NilsBriggen
e256508c44
The Android CI job recompiled all native C++ (whisper.cpp, reanimated, worklets, gesture-handler) from scratch every push — ~41 min on the shared runner — because nothing survived the `COPY . .` layer invalidation, and `--load` exported the whole ~15GB image just to copy one APK out. Dockerfile (docker/android.Dockerfile): - Install ccache and patch the NDK CMake toolchain file so every module's externalNativeBuild routes C/C++ compiles + links through ccache (the third-party modules don't honor ccache on their own). ccache dir is a persistent BuildKit cache mount, so objects survive across builds. - Mount GRADLE_USER_HOME as a cache and pass --build-cache --parallel, so Kotlin/Java/resource/dex tasks and resolved deps persist too. - Cache-mount the bun install dir; move the nodejs install into the cached toolchain layer (was reinstalling on every source change). - New `apk` stage (FROM scratch) holding just the signed APK, for --output type=local extraction without loading the 15GB image. CI (.gitea/workflows/ci.yml): - build-apk now builds `--target apk --output type=local,dest=./out` and streams ./out/app-release.apk to /srv/wisp/wisp.apk — no --load, no docker create/cp, and the build image no longer piles up on the host. Validated locally (arm64, real sign+assemble path): cold gradle 5m5s -> warm gradle 57s; 61/61 C++ compiles served from ccache (100% hit); 226/578 Gradle tasks from cache; export-only build 60s wall, APK valid + v2/v3 signed. The first server build after this is still cold (~41 min) to populate the caches; pushes after that are much faster. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
164 lines
7.8 KiB
Docker
164 lines
7.8 KiB
Docker
# syntax=docker/dockerfile:1
|
|
#
|
|
# docker/android.Dockerfile — full Android build for Wisp -> signed release APK.
|
|
# ---------------------------------------------------------------------------
|
|
# Two stages:
|
|
# toolchain : JDK17 + Android SDK (platform/build-tools) + NDK + CMake + Bun.
|
|
# Reusable on its own (docker build --target toolchain).
|
|
# build : COPYs the repo, installs JS deps, `expo prebuild`, signs, and
|
|
# runs `gradlew assembleRelease`. The signed APK lands at
|
|
# /workspace/android/app/build/outputs/apk/release/app-release.apk
|
|
#
|
|
# This is socket-safe for CI: the whole build happens inside `docker build`
|
|
# (context streamed to the daemon — NO host bind mounts, which don't resolve
|
|
# over a mounted Docker socket). CI then extracts the APK via docker create/cp.
|
|
#
|
|
# Signing secrets are passed as BuildKit secrets (NOT build args / layers):
|
|
# --secret id=ks_b64 -> ANDROID_KEYSTORE_BASE64 (base64 of the .keystore)
|
|
# --secret id=ks_pass -> ANDROID_KEYSTORE_PASSWORD
|
|
# --secret id=ks_alias -> ANDROID_KEY_ALIAS
|
|
# --secret id=ks_keypass -> ANDROID_KEY_PASSWORD
|
|
#
|
|
# JDK pinned to 17 (AGP 8.x); versions track RN 0.85 / Expo SDK 56
|
|
# (compileSdk/targetSdk 36, buildTools 36.0.0, NDK 27.1.12297006).
|
|
# ---------------------------------------------------------------------------
|
|
|
|
FROM eclipse-temurin:17-jdk AS toolchain
|
|
|
|
ARG ANDROID_PLATFORM_VERSION=36
|
|
ARG ANDROID_BUILD_TOOLS_VERSION=36.0.0
|
|
ARG ANDROID_NDK_VERSION=27.1.12297006
|
|
ARG ANDROID_CMAKE_VERSION=3.22.1
|
|
ARG ANDROID_CMDLINE_TOOLS_VERSION=11076708
|
|
|
|
ENV ANDROID_HOME=/opt/android-sdk \
|
|
ANDROID_SDK_ROOT=/opt/android-sdk \
|
|
DEBIAN_FRONTEND=noninteractive \
|
|
GRADLE_USER_HOME=/opt/gradle \
|
|
BUN_INSTALL=/root/.bun
|
|
ENV PATH=/root/.bun/bin:${ANDROID_HOME}/cmdline-tools/latest/bin:${ANDROID_HOME}/platform-tools:${PATH}
|
|
|
|
# OS deps for SDK download + RN/CMake native builds.
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
unzip curl git bash ca-certificates \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Bun (Wisp's package manager / task runner).
|
|
RUN curl -fsSL https://bun.sh/install | bash && bun --version
|
|
|
|
# Android commandline-tools into $ANDROID_HOME/cmdline-tools/latest.
|
|
RUN mkdir -p ${ANDROID_HOME}/cmdline-tools \
|
|
&& curl -fsSL -o /tmp/ct.zip \
|
|
"https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_CMDLINE_TOOLS_VERSION}_latest.zip" \
|
|
&& unzip -q /tmp/ct.zip -d /tmp/ct \
|
|
&& mv /tmp/ct/cmdline-tools ${ANDROID_HOME}/cmdline-tools/latest \
|
|
&& rm -rf /tmp/ct.zip /tmp/ct
|
|
|
|
# Licenses + SDK packages. NDK + CMake are required because whisper.rn compiles
|
|
# native C++ (whisper.cpp) via externalNativeBuild.
|
|
RUN yes | sdkmanager --licenses > /dev/null \
|
|
&& sdkmanager --install \
|
|
"platform-tools" \
|
|
"platforms;android-${ANDROID_PLATFORM_VERSION}" \
|
|
"build-tools;${ANDROID_BUILD_TOOLS_VERSION}" \
|
|
"ndk;${ANDROID_NDK_VERSION}" \
|
|
"cmake;${ANDROID_CMAKE_VERSION}" > /dev/null \
|
|
&& yes | sdkmanager --licenses > /dev/null
|
|
|
|
# Build accelerators, installed AFTER the (large, slow) SDK/NDK layer so that
|
|
# layer stays cached:
|
|
# - ccache : short-circuits recompiling unchanged C++ across builds. The bulk
|
|
# of the build is third-party native modules (whisper.cpp via
|
|
# whisper.rn, reanimated/worklets, gesture-handler); see the NDK
|
|
# toolchain patch below for how it's wired in globally.
|
|
# - nodejs : scripts/ci-android-sign.sh runs a small Node patch script (this
|
|
# image ships Bun, not Node). Lives here so it's cached, not
|
|
# reinstalled on every source change.
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends ccache nodejs \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Route EVERY NDK CMake C/C++ compile + link through ccache. AGP invokes each
|
|
# externalNativeBuild project with this NDK toolchain file, so setting the
|
|
# compiler launcher here applies ccache to all native modules at once — no need
|
|
# to patch each module's CMakeLists (third-party ones don't honor ccache on
|
|
# their own). Guarded on ccache being present; a no-op otherwise.
|
|
RUN printf '\n%s\n' \
|
|
'# --- WISP: route NDK CMake compiles/links through ccache ---' \
|
|
'find_program(WISP_CCACHE_PROGRAM ccache)' \
|
|
'if(WISP_CCACHE_PROGRAM)' \
|
|
' set(CMAKE_C_COMPILER_LAUNCHER "${WISP_CCACHE_PROGRAM}")' \
|
|
' set(CMAKE_CXX_COMPILER_LAUNCHER "${WISP_CCACHE_PROGRAM}")' \
|
|
'endif()' \
|
|
>> "${ANDROID_HOME}/ndk/${ANDROID_NDK_VERSION}/build/cmake/android.toolchain.cmake"
|
|
|
|
# ccache tuning. CCACHE_DIR is mounted as a persistent BuildKit cache on the
|
|
# gradle RUN below, so hits survive across builds. COMPILERCHECK=content keeps
|
|
# hits valid across image rebuilds; the sloppiness flags raise the hit rate for
|
|
# RN's headers without risking incorrect cache hits for release builds.
|
|
ENV CCACHE_DIR=/root/.ccache \
|
|
CCACHE_MAXSIZE=5G \
|
|
CCACHE_COMPILERCHECK=content \
|
|
CCACHE_SLOPPINESS=time_macros,include_file_mtime,include_file_ctime,file_macro \
|
|
CCACHE_BASEDIR=/workspace
|
|
|
|
WORKDIR /workspace
|
|
|
|
# ---------------------------------------------------------------------------
|
|
FROM toolchain AS build
|
|
|
|
# Install JS deps first (cached unless package.json/bun.lock change). The bun
|
|
# cache mount keeps re-installs fast even when the lockfile DOES change.
|
|
COPY package.json bun.lock ./
|
|
RUN --mount=type=cache,target=/root/.bun/install/cache \
|
|
bun install --frozen-lockfile
|
|
|
|
# Bring in the rest of the source (.dockerignore excludes node_modules, android/, ios/, dist...).
|
|
COPY . .
|
|
|
|
# Generate the native android/ project (autolinks whisper.rn, reanimated, etc.).
|
|
RUN bunx expo prebuild --platform android --no-install
|
|
|
|
# Decode the keystore + wire release signing, then assemble the signed APK.
|
|
# Secrets are mounted as files under /run/secrets and exported only for this RUN.
|
|
#
|
|
# Two persistent BuildKit cache mounts make rebuilds fast (they survive the
|
|
# `COPY . .` layer invalidation that happens on every source change):
|
|
# - /root/.ccache : compiled C++ objects (the big win — whisper.cpp etc.).
|
|
# - /opt/gradle : GRADLE_USER_HOME — the Gradle build cache (--build-cache),
|
|
# resolved dependencies, and the downloaded wrapper dist.
|
|
# --build-cache + --parallel let unchanged Kotlin/Java/resource/dex tasks be
|
|
# served from cache and independent module tasks run concurrently.
|
|
RUN --mount=type=secret,id=ks_b64 \
|
|
--mount=type=secret,id=ks_pass \
|
|
--mount=type=secret,id=ks_alias \
|
|
--mount=type=secret,id=ks_keypass \
|
|
--mount=type=cache,target=/root/.ccache \
|
|
--mount=type=cache,target=/opt/gradle \
|
|
ANDROID_KEYSTORE_BASE64="$(cat /run/secrets/ks_b64)" \
|
|
ANDROID_KEYSTORE_PASSWORD="$(cat /run/secrets/ks_pass)" \
|
|
ANDROID_KEY_ALIAS="$(cat /run/secrets/ks_alias)" \
|
|
ANDROID_KEY_PASSWORD="$(cat /run/secrets/ks_keypass)" \
|
|
bash scripts/ci-android-sign.sh \
|
|
&& cd android \
|
|
&& ./gradlew assembleRelease \
|
|
-PreactNativeArchitectures=arm64-v8a \
|
|
--build-cache --parallel \
|
|
--no-daemon --max-workers=2 -x lint --stacktrace \
|
|
&& ccache --show-stats --verbose 2>/dev/null | grep -iE 'cacheable|hit|miss|cache size' || true
|
|
|
|
# The signed APK lands at:
|
|
# /workspace/android/app/build/outputs/apk/release/app-release.apk
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# apk : a tiny export-only stage holding JUST the signed APK.
|
|
#
|
|
# CI builds this stage with `--target apk --output type=local,dest=...`, which
|
|
# writes only app-release.apk to the host — NO `--load` of the ~15GB build image
|
|
# (saves the multi-minute image export/unpack on every push AND avoids that
|
|
# image piling up on the server's disk). The COPY pulls from `build`, so the
|
|
# whole build still runs; only the output is slimmed.
|
|
FROM scratch AS apk
|
|
COPY --from=build /workspace/android/app/build/outputs/apk/release/app-release.apk /app-release.apk
|