Share

CW-11: Visualise workflows

By c03rad0r March 14, 2025 · Edited March 14, 2025

### [Building a TollGate module](https://github.com/OpenTollGate/tollgate-module-relay-go/actions/runs/13858291486) (nostr client)

The following mermaid graph visualises what happens when we make changes to a module.

  • Build binary

    • - [x] verifies that the code can be compiled for all the targets that we care about in less than two minutes.

  • Package module

    • - [x] Uses the OpenWRT SDK to generate packages quickly

    - [x] Builds a newer golang version, which is required by the relay

    - [x] Was using the wrong golang-package.mk. See [here](

    c03rad0r
    Mar 14, 2025 14:32

    This change is needed because you’re moving from using OpenWRT’s standard Go package infrastructure to a custom implementation. Let me break it down:

    Original Line

    include $(INCLUDE_DIR)/golang-package.mk

    This tries to use OpenWRT’s built-in Go package infrastructure which:

    Expects specific file locations

    Requires the official golang-package.mk file

    Uses predefined build rules that may conflict with your custom setup

    New Implementation

    $(eval $(call GoPackage))

    This instead activates your custom Go package definition that:

    Is defined in your golang.mk file

    Uses your own build rules and paths

    Bypasses OpenWRT’s default Go build system

    Key Differences

    include \((INCLUDE_DIR)/golang-package.mk\)(eval $(call GoPackage))SourceOpenWRT’s core packageYour custom golang.mkBuild RulesPredefined by OpenWRTCustomized for your needsFile LocationsRequires standard pathsUses your custom pathsWorkspace HandlingStandard Go workspaceCustom .go_work directory

    Why This Fixes Your Error

    Avoids Missing File The error occurred because the build system couldn’t find OpenWRT’s official golang-package.mk. By using your own implementation, you eliminate this dependency.

    Custom Build Control Your golang.mk likely contains a GoPackage macro that implements alternative build steps better suited for your workspace setup.

    Path Consistency Your custom implementation uses paths like .go_work/build that match what you’re actually creating in the build process.

    Implementation Notes

    Ensure golang.mk contains the GoPackage macro definition

    Verify your custom implementation handles:

    define GoPackage

    Your custom build rules here

    Package name handling

    Dependency resolution

    Install paths

    endef

    Remove any remaining references to OpenWRT’s default Go infrastructure

    This change allows full control over the Go build process while maintaining compatibility with OpenWRT’s package framework.

    View full event
    ).

    - [ ] [Packages the module](https://github.com/OpenTollGate/tollgate-sdk/actions/runs/13858302129) for OpenWRT

    - [x] Resolved sonic error

    - [ ] [Separate step](

    c03rad0r
    Mar 14, 2025 14:51

    The relay specific sonic stuff doesn’t apply to other modules, so perhaps it should be in a separate step that gets skipped when packaging other modules..

    https://github.com/OpenTollGate/tollgate-sdk/blob/eab6f2a6c728ddff5734cc50e4b999e17d8d7fe3/action.yml#L402-L422

    View full event
    ) for relay specific stuff

    - [ ] Currently hitting errors when installing the package - migration to action

    - [ ] opkg-make-index not required for now

    - [ ] package format seems wrong - perhaps opkg-build is required?

    - [ ] [[Prebuild Golang]] to save about 10 minutes per run. Issue [here](

    c03rad0r
    Mar 14, 2025 14:37

    Yes! You can significantly reduce build time by pre-building the Golang package and reusing it across runs. Here’s how to implement this optimization:

    1. Create a Pre-Build Workflow (.github/workflows/prebuild-golang.yml):

    name: Prebuild Golang on: workflow_dispatch: schedule: - cron: ‘0 0 * * 0’ # Weekly rebuild push: paths: - ‘lang/golang/**’

    jobs: prebuild: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v4

    - name: Setup SDK
  uses: ./.github/actions/your-sdk-setup-action
  with:
    model: gl-mt3000 # Use your most common target

- name: Build Golang IPK
  run: |
    SDK_PATH="/tmp/openwrt-sdk/..." # Your SDK path logic
    make -C \(SDK_PATH package/lang/golang/compile V=s -j\)(nproc)

- name: Upload Artifact
  uses: actions/upload-artifact@v3
  with:
    name: prebuilt-golang
    path: ${{ SDK_PATH }}/bin/packages/**/golang*.ipk
    1. Modify Your Main Workflow:

    • name: Get Prebuilt Golang uses: actions/download-artifact@v3 with: name: prebuilt-golang path: ${{ SDK_PATH }}/bin/packages/

    • name: Install Prebuilt Golang run: | opkg install ${{ SDK_PATH }}/bin/packages//golang.ipk shell: bash

    Remove these from your current workflow:

    - The entire “Copy golang sources” section

    - make package/lang/golang/compile

    1. Optimize Docker Cache (Even Better Approach):

    FROM openwrt-sdk-base AS builder

    Build golang once

    RUN make package/lang/golang/compile

    FROM openwrt-sdk-base AS runtime COPY –from=builder /sdk/bin/packages/ /sdk/bin/packages/

    1. Implement Local Cache (Alternative):
    • name: Cache Golang uses: actions/cache@v3 with: path: | ${{ SDK_PATH }}/build_dir/target-/golang \({{ SDK_PATH }}/staging_dir/target-*/golang* key: golang-\){{ hashFiles(‘lang/golang/*’) }}

    Expected Time Savings:

    StepOriginal TimeOptimized TimeGolang Compilation8-10 mins0-30 secsDependency Resolution2-3 mins0-15 secsTotal Savings10 mins1 min

    Verification Steps:

    Add debug output after installation:

    • name: Verify Golang Installation run: | ls -la ${{ SDK_PATH }}/staging_dir/target-/bin/go file ${{ SDK_PATH }}/staging_dir/target-/bin/go \({{ SDK_PATH }}/staging_dir/target-*/bin/go version Monitor build logs for: Checking 'golang'... already installed Skipping package golang dependencies Important Considerations: Version Pinning: Add a hash check in your workflow: - name: Verify Package Integrity run: | echo "golang_checksum=\)(sha256sum bin/packages//golang.ipk)“ >> \(GITHUB_OUTPUT shell: bash Cross-architecture Support: Store separate packages for each target: # In prebuild job ARCH_SUFFIX=\)(echo \(TARGET | md5sum | cut -c1-6) mv golang*.ipk golang-\)ARCH_SUFFIX.ipk

    This optimization should reduce your build times by 80-90% while maintaining build reproducibility.

    View full event
    )

  • Generate image

    • - [ ] Installs the package from the previous step in an OpenWRT image

    ```mermaid

    graph TD

    Trigger[Push/PR to main] --> build-binary

    build-binary -->|Matrix: Targets| Build

    Build --> UploadBlossom

    UploadBlossom --> PublishNIP94[Publish NIP-94 announcement]

    PublishNIP94 --> Verify

    Verify --> UploadArtifact

    build-binary --> build-package

    build-package -->|Matrix: 3 Models| BuildPackage

    BuildPackage --> UploadBlossomPackage

    UploadBlossomPackage --> PublishNIP94Package[Publish NIP-94 announcement]

    PublishNIP94Package --> VerifyPackage

    VerifyPackage --> UploadArtifactPackage

    build-package --> build-firmware

    build-firmware -->|Matrix: 3 Models| BuildFirmware

    BuildFirmware --> UploadBlossomFirmware

    UploadBlossomFirmware --> PublishNIP94Firmware[Publish NIP-94 announcement]

    PublishNIP94Firmware --> VerifyFirmware

    VerifyFirmware --> UploadArtifactFirmware

    %% Define classes for styling

    classDef bitcoinOrange fill:#f7931a,stroke:#e68a00,color:white;

    classDef nostrPurple fill:#8624f5,stroke:#6c13d8,color:white;

    classDef nip94Color fill:#00bcd4,stroke:#0097a7,color:white;

    %% Apply classes to nodes

    class build-binary,build-package,build-firmware bitcoinOrange;

    class UploadBlossom,UploadBlossomPackage,UploadBlossomFirmware nostrPurple;

    class PublishNIP94,PublishNIP94Package,PublishNIP94Firmware nip94Color;

    ```

    ### Making a TollGate release (OpenWRT image)

    The packages that were announced with NIP-94 events when building modules need to be installed in an OpenWRT image.

    The tollgate-os image builder

    - [ ] searches for the NIP-94 events that match the version numbers in the release json

    - [x] downloads the respective packages from nostr

    - [ ] installs the packages in an OpenWRT image

    - [x] uploads the image to blossom

    - [x] publishes a NIP-94 event announcing the image

    #### Example release

    ```

    {

    "modules": {

    "tollgate-module-crowsnest-go": {

    "version": "0.1.0",

    "description": "Crowsnest module for observation and monitoring"

    },

    "tollgate-module-janitor-go": {

    "version": "0.1.0",

    "description": "Janitor module for system maintenance and cleanup"

    },

    "tollgate-module-merchant-go": {

    "version": "0.1.0",

    "description": "Merchant module for payment processing"

    },

    "tollgate-module-relay-go": {

    "version": "0.1.0",

    "description": "Relay module for message handling and forwarding"

    },

    "tollgate-module-valve-go": {

    "version": "0.1.0",

    "description": "Valve module for traffic control and access management"

    },

    "tollgate-module-whoami-go": {

    "version": "0.1.0",

    "description": "WhoAmI module for identity and authentication services"

    }

    },

    "nip94eventwhitelist": {

    "authorized_npubs": [

    "...",

    "...",

    "..."

    ]

    },

    "relays": [

    "wss://relay.damus.io",

    "wss://nos.lol",

    "wss://nostr.mom",

    "wss://relay.nostr.band",

    "wss://relay.snort.social"

    ],

    "last_updated": "2025-03-13"

    }

    ```

    ```mermaid

    graph TD

    Start[Start Image Builder] --> ReadConfig[Read Release JSON]

    ReadConfig --> FetchModules[Fetch Modules via NIP-94]

    FetchModules --> BuildImage[Build OpenWRT Image]

    BuildImage --> UploadImage[Upload to Blossom]

    UploadImage --> AnnounceRelease[Publish NIP-94 Announcement]

    %% Define classes for styling

    classDef input fill:#4caf50,stroke:#2e7d32,color:white;

    classDef process fill:#2196f3,stroke:#0d47a1,color:white;

    classDef nostr fill:#8624f5,stroke:#6c13d8,color:white;

    classDef output fill:#f7931a,stroke:#e68a00,color:white;

    %% Apply classes to nodes

    class Start,ReadConfig input;

    class BuildImage process;

    class FetchModules,AnnounceRelease nostr;

    class UploadImage output;

    ```

    ### Goals for the coming week

    - [ ] Start a TIP-01 draft

    - [ ] Pre-build golang

    - [ ] Get the package from the new SDK action to install

    - [ ] Implement the release image build from NIP-94 events

    - [ ] Figure out how to build rust modules with the SDK (merchant)

    #f7931a #e68a00 #8624f5 #6c13d8 #00bcd4 #0097a7 #4caf50 #2e7d32 #2196f3 #0d47a1
    Write a comment
    No comments yet.

    Related articles

    by TFTC (News Bot)

    March 30 2026 · Edited Mar 30, 2026
    Cover image for The PARITY Act Confirms It: Stablecoins Get the Tax Break, Bitcoin Gets Left Behind

    The PARITY Act Confirms It: Stablecoins Get the Tax Break, Bitcoin Gets Left Behind

    The latest PARITY Act draft gives stablecoins a $200 de minimis exemption while leaving Bitcoin out entirely. It confirms TFTC reporting that Coinbase lobbied for exactly this outcome.

    by TFTC (News Bot)

    March 19 2026 · Edited Mar 19, 2026
    Cover image for Bitcoin Infrastructure Is Being Built at Wartime Speed

    Bitcoin Infrastructure Is Being Built at Wartime Speed

    Square just auto-enabled Lightning for 4 million merchants. Tether funded Ark Labs. Lightning hit $1.17B monthly volume. A 4nm mining chip taped out. The Fed held rates on $39T in debt. The builders aren't waiting.

    by TFTC (News Bot)

    March 18 2026 · Edited Mar 18, 2026
    Cover image for The Oil Market Fractured, and the Petrodollar Is Cracking in the Strait of Hormuz

    The Oil Market Fractured, and the Petrodollar Is Cracking in the Strait of Hormuz

    The Strait of Hormuz has been closed for 19 days, fracturing oil into a two-tier market. Dubai crude hit $152 while WTI sits at $94. An Indian tanker paid in yuan for safe passage. The petrodollar is cracking in real time.