Skip to content

Conversation

@justice-adams-apple
Copy link
Contributor

@justice-adams-apple justice-adams-apple commented Dec 12, 2025

  • Add artifact_manager for downloading toolchain artifacts used by stage 2 builds.

This will replace the fetch functionality in monorepo_build.py and be used by all clang jobs in the future, both the mainline (standard post merge jobs which exist now) and the bisection jobs (jobs which will be setup to mimic the mainline jobs in a separate folder)

The script will search for a stage1 build of both the mainline and bisection version of the job if needed and download the appropriate package. This saves on build time as we can utilize already built stage 1 packages. If no stage 1 artifact exist, the data is written to a props file and the CI system will fire off a stage 1 build accordingly.

This handles all of the logic around determining which toolchain package to download making it simple for CI libraries in the future to invoke the script as such:

def pythonCmd = """
    python ./artifact_manager.py \\
        --workspace "\$WORKSPACE" \\
        --output-file artifact_result.properties
"""

if (artifactParam) {
    pythonCmd += " --artifact \"${artifactParam}\""
} else {
    pythonCmd += " --job-name \"${jobName}\"" 
}

def scriptResult = sh(
    script: pythonCmd,
    returnStatus: true
)

// Get result from props file
def resultProps = readProperties file: 'artifact_result.properties'
def artifactFound = resultProps.ARTIFACT_FOUND == 'true'
def usedArtifact = resultProps.USED_ARTIFACT
def needsStage1 = resultProps.NEEDS_STAGE1 == 'true'

if (needsStage1) {
    echo "Triggering stage 1 build for artifact: ${usedArtifact}"

    // Trigger stage 1 job and wait for completion
    def stage1Build = build(
        job: stage1Job,
        wait: true,
        propagate: true
    )

    echo "Stage 1 build completed successfully. Build number: ${stage1Build.number}"

    // Retry fetching the artifact after stage 1 completes
    def retryCmd = """
        python ./artifact_manager.py \\
            --workspace "\$WORKSPACE" \\
            --output-file artifact_result.properties
            --artifact \"${usedArtifact}\""
    """

    sh retryCmd
}

By extracting the logic to the python scripts we can make the functionality portable across CI systems.

@justice-adams-apple justice-adams-apple force-pushed the jadams/add-artifact-manager branch from 32c2545 to 30592f3 Compare December 12, 2025 00:05
@justice-adams-apple
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant