Performance: GitHub Action caching

[natrimmer]

Optimize CI/CD Pipeline Performance with Caching

This PR improves the performance of the fAIr CI/CD pipeline by implementing caching across workflows:

Changes

• Added npm caching for frontend builds using actions/cache@v4
• Implemented Docker layer caching for API and Worker images using BuildKit
• Added pip caching for backend Python dependencies

Benefits

• Reduces CI runtime by caching node_modules, Docker layers, and Python packages
• Decreases GitHub Actions compute time and costs
• Speeds up PR checks and deployments

Resources

• Docker: Cache management with GitHub Actions
• Python: Caching Packages Dependencies
• Node: Caching Packages Dependencies
• Node: Better Cache Hits

[natrimmer]

I had to remove the docker layer caching as it was causing the action to hang. This seems to be an issue others are having where the cache export step gets caught. I'm looking into fixes.

moby/buildkit#4628

[kshitijrajsharma]

Sure , Thank you for looking into this ! Let me know if you have some updates

[natrimmer]

@kshitijrajsharma Thank you! I'll open another PR shortly!

[natrimmer]

@kshitijrajsharma

Docker Build Cache Impact

An analysis of GitHub Actions workflow data shows strong time savings through Docker layer caching in the docker_build.yml workflow. From historical averages before caching (116 runs/year), workflows took 708.2s - now dropping to 55.0s with cache hits (92.2% time reduction).

Impact from caching in docker_build.yml

Based on historical run frequency:

• Original workflow runtime: 708.2s
• New workflow runtime (with cache): 55.0s
• Time saved per run: 653.2s
• Annual runs: 116 (~10 per month)
• Total potential annual time savings: 75,771.2 seconds

Combined savings formula:

Annual minutes saved = (cache hit rate %) × 1,262.9 minutes
  (derived from 75,771.2 seconds / 60)

Monthly minutes saved = Annual savings / 12
  = (cache hit rate %) × 105.2 minutes

Projected workflow time savings at different cache hit rates:

Cache Hit Rate	Monthly (min)	Annual (min)
25%	26.3	315.7
50%	52.6	631.5
75%	78.9	947.2
100%	105.2	1,262.9

Other Workflows

Similar time savings can be expected in docker_publish_image.yml as it mirrors the build workflow structure. Historical averages show even longer pre-cache build times, averaging 1,053.0s across 148 runs/year, suggesting potentially larger absolute time savings. Cache performance data from this and other workflows doesn't exist yet.

Limitations

• Current performance metrics only available from docker_build.yml
• Historical averages based on 1 year period before caching
• Cache hit effectiveness will vary across workflows based on:
  • Development patterns
  • Dependency update frequency
  • Branch structures
  • Pull request patterns

Workflow data from fAIr's Action Performance Metrics.

[kshitijrajsharma]

Awesomeeee