Enable Link-Time Optimization (LTO) and codegen-units = 1

[zamazan4ik]

Hi!

I noticed that in the Cargo.toml file Link-Time Optimization (LTO) for the project is not enabled. I suggest switching it on since it will reduce the binary size (always a good thing to have) and will likely improve the application's performance. If you want to read more about LTO and its possible modes, I recommend starting from this Rustc documentation.

I think you can enable LTO only for the Release builds so as not to sacrifice the developers' experience while working on the project since LTO consumes an additional amount of time to finish the compilation routine. If you think that a regular Release build should not be affected by such a change as well, then I suggest adding an additional dist or release-lto profile where in addition to regular release optimizations LTO will also be added. Such a change simplifies life for maintainers and others interested in the project persons who want to build the most optimized version of the application. However, if we enable it on the Cargo profile level for the Release profile, users, who install the application with cargo install will get the LTO-optimized version of the game "automatically". E.g., check cargo-outdated Release profile. You also could be interested in other optimization options like codegen-units = 1 - this option also brings performance improvements to the project.

Basically, it can be enabled with the following lines:

[profile.release]
codegen-units = 1
lto = true

I have made quick tests (Fedora 41, Rust 1.83, the latest version of the project at the moment) - here are the results:

libllm_gateway.so:

• Release (current default): 11 Mib
• Release + codegen-units = 1 + Fat LTO: 9.8 Mib

libprompt_gateway.so:

• Release (current default): 1.7 Mib
• Release + codegen-units = 1 + Fat LTO: 1.4 Mib

Thank you.

[adilhafeez]

Nice - that's like 10% reduction in size. The cost is additional build time. We can enable this for release docker images. Thanks for reporting it.

[samadpls]

hi @adilhafeez, Can I work on this issue

[adilhafeez]

@samadpls thanks. Pls create a PR when you're ready and I can review you work.