Support for detecting breaking changes before provider releases

[marcindulak]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Description

There were two recent breaking changes in the minor versions of the provider. The reference to the minor version is not that important, the focus is on detecting breaking changes.

v3.95.0 results in opening the app service firewall #25244
v3.97.0 results in removal of storage accounts #25364

This is a meta issue, created for the purpose of requesting the feedback from the core contributors, about what are the plans to improve this situation.

New or Affected Resource(s)/Data Source(s)

azurerm

Potential Terraform Configuration

No response

References

No response

[rcskosir]

👋 Thank you for taking the time to open this issue. We appreciate feedback from the community and are always looking for ways to improve.

The topic of breaking changes and testing is something that we have given much thought to internally, unfortunately there are a lot of hurdles to implementing a robust and accurate breaking change detector due to limitations with our testing framework.

Despite this we have made some recent enhancements to the provider to help stop bugs and breaking changes from going out in minor releases. As mentioned by Tom here #25364 (comment). hashicorp/go-azure-sdk with its additional validation and checks is one way we are improving the codebase's resilience to a number of bug types.

Additionally with the addition of the PR template #25103 we hope to have better insight to when a breaking change is being made and help with the review process and improve traceability.

As for future plans to improve the process, as mentioned a robust breaking change testing is required, which requires bandwidth that we don’t currently have along with updates like Terraform Plugin Framework, so while this is on our backlog and is something we would like to implement, it is not currently on the roadmap.

[marcindulak]

I hope the team can expand on the topic, and a) provide the outcome of the thoughts about the hurdles to the public b) provide issue tracker for the plans for hashicorp/go-azure-sdk with its additional validation and checks.

This issue is one aspect of terraform, another one, currently affecting the use of terraform, being a subjective increase of nondeterminism in terraform behaviour, for which I opened a dedicated issue #25585.