[0.4.8] object attributes are now case sensitive (breaking change!)

[mblaschke-daimlertruck]

Starting with 0.4.8 all object attributes are case sensitive but were case insensitive before

example yaml file:

foobar: barfoo

code:

$foo = Get-Content ... | ConvertFrom-Yaml
$foo.Foobar # not working anymore, worked with 0.4.7
$foo.foobar # works with 0.4.7 and 0.4.8

This is a breaking change in a minor or patch version, right before xmas is not a good time to introduce such changes 😔

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.6
PSEdition                      Core
GitCommitId                    7.4.6
OS                             Darwin 23.6.0 Darwin Kernel Version 23.6.0: Fri Nov 15 15:13:15 PST 2024; root:xnu-10063.141.1.702.7~1/RELEASE_ARM64_T6000
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[SpecterShell]

Might be related: PowerShell/PowerShell#19928

[gabriel-samfira]

Hi @mblaschke-daimlertruck

My apologies for the issues you're having with powershell-yaml.

I was hoping that the 4 months we had the pre release up would be enough to catch things like this. But I did not take into account that not many people try out pre-releases.

I will have a look this evening at options to mitigate this short term and will probably push an update. I am currently traveling so this will happen tomorrow at the latest.

The case insensitivity when deserializing was broken behavior in previous versions. It essentially broke round-tripping as yaml itself is case sensitive. So after reading from yaml, you would loose keys as foo and Foo could hold 2 sets of completely different data. And data loss is not the best feature to have.

For the short term I will probably revert that change until after the holidays and release a 0.5 version with case sensitivity enabled.

If this breaks things badly for you, and can't wait a few hours, please explicitly use/install 0.4.7 for now by passing the -MaximumVersion flag to Install-Module ultil I get a chance to get to a PC.

[gabriel-samfira]

This is the PR that fixed the broken case insensitive behavior of powershell-yaml: https://github.com/cloudbase/powershell-yaml/pull/139/files. This PR was a bug fix, not a feature, which is why it was not released as part of a major version release. Relying on case insensitivity is something that should be avoided, especially if the source of the data that is being used, is case sensitive.

I am reverting it for now and pushing a new version on Monday at the latest, but keep in mind, that we are now reverting to broken behavior which will lead to data loss under certain conditions:

foo: "this is important data"
Foo: "this is equally important data, but different"

In the above example, foo is lost. Leaving aside the fact that relying on case sensitivity to store data or make decisions may sound weird, if it's possible to do, someone is probably doing it already. There is at least one programming language that does rely on case sensitivity to make meaningful decisions (Go).

YAML is case sensitive. Any parser that deals with YAML must also be case sensitive. This change will be added back in version 0.5.0 which will released at the beginning of 2025.

I encourage anyone that uses powershell-yaml to watch this repository for changes and please help test out pre-releases whenever they are available. Especially if those pre-releases have been up for a long time. I don't expect anyone to run a pre-release of powershell-yaml in production, but a Invoke-Pester in any project that relies on powershell-yaml will help catch issues such as this in time.

I will also be creating github releases from now on, with a description of every change in that release. We will have at least one pre-release, which will also be published to the gallery. The pre-release will stay for at least 1 week, after which if no issues are reported with that pre-release, a final release will be cut.

[mblaschke-daimlertruck]

@gabriel-samfira
ConvertFrom-Json has another approach:

$bar = '{"foo":"bar","Foo":"bar2"}' | ConvertFrom-Json
ConvertFrom-Json: Cannot convert the JSON string because it contains keys with different casing. Please use the -AsHashTable switch instead. The key that was attempted to be added to the existing key 'foo' was 'Foo'.

which also might be a solution and is backwards compatible.

I'm also not happy about the case insensitive attributes, but it needs time to fix it :)
Thanks for the revert.

[gabriel-samfira]

ConvertFrom-Json has another approach:

I would love to implement something like that, but I don't think it's a good fit for powershell-yaml. ConvertFrom-Json went with PScuscomObjects from the beginning (which is case insensitive) and later added the ability to use hash maps via the flag you see in the error message. We went with hash maps from the start because that is the native data structure for kye/value pairs in powershell. PSCustomObject is just a generic wrapper which adds overhead and has some gotchas that bite you in the behind later on (like case insensitivity).

If we are to change the way we deserialize, I think we're just going to go with something like this:

• [WiP] Implement deserializing to specific type #142

Where we add the ability for the user to define a "model" into which we deserialize a yaml. So instead of adding numerous switches and options, we just let the user define the data structure they want to deserialize into.

In any case, I will publish a new version with the revert later today.

Please click on "watch" for this repo and if your project has a CI, please consider using adding the -AllowPrerelease flag to the Install-Module command when installing powershell-yaml. I try to not break things as much as I can, but sometimes it can't be helped. Installing a pre-release version during testing will help us catch issues and will give you a heads up in regards to changes that may be incoming in the next stable release of powershell-yaml. I realize this may not be possible, but it would help if more people would test pre-releases.

I will try to do a much better job in communicating changes that are part of new releases. Right now transparency from my side is abysmal, and I need to fix that.

[gabriel-samfira]

A new version has been released: https://www.powershellgallery.com/packages/powershell-yaml/0.4.9

A new release has been generated on GitHub: https://github.com/cloudbase/powershell-yaml/releases/tag/v0.4.9

Please test it out and let me know if all is well.