C# Deserialization error with Databricks GetMwsWorkspaces - expects string, gets Double

[automagic]

Describe what happened

When calling GetMwsWorkspaces, Pulumi raises a serialization error:

warning: Expected System.String but got System.Double deserializing Pulumi.Databricks.GetMwsWorkspacesResult(ids)

This appears to be a regression, as this has worked fine before.

Sample program

var workspaces = Databricks.GetMwsWorkspaces.Invoke(null, new InvokeOptions
                {
                    Provider = cfg.DatabricksMwsProvider,  // i've instantiated this elsewhere. this is def not the issue because i never changed my code.
                    Parent = this,
                });

Log output

No response

Affected Resource(s)

No response

Output of pulumi about

CLI
Version 3.144.1
Go Version go1.23.4
Go Compiler gc

Host
OS darwin
Version 14.4.1
Arch arm64

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[iwahbe]

Hi @automagic. I'm sorry you're hitting this bug.

Can you clarify what you are hitting? The title says "error", but what you pasted in says "warning"? Does it stop program executing, or just warn?

I'm not able to replicate this by just invoking Databricks.GetMwsWorkspaces.Invoke. I tried with this program, and it worked as expected:

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Databricks = Pulumi.Databricks;

return await Deployment.RunAsync(() => 
{
    Databricks.GetMwsWorkspaces.Invoke(null);
    return new Dictionary<string, object?>
    {
    };
});

Do you have repro for this warning?

[pstrittmater-sd]

@iwahbe hi there. you're correct. it's a warning.

warning: Expected System.String but got System.Double deserializing Pulumi.Databricks.GetMwsWorkspacesResult(ids)

however, i think there is actually an issue under the hood. as you can see, none of the ids actually has a value:

code

:

                var workspaces = Databricks.GetMwsWorkspaces.Invoke(null, new InvokeOptions
                {
                    Provider = cfg.DatabricksMwsProvider,
                    Parent = this,
                });
                workspaces.Apply(result =>
                {
                    Console.WriteLine($"Found {result.Ids.Count} workspaces:");
                    foreach (var (name, id) in result.Ids)
                    {
                        Console.WriteLine($"  `{name}`: `{id}`");
                        Console.WriteLine($"workspace id is null: {id == null}");
                    }
                    return result;
                });

output

:

Found 12 workspaces:
  `<obfuscated>`: ``
workspace id is null: True
  `<obfuscated>`: ``
workspace id is null: True

and if i try to actually use the value, it throws an exception:

code

:

var workspaceId = workspaces.Apply(result => result.Ids[workspaceName].ToString());

output

:

error: Running program '<obfuscated>' failed with an unhandled exception:
System.NullReferenceException: Object reference not set to an instance of an object.
   at <obfuscated>+(GetMwsWorkspacesResult result) => { } in <obfuscated>:line 195

again, this previously worked. i'm currently working around it with a hardcoded static dictionary that contains the workspace ids, but obviously that's not ideal, as it would be better practice to use state.

[VenelinMartinov]

Thanks for reporting back @pstrittmater-sd. I've managed to reproduce the problem with the following program:

using System;
using Pulumi;
using Pulumi.Databricks;

return await Deployment.RunAsync(() =>
{
    var workspaces = GetMwsWorkspaces.Invoke();

    workspaces.Apply(result =>
                {
                    Console.WriteLine($"Found {result.Ids.Count} workspaces:");
                    return result;
                });
});

I'l confirm if this is a regression next.

[VenelinMartinov]

This does not seem to be a regression - I went back 2 years to pulumi-databricks v1.8.0 and the behaviour is the same. This might a result of a configuration change in the provider. Perhaps the datasource does not handle no workspaces well. I'll test that next.

Confirmed we have no workspaces configured.

Once I added a workspace the program now runs fine.

@pstrittmater-sd can you please confirm if your account does not have any workspaces? That seemed to be the source of the issue for me. It's unfortunate that the provider does not handle that very well - I'll see why that is.

[VenelinMartinov]

This looks to be an issue in Terraform as well:

terraform {
  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = ">= 1.63.0"
    }
  }
}

data "databricks_mws_workspaces" "example" {}

output "workspace_ids" {
  value = data.databricks_mws_workspaces.example.ids
}

output "workspace_count" {
  value = length(data.databricks_mws_workspaces.example.ids)
}

I'll report it upstream.

The workaround is to add an explicit null check:

return await Deployment.RunAsync(() =>
{
    var workspaces = GetMwsWorkspaces.Invoke();

    workspaces.Apply(result =>
                {
                    if (result.Ids == null)
                    {
                        Console.WriteLine("No workspaces found");
                    }
                    else
                    {
                        Console.WriteLine($"Found {result.Ids.Count} workspaces:");
                    }
                    return result;
                });
});

[VenelinMartinov]

Raised databricks/terraform-provider-databricks#4413 upstream as well as a PR: databricks/terraform-provider-databricks#4414