Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gazer failing to Export Dashboards #244

Closed
jondu2022 opened this issue Apr 11, 2024 · 21 comments
Closed

Gazer failing to Export Dashboards #244

jondu2022 opened this issue Apr 11, 2024 · 21 comments
Assignees
@drstrangelooker
Labels
need triage p3

Comments

@jondu2022
Copy link

jondu2022 commented Apr 11, 2024 •
edited
Loading

C:\Users\username>gzr space export 1093 --host myhosturl: --no-verify-ssl --zip demo.zip
Error querying query(12345)
(GET https://myhosturl:19999/api/4.0/queries/12345: 404 - Not found
// See: https://cloud.google.com/looker/docs/r/err/4.0/404/get/queries/:query_id
// And: https://marketplace-api.looker.com/errorcodes/404.md)
I find there is a zip file generated in my local folder however, there is No dashboard json file, I only see some Look json files and one folder json file,
by the way, when I run "gzr user me" with the same host below, it is working fine.
Please advise
@jondu2022
Copy link
Author

Please note the host url with a little change to mark its real one

@drstrangelooker
Copy link
Collaborator

What version of gzr are you running?

@jondu2022
Copy link
Author

jondu2022 commented Apr 12, 2024 •
edited
Loading

@drstrangelooker
I find it is "user_agent=>Gazer 0.3.10"
Yes, confirmed by running ">gzr version
v0.3.10".
Do I need to upgrade to newest version by running the command "$ gem install gazer"?
Or any other way to upgrade it? Please advise

I run this command and it says it successfully installed ,

gem install gazer
Fetching gazer-0.3.14.gem
Successfully installed gazer-0.3.14
Parsing documentation for gazer-0.3.14
Installing ri documentation for gazer-0.3.14
Done installing documentation for gazer after 1 seconds
1 gem installed

Ruby version is ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [x64-mingw-ucrt]

But the error still here --------- error still here

@jondu2022
Copy link
Author

@drstrangelooker
I use the similar 'gzr space export newfoldernumber --host myhosturl --no-verify-ssl --zip export_testlook.
zip'. It is successful to download it to my local machine.
That means, some succeed, some fails. But I run the command in the past, it works

@jondu2022
Copy link
Author

Hey @drstrangelooker
Any updates yet?
Looking forward to your help.
thanks

@drstrangelooker
Copy link
Collaborator

I still don't have an answer. There were some changes to this code from 3.10 to 3.14 so I would have expected the latest version to work. Still investigating.

@jondu2022
Copy link
Author

I still don't have an answer. There were some changes to this code from 3.10 to 3.14 so I would have expected the latest version to work. Still investigating.

Thanks @drstrangelooker Looking forward to your magic,
By the way, I also try this Looker Deployer which is also not working for me.
Looker Deployer is the wrapper of Gazer.
https://www.googlecloudcommunity.com/gc/Administering-Looker/Gazer-Cannot-export-Dashboard-from-folder/m-p/736559#M1456

@drstrangelooker
Copy link
Collaborator

If you can run this with version 0.3.14 and the --debug flag then post the whole error I might get some information that would help me.

@jondu2022
Copy link
Author

Hey @drstrangelooker
Please see the error below, I add "--debug"
Looking forward to your magic

C:\Users\Username>gzr space export 1428 --host our_host_url  --no-verify-ssl --debug --zip export_CA_Inhouse_Overview20240423.zip
options: {"debug"=>true, "host"=>"our_host_url", "port"=>"19999", "ssl"=>true, "verify_ssl"=>false, "timeout"=>60, "force"=>false, "token_file"=>false, "dir"=>".", "zip"=>"export_CA_Inhouse_Overview20240423.zip"}
using options ["debug=>true", "host=>our_host_url", "port=>19999", "ssl=>true", "verify_ssl=>false", "timeout=>60", "force=>false", "token_file=>false", "dir=>.", "zip=>export_CA_Inhouse_Overview20240423.zip"]
API current_version 4.0
Supported API versions ["4.0"]
connecting to ["api_endpoint=>https://our_host_url:19999/api/4.0", "connection_options=>{:ssl=>{:verify=>false, :verify_mode=>0}, :request=>{:timeout=>60}}", "user_agent=>Gazer 0.3.14", "netrc=>true", "netrc_file=>~/.netrc"]
check for connectivity: true
verify authentication: true
{:dashboard_element_id=>"3905", :id=>"9"}
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
{:dashboard_element_id=>"3905", :id=>"9"}
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
{:dashboard_element_id=>"3905", :id=>"9"}
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
{:dashboard_element_id=>"3905", :id=>"9"}
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
{:dashboard_element_id=>"3905", :id=>"9"}
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Found alerts []
Looked up alerts entries []
Error querying query(47271)
GET https://our_host_url:19999/api/4.0/queries/47271: 404 - Not found
 // See: https://cloud.google.com/looker/docs/r/err/4.0/404/get/queries/:query_id
 // And: https://marketplace-api.looker.com/errorcodes/404.md

C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/looker-sdk-0.1.6/lib/looker-sdk/response/raise_error.rb:37:in `on_complete'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/faraday-2.7.10/lib/faraday/middleware.rb:18:in `block in call'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/faraday-2.7.10/lib/faraday/response.rb:42:in `on_complete'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/faraday-2.7.10/lib/faraday/middleware.rb:17:in `call'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/faraday-2.7.10/lib/faraday/rack_builder.rb:153:in `build_response'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/faraday-2.7.10/lib/faraday/connection.rb:444:in `run_request'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/faraday-2.7.10/lib/faraday/connection.rb:200:in `get'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/sawyer-0.9.2/lib/sawyer/agent.rb:99:in `call'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/looker-sdk-0.1.6/lib/looker-sdk/client.rb:316:in `request'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/looker-sdk-0.1.6/lib/looker-sdk/client.rb:99:in `get'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/looker-sdk-0.1.6/lib/looker-sdk/client/dynamic.rb:146:in `invoke_remote'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/looker-sdk-0.1.6/lib/looker-sdk/client/dynamic.rb:107:in `method_missing'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/command.rb:79:in `query'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/modules/dashboard.rb:304:in `block (2 levels) in cat_dashboard'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/modules/dashboard.rb:302:in `each_index'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/modules/dashboard.rb:302:in `block in cat_dashboard'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/modules/dashboard.rb:283:in `each_index'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/modules/dashboard.rb:283:in `cat_dashboard'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/commands/folder/export.rb:108:in `block in process_folder'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/commands/folder/export.rb:107:in `each'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/commands/folder/export.rb:107:in `process_folder'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/commands/folder/export.rb:77:in `block in execute'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/modules/session.rb:306:in `with_session'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/commands/folder/export.rb:51:in `execute'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/lib/gzr/commands/folder.rb:84:in `export'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:116:in `invoke'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:243:in `block in subcommand'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/gazer-0.3.14/exe/gzr:36:in `<top (required)>'
C:/Ruby31-x64/bin/gzr:32:in `load'
C:/Ruby31-x64/bin/gzr:32:in `<main>'
logout
ERROR: Not found

@jondu2022
Copy link
Author

Hey @drstrangelooker
Inside No Dashboard file
Even I get the error but I can find the zip folder on my machine, however, I do not see the 'dashboard json file' in the folder.
See the attached screenshot,

@jondu2022
Copy link
Author

Hwy @drstrangelooker
Hope you are well.
Any updates yet?
Looking forward to your fix

@jondu2022
Copy link
Author

Hi @drstrangelooker
Hope you are well.
Any updates yet?
Looking forward to your fix.

By the way My colleague found for whatever reasons, the dashboard is still working but it is kind of corrupted. Then we add all those tiles on the dashboard in problem to the new Dashboard then re-link those dashboard filters. We have to use the new dashboard then export it and import it to all other instances.

@drstrangelooker
Copy link
Collaborator

I haven't found an underlying issue that I can duplicate, so it is probably some sort of corruption or weird state in your instance.

@jondu2022
Copy link
Author

jondu2022 commented May 9, 2024 via email

@ashamapant
Copy link

ashamapant commented May 20, 2024 •
edited
Loading

Hi, I am @jondu2022's colleague. Adding few more points from my end.
The errors are similar to the one below
GET https://host/api/4.0/queries/105799: 404 - Not found

  1. checked with Looker support team and they have confirmed that Get Query API end point now needs a query slug and not a query ID. Since we are on a customer hosted Looker instance and that we are on one of the supported Looker versions 24.0, we do not have any option to turn on/off the legacy feature. Looker hosted looker instances can still see the option to turn off "Disallow Numeric Queries ID option"
    https://cloud.google.com/looker/docs/best-practices/query-id-update-instructions#what_changed

  2. Export is an issue only with dashboards having merge queries.

Fix needs to be on Gazer end to be able to pass the query slug instead of the query id to be able to export dashboards.

We heavily use Gazer for our promotion management. We have releases planned in the near future and getting this fixed will help us immensely.
Any help here is highly appreciated.

Thanks

@ashamapant
Copy link

Hi there, Adding more information based on the investigation of the code. Experts, please advise if my understanding is incorrect.

Export fails for dashboards having merged queries.

  1. lib/gzr/command.rb -- query(query_id) functions - calls get query api end point via Looker SDK.
  2. lib/gzr/modules/dashboard.rb -- merge_result - Get merge query end point gives a list of all the source queries with their query ids. We are passing the query slugs / query ids to the query function in Step 1. Since the API call - merge query only furnishes query id, query ID is passed to Get Query API end point, which now works only with query slug. As a result, we end up with errors GET https://host/api/4.0/queries/105799: 404 - Not found

I have contacted Looker support team checking if there is a way we could extract the query slug using a query id via an API call or add query slug information to the Get merge query API call. They are yet to get back to me.

This issue might not exist with Looker hosted Looker instances as they have the option of turning off the legacy feature.

Happy for the experts to share their insights on this issue and advise if there are any ways of fixing this issue for customer hosted looker instances.

Thanks

@drstrangelooker
Copy link
Collaborator

I'm looking right now. We recently pushed a patch to handle query slugs instead of ids. The code tries to use a slug, then falls back to a query id if the slug fails.

@drstrangelooker
Copy link
Collaborator

Can you tell me what version of looker you are running? I have checked this case here and it is working properly.

@ashamapant
Copy link

Hi @drstrangelooker ,
Thanks for looking into.
We are on gazer 0.3.14

Can you please check if you can find this in legacy features of your Looker instance

Admin - Legacy
Disallow Numeric Query Ids - Is this "Off"? If Off - yes gazer will work as expected.

For information - we are not looker hosted looker instance and hence do not see the legacy feature to toggle off. It is turned on by default - which means Get Query APi end point will only accept query slugs and merge query API end point only gives query ids. We are on Looker version 24.0.39

Thanks

@drstrangelooker
Copy link
Collaborator

Can you upgrade to 24.0.72 and try again?

@drstrangelooker
Copy link
Collaborator

I confirmed that the code needed to get the query_slug with the merge_query shows up starting in 24.0.43. So if you upgrade this will work properly.

@drstrangelooker drstrangelooker self-assigned this May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@drstrangelooker drstrangelooker

Labels
need triage p3
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@drstrangelooker @ashamapant @jondu2022