Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using Artifactory.search() goes through the list of objects with GET -> slow performance #73

Open
iiro opened this issue Oct 24, 2016 · 1 comment

Comments

@iiro
Copy link

iiro commented Oct 24, 2016

Hi,

I'm trying to use the Artifactory.search() here like this:

Artifactory::Resource::Artifact.search(name: 'project*.gz', repos: 'project-artifacts-local')

And it does produce the list of objects yes - but it also goes through every object one by one with GET (at least by looking Artifactory's access.log) - which means it takes about a minute to run this (with current amount of artifacts on search scope).

Is this intentional?

@ThomVivet
Copy link

I encountered the same issue while using this library.
The large number of request is due to from_url() method which is called for every single objects returned by a search request. This method calls Artifactory::Client::request at the end...
For my needs I have extended the Resource::Artifact class with a new method (in order to deal with NPM packages).
This method use an AQL query and then the from_url() method only once for the element which will be downloaded further...
So performances are improved since only 2 requests to the server are needed to return the searched object.

Code exemple:

      def npm(options = {})
        client  = extract_client!(options)
        repos   = options[:repos]
        name    = options[:name]
        version = options[:version]

        aql = %Q(items.find({"repo":"#{repos}"},{"@npm.name":"#{name}"},{"@npm.version":{"$match":"#{version}"}}).include("repo","name","path").sort({"$desc":["name"]}))

        results = client.post('/api/search/aql', aql)['results']
        if not results.empty?
          sorted = results.sort {|x,y| Gem::Version.new(x['name'].match(/-(\d+[\.\d-]+)\.tgz/)[1].gsub('-', '.')) <=> Gem::Version.new(y['name'].match(/-(\d+[\.\d-]+)\.tgz/)[1].gsub('-', '.'))}
          npm = sorted[-1]

          # Call from_url only once in order to improve performances
          from_url(File.join('/api/storage/' + npm['repo'], npm['path'], npm['name']), client: client)
        else
          nil
        end
      rescue Error::HTTPError => e
        raise unless e.code == 404
        nil
      end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants