Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use package_revise instead of resource_create #28

Closed
paulmueller opened this issue Aug 30, 2021 · 3 comments
Closed

Use package_revise instead of resource_create #28

paulmueller opened this issue Aug 30, 2021 · 3 comments
Labels
bug Something isn't working

Comments

@paulmueller
Copy link
Member

Affected code:

# Attempt upload
e = MultipartEncoder(fields={
'package_id': dataset_id,
'name': resource_name,
'upload': (resource_name, path.open('rb'))})
m = MultipartEncoderMonitor(e, monitor_callback)
# perform upload
data = api.post("resource_create",
data=m,
dump_json=False,
headers={"Content-Type": m.content_type})

Using package_revise instead of resource_create was @wardi's suggestion in the ckan gitter.im chat. package_revise can be used with the __extend syntax to support file uploads.

The original problem were race conditions when resources were added and at the same time package_revise was called from other threads. package_revise should be atomic if and only if only package_revise is used.

@paulmueller paulmueller added the bug Something isn't working label Aug 30, 2021
@paulmueller
Copy link
Member Author

ckan/ckanapi#181

@paulmueller
Copy link
Member Author

In 4db9a95, a temporary workaround for failing tests or hanging "online" states is implemented. If package_revise works, then everything will be much more efficient.

@paulmueller
Copy link
Member Author

Package revise works now. However, the server-side implementation is still pending. Currently, I am using a workaround for ckan/ckan#2949 and manually call after_create and generate the resource views (see ckan/ckan#6472).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant