[WIP] Update in background job #49
Conversation
|
@phargogh before I wrap this up I thought I'd show you what I have so far here--feels like it will be a bit more convenient than what we have right now. |
| echo "Starting background jobs worker" | ||
| ckan jobs worker & |
There was a problem hiding this comment.
Start the jobs worker in the background.
There was a problem hiding this comment.
Would this need to be nohuped in order to keep it running past the end of the shell process, and still have it running in the background (thanks to &)?
There was a problem hiding this comment.
Ah yeah that could be necessary, locally this works but I can see how it would not when deployed. I can try it on staging first.
| def after_dataset_update(self, context, package): | ||
| resources = [res.as_dict(core_columns_only=False) for res in context['package'].resources] | ||
| toolkit.enqueue_job(update_dataset, [context['user'], package, resources]) |
There was a problem hiding this comment.
Hook that is fired on dataset update via the interface or API.
| ctx = { 'user': user } | ||
| updates = {'id': dataset['id'], 'extras': extras} | ||
| toolkit.get_action('package_patch')(ctx, updates) |
There was a problem hiding this comment.
We patch the dataset with our updated extras to avoid overwriting other fields.
| def update_mappreview(dataset, metadata, extras): | ||
| # TODO | ||
| return extras |
There was a problem hiding this comment.
Will port over from sync-datasets.py.
|
@phargogh I believe this is ready now! I tried this on staging without a Steps to test:
So now adding / editing via the interface or API should have the same result as running |
This is a work in progress PR, please do not merge.
The intention here is to use background jobs to update dataset metadata needed for zip file expansion and map previews. Currently we run a separate script to do this which can lead to confusion or errors.
To do: