Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] When importing events preserve VTIMEZONES #513

Merged
merged 13 commits into from
Oct 6, 2016
Merged

Conversation

geier
Copy link
Member

@geier geier commented Oct 4, 2016

This is an important fix for an issue reported in #498 by @waynew.
The underlying issue can lead to data-loss or rather wrongly displayed events, see draft for release announcement.

@untitaker @hobarrera while this is not entirely finished yet, it would be great if you had the time to look over this already, as I want to get it out as soon as possible.

Todo

  • re-enable setting of random-uid
  • backport to v0.8.x
  • put VEVENTs into ics file in "right" order (deferred, because of a nasty circular dependency if we were to import sort_key into khal/utils.py that needs to be resolved first, but should be okay, as we keep the original order and when actually inserting it into the db we order it correctly

fixes #498

@waynew
Copy link

waynew commented Oct 4, 2016

If there's anything that I can test out I'll be happy to :)

Copy link
Member

@untitaker untitaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i

# print all sub-events
for sub_event in vevent:
cal = icalendar.Calendar.from_ical(vevent)
for sub_event in [item for item in cal.walk() if item.name == 'VEVENT']:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just have a if item.name == 'VEVENT': continue in the for-loop. This creates a new list for no good reason.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using () instead of [] would at least make it a generator comprehension instead of listcomp, and have the same effect of skipping creation of an intermediate list.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's still overhead. I don't see a reason to use either list comprs or generator exprs.

same UID, i.e., one "master" event and (optionally) 1+ RECURRENCE-ID events
:type vevent: list(str)
"""
# TODO re-enable random_uid
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calling khal import --random-uid somefile.ics would import all events but assign a random uid to each VEVENT, I want to re-enable that functionality.

if datetime_.tzinfo is None and 'TZID' in item.params and \
item.params['TZID'] not in missing_tz:
logger.warn(
'Cannot find timezone `{}` in .ics file, using default timezone. '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think khal should print the event and ask for the tz

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably a good idea, not sure if we need to address it with this PR though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be useful as a separate command, like repair, which might eventually have other features as well.

I agree that it's best left outside the scope of this issue (so we can close this bug faster).

'information in {} in {}'.format(prop, href, calendar))
logger.warn(
"{} localized in invalid or incomprehensible timezone `{}` in {}/{}. "
"This could lead to this event being wronlgy displayed."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo.

@@ -37,12 +38,14 @@


def timefstr(dtime_list, timeformat):
"""converts a time (as a string) to a datetimeobject
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change, also why is that API designed like that? Why not take just a time as a string? Why does it delete the item from the list?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for consistency reason with the other xfstr()s

:rtype list:
"""
cal = icalendar.Calendar.from_ical(ics)
vevents = [item for item in cal.walk() if item.name == 'VEVENT']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary list. Can be again done in a single loop.

@geier geier force-pushed the fix_import_timezone branch from 412f444 to e2842df Compare October 6, 2016 15:15
@geier geier force-pushed the fix_import_timezone branch 2 times, most recently from d103d67 to 21adf38 Compare October 6, 2016 15:57
@geier geier force-pushed the fix_import_timezone branch from 21adf38 to 9334546 Compare October 6, 2016 16:03
geier added a commit that referenced this pull request Oct 6, 2016
@geier geier force-pushed the fix_import_timezone branch from 9334546 to 7b360dc Compare October 6, 2016 16:51
@geier geier merged commit e805062 into master Oct 6, 2016
@geier
Copy link
Member Author

geier commented Oct 6, 2016

@pimutils/packaging There is a new release of khal (0.8.4), as it fixes a bug in import that can lead to data loss, please update all packages as soon as possible. If you need any help with that, please ping me.
Also, if you are maintaining a v0.7.x package that can not be upgrade because of the python 2/3 issue (and people are actually using it), ping me, so I can backport this fix onto 0.7.0.

@geier geier removed the in progress label Oct 6, 2016
@geier geier deleted the fix_import_timezone branch October 6, 2016 19:21
@part1zano
Copy link

Updating the port right now. I guess it might get committed during the following week or so.

@part1zano
Copy link

Couldn't upgrade the port now as it contains py3-specific code. Could you kindly release something like 0.7.5?

@untitaker
Copy link
Member

Khal 0.8 doesn't support Python 2.

@untitaker
Copy link
Member

Ok, it appears @geier offered to backport this to 0.7 anyway, so nvm

@part1zano
Copy link

@untitaker thank you for the info!

@untitaker
Copy link
Member

The same thing applies for vdirsyncer 0.13 btw, but it has been outputting warnings about Python 2 for a long time now.

@untitaker
Copy link
Member

BTW feel free to open issues if you want to have a different way to be notified about releases.

@mathstuf
Copy link

mathstuf commented Oct 8, 2016

Building for Rawhide now; will copy down to stable releases afterwards.

As for notifications, a separate issue with a better subject line would be better at least.

@mathstuf
Copy link

mathstuf commented Oct 8, 2016

Fedora 23 is still using the Python 2 version, so a new 0.7 would be good there.

@part1zano
Copy link

Regarding notifications, I wish a system that notifies me via email. The mentions thing implemented here works just fine in this case. If the separate issue approach mentioned above works the same way, I'm for it.

@untitaker
Copy link
Member

Let's continue in pimutils/pimutils.org#5

geier added a commit that referenced this pull request Oct 10, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DTSTART/DTEND has invalid or incomprehensible timezone information from Outlook appointment
6 participants