-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fails to convert #113
Comments
Also, I noticed that the |
I think you missed step 3. This is a critical step since the rest of the process depends on the I was able to convert the data source you link above following these steps:
node bin/gtfs2lc.js -f jsonld -o path/to/output/folder /path/to/source/gtfs Note: The period covered by this GTFS source is quite long (from 2021-02-11 until 2021-12-11) and given its high amount of trips, it will result in a very big Linked Connections file. I didn't have enough disk space to completely process it, so for testing purposes I changed its coverage period (in As for the failing tests you mentioned, I think it is because you are also lacking the sorting pre-processing there. If you check the testing command in package.json you will see that it performs this step first. Running the tests using the npm command Please let me know if this solves the issue :) |
Thanks for trying this out!
I didn't pay enough attention to the log output. It fails (without aborting!) on the $ ../gtfs2lc/bin/gtfs2lc-sort.sh gtfs
# Converting newlines dos2unix
# Removing UTF-8 artifacts in directory gtfs
# sort: -k d,: Invalid argument
# Creating connection files according to the number of CPU processors available
# Converted 0 stop_times to connections
# Sorting files in directory gtfs
$ ls -l $(which sort)
# -rwxr-xr-x 1 root wheel 74912 Sep 22 02:30 /usr/bin/sort
$ man sort
#
# SORT(1) BSD General Commands Manual SORT(1)
#
# NAME
# sort -- sort or merge records (lines) of text and binary files
#
# SYNOPSIS
# sort [-bcCdfghiRMmnrsuVz] [-k field1[,field2]] [-S memsize] [-T dir] [-t char] [-o output] [file ...]
# sort --help
# sort --version
#
# DESCRIPTION Could it be that your Also, in #115, I have added a |
That is very strange. This is the version of
|
you have GNU sort, i have the BSD sort. almost always, an up-to-date GNU CLI has more features than the macOS BSD one. same with if you're on a gnu/linux system, it's because then the gnu variants are canonical. if you're on macOS, it's probably because you have installed sort via homebrew. can you run |
So is a problem with macOS
|
I'll run with |
Unfortunately, the current $ head -n 1 gtfs/stop_times.txt
# "tip_id","arrival_time","departure_time","stop_id","stop_sequence","pickup_type","drop_off_type","stop_headsign"
$ head -n 1 gtfs/trips.txt
# "oute_id","service_id","trip_id","trip_headsign","trip_short_name","direction_id","block_id","shape_id","wheelchair_accessible","bikes_allowed" You could use Miller & mlr --csv sort -f trip_id trips.csv | sponge trips.csv
mlr --csv sort -f trip_id -n stop_sequence stop_times.csv | sponge trips.csv
mlr --csv sort -f service_id calendar.csv | sponge trips.csv
mlr --csv sort -f service_id,date calendar_dates.csv | sponge trips.csv It would also take care of the UTF-8 Byte Order Mark (BOM) & non-Unix-newlines in one pass, and prevent the current race conditions with |
I also tried running with gsort --version
# sort (GNU coreutils) 9.0
# Copyright (C) 2021 Free Software Foundation, Inc.
# License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
# This is free software: you are free to change and redistribute it.
# There is NO WARRANTY, to the extent permitted by law.
# Written by Mike Haertel and Paul Eggert.
/bin/gtfs2lc-sort.sh gtfs
# Converting newlines dos2unix
# Removing UTF-8 artifacts in directory gtfs
# Trimming EOLs and removing continuous double quotes
# sort: -k d,: Invalid argument
# Creating connection files according to the number of CPU processors available
# Converted 0 stop_times to connections
# Sorting files in directory gtfs |
I tried to convert the 2021-02-12 VBB GTFS feed.
It has also created 4 files inside
gtfs
:The text was updated successfully, but these errors were encountered: