diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/404.html b/404.html new file mode 100644 index 00000000..fd5635ec --- /dev/null +++ b/404.html @@ -0,0 +1,939 @@ + + + +
+ + + + + + + + + + + + + + + + +Skyscraper allows you to fully customize how you want the final frontend artwork to appear by editing the file /home/<USER>/.skyscraper/artwork.xml
. Each piece of artwork can be composited from one or more of the other artwork resources and you can even apply several effects to them.
<output>
node exports a single piece of artwork.<layer>
nodes are nested in an output node, it will simply export the defined type as raw unmanipulated artwork.Read on for an example artwork.xml
and a more thorough description of the <output>
, <layer>
and the various available effect nodes.
Warning
+Some users seem to have a habbit of editing the /home/<USER>/skysource/artwork.xml
file. This is not the correct file! Please read the first paragraph. :)
Tip
+Every time you change the artwork configuration, remember to regenerate the game list. Your changes won't take effect until you do so. Read more about this here.
+Watch a video demonstrating the artwork compositing features here. Note that the video is quite old and only demonstrates a fraction of the possibilities you have with the Skyscraper compositor. It's all thoroughly documented below.
+An example could look something like this:
+<?xml version="1.0" encoding="UTF-8"?>
+<artwork>
+ <output type="screenshot" width="640" height="480">
+ <layer resource="screenshot" x="20" width="520" height="390" align="center" valign="middle">
+ <rounded radius="10"/>
+ <stroke width="5"/>
+ </layer>
+ <layer resource="cover" height="250" x="0" y="-10" valign="bottom">
+ <gamebox side="wheel" rotate="90"/>
+ <shadow distance="5" softness="5" opacity="70"/>
+ </layer>
+ <layer resource="wheel" width="250" x="-10" align="right">
+ <shadow distance="5" softness="5" opacity="70"/>
+ </layer>
+ </output>
+</artwork>
+
Which results in a screenshot being exported to look like this:
+ +Click the following links to quickly go to a desired section:
+Nodes: XML Preamble, Artwork node, Output node, Layer node
+Effects: Blur effect, Brightness effect, Color balance effect, Colorize effect, Contrast effect, Frame effect, Gamebox effect, Hue effect, Mask effect, Opacity effect, Rotate effect, Rounded effect, Saturation effect, Scanlines effect, Shadow effect, Stroke effect.
+Other: Custom image resources.
+This is a special node used to describe the XML language used in the document. Don't change it.
+This node is the document base node. It contains all other nodes. Don't change or delete it.
+The output node describes a single piece of artwork to be exported when scraping. You can have as many (or few) output nodes as you like.
+An output node with no nested 'layer' nodes will export the type with no compositing.
+The type to be exported. Can be:
+The resource attribute can be set to one of the following:
+For instance, if you wish to export a 'marquee' image but want it to make use of the 'wheel' resource instead, you can set <output type="marquee" resource="wheel"/>
. It will then export a 'marquee' but use the raw unmanipulated 'wheel' image when doing so. So if your frontend theme, such as some EmulationStation themes, makes use of the 'marquee' artwork, it will then be using the 'wheel' for it. If this attribute is left out, 'resource' will be set to the same as 'type'.
Info
+The screenshot
, cover
, wheel
and marquee
types are hardcoded. It's not currently possible to create custom types.
Set the desired area size for the image resource in mpixels (eg '0.1'). This can be used as an alternative to setting width or height and is useful in cases where images vary a lot in size. 'width' and 'height' attributes will be ignored if this is set.
+Sets the width and height of the artwork export. If both are left out it will use the original size of the scraped artwork. If one is left out, it will keep the aspect ratio relative to that.
+<layer resource="cover" mode="overlay" opacity="50" x="5" y="5" mpixels="0.1" width="10" height="10" align="center" valign="middle"/>
+...
+</layer>
+
Like layers in Gimp or almost any other respectable photo manipulation software, you can add as many of these nodes as you like. You can even nest them inside of each other, in which case the child layer will be anchored to the topleft corner of the parent layer, making it easier to align.
+Note
+Layer and effect nodes are rendered from top to bottom.
+The artwork resource to be used by the layer. It can be:
+If the resource attribute is left out the layer will simply be transparent. This is useful when aligning nested layers that differ in size across games.
+Sets the layer render mode. It can be:
+If left out the layer is simply rendered as is on top of the parent layer.
+ +Defines the opacity of the layer. 100 is completely visible. 0 is completely transparent.
+X and Y coordinates for placement. Relative to 'align' and 'valign' and the parent layer. Either can be left out in which case it will be set to 0.
+Set the desired area size for the image resource in mpixels (eg '0.1'). This can be used as an alternative to setting width or height and is useful in cases where images vary a lot in size. 'width' and 'height' attributes will be ignored if this is set.
+Sets the width and height of the layer. If either is left out it will keep the aspect ratio relative to the other. If both are left out it will use the original dimensions of the scraped resource.
+The horizontal alignment of the layer. It can be:
+The alignment is relative to the parent layer.
+The vertical alignment of the layer. It can be:
+The alignment is relative to the parent layer.
+Must be nested inside a layer node. Adjusts the color balance of the parent layer.
+The red color adjustment. Can be -255 to 255.
+The green color adjustment. Can be -255 to 255.
+The blue color adjustment. Can be -255 to 255.
+Must be nested inside a layer node. Blurs the parent layer.
+Defines the radius of the blur. Higher means blurrier.
+Must be nested inside a layer node. Adjusts the brightness of the parent layer.
+The difference value for the adjustment. Can be -255 to 255.
+Must be nested inside a layer node. Colorizes the parent layer with a single hue.
+Sets the hue in degrees. Can be 0 to 360.
+Sets the saturation delta value of the colorize effect. Can be -127 to 127.
+Must be nested inside a layer node. Adjusts the contrast of the parent layer.
+The difference value for the adjustment. Can be -255 to 255.
+Must be nested inside a layer node. This applies a frame to the parent layer.
+Tip
+You can get some cool results by first applying a mask, then applying a frame. This is useful for frames that aren't square.
+The filename of the custom image resource to be used as frame.
+Sets the width of the frame in pixels. If left out it will be set to the width of the parent layer.
+Sets the height of the frame in pixels. If left out it will be set to the height of the parent layer.
+Sets the x coordinate of the frame relative to the parent layer. If left out it will be set to 0.
+Sets the y coordinate of the frame relative to the parent layer. If left out it will be set to 0.
+Left image shows the result with the wheel artwork applied to the side. Right image shows the original cover layer.
+<layer resource="cover" height="200">
+ <gamebox side="[custom image resource]" rotate="90" sidescaling="both"/>
+</layer>
+
Must be nested inside a layer node. Renders a nifty looking 3D game box. It uses the parent layer image on the front of the box.
+The filename of the custom image resource to be used on the side of the box.
+Defines the rotation of the side image in degrees.
+Defines how to scale the side / spine image. If left out, it will autoscale depending on the size of the artwork resource. Can be 'width', 'height' or 'both'.
+Must be nested inside a layer node. Rotates the hue of the parent layer.
+The difference value for the adjustment. Can be 0 to 359.
+Left image shows the result. Right image shows the mask used. The white part of the mask is transparent.
+ +Must be nested inside a layer node. This applies a mask to the parent layer.
+The filename of the custom image resource to be used as mask. The alpha channel of the mask will blind out the underlying parts of the parent layer.
+Sets the width of the mask in pixels. If left out it will be set to the width of the parent layer.
+Sets the height of the mask in pixels. If left out it will be set to the height of the parent layer.
+Sets the x coordinate of the mask relative to the parent layer. If left out it will be set to 0.
+Sets the y coordinate of the mask relative to the parent layer. If left out it will be set to 0.
+Must be nested inside a layer node. Adjusts the opacity of the parent layer.
+The opacity of the layer. Can be 0-100 where 0 is completely transparent and 100 is opaque.
+Must be nested inside a layer node. Rotates the parent layer around the x, y or z axis.
+Sets how many degrees the parent layer will be rotated. Can be -360 to 360.
+Sets which axis the parent layer should be rotated around. Can be 'x', 'y' or 'z'. If left out it will be set to 'z'.
+Must be nested inside a layer node. Rounds the corners of the parent layer.
+Defines the radius of the corners in pixels.
+Must be nested inside a layer node. Adjusts the color saturation of the parent layer.
+Tip
+If you want to completely greyscale your layer I highly recommend using the 'saturation' attribute of the 'colorize' effect instead. It uses a better estimation of percieved luminance.
+The difference value for the adjustment. Can be -255 to 255.
+Must be nested inside a layer node. Adds a CRT monitor scanline effect to the parent layer.
+The filename of the custom image resource to be used as the scanline overlay. If left out it defaults to scanlines1.png
.
Sets the scale factor of the overlay. Can be 0.1 to 2.0. If left out it defaults to 1.0.
+Sets the opacity of the overlay. Can be 0 to 100. If left out it defaults to 100.
+Sets the overlay compositing mode. It can be:
+If left out the overlay is rendered using the overlay
method.
Must be nested inside a layer node. Renders a dropshadow on the parent layer using the attributes provided. If either of the attributes are left out, the shadow won't be drawn.
+Distance in pixels from the layer. The distance is always down to the right.
+Defines how soft (radius) the shadow will appear. A value of 0 is sharpest.
+Defines the opacity of the shadow. 100 is completely visible. 0 is completely transparent.
+Must be nested inside a layer node. Renders a colored outline on the parent layer. If all color attributes are left out, it averages a suitable color from the parent layer.
+The width of the outline in pixels.
+The red color value for the outline. Can be 0-255. If left out it is set to 0.
+The green color value for the outline. Can be 0-255. If left out it is set to 0.
+The blue color value for the outline. Can be 0-255. If left out it is set to 0.
+Provides the color to use hex-style. This can be used instead of the 'red', 'green' and 'blue' attributes described above. An example could be 'color="#ff0099"'.
+From Skyscraper version 2.3.0 you can use custom image resources wherever the documentation says so. Place your custom resources in the '/home/<USER>/.skyscraper/resources
' folder and use it by adding the filename to the attribute.
Example:
+ +The file megadrive_logo.png will then be loaded from the '/home/<USER>/.skyscraper/resources/megadrive_logo.png
' file. Feel free to create subfolders inside the resources folder. If you do, remember to add the partial path to the filename like so:
This will look for the file at '/home/<USER>/.skyscraper/resources/platform_logos/megadrive_logo.png
'.
Warning
+All custom image resources are loaded into memory when Skyscraper starts to avoid repeatedly reading them from disc / SD card. So please keep your resources to a minimum - otherwise it might exceed the total memory on the Pi which will cause Skyscraper to get killed by the kernel.
+Whenever you scrape any platform with any scraping module, Skyscraper caches each resource internally. A resource can, for instance, be a game description
or a game screenshot
. Each game can have several versions of each resource cached locally. One of each type per scraping module. This comes in handy when generating the game list and artwork compositions.
Think of the resource cache as the cache in an internet browser. Most of the data on webpages don't change much. By caching some of the data locally, it can load parts of a webpage from that data instead of loading it from the remote server. This is exactly how the Skyscraper cache works. It helps keep the online servers healthy by not hammering them whenever you need resources you already downloaded once. And it allows you to re-generate the frontend game lists if you add new games or perhaps want to change the style of the exported artwork.
+Default resource cache folder
+The default base folder for all of Skyscrapers' locally cached data is in the /home/<USER>/.skyscraper/cache
folder. In this folder you'll find individual platform subfolders. Any of these are selfcontained and contains all of the cached data for that particular platform.
Resource and scraping module priorities
+There is ONE file that you can and should edit inside each of the /home/<USER>/.skyscraper/cache/<PLATFORM>
folders. That file is called priorities.xml
and decides the scraper priority of resources for each resource type. For instance, if you know that thegamesdb
always provides the best descriptions
for games, you'd add an <order type="description">
node with a <source>thegamesdb</source>
subnode. You can have multiple <source>
nodes, Skyscraper will then prefer the topmost source when generating a game list. If the topmost isn't found it'll prioritize the next one and so forth. Any source that isn't listed with an <order>
node will be prioritized using timestamps (newest wins) for when each resource was added to the cache. So you don't have to add all of them.
Skyscraper provides the example file /home/<USER>/.skyscraper/cache/priorities.xml.example
. Please don't edit this file manually, as it will be overwritten when you update Skyscraper. When a platform is scraped for the first time, it will automatically copy the example file to /home/<USER>/.skyscraper/cache/<PLATFORM>/priorities.xml
unless it already exists. You can of course also copy the file yourself before scraping a platform. If you do so, be sure to remove the .example
part of the filename so it's just called priorities.xml
.
Update locally cached data
+If you wish to update / refresh the locally cached resources for a particular platform and scraping module, Skyscraper provides the --refresh
option. If this flag is set on the command line, any data in the resource cache will be updated with the new incoming data.
If you wish to just refresh the data for a single rom simply scrape it with -p <PLATFORM> -s <SCRAPING MODULE> <RELATIVE OR FULL ROM PATH AND FILENAME>
and the locally cached data for that particular rom will be updated / refreshed. You can add more filenames one after the other if you like. If any filename or paths has spaces in it, remember to double-quote it like so "relative path/to rom/rom filename.sfc"
. You don't need the --refresh
option when updating single files - it is implicit.
When you've updated information in the resource cache, always remember to re-generate the game list by simply running Skyscraper -p <PLATFORM>
when you're done. The updated resources won't be visible in your frontend until you do.
Preferred way of editing the cache is via the various --cache
commands.
Do not edit
+MOST FILES AND FOLDERS INSIDE THE /home/<USER>/.skyscraper/cache
FOLDER ARE NOT MEANT TO BE MANIPULATED BY HAND! It can be done, but don't complain to me about the format of the database. It is NOT meant to be understood by humans. It is meant to be efficient for reading and parsing by Skyscraper itself. Same goes for the media files that reside in the subfolders.
Don't leave custom files in here
+If you decide to add your own files to the subfolders, you risk them being deleted by Skyscraper later on if it is run with one of the cache cleanup command line options. You've been warned!
+Other cool stuff you CAN DO
+Each subfolder in the /home/<USER>/.skyscraper/cache/
folder is self-contained and can be copied to other Skyscraper installations at your convenience. Just copy the folder itself over to some other computer that has Skyscraper 1.6.0 or later installed, and you can make use of the data when generating game lists. If you add it at a non-default location, set the custom folder with -d <FOLDER>
.
I do not recommend editing the db.xml
resource cache files manually. But the format is simple, so you certainly can if you want to.
Resource id
+The database consists of resource entries connected to a unique id. The id is calculated from the rom data or, in special cases, the filename (in cases where the file data is a script or a ROM gets changed by the gameplay). An entry can look like this:
+<resource id="<ID KEY>"
+ type="<RESOURCE TYPE>"
+ source="<SCRAPING SOURCE>"
+ timestamp="<UNIX TIMESTAMP IN MSECS>">Resource data</resource>
+
Note
+Pre-3.3.0 versions of Skyscraper used sha1
as the name of the unique id key. Later versions use id
.
A game title
+A game platform
+A game description
+The publisher of a game
+The developer of a game
+How many players are supported by a game
+List of game tags, most often genre related
+The release date of a game
+Game age rating, defined as youngest allowed age in years
+Game rating, real number between 0 and 1
+A cover image filename for a game (file exists in covers
subfolder)
A screenshot image filename for a game (file exists in screenshots
subfolder)
A wheel image filename for a game (file exists in wheels
subfolder)
A marquee image filename for a game (file exists in marquees
subfolder)
A texture image (disc/cartridge) filename for a game (file exists in textures
subfolder)
A video file filename for a game (file exists in videos
subfolder)
This page summarizes the changes of each Skyscraper release, a changlog for +humans.
+innerBracketsReplace
for examples.
+ Thanks for the suggestion, @retrobit.peas.json
and platformid_map.csv
to make use of it.--hint
, it shows a random Tip of the Day./home/<USER>
replaced with the actual user's
+ home directory screen messages. Thanks for highlighting it on the Mac,
+ @cdaters whdload.xml
file for platform Amiga will be not
+ downloaded again until the server indicates. Manually removing
+ /home/<USER>/.skyscraper/whdload_cached_etag.txt
will force a new download.--cache edit
mode will now dismiss any changes
+ made instead of persisting thempeas_local.json
(same format as
+ peas.json
), to extend platform information or overwrite existing platform
+ information*_platform.json
files as reference for supported platforms of
+ various scraperspeas.json
. Thanks for hinting,
+ @vircon32scrapers
entries in peas.json
, as it did not provide any use.includeFiles
superseded by
+ includePattern
; excludeFiles
superseded by excludePattern
;
+ gamelistFolder
superseded by gameListFolder
; fromfile
superseded by
+ includefrom
. These were deprecated since v3.7.0.manuals=true
and
+ gameListVariants=enable-manuals
. Thanks for
+ the initial PR, @pandinoESDE_APPDATA_DIR
if
+ present. Thanks for the hint, @ASHGOLDOFFICIALaliasMap.csv
).
+ Verbose info in #59. Thanks,
+ @mjkayeonlymissing
for counting and scraping only
+ games which do not have any game data in the cache. This is a commodity config
+ option to the already existing flag with the same name. Plus: If you use a
+ scraper with a scraping limit for games to be scraped at once (e.g. MobyGames)
+ you may stay below that limit. Documentation here.
+ Thanks for the suggestion, @sleve_mcdichaelformats
in Skyscraper's peas.json
file) with
+ info from RetroPie's platform.cfg
(commit
+ 5e0ab1f
)frontend=esde
in config.ini
and see
+ documentation on the
+ default settings. Thanks for the hints and for testing, @maxexcloo, @NargashaliasMap.csv
+ are now also applicable for Screenscraper. Thanks, @retrobit.ss
) and screenshot from title screen (type:
+ sstitle
).<folder/>
nodes in gamelist or create skeleton
+ <folder/>
nodes when ROMs are stored in subfolders within a system folder,
+ see frontend documentation and the gamelist
+ specification.supplementary/bash-completion/Skyscraper.bash
into
+ $XDG_DATA_HOME/bash-completion/completions/
($XDG_DATA_HOME
is
+ equivalent to $HOME/.local/share
). Open a new bash -- et voila!make install
. See
+ PREFIX
in skyscraper.pro
.update_skyscraper.sh
(thanks, @jeantichoc) and Docker support via Dev
+ Container (kudos, @retrobit).tidyDesc
added. See documentation
+ hereCONFIGINI.md
. Details
+ see hereskyscraper.cpp
class. Factorised configuration settings
+ into settings.cpp
-p <PLATFORM>
nor --cache help
nor
+ --flags help
is providedscreenshot/
and as well
+ screenshots/
URL path as some platforms (supported since configurable
+ platforms) have their screenshots served from screenshots/
.--query=<gameid>
. Handy to hint to the right game information when usual
+ search returns false positives.<scraper>_platforms.json
files). These files are used as reference.platforms.json
.platforms.json
is
+ replaced by peas.json
(platforms, extensions/formats, aliases and scrapers).
+ Precise platform determination for Screenscraper, Mobygames and TGDB via
+ platform_idmap.csv
See platforms documentation for
+ details.gamelist.xml
internal rating range). See
+ also import formatsdb.xml
files.--query
free-text search in OpenRetro scraper fixed. This bug
+ did not occur when the switch is ommitted and an Amiga WHDLoad file is provided
+ to Skyscraper.--unattend
(Thanks
+ @windg)mamemap.csv
from MAME 0.240 (Arcade).dat, fbneo.dat,
+ mame2003-plus.xml and cleanup of surplus device informationmamemap.csv
does no longer rely on mame binary*.php
scriptsplatforms.json
sorted and formattedmobygames.cpp
.to
+ mobygames.json
These pull requests from other repos have been merged into this fork.
+Look below to find a thorough description of all command line options available when using Skyscraper. These options should be used with the Skyscraper
command.
Also consider that almost all of these options are set at a useful default (and can therefore be left out) and should only be used / changed if your use case requires it.
+Most of the options can also be set in the /home/<USER>/.skyscraper/config.ini
file thus removing the need to type them on command line all the time. Check the config.ini doc here for more info on this.
From version 3.9.3 onwards Skyscraper provides Bash Tab completion (aka +programmable completion).
+On RetroPie the
+scriptmodule
+will handle the installation. For the curious: It lands in
+/etc/bash_completion.d/
.
On non RaspiOS-based RetroPie-Installments put the file from
+supplementary/bash-completion/Skyscraper.bash
to
+$XDG_DATA_HOME/bash-completion/completions/
(respective to
+$HOME/.local/share/bash-completion/completions/
).
In either case: Open a new bash and press Tab twice to see it in action.
+The most prevalent short options you will use are most likely -s
and -p
. But
+you will information about the other options here too.
Sets a non-default XML file to use when setting up the artwork compositing. By default Skyscraper uses the file /home/<USER>/.skyscraper/artwork.xml
. If you provide a relative filepath it will be expanded relative to the current working directory. Consider setting this in config.ini
instead.
+Read more about the artwork.xml format and customization options here.
Example(s)
+ +Sets a non-default config file. By default Skyscraper uses the file /home/<USER>/.skyscraper/config.ini
. A relative filepath will be prepended with the current working directory. If you provide a config file that does not exist, Skyscraper will print a warning and will continue by using built-in default values for the configuration.
Example(s)
+ +Sets a non-default location for the storing and loading of cached game resources. This is what is referred to in the docs as the resource cache. By default this folder is set to /home/<USER>/.skyscraper/cache/<PLATFORM>
. Don't change this unless you have a good reason to. The folder pointed to should be a folder with a Skyscraper db.xml
file and its required subfolders inside of it (covers
, screenshots
etc.).
Note
+If you wish to always use a certain location as base folder for your resource cache ((for instance if you want your cache to reside on a USB drive), it is strongly recommended to set this in the config.ini
file instead. Read more about the relevant config.ini
option here.
Example(s)
+ +Info
+This option is applicable only when using the -f attractmode
or the -f pegasus
option.
When using -f attractmode
it is required to set the emulator to be used when generating the attractmode
game list. On RetroPie the emulator name is mostly the same as the platform. Consider setting this in config.ini
instead.
It can optionally be used with -f pegasus
to set the launch command used by the Pegasus game list. On RetroPie this defaults to the RetroPie launch command which works with RetroPie. Consider setting this in config.ini
instead.
Example(s)
+Skyscraper -p snes -f attractmode -e snes
+# On RetroPie
+Skyscraper -p snes -f pegasus -e "/opt/retropie/supplementary/runcommand/runcommand.sh 0 _SYS_ snes"
+
Sets the frontend you wish to export a game list for. By default Skyscraper will export an EmulationStation game list, but other frontends are supported as well. If exporting for example for the attractmode
frontend, please also take note of the required -e
option that goes along with using the attractmode
frontend. Consider setting this in config.ini
instead.
Check all supported frontends with '--help' and read a more about the details concerning each of them here.
+Example(s)
+ +Sets the game list export folder. By default Skyscraper exports the game list to the same directory as the rom input folder. This enables you to change that to a non-default location. Consider setting this in config.ini
instead.
Example(s)
+ +Outputs the help text for all command line options to the terminal.
+Example(s)
+ +Displays one of Skyscrapers's 'Did you know?' tips and exits.
+Sets the rom input folder. By default Skyscraper will look for roms in the /home/<user>/RetroPie/roms/<PLATFORM>
folder. If your roms are located in a non-default location, you can set the input path using this option. Consider setting this in config.ini
instead.
Example(s)
+ +Sets the maximum length of returned game descriptions. This is a convenience option if you feel like game descriptions are too long. By default it is set to 2500. Consider setting this in config.ini
instead.
Example(s)
+ +Some scraping modules are based on a filename or title based search. This option sets the minimum percentage any returned results need to match with in order for it to be accepted. For instance, the game Wonderboy in Monsterland
might return the title Wonder Boy in Monster Land
which is clearly a match. But it's not a 100% match. So it needs to be set relatively high, while still ignoring bad matches. By default it is set to 65 which has been tested to be a good middle-ground. Consider setting this in config.ini
instead.
Example(s)
+ +Sets the artwork / media output folder. By default Skyscraper outputs the composited artwork files to the game list export folder + /media
. This allows you to change that to a non-default location. Read more about the artwork compositing here. Consider setting this in config.ini
instead.
Example(s)
+ +Sets the platform you wish to scrape. Supported platforms can be seen using the --help
option described above.
Running the following commands will scrape from all cached resources and generate a game list and composite artwork using the recipe in /home/<USER>/.skyscraper/artwork.xml
(check the artwork documentation for more info on this here.
Before running these commands you need to first gather some data into the cache. Please read the description of -s <MODULE>
below.
Example(s)
+ +Sets which scraping module you wish to gather data from. All data scraped from any of the modules will be cached in the resource cache and can then later be used to generate a game list for your frontend. Read more about this in the -p <PLATFORM>
description above.
To generate a game list from the resource cache, just leave out the -s
option entirely.
Example(s)
+Skyscraper -p amiga -s openretro
+Skyscraper -p snes -s screenscraper
+Skyscraper -p amiga -s esgamelist
+Skyscraper -p snes -s import
+
To read more about each scraping module, check here.
+Sets the desired number of parallel threads to be run when scraping. By default it is set to 4.
+Note
+Some modules have maximum allowed threads. If you set this higher than the allowed value, it will be auto-adjusted.
+Example(s)
+ +Some scraping modules require a user key or a user id + password to work. Check the scraping module overview to see the specific requirements for each module here. Consider setting this in config.ini
instead.
Example(s)
+ +If you have a rom that Skyscraper doesn't even try to gather data for, it might be because it has a file extension that isn't currently supported. This option allows you to temporarily add support for any file extension. If you feel like you are using a file extension that ought to be supported by default, please report it so it can be added in a later version of Skyscraper. Consider setting this in config.ini
instead.
Example(s)
+Skyscraper -p snes -s thegamesdb --addext *.ext
+Skyscraper -p snes -s thegamesdb --addext "*.ext1 *.ext2"
+
This is the cache master option. It contains several subcommands that allows you to manipulate the cached data for the selected platform.
+Tip
+For any of these commands you can set a non-default resource cache folder with the -d
option. The folder pointed to should be a folder with a Skyscraper db.xml
file and its required subfolders inside of it (covers
, screenshots
etc.).
Read more about the resource cache here.
+Outputs a description of all available --cache
functions.
Allows editing of any cached resources connected to your roms. The editing mode will go through each of the files in the queue one by one, allowing you to add and remove resources as needed. Any resource you add manually will be prioritized above all others.
+You can provide one or more filenames to the end of the command line or use the --includefrom
option to edit the resources for just those files. You can use the --startat
and --endat
options to edit a span of roms. If none of those options are used, it will edit all of the roms in the input folder one by one.
For efficiency, when adding a lot of resources of the same type, you can also add the optional new=<TYPE>
which will make it very easy to batch insert resources of the defined type to all the files you are editing. <TYPE>
can be any of the known textual resources: title
, platform
, releasedate
, developer
, publisher
, players
, ages
, genres
, rating
, description
.
Example(s)
+Skyscraper -p snes --cache edit <FILENAME 1> <FILENAME 2>
+Skyscraper -p snes --cache edit --startat <FILENAME> --endat <FILENAME>
+Skyscraper -p snes --cache edit:new=developer --startat <FILENAME> --endat <FILENAME>
+Skyscraper -p snes --cache edit
+Skyscraper -p snes --cache edit --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_developer-20190708.txt"
+Skyscraper -p snes --cache edit:new=ages --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_ages-20190708.txt"
+
This option allows you to merge two resource caches together. It will merge the cache located at the <PATH>
location into the default cache for the chosen platform. The path specified must be a path containing the db.xml
file. You can also set a non-default destination to merge to with the -d
option.
Example(s)
+Skyscraper -p snes --cache merge:"path to/source/cache/snes"
+Skyscraper -p snes --cache merge:"path to/source/cache/snes" -d "/path to/nondefault/destination/cache/snes"
+
This is a powerful option that allows you to purge the requested resources from the resource cache connected to the selected platform.
+You can purge all resources from the cache for the chosen platform using the keyword all
.
You can purge specific resources from a certain module with m=<MODULE>
or of a certain type with t=<TYPE>
or a combination of the two separated by a ,
.
Supported modules can be seen under -s
when using the --help
option. Supported types are: title
, platform
, description
, publisher
, developer
, ages
, tags
, rating
, releasedate
, cover
, screenshot
, wheel
, marquee
, video
.
Possible dangerous command
+Purging anything from the cache cannot be undone, so please consider making a backup.
+Example(s)
+Skyscraper -p snes --cache purge:all
+Skyscraper -p snes --cache purge:m=thegamesdb
+Skyscraper -p snes --cache purge:t=cover
+Skyscraper -p snes --cache purge:m=thegamesdb,t=cover
+
Same as --refresh.
+Example(s)
+ +Will create report(s) containing all filenames of games missing the selected resource type(s). File(s) will be exported to /home/<USER>/.skyscraper/reports/report-<PLATFORM>-missing_<RESOURCE>-yyyymmdd.txt
You can use any of the following:
+all
: Creates reports for all resource typestextual
: Creates reports for all textual resource typesartwork
: Creates reports for all artwork related resource types excluding 'video'media
: Creates reports for all media resource types including 'video'type1
,type2
,type3
,...:Supported resource types are: title
, platform
, description
, publisher
, developer
, ages
, tags
, rating
, releasedate
, cover
, screenshot
, wheel
, marquee
, video
.
Tip
+The reports can be fed back into Skyscraper using the --includefrom <REPORTFILE>
option, which tells Skyscraper to only work on the files contained in the report. This is useful in combination with, for instance, the --cache edit
option or the --cache refresh
/--refresh
(they are the same) option(s).
Example(s)
+Skyscraper -p snes --cache report:missing=textual
+Skyscraper -p snes --cache report:missing=artwork
+Skyscraper -p snes --cache report:missing=developer
+Skyscraper -p snes --cache report:missing=publisher,screenshot
+
Shows the cache stats for the chosen platform. It will list how many resources of each type you currently have cached for each scraping module.
+Example(s)
+ +You can purge all resources that don't have any connection to your current romset for the selected platform by using the vacuum
command. This is extremely useful if you've removed a bunch of roms from your collection and you wish to purge any cached data you don't need anymore.
Possible dangerous command
+Vacuuming the cache cannot be undone, so please consider making a backup.
+Example(s)
+ +This will test the integrity of the resource cache connected to the chosen platform. It will remove / clean out any stray files that aren't connected to an entry in the cache and vice versa. It's not really necessary to use this option unless you have manually deleted any of the cached files or entries in the db.xml
file connected to the platform.
Note
+This option doesn't clean up your game list media folders. You will need to do that yourself since Skyscraper has no idea what files you might keep in those folders. This option only relates to the resource cache database and related files.
+Example(s)
+ +If you wish to work on a subset of your roms you can use this option to set the rom to end at. Use it in conjunction with the --startat
option described above to further narrow the subset of files.
Note
+Enabling this option automatically sets the --refresh
option and enables the nosubdirs
flag.
Tip
+Instead of using this option, if you just want to gather resources for one or two roms, you can provide the filename(s) directly on the command like so: $ Skyscraper -p snes -s thegamesdb "/full/or/partial/path/to/rom.zip"
. You can also use the --includefrom
option.
Example(s)
+Skyscraper -p snes --cache edit --endat "rom name.zip"
+Skyscraper -p snes -s thegamesdb --endat "partial/path/to/rom name.zip"
+
Tells Skyscraper to exclude the files listed in FILENAME. One filename per line (with FULL path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip').
+This file can be generated with the --cache report:missing
option or made manually.
Tip
+You might also want to check out the excludepattern
option.
Example(s)
+ +Per platform Skyscraper have default file extensions that it will accept. This option allows you to exclude certain files within that scope. The pattern is a simple asterisk type pattern. You can add several patterns by separating them with ','. In cases where you need to match for a comma you need to escape it as '\,' (see last example).
+Info
+Remember to double-quote the pattern as seen in the examples to avoid odd behaviour.
+Tip
+--excludefrom
option..skyscraperignore
within any subfolder of the input dir, all files from that directory will be ignored by Skyscraper.Example(s)
+Skyscraper -p snes -s screenscraper --excludepattern "*[BIOS]*"
+Skyscraper -p amiga --excludepattern "AGA*"
+Skyscraper -p amiga --excludepattern "*AGA*,Super*"
+Skyscraper -p amiga --excludepattern "*AGA*,Super*,*\, The"
+
From Skyscraper 3.5.0 all command-line options that change the scraping behaviour have been combined into this option. Check below for a complete list of all the available flags and what they do. You can also get this list by using --flags help
.
To enable multiple flags separate them by commas (eg. --flags FLAG1,FLAG2
) or apply --flags
option multiple times.
This flag forces Skyscraper to use the filename (excluding extension) instead of the cached titles when generating a game list. Consider setting this in config.ini
instead.
Note
+If nameTemplate
is set in config.ini
this flag is ignored.
When gathering data from any of the scraping modules many potential entries will be returned. Normally Skyscraper chooses the best entry for you. But should you wish to choose the best entry yourself, you can enable this flag. Skyscraper will then list the returned entries and let you choose which one is the best one.
+By default Skyscraper doesn't scrape and cache game manuals resources because not all scraping sites provide this data and also only some frontends support PDF display of these game manuals. You can enable it by using this flag. Consider setting this in config.ini
instead.
Use this flag to disable any bracket notes when generating the game list. It will disable notes such as (Europe)
and [AGA]
completely. This flag is only relevant when generating the game list. It makes no difference when gathering data into the resource cache. Consider setting this in config.ini
instead.
Note
+If nameTemplate
is set in config.ini
this flag is ignored.
Disables the caching of the resource type cover
when scraping with any module. If you never use covers in your artwork configuration, this flag can save you some space. Consider setting this in config.ini
instead.
Disables cropping away the black borders around screenshot resources when compositing the final frontend artwork.
+Disables the "Did you know" hints when running Skyscraper. Consider setting this in config.ini
instead.
Disables the caching of the resource type marquee
when scraping with any module. If you never use marquees in your artwork configuration, this flag can save you some space. Consider setting this in config.ini
instead.
By default, to save space, Skyscraper resizes large pieces of artwork before adding them to the resource cache. Adding this flag will disable this and save the artwork files exactly as they are retrieved from the scraping module. Consider setting this in config.ini
instead.
Note
+This is not related to the artwork compositing that happens when generating a game list. This is only related to how Skyscraper handles artwork when adding it to the resource cache while gathering data from the scraping modules.
+Disables the caching of the resource type screenshot
when scraping with any module. If you never use screenshots in your artwork configuration, this flag can save you some space. Consider setting this in config.ini
instead.
By default Skyscraper will include roms located in subfolders. By adding this flag Skyscraper will only scrape the roms located directly in the input folder. See -i <PATH>
above to read more about the rom input folder. Consider setting this in config.ini
instead.
Disables the caching of the resource type texture
when scraping with any module. If you never use textures (disc/cartridge) in your artwork configuration, this flag can save you some space. Consider setting this in config.ini
instead.
Disables clean up some misformatting in scraped description:
+…
) is replace with three dotsConsider setting this in config.ini
instead.
Disables the caching of the resource type wheel
when scraping with any module. If you never use wheels in your artwork configuration, this flag can save you some space. Consider setting this in config.ini
instead.
This flag tells Skyscraper to skip all files which already have any piece of data from any source in the cache. This is useful if you just scraped almost all files from a platform succesfully with one source, and then want to only scrape the remaining games with a different source to fill in the holes. Normally Skyscraper will scrape all files again with the second source.
+This flag is only relevant when generating a game list (by leaving out the -s <MODULE>
option). It disables the game list generator and artwork compositor and only outputs the results of the potential game list generation to the terminal. It can be very useful to check exactly what and how the data will be combined from the resource cache.
Only relevant when generating an EmulationStation, a Retrobat or a Pegasus game list, with the -f
option. Emulationstation is the default frontend when the -f
option is left out. This forces the rom and any media paths (if they are the same as the input folder) inside the game list to be relative to the rom input folder. Consider setting this in config.ini
instead.
When generating gamelists, skip processing covers that already exist in the media output folder.
+When generating gamelists, skip copying manuals that already exist in the media output folder.
+When generating gamelists, skip processing marquees that already exist in the media output folder.
+When generating gamelists, skip processing screenshots that already exist in the media output folder.
+When generating gamelists, skip copying videos that already exist in the media output folder.
+When generating gamelists, skip processing wheels that already exist in the media output folder.
+If a rom has no resources attached to it in the cache, it will be left out when generating a game list file. It will still show up in the frontend (at least it does for EmulationStation) but it won't exist in the game list file. You can safely leave out this flag unless you need the empty entries for some reason. Consider setting this in config.ini
instead.
Enabling this flag is currently only relevant while also using the videos
flag. It basically means that Skyscraper will create a link to the cached videos instead of copying them when generating the game list media files. This will save a lot of space, but has the caveat that if you somehow remove the videos from the cache, the links will be broken and the videos then won't show anymore. Consider setting this in config.ini
instead.
Game titles are returned from the scraping sources sometimes as 'The Game' and other times as 'Game, The'. Enabling this flag will force Skyscraper to always try and move 'The' to the front of the titles. If it is not enabled, Skyscraper will always try and move it to the end of the title, regardless of how it was originally returned by the scraping sources.
+Note
+When generating gamelists Skyscraper will still sort the games as if the game titles didn't have 'The' at the beginning.
+When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By using this flag Skyscraper will always overwrite an existing game list and never skip existing entries. This is useful when scripting Skyscraper to avoid the need for user input. Consider setting this in config.ini
instead.
When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By using this flag Skyscraper will always overwrite an existing game list and always skip existing entries. This is useful when scripting Skyscraper to avoid the need for user input. Consider setting this in config.ini
instead.
Some scraping modules use file checksums to identify the game in their databases. If you've compressed your roms to zip or 7z files yourself, this can pose a problem in getting a good result. You can then try to use this flag. Doing so will extract the rom and do the file checksum on the rom itself instead of the compressed file.
+Info
+Only use this flag if you are having problems getting the roms identified from the compressed files. It slows down the scraping process significantly and should therefore be avoided if possible.
+By default Skyscraper doesn't scrape and cache video resources because of the significant disk space required to save them. You can enable videos using this flag. Consider setting this in config.ini
instead.
Example(s)
+Skyscraper -p amiga --flags forcefilename,nosubdirs,skipexistingwheels
+Skyscraper -p nes --flags videos,nomarquees
+
Tells Skyscraper to only include the files listed in FILENAME. One filename per line (with absolute path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip').
+The option is the equivalent to adding a bunch of filenames to work on directly on the commandline. It reads one line at a time from <FILENAME>
and adds them to the queue of files to work on. This is very useful in combination with the --cache edit
option or if you want to refresh data for just those files using -s <SCRAPING MODULE>
.
This file can be generated with the '--cache report:missing' option or made manually.
+Tip
+You might also want to check out the includepattern
option.
Example(s)
+Skyscraper -p snes -s screenscraper --includefrom "/home/pi/.skyscraper/includes.txt"
+# After running --cache report:missing
+Skyscraper -p snes --cache edit --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_developer-20190708.txt"
+Skyscraper -p snes -s screenscraper --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_developer-20190708.txt"
+
Per platform Skyscraper have default file extensions that it will accept. This option allows you to only include certain files within that scope. The pattern is a simple asterisk type pattern. You can add several patterns by separating them with ','. In cases where you need to match for a comma you need to escape it as '\,' (see last example).
+Info
+Remember to double-quote the pattern as seen in the examples to avoid odd behaviour.
+Tip
+You might also want to check out the file extension options.
+Example(s)
+Skyscraper -p snes -s screenscraper --includepattern "Super*"
+Skyscraper -p amiga --includepattern "*AGA*"
+Skyscraper -p amiga --includepattern "*AGA*,Super*"
+Skyscraper -p amiga --includepattern "*AGA*,Super*,*\, The"
+
Adds the specified language to the top of the existing default internal language priority list. Read more about languages here. Only one language is supported with this configuration. For a permanent setup you should consider setting this in config.ini
instead.
Example(s)
+ +Not all scraping modules support all platforms. This means that you can potentially start a scraping run with a module and a platform that is incompatible. This will hammer the servers for potentially hundreds of roms but provide 0 results for any of them. To avoid this Skyscraper has a builtin limit for initially allowed failed rom lookups. If this is reached it will quit. Setting this option allows you to set this limit yourself, but not above a maximum of 200. The default limit is 42. Don't change this unless you have a very good reason to do so.
+Example(s)
+ +For most modules a search query is sent to the scraping module in an URL format. This means that a filename such as "Rick Dangerous.lha" becomes "rick+dangerous". The '+' here means a space. You could probably also use the URL encoded space "rick%20dangerous" but my tests show that most modules expect spaces as '+'. And it is the "rick+dangerous" part that you, as the user, can pass as the query, like so:
+ +Remember to also add a filename that you wish to use the override with. Otherwise the query will be ignored.
+But not all of the scraping modules are search name based. For instance, the screenscraper
module can use a variety of different search methods. So for screenscraper you also have the option of overriding the checksums it uses to search for a game. This is especially convenient in cases where a filename exists multiple times in their database and your own local file doesn't match with any of the connected checksums (maybe you've compressed the rom yourself). In this case you can look up one of the working checksums on the Screenscraper website (screenscraper.fr) and override the checksum.
You can use any combination of crc=<CHECKSUM>
, md5=<CHECKSUM>
, sha1=<CHECKSUM>
and romnom=<FILENAME>
(without the <
and >
! Also "romnom" is "rom name" in French - Screenscraper is operated from France). Most times you only need one of these, but you can combine them by separating them with a &
.
The Mobygames scraper supports the romname directly in the --query
parameter, but also accepts the game id from the mobygames site which you can find when manually looking up a game in the "Identifiers" section of the game details page.
The --query
option is an experts only option, but it's very useful to get results for those last difficult roms.
Example(s)
+$ Skyscraper -p snes -s thegamesdb --query "rick+dangerous" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p snes -s screenscraper --query "md5=<CHECKSUM>" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p snes -s screenscraper --query "romnom=file%20name.zip" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p snes -s screenscraper --query "sha1=<CHECKSUM>&romnom=file%20name.zip" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p ports -s mobygames --query "14576" "/home/pi/RetroPie/roms/ports/Head over Heels.sh"
+
Skyscraper has a resource cache which works just like the browser cache in Firefox. If you scrape and gather resources for a platform with the same scraping module twice, it will grab the data from the cache instead of hammering the online servers again. This has the advantage in the case where you scrape a rom set twice, only the roms that weren't recognized the first time around will be fetched from the online servers. Everything else will be loaded from the cache.
+You can force all data to be refetched from the servers by setting this option, effectively updating the cached data with new data from the source.
+Note
+Only use this option if you know data has changed for several roms at the source. Otherwise you are hammering the servers for no reason.
+Example(s)
+ +Adds the specified region to the top of the existing default internal region priority list. Only one region is supported with this configuration. For a permanent setup you should consider setting this in config.ini
instead.
Read more about how regions are handled in general here.
+Info
+Setting this will overwrite any region auto-detected from the file name.
+Example(s)
+ +If you wish to work on a subset of your roms you can use this option to set the starting rom. Use it in conjunction with the --endat
option described below to further narrow the subset of files.
Note
+Enabling this option automatically sets the --refresh
option and enables the nosubdirs
flag.
Tip
+Instead of using this option, if you just want to gather resources for one or two roms, you can provide the filename(s) directly on the command like so: $ Skyscraper -p snes -s thegamesdb "/full/or/partial/path/to/rom.zip"
. You can also use the --includefrom
option.
Example(s)
+Skyscraper -p snes --cache edit --startat "rom name.zip"
+Skyscraper -p snes -s thegamesdb --startat "partial/path/to/rom name.zip"
+
Sets how verbose Skyscraper should be when running. Default level is 0. The higher the value, the more info Skyscraper will output to the terminal while running. Consider setting this in config.ini
instead.
Example(s)
+ + + + + + + + + + + + + + +Look below to find a thorough description of all options and sections available in the /home/<USER>/.skyscraper/config.ini
configuration file. These options will then be applied whenever running Skyscraper with the Skyscraper
command.
Please take note that almost all of these options are set at a useful default as documented here (and can therefore be left out) and should only be set if your use case requires it.
+Warning
+Some users seem to have a habbit of editing the /home/<USER>/skysource/config.ini
file. This is not the correct file! Please read the first paragraph. :)
Options can be set on four levels; either [main]
, [<PLATFORM>]
, [<FRONTEND>]
or [<SCRAPER>]
.
Settings in the [main]
section will always be set regardless of selected platform, frontend and module.
<PLATFORM>
can be any of the supported platforms (check list with --help
under the -p
option), in which case the settings will only be applied while scraping that particular platform.
<FRONTEND>
can be any of the supported frontends (check list with --help
under the -f
option), in which case the settings will only be applied while scraping for that particular frontend.
<SCRAPER>
can be any of the supported scraping modules (check list with --help
under the -s
option), in which case the settings will only be applied while scraping with that particular scraping module.
Each section can have overlapping parameters. In case where a certain option exists in several sections they are prioritized as scraping module first, then frontend, then platform and lastly main. Any commandline option which relates to an configuration setting in config.ini
has highest precedence, regardless of the other four levels respective sections.
You can find an example config file at /home/<USER>/.skyscraper/config.ini.example
. This file contains all available options. Just copy the file to config.ini
and uncomment and edit the ones you wish to use by removing the #
or ;
in front of the variables. Remember to also uncomment the section the option relates to such as [main]
or [amiga]
.
Note
+You can set a custom configuration file with the -c <FILENAME>
command line option. Read more about all available command line options here.
Example config.ini
:
[main]
+inputFolder="/path/to/your/roms"
+cacheFolder="/path/to/your/desired/db/cache/base/folder"
+gameListBackup="true"
+videos="true"
+unattend="true"
+verbosity="1"
+lang="ja"
+region="jp"
+cacheMarquees="false"
+
+[amiga]
+inputFolder="/specific/path/just/for/amiga/roms"
+cacheFolder="/specific/path/just/for/amiga/cache"
+minMatch="50"
+maxLength="200"
+
+[pegasus]
+artworkXml="artwork-pegasus.xml"
+maxLength="10000"
+
+[screenscraper]
+userCreds="USER:PASSWORD"
+
This is an alphabetical index of all configuration options including the sections they may be applied to.
+Parameter | +[main] |
+[<PLATFORM>] |
+[<FRONTEND>] |
+[<SCRAPER>] |
+
---|---|---|---|---|
addExtensions | +Y | +Y | ++ | + |
addFolders | ++ | + | Y | ++ |
artworkXml | +Y | +Y | +Y | ++ |
brackets | +Y | +Y | +Y | ++ |
cacheCovers | +Y | +Y | ++ | Y | +
cacheFolder | +Y | +Y | ++ | + |
cacheMarquees | +Y | +Y | ++ | Y | +
cacheRefresh | +Y | ++ | + | Y | +
cacheResize | +Y | +Y | ++ | Y | +
cacheScreenshots | +Y | +Y | ++ | Y | +
cacheTextures | +Y | +Y | ++ | Y | +
cacheWheels | +Y | +Y | ++ | Y | +
cropBlack | +Y | +Y | +Y | ++ |
emulator | +Y | +Y | +Y | ++ |
endAt | ++ | Y | +Y | ++ |
excludeFrom | +Y | +Y | ++ | + |
excludePattern | +Y | +Y | +Y | ++ |
extensions | ++ | Y | ++ | + |
forceFilename | +Y | +Y | +Y | ++ |
frontend | +Y | ++ | + | + |
gameListBackup | +Y | ++ | Y | ++ |
gameListFolder | +Y | +Y | +Y | ++ |
gameListVariants | ++ | + | Y | ++ |
hints | +Y | ++ | + | + |
importFolder | +Y | +Y | ++ | + |
includeFrom | +Y | +Y | ++ | + |
includePattern | +Y | +Y | +Y | ++ |
innerBracketsReplace | +Y | ++ | + | + |
innerParenthesesReplace | +Y | ++ | + | + |
inputFolder | +Y | +Y | ++ | + |
interactive | +Y | +Y | ++ | Y | +
jpgQuality | +Y | +Y | ++ | Y | +
lang | +Y | +Y | ++ | + |
langPrios | +Y | +Y | ++ | + |
launch | +Y | +Y | +Y | ++ |
manuals | +Y | +Y | ++ | + |
maxFails | +Y | ++ | + | + |
maxLength | +Y | +Y | +Y | +Y | +
mediaFolder | +Y | +Y | +Y | ++ |
mediaFolderHidden | ++ | + | Y | ++ |
minMatch | +Y | +Y | ++ | Y | +
nameTemplate | +Y | +Y | ++ | + |
onlyMissing | +Y | +Y | ++ | Y | +
platform | +Y | ++ | + | + |
pretend | +Y | +Y | ++ | + |
region | +Y | +Y | ++ | + |
regionPrios | +Y | +Y | ++ | + |
relativePaths | +Y | +Y | ++ | + |
scummIni | +Y | ++ | + | + |
skipped | +Y | +Y | +Y | ++ |
spaceCheck | +Y | ++ | + | + |
startAt | ++ | Y | +Y | ++ |
subdirs | +Y | +Y | ++ | + |
symlink | +Y | +Y | +Y | ++ |
theInFront | +Y | +Y | +Y | ++ |
threads | +Y | +Y | ++ | Y | +
tidyDesc | +Y | +Y | ++ | Y | +
unattend | +Y | +Y | +Y | +Y | +
unattendSkip | +Y | +Y | +Y | +Y | +
unpack | +Y | +Y | ++ | + |
userCreds | ++ | + | + | Y | +
verbosity | +Y | +Y | +Y | ++ |
videoConvertCommand | +Y | ++ | + | Y | +
videoConvertExtension | +Y | ++ | + | Y | +
videoPreferNormalized | ++ | + | + | Y | +
videos | +Y | +Y | +Y | +Y | +
videoSizeLimit | +Y | +Y | ++ | Y | +
Sets the rom input folder. By default Skyscraper will look for roms in the /home/<USER>/RetroPie/roms/<PLATFORM>
folder. If your roms are located in a non-default location, you can set the input path using this option.
Note
+If this is set in the [main]
section it will automatically add /<PLATFORM>
to the end of the path. If you want better control consider adding it to a [<PLATFORM>]
section instead where it will be used as is.
Default value: /home/<USER>/RetroPie/roms/<PLATFORM>
+Allowed in sections: [main]
, [<PLATFORM>]
Sets the game list export folder. By default Skyscraper exports the game list to the same directory as the rom input folder. This enables you to change that to a non-default location.
+Note
+If this is set in the [main]
or [<FRONTEND>]
section it will automatically add /<PLATFORM>
to the end of the path. If you want better control consider adding it to a [<PLATFORM>]
section instead where it will be used as is.
Default value: /home/<USER>/RetroPie/roms/<PLATFORM>
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
When set to true Skyscraper will create a backup of the existing game list each time it is run in game list generation mode (this mode is enabled by leaving out the -s MODULE
option). The filename and path of the backup will be equal to the existing game list but have a timestamp appended to the name of the format -yyyyMMdd-hhmmss
(eg. gamelist.xml-20200530-115900
).
Default value: false
+Allowed in sections: [main]
, [<FRONTEND>]
Sets the artwork / media output folder. By default (ie. if the option mediaFolder
is not explicitly set or pre-defined by the frontend) Skyscraper outputs the composited artwork files to the game list export folder concatenated with /media
. The mediaFolder
setting enables you to change that to a non-default location.
Read more about the artwork compositing here.
+Note
+If this is set in the [main]
or [<FRONTEND>]
section it will automatically add /<PLATFORM>
to the end of the path. If you want better control consider adding it to a [<PLAFORM>]
section instead where it will be used as is.
Default value: /home/<USER>/RetroPie/roms/<PLATFORM>/media
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
By default Skyscraper uses the <PLATFORM>/media
folder when generating EmulationStation media. Setting this option will change that to <PLATFORM>/.media
. Notice the .
which hides the folder, which can speed up the initial EmulationStation loading sequence when using slow storage such as network file systems.
This option is ignored if you set the media folder manually.
+Note
+Remember to remove your old <PLATFORM>/media
folders if you enable this option.
Default value: false
+Allowed in sections: Only for frontends [emulationstation]
or [retrobat]
Sets a non-default location for the storing and loading of cached game resources. This is what is referred to in the docs as the resource cache. By default this folder is set to /home/<USER>/.skyscraper/cache/<PLATFORM>
. Don't change this unless you have a good reason to (for instance if you want your cache to reside on a USB drive).
Note
+If this is set in the [main]
section (recommended) it will automatically add /<PLATFORM>
to the end of the path. If you want better control consider adding it to a [<PLAFORM>]
section instead where it will be used as is.
Default value: /home/<USER>/.skyscraper/cache/<PLATFORM>
+Allowed in sections: [main]
, [<PLATFORM>]
By default, to save space, Skyscraper resizes large pieces of artwork before adding them to the resource cache. Setting this option to "false"
will disable this and save the artwork files with their original resolution. Beware that Skyscraper converts all artwork resources to lossless PNG's when saving them. High resolution images such as covers will take up a lot of space! So set this to "false"
with caution.
Note
+This is not related to the artwork compositing that happens when generating a game list. This is only related to how Skyscraper handles artwork when adding it to the resource cache while gathering data from the scraping modules.
+To read about artwork compositing go here instead.
+Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Skyscraper has a resource cache which works just like the browser cache in Firefox. If you scrape and gather resources for a platform with the same scraping module twice, it will grab the data from the cache instead of hammering the online servers again. This has the advantage in the case where you scrape a rom set twice, only the roms that weren't recognized the first time around will be fetched from the online servers. Everything else will be loaded from the cache.
+You can force all data to be refetched from the servers by setting this option to cacheRefresh="true"
, effectively updating the cached data with new data from the source.
Warning
+Only set this option to true if you know data has changed for several roms at the source. Otherwise you are hammering the servers for no reason.
+Default value: false
+Allowed in sections: [main]
, [<SCRAPER>]
By default Skyscraper uses just the title as the game name when generating gamelists for any frontend. You can change this to suit your personal preference by setting this option. It works by replacing certain tokens in the name template with the corresponding data. The valid tokens are:
+%t
: The game title as returned by the scraping sources without bracket information (see %b
and %B
below)%f
: The game filename without extension and bracket information (see %b
and %B
below)%b
: The game ()
parentheses information. This information often comes from the filename, but can also come from the scraping source title (eg. (USA)
or (en,fr,de)
)%B
: The game []
bracket information. This information often comes from the filename, but can also come from the scraping source title (eg. [disk 1 of 2]
or [AGA]
)%a
: The age restriction as returned by the scraping sources (eg. 16+
)%d
: The game developer as returned by the scraping sources%p
: The game publisher as returned by the scraping sources%r
: The game rating as returned by the scraping sources (0.0
to 5.0
)%P
: The number of players as returned by the scraping sources%D
: The game release date with format yyyy-mm-dd
This option also support template groups separated by ;
within the template. The template parser will go over each group. If a group only has empty variables it will not be included in the final game name. So for a template such as %t;, %P player(s)
where the %P
is empty because no scraping source has provided the info, it will leave out the , %P player(s)
part entirely resulting in the title Game Name
. If this was not separated by ;
the resulting game name would end up being Game Name, player(s)
.
Example(s)
+ +Will result in: 1945k III [1945kiii], 2 player(s)
Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
Cleans up some misformatting in scraped description:
+…
) is replaced with three dot charactersQuote
+'Multiple exclamation marks,' he went on, shaking his head, 'are a sure sign of a diseased mind.'
+-- Eric by Terry Pratchett
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Sets the default jpg quality when saving image resources to the cache. This will be ignored if --flags noresize
is set. Default is 95.
All screenshots and any image resource that uses transparency will always be saved as PNG images. In those cases this setting will be ignored as PNG format is lossless.
+Default value: 95
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Enables/disables the caching of the resource type cover
when scraping with any module. If you never use covers in your artwork configuration, setting this to "false"
can save you some space.
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Enables/disables the caching of the resource type cover
when scraping with any module. If you never use covers in your artwork configuration, setting this to "false"
can save you some space.
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Enables/disables the caching of the resource type screenshot
when scraping with any module. If you never use screenshots in your artwork configuration, setting this to "false"
can save you some space.
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Enables/disables cropping away the black borders around the screenshot resources when compositing the final frontend gamelist artwork.
+Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Enables/disables the caching of the resource type wheel
when scraping with any module. If you never use wheels in your artwork configuration, setting this to "false"
can save you some space.
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Enables/disables the caching of the resource type marquee
when scraping with any module. If you never use marquees in your artwork configuration, setting this to "false"
can save you some space.
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Sets a non-default folder when scraping using the -s import
module. By default this is set to /home/<USER>/.skyscraper/import
and will also look for a /<PLATFORM>
inside of the chosen folder. Read more about the -s import
module here.
Default value: /home/<USER>/.skyscraper/import/<PLATFORM>
+Allowed in sections: [main]
, [<PLATFORM>]
Some scraping modules use file checksums to identify the game in their databases. If you've compressed your roms to zip or 7z files yourself, this can pose a problem in getting a good result. You can then try setting this option to "true"
. Doing so will extract the rom and do the file checksum on the rom itself instead of the compressed file.
Info
+Only enable this option if you are having problems getting the roms identified from the compressed files. It slows down the scraping process significantly and should therefore be avoided if possible.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
Sets the frontend you wish to export a game list for. By default Skyscraper will export an EmulationStation game list, but other frontends are supported as well.
+If exporting for the attractmode
frontend, please also take note of the required emulator=""
option that goes along with using the attractmode
frontend.
If exporting for the pegasus
frontend, please also take note of the optional launch=""
option that allows you to set the launch parameter used by Pegasus.
Use as value for the frontend option the frontend name in all lowercase and with alphabetical characters only: emulationstation
, esde
, pegasus
, retrobat
, attractmode
, aso. Check all supported frontends with '--help' and read a more about the details concerning each of them here.
Default value: emulationstation
+Allowed in sections: [main]
This option is only applicable when also setting the frontend="attractmode"
option. It sets the emulator to be used when generating the attractmode
game list. On RetroPie the emulator name is mostly the same as the platform.
Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
This option is only applicable when also setting the frontend="pegasus"
option. It sets the launch command to be used when generating the pegasus
game list. This is optional. It defaults to the RetroPie launch options which works on RetroPie.
Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
By default Skyscraper doesn't scrape and cache video resources because of the significant disk space required to save them. You can enable videos using this option.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
, [<SCRAPER>]
If video scraping is enabled you can set the maximum allowed video file size with this variable. The size is in Megabytes (1.000.000 bytes). If this size is exceeded the video file won't be saved to the cache.
+Default value: 100
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Some scraping modules deliver videos that use a codec or color format that some frontends don't support. In those cases it can be useful to convert the videos before saving them in the Skyscraper resource cache.
+This setting allows you to set a command that will be run on each video after it has been downloaded from the selected scraping module. See the examples below for inspiration.
+The %i
and %o
are required and will be replaced with the internally used video input (original) and output (converted) filenames as needed by Skyscraper.
If your command / script always converts to a videofile with a specific extension, you also need to set videoConvertExtension
.
Tip
+Set --verbosity 3
to route all output from your command / script to the terminal while Skyscraper runs. This will help you ensure everything is working as intended.
The first example below makes use of the excellent ffmpeg
tool. If you want to use this specific example you need to install ffmpeg
first. On RetroPie and other Debian-derived distros you can install it with sudo apt install ffmpeg
. If you want to use a script for the video conversion and run it directly without path, you need to place it in the /home/<USER>/.skyscraper
folder.
Tip
+If you create a script that includes checks on the input video and it sometimes decides not to convert them, you should simply let the script copy the video with cp %i %o
. The %i
is a temporary file created internally by Skyscraper. And the %o
file is the file used by the cache. Both files should exist after a successful conversion. Skyscraper removes the temporary %i
file automatically.
Example(s)
+videoConvertCommand="ffmpeg -i %i -y -pix_fmt yuv420p -t 00:00:10 -c:v libx264 -crf 23 -c:a aac -b:a 64k -vf scale=640:480:force_original_aspect_ratio=decrease,pad=640:480:(ow-iw)/2:(oh-ih)/2,setsar=1 %o"
+videoConvertCommand="videoconvert.sh %i %o"
+
Default value: unset
+Allowed in sections: [main]
, [<SCRAPER>]
If you want to force an extension for the converted video file created by the videoConvertCommand
command, you need to set this option. The converted file will then automatically have this extension no matter what the input file extension is.
For instance, if a scraping module delivers the file videofile.avi
and you always want the converted files to be mp4
files, you simply set this option to videoConvertExtension="mp4"
.
It is up to you to make sure that the command you provide in videoConvertCommand
actually converts to a video file using the selected extension. For some conversion tools such as ffmpeg
this is handled simply by setting the extension of the output file. But for other tools you might have to provide additional options to the videoConvertCommand
above.
Default value: unset
+Allowed in sections: [main]
, [<SCRAPER>]
This option is only applicable when scraping with the -s screenscraper
module. ScreenScraper offers two versions of some of their videos. A normalized version, which adheres to some defined standard they made, and the originals. If you prefer converting or standardizing the videos yourself (see videoConvertCommand
above) then you can set this to false
. If you do so Skyscraper will fetch the original videos from ScreenScraper instead of the normalized ones.
Be aware that the original videos often vary a lot in codec, color format and size. So it is recommended to convert them afterwards using the videoConvertCommand
.
Example(s)
+ +Default value: true
+Allowed in sections: Only for scraper [screenscraper]
Enabling this option is only relevant while also setting the videos="true"
option. It basically means that Skyscraper will create a link to the cached videos instead of copying them when generating the game list media files. This will save a lot of space, but has the caveat that if you somehow remove the videos from the cache, the links will be broken and the videos then won't show anymore.
Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Game titles are returned from the scraping sources sometimes as 'The Game' and other times as 'Game, The'. Enabling this option will force Skyscraper to always try and move 'The' to the front of the titles. If it is not enabled, Skyscraper will always try and move it to the end of the title, regardless of how it was originally returned by the scraping sources.
+Info
+When generating gamelists Skyscraper will still sort the games as if the game titles didn't have 'The' at the beginning.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Disable this option to remove any bracket notes when generating the game list. It will disable notes such as (Europe)
and [AGA]
completely. This option is only relevant when generating the game list. It makes no difference when gathering data into the resource cache. Default (true) will keep brackets in the game title.
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Only in use when the option brackets
is set to true
for gamelist creation:
+This replaces consecutive brackets ][
in the game title with whatever is
+defined in this option. This setting has no effect, if there is only one bracket
+present in the game filename.
+Use the option innerParenthesesReplace
for the same effect on round brackets
+)(
(aka. parentheses).
Example(s)
+Filename: Oddworld - Abe's Exoddus [NTSC-U] [SLUS-00710].m3u
innerBracketsReplace=""
(unset), gamelist game title output: Oddworld - Abe's Exoddus [NTSC-U][SLUS-00710]
innerBracketsReplace="] ["
, gamelist game title output (note the space.): Oddworld - Abe's Exoddus [NTSC-U] [SLUS-00710]
innerBracketsReplace=","
, gamelist game title output: Oddworld - Abe's Exoddus [NTSC-U,SLUS-00710]
Default value: unset
+Allowed in sections: [main]
Same as innerBracketsReplace but for parentheses )(
+(aka. round brackets).
Sets the maximum length of returned game descriptions. This is a convenience option if you feel like game descriptions are too long. By default it is set to 2500.
+Default value: 2500
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
, [<SCRAPER>]
Some scraping modules are based on a file name or title based search. This option sets the minimum percentage any returned results need to match with in order for it to be accepted. For instance, the game Wonderboy in Monsterland
might return the title Wonder Boy in Monster Land
which is clearly a match. But it's not a 100% match. So it needs to be set relatively high, while still ignoring bad matches. By default it is set to 65 which has been tested to be a good middle-ground.
Default value: 65
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Sets the desired number of parallel threads to be run when scraping. Some modules have maximum allowed threads. If you set this higher than the allowed value, it will be auto-adjusted. By default it is set to 4.
+Default value: 4
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
This option is only relevant when generating a game list (by leaving out the -s <SCRAPER>
command line option). It disables the game list generator and artwork compositor and only outputs the results of the potential game list generation to the terminal. It is mostly useful when used as a command line flag with --flags pretend
. It makes little sense to set it here, but you can if you want to.
Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By enabling this option Skyscraper will always overwrite an existing game list and never skip existing entries, in other words: the game list will be newly created. This is flag useful for example when scripting Skyscraper to avoid the need for user input.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
, [<SCRAPER>]
When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By enabling this option Skyscraper will always overwrite an existing game list and always skip existing entries, in other words: game list entries are added if not present in the gamelist, existing entries are left untouched. This flag is useful for example when scripting Skyscraper to avoid the need for user input.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
, [<SCRAPER>]
When gathering data from any of the scraping modules many potential entries will be returned. Normally Skyscraper chooses the best entry for you. But should you wish to choose the best entry yourself, you can enable this option. Skyscraper will then list the returned entries and let you choose which one is the best one. It is recommended to use the command line flag --flags interactive
instead in the (hopefully) rare cases where this mode is necessary.
Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
Enable this option to force Skyscraper to use the file name (excluding extension) instead of the cached scraping module titles when generating a game list.
+Tip
+If you set forceFilename="true"
and your filenames contain bracket notes such as (this)
or [that]
at the end, these will be combined with whatever bracket notes are at the end of the titles returned from the sources. This can cause some confusion. For instance, if you have the filename Gran Turismo 2 (USA) (Arcade Mode)
and the cached title is Gran Turismo 2 (Arcade Mode)
, then the gamelist name will become Gran Turismo 2 (Arcade Mode)(USA)(Arcade Mode)
. You can disable them altogether with the brackets="no"
option.
Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Sets how verbose Skyscraper should be when running. Default level is 0. The higher the value, the more info Skyscraper will output to the terminal while running. Anything higher than 3 does not further increase the verbosity of the output.
+Default value: 0
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Only has an effect if a rom has no resources attached to it in the cache. If false (default) the rom will be left out when generating a game list file. However, it will still show up in the frontend (at least it does for EmulationStation), but it won't exist in the game list file. If unsure leave it to false, unless you need generic gamelist entries, consisting of <path/>
and <name/>
, for some reason. In other words: If skipped is set to false and no entry for a rom is found in the cache nothing will be added to the gamelist for that rom. If skipped is true and no entry for a rom is found in the cache a generic entry will be added to the gamelist file for that rom. If a rom has data in Skyscraper's cache then this flag has no effect.
Note
+When skipped is set true, any generic added game entry is not eligible for formatting with a nameTemplate.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Not all scraping modules support all platforms. This means that you can potentially start a scraping run with a module and a platform that is incompatible. This will hammer the servers for potentially hundreds of roms but provide 0 results for any of them. To avoid this Skyscraper has a builtin limit for initially allowed failed rom lookups. If this is reached it will quit. Setting this option allows you to set this limit yourself, but not above a maximum of 200. The default limit is 42. Don't change this unless you have a very good reason to do so.
+Default value: 42
+Allowed in sections: [main]
Adds the specified region to the top of the existing default internal region priority list. Only one region is supported with this configuration. To overwrite the internal default region priority list entirely check here. Read more about how regions are handled in general here.
+Info
+Setting the region will overwrite any region auto-detected from the file name.
+Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
Example(s)
+ +or
+ +Completely overwrites the internal region priority list inside of Skyscraper. Multiple regions can be configured here separated by commas. Read more about how regions are handled in general here.
+Info
+Any region auto-detected from the file name will still be added to the top of this list.
+Default value: eu, us, ss, uk, wor, jp, au, ame, de, cus, cn, kr, asi, br, sp, fr, gr, it, no, dk, nz, nl, pl, ru, se, tw, ca
+Allowed in sections: [main]
, [<PLATFORM>]
Example(s)
+ +or
+ +Adds the specified language to the top of the existing default internal language priority list. Only one language is supported with this configuration. Read more about how languages are handles in general here.
+Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
Example(s)
+ +or
+ +Completely overwrites the internal language priority list inside of Skyscraper. Multiple languages can be configured here separated by commas. Read more about how languages are handles in general here.
+Default value: en, de, fr, es
+Allowed in sections: [main]
, [<PLATFORM>]
Example(s)
+ +or
+ +Sets a non-default xml file to use when setting up the artwork compositing. By default Skyscraper uses the file /home/<USER>/.skyscraper/artwork.xml
. If you provide a relative filepath it will be expanded to /home/<USER>/.skyscraper/<artworkXml>
, respective to $XDG_CONFIG_HOME/skyscraper/<artworkXml>
, if you use Skyscraper in XDG mode.
+Read more about the artwork.xml format and customization options here.
Tip
+It can be very useful to set this in any platform section or frontend section where you want a specific artwork setup / look.
+Default value: artwork.xml
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Enabling this forces the rom and any media paths inside the game list to be relative to the rom input folder. Currently only relevant when generating an EmulationStation, a Retrobat or a Pegasus game list (see also frontend option).
+Info
+This setting has no effect when the frontend is set to attractmode.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
Completely overwrites the rom extensions Skyscraper allows for the chosen platform. Use a space between each extension (e.g., *.abc *.def
).
Info
+If you feel like you are using a file extension that ought to be supported by default, please report it so it can be added in a later version of Skyscraper.
+Default value: unset. If set it ignores "formats": [ ... ]
in peas.json
file
+Allowed in sections: [<PLATFORM>]
Adds the rom extensions to the ones that are already supported by the platform, use a space between each extension.
+Info
+If you feel like you are using a file extension that ought to be supported by default, please report it so it can be added in a later version of Skyscraper.
+Default value: *.zip *.7z *.ml
(if and only if extensions
parameter is unset)
+Allowed in sections: [main]
, [<PLATFORM>]
If set to false it disables the "Did you know" hints when running Skyscraper.
+Default value: true
+Allowed in sections: [main]
By default Skyscraper will include roms located in subfolders. By disabling this option Skyscraper will only scrape the roms located directly in the input folder. See inputFolder="<PATH>"
further up to read more about the rom input folder.
Default value: true
+Allowed in sections: [main]
, [<PLATFORM>]
This flag, when set true, will only scrape game information for ROMs that do not have any data in Skyscraper's cache. The cache information of a ROM that has a cache hit will not be updated or extended when 'only missing' is true. A use case could be that you want to exclude already scraped games from a scraping process and you are using a scraping module which has a limit on how many ROMs may be scraped: With 'only missing' enabled the ROMs with cache data are subtracted from all ROMs to be scraped for a platform, thus the remaining ROM count may be below the scrape module's limit.
+Tip
+Please consider using the command line flag option --flags onlymissing
instead. It makes little sense to set this permanently in your configuration unless you always want to use the cached data for already cached game entries. Also when 'only missing' is set and the cache has any data for a game entry, the refresh option has no effect. But you still can use the 'only missing' in the config file if you absolutely want to.
Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
, [<SCRAPER>]
If you only ever wish to gather data for a subset of your roms from the scraping modules you can use this option to set the starting rom. It will then scrape alphabetically from that rom and onwards.
+Tip
+Please consider using the command line option --startat <FILENAME>
instead. It makes little sense to set this permanently in your configuration unless you always want it to only ever scrape from a certain file and onward. But you can if you absolutely want to.
Default value: unset
+Allowed in sections: [<PLATFORM>]
, [<FRONTEND>]
If you only ever wish to gather data for a subset of your roms from the scraping modules you can use this option to set the rom to end at. It will then scrape alphabetically until it reaches this rom, then stop.
+Tip
+Please consider using the command line option --endat <FILENAME>
instead. It makes little sense to set this permanently in your configuration unless you always want it to only ever scrape up until it reaches a certain file. But you can if you absolutely want to.
Default value: unset
+Allowed in sections: [<PLATFORM>]
, [<FRONTEND>]
Per platform Skyscraper have default file extensions that it will accept. This option allows you to only include certain files within that scope. The pattern is a simple asterisk type pattern. In cases where you need to match for a comma you need to escape it as \,
.
Note
+You might also want to check out the file extension options and the the 'includeFrom' option.
+Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Per platform Skyscraper have default file extensions that it will accept. This option allows you to exclude certain files within that scope. The pattern is a simple asterisk type pattern. In cases where you need to match for a comma you need to escape it as \,
.
Note
+You might also want to check out the file extension options and the 'excludeFrom' option.
+Tip
+If you create a file named .skyscraperignore
within any subfolder of the input dir, all files from that directory will be ignored by Skyscraper. Additionally, if you put an empty file .skyscraperignoretree
in a folder, all files in that directory and all subdirectories are ignored.
Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
, [<FRONTEND>]
Tells Skyscraper to only include the files listed in FILENAME. One filename per line (with FULL path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip').
+This file can be generated with the '--cache report:missing' option or made manually.
+Note
+You might also want to check out the 'includePattern' option.
+Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
Tells Skyscraper to exclude the files listed in FILENAME. One filename per line (with FULL path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip').
+This file can be generated with the '--cache report:missing' option or made manually.
+Note
+You might also want to check out the 'excludePattern' option.
+Tip
+If you create a file named '.skyscraperignore' within any subfolder of the input dir, all files from that directory will be ignored by Skyscraper. Additionally, if you put an empty file .skyscraperignoretree
in a folder, all files in that directory and all subdirectories are ignored.
Default value: unset
+Allowed in sections: [main]
, [<PLATFORM>]
Some scraping modules require a key
or user:password
to work. You can create a [<SCRAPER>]
section and add a userCreds="user:pass"
or userCreds="key"
line beneath it. Doing this will always use these credentials when scraping from the module in question. Check the scraping module overview to see the specific requirements for each module here.
Example(s)
+ +or
+ +Default value: unset
+Allowed in sections: [<SCRAPER>]
Skyscraper will continuously check if you are running low on disk space. If you go below 200 MB in either the game list export folder or the resource cache folder, it will quit to make sure your system doesn't become unstable. Some types of file systems provide a faulty result to Skyscraper when it comes to these checks and thus it can be necessary to disable it altogether. You can use this option to do just that.
+Default value: true
+Allowed in sections: [main]
Allows you to set a non-default path of the scummvm.ini
file. This file is used whenever scraping the scummvm
platform. It converts the shortname such as monkey2
to the more search-friendly name Monkey Island 2: LeChuck's Revenge
whenever using one of the file name search based scraping modules.
Default value: If ~/.scummvmrc
has precedence over /opt/retropie/configs/scummvm/scummvm.ini
+Allowed in sections: [main]
Allows you to set a platform, which is applied when no command line switch -p
is given.
Default value: unset
+Allowed in sections: [main]
If you arrange your ROMs in directories below a platform directory, this flag
+comes into play. The gamelist
+specification
+allows you to define also metadata for directories. This metadata is held in
+<folder/>
elements in the gamelist XML file. In minimum a <folder/>
has a
+<path/>
and a <name/>
inner element. If no <name/>
is given, the last
+subdirectory of the path element is taken as name. Which additionally
+subelements are interpreted and rendered is dependent on the frontend, for
+example EmulationStation for RetroPie supports these
+elements
+to be used inside a folder element.
If this flag set to true, Skyscraper will create generic folder elements in the +gamelist file, containing path and name for each directory found. If false, no +folder elements will be created for the directories with ROM(s) (example).
+In both cases the existing <folder/>
elements of a gamelist will be preserved.
+However, folder data is not cached by Skyscraper, which means if you delete your
+gamelist.xml
Skyscraper can not restore the edited folder elements.
EmulationStation 2.11.2rp and earlier
+EmulationStation 2.11.2 (RetroPie) and earlier have a bug that adds a <folder/>
element everytime you edit and save metadata for the same folder within EmulationStation. The last edit will be the latest folder element in the gamelist file. Skyscraper in contrast expects in maximum one <folder/>
element per each path. If you regenerate a gamelist with Skyscraper, you will lose the latest edit of the folder metadata. This bug is described here and fixed in the next release of EmulationStation respective is fixed already in the EmulationStation-dev version.
Default value: false
+Allowed in sections: Only for frontends [emulationstation]
, [esde]
or [retrobat]
By default Skyscraper doesn't scrape and cache game manuals resources because not all scraping sites provide this data and also only some frontends support PDF display of these game manuals. If enabled Skyscraper will collect game manuals for the scraping modules that provide this data. For frontend ES-DE no further option must be set to enable the output of the PDF manuals to the appropriate folder. For other EmulationStation forks see also option gameListVariants.
+Default value: false
+Allowed in sections: [main]
, [<PLATFORM>]
This is a comma separated list of options for the different gamelist variants used by the various EmulationStation forks. Currently only enable-manuals
is evaluated as variant: It generates <manual/>
entries in the gamelist for the game manuals scraped or found in the cache, if also the manuals configuration option is enabled. This option is not needed for the ES-DE frontend to output game manuals.
Example(s)
+ +Default value: unset
+Allowed in sections: Only for frontend [emulationstation]
git checkout -b my-new-feature
)clang-format-15 -i src/<changedfiles>
)git rebase --keep-base -i master
)git push origin my-new-feature
)Found something missing, or an existing section confusing or outdated? Any +documentation fix is also appreciated.
+ + + + + + + + + + + + + +A quick note before I let you loose on the questions:
+Skyscraper --help
config.ini
options hereAnd now for the questions (with answers)...
+You probably forgot to generate the gamelist. Skyscraper has two modes; resource gathering mode and gamelist generation mode. First you gather data into Skyscraper's resource cache by scraping the platform with any of the supported scraping modules (eg. Skyscraper -p snes -s thegamesdb
). When you feel like you have gathered all the resources that you need, you then generate the gamelist by simply leaving out the -s <MODULE>
option (eg. Skyscraper -p snes
).
+Read more about the resource cache here.
Almost all of the scraping modules, such as ScreenScraper and TheGamesDb, have restrictions applied to avoid users overloading their servers. These restrictions are set by them, not Skyscraper. Some modules even require you to log in to use them. Read more about the restrictions and how to deal with them here. If you've created a user at ScreenScraper and want to use your credentials with Skyscraper, just add the following section to /home/<USER>/.skyscraper/config.ini
. Type it exactly like this, but with your own user and pass.
+
Skyscraper includes a powerful artwork compositor you can set up any way you like. Read more about the possibilities here. If you just want it to export the raw unmodified screenshot and other pieces of artwork, you can use the /home/<USER>/.skyscraper/artwork.xml.example2
and copy it to /home/<USER>/.skyscraper/artwork.xml
. Remember to regenerate the gamelists after you've done this to make use of the new artwork configuration. This can be done simply by running Skyscraper with Skyscraper -p PLATFORM
. This will regenerate the gamelist for that platform using this new artwork configuration.
Whenever you make any gamelist specific changes to Skyscraper's configurations, you need to regenerate the gamelists for your chosen frontend afterwards for the changes to take effect. This can be done simply by running Skyscraper with Skyscraper -p PLATFORM
. This will regenerate the gamelist for that platform for the default EmulationStation frontend using your new configuration.
+If you wish to export for one of the other supported frontends, you need to specify this. Read more about the frontends and their requirements here.
Did you remember to quit EmulationStation before running Skyscraper? EmulationStation writes its current gamelists back to disk when it is closed. If you run Skyscraper while EmulationStation is also running, it will overwrite any gamelists you just generated. You need to quit EmulationStation before generating the gamelists with Skyscraper to avoid this. You can quit EmulationStation by pressing F4 on a connected keyboard, or by selecting quit from the menus.
+YES! Absolutely. In addition to taking options from the command-line, Skyscraper also reads the /home/<USER>/.skyscraper/config.ini
file, which is where you should set everything up. This file allows you to configure things both globally, per-platform, per-frontend and per-scraping module. Read more about the available options and option priorities here.
You are probably running, or at some point ran, Skyscraper with sudo Skyscraper
instead of just Skyscraper
. Doing so will result in all sorts of weird behaviour as Skyscraper was then run as the root
user instead of the ordinary user. Depending on what command-line parameters you ran Skyscraper with, this will have caused permission issues for your files in /home/<USER>/RetroPie/roms
subfolders and might also have caused permission issues with the /home/<USER>/.skyscraper
folder.
+This is a common problem for new Linux users who are used to working with Windows. In Linux you rarely need Administrator privileges. Running a command with sudo
in front of it will run it as the root / Administrator user. And any folders or files being generated while that command runs, will then be owned by root. When you run software as the normal user afterwards and it tries to write to those folders or files, it will fail.
+You need to reset those permissions back to be owned by your ordinary user. If you are running RetroPie, the following commands might fix your problem (RUN AT YOUR OWN RISK!):
+
sudo
, as we need root permission to reset the folders and files (hence the warning). But in general, never run anything with sudo
unless you are specifically told to do so.
+Skyscraper contains a powerful artwork compositor that can customize the exported screenshot, boxart etc. any way you like. This is done through the /home/<USER>/.skyscraper/artwork.xml
file. The default configuration for this file is made to look good when exporting for the default frontend EmulationStation. When you use a different frontend, you need to customize this file a bit to make it look good. For Pegasus I suggest something like this:
+
<?xml version="1.0" encoding="UTF-8"?>
+<artwork>
+ <output type="screenshot" width="640"/>
+ <output type="cover" width="640" height="480">
+ <layer resource="cover" height="480" align="center" valign="middle">
+ <gamebox side="wheel" rotate="90"/>
+ </layer>
+ </output>
+ <output type="wheel" height="200"/>
+</artwork>
+
artwork.xml
here.
+artwork.xml
configuration and regenerated the game list, but it still looks the same in the frontend. Why is that?Did you perhaps edit /home/<USER>/skysource/artwork.xml
instead of /home/<USER>/.skyscraper/artwork.xml
? The latter is the one you need to edit.
+Read more about how to set up a custom artwork.xml
here.
When generating a game list with Skyscraper you have the option of generating it for several different frontends. A frontend is the graphical interface that lists and launches your games.
+Setting a frontend when generating a game list is done by setting the -f <FRONTEND>
command-line parameter as explained here or by setting it in /home/<USER>/.skyscraper/config.ini
as explained here. Use for the <FRONTEND>
value the frontend name all lowercase and with alphabetical characters only: emulationstation
, esde
, pegasus
, retrobat
, attractmode
. Some frontends have further options that are either optional or required. Check the frontend sections below for more information on this.
Warning
+Skyscraper will overwrite your game list (obviously). So if you have spend a lot of time hand-crafting metadata in a game list for any frontend, please remember to create a backup before overwriting it with Skyscraper. You can also tell Skyscraper to auto-backup old game lists prior to overwriting them. Read more about that here.
+When generating a game list for any frontend, Skyscraper will try to preserve certain metadata. Check the frontend sections below for more information on what metadata is preserved per frontend.
+/home/<USER>/RetroPie/roms/<PLATFORM>
gamelist.xml
This is the default frontend used when generating a game list with Skyscraper. If no frontend is defined on command-line or in config.ini
it will fall back to generating for EmulationStation. By default the <USER>
is pi
on RetroPie setups.
Skyscraper will preserve the following metadata when re-generating a game list for EmulationStation: favorite
, hidden
, kidgame
, lastplayed
, playcount
, sortname
. Also existing <folder/>
elements of a gamelist file will be preserved. The user editable sub-XML elements for a folder are listed in the Metadata.cpp
of EmulationStation.
Warning
+Folder data is not cached by Skyscraper, thus if you delete your gamelist.xml
, Skyscraper can not restore the edited folder elements from cache.
Automatic addition of folder elements if addFolder
is true:
+If at least one ROM is within a subfolder and this subfolder is not yet part of the gamelist.xml
file, it will be added with two mandatory subelements:
<path/>
reflects the relative subpath from the system folder and <name/>
, which represents the direct parent folder of a ROM by default. However, you may edit this to any name which should be shown in EmulationStation.Example
+Consider this folder structure below snes
, whereas each lowest folder contains at least one ROM:
+
gamelist.xml
and only if addFolders
is set true:
+[...]
+ <folder>
+ <path>./Retail</path>
+ <name>Retail</name>
+ </folder>
+ <folder>
+ <path>./Retail/USA</path>
+ <name>USA</name>
+ </folder>
+ <folder>
+ <path>./Retail/JP</path>
+ <name>JP</name>
+ </folder>
+ <folder>
+ <path>./Retail/EUR</path>
+ <name>EUR</name>
+ </folder>
+[...]
+
Retail
folder is added even if it does not contain a ROM because it is part of the path to the ROMs in the lowest folders.
+This is very similar to the default EmulationStation. This section only +highlights the differences.
+/home/<USER>/ES-DE/gamelists/<PLATFORM>
gamelist.xml
ROMs are expected to be in the input folder /home/<USER>/ROMs/<PLATFORM>
for
+every <PLATFORM>
you scrape.
Media filepaths (screenshots, marquees, videos, aso.) are not explicitly stored
+in the gamelist file. This is ES-DE default. However, Skyscraper will put the
+media files into the downloaded_media
folder (e.g.
+~/ES-DE/downloaded_media/<PLATFORM>/screenshots/
for screenshots) from where
+ES-DE will pick them up. Note that ES-DE does not support textures currently.
Tip
+You may override the default storage folder of ES-DE (/home/<USER>/ES-DE
)
+on a non-Windows build by setting the environment variable ESDE_APPDATA_DIR
+to a different folder as documented in the ES-DE documentation here.
All the gory details of ES-DE's gamelist file format you can find in the section +"Gamelist Reference" +here.
+Skyscraper will, additionally to EmulationStation preserved metadata, retain the
+following metadata when re-generating a game list for ES-DE: altemulator
,
+broken
, collectionsortname
, completed
, controller
, hidemetadata
,
+nogamecount
, nomultiscrape
. Also existing <folder/>
elements of a gamelist
+file will be preserved: For these on top is the folderlink
element is preserved.
/home/<USER>/RetroPie/roms/<PLATFORM>
gamelist.xml
This is modeled after EmualtionStation as it uses it with slight differences.
+/home/<USER>/.attract/romlists
<EMULATOR/PLATFORM>.txt
Attract-Mode is a bit more abstract when it comes to how it saves its game lists and media. To export for Attract-Mode you need to, in addition to setting the frontend, set -e <EMULATOR>
on command-line or in config.ini
. The <EMULATOR>
is a file that describes the platform / emulator you are generating a game list for. The file contains everything needed to tell Attract-Mode how to launch games for the platform, and even where to find the media files for the games (such as screenshots and videos).
If you are running RetroPie most of the <EMULATOR>
files will have been auto-generated for you. They are usually named the same as the platform you are generating a game list for. So, if you are generating for snes
you simply use Skyscraper -p snes -f attractmode -e snes
and that should work just fine. The <EMULATOR>
files are usually located at /home/<USER>/.attract/emulators/
. Check them out if you are curious.
Skyscraper will preserve the following metadata when re-generating a game list for Attract-Mode: altromname
, alttitle
, buttons
, cloneof
, control
, displaycount
, displaytype
, extra
, rotation
, status
.
/home/<USER>/RetroPie/roms/<PLATFORM>
metadata.pegasus.txt
Pegasus is easy and simple to generate a game list for. Simply do Skyscraper -p <PLATFORM> -f pegasus
. If you want to specify a custom launch command (if you are using RetroPie you don't have to, a default one will be used), you can set it on command-line with -e "<COMMAND>"
or in config.ini
with:
You need to add the individual platform rom directories to Pegasus (if they are not already defined) before any of them will show up! Start the Pegasus frontend, press ESC on the keyboard and choose Settings -> Set game directories. Simply point it to each individual platform sub-directory. For RetroPie you should have a path for each platform (eg. /home/<USER>/RetroPie/roms/snes
, /home/<USER>/RetroPie/roms/megadrive
etc.).
Info
+If you are generating game lists for Pegasus, it is highly recommended to disable third-party game list data sources! Otherwise you will have a mish-mash or different sources showing up in Pegasus. Start the Pegasus frontend, press ESC on the keyboard and choose Settings -> Enable/disable data sources and disable everything in that submenu.
+Then reload the game lists or restart Pegasus, and all of the platforms should show up with media and game information generated by Skyscraper.
Skyscraper will preserve any metadata key-value pairs added to the header and / or individual game list entries.
+ + + + + + + + + + + + + +The following describes how to import your own custom textual, artwork and / or video data into the resource cache for later use when generating game lists (enable game list generation mode simply by leaving out the -s
command line option).
Tip
+For any path in the following description, you can also add a platform subfolder. Skyscraper will auto-detect this and use it instead of the base import folder. For instance, you can create the folder /home/<USER>/.skyscraper/import/amiga
and it will use that as base instead of /home/<USER>/.skyscraper/import/
when importing for the amiga
platform.
Info
+Be sure to also check the --cache edit
option here.
To import videos or images into the resource cache, use the following procedure:
+Bubble Bobble.nes
will import images with a filename of Bubble Bobble.jpg
or Bubble Bobble.png
or other well-known image formats. As long as the base name is an exact match. Same goes for video files. I recommend only making use of well-known video formats since Skyscraper imports them directly without conversion (unless you convert them as described here), so they need to be supported directly by the frontend you plan to use..pdf
. The base name must match the ROM file, thus the game manual of the example is Bubble Bobble.pdf
./home/<USER>/.skyscraper/import/<PLATFORM>/screenshots
, covers
, wheels
, marquees
, videos
or manuals
folders.Skyscraper -p <PLATFORM> -s import
. If you named your files correctly, they will now be imported. Look for the green 'YES' in the output at the rom(s) you've placed files for. This will tell you if it succeeded or not.If you are importing videos, you also need to add the command line flag --flags videos
for this to work. Videos aren't imported or scraped by default, since it is considered a huge disk space hog. So keep this in mind if you want to import videos into the cache. If you plan on always using videos, consider adding this option to the config.ini instead.
Skyscraper also allows you to import textual data for any rom you have in your collection. All you need to do is to prepare files for each rom with an exactly matching base name. For instance Bubble Bobble.nes
should have a file called Bubble Bobble.txt
or Bubble Bobble.xml
or whatever suffix you want to use. The suffix is not important. What is important is that you place all of these raw data files into the /home/<USER>/.skyscraper/import/textual
folder. And then you need to make a definitions file so Skyscraper has a recipe for these files.
In order for Skyscraper to understand your textual data files, it needs a recipe. Or a definition of your format if you like. The format is completely up to you. The file must be placed at /home/<USER>/.skyscraper/import/definitions.dat
. Here follows a few examples with a matching data file for comparison:
Example 1
+Definitions file: /home/<USER>/.skyscraper/import/definitions.dat
<game>
+ <title>###TITLE###</title>
+ <description>###DESCRIPTION###</description>
+ <developer>###DEVELOPER###</developer>
+ <publisher>###PUBLISHER###</publisher>
+ <players>###PLAYERS###</players>
+ <rating>###RATING###</rating>
+ <ages>###AGES###</ages>
+ <genre>###TAGS###</genre>
+ <releasedate>###RELEASEDATE###</releasedate>
+</game>
+
Import file: /home/<USER>/.skyscraper/import/textual/<EXACT ROM BASE NAME>.xml
<game>
+ <title>The Game Title</title>
+ <description>This game is about yada, yada yada.</description>
+ <developer>The game developer</developer>
+ <publisher>The game publisher</publisher>
+ <players>Maximum number of players supported (example '4')</players>
+ <rating>The game rating 0-5 (example '3.5')</rating>
+ <ages>Minimum age restriction as integer (example '16')</ages>
+ <genre>Genres, Comma-separated</genre>
+ <releasedate>The game release date (example '1985-06-01')</releasedate>
+</game>
+
Tip
+As with Skyscraper 3.9.2 onwards you are no longer required to have the same indention or whitespace (hard tabs, spaces) as in the XML-like definitions.dat
. Matching is solely done via XML elements (e.g. <title>
), which saves you headache in identifiying whitespace mismatch between the XML template and your XML import file. If you use a custom XML-like definitions.dat
: Keep the opening and closing XML element on one line and use one line per ###tag###
as in the example definition shown above. This is the default in the provided definitions.dat
. In your import file just assure to use also closing elements (</title>
) and the same caSiNg in the elements between template and import file, ie. provide a valid XML file.
Example 2
+Definitions file: /home/<USER>/.skyscraper/import/definitions.dat
Title : ###TITLE###
+Description: ###DESCRIPTION###
+Developer : ###DEVELOPER###
+Publisher : ###PUBLISHER###
+Players : ###PLAYERS###
+Rating : ###RATING###
+Rating : ###AGES###
+Genre : ###TAGS###
+Date : ###RELEASEDATE###
+
Import file: /home/<USER>/.skyscraper/import/textual/<EXACT ROM BASE NAME>.txt
Title : Game Title
+Description: This game is about yada, yada yada.
+Developer : The game developer
+Publisher : The game publisher
+Players : Maximum number of players supported (example '4')
+Rating : The game rating 0-5 (example '3.5')
+Ages : Minimum age restriction as integer (example '16')
+Genre : Genres, Comma-separated
+Date : The game release date (example '1985-06-01')
+
Warning
+Make sure any line matches exactly with the line in the recipe including white-space characters such as newline characters, spaces and tabs! Otherwise Skyscraper won't recognize it. Especially for newline characters this can be difficult to debug. Newline characters appear the same in an editor so you won't notice the problem until you load it up in a hex editor.
+From the examples above you'll notice the ###SOMETHING###
tags. This is what Skyscraper recognizes your data from. The supported tags are:
###TITLE###
###DESCRIPTION###
###DEVELOPER###
###PUBLISHER###
###PLAYERS###
###AGES###
###RATING###
###TAGS###
###RELEASEDATE###
For ###PLAYERS###
, ###AGES###
, ###RELEASEDATE###
and ###RATING###
a certain format is required.
Release date must be of one of the following formats:
+yyyy
yyyy-MM
yyyy-MM-dd
(full ISO 8601 date)yyyy-MMM-dd
(MMM is Jan, Feb and so on...)MM/dd/yyyy
MMM, yyyy
(MMM is Jan, Feb and so on...)MMM dd, yyyy
(MMM is Jan, Feb and so on...)Rating can be either a number between 0 and 5, use as fraction only .5. Examples: 3 or 4.5 but not 4.25 or 1.0. This value will be divided by five to match the scale of the rating in a gamelist. Additionally, a value between 0.0 and 1.0 can be used. This value will not be recalculated. If you want to use 0.5 in this metric, provide it as .5 (note the missing zero). Do add a fraction of zero explicitly to denote a rating of 1.0 (100%), without this trailing fraction it will be interpreted as if it would be in the range of 0 to 5 (thus 20% for 1).
+When you've imported all of your data into the resource cache, you can make use of it by enabling Skyscrapers game list generation mode simply by leaving out the -s
command line option (eg. Skyscraper -p <PLATFORM>
). The game list generator will then make use of your imported data. If you don't know what the resource cache is, read more about it here.
Some game information comes in several languages. Skyscraper provides several ways of configuring these for your convenience. It is important to understand the distinction between game region and language. Setting a language is a user-preferred thing and will only affect the game descriptions and tags (genres). The remaining game data is tied to the region instead (artwork (screenshots, marquees, ...) and, in some cases, the game name).
+Below follows a general list of supported languages. Please note that not all languages are supported by all modules (support is noted inside the parenthesis for each language).
+When configuring languages be sure to use the short-names as shown (eg. 'no' for Norwegian).
+Skyscraper's default internal language priority list is as follows. Topmost language has highest priority and so forth.
+--lang LANGUAGE
or lang="LANGUAGE"
It is possible to manipulate the default internal language priority list. You can either do this on command-line or through /home/<USER>/.skyscraper/config.ini
. It is recommended to set it in config.ini
for a permanent setup.
Read this.
+ + + + + + + + + + + + + +--flags unpack
which broke with 3.6.8 due to deprecated Qt function calls<cover>
node and replaced it with <thumbnail>
for ES gamelist generation (Thank you to 'yisraeldov' for letting me know)--cache report
now supports symlinks (Thank you to 'vanativo' for reporting this)$ sudo apt-get install p7zip-full
for Debian base systems)<kidgame>bool</kidgame>
' output to Emulationstation gamelist generationThe ARTWORK release. Check artwork documentation here +* MAJOR: Completely rewrote the artwork compositing engine + * Now supports nested layers which anchors to the parent layer for easy placement + * Implemented 'balance' effect that adjusts the colors of the parent layer + * Implemented 'blur' effect that blurs the parent layer + * Implemented 'brightness' effect that adjusts the brightness of the parent layer + * Implemented 'contrast' effect that adjusts the contrast of the parent layer + * Implemented 'frame' effect that allows you to add a graphical frame to the parent layer + * Implemented 'gamebox' effect that turns the parent layer into a nice looking 3D game box + * Implemented 'mask' effect that allows you to mask out certain parts of the parent layer + * Implemented 'opacity' effect that adjusts the opacity of the parent layer + * Implemented 'rounded' effect that rounds the corners of the parent layer + * Implemented 'stroke' effect that outlines the parent layer + * Improved 'shadow' effect to adhere perfectly to softness as radius +* Added '-a' command line option for setting custom artwork xml config file +* Added 'artworkXml' config file options for setting custom artwork xml config file +* Implemented resource system that allows user to place files in '~/.skyscraper/resources' and use them in the '~/.skyscraper/artwork.xml' layers and effects +* Added 'From cache' boolean to output plus note about '--updatedb' +* 'simple mode' now also accepts "Y" as a yes answer instead of just "y" +* Now also looks for 'jp' region if no english region media is found for 'screenscraper' module +* Now always accepts 'screenscraper' results no matter if platform matches or not +* Now sets 'minMatch' to 0 by default for 'localdb' , 'arcadedb' and 'screenscraper' scraping modules. Can be overruled on command line and in config file +* Made localdb more thread safe, might've fixed rare issues of resources being mixed up internally +* Now works with filenames provided on command line even if they don't include full path +* Added resource sources to output +* Added 'wonderswan' and 'wonderswancolor' platforms
+Get a list of supported platforms with Skyscraper --help
.
After the initial work from torresflo @ GitHub it is possible to add new
+platforms by editing the platforms.json
file. However, since version 3.9.0 of
+Skyscraper this file is replaced by peas.json
(short for platforms,
+extensions/formats, aliases and scrapers. The scrapers list has been
+removed with 3.13.0 as it did not have any use) in the same folder.
Take this example from the peas.json
file:
[...]
+ "megadrive": {
+ "aliases": [
+ "genesis",
+ "mega drive",
+ "sega mega drive",
+ "sega mega drive/genesis"
+ ],
+ "formats": [
+ "*.bin",
+ "*.gen",
+ "*.md",
+ "*.sg",
+ "*.smd"
+ ]
+ },
+ [...]
+
<key>
: the key (megadrive
in the example) reflects the platform/folder
+ name (or more generally the handle), usually provided with -p
on the command
+ line.aliases
: these aliases can be used to allow additional hits for a search.
+ The use of aliases is handy especially for scraping websites which do not
+ provide a web API. Currently Screenscraper, Mobygames or The Games DB are
+ providing an web API. See also section about
+ platforms_idmap.csv
and Updating the
+ files below.formats
: set of ROM file extensions which will be included in scraping if
+ a ROM file is not provided explicitly via command line.Tip
+Since Skyscraper 3.13.0 you may also maintain local changes to the peas.json
+in a separate file called peas_local.json
alongside to the peas.json
. The
+format is identical to the peas.json
. Precedence: Any platform entry from
+peas_local.json
with the same platform name as in peas.json
overwrites
+the one from peas.json
.
Tip
+If you need a specific folder name for a platform (on your setup or due to an
+EmulationStation theme) use a symbolic link (see megadrive
(=folder) and
+genesis
(=symlink) for example on RetroPie or plus4
(=folder) and c16
+(=symlink)) instead of adding a new platform in the JSON file.
The file platforms_idmap.csv
defines exact platform id for the web API of the
+three before mentioned scraping sites. It is a CSV file which maps the platform
+handle (e.g. megadrive
) to the respective platform id of the scraping sites:
You can display the number with their platform name on each of the three
+scraping sites with the script peas_and_idmap_verify.py
. Find the script
+sibling to the Skyscraper executable. Below is a part of the output:
[...]
+ ├── megadrive
+ │ ├── ScrS 1: Megadrive
+ │ ├── Moby 16: Genesis
+ │ └── TGDB 36: Sega Mega Drive
+[...]
+
Outline:
+peas_local.json
sibling to peas.json
. Enter in this file an
+ empty {}
JSON object.peas_local.json
inside the outer (empty)
+ block created before, or copy an existing block and adapt to your needs. For
+ RetroPie your chosen <platform_name>
must match the folder in
+ ~/RetroPie/roms/<platform_name>
.<platform_name>
also in platforms_idmap.csv
(see below for details).es_systems.cfg
+ as documented
+ here.There is also a an verbatim example, you may skip the next section initially and +can continue with the hands-on example.
+peas.json
(or peas_local.json
) and platforms_idmap.csv
¶These two files are ment to be locally edited and extended for additional
+platforms. Whenever you add a new platform block to the peas.json
do also
+lookup the corresponding platform ids and add them to platforms_idmap.csv
for
+the scraping sites with an API.
To find the platform ids for Screenscraper, Mobygames and The Games DB, please
+consult the files screenscraper_platforms.json
, mobygames_platforms.json
and
+tgdb_platforms.json
which are located sibling to your config.ini
of the
+Skyscraper installation. If you can not identify an id in these files use -1
+as value in the CSV. If you add -1
to CSV, the aliases
from peas are tried
+to find a match upon scraping. Edits in screenscraper_platforms.json
,
+mobygames_platforms.json
and tgdb_platforms.json
are not needed and moreover
+they will be overwritten with each Skyscraper update, as these files are only a
+reference for finding the id values for the platforms_idmap.csv
.
For those scraping sites without an API or without exact id match do use the
+platform name which is used on the scraping site and put it into the in the aliases
+list in the peas.json
for the respective platform/system at <platform_name>
.
Example
+The platforms ScummVM or Steam do not have an exact match on Mobygames, however you may scrape successfully for ScummVM and Steam games if you use 'PC', 'DOS', 'Windows', 'Linux' or similar as "aliases": ...
in the "scummvm": ...
or "steam": ...
section of peas.json
. Usually you find the platform information if you lookup the game manually on the scraping website.
Let the platform/systemname be satellaview
. You may read about this SNES
+enhancing peripheral here.
peas_local.json
¶Actually, this is a copy of the "snes:
+..."
+block, with slight modifications.
platforms_idmap.csv
¶To fill in the values into the CSV file do consult the
+screenscraper_platforms.json
,
+mobygames_platforms.json
+and
+tgdb_platforms.json
+files. Try to find the ID for the platform in these files. For satellaview
+only Screenscraper has an exact match:
+107
.
+Use -1
as ID when no exact match is provided for the scraping site. Whenever
+an -1
is encountered Skyscraper tries the "aliases":
from peas.json
(and
+from peas_local.json
if present) to find scraping data.
Add this information at the end of the CSV file. Why at the end? On rare
+occasions the platforms_idmap.csv
may be updated on a new release. On RetroPie
+the installed file is then named platforms_idmap.csv.rp-dist
, having changes
+at the end may be a less cumbersome manual merge with your local
+platforms_idmap.csv
.
Add this information: +
+This part should be added to your ~/.emulationstation/es_systems.cfg
. See
+here for
+additional information on this.
Warning
+Stop EmulationStation before editing the config file.
+/home/pi/RetroPie/roms/satellaview
."formats":
block of peas.json
.
+ However, Skyscraper uses case insensitive extension mapping. The
+ extensions .7z
and .zip
are added automagically by Skyscraper, thus the
+ "formats":
list is usually shorter than the EmulationStation <extension/>
+ list.snes
as
+ Note
+If you run a different frontend than EmulationStation, consult the documentation for your frontend on how to add additional systems.
+satellaview/gamelist.xml
as in the introductive
+ use case using Skyscraper -p
+ satellaview -s screenscraper
and Skyscraper -p satellaview
Kudos
+Thanks to retrobit @ GitHub for contributing this usecase.
+platforms.json
and screenscraper.json
¶Info
+If you neither edited platforms.json
nor screenscraper.json
or do not
+have these files in the Skyscraper config folder (sibling to the
+config.ini
) you can safely ignore this section.
If you have local changes in these files you can transfer your local
+changes to the new files. The new peas.json
is the platforms.json
but in a
+more compact form which has less visual overhead and also allows faster parsing.
+The platforms_idmap.csv
maps the former screenscraper.json
platform ids plus
+the platform ids of two more web API sites.
Use the script convert_platforms_json.py
(sibling to the Skyscraper
+executable) to convert the platforms.json
to a peas_mine.json
file which you
+can then diff to the peas.json
and transfer your changes to peas.json
or
+peas_local.json
.
Use the script check_screenscraper_json_to_idmap.py
to identify differences
+from your screenscraper.json
. Then use the three <scraper>_platforms.json
+files to identify the matching platform ids to be entered in
+platform_idmap.csv
. Use -1
in this file, if there is no matching platform id.
Filenames shown strikethrough are superseded.
+Filenames shown italic are user editable.
File |
+Introduced with Version | +Notes (version) | +
---|---|---|
mobygames.json |
+3.8.0 | +superseded by mobygames_platforms.json (v3.9.0); not to be edited; IDs are used in platforms_idmap.csv |
+
platforms.json |
+3.7.7-2 (@detain) | +superseded by peas.json (3.9.0) and peas_local.json (3.13.0); do edit the latter to add/change platforms; these files use a leaner format than platforms.json |
+
platforms_idmap.csv |
+3.9.0 | +maps the platform names (handles) from peas.json / peas_local.json to exact platform IDs used in scrapers MobyGames, Screenscraper or TGDB; do edit to add new platforms |
+
screenscraper.json |
+3.7.7-2 (@detain) | +IDs formerly used in here are part of platforms_idmap.csv (3.9.0); superseded by screenscraper_platforms.json (3.9.0) which is not to be edited |
+
tgdb_developers.json |
+2.5.3 (@muldjord) | +API mapping of 'Developers'; Uses leaner format as before (3.9.0); not to be edited | +
tgdb_genres.json |
+3.9.0 | +API mapping of 'Genres' (3.9.0); not to be edited | +
tgdb_platforms.json |
+3.9.0 | +API mapping of 'Platforms' (3.9.0); not to be edited; IDs are used in platforms_idmap.csv |
+
tgdb_publishers.json |
+2.5.3 (@muldjord) | +API mapping of 'Publishers'; Uses leaner format as before (3.9.0); not to be edited | +
Some game information and / or game media is region-based. Skyscraper provides several ways of configuring these for your convenience. But most importantly; it supports region auto-detection directly from the file names. Read on for more information about how regions are handled by Skyscraper.
+Below follows a general list of supported regions. Please note that not all regions are supported by all modules (support is noted inside the parenthesis for each country or region).
+When configuring regions be sure to use the short-names as shown (eg. 'fr' for France).
+Region Key | +Country/Region (supported by scraping site) | +
---|---|
ame | +American continent (screenscraper) | +
asi | +Asia (screenscraper, igdb) | +
au | +Australia (screenscraper, igdb, mobygames) | +
bg | +Bulgaria (screenscraper, mobygames) | +
br | +Brazil (screenscraper, mobygames) | +
ca | +Canada (screenscraper, mobygames) | +
cl | +Chile (screenscraper, mobygames) | +
cn | +China (screenscraper, igdb, mobygames) | +
cus | +Custom (screenscraper) | +
cz | +Czech republic (screenscraper, mobygames) | +
de | +Germany (screenscraper, mobygames) | +
dk | +Denmark (screenscraper, mobygames) | +
eu | +Europe (screenscraper, igdb) | +
fi | +Finland (screenscraper, mobygames) | +
fr | +France (screenscraper, mobygames) | +
gr | +Greece (screenscraper, mobygames) | +
hu | +Hungary (screenscraper, mobygames) | +
il | +Israel (screenscraper, mobygames) | +
it | +Italy (screenscraper, mobygames) | +
jp | +Japan (screenscraper, igdb, mobygames) | +
kr | +Korea (screenscraper, mobygames) | +
kw | +Kuwait (screenscraper) | +
mor | +Middle East (screenscraper) | +
nl | +Netherlands (screenscraper, mobygames) | +
no | +Norway (screenscraper, mobygames) | +
nz | +New Zealand (screenscraper, igdb, mobygames) | +
oce | +Oceania (screenscraper) | +
pe | +Peru (screenscraper) | +
pl | +Poland (screenscraper, mobygames) | +
pt | +Portugal (screenscraper, mobygames) | +
ru | +Russia (screenscraper, mobygames) | +
se | +Sweden (screenscraper, mobygames) | +
sk | +Slovakia (screenscraper, mobygames) | +
sp | +Spain (screenscraper, mobygames) | +
ss | +ScreenScraper (screenscraper) | +
tr | +Turkey (screenscraper, mobygames) | +
tw | +Taiwan (screenscraper, mobygames) | +
uk | +United Kingdom (screenscraper, mobygames) | +
us | +USA (screenscraper, igdb, mobygames) | +
wor | +World (screenscraper, igdb, mobygames) | +
Skyscraper will try to auto-detect the region from the file name. It will look for designations such as (Europe)
or (e)
and set the region accordingly. This currently works for the following regions and / or countries:
So if your files are named like Game Name (Europe).zip
, there's no need to configure regions manually. Skyscraper will recognize Europe
and add it to the top of the internal region priority list. If info or media isn't found for the auto-detected region, it will move down the list and check the next region on the list until it finds one that has data for the requested resource.
Skyscraper's default internal region priority list is as follows. Topmost region has highest priority and so forth.
+--region REGION
(command line) or region="REGION"
(config.ini)If you insist, of course you can configure the region manually. You can either do this on command-line or through /home/<USER>/.skyscraper/config.ini
. It is recommended to set it in config.ini
for a permanent setup.
Read this.
+ + + + + + + + + + + + + +Skyscraper supports several online and local sources when scraping data for your roms. This makes Skyscraper a hugely versatile tool since it also caches any resources that are gathered from any of the modules. The cached data can then be used to generate a game list and composite artwork later.
+Choosing a scraping module is as simply as setting the -s <MODULE>
option when running Skyscraper on the command line. It also requires a platform to be set with -p <PLATFORM>
. If you leave out the -s
option Skyscraper goes into game list generation mode and combines your cached data into a game list for the chosen platform and frontend. Read more about the resource cache here.
For scraping modules that support or require user credentials you have the option of either setting it on commandline with -u <USER:PASSWD>
or -u <KEY>
or better yet, by adding it permanently to the Skyscraper configuration at /home/<USER>/.skyscraper/config.ini
as described here
Below follows a description of all scraping modules.
+screenscraper
screenscraper_platforms.json
sibling to your config.ini
cover
, screenshot
, wheel
, manual
, marquee
, video
Skyscraper -p snes -s screenscraper
ScreenScraper is probably the most versatile and complete retro gaming database out there. It searches for games using either the checksums of the files or by comparing the exact file name to entries in their database.
+It can be used for gathering data for pretty much all platforms, but it does have issues with platforms that are ISO based. Still, even for those platforms, it does locate some games.
+It has the best support for the wheel
and marquee
artwork types of any of the databases, and also contains videos and manuals for a lot of the games.
I strongly recommend supporting them by contributing data to the database, or by supporting them with a bit of money. This can also give you more threads to scrape with.
+Note
+Exact file name matching does not work well for the arcade
derived platforms in cases where a data checksum doesn't match. The reason being that arcade
and other arcade-like platforms are made up of several subplatforms. Each of those subplatforms have a high chance of containing the same file name entry. In those cases ScreenScraper can't determine a unique game and will return an empty result.
thegamesdb
tgdb_platforms.json
sibling to your config.ini
cover
, screenshot
, wheel
, marquee
Skyscraper -p snes -s thegamesdb
For newer games there's no way around TheGamesDb. It recently had a huge redesign and their database remains one of the best out there. I would recommend scraping your roms with screenscraper
first, and then use thegamesdb
to fill out the gaps in your cache.
There's a small caveat to this module, as it has a monthly request limit (see above) per IP per month. But this should be plenty for most people.
+Their API is based on a file name search. This means that the returned results do have a chance of being faulty. Skyscraper does a lot internally to make sure accepted data is for the correct game. But it is impossible to ensure 100% correct results, so do keep that in mind when using it. Consider using the --flags interactive
command line flag if you want complete control of the accepted entries.
arcadedb
cover
, screenshot
, wheel
, marquee
, video
Skyscraper -p fba -s arcadedb
Several Arcade databases using the MAME file name id's have existed throughout the years. Currently the best one, in my opinion, is the ArcadeDB made by motoschifo. It goes without saying that this module is best used for arcade platforms such as fba
, arcade
and any of the mame sub-platforms.
As it relies on the MAME file name id when searching, there's no use trying to use this module for any non-MAME files. It won't give you any results.
+This module also supports videos for many games.
+openretro
cover
, screenshot
, marquee
Skyscraper -p amiga -s openretro
If you're looking to scrape the Amiga RetroPlay LHA files, there's no better way to do this than using the openretro
module. It is by far the best WHDLoad Amiga database on the internet when it comes to data scraping, and maybe even the best Amiga game info database overall.
The database also supports many non-Amiga platforms, but there's no doubt that Amiga is the strong point.
+mobygames
mobygames_platforms.json
sibling to your config.ini
cover
, screenshot
Skyscraper -p fba -s mobygames <SINGLE FILE TO SCRAPE>
Skyscraper -p fba -s mobygames --startat <FILE TO START AT> --endat <FILE TO END AT>
MobyGames. What can I say. If you haven't heard about this database before you've been missing out. It's one of the best and oldest games databases on the internet. You'll probably come across references to MobyGames on other sites when searching for retro games. There's a reason for that - it's that good.
+There's a caveat to the module as it has quite strong restrictions for the number of requests that are allowed at any given time. This restriction is global for the entire Skyscraper user base, which means that it might quit on you if other users are currently scraping from it. For this reason it has been strongly limited inside of Skyscraper by forcing a maximum number of rom scrapings per run.
+Please use this module sparingly. And only ever use it to scrape those last few roms you can't get any data for using any of the other sources.
+igdb
Skyscraper -p fba -s igdb <SINGLE FILE TO SCRAPE>
Skyscraper -p fba -s igdb --startat <FILE TO START AT> --endat <FILE TO END AT>
IGDB is a relatively new database on the market. But absolutely not a bad one at that. It has a couple caveats though, as the database doesn't distinguish between platform versions of the same game when it comes to any artwork resources (they are working to implement this at some point). This makes it less usable in a retro game scraping context as many of the games differ drastically visually between the old platforms. For that reason alone, this module will only provide textual data for your roms for the time being.
+It is required to register with the Twitch dev program (IGDB is owned by Twitch) and create a free client-id and secret-key pair for use with Skyscraper. The process of getting this free client-id and secret-key pair is quite easy. Just follow the following steps:
+http://localhost
as OAuth redirect URLNew secret
/home/<USER>/.skyscraper/config.ini
):Substitute CLIENTID and SECRETKEY with your own details. And that's it, you should now be able to use the IGDB module.
+worldofspectrum
cover
, screenshot
Skyscraper -p zxspectrum -s worldofspectrum
If you're looking specifically for ZX Spectrum data, this is the module to use. World of Spectrum is probably the most complete ZX Spectrum resource and information database in existence. I strongly recommend visiting the site if you have any interest in these little machines. It's a cornucopia of information on the platform.
+esgamelist
screenshot
, marquee
, video
Skyscraper -p snes -s esgamelist
This module allows you to import data from an existing EmulationStation game list into the Skyscraper cache. This is useful if you already have a lot of data and artwork in a gamelist.xml file and you wish to use it with Skyscraper.
+Skyscraper will search for the gamelist.xml
file at <INPUT FOLDER>/gamelist.xml
which by default is /home/<USER>/RetroPie/roms/<PLATFORM>/gamelist.xml
. If that file isn't found it will look for it at /home/<USER>/.skyscraper/import/<PLATFORM>/gamelist.xml
.
import
cover
, screenshot
, wheel
, marquee
, video
Skyscraper -p snes -s import
Read a thorough description of this module here.
+ + + + + + + + + + + + + +The following describes a typical way of using Skyscraper. It's really simple to use once you understand the core concept. It has a couple of layers you should familiarize yourself with in order to get the best results.
+Info
+Always remember to close your frontend before using Skyscraper! Otherwise any game list generated by Skyscraper will be overwritten when you exit the frontend.
+Info
+Some of the scraping modules either require or work best when you have registered a user with their service and configured Skyscraper to make use of these credentials when scraping with the module. The suggested screenscraper
source used in the command a bit further down will work without credentials. But when the service is busy, it will give you an error! So consider registering users with the relevant sources and configure Skyscraper to make use of them. Read about all scraping modules here.
Now, let's dig in!
+You probably installed Skyscraper in order to get some data and artwork on your frontend game lists. So let's say that you have a bunch of Super Nintendo files you wish to populate with data. You'd first gather some data into the Skyscraper resource cache and then generate a game list using the cached data afterwards. So it's a two-step process.
+ +Skyscraper can gather data from several scraping modules / sources (set with -s <MODULE>
). So you'd start by scraping the snes
platform (set with -p <PLATFORM>
) with one or more of those modules. Each time you do so, all of the data will be saved in the Skyscraper cache.
This should have given you a lot of data for your games. But some of it might still be missing. So let's continue gathering some data from another module.
+ +Each time you scrape your roms with a new scraping module, all of the data that was gathered will be cached inside Skyscraper's internal resource cache. The cache can contain one of each resource for each of the scraping modules.
+By having gathered data from both sources (screenscraper
and thegamesdb
) there's a good chance that you will get a complete result if the data from both is combined. And that's exactly what Skyscraper can do for you.
Running that command will generate a game list for the chosen frontend (default is EmulationStation). It combines all of the cached data into the most complete results. This phase also includes the artwork compositing. If you use a non-default frontend, you can configure this in Skyscraper's configuration as described here. Or you can set it on command-line as described here.
+The source priority for each resource is defined per-platform and can be changed at any time. Read more about that here.
+You're done! The game list has been generated and you can relaunch your frontend to enjoy the game data and artwork. Or you can move on and scrape data for another platform. Just remember to distinguish between the gathering phase and the game list generation phase. Always gather first, then generate the game list afterwards.
+Tip
+You can always re-generate a game list if you changed the artwork configuration or gathered new data for any of your roms. Simply rerun the Skyscraper -p <PLATFORM>
command again, and the game list will be re-generated with your new settings using all of the previously cached data.
Tip
+If you re-gather data for a platform, it will skip games that already have data in the resource cache to save bandwidth for the online sources. To force a refresh of the data directly from the source, you need to set the --refresh
option.
Yes, yes, yes. This is just the basics. You can customize the artwork completely to your liking. You can prioritize the different cached resources any way you like. You can scrape single games in order to gather information for just that one game. You can enable video scraping. You can import your own data into the cache. You can edit the cached data. You can clean your cache. You can create aliases for games that are difficult to scrape (see aliasMap.csv
sibling to your config.ini
). You can change your preferred region and language for the modules that support it. You can set up keys or user id's and password for the scraping modules that take advantage of this. There's a lot you can do if you want to.
The important thing to note is that the defaults for Skyscraper is set to work well for most users. So unless you see something you want to change, you don't have to. But it's equally important to know that you can.
+If this piqued your interest, please read the entire documentation thorougly. Everything is documented to the last detail. It can seem overwhelming. But it gives you some really powerful options to scrape just the way you want. Read more about the advanced features here.
+ + + + + + + + + + + + + +If you are a fellow Linux user you may be familiar with the XDG Base Directory +Specification (BDS) as part of the freedesktop.org project. If not, you can find +the short specification +here, the +ArchWiki has also +additional information and list tools that support the specification. Also, +wikipedia +has a brief section on the topic.
+Tip
+In this documentation you will note paths and file locations which relate to the +"genuine Skyscraper" approach. With the table below you can map these +locations to their counterpart in XDG BDS.
+Type | +Genuine Skyscraper Loation | +XDG BDS Location | +
---|---|---|
CONFIG | +All files, but no subfolders of ~/.skyscraper/* |
+$XDG_CONFIG_HOME/skyscraper/ |
+
CACHE | +All of ~/.skyscraper/cache/ |
+$XDG_CACHE_HOME/skyscraper/ |
+
IMPORT | +All of ~/.skyscraper/import/ |
+$XDG_DATA_HOME/skyscraper/import/ |
+
RESOURCE | +All of ~/.skyscraper/resources/ |
+$XDG_DATA_HOME/skyscraper/resources/ |
+
LOG | +Skipped files/games during processing ~/.skyscraper/skipped*.txt |
+$XDG_STATE_HOME/skyscraper/ |
+
REPORT | +Cache reports ~/.skyscraper/reports/* |
+$XDG_STATE_HOME/skyscraper/reports/ |
+
LOG | +Rare Screenscraper Error Log ~/.skyscraper/screenscraper_error.json |
+$XDG_STATE_HOME/skyscraper/ |
+
As the per specifications: All paths must be absolute, relative paths will be +ignored by Skyscraper. Non-existing paths will be created by Skyscraper.
+The XDG feature can be used with any modern Linux distribution, but should not +be enabled on RetroPie-Installments as they rely on the genuine Skyscraper +folder structure.
+Review the Skyscraper project file (skyscraper.pro
). Enable the line
+DEFINES+=XDG
, rebuild and install Skyscraper. Then transfer the files from the
+genuine Skyscraper location to their XDG counterpart. If you start from scratch
+Skyscraper will be deploy its configuration files to the XDG destinations (by
+default sourced from /usr/local/etc/skyscraper
).
If you want to verify if Skyscraper is compiled with XDG support run Skyscraper
+--version
. If you see a XDG label below the version it means XDG is enabled and
+supported.
To disable XDG support undo the steps above.
+The Fine Print
+Skyscraper supports all user directories of the XDG BDS (except
+$HOME/.local/bin
for user binaries). The system
+directories properties (XDG_DATA_DIRS
and XDG_CONFIG_DIRS
) are
+not utilized by Skyscraper.