diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/404.html b/404.html new file mode 100644 index 00000000..fd5635ec --- /dev/null +++ b/404.html @@ -0,0 +1,939 @@ + + + + + + + + + + + + + + + + + + + + + Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ +

404 - Not found

+ +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/ARTWORK/index.html b/ARTWORK/index.html new file mode 100644 index 00000000..61048a4c --- /dev/null +++ b/ARTWORK/index.html @@ -0,0 +1,3070 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Artwork & Layout - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + +

Artwork & Layout

+ +

Artwork Look and Effects

+

Skyscraper allows you to fully customize how you want the final frontend artwork to appear by editing the file /home/<USER>/.skyscraper/artwork.xml. Each piece of artwork can be composited from one or more of the other artwork resources and you can even apply several effects to them.

+
    +
  • Each <output> node exports a single piece of artwork.
  • +
  • If no <layer> nodes are nested in an output node, it will simply export the defined type as raw unmanipulated artwork.
  • +
  • All layer nodes and their nested effects are rendered / applied top-to-bottom.
  • +
+

Read on for an example artwork.xml and a more thorough description of the <output>, <layer> and the various available effect nodes.

+
+

Warning

+

Some users seem to have a habbit of editing the /home/<USER>/skysource/artwork.xml file. This is not the correct file! Please read the first paragraph. :)

+
+
+

Tip

+

Every time you change the artwork configuration, remember to regenerate the game list. Your changes won't take effect until you do so. Read more about this here.

+
+

Watch a video demonstrating the artwork compositing features here. Note that the video is quite old and only demonstrates a fraction of the possibilities you have with the Skyscraper compositor. It's all thoroughly documented below.

+

Example artwork.xml

+

An example could look something like this:

+
<?xml version="1.0" encoding="UTF-8"?>
+<artwork>
+  <output type="screenshot" width="640" height="480">
+    <layer resource="screenshot" x="20" width="520" height="390" align="center" valign="middle">
+      <rounded radius="10"/>
+      <stroke width="5"/>
+    </layer>
+    <layer resource="cover" height="250" x="0" y="-10" valign="bottom">
+      <gamebox side="wheel" rotate="90"/>
+      <shadow distance="5" softness="5" opacity="70"/>
+    </layer>
+    <layer resource="wheel" width="250" x="-10" align="right">
+      <shadow distance="5" softness="5" opacity="70"/>
+    </layer>
+  </output>
+</artwork>
+
+

Which results in a screenshot being exported to look like this:

+
+

Artwork example +

+
Artwork example
+
+

XML Definitions

+

Click the following links to quickly go to a desired section:

+

Nodes: XML Preamble, Artwork node, Output node, Layer node

+

Effects: Blur effect, Brightness effect, Color balance effect, Colorize effect, Contrast effect, Frame effect, Gamebox effect, Hue effect, Mask effect, Opacity effect, Rotate effect, Rounded effect, Saturation effect, Scanlines effect, Shadow effect, Stroke effect.

+

Other: Custom image resources.

+

XML Preamble (Mandatory)

+
<?xml version="1.0" encoding="UTF-8"?>
+
+

This is a special node used to describe the XML language used in the document. Don't change it.

+

'artwork' node (Mandatory)

+
<artwork>
+...
+</artwork>
+
+

This node is the document base node. It contains all other nodes. Don't change or delete it.

+

'output' node(s) (Optional)

+
<output type="screenshot" resource="wheel" mpixels="0.1" width="640" height="400">
+...
+<output>
+
+

The output node describes a single piece of artwork to be exported when scraping. You can have as many (or few) output nodes as you like.

+

An output node with no nested 'layer' nodes will export the type with no compositing.

+
'type' attribute (Mandatory)
+

The type to be exported. Can be:

+
    +
  • screenshot
  • +
  • cover
  • +
  • wheel
  • +
  • marquee
  • +
  • texture
  • +
+
'resource' attribute [from v2.7.6] (Optional)
+

The resource attribute can be set to one of the following:

+
    +
  • screenshot
  • +
  • cover
  • +
  • wheel
  • +
  • marquee
  • +
  • texture
  • +
+

For instance, if you wish to export a 'marquee' image but want it to make use of the 'wheel' resource instead, you can set <output type="marquee" resource="wheel"/>. It will then export a 'marquee' but use the raw unmanipulated 'wheel' image when doing so. So if your frontend theme, such as some EmulationStation themes, makes use of the 'marquee' artwork, it will then be using the 'wheel' for it. If this attribute is left out, 'resource' will be set to the same as 'type'.

+
+

Info

+

The screenshot, cover, wheel and marquee types are hardcoded. It's not currently possible to create custom types.

+
+
'mpixels' attribute [from v2.7.8] (Optional)
+

Set the desired area size for the image resource in mpixels (eg '0.1'). This can be used as an alternative to setting width or height and is useful in cases where images vary a lot in size. 'width' and 'height' attributes will be ignored if this is set.

+
'width' and 'height' attributes (Optional)
+

Sets the width and height of the artwork export. If both are left out it will use the original size of the scraped artwork. If one is left out, it will keep the aspect ratio relative to that.

+

'layer' node(s) (Optional)

+
<layer resource="cover" mode="overlay" opacity="50" x="5" y="5" mpixels="0.1" width="10" height="10" align="center" valign="middle"/>
+...
+</layer>
+
+

Like layers in Gimp or almost any other respectable photo manipulation software, you can add as many of these nodes as you like. You can even nest them inside of each other, in which case the child layer will be anchored to the topleft corner of the parent layer, making it easier to align.

+
+

Note

+

Layer and effect nodes are rendered from top to bottom.

+
+
'resource' attribute (Optional)
+

The artwork resource to be used by the layer. It can be:

+
    +
  • screenshot
  • +
  • cover
  • +
  • wheel
  • +
  • marquee
  • +
+

If the resource attribute is left out the layer will simply be transparent. This is useful when aligning nested layers that differ in size across games.

+
'mode' attribute [from v2.3.3] (Optional)
+

Sets the layer render mode. It can be:

+
    +
  • multiply
  • +
  • screen
  • +
  • overlay
  • +
  • darken
  • +
  • lighten
  • +
  • colordodge
  • +
  • colorburn
  • +
  • hardlight
  • +
  • softlight
  • +
  • difference
  • +
  • exclusion
  • +
  • sourcein
  • +
  • destinationin
  • +
  • sourceout
  • +
  • destinationout
  • +
  • sourceatop
  • +
  • destinationatop
  • +
  • xor
  • +
+

If left out the layer is simply rendered as is on top of the parent layer.

+
+

Mode examples +

+
Mode examples (note the in-image captions)
+
+
'opacity' attribute [from v2.3.3] (Optional)
+

Defines the opacity of the layer. 100 is completely visible. 0 is completely transparent.

+
'x' and 'y' attributes (Optional)
+

X and Y coordinates for placement. Relative to 'align' and 'valign' and the parent layer. Either can be left out in which case it will be set to 0.

+
'mpixels' attribute [from v2.7.8] (Optional)
+

Set the desired area size for the image resource in mpixels (eg '0.1'). This can be used as an alternative to setting width or height and is useful in cases where images vary a lot in size. 'width' and 'height' attributes will be ignored if this is set.

+
'width' and 'height' attributes (Optional)
+

Sets the width and height of the layer. If either is left out it will keep the aspect ratio relative to the other. If both are left out it will use the original dimensions of the scraped resource.

+
'align' attribute (Optional)
+

The horizontal alignment of the layer. It can be:

+
    +
  • left
  • +
  • center
  • +
  • right
  • +
+

The alignment is relative to the parent layer.

+
'valign' attribute (Optional)
+

The vertical alignment of the layer. It can be:

+
    +
  • top
  • +
  • middle
  • +
  • bottom
  • +
+

The alignment is relative to the parent layer.

+

'balance' effect node (Optional)

+
+

Effect example +

+
Balance effect sample
+
+
<layer>
+  <balance red="10" green="20" blue="30"/>
+</layer>
+
+

Must be nested inside a layer node. Adjusts the color balance of the parent layer.

+
'red' attribute (Optional)
+

The red color adjustment. Can be -255 to 255.

+
'green' attribute (Optional)
+

The green color adjustment. Can be -255 to 255.

+
'blue' attribute (Optional)
+

The blue color adjustment. Can be -255 to 255.

+

'blur' effect node (Optional)

+
+

Effect example +

+
Blur effect sample
+
+
<layer>
+  <blur softness="5"/>
+</layer>
+
+

Must be nested inside a layer node. Blurs the parent layer.

+
'softness' attribute (Mandatory)
+

Defines the radius of the blur. Higher means blurrier.

+

'brightness' effect node (Optional)

+
+

Effect example +

+
Brightness effect sample
+
+
<layer>
+  <brightness value="10"/>
+</layer>
+
+

Must be nested inside a layer node. Adjusts the brightness of the parent layer.

+
'value' attribute (Mandatory)
+

The difference value for the adjustment. Can be -255 to 255.

+

'colorize' effect node [from v2.3.3] (Optional)

+
+

Effect example +

+
Colorize effect sample
+
+
<layer>
+  <colorize hue="180" saturation="-50"/>
+</layer>
+
+

Must be nested inside a layer node. Colorizes the parent layer with a single hue.

+
'hue' attribute (Mandatory)
+

Sets the hue in degrees. Can be 0 to 360.

+
'saturation' attribute (Optional)
+

Sets the saturation delta value of the colorize effect. Can be -127 to 127.

+

'contrast' effect node (Optional)

+
+

Effect example +

+
Contrast effect sample
+
+
<layer>
+  <contrast value="10"/>
+</layer>
+
+

Must be nested inside a layer node. Adjusts the contrast of the parent layer.

+
'value' attribute (Mandatory)
+

The difference value for the adjustment. Can be -255 to 255.

+

'frame' effect node (Optional)

+
+

Effect example +

+
Frame effect sample
+
+
<layer>
+  <frame file="[custom image resource]" x="0" y="0" width="100" height="100"/>
+</layer>
+
+

Must be nested inside a layer node. This applies a frame to the parent layer.

+
+

Tip

+

You can get some cool results by first applying a mask, then applying a frame. This is useful for frames that aren't square.

+
+
'file' attribute (Mandatory)
+

The filename of the custom image resource to be used as frame.

+
'width' attribute (Optional)
+

Sets the width of the frame in pixels. If left out it will be set to the width of the parent layer.

+
'height' attribute (Optional)
+

Sets the height of the frame in pixels. If left out it will be set to the height of the parent layer.

+
'x' attribute (Optional)
+

Sets the x coordinate of the frame relative to the parent layer. If left out it will be set to 0.

+
'y' attribute (Optional)
+

Sets the y coordinate of the frame relative to the parent layer. If left out it will be set to 0.

+

'gamebox' effect node (Optional)

+
+

Effect example +

+
Gamebox effect sample
+
+

Left image shows the result with the wheel artwork applied to the side. Right image shows the original cover layer.

+
<layer resource="cover" height="200">
+  <gamebox side="[custom image resource]" rotate="90" sidescaling="both"/>
+</layer>
+
+

Must be nested inside a layer node. Renders a nifty looking 3D game box. It uses the parent layer image on the front of the box.

+
'side' attribute (Optional)
+

The filename of the custom image resource to be used on the side of the box.

+
'rotate' attribute (Optional)
+

Defines the rotation of the side image in degrees.

+
'sidescaling' attribute [from v2.7.1] (Optional)
+

Defines how to scale the side / spine image. If left out, it will autoscale depending on the size of the artwork resource. Can be 'width', 'height' or 'both'.

+

'hue' effect node [from v2.3.3] (Optional)

+
+

Effect example +

+
Hue effect sample
+
+
<layer>
+  <hue value="100"/>
+</layer>
+
+

Must be nested inside a layer node. Rotates the hue of the parent layer.

+
'value' attribute (Mandatory)
+

The difference value for the adjustment. Can be 0 to 359.

+

'mask' effect node (Optional)

+
+

Effect example +

+
Masking sample
+
+

Left image shows the result. Right image shows the mask used. The white part of the mask is transparent.

+
<layer>
+  <mask file="[custom image resource]" x="0" y="0" width="100" height="100"/>
+</layer>
+
+

Must be nested inside a layer node. This applies a mask to the parent layer.

+
'file' attribute (Mandatory)
+

The filename of the custom image resource to be used as mask. The alpha channel of the mask will blind out the underlying parts of the parent layer.

+
'width' attribute (Optional)
+

Sets the width of the mask in pixels. If left out it will be set to the width of the parent layer.

+
'height' attribute (Optional)
+

Sets the height of the mask in pixels. If left out it will be set to the height of the parent layer.

+
'x' attribute (Optional)
+

Sets the x coordinate of the mask relative to the parent layer. If left out it will be set to 0.

+
'y' attribute (Optional)
+

Sets the y coordinate of the mask relative to the parent layer. If left out it will be set to 0.

+

'opacity' effect node (Optional)

+
+

Effect example +

+
Opacity effect sample
+
+
<layer>
+  <opacity value="50"/>
+</layer>
+
+

Must be nested inside a layer node. Adjusts the opacity of the parent layer.

+
'value' attribute (Mandatory)
+

The opacity of the layer. Can be 0-100 where 0 is completely transparent and 100 is opaque.

+

'rotate' effect node [from v2.3.3] (Optional)

+
+

Effect example +

+
Rotation examples
+
+
<layer>
+  <rotate degrees="45" axis="y"/>
+</layer>
+
+

Must be nested inside a layer node. Rotates the parent layer around the x, y or z axis.

+
'degrees' attribute (Mandatory)
+

Sets how many degrees the parent layer will be rotated. Can be -360 to 360.

+
'axis' attribute (Optional)
+

Sets which axis the parent layer should be rotated around. Can be 'x', 'y' or 'z'. If left out it will be set to 'z'.

+

'rounded' effect node (Optional)

+
+

Effect example +

+
Corner rounding sample
+
+
<layer>
+  <rounded radius="10"/>
+</layer>
+
+

Must be nested inside a layer node. Rounds the corners of the parent layer.

+
'radius' attribute (Mandatory)
+

Defines the radius of the corners in pixels.

+

'saturation' effect node [from v2.3.3] (Optional)

+
+

Effect example +

+
Saturation effect sample
+
+
<layer>
+  <saturation value="-100"/>
+</layer>
+
+

Must be nested inside a layer node. Adjusts the color saturation of the parent layer.

+
+

Tip

+

If you want to completely greyscale your layer I highly recommend using the 'saturation' attribute of the 'colorize' effect instead. It uses a better estimation of percieved luminance.

+
+
'value' attribute (Mandatory)
+

The difference value for the adjustment. Can be -255 to 255.

+

'scanlines' effect node [from v2.9.0] (Optional)

+
+

Effect example +

+
Scanlines effect sample
+
+
<layer>
+  <scanlines file="scanlines1.png" scale="1.0" opacity="100" mode="overlay"/>
+</layer>
+
+

Must be nested inside a layer node. Adds a CRT monitor scanline effect to the parent layer.

+
'file' attribute (Optional)
+

The filename of the custom image resource to be used as the scanline overlay. If left out it defaults to scanlines1.png.

+
'scale' attribute (Optional)
+

Sets the scale factor of the overlay. Can be 0.1 to 2.0. If left out it defaults to 1.0.

+
'opacity' attribute (Optional)
+

Sets the opacity of the overlay. Can be 0 to 100. If left out it defaults to 100.

+
'mode' attribute (Optional)
+

Sets the overlay compositing mode. It can be:

+
    +
  • multiply
  • +
  • screen
  • +
  • overlay
  • +
  • darken
  • +
  • lighten
  • +
  • colordodge
  • +
  • colorburn
  • +
  • hardlight
  • +
  • softlight
  • +
  • difference
  • +
  • exclusion
  • +
  • sourcein
  • +
  • destinationin
  • +
  • sourceout
  • +
  • destinationout
  • +
  • sourceatop
  • +
  • destinationatop
  • +
  • xor
  • +
+

If left out the overlay is rendered using the overlay method.

+
+

Mode examples +

+
Layering examples (note the in-image captions)
+
+

'shadow' effect node (Optional)

+
+

Effect example +

+
Dropshadow effect sample
+
+
<layer>
+  <shadow distance="10" softness="5" opacity="50"/>
+</layer>
+
+

Must be nested inside a layer node. Renders a dropshadow on the parent layer using the attributes provided. If either of the attributes are left out, the shadow won't be drawn.

+
'distance' attribute (Mandatory)
+

Distance in pixels from the layer. The distance is always down to the right.

+
'softness' attribute (Mandatory)
+

Defines how soft (radius) the shadow will appear. A value of 0 is sharpest.

+
'opacity' attribute (Mandatory)
+

Defines the opacity of the shadow. 100 is completely visible. 0 is completely transparent.

+

'stroke' effect node (Optional)

+
+

Effect example +

+
Stroke effect sample
+
+
<layer>
+  <stroke width="5" red="10" green="20" blue="30"/>
+</layer>
+
+

Must be nested inside a layer node. Renders a colored outline on the parent layer. If all color attributes are left out, it averages a suitable color from the parent layer.

+
'width' attribute (Mandatory)
+

The width of the outline in pixels.

+
'red' attribute (Optional)
+

The red color value for the outline. Can be 0-255. If left out it is set to 0.

+
'green' attribute (Optional)
+

The green color value for the outline. Can be 0-255. If left out it is set to 0.

+
'blue' attribute (Optional)
+

The blue color value for the outline. Can be 0-255. If left out it is set to 0.

+
'color' attribute [from v2.3.1] (Optional)
+

Provides the color to use hex-style. This can be used instead of the 'red', 'green' and 'blue' attributes described above. An example could be 'color="#ff0099"'.

+

Custom image resources

+

From Skyscraper version 2.3.0 you can use custom image resources wherever the documentation says so. Place your custom resources in the '/home/<USER>/.skyscraper/resources' folder and use it by adding the filename to the attribute.

+

Example:

+
<gamebox side="megadrive_logo.png" rotate="90"/>
+
+

The file megadrive_logo.png will then be loaded from the '/home/<USER>/.skyscraper/resources/megadrive_logo.png' file. Feel free to create subfolders inside the resources folder. If you do, remember to add the partial path to the filename like so:

+
<gamebox side="platform_logos/megadrive_logo.png" rotate="90"/>
+
+

This will look for the file at '/home/<USER>/.skyscraper/resources/platform_logos/megadrive_logo.png'.

+
+

Warning

+

All custom image resources are loaded into memory when Skyscraper starts to avoid repeatedly reading them from disc / SD card. So please keep your resources to a minimum - otherwise it might exceed the total memory on the Pi which will cause Skyscraper to get killed by the kernel.

+
+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/CACHE/index.html b/CACHE/index.html new file mode 100644 index 00000000..f171594e --- /dev/null +++ b/CACHE/index.html @@ -0,0 +1,1481 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Skyscraper Cache - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+ +
+
+ + + +
+
+ + + + + + + +

Skyscraper Cache

+ +

Resource Cache

+

Whenever you scrape any platform with any scraping module, Skyscraper caches each resource internally. A resource can, for instance, be a game description or a game screenshot. Each game can have several versions of each resource cached locally. One of each type per scraping module. This comes in handy when generating the game list and artwork compositions.

+

Think of the resource cache as the cache in an internet browser. Most of the data on webpages don't change much. By caching some of the data locally, it can load parts of a webpage from that data instead of loading it from the remote server. This is exactly how the Skyscraper cache works. It helps keep the online servers healthy by not hammering them whenever you need resources you already downloaded once. And it allows you to re-generate the frontend game lists if you add new games or perhaps want to change the style of the exported artwork.

+

Default resource cache folder

+

The default base folder for all of Skyscrapers' locally cached data is in the /home/<USER>/.skyscraper/cache folder. In this folder you'll find individual platform subfolders. Any of these are selfcontained and contains all of the cached data for that particular platform.

+

Resource and scraping module priorities

+

There is ONE file that you can and should edit inside each of the /home/<USER>/.skyscraper/cache/<PLATFORM> folders. That file is called priorities.xml and decides the scraper priority of resources for each resource type. For instance, if you know that thegamesdb always provides the best descriptions for games, you'd add an <order type="description"> node with a <source>thegamesdb</source> subnode. You can have multiple <source> nodes, Skyscraper will then prefer the topmost source when generating a game list. If the topmost isn't found it'll prioritize the next one and so forth. Any source that isn't listed with an <order> node will be prioritized using timestamps (newest wins) for when each resource was added to the cache. So you don't have to add all of them.

+

Skyscraper provides the example file /home/<USER>/.skyscraper/cache/priorities.xml.example. Please don't edit this file manually, as it will be overwritten when you update Skyscraper. When a platform is scraped for the first time, it will automatically copy the example file to /home/<USER>/.skyscraper/cache/<PLATFORM>/priorities.xml unless it already exists. You can of course also copy the file yourself before scraping a platform. If you do so, be sure to remove the .example part of the filename so it's just called priorities.xml.

+

Update locally cached data

+

If you wish to update / refresh the locally cached resources for a particular platform and scraping module, Skyscraper provides the --refresh option. If this flag is set on the command line, any data in the resource cache will be updated with the new incoming data.

+

If you wish to just refresh the data for a single rom simply scrape it with -p <PLATFORM> -s <SCRAPING MODULE> <RELATIVE OR FULL ROM PATH AND FILENAME> and the locally cached data for that particular rom will be updated / refreshed. You can add more filenames one after the other if you like. If any filename or paths has spaces in it, remember to double-quote it like so "relative path/to rom/rom filename.sfc". You don't need the --refresh option when updating single files - it is implicit.

+

When you've updated information in the resource cache, always remember to re-generate the game list by simply running Skyscraper -p <PLATFORM> when you're done. The updated resources won't be visible in your frontend until you do.

+

The Resource Cache Folder and File Structures

+

Preferred way of editing the cache is via the various --cache commands.

+
+

Do not edit

+

MOST FILES AND FOLDERS INSIDE THE /home/<USER>/.skyscraper/cache FOLDER ARE NOT MEANT TO BE MANIPULATED BY HAND! It can be done, but don't complain to me about the format of the database. It is NOT meant to be understood by humans. It is meant to be efficient for reading and parsing by Skyscraper itself. Same goes for the media files that reside in the subfolders.

+
+
+

Don't leave custom files in here

+

If you decide to add your own files to the subfolders, you risk them being deleted by Skyscraper later on if it is run with one of the cache cleanup command line options. You've been warned!

+
+

Other cool stuff you CAN DO

+

Each subfolder in the /home/<USER>/.skyscraper/cache/ folder is self-contained and can be copied to other Skyscraper installations at your convenience. Just copy the folder itself over to some other computer that has Skyscraper 1.6.0 or later installed, and you can make use of the data when generating game lists. If you add it at a non-default location, set the custom folder with -d <FOLDER>.

+

Resource Cache Format

+

I do not recommend editing the db.xml resource cache files manually. But the format is simple, so you certainly can if you want to.

+

Resource id

+

The database consists of resource entries connected to a unique id. The id is calculated from the rom data or, in special cases, the filename (in cases where the file data is a script or a ROM gets changed by the gameplay). An entry can look like this:

+
<resource id="<ID KEY>" 
+        type="<RESOURCE TYPE>" 
+      source="<SCRAPING SOURCE>"
+   timestamp="<UNIX TIMESTAMP IN MSECS>">Resource data</resource>
+
+
+

Note

+

Pre-3.3.0 versions of Skyscraper used sha1 as the name of the unique id key. Later versions use id.

+
+

Resource Types

+
title
+

A game title

+
platform
+

A game platform

+
description
+

A game description

+
publisher
+

The publisher of a game

+
developer
+

The developer of a game

+
players
+

How many players are supported by a game

+
tags
+

List of game tags, most often genre related

+
releasedate
+

The release date of a game

+
ages
+

Game age rating, defined as youngest allowed age in years

+
rating
+

Game rating, real number between 0 and 1

+
cover
+

A cover image filename for a game (file exists in covers subfolder)

+
screenshot
+

A screenshot image filename for a game (file exists in screenshots subfolder)

+
wheel
+

A wheel image filename for a game (file exists in wheels subfolder)

+
marquee
+

A marquee image filename for a game (file exists in marquees subfolder)

+
texture
+

A texture image (disc/cartridge) filename for a game (file exists in textures subfolder)

+
video
+

A video file filename for a game (file exists in videos subfolder)

+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/CHANGELOG/index.html b/CHANGELOG/index.html new file mode 100644 index 00000000..ee4a5365 --- /dev/null +++ b/CHANGELOG/index.html @@ -0,0 +1,1460 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Changelog - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + + + + + +
+
+ + + + + + + +

Changelog

+ +

Changes

+

This page summarizes the changes of each Skyscraper release, a changlog for +humans.

+

Version 3.14.0 (TBA)

+
    +
  • Added: Support for XDG Base Directories, thanks for the suggestion + @ASHGOLDOFFICIAL.
  • +
  • Added: Option to allow any delimiter between consecutive brackets and + parentheses in gamelist title. See + innerBracketsReplace for examples. + Thanks for the suggestion, @retrobit.
  • +
  • Added: Platform 'Fujitsu + FM-Towns'. Manually update + your peas.json and platformid_map.csv to make use of it.
  • +
  • Added: Option --hint, it shows a random Tip of the Day.
  • +
  • Updated: Skyscraper's hardcoded /home/<USER> replaced with the actual user's + home directory screen messages. Thanks for highlighting it on the Mac, + @cdaters
  • +
  • Updated: A downloaded whdload.xml file for platform Amiga will be not + downloaded again until the server indicates. Manually removing + /home/<USER>/.skyscraper/whdload_cached_etag.txt will force a new download.
  • +
  • Fixed: Performing Ctrl-C in --cache edit mode will now dismiss any changes + made instead of persisting them
  • +
+

Version 3.13.0 (2024-11-06)

+
    +
  • Added: Option to provide user file peas_local.json (same format as + peas.json), to extend platform information or overwrite existing platform + information
  • +
  • Updated: *_platform.json files as reference for supported platforms of + various scrapers
  • +
  • Added: Support for Vircon32 platform in peas.json. Thanks for hinting, + @vircon32
  • +
  • Removed: scrapers entries in peas.json, as it did not provide any use.
  • +
  • Removed: Deprecated flags and options: includeFiles superseded by + includePattern; excludeFiles superseded by excludePattern; + gamelistFolder superseded by gameListFolder; fromfile superseded by + includefrom. These were deprecated since v3.7.0.
  • +
+

Version 3.12.0 (2024-07-01)

+
    +
  • Added: Support for scraping of PDF manuals (for scrape modules screenscraper, + import and esgamelist) and gamelist output with these manuals for frontends + (ES-DE Frontend and some EmulationStation variants). See configurations + options manuals=true and + gameListVariants=enable-manuals. Thanks for + the initial PR, @pandino
  • +
  • Added: For frontend ES-DE, evaluate environment variable ESDE_APPDATA_DIR if + present. Thanks for the hint, @ASHGOLDOFFICIAL
  • +
  • Added: Use also release year as hint on user interaction. This is useful when + in interactive mode and more than one game with the same name is found. + Skyscraper can be guided to prefer a specific game when the release year is + added in parenthesis as part of the ROM name (or alias in aliasMap.csv). + Verbose info in #59. Thanks, + @mjkaye
  • +
  • Changed: Persistent config option onlymissing for counting and scraping only + games which do not have any game data in the cache. This is a commodity config + option to the already existing flag with the same name. Plus: If you use a + scraper with a scraping limit for games to be scraped at once (e.g. MobyGames) + you may stay below that limit. Documentation here. + Thanks for the suggestion, @sleve_mcdichael
  • +
  • Update: Valid extensions (= formats in Skyscraper's peas.json file) with + info from RetroPie's platform.cfg (commit + 5e0ab1f)
  • +
  • Fixed: The update + script + for recent MacOS versions. Thanks, @calumbrodie
  • +
+

Version 3.11.0 (2024-04-15)

+
    +
  • Added: Support for EmulationStation Desktop Edition (ES-DE Frontend). Use + frontend=esde in config.ini and see + documentation on the + default settings. Thanks for the hints and for testing, @maxexcloo, @Nargash
  • +
  • Added: Entries in + aliasMap.csv + are now also applicable for Screenscraper. Thanks, @retrobit.
  • +
  • Added: Enhanced game name detection for ScummVM platform.
  • +
  • Changed: In module Screenscraper, in some cases if multiple media types are + queried (NB: most queries are single-typed), now the first type is selected + with the first matching region. Only if this does exists in any provided + region, try the next type with all provided regions (in short: type has + precedence over region). Previously the regions have been tried to match on + any media type (=region had precedence over type), which resulted in picking + the alternative type. This approach gave less suitable media files, especially + for screenshots (type:ss) and screenshot from title screen (type: + sstitle).
  • +
  • Fixed: Some corner-case bugs fixed, thanks to all reporters!
  • +
+

Version 3.10.0 (2024-02-10)

+
    +
  • Feature: Preserve existing <folder/> nodes in gamelist or create skeleton + <folder/> nodes when ROMs are stored in subfolders within a system folder, + see frontend documentation and the gamelist + specification.
  • +
  • Feature: Bash Completion on Linux + installations. Use Tab twice for + completion of Skyscraper options. On RetroPie the scriptmodule will handle + the installation. On non-RetroPie-Linux put the file + supplementary/bash-completion/Skyscraper.bash into + $XDG_DATA_HOME/bash-completion/completions/ ($XDG_DATA_HOME is + equivalent to $HOME/.local/share). Open a new bash -- et voila!
  • +
  • Feature: Customizable installation folder when running make install. See + PREFIX in skyscraper.pro.
  • +
  • Feature: Improved MacOS support. Unified update script + update_skyscraper.sh (thanks, @jeantichoc) and Docker support via Dev + Container (kudos, @retrobit).
  • +
  • Fix: Various minor fixes reported from the community on different setups, + thanks!
  • +
+

2023-12-01 (Version 3.9.2)

+
    +
  • Feature: Import of data in XML format is now more lax (does not rely on + strict identical indention). Read also the hint + here
  • +
  • Feature: Configuration option tidyDesc added. See documentation + here
  • +
  • Feature: Documentation reviewed and hosted with mkdocs for ease of access. + See here
  • +
  • Update: Added index of configuration parameters to CONFIGINI.md. Details + see here
  • +
  • Update: Refactored skyscraper.cpp class. Factorised configuration settings + into settings.cpp
  • +
  • Update: Various other refactorings to remove duplicated code
  • +
  • Fix: Quit Skyscraper when neither -p <PLATFORM> nor --cache help nor + --flags help is provided
  • +
  • Fix: Warning remediated when NULL image was applied in composer/gamebox + rendering
  • +
+

2023-10-22 (Version 3.9.1)

+
    +
  • Feature: Mobygames scraper genres limited to two most relevant genre + categories ('Basic Genre' and 'Gameplay')
  • +
  • Update: Removed legacy and unused code
  • +
  • Update: Code formatting (LLVM)
  • +
  • Fix: TGDB scraper retrieves screens from screenshot/ and as well + screenshots/ URL path as some platforms (supported since configurable + platforms) have their screenshots served from screenshots/.
  • +
+

2023-10-20 (Version 3.9.0)

+
    +
  • Feature: Mobygames scraper respects game id from mobygames.com website via + --query=<gameid>. Handy to hint to the right game information when usual + search returns false positives.
  • +
  • Feature: Scrapers which provide an web API (Screenscraper, Mobygames, The + Games DB (tgdb)) have the full platform information shipped with this release + (see <scraper>_platforms.json files). These files are used as reference.
  • +
  • Feature: Less 'aliases' maintenance needed in former platforms.json.
  • +
  • Update: Streamlined external platform configuration. File platforms.json is + replaced by peas.json (platforms, extensions/formats, aliases and scrapers). + Precise platform determination for Screenscraper, Mobygames and TGDB via + platform_idmap.csv See platforms documentation for + details.
  • +
  • Fix: Failed media download when TGDB provides PNG files instead of JPG files + and vice versa.
  • +
  • Update: Scriptmodule for this Skyscraper now official part of RetroPie.
  • +
+

2023-09-23 (Version 3.8.1.2309)

+
    +
  • Feature: OpenRetro scraper retrieves now also score/rating, if available for a + game. Precedence is to use reviews from external websites first (right + header), then the Score label (above the game details). See this + example
  • +
  • Feature: Additionally to the existing import with rating values of 0, 0.5, 1, + 1.5 ... 5 ("Star rating") it is possible to use 0.1 ... 1.0 scale for rating + in import files (EmulationStation gamelist.xml internal rating range). See + also import formats
  • +
  • Fix: Wrong score/rating calculation from Mobygames scraper / Mobygames API. + See also this + gist on how + to update your existing Skyscraper db.xml files.
  • +
  • Fix: Use of --query free-text search in OpenRetro scraper fixed. This bug + did not occur when the switch is ommitted and an Amiga WHDLoad file is provided + to Skyscraper.
  • +
  • Fix: RetroPie Scriptmodule, removed surplus boolean negation (Thanks + @s1eve-mcdichae1)
  • +
  • Fix: RetroPie Scriptmodule, fixed use of legacy option --unattend (Thanks + @windg)
  • +
  • Update: RetroPie Scriptmodule, relaxed the remove function of the scriptmodule + to not zap the Skyscraper cache. Plus various cleanups.
  • +
+

2023-09-15 (Version 3.8.0.2309)

+
    +
  • updated mamemap.csv from MAME 0.240 (Arcade).dat, fbneo.dat, + mame2003-plus.xml and cleanup of surplus device information
  • +
  • update script for make mamemap.csv does no longer rely on mame binary
  • +
  • removed discontinued *.php scripts
  • +
  • platforms.json sorted and formatted
  • +
  • updated documentation especially to reflect the supported platforms
  • +
  • scriptmodule file for RetroPie aligned to their naming convention
  • +
  • Mobygames platform information refactored from hardwired mobygames.cpp.to + mobygames.json
  • +
+

Included Pull Requests from Parent Skyscraper Repos

+

These pull requests from other repos have been merged into this fork.

+ + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/CLIHELP/index.html b/CLIHELP/index.html new file mode 100644 index 00000000..cae447cb --- /dev/null +++ b/CLIHELP/index.html @@ -0,0 +1,2862 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Command Line Options - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + +

Command Line Options

+ +

Overview

+

Look below to find a thorough description of all command line options available when using Skyscraper. These options should be used with the Skyscraper command.

+

Also consider that almost all of these options are set at a useful default (and can therefore be left out) and should only be used / changed if your use case requires it.

+

Most of the options can also be set in the /home/<USER>/.skyscraper/config.ini file thus removing the need to type them on command line all the time. Check the config.ini doc here for more info on this.

+

Programmable Completion

+

From version 3.9.3 onwards Skyscraper provides Bash Tab completion (aka +programmable completion).

+

On RetroPie the +scriptmodule +will handle the installation. For the curious: It lands in +/etc/bash_completion.d/.

+

On non RaspiOS-based RetroPie-Installments put the file from +supplementary/bash-completion/Skyscraper.bash to +$XDG_DATA_HOME/bash-completion/completions/ (respective to +$HOME/.local/share/bash-completion/completions/).

+

In either case: Open a new bash and press Tab twice to see it in action.

+

Short Options

+

The most prevalent short options you will use are most likely -s and -p. But +you will information about the other options here too.

+

-a <FILENAME>

+

Sets a non-default XML file to use when setting up the artwork compositing. By default Skyscraper uses the file /home/<USER>/.skyscraper/artwork.xml. If you provide a relative filepath it will be expanded relative to the current working directory. Consider setting this in config.ini instead.
+Read more about the artwork.xml format and customization options here.

+

Example(s)

+
Skyscraper -p snes -a "/path/to/artwork.xml"
+
+

-c <FILENAME>

+

Sets a non-default config file. By default Skyscraper uses the file /home/<USER>/.skyscraper/config.ini. A relative filepath will be prepended with the current working directory. If you provide a config file that does not exist, Skyscraper will print a warning and will continue by using built-in default values for the configuration.

+

Example(s)

+
Skyscraper -p snes -c "/path/to/config.ini"
+
+

-d <PATH>

+

Sets a non-default location for the storing and loading of cached game resources. This is what is referred to in the docs as the resource cache. By default this folder is set to /home/<USER>/.skyscraper/cache/<PLATFORM>. Don't change this unless you have a good reason to. The folder pointed to should be a folder with a Skyscraper db.xml file and its required subfolders inside of it (covers, screenshots etc.).

+
+

Note

+

If you wish to always use a certain location as base folder for your resource cache ((for instance if you want your cache to reside on a USB drive), it is strongly recommended to set this in the config.ini file instead. Read more about the relevant config.ini option here.

+
+

Example(s)

+
Skyscraper -p snes -d "/custom/cache/path"
+
+

-e <STRING>

+
+

Info

+

This option is applicable only when using the -f attractmode or the -f pegasus option.

+
+

When using -f attractmode it is required to set the emulator to be used when generating the attractmode game list. On RetroPie the emulator name is mostly the same as the platform. Consider setting this in config.ini instead.

+

It can optionally be used with -f pegasus to set the launch command used by the Pegasus game list. On RetroPie this defaults to the RetroPie launch command which works with RetroPie. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -f attractmode -e snes
+# On RetroPie
+Skyscraper -p snes -f pegasus -e "/opt/retropie/supplementary/runcommand/runcommand.sh 0 _SYS_ snes"
+
+

-f <FRONTEND>

+

Sets the frontend you wish to export a game list for. By default Skyscraper will export an EmulationStation game list, but other frontends are supported as well. If exporting for example for the attractmode frontend, please also take note of the required -e option that goes along with using the attractmode frontend. Consider setting this in config.ini instead.

+

Check all supported frontends with '--help' and read a more about the details concerning each of them here.

+

Example(s)

+
Skyscraper -p snes -f pegasus
+Skyscraper -p snes -f attractmode -e snes
+
+

-g <PATH>

+

Sets the game list export folder. By default Skyscraper exports the game list to the same directory as the rom input folder. This enables you to change that to a non-default location. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -s screenscraper -g "/your/desired/game list/export/path"
+
+

-h, --help

+

Outputs the help text for all command line options to the terminal.

+

Example(s)

+
Skyscraper --help
+Skyscraper -h
+
+

--hint

+

Displays one of Skyscrapers's 'Did you know?' tips and exits.

+

-i <PATH>

+

Sets the rom input folder. By default Skyscraper will look for roms in the /home/<user>/RetroPie/roms/<PLATFORM> folder. If your roms are located in a non-default location, you can set the input path using this option. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -i "/path/to/your/snes/roms"
+
+

-l <0-10000>

+

Sets the maximum length of returned game descriptions. This is a convenience option if you feel like game descriptions are too long. By default it is set to 2500. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -l 500
+
+

-m <0-100>

+

Some scraping modules are based on a filename or title based search. This option sets the minimum percentage any returned results need to match with in order for it to be accepted. For instance, the game Wonderboy in Monsterland might return the title Wonder Boy in Monster Land which is clearly a match. But it's not a 100% match. So it needs to be set relatively high, while still ignoring bad matches. By default it is set to 65 which has been tested to be a good middle-ground. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -s thegamesdb -m 50
+
+

-o <PATH>

+

Sets the artwork / media output folder. By default Skyscraper outputs the composited artwork files to the game list export folder + /media. This allows you to change that to a non-default location. Read more about the artwork compositing here. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -s screenscraper -o "/path/to/where/you/want/the/artwork/files"
+
+

-p <PLATFORM>

+

Sets the platform you wish to scrape. Supported platforms can be seen using the --help option described above.

+

Running the following commands will scrape from all cached resources and generate a game list and composite artwork using the recipe in /home/<USER>/.skyscraper/artwork.xml (check the artwork documentation for more info on this here.

+

Before running these commands you need to first gather some data into the cache. Please read the description of -s <MODULE> below.

+

Example(s)

+
Skyscraper -p amiga
+Skyscraper -p snes
+
+

-s <MODULE>

+

Sets which scraping module you wish to gather data from. All data scraped from any of the modules will be cached in the resource cache and can then later be used to generate a game list for your frontend. Read more about this in the -p <PLATFORM> description above.

+

To generate a game list from the resource cache, just leave out the -s option entirely.

+

Example(s)

+
Skyscraper -p amiga -s openretro
+Skyscraper -p snes -s screenscraper
+Skyscraper -p amiga -s esgamelist
+Skyscraper -p snes -s import
+
+

To read more about each scraping module, check here.

+

-t <1-8>

+

Sets the desired number of parallel threads to be run when scraping. By default it is set to 4.

+
+

Note

+

Some modules have maximum allowed threads. If you set this higher than the allowed value, it will be auto-adjusted.

+
+

Example(s)

+
Skyscraper -p snes -s thegamesdb -t 5
+
+

-u <KEY or USERID:PASSWORD>

+

Some scraping modules require a user key or a user id + password to work. Check the scraping module overview to see the specific requirements for each module here. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -s screenscraper -u <userid:password>
+
+

Long Options

+

--addext <EXTENSION>

+

If you have a rom that Skyscraper doesn't even try to gather data for, it might be because it has a file extension that isn't currently supported. This option allows you to temporarily add support for any file extension. If you feel like you are using a file extension that ought to be supported by default, please report it so it can be added in a later version of Skyscraper. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -s thegamesdb --addext *.ext
+Skyscraper -p snes -s thegamesdb --addext "*.ext1 *.ext2"
+
+

--cache

+

This is the cache master option. It contains several subcommands that allows you to manipulate the cached data for the selected platform.

+
+

Tip

+

For any of these commands you can set a non-default resource cache folder with the -d option. The folder pointed to should be a folder with a Skyscraper db.xml file and its required subfolders inside of it (covers, screenshots etc.).

+
+

Read more about the resource cache here.

+

--cache help

+

Outputs a description of all available --cache functions.

+

--cache edit[:new=<TYPE>]

+

Allows editing of any cached resources connected to your roms. The editing mode will go through each of the files in the queue one by one, allowing you to add and remove resources as needed. Any resource you add manually will be prioritized above all others.

+

You can provide one or more filenames to the end of the command line or use the --includefrom option to edit the resources for just those files. You can use the --startat and --endat options to edit a span of roms. If none of those options are used, it will edit all of the roms in the input folder one by one.

+

For efficiency, when adding a lot of resources of the same type, you can also add the optional new=<TYPE> which will make it very easy to batch insert resources of the defined type to all the files you are editing. <TYPE> can be any of the known textual resources: title, platform, releasedate, developer, publisher, players, ages, genres, rating, description.

+

Example(s)

+
Skyscraper -p snes --cache edit <FILENAME 1> <FILENAME 2>
+Skyscraper -p snes --cache edit --startat <FILENAME> --endat <FILENAME>
+Skyscraper -p snes --cache edit:new=developer --startat <FILENAME> --endat <FILENAME>
+Skyscraper -p snes --cache edit
+Skyscraper -p snes --cache edit --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_developer-20190708.txt"
+Skyscraper -p snes --cache edit:new=ages --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_ages-20190708.txt"
+
+

--cache merge:<PATH>

+

This option allows you to merge two resource caches together. It will merge the cache located at the <PATH> location into the default cache for the chosen platform. The path specified must be a path containing the db.xml file. You can also set a non-default destination to merge to with the -d option.

+

Example(s)

+
Skyscraper -p snes --cache merge:"path to/source/cache/snes"
+Skyscraper -p snes --cache merge:"path to/source/cache/snes" -d "/path to/nondefault/destination/cache/snes"
+
+

--cache purge:<KEYWORD|MODULE and/or TYPE>

+

This is a powerful option that allows you to purge the requested resources from the resource cache connected to the selected platform.

+

You can purge all resources from the cache for the chosen platform using the keyword all.

+

You can purge specific resources from a certain module with m=<MODULE> or of a certain type with t=<TYPE> or a combination of the two separated by a ,.

+

Supported modules can be seen under -s when using the --help option. Supported types are: title, platform, description, publisher, developer, ages, tags, rating, releasedate, cover, screenshot, wheel, marquee, video.

+
+

Possible dangerous command

+

Purging anything from the cache cannot be undone, so please consider making a backup.

+
+

Example(s)

+
Skyscraper -p snes --cache purge:all
+Skyscraper -p snes --cache purge:m=thegamesdb
+Skyscraper -p snes --cache purge:t=cover
+Skyscraper -p snes --cache purge:m=thegamesdb,t=cover
+
+

--cache refresh

+

Same as --refresh.

+

Example(s)

+
Skyscraper -p snes -s screenscraper --cache refresh
+
+

--cache report:missing=<all, textual, artwork, media or RESOURCE1,RESOURCE2,...>

+

Will create report(s) containing all filenames of games missing the selected resource type(s). File(s) will be exported to /home/<USER>/.skyscraper/reports/report-<PLATFORM>-missing_<RESOURCE>-yyyymmdd.txt

+

You can use any of the following:

+
    +
  • all: Creates reports for all resource types
  • +
  • textual: Creates reports for all textual resource types
  • +
  • artwork: Creates reports for all artwork related resource types excluding 'video'
  • +
  • media: Creates reports for all media resource types including 'video'
  • +
  • type1,type2,type3,...:
  • +
+

Supported resource types are: title, platform, description, publisher, developer, ages, tags, rating, releasedate, cover, screenshot, wheel, marquee, video.

+
+

Tip

+

The reports can be fed back into Skyscraper using the --includefrom <REPORTFILE> option, which tells Skyscraper to only work on the files contained in the report. This is useful in combination with, for instance, the --cache edit option or the --cache refresh/--refresh (they are the same) option(s).

+
+

Example(s)

+
Skyscraper -p snes --cache report:missing=textual
+Skyscraper -p snes --cache report:missing=artwork
+Skyscraper -p snes --cache report:missing=developer
+Skyscraper -p snes --cache report:missing=publisher,screenshot
+
+

--cache show

+

Shows the cache stats for the chosen platform. It will list how many resources of each type you currently have cached for each scraping module.

+

Example(s)

+
Skyscraper -p snes --cache show
+
+

--cache vacuum

+

You can purge all resources that don't have any connection to your current romset for the selected platform by using the vacuum command. This is extremely useful if you've removed a bunch of roms from your collection and you wish to purge any cached data you don't need anymore.

+
+

Possible dangerous command

+

Vacuuming the cache cannot be undone, so please consider making a backup.

+
+

Example(s)

+
Skyscraper -p snes --cache vacuum
+
+

--cache validate

+

This will test the integrity of the resource cache connected to the chosen platform. It will remove / clean out any stray files that aren't connected to an entry in the cache and vice versa. It's not really necessary to use this option unless you have manually deleted any of the cached files or entries in the db.xml file connected to the platform.

+
+

Note

+

This option doesn't clean up your game list media folders. You will need to do that yourself since Skyscraper has no idea what files you might keep in those folders. This option only relates to the resource cache database and related files.

+
+

Example(s)

+
Skyscraper -p snes --cache validate
+
+

--endat <FILENAME>

+

If you wish to work on a subset of your roms you can use this option to set the rom to end at. Use it in conjunction with the --startat option described above to further narrow the subset of files.

+
+

Note

+

Enabling this option automatically sets the --refresh option and enables the nosubdirs flag.

+
+
+

Tip

+

Instead of using this option, if you just want to gather resources for one or two roms, you can provide the filename(s) directly on the command like so: $ Skyscraper -p snes -s thegamesdb "/full/or/partial/path/to/rom.zip". You can also use the --includefrom option.

+
+

Example(s)

+
Skyscraper -p snes --cache edit --endat "rom name.zip"
+Skyscraper -p snes -s thegamesdb --endat "partial/path/to/rom name.zip"
+
+

--excludefrom <FILENAME>

+

Tells Skyscraper to exclude the files listed in FILENAME. One filename per line (with FULL path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip').

+

This file can be generated with the --cache report:missing option or made manually.

+
+

Tip

+

You might also want to check out the excludepattern option.

+
+

Example(s)

+
Skyscraper -p snes -s screenscraper --excludefrom "/home/pi/.skyscraper/excludes.txt"
+
+

--excludepattern <PATTERN 1, PATTERN 2>

+

Per platform Skyscraper have default file extensions that it will accept. This option allows you to exclude certain files within that scope. The pattern is a simple asterisk type pattern. You can add several patterns by separating them with ','. In cases where you need to match for a comma you need to escape it as '\,' (see last example).

+
+

Info

+

Remember to double-quote the pattern as seen in the examples to avoid odd behaviour.

+
+
+

Tip

+
    +
  1. You might also want to check out the file extension options.
  2. +
  3. You might also want to check out the --excludefrom option.
  4. +
  5. If you create a file named .skyscraperignore within any subfolder of the input dir, all files from that directory will be ignored by Skyscraper.
  6. +
+
+

Example(s)

+
Skyscraper -p snes -s screenscraper --excludepattern "*[BIOS]*"
+Skyscraper -p amiga --excludepattern "AGA*"
+Skyscraper -p amiga --excludepattern "*AGA*,Super*"
+Skyscraper -p amiga --excludepattern "*AGA*,Super*,*\, The"
+
+

--flags

+

From Skyscraper 3.5.0 all command-line options that change the scraping behaviour have been combined into this option. Check below for a complete list of all the available flags and what they do. You can also get this list by using --flags help.

+

To enable multiple flags separate them by commas (eg. --flags FLAG1,FLAG2) or apply --flags option multiple times.

+

forcefilename

+

This flag forces Skyscraper to use the filename (excluding extension) instead of the cached titles when generating a game list. Consider setting this in config.ini instead.

+
+

Note

+

If nameTemplate is set in config.ini this flag is ignored.

+
+

interactive

+

When gathering data from any of the scraping modules many potential entries will be returned. Normally Skyscraper chooses the best entry for you. But should you wish to choose the best entry yourself, you can enable this flag. Skyscraper will then list the returned entries and let you choose which one is the best one.

+

manuals

+

By default Skyscraper doesn't scrape and cache game manuals resources because not all scraping sites provide this data and also only some frontends support PDF display of these game manuals. You can enable it by using this flag. Consider setting this in config.ini instead.

+

nobrackets

+

Use this flag to disable any bracket notes when generating the game list. It will disable notes such as (Europe) and [AGA] completely. This flag is only relevant when generating the game list. It makes no difference when gathering data into the resource cache. Consider setting this in config.ini instead.

+
+

Note

+

If nameTemplate is set in config.ini this flag is ignored.

+
+

nocovers

+

Disables the caching of the resource type cover when scraping with any module. If you never use covers in your artwork configuration, this flag can save you some space. Consider setting this in config.ini instead.

+

nocropblack

+

Disables cropping away the black borders around screenshot resources when compositing the final frontend artwork.

+

nohints

+

Disables the "Did you know" hints when running Skyscraper. Consider setting this in config.ini instead.

+

nomarquees

+

Disables the caching of the resource type marquee when scraping with any module. If you never use marquees in your artwork configuration, this flag can save you some space. Consider setting this in config.ini instead.

+

noresize

+

By default, to save space, Skyscraper resizes large pieces of artwork before adding them to the resource cache. Adding this flag will disable this and save the artwork files exactly as they are retrieved from the scraping module. Consider setting this in config.ini instead.

+
+

Note

+

This is not related to the artwork compositing that happens when generating a game list. This is only related to how Skyscraper handles artwork when adding it to the resource cache while gathering data from the scraping modules.

+
+

noscreenshots

+

Disables the caching of the resource type screenshot when scraping with any module. If you never use screenshots in your artwork configuration, this flag can save you some space. Consider setting this in config.ini instead.

+

nosubdirs

+

By default Skyscraper will include roms located in subfolders. By adding this flag Skyscraper will only scrape the roms located directly in the input folder. See -i <PATH> above to read more about the rom input folder. Consider setting this in config.ini instead.

+

notextures

+

Disables the caching of the resource type texture when scraping with any module. If you never use textures (disc/cartridge) in your artwork configuration, this flag can save you some space. Consider setting this in config.ini instead.

+

notidydesc

+

Disables clean up some misformatting in scraped description:

+
    +
  1. Heading and trailing spaces are stripped
  2. +
  3. Multiple spaces between sentences are reduced to one space
  4. +
  5. Bulletpoint beginning with * or ● are replaced with a dash
  6. +
  7. Stylized ellipsis (… Unicode:&#8230;) is replace with three dots
  8. +
  9. Multiple exclamation marks are reduced to one, unless for games titles are explicitly typed like that, like 'Super Punch-Out!!'.
  10. +
+

Consider setting this in config.ini instead.

+

nowheels

+

Disables the caching of the resource type wheel when scraping with any module. If you never use wheels in your artwork configuration, this flag can save you some space. Consider setting this in config.ini instead.

+

onlymissing

+

This flag tells Skyscraper to skip all files which already have any piece of data from any source in the cache. This is useful if you just scraped almost all files from a platform succesfully with one source, and then want to only scrape the remaining games with a different source to fill in the holes. Normally Skyscraper will scrape all files again with the second source.

+

pretend

+

This flag is only relevant when generating a game list (by leaving out the -s <MODULE> option). It disables the game list generator and artwork compositor and only outputs the results of the potential game list generation to the terminal. It can be very useful to check exactly what and how the data will be combined from the resource cache.

+

relative

+

Only relevant when generating an EmulationStation, a Retrobat or a Pegasus game list, with the -f option. Emulationstation is the default frontend when the -f option is left out. This forces the rom and any media paths (if they are the same as the input folder) inside the game list to be relative to the rom input folder. Consider setting this in config.ini instead.

+

skipexistingcovers

+

When generating gamelists, skip processing covers that already exist in the media output folder.

+

skipexistingmanuals

+

When generating gamelists, skip copying manuals that already exist in the media output folder.

+

skipexistingmarquees

+

When generating gamelists, skip processing marquees that already exist in the media output folder.

+

skipexistingscreenshots

+

When generating gamelists, skip processing screenshots that already exist in the media output folder.

+

skipexistingvideos

+

When generating gamelists, skip copying videos that already exist in the media output folder.

+

skipexistingwheels

+

When generating gamelists, skip processing wheels that already exist in the media output folder.

+

skipped

+

If a rom has no resources attached to it in the cache, it will be left out when generating a game list file. It will still show up in the frontend (at least it does for EmulationStation) but it won't exist in the game list file. You can safely leave out this flag unless you need the empty entries for some reason. Consider setting this in config.ini instead.

+ +

Enabling this flag is currently only relevant while also using the videos flag. It basically means that Skyscraper will create a link to the cached videos instead of copying them when generating the game list media files. This will save a lot of space, but has the caveat that if you somehow remove the videos from the cache, the links will be broken and the videos then won't show anymore. Consider setting this in config.ini instead.

+

theinfront

+

Game titles are returned from the scraping sources sometimes as 'The Game' and other times as 'Game, The'. Enabling this flag will force Skyscraper to always try and move 'The' to the front of the titles. If it is not enabled, Skyscraper will always try and move it to the end of the title, regardless of how it was originally returned by the scraping sources.

+
+

Note

+

When generating gamelists Skyscraper will still sort the games as if the game titles didn't have 'The' at the beginning.

+
+

unattend

+

When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By using this flag Skyscraper will always overwrite an existing game list and never skip existing entries. This is useful when scripting Skyscraper to avoid the need for user input. Consider setting this in config.ini instead.

+

unattendskip

+

When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By using this flag Skyscraper will always overwrite an existing game list and always skip existing entries. This is useful when scripting Skyscraper to avoid the need for user input. Consider setting this in config.ini instead.

+

unpack

+

Some scraping modules use file checksums to identify the game in their databases. If you've compressed your roms to zip or 7z files yourself, this can pose a problem in getting a good result. You can then try to use this flag. Doing so will extract the rom and do the file checksum on the rom itself instead of the compressed file.

+
+

Info

+

Only use this flag if you are having problems getting the roms identified from the compressed files. It slows down the scraping process significantly and should therefore be avoided if possible.

+
+

videos

+

By default Skyscraper doesn't scrape and cache video resources because of the significant disk space required to save them. You can enable videos using this flag. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p amiga --flags forcefilename,nosubdirs,skipexistingwheels
+Skyscraper -p nes --flags videos,nomarquees
+
+

--includefrom <FILENAME>

+

Tells Skyscraper to only include the files listed in FILENAME. One filename per line (with absolute path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip'). +The option is the equivalent to adding a bunch of filenames to work on directly on the commandline. It reads one line at a time from <FILENAME> and adds them to the queue of files to work on. This is very useful in combination with the --cache edit option or if you want to refresh data for just those files using -s <SCRAPING MODULE>.

+

This file can be generated with the '--cache report:missing' option or made manually.

+
+

Tip

+

You might also want to check out the includepattern option.

+
+

Example(s)

+
Skyscraper -p snes -s screenscraper --includefrom "/home/pi/.skyscraper/includes.txt"
+# After running --cache report:missing
+Skyscraper -p snes --cache edit --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_developer-20190708.txt"
+Skyscraper -p snes -s screenscraper --includefrom "/home/pi/.skyscraper/reports/report-snes-missing_developer-20190708.txt"
+
+

--includepattern <PATTERN 1,PATTERN 2>

+

Per platform Skyscraper have default file extensions that it will accept. This option allows you to only include certain files within that scope. The pattern is a simple asterisk type pattern. You can add several patterns by separating them with ','. In cases where you need to match for a comma you need to escape it as '\,' (see last example).

+
+

Info

+

Remember to double-quote the pattern as seen in the examples to avoid odd behaviour.

+
+
+

Tip

+

You might also want to check out the file extension options.

+
+

Example(s)

+
Skyscraper -p snes -s screenscraper --includepattern "Super*"
+Skyscraper -p amiga --includepattern "*AGA*"
+Skyscraper -p amiga --includepattern "*AGA*,Super*"
+Skyscraper -p amiga --includepattern "*AGA*,Super*,*\, The"
+
+

--lang <CODE>

+

Adds the specified language to the top of the existing default internal language priority list. Read more about languages here. Only one language is supported with this configuration. For a permanent setup you should consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -s screenscraper --lang es
+
+

--maxfails <1-200>

+

Not all scraping modules support all platforms. This means that you can potentially start a scraping run with a module and a platform that is incompatible. This will hammer the servers for potentially hundreds of roms but provide 0 results for any of them. To avoid this Skyscraper has a builtin limit for initially allowed failed rom lookups. If this is reached it will quit. Setting this option allows you to set this limit yourself, but not above a maximum of 200. The default limit is 42. Don't change this unless you have a very good reason to do so.

+

Example(s)

+
Skyscraper -p snes -s thegamesdb --maxfails 75
+
+

--query <STRING>

+

For most modules a search query is sent to the scraping module in an URL format. This means that a filename such as "Rick Dangerous.lha" becomes "rick+dangerous". The '+' here means a space. You could probably also use the URL encoded space "rick%20dangerous" but my tests show that most modules expect spaces as '+'. And it is the "rick+dangerous" part that you, as the user, can pass as the query, like so:

+
$ Skyscraper -p <PLATFORM> -s <MODULE> --query "rick+dangerous" <FILENAME>
+
+

Remember to also add a filename that you wish to use the override with. Otherwise the query will be ignored.

+

But not all of the scraping modules are search name based. For instance, the screenscraper module can use a variety of different search methods. So for screenscraper you also have the option of overriding the checksums it uses to search for a game. This is especially convenient in cases where a filename exists multiple times in their database and your own local file doesn't match with any of the connected checksums (maybe you've compressed the rom yourself). In this case you can look up one of the working checksums on the Screenscraper website (screenscraper.fr) and override the checksum.

+

You can use any combination of crc=<CHECKSUM>, md5=<CHECKSUM>, sha1=<CHECKSUM> and romnom=<FILENAME> (without the < and >! Also "romnom" is "rom name" in French - Screenscraper is operated from France). Most times you only need one of these, but you can combine them by separating them with a &.

+

The Mobygames scraper supports the romname directly in the --query parameter, but also accepts the game id from the mobygames site which you can find when manually looking up a game in the "Identifiers" section of the game details page.

+

The --query option is an experts only option, but it's very useful to get results for those last difficult roms.

+

Example(s)

+
$ Skyscraper -p snes -s thegamesdb --query "rick+dangerous" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p snes -s screenscraper --query "md5=<CHECKSUM>" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p snes -s screenscraper --query "romnom=file%20name.zip" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p snes -s screenscraper --query "sha1=<CHECKSUM>&romnom=file%20name.zip" /full/or/partial/path/to/rom.zip
+$ Skyscraper -p ports -s mobygames --query "14576" "/home/pi/RetroPie/roms/ports/Head over Heels.sh"
+
+

--refresh

+

Skyscraper has a resource cache which works just like the browser cache in Firefox. If you scrape and gather resources for a platform with the same scraping module twice, it will grab the data from the cache instead of hammering the online servers again. This has the advantage in the case where you scrape a rom set twice, only the roms that weren't recognized the first time around will be fetched from the online servers. Everything else will be loaded from the cache.

+

You can force all data to be refetched from the servers by setting this option, effectively updating the cached data with new data from the source.

+
+

Note

+

Only use this option if you know data has changed for several roms at the source. Otherwise you are hammering the servers for no reason.

+
+

Example(s)

+
Skyscraper -p snes -s screenscraper --refresh
+
+

--region <CODE>

+

Adds the specified region to the top of the existing default internal region priority list. Only one region is supported with this configuration. For a permanent setup you should consider setting this in config.ini instead.

+

Read more about how regions are handled in general here.

+
+

Info

+

Setting this will overwrite any region auto-detected from the file name.

+
+

Example(s)

+
Skyscraper -p snes -s screenscraper --region jp
+
+

--startat <FILENAME>

+

If you wish to work on a subset of your roms you can use this option to set the starting rom. Use it in conjunction with the --endat option described below to further narrow the subset of files.

+
+

Note

+

Enabling this option automatically sets the --refresh option and enables the nosubdirs flag.

+
+
+

Tip

+

Instead of using this option, if you just want to gather resources for one or two roms, you can provide the filename(s) directly on the command like so: $ Skyscraper -p snes -s thegamesdb "/full/or/partial/path/to/rom.zip". You can also use the --includefrom option.

+
+

Example(s)

+
Skyscraper -p snes --cache edit --startat "rom name.zip"
+Skyscraper -p snes -s thegamesdb --startat "partial/path/to/rom name.zip"
+
+

--verbosity <0-3>

+

Sets how verbose Skyscraper should be when running. Default level is 0. The higher the value, the more info Skyscraper will output to the terminal while running. Consider setting this in config.ini instead.

+

Example(s)

+
Skyscraper -p snes -s screenscraper --verbosity 3
+
+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/CONFIGINI/index.html b/CONFIGINI/index.html new file mode 100644 index 00000000..1e0bd7d0 --- /dev/null +++ b/CONFIGINI/index.html @@ -0,0 +1,3384 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + The config.ini File - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + +

The config.ini File

+ +

config.ini Options

+

Look below to find a thorough description of all options and sections available in the /home/<USER>/.skyscraper/config.ini configuration file. These options will then be applied whenever running Skyscraper with the Skyscraper command.

+

Please take note that almost all of these options are set at a useful default as documented here (and can therefore be left out) and should only be set if your use case requires it.

+
+

Warning

+

Some users seem to have a habbit of editing the /home/<USER>/skysource/config.ini file. This is not the correct file! Please read the first paragraph. :)

+
+

Options can be set on four levels; either [main], [<PLATFORM>], [<FRONTEND>] or [<SCRAPER>].

+

Settings in the [main] section will always be set regardless of selected platform, frontend and module.

+

<PLATFORM> can be any of the supported platforms (check list with --help under the -p option), in which case the settings will only be applied while scraping that particular platform.

+

<FRONTEND> can be any of the supported frontends (check list with --help under the -f option), in which case the settings will only be applied while scraping for that particular frontend.

+

<SCRAPER> can be any of the supported scraping modules (check list with --help under the -s option), in which case the settings will only be applied while scraping with that particular scraping module.

+

Each section can have overlapping parameters. In case where a certain option exists in several sections they are prioritized as scraping module first, then frontend, then platform and lastly main. Any commandline option which relates to an configuration setting in config.ini has highest precedence, regardless of the other four levels respective sections.

+

You can find an example config file at /home/<USER>/.skyscraper/config.ini.example. This file contains all available options. Just copy the file to config.ini and uncomment and edit the ones you wish to use by removing the # or ; in front of the variables. Remember to also uncomment the section the option relates to such as [main] or [amiga].

+
+

Note

+

You can set a custom configuration file with the -c <FILENAME> command line option. Read more about all available command line options here.

+
+

Example config.ini:

+
[main]
+inputFolder="/path/to/your/roms"
+cacheFolder="/path/to/your/desired/db/cache/base/folder"
+gameListBackup="true"
+videos="true"
+unattend="true"
+verbosity="1"
+lang="ja"
+region="jp"
+cacheMarquees="false"
+
+[amiga]
+inputFolder="/specific/path/just/for/amiga/roms"
+cacheFolder="/specific/path/just/for/amiga/cache"
+minMatch="50"
+maxLength="200"
+
+[pegasus]
+artworkXml="artwork-pegasus.xml"
+maxLength="10000"
+
+[screenscraper]
+userCreds="USER:PASSWORD"
+
+

Index of Options with Applicable Sections

+

This is an alphabetical index of all configuration options including the sections they may be applied to.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Parameter[main][<PLATFORM>][<FRONTEND>][<SCRAPER>]
addExtensionsYY
addFoldersY
artworkXmlYYY
bracketsYYY
cacheCoversYYY
cacheFolderYY
cacheMarqueesYYY
cacheRefreshYY
cacheResizeYYY
cacheScreenshotsYYY
cacheTexturesYYY
cacheWheelsYYY
cropBlackYYY
emulatorYYY
endAtYY
excludeFromYY
excludePatternYYY
extensionsY
forceFilenameYYY
frontendY
gameListBackupYY
gameListFolderYYY
gameListVariantsY
hintsY
importFolderYY
includeFromYY
includePatternYYY
innerBracketsReplaceY
innerParenthesesReplaceY
inputFolderYY
interactiveYYY
jpgQualityYYY
langYY
langPriosYY
launchYYY
manualsYY
maxFailsY
maxLengthYYYY
mediaFolderYYY
mediaFolderHiddenY
minMatchYYY
nameTemplateYY
onlyMissingYYY
platformY
pretendYY
regionYY
regionPriosYY
relativePathsYY
scummIniY
skippedYYY
spaceCheckY
startAtYY
subdirsYY
symlinkYYY
theInFrontYYY
threadsYYY
tidyDescYYY
unattendYYYY
unattendSkipYYYY
unpackYY
userCredsY
verbosityYYY
videoConvertCommandYY
videoConvertExtensionYY
videoPreferNormalizedY
videosYYYY
videoSizeLimitYYY
+

inputFolder

+

Sets the rom input folder. By default Skyscraper will look for roms in the /home/<USER>/RetroPie/roms/<PLATFORM> folder. If your roms are located in a non-default location, you can set the input path using this option.

+
+

Note

+

If this is set in the [main] section it will automatically add /<PLATFORM> to the end of the path. If you want better control consider adding it to a [<PLATFORM>] section instead where it will be used as is.

+
+

Default value: /home/<USER>/RetroPie/roms/<PLATFORM>
+Allowed in sections: [main], [<PLATFORM>]

+
+

gameListFolder

+

Sets the game list export folder. By default Skyscraper exports the game list to the same directory as the rom input folder. This enables you to change that to a non-default location.

+
+

Note

+

If this is set in the [main] or [<FRONTEND>] section it will automatically add /<PLATFORM> to the end of the path. If you want better control consider adding it to a [<PLATFORM>] section instead where it will be used as is.

+
+

Default value: /home/<USER>/RetroPie/roms/<PLATFORM>
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

gameListBackup

+

When set to true Skyscraper will create a backup of the existing game list each time it is run in game list generation mode (this mode is enabled by leaving out the -s MODULE option). The filename and path of the backup will be equal to the existing game list but have a timestamp appended to the name of the format -yyyyMMdd-hhmmss (eg. gamelist.xml-20200530-115900).

+

Default value: false
+Allowed in sections: [main], [<FRONTEND>]

+
+

mediaFolder

+

Sets the artwork / media output folder. By default (ie. if the option mediaFolder is not explicitly set or pre-defined by the frontend) Skyscraper outputs the composited artwork files to the game list export folder concatenated with /media. The mediaFolder setting enables you to change that to a non-default location.

+

Read more about the artwork compositing here.

+
+

Note

+

If this is set in the [main] or [<FRONTEND>] section it will automatically add /<PLATFORM> to the end of the path. If you want better control consider adding it to a [<PLAFORM>] section instead where it will be used as is.

+
+

Default value: /home/<USER>/RetroPie/roms/<PLATFORM>/media
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

mediaFolderHidden

+

By default Skyscraper uses the <PLATFORM>/media folder when generating EmulationStation media. Setting this option will change that to <PLATFORM>/.media. Notice the . which hides the folder, which can speed up the initial EmulationStation loading sequence when using slow storage such as network file systems.

+

This option is ignored if you set the media folder manually.

+
+

Note

+

Remember to remove your old <PLATFORM>/media folders if you enable this option.

+
+

Default value: false
+Allowed in sections: Only for frontends [emulationstation] or [retrobat]

+
+

cacheFolder

+

Sets a non-default location for the storing and loading of cached game resources. This is what is referred to in the docs as the resource cache. By default this folder is set to /home/<USER>/.skyscraper/cache/<PLATFORM>. Don't change this unless you have a good reason to (for instance if you want your cache to reside on a USB drive).

+
+

Note

+

If this is set in the [main] section (recommended) it will automatically add /<PLATFORM> to the end of the path. If you want better control consider adding it to a [<PLAFORM>] section instead where it will be used as is.

+
+

Default value: /home/<USER>/.skyscraper/cache/<PLATFORM>
+Allowed in sections: [main], [<PLATFORM>]

+
+

cacheResize

+

By default, to save space, Skyscraper resizes large pieces of artwork before adding them to the resource cache. Setting this option to "false" will disable this and save the artwork files with their original resolution. Beware that Skyscraper converts all artwork resources to lossless PNG's when saving them. High resolution images such as covers will take up a lot of space! So set this to "false" with caution.

+
+

Note

+

This is not related to the artwork compositing that happens when generating a game list. This is only related to how Skyscraper handles artwork when adding it to the resource cache while gathering data from the scraping modules.

+
+

To read about artwork compositing go here instead.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

cacheRefresh

+

Skyscraper has a resource cache which works just like the browser cache in Firefox. If you scrape and gather resources for a platform with the same scraping module twice, it will grab the data from the cache instead of hammering the online servers again. This has the advantage in the case where you scrape a rom set twice, only the roms that weren't recognized the first time around will be fetched from the online servers. Everything else will be loaded from the cache.

+

You can force all data to be refetched from the servers by setting this option to cacheRefresh="true", effectively updating the cached data with new data from the source.

+
+

Warning

+

Only set this option to true if you know data has changed for several roms at the source. Otherwise you are hammering the servers for no reason.

+
+

Default value: false
+Allowed in sections: [main], [<SCRAPER>]

+
+

nameTemplate

+

By default Skyscraper uses just the title as the game name when generating gamelists for any frontend. You can change this to suit your personal preference by setting this option. It works by replacing certain tokens in the name template with the corresponding data. The valid tokens are:

+
    +
  • %t: The game title as returned by the scraping sources without bracket information (see %b and %B below)
  • +
  • %f: The game filename without extension and bracket information (see %b and %B below)
  • +
  • %b: The game () parentheses information. This information often comes from the filename, but can also come from the scraping source title (eg. (USA) or (en,fr,de))
  • +
  • %B: The game [] bracket information. This information often comes from the filename, but can also come from the scraping source title (eg. [disk 1 of 2] or [AGA])
  • +
  • %a: The age restriction as returned by the scraping sources (eg. 16+)
  • +
  • %d: The game developer as returned by the scraping sources
  • +
  • %p: The game publisher as returned by the scraping sources
  • +
  • %r: The game rating as returned by the scraping sources (0.0 to 5.0)
  • +
  • %P: The number of players as returned by the scraping sources
  • +
  • %D: The game release date with format yyyy-mm-dd
  • +
+

This option also support template groups separated by ; within the template. The template parser will go over each group. If a group only has empty variables it will not be included in the final game name. So for a template such as %t;, %P player(s) where the %P is empty because no scraping source has provided the info, it will leave out the , %P player(s) part entirely resulting in the title Game Name. If this was not separated by ; the resulting game name would end up being Game Name, player(s).

+

Example(s)

+
nameTemplate="%t [%f];, %P player(s)"
+
+

Will result in: 1945k III [1945kiii], 2 player(s)

+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>]

+
+

tidyDesc

+

Cleans up some misformatting in scraped description:

+
    +
  1. Heading and trailing spaces are stripped
  2. +
  3. Multiple spaces between sentences are reduced to one space
  4. +
  5. Bulletpoint beginning with * or ● are replaced with a dash
  6. +
  7. Stylized ellipsis (… Unicode:&#8230;) is replaced with three dot characters
  8. +
  9. Multiple exclamation marks are reduced to one, unless for game titles are explicitly typed like that, like 'Super Punch-Out!!'.
  10. +
+
+

Quote

+

'Multiple exclamation marks,' he went on, shaking his head, 'are a sure sign of a diseased mind.'
+-- Eric by Terry Pratchett

+
+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

jpgQuality

+

Sets the default jpg quality when saving image resources to the cache. This will be ignored if --flags noresize is set. Default is 95.

+

All screenshots and any image resource that uses transparency will always be saved as PNG images. In those cases this setting will be ignored as PNG format is lossless.

+

Default value: 95
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

cacheCovers

+

Enables/disables the caching of the resource type cover when scraping with any module. If you never use covers in your artwork configuration, setting this to "false" can save you some space.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

cacheTextures

+

Enables/disables the caching of the resource type cover when scraping with any module. If you never use covers in your artwork configuration, setting this to "false" can save you some space.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

cacheScreenshots

+

Enables/disables the caching of the resource type screenshot when scraping with any module. If you never use screenshots in your artwork configuration, setting this to "false" can save you some space.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

cropBlack

+

Enables/disables cropping away the black borders around the screenshot resources when compositing the final frontend gamelist artwork.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

cacheWheels

+

Enables/disables the caching of the resource type wheel when scraping with any module. If you never use wheels in your artwork configuration, setting this to "false" can save you some space.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

cacheMarquees

+

Enables/disables the caching of the resource type marquee when scraping with any module. If you never use marquees in your artwork configuration, setting this to "false" can save you some space.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

importFolder

+

Sets a non-default folder when scraping using the -s import module. By default this is set to /home/<USER>/.skyscraper/import and will also look for a /<PLATFORM> inside of the chosen folder. Read more about the -s import module here.

+

Default value: /home/<USER>/.skyscraper/import/<PLATFORM>
+Allowed in sections: [main], [<PLATFORM>]

+
+

unpack

+

Some scraping modules use file checksums to identify the game in their databases. If you've compressed your roms to zip or 7z files yourself, this can pose a problem in getting a good result. You can then try setting this option to "true". Doing so will extract the rom and do the file checksum on the rom itself instead of the compressed file.

+
+

Info

+

Only enable this option if you are having problems getting the roms identified from the compressed files. It slows down the scraping process significantly and should therefore be avoided if possible.

+
+

Default value: false
+Allowed in sections: [main], [<PLATFORM>]

+
+

frontend

+

Sets the frontend you wish to export a game list for. By default Skyscraper will export an EmulationStation game list, but other frontends are supported as well.

+

If exporting for the attractmode frontend, please also take note of the required emulator="" option that goes along with using the attractmode frontend.

+

If exporting for the pegasus frontend, please also take note of the optional launch="" option that allows you to set the launch parameter used by Pegasus.

+

Use as value for the frontend option the frontend name in all lowercase and with alphabetical characters only: emulationstation, esde, pegasus, retrobat, attractmode, aso. Check all supported frontends with '--help' and read a more about the details concerning each of them here.

+

Default value: emulationstation
+Allowed in sections: [main]

+
+

emulator

+

This option is only applicable when also setting the frontend="attractmode" option. It sets the emulator to be used when generating the attractmode game list. On RetroPie the emulator name is mostly the same as the platform.

+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

launch

+

This option is only applicable when also setting the frontend="pegasus" option. It sets the launch command to be used when generating the pegasus game list. This is optional. It defaults to the RetroPie launch options which works on RetroPie.

+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

videos

+

By default Skyscraper doesn't scrape and cache video resources because of the significant disk space required to save them. You can enable videos using this option.

+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>], [<SCRAPER>]

+
+

videoSizeLimit

+

If video scraping is enabled you can set the maximum allowed video file size with this variable. The size is in Megabytes (1.000.000 bytes). If this size is exceeded the video file won't be saved to the cache.

+

Default value: 100
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

videoConvertCommand

+

Some scraping modules deliver videos that use a codec or color format that some frontends don't support. In those cases it can be useful to convert the videos before saving them in the Skyscraper resource cache.

+

This setting allows you to set a command that will be run on each video after it has been downloaded from the selected scraping module. See the examples below for inspiration.

+

The %i and %o are required and will be replaced with the internally used video input (original) and output (converted) filenames as needed by Skyscraper.

+

If your command / script always converts to a videofile with a specific extension, you also need to set videoConvertExtension.

+
+

Tip

+

Set --verbosity 3 to route all output from your command / script to the terminal while Skyscraper runs. This will help you ensure everything is working as intended.

+
+

The first example below makes use of the excellent ffmpeg tool. If you want to use this specific example you need to install ffmpeg first. On RetroPie and other Debian-derived distros you can install it with sudo apt install ffmpeg. If you want to use a script for the video conversion and run it directly without path, you need to place it in the /home/<USER>/.skyscraper folder.

+
+

Tip

+

If you create a script that includes checks on the input video and it sometimes decides not to convert them, you should simply let the script copy the video with cp %i %o. The %i is a temporary file created internally by Skyscraper. And the %o file is the file used by the cache. Both files should exist after a successful conversion. Skyscraper removes the temporary %i file automatically.

+
+

Example(s)

+
videoConvertCommand="ffmpeg -i %i -y -pix_fmt yuv420p -t 00:00:10 -c:v libx264 -crf 23 -c:a aac -b:a 64k -vf scale=640:480:force_original_aspect_ratio=decrease,pad=640:480:(ow-iw)/2:(oh-ih)/2,setsar=1 %o"
+videoConvertCommand="videoconvert.sh %i %o"
+
+

Default value: unset
+Allowed in sections: [main], [<SCRAPER>]

+
+

videoConvertExtension

+

If you want to force an extension for the converted video file created by the videoConvertCommand command, you need to set this option. The converted file will then automatically have this extension no matter what the input file extension is.

+

For instance, if a scraping module delivers the file videofile.avi and you always want the converted files to be mp4 files, you simply set this option to videoConvertExtension="mp4".

+

It is up to you to make sure that the command you provide in videoConvertCommand actually converts to a video file using the selected extension. For some conversion tools such as ffmpeg this is handled simply by setting the extension of the output file. But for other tools you might have to provide additional options to the videoConvertCommand above.

+

Default value: unset
+Allowed in sections: [main], [<SCRAPER>]

+
+

videoPreferNormalized

+

This option is only applicable when scraping with the -s screenscraper module. ScreenScraper offers two versions of some of their videos. A normalized version, which adheres to some defined standard they made, and the originals. If you prefer converting or standardizing the videos yourself (see videoConvertCommand above) then you can set this to false. If you do so Skyscraper will fetch the original videos from ScreenScraper instead of the normalized ones.

+

Be aware that the original videos often vary a lot in codec, color format and size. So it is recommended to convert them afterwards using the videoConvertCommand.

+

Example(s)

+
[screenscraper]
+videoPreferNormalized="false"
+
+

Default value: true
+Allowed in sections: Only for scraper [screenscraper]

+
+ +

Enabling this option is only relevant while also setting the videos="true" option. It basically means that Skyscraper will create a link to the cached videos instead of copying them when generating the game list media files. This will save a lot of space, but has the caveat that if you somehow remove the videos from the cache, the links will be broken and the videos then won't show anymore.

+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

theInFront

+

Game titles are returned from the scraping sources sometimes as 'The Game' and other times as 'Game, The'. Enabling this option will force Skyscraper to always try and move 'The' to the front of the titles. If it is not enabled, Skyscraper will always try and move it to the end of the title, regardless of how it was originally returned by the scraping sources.

+
+

Info

+

When generating gamelists Skyscraper will still sort the games as if the game titles didn't have 'The' at the beginning.

+
+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

brackets

+

Disable this option to remove any bracket notes when generating the game list. It will disable notes such as (Europe) and [AGA] completely. This option is only relevant when generating the game list. It makes no difference when gathering data into the resource cache. Default (true) will keep brackets in the game title.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

innerBracketsReplace

+

Only in use when the option brackets is set to true for gamelist creation: +This replaces consecutive brackets ][ in the game title with whatever is +defined in this option. This setting has no effect, if there is only one bracket +present in the game filename.
+Use the option innerParenthesesReplace for the same effect on round brackets +)( (aka. parentheses).

+

Example(s)

+

Filename: Oddworld - Abe's Exoddus [NTSC-U] [SLUS-00710].m3u

+
    +
  • innerBracketsReplace="" (unset), gamelist game title output: Oddworld - Abe's Exoddus [NTSC-U][SLUS-00710]
  • +
  • innerBracketsReplace="] [", gamelist game title output (note the space.): Oddworld - Abe's Exoddus [NTSC-U] [SLUS-00710]
  • +
  • innerBracketsReplace=",", gamelist game title output: Oddworld - Abe's Exoddus [NTSC-U,SLUS-00710]
  • +
+

Default value: unset
+Allowed in sections: [main]

+
+

innerParenthesesReplace

+

Same as innerBracketsReplace but for parentheses )( +(aka. round brackets).

+
+

maxLength

+

Sets the maximum length of returned game descriptions. This is a convenience option if you feel like game descriptions are too long. By default it is set to 2500.

+

Default value: 2500
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>], [<SCRAPER>]

+
+

minMatch

+

Some scraping modules are based on a file name or title based search. This option sets the minimum percentage any returned results need to match with in order for it to be accepted. For instance, the game Wonderboy in Monsterland might return the title Wonder Boy in Monster Land which is clearly a match. But it's not a 100% match. So it needs to be set relatively high, while still ignoring bad matches. By default it is set to 65 which has been tested to be a good middle-ground.

+

Default value: 65
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

threads

+

Sets the desired number of parallel threads to be run when scraping. Some modules have maximum allowed threads. If you set this higher than the allowed value, it will be auto-adjusted. By default it is set to 4.

+

Default value: 4
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

pretend

+

This option is only relevant when generating a game list (by leaving out the -s <SCRAPER> command line option). It disables the game list generator and artwork compositor and only outputs the results of the potential game list generation to the terminal. It is mostly useful when used as a command line flag with --flags pretend. It makes little sense to set it here, but you can if you want to.

+

Default value: false
+Allowed in sections: [main], [<PLATFORM>]

+
+

unattend

+

When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By enabling this option Skyscraper will always overwrite an existing game list and never skip existing entries, in other words: the game list will be newly created. This is flag useful for example when scripting Skyscraper to avoid the need for user input.

+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>], [<SCRAPER>]

+
+

unattendSkip

+

When generating a game list Skyscraper will check if it already exists and ask if you want to overwrite it. And it will also ask if you wish to skip existing game list entries. By enabling this option Skyscraper will always overwrite an existing game list and always skip existing entries, in other words: game list entries are added if not present in the gamelist, existing entries are left untouched. This flag is useful for example when scripting Skyscraper to avoid the need for user input.

+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>], [<SCRAPER>]

+
+

interactive

+

When gathering data from any of the scraping modules many potential entries will be returned. Normally Skyscraper chooses the best entry for you. But should you wish to choose the best entry yourself, you can enable this option. Skyscraper will then list the returned entries and let you choose which one is the best one. It is recommended to use the command line flag --flags interactive instead in the (hopefully) rare cases where this mode is necessary.

+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

forceFilename

+

Enable this option to force Skyscraper to use the file name (excluding extension) instead of the cached scraping module titles when generating a game list.

+
+

Tip

+

If you set forceFilename="true" and your filenames contain bracket notes such as (this) or [that] at the end, these will be combined with whatever bracket notes are at the end of the titles returned from the sources. This can cause some confusion. For instance, if you have the filename Gran Turismo 2 (USA) (Arcade Mode) and the cached title is Gran Turismo 2 (Arcade Mode), then the gamelist name will become Gran Turismo 2 (Arcade Mode)(USA)(Arcade Mode). You can disable them altogether with the brackets="no" option.

+
+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

verbosity

+

Sets how verbose Skyscraper should be when running. Default level is 0. The higher the value, the more info Skyscraper will output to the terminal while running. Anything higher than 3 does not further increase the verbosity of the output.

+

Default value: 0
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

skipped

+

Only has an effect if a rom has no resources attached to it in the cache. If false (default) the rom will be left out when generating a game list file. However, it will still show up in the frontend (at least it does for EmulationStation), but it won't exist in the game list file. If unsure leave it to false, unless you need generic gamelist entries, consisting of <path/> and <name/>, for some reason. In other words: If skipped is set to false and no entry for a rom is found in the cache nothing will be added to the gamelist for that rom. If skipped is true and no entry for a rom is found in the cache a generic entry will be added to the gamelist file for that rom. If a rom has data in Skyscraper's cache then this flag has no effect.

+
+

Note

+

When skipped is set true, any generic added game entry is not eligible for formatting with a nameTemplate.

+
+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

maxFails

+

Not all scraping modules support all platforms. This means that you can potentially start a scraping run with a module and a platform that is incompatible. This will hammer the servers for potentially hundreds of roms but provide 0 results for any of them. To avoid this Skyscraper has a builtin limit for initially allowed failed rom lookups. If this is reached it will quit. Setting this option allows you to set this limit yourself, but not above a maximum of 200. The default limit is 42. Don't change this unless you have a very good reason to do so.

+

Default value: 42
+Allowed in sections: [main]

+
+

region

+

Adds the specified region to the top of the existing default internal region priority list. Only one region is supported with this configuration. To overwrite the internal default region priority list entirely check here. Read more about how regions are handled in general here.

+
+

Info

+

Setting the region will overwrite any region auto-detected from the file name.

+
+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>]

+

Example(s)

+
[main]
+region="de"
+
+

or

+
[snes]
+region="de"
+
+
+

regionPrios

+

Completely overwrites the internal region priority list inside of Skyscraper. Multiple regions can be configured here separated by commas. Read more about how regions are handled in general here.

+
+

Info

+

Any region auto-detected from the file name will still be added to the top of this list.

+
+

Default value: eu, us, ss, uk, wor, jp, au, ame, de, cus, cn, kr, asi, br, sp, fr, gr, it, no, dk, nz, nl, pl, ru, se, tw, ca
+Allowed in sections: [main], [<PLATFORM>]

+

Example(s)

+
[main]
+regionPrios="de,us,jp"
+
+

or

+
[megadrive]
+regionPrios="de,us,jp"
+
+
+

lang

+

Adds the specified language to the top of the existing default internal language priority list. Only one language is supported with this configuration. Read more about how languages are handles in general here.

+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>]

+

Example(s)

+
[main]
+lang="it"
+
+

or

+
[snes]
+lang="it"
+
+
+

langPrios

+

Completely overwrites the internal language priority list inside of Skyscraper. Multiple languages can be configured here separated by commas. Read more about how languages are handles in general here.

+

Default value: en, de, fr, es
+Allowed in sections: [main], [<PLATFORM>]

+

Example(s)

+
[main]
+langPrios="it,en"
+
+

or

+
[megadrive]
+langPrios="it,en"
+
+
+

artworkXml

+

Sets a non-default xml file to use when setting up the artwork compositing. By default Skyscraper uses the file /home/<USER>/.skyscraper/artwork.xml. If you provide a relative filepath it will be expanded to /home/<USER>/.skyscraper/<artworkXml>, respective to $XDG_CONFIG_HOME/skyscraper/<artworkXml>, if you use Skyscraper in XDG mode.
+Read more about the artwork.xml format and customization options here.

+
+

Tip

+

It can be very useful to set this in any platform section or frontend section where you want a specific artwork setup / look.

+
+

Default value: artwork.xml
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

relativePaths

+

Enabling this forces the rom and any media paths inside the game list to be relative to the rom input folder. Currently only relevant when generating an EmulationStation, a Retrobat or a Pegasus game list (see also frontend option).

+
+

Info

+

This setting has no effect when the frontend is set to attractmode.

+
+

Default value: false
+Allowed in sections: [main], [<PLATFORM>]

+
+

extensions

+

Completely overwrites the rom extensions Skyscraper allows for the chosen platform. Use a space between each extension (e.g., *.abc *.def).

+
+

Info

+

If you feel like you are using a file extension that ought to be supported by default, please report it so it can be added in a later version of Skyscraper.

+
+

Default value: unset. If set it ignores "formats": [ ... ] in peas.json file
+Allowed in sections: [<PLATFORM>]

+
+

addExtensions

+

Adds the rom extensions to the ones that are already supported by the platform, use a space between each extension.

+
+

Info

+

If you feel like you are using a file extension that ought to be supported by default, please report it so it can be added in a later version of Skyscraper.

+
+

Default value: *.zip *.7z *.ml (if and only if extensions parameter is unset)
+Allowed in sections: [main], [<PLATFORM>]

+
+

hints

+

If set to false it disables the "Did you know" hints when running Skyscraper.

+

Default value: true
+Allowed in sections: [main]

+
+

subdirs

+

By default Skyscraper will include roms located in subfolders. By disabling this option Skyscraper will only scrape the roms located directly in the input folder. See inputFolder="<PATH>" further up to read more about the rom input folder.

+

Default value: true
+Allowed in sections: [main], [<PLATFORM>]

+
+

onlyMissing

+

This flag, when set true, will only scrape game information for ROMs that do not have any data in Skyscraper's cache. The cache information of a ROM that has a cache hit will not be updated or extended when 'only missing' is true. A use case could be that you want to exclude already scraped games from a scraping process and you are using a scraping module which has a limit on how many ROMs may be scraped: With 'only missing' enabled the ROMs with cache data are subtracted from all ROMs to be scraped for a platform, thus the remaining ROM count may be below the scrape module's limit.

+
+

Tip

+

Please consider using the command line flag option --flags onlymissing instead. It makes little sense to set this permanently in your configuration unless you always want to use the cached data for already cached game entries. Also when 'only missing' is set and the cache has any data for a game entry, the refresh option has no effect. But you still can use the 'only missing' in the config file if you absolutely want to.

+
+

Default value: false
+Allowed in sections: [main], [<PLATFORM>], [<SCRAPER>]

+
+

startAt

+

If you only ever wish to gather data for a subset of your roms from the scraping modules you can use this option to set the starting rom. It will then scrape alphabetically from that rom and onwards.

+
+

Tip

+

Please consider using the command line option --startat <FILENAME> instead. It makes little sense to set this permanently in your configuration unless you always want it to only ever scrape from a certain file and onward. But you can if you absolutely want to.

+
+

Default value: unset
+Allowed in sections: [<PLATFORM>], [<FRONTEND>]

+
+

endAt

+

If you only ever wish to gather data for a subset of your roms from the scraping modules you can use this option to set the rom to end at. It will then scrape alphabetically until it reaches this rom, then stop.

+
+

Tip

+

Please consider using the command line option --endat <FILENAME> instead. It makes little sense to set this permanently in your configuration unless you always want it to only ever scrape up until it reaches a certain file. But you can if you absolutely want to.

+
+

Default value: unset
+Allowed in sections: [<PLATFORM>], [<FRONTEND>]

+
+

includePattern

+

Per platform Skyscraper have default file extensions that it will accept. This option allows you to only include certain files within that scope. The pattern is a simple asterisk type pattern. In cases where you need to match for a comma you need to escape it as \,.

+
+

Note

+

You might also want to check out the file extension options and the the 'includeFrom' option.

+
+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

excludePattern

+

Per platform Skyscraper have default file extensions that it will accept. This option allows you to exclude certain files within that scope. The pattern is a simple asterisk type pattern. In cases where you need to match for a comma you need to escape it as \,.

+
+

Note

+

You might also want to check out the file extension options and the 'excludeFrom' option.

+
+
+

Tip

+

If you create a file named .skyscraperignore within any subfolder of the input dir, all files from that directory will be ignored by Skyscraper. Additionally, if you put an empty file .skyscraperignoretree in a folder, all files in that directory and all subdirectories are ignored.

+
+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>], [<FRONTEND>]

+
+

includeFrom

+

Tells Skyscraper to only include the files listed in FILENAME. One filename per line (with FULL path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip').

+

This file can be generated with the '--cache report:missing' option or made manually.

+
+

Note

+

You might also want to check out the 'includePattern' option.

+
+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>]

+
+

excludeFrom

+

Tells Skyscraper to exclude the files listed in FILENAME. One filename per line (with FULL path, eg. '/home/pi/RetroPie/roms/snes/subdir/somefile.zip').

+

This file can be generated with the '--cache report:missing' option or made manually.

+
+

Note

+

You might also want to check out the 'excludePattern' option.

+
+
+

Tip

+

If you create a file named '.skyscraperignore' within any subfolder of the input dir, all files from that directory will be ignored by Skyscraper. Additionally, if you put an empty file .skyscraperignoretree in a folder, all files in that directory and all subdirectories are ignored.

+
+

Default value: unset
+Allowed in sections: [main], [<PLATFORM>]

+
+

userCreds

+

Some scraping modules require a key or user:password to work. You can create a [<SCRAPER>] section and add a userCreds="user:pass" or userCreds="key" line beneath it. Doing this will always use these credentials when scraping from the module in question. Check the scraping module overview to see the specific requirements for each module here.

+

Example(s)

+
[screenscraper]
+userCreds="user:password"
+
+

or

+
[igdb]
+userCreds="key"
+
+

Default value: unset
+Allowed in sections: [<SCRAPER>]

+
+

spaceCheck

+

Skyscraper will continuously check if you are running low on disk space. If you go below 200 MB in either the game list export folder or the resource cache folder, it will quit to make sure your system doesn't become unstable. Some types of file systems provide a faulty result to Skyscraper when it comes to these checks and thus it can be necessary to disable it altogether. You can use this option to do just that.

+

Default value: true
+Allowed in sections: [main]

+
+

scummIni

+

Allows you to set a non-default path of the scummvm.ini file. This file is used whenever scraping the scummvm platform. It converts the shortname such as monkey2 to the more search-friendly name Monkey Island 2: LeChuck's Revenge whenever using one of the file name search based scraping modules.

+

Default value: If ~/.scummvmrc has precedence over /opt/retropie/configs/scummvm/scummvm.ini
+Allowed in sections: [main]

+
+

platform

+

Allows you to set a platform, which is applied when no command line switch -p is given.

+

Default value: unset
+Allowed in sections: [main]

+
+

addFolders

+

If you arrange your ROMs in directories below a platform directory, this flag +comes into play. The gamelist +specification +allows you to define also metadata for directories. This metadata is held in +<folder/> elements in the gamelist XML file. In minimum a <folder/> has a +<path/> and a <name/> inner element. If no <name/> is given, the last +subdirectory of the path element is taken as name. Which additionally +subelements are interpreted and rendered is dependent on the frontend, for +example EmulationStation for RetroPie supports these +elements +to be used inside a folder element.

+

If this flag set to true, Skyscraper will create generic folder elements in the +gamelist file, containing path and name for each directory found. If false, no +folder elements will be created for the directories with ROM(s) (example).

+

In both cases the existing <folder/> elements of a gamelist will be preserved. +However, folder data is not cached by Skyscraper, which means if you delete your +gamelist.xml Skyscraper can not restore the edited folder elements.

+
+

EmulationStation 2.11.2rp and earlier

+

EmulationStation 2.11.2 (RetroPie) and earlier have a bug that adds a <folder/> element everytime you edit and save metadata for the same folder within EmulationStation. The last edit will be the latest folder element in the gamelist file. Skyscraper in contrast expects in maximum one <folder/> element per each path. If you regenerate a gamelist with Skyscraper, you will lose the latest edit of the folder metadata. This bug is described here and fixed in the next release of EmulationStation respective is fixed already in the EmulationStation-dev version.

+
+

Default value: false
+Allowed in sections: Only for frontends [emulationstation], [esde] or [retrobat]

+
+

manuals

+

By default Skyscraper doesn't scrape and cache game manuals resources because not all scraping sites provide this data and also only some frontends support PDF display of these game manuals. If enabled Skyscraper will collect game manuals for the scraping modules that provide this data. For frontend ES-DE no further option must be set to enable the output of the PDF manuals to the appropriate folder. For other EmulationStation forks see also option gameListVariants.

+

Default value: false
+Allowed in sections: [main], [<PLATFORM>]

+
+

gameListVariants

+

This is a comma separated list of options for the different gamelist variants used by the various EmulationStation forks. Currently only enable-manuals is evaluated as variant: It generates <manual/> entries in the gamelist for the game manuals scraped or found in the cache, if also the manuals configuration option is enabled. This option is not needed for the ES-DE frontend to output game manuals.

+

Example(s)

+
[emulationstation]
+gameListVariants="enable-manuals"
+
+

Default value: unset
+Allowed in sections: Only for frontend [emulationstation]

+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/CONTRIBUTING/index.html b/CONTRIBUTING/index.html new file mode 100644 index 00000000..3ea5c1c7 --- /dev/null +++ b/CONTRIBUTING/index.html @@ -0,0 +1,1063 @@ + + + + + + + + + + + + + + + + + + + + + + + + + Contributing - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + +

Contributing

+ +

Source Code

+
    +
  1. Fork this repo
  2. +
  3. It is appreciated (but not required) if you raise an issue first
  4. +
  5. Create your feature branch (git checkout -b my-new-feature)
  6. +
  7. Code, test, commit, repeat
  8. +
  9. Apply code formatting (e.g. clang-format-15 -i src/<changedfiles>)
  10. +
  11. Squash your commits to one commit (e.g. git rebase --keep-base -i master)
  12. +
  13. Push to the branch (git push origin my-new-feature)
  14. +
  15. Create new Pull Request
  16. +
+

Documentation

+

Found something missing, or an existing section confusing or outdated? Any +documentation fix is also appreciated.

+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/FAQ/index.html b/FAQ/index.html new file mode 100644 index 00000000..7b2e84b3 --- /dev/null +++ b/FAQ/index.html @@ -0,0 +1,1102 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + FAQ - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + +

FAQ

+ +

The official Skyscraper Frequently Asked Questions

+

A quick note before I let you loose on the questions:

+
    +
  • You can get a list of all available command-line options by running Skyscraper --help
  • +
  • For a more thorough description of the command-line options go here
  • +
  • You can read all about the available config.ini options here
  • +
  • If you are using Skyscraper with RetroPie search their forum or raise a question there if you are puzzled
  • +
+

And now for the questions (with answers)...

+
+I ran Skyscraper and it scraped a lot of artwork and game data, but no data is showing up in my gamelists. What's going on? +

You probably forgot to generate the gamelist. Skyscraper has two modes; resource gathering mode and gamelist generation mode. First you gather data into Skyscraper's resource cache by scraping the platform with any of the supported scraping modules (eg. Skyscraper -p snes -s thegamesdb). When you feel like you have gathered all the resources that you need, you then generate the gamelist by simply leaving out the -s <MODULE> option (eg. Skyscraper -p snes). +Read more about the resource cache here.

+
+
+Skyscraper keeps quitting on me with errors about request limits or similar. What's going on? +

Almost all of the scraping modules, such as ScreenScraper and TheGamesDb, have restrictions applied to avoid users overloading their servers. These restrictions are set by them, not Skyscraper. Some modules even require you to log in to use them. Read more about the restrictions and how to deal with them here. If you've created a user at ScreenScraper and want to use your credentials with Skyscraper, just add the following section to /home/<USER>/.skyscraper/config.ini. Type it exactly like this, but with your own user and pass. +

[screenscraper]
+userCreds="USER:PASS"
+

+
+
+Skyscraper by default generates a composited screenshot that combines both boxart, screenshot and wheel. I just want it to export the raw artwork, how do I do that? +

Skyscraper includes a powerful artwork compositor you can set up any way you like. Read more about the possibilities here. If you just want it to export the raw unmodified screenshot and other pieces of artwork, you can use the /home/<USER>/.skyscraper/artwork.xml.example2 and copy it to /home/<USER>/.skyscraper/artwork.xml. Remember to regenerate the gamelists after you've done this to make use of the new artwork configuration. This can be done simply by running Skyscraper with Skyscraper -p PLATFORM. This will regenerate the gamelist for that platform using this new artwork configuration.

+
+
+I've edited the artwork.xml or some other part of Skyscraper's gamelist specific configurations, but the games still show up the same inside my frontend. Why is that? +

Whenever you make any gamelist specific changes to Skyscraper's configurations, you need to regenerate the gamelists for your chosen frontend afterwards for the changes to take effect. This can be done simply by running Skyscraper with Skyscraper -p PLATFORM. This will regenerate the gamelist for that platform for the default EmulationStation frontend using your new configuration. +If you wish to export for one of the other supported frontends, you need to specify this. Read more about the frontends and their requirements here.

+
+
+I used Skyscraper to generate EmulationStation gamelists for one or more platforms, but when I restarted the system the gamelist hadn't changed. Why is that? +

Did you remember to quit EmulationStation before running Skyscraper? EmulationStation writes its current gamelists back to disk when it is closed. If you run Skyscraper while EmulationStation is also running, it will overwrite any gamelists you just generated. You need to quit EmulationStation before generating the gamelists with Skyscraper to avoid this. You can quit EmulationStation by pressing F4 on a connected keyboard, or by selecting quit from the menus.

+
+
+I want to set up different options for different platforms. It's really tiresome setting it all on command-line. Can this be done easier? +

YES! Absolutely. In addition to taking options from the command-line, Skyscraper also reads the /home/<USER>/.skyscraper/config.ini file, which is where you should set everything up. This file allows you to configure things both globally, per-platform, per-frontend and per-scraping module. Read more about the available options and option priorities here.

+
+
+When I try to scrape data or generate gamelists I get all sorts of write permission errors and missing configurations. What's going on? +

You are probably running, or at some point ran, Skyscraper with sudo Skyscraper instead of just Skyscraper. Doing so will result in all sorts of weird behaviour as Skyscraper was then run as the root user instead of the ordinary user. Depending on what command-line parameters you ran Skyscraper with, this will have caused permission issues for your files in /home/<USER>/RetroPie/roms subfolders and might also have caused permission issues with the /home/<USER>/.skyscraper folder.
+This is a common problem for new Linux users who are used to working with Windows. In Linux you rarely need Administrator privileges. Running a command with sudo in front of it will run it as the root / Administrator user. And any folders or files being generated while that command runs, will then be owned by root. When you run software as the normal user afterwards and it tries to write to those folders or files, it will fail.
+You need to reset those permissions back to be owned by your ordinary user. If you are running RetroPie, the following commands might fix your problem (RUN AT YOUR OWN RISK!): +

$ sudo chown -R pi:pi /home/pi/.skyscraper
+$ sudo chown -R pi:pi /home/pi/RetroPie/roms
+
+The above commands must be run with sudo, as we need root permission to reset the folders and files (hence the warning). But in general, never run anything with sudo unless you are specifically told to do so.

+
+
+I generated a game list for the Pegasus frontend. When I fire up Pegasus the data is there, but it doesn't look very good. Why is that? +

Skyscraper contains a powerful artwork compositor that can customize the exported screenshot, boxart etc. any way you like. This is done through the /home/<USER>/.skyscraper/artwork.xml file. The default configuration for this file is made to look good when exporting for the default frontend EmulationStation. When you use a different frontend, you need to customize this file a bit to make it look good. For Pegasus I suggest something like this: +

<?xml version="1.0" encoding="UTF-8"?>
+<artwork>
+  <output type="screenshot" width="640"/>
+  <output type="cover" width="640" height="480">
+    <layer resource="cover" height="480" align="center" valign="middle">
+      <gamebox side="wheel" rotate="90"/>
+    </layer>
+  </output>
+  <output type="wheel" height="200"/>
+</artwork>
+
+Read more about how to set up a custom artwork.xml here.

+
+
+I've changed my artwork.xml configuration and regenerated the game list, but it still looks the same in the frontend. Why is that? +

Did you perhaps edit /home/<USER>/skysource/artwork.xml instead of /home/<USER>/.skyscraper/artwork.xml? The latter is the one you need to edit.
+Read more about how to set up a custom artwork.xml here.

+
+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/FRONTENDS/index.html b/FRONTENDS/index.html new file mode 100644 index 00000000..f4125fa3 --- /dev/null +++ b/FRONTENDS/index.html @@ -0,0 +1,1373 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Frontends - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + +

Frontends

+ +

Overview

+

When generating a game list with Skyscraper you have the option of generating it for several different frontends. A frontend is the graphical interface that lists and launches your games.

+

Setting a frontend when generating a game list is done by setting the -f <FRONTEND> command-line parameter as explained here or by setting it in /home/<USER>/.skyscraper/config.ini as explained here. Use for the <FRONTEND> value the frontend name all lowercase and with alphabetical characters only: emulationstation, esde, pegasus, retrobat, attractmode. Some frontends have further options that are either optional or required. Check the frontend sections below for more information on this.

+
+

Warning

+

Skyscraper will overwrite your game list (obviously). So if you have spend a lot of time hand-crafting metadata in a game list for any frontend, please remember to create a backup before overwriting it with Skyscraper. You can also tell Skyscraper to auto-backup old game lists prior to overwriting them. Read more about that here.

+
+

When generating a game list for any frontend, Skyscraper will try to preserve certain metadata. Check the frontend sections below for more information on what metadata is preserved per frontend.

+

EmulationStation (default)

+
    +
  • Default game list location: /home/<USER>/RetroPie/roms/<PLATFORM>
  • +
  • Default game list filename: gamelist.xml
  • +
+

This is the default frontend used when generating a game list with Skyscraper. If no frontend is defined on command-line or in config.ini it will fall back to generating for EmulationStation. By default the <USER> is pi on RetroPie setups.

+

Metadata preservation

+

Skyscraper will preserve the following metadata when re-generating a game list for EmulationStation: favorite, hidden, kidgame, lastplayed, playcount, sortname. Also existing <folder/> elements of a gamelist file will be preserved. The user editable sub-XML elements for a folder are listed in the Metadata.cpp of EmulationStation.

+
+

Warning

+

Folder data is not cached by Skyscraper, thus if you delete your gamelist.xml, Skyscraper can not restore the edited folder elements from cache.

+
+

Automatic addition of folder elements if addFolder is true:
+If at least one ROM is within a subfolder and this subfolder is not yet part of the gamelist.xml file, it will be added with two mandatory subelements:

+
    +
  • <path/> reflects the relative subpath from the system folder and
  • +
  • <name/>, which represents the direct parent folder of a ROM by default. However, you may edit this to any name which should be shown in EmulationStation.
  • +
+
+

Example

+

Consider this folder structure below snes, whereas each lowest folder contains at least one ROM: +

snes
+└── Retail
+    ├── EUR
+    ├── JP
+    └── USA
+
+Skyscraper will generate these elements if not present in gamelist.xml and only if addFolders is set true: +
[...]
+    <folder>
+        <path>./Retail</path>
+        <name>Retail</name>
+    </folder>
+    <folder>
+        <path>./Retail/USA</path>
+        <name>USA</name>
+    </folder>
+    <folder>
+        <path>./Retail/JP</path>
+        <name>JP</name>
+    </folder>
+    <folder>
+        <path>./Retail/EUR</path>
+        <name>EUR</name>
+    </folder>
+[...]
+
+Note that the Retail folder is added even if it does not contain a ROM because it is part of the path to the ROMs in the lowest folders.

+
+

EmulationStation Desktop Edition (ES-DE)

+

This is very similar to the default EmulationStation. This section only +highlights the differences.

+
    +
  • Default game list location: /home/<USER>/ES-DE/gamelists/<PLATFORM>
  • +
  • Default game list filename: gamelist.xml
  • +
+

ROMs are expected to be in the input folder /home/<USER>/ROMs/<PLATFORM> for +every <PLATFORM> you scrape.

+

Media filepaths (screenshots, marquees, videos, aso.) are not explicitly stored +in the gamelist file. This is ES-DE default. However, Skyscraper will put the +media files into the downloaded_media folder (e.g. +~/ES-DE/downloaded_media/<PLATFORM>/screenshots/ for screenshots) from where +ES-DE will pick them up. Note that ES-DE does not support textures currently.

+
+

Tip

+

You may override the default storage folder of ES-DE (/home/<USER>/ES-DE) +on a non-Windows build by setting the environment variable ESDE_APPDATA_DIR +to a different folder as documented in the ES-DE documentation here.

+
+

All the gory details of ES-DE's gamelist file format you can find in the section +"Gamelist Reference" +here.

+

Metadata preservation

+

Skyscraper will, additionally to EmulationStation preserved metadata, retain the +following metadata when re-generating a game list for ES-DE: altemulator, +broken, collectionsortname, completed, controller, hidemetadata, +nogamecount, nomultiscrape. Also existing <folder/> elements of a gamelist +file will be preserved: For these on top is the folderlink element is preserved.

+

RetroBat

+
    +
  • Default game list location: /home/<USER>/RetroPie/roms/<PLATFORM>
  • +
  • Default game list filename: gamelist.xml
  • +
+

This is modeled after EmualtionStation as it uses it with slight differences.

+

Attract-Mode

+
    +
  • Default game list location: /home/<USER>/.attract/romlists
  • +
  • Default game list filename: <EMULATOR/PLATFORM>.txt
  • +
+

Attract-Mode is a bit more abstract when it comes to how it saves its game lists and media. To export for Attract-Mode you need to, in addition to setting the frontend, set -e <EMULATOR> on command-line or in config.ini. The <EMULATOR> is a file that describes the platform / emulator you are generating a game list for. The file contains everything needed to tell Attract-Mode how to launch games for the platform, and even where to find the media files for the games (such as screenshots and videos).

+

If you are running RetroPie most of the <EMULATOR> files will have been auto-generated for you. They are usually named the same as the platform you are generating a game list for. So, if you are generating for snes you simply use Skyscraper -p snes -f attractmode -e snes and that should work just fine. The <EMULATOR> files are usually located at /home/<USER>/.attract/emulators/. Check them out if you are curious.

+

Metadata preservation

+

Skyscraper will preserve the following metadata when re-generating a game list for Attract-Mode: altromname, alttitle, buttons, cloneof, control, displaycount, displaytype, extra, rotation, status.

+

Pegasus

+
    +
  • Default game list location: /home/<USER>/RetroPie/roms/<PLATFORM>
  • +
  • Default game list filename: metadata.pegasus.txt
  • +
+

Pegasus is easy and simple to generate a game list for. Simply do Skyscraper -p <PLATFORM> -f pegasus. If you want to specify a custom launch command (if you are using RetroPie you don't have to, a default one will be used), you can set it on command-line with -e "<COMMAND>" or in config.ini with:

+
[pegasus]
+launch="<COMMAND>"
+
+

You need to add the individual platform rom directories to Pegasus (if they are not already defined) before any of them will show up! Start the Pegasus frontend, press ESC on the keyboard and choose Settings -> Set game directories. Simply point it to each individual platform sub-directory. For RetroPie you should have a path for each platform (eg. /home/<USER>/RetroPie/roms/snes, /home/<USER>/RetroPie/roms/megadrive etc.).

+
+

Info

+

If you are generating game lists for Pegasus, it is highly recommended to disable third-party game list data sources! Otherwise you will have a mish-mash or different sources showing up in Pegasus. Start the Pegasus frontend, press ESC on the keyboard and choose Settings -> Enable/disable data sources and disable everything in that submenu.
+Then reload the game lists or restart Pegasus, and all of the platforms should show up with media and game information generated by Skyscraper.

+
+

Metadata preservation

+

Skyscraper will preserve any metadata key-value pairs added to the header and / or individual game list entries.

+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/IMPORT/index.html b/IMPORT/index.html new file mode 100644 index 00000000..fabf2f85 --- /dev/null +++ b/IMPORT/index.html @@ -0,0 +1,1314 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Import of Game Data - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + + + + + +
+
+ + + + + + + +

Import of Game Data

+ +

Custom Data Import

+

The following describes how to import your own custom textual, artwork and / or video data into the resource cache for later use when generating game lists (enable game list generation mode simply by leaving out the -s command line option).

+
+

Tip

+

For any path in the following description, you can also add a platform subfolder. Skyscraper will auto-detect this and use it instead of the base import folder. For instance, you can create the folder /home/<USER>/.skyscraper/import/amiga and it will use that as base instead of /home/<USER>/.skyscraper/import/ when importing for the amiga platform.

+
+
+

Info

+

Be sure to also check the --cache edit option here.

+
+

Images, Videos and Game Manuals

+

To import videos or images into the resource cache, use the following procedure:

+
    +
  • Name your image or video file with the exact base name of the rom you wish to connect it to. Example: Bubble Bobble.nes will import images with a filename of Bubble Bobble.jpg or Bubble Bobble.png or other well-known image formats. As long as the base name is an exact match. Same goes for video files. I recommend only making use of well-known video formats since Skyscraper imports them directly without conversion (unless you convert them as described here), so they need to be supported directly by the frontend you plan to use.
  • +
  • Game manuals are expected to use PDF format and have the extension .pdf. The base name must match the ROM file, thus the game manual of the example is Bubble Bobble.pdf.
  • +
  • Place all of your images, videos or game manuals in the /home/<USER>/.skyscraper/import/<PLATFORM>/screenshots, covers, wheels, marquees, videos or manuals folders.
  • +
  • Now run Skyscraper with Skyscraper -p <PLATFORM> -s import. If you named your files correctly, they will now be imported. Look for the green 'YES' in the output at the rom(s) you've placed files for. This will tell you if it succeeded or not.
  • +
  • The data is now imported into the resource cache. To make use of if read here.
  • +
+

Special Note for Videos

+

If you are importing videos, you also need to add the command line flag --flags videos for this to work. Videos aren't imported or scraped by default, since it is considered a huge disk space hog. So keep this in mind if you want to import videos into the cache. If you plan on always using videos, consider adding this option to the config.ini instead.

+

Textual Data (Publisher, Players, Rating, ...)

+

Skyscraper also allows you to import textual data for any rom you have in your collection. All you need to do is to prepare files for each rom with an exactly matching base name. For instance Bubble Bobble.nes should have a file called Bubble Bobble.txt or Bubble Bobble.xml or whatever suffix you want to use. The suffix is not important. What is important is that you place all of these raw data files into the /home/<USER>/.skyscraper/import/textual folder. And then you need to make a definitions file so Skyscraper has a recipe for these files.

+

Textual data definitions file

+

In order for Skyscraper to understand your textual data files, it needs a recipe. Or a definition of your format if you like. The format is completely up to you. The file must be placed at /home/<USER>/.skyscraper/import/definitions.dat. Here follows a few examples with a matching data file for comparison:

+

Example 1

+

Definitions file: /home/<USER>/.skyscraper/import/definitions.dat

+
<game>
+  <title>###TITLE###</title>
+  <description>###DESCRIPTION###</description>
+  <developer>###DEVELOPER###</developer>
+  <publisher>###PUBLISHER###</publisher>
+  <players>###PLAYERS###</players>
+  <rating>###RATING###</rating>
+  <ages>###AGES###</ages>
+  <genre>###TAGS###</genre>
+  <releasedate>###RELEASEDATE###</releasedate>
+</game>
+
+

Import file: /home/<USER>/.skyscraper/import/textual/<EXACT ROM BASE NAME>.xml

+
<game>
+  <title>The Game Title</title>
+  <description>This game is about yada, yada yada.</description>
+  <developer>The game developer</developer>
+  <publisher>The game publisher</publisher>
+  <players>Maximum number of players supported (example '4')</players>
+  <rating>The game rating 0-5 (example '3.5')</rating>
+  <ages>Minimum age restriction as integer (example '16')</ages>
+  <genre>Genres, Comma-separated</genre>
+  <releasedate>The game release date (example '1985-06-01')</releasedate>
+</game>
+
+
+

Tip

+

As with Skyscraper 3.9.2 onwards you are no longer required to have the same indention or whitespace (hard tabs, spaces) as in the XML-like definitions.dat. Matching is solely done via XML elements (e.g. <title>), which saves you headache in identifiying whitespace mismatch between the XML template and your XML import file. If you use a custom XML-like definitions.dat: Keep the opening and closing XML element on one line and use one line per ###tag### as in the example definition shown above. This is the default in the provided definitions.dat. In your import file just assure to use also closing elements (</title>) and the same caSiNg in the elements between template and import file, ie. provide a valid XML file.

+
+

Example 2

+

Definitions file: /home/<USER>/.skyscraper/import/definitions.dat

+
Title      : ###TITLE###
+Description: ###DESCRIPTION###
+Developer  : ###DEVELOPER###
+Publisher  : ###PUBLISHER###
+Players    : ###PLAYERS###
+Rating     : ###RATING###
+Rating     : ###AGES###
+Genre      : ###TAGS###
+Date       : ###RELEASEDATE###
+
+

Import file: /home/<USER>/.skyscraper/import/textual/<EXACT ROM BASE NAME>.txt

+
Title      : Game Title
+Description: This game is about yada, yada yada.
+Developer  : The game developer
+Publisher  : The game publisher
+Players    : Maximum number of players supported (example '4')
+Rating     : The game rating 0-5 (example '3.5')
+Ages       : Minimum age restriction as integer (example '16')
+Genre      : Genres, Comma-separated
+Date       : The game release date (example '1985-06-01')
+
+
+

Warning

+

Make sure any line matches exactly with the line in the recipe including white-space characters such as newline characters, spaces and tabs! Otherwise Skyscraper won't recognize it. Especially for newline characters this can be difficult to debug. Newline characters appear the same in an editor so you won't notice the problem until you load it up in a hex editor.

+
+

List of known tags

+

From the examples above you'll notice the ###SOMETHING### tags. This is what Skyscraper recognizes your data from. The supported tags are:

+
    +
  • ###TITLE###
  • +
  • ###DESCRIPTION###
  • +
  • ###DEVELOPER###
  • +
  • ###PUBLISHER###
  • +
  • ###PLAYERS###
  • +
  • ###AGES###
  • +
  • ###RATING###
  • +
  • ###TAGS###
  • +
  • ###RELEASEDATE###
  • +
+

Resource formats

+

For ###PLAYERS###, ###AGES###, ###RELEASEDATE### and ###RATING### a certain format is required.

+
    +
  • Players must contain just the maximum number of players as an integer such as '4'.
  • +
  • Ages must be an integer between 1 and 18 (for instance "16" means it is suitable from ages 16 and up).
  • +
  • +

    Release date must be of one of the following formats:

    +
      +
    • yyyy
    • +
    • yyyy-MM
    • +
    • yyyy-MM-dd (full ISO 8601 date)
    • +
    • yyyy-MMM-dd (MMM is Jan, Feb and so on...)
    • +
    • MM/dd/yyyy
    • +
    • MMM, yyyy (MMM is Jan, Feb and so on...)
    • +
    • MMM dd, yyyy (MMM is Jan, Feb and so on...)
    • +
    +
  • +
  • +

    Rating can be either a number between 0 and 5, use as fraction only .5. Examples: 3 or 4.5 but not 4.25 or 1.0. This value will be divided by five to match the scale of the rating in a gamelist. Additionally, a value between 0.0 and 1.0 can be used. This value will not be recalculated. If you want to use 0.5 in this metric, provide it as .5 (note the missing zero). Do add a fraction of zero explicitly to denote a rating of 1.0 (100%), without this trailing fraction it will be interpreted as if it would be in the range of 0 to 5 (thus 20% for 1).

    +
  • +
+

How to actually use the data?

+

When you've imported all of your data into the resource cache, you can make use of it by enabling Skyscrapers game list generation mode simply by leaving out the -s command line option (eg. Skyscraper -p <PLATFORM>). The game list generator will then make use of your imported data. If you don't know what the resource cache is, read more about it here.

+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/LANGUAGES/index.html b/LANGUAGES/index.html new file mode 100644 index 00000000..eefb6ff5 --- /dev/null +++ b/LANGUAGES/index.html @@ -0,0 +1,1223 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Languages - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+ +
+
+ + + +
+
+ + + + + + + +

Languages

+ +

Overview

+

Some game information comes in several languages. Skyscraper provides several ways of configuring these for your convenience. It is important to understand the distinction between game region and language. Setting a language is a user-preferred thing and will only affect the game descriptions and tags (genres). The remaining game data is tied to the region instead (artwork (screenshots, marquees, ...) and, in some cases, the game name).

+

Scraping Modules that Support Languages

+
    +
  • screenscraper (game description and tags / genres)
  • +
+

Below follows a general list of supported languages. Please note that not all languages are supported by all modules (support is noted inside the parenthesis for each language).

+

When configuring languages be sure to use the short-names as shown (eg. 'no' for Norwegian).

+

List of Supported Languages

+
    +
  • cz: Czech (screenscraper)
  • +
  • da: Danish (screenscraper)
  • +
  • de: German (screenscraper)
  • +
  • en: English (screenscraper)
  • +
  • es: Spanish (screenscraper)
  • +
  • fi: Finnish (screenscraper)
  • +
  • fr: French (screenscraper)
  • +
  • hu: Hungarian (screenscraper)
  • +
  • it: Italian (screenscraper)
  • +
  • ja: Japanese (screenscraper)
  • +
  • ko: Korean (screenscraper)
  • +
  • nl: Dutch (screenscraper)
  • +
  • no: Norwegian (screenscraper)
  • +
  • pl: Polish (screenscraper)
  • +
  • pt: Portuguese (screenscraper)
  • +
  • ru: Russian (screenscraper)
  • +
  • sk: Slovakian (screenscraper)
  • +
  • sv: Swedish (screenscraper)
  • +
  • tr: Turkish (screenscraper)
  • +
  • zh: Chinese (screenscraper)
  • +
+

Default Language Precedence

+

Skyscraper's default internal language priority list is as follows. Topmost language has highest priority and so forth.

+
    +
  • Auto-detected language or language set with --lang LANGUAGE or lang="LANGUAGE"
  • +
  • en
  • +
  • de
  • +
  • fr
  • +
  • es
  • +
+

Configuring Language

+

It is possible to manipulate the default internal language priority list. You can either do this on command-line or through /home/<USER>/.skyscraper/config.ini. It is recommended to set it in config.ini for a permanent setup.

+

Permanent setting in config.ini

+

Read this and this.

+

Command Line

+

Read this.

+ + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + \ No newline at end of file diff --git a/OLDERRELEASES/index.html b/OLDERRELEASES/index.html new file mode 100644 index 00000000..d595e74d --- /dev/null +++ b/OLDERRELEASES/index.html @@ -0,0 +1,3782 @@ + + + + + + + + + + + + + + + + + + + + + + + OLDERRELEASES - Skyscraper + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + +

OLDERRELEASES

+ +

Version x.x.x (Features under consideration, all unimplemented)

+
    +
  • Add the option to scrape from cached data purely originating from certain scraping modules
  • +
  • Consider making aliasMap the global baseName instead of just the search term base name. This will fix missing brackets in alias names being ignored later on
  • +
  • Make all artwork types custom meaning that their type can be whatever the user wants. The sources will create ones with known types such as 'screenshot' and 'cover', but the user can import other types they define themselves, such as 'cabinet' or whatever else. Internally artwork is then handled by a list of artwork instead of 4 hardcoded types in GameEntry
  • +
  • Introduce the ability to use in the 'file="somefolder/.png"' in artwork.xml which would then look for a file in 'resources' with the game basename.png allowing users to use their own custom artwork data as a workaround to adding new types to import (Thank you to 'jueank' for suggesting this)
  • +
  • Add the option of scraping custom platforms by configuring them in the config with an alias to an already existing platform. Example: scrape 'pcenginecd' could be scraped as 'pcengine' in case you have those files in a 'roms/pcenginecd' folder instead of the pcengine folder. Check here: https://github.com/muldjord/skyscraper/issues/136
  • +
  • Create a testmode for the artwork compositor that let's you quickly render an example to see if you got everything set up right in the artwork xml
  • +
  • Allow 'region' to be a list similar to 'regionPrios'. When using 'region' it should simply keep the default priority list and add those from 'region' to the top. 'regionPrios' should still overwrite it entirely. Naming change probably a good idea, for instance rename 'region' to 'regionsPrefer' or something. 'regionPrios' should probably also be changed to 'regionsOverride'. (Thank you to 'corezon' for suggesting this).
  • +
  • Implement a less rigid filename match for the 'import' module to allow for close match filenames
  • +
  • Add support for grouping multi-disk games so they only have one entry in the gamelists. See issues/232 (Thank you to 'igno2k' for suggesting this)
  • +
+

Version 3.7.7-2

+
    +
  • Added script to quickly generate the mapMap.csv
  • +
  • Added script to quickly generate the platforms.json from an existing platform.cpp file
  • +
  • Moved the list of platforms and thier associated scrapers, file masks, and aliases to be loaded from a JSON file instead of hardcoded in CPP from torresflo/skyscraper-enhanced
  • +
  • Added 'texture' image type to be stored in textures dir from InfiRandia/skyscraper
  • +
  • Updated MAME map to 0.245
  • +
  • Added 'actionmax', 'naomi2', 'ps3', 'ps4', 'ps5', and 'xbox' platforms
  • +
  • Added '*.chd' to 'naomi' and 'naomi2'
  • +
  • Added '*.nca' to 'switch'
  • +
  • Fixed the 'pc88' and 'x1' ScreenScraper IDs
  • +
  • Added automatic generation of compiled builds on new tag
  • +
+

Version 3.7.7 (28th June 2022)

+
    +
  • Added '*.m3u' to 'segacd' platform (Thank you to user 'bmn001' for suggesting it)
  • +
+

Version 3.7.6 (12th June 2022)

+
    +
  • Added 'pico8' platform (Thank you to user 'zerojay' for suggesting it)
  • +
+

Version 3.7.5 (31st January 2022)

+
    +
  • Now allows ':' in Pegasus command
  • +
+

Version 3.7.4 (23rd January 2022)

+
    +
  • '--includefrom' and '--excludefrom' now works with relative paths (Thank you to use 'sleve_mcdichael' for reporting this
  • +
  • Skipped file entries now conforms to same format as report files to allow using them with '--excludefrom' and '--includefrom' (Thank you to user 'TomFury' for suggesting this)
  • +
+

Version 3.7.3 (23rd January 2022)

+
    +
  • Added 'switch' platform (Thank you to user 'Redemp' for providing info)
  • +
+

Version 3.7.2 (20th January 2022)

+
    +
  • Skyscraper now only ignore files when using the '.skyscraperignore[tree]' files when scraping for new data. They will NOT be ignored when generating game lists
  • +
+

Version 3.7.1 (19th January 2022)

+
    +
  • Skyscraper will now ignore an entire tree of subfolders where a file called '.skyscraperignoretree' is found (Thank you to user 'sromeroi' for suggesting this)
  • +
+

Version 3.7.0 (17th January 2022)

+
    +
  • Moved '--fromfile' option to '--includefrom'. '--fromfile' still works, but is considered deprecated
  • +
  • Moved '--includefiles' option to '--includepattern'. '--includefiles' still works, but is considered deprecated
  • +
  • Moved '--excludefiles' option to '--excludepattern'. '--excludefiles' still works, but is considered deprecated
  • +
  • Added '--excludefrom' option similar to '--includefrom' only the opposite (Thank you to user 'TomFury' for suggesting this)
  • +
  • Skyscraper will now ignore any subfolders within the input folder where a file called '.skyscraperignore' is found (Thank you to user 'sromeroi' for suggesting this)
  • +
  • Added platform 'easyrpg', only usable using the 'screenscraper' scraping module (Thank you to user 'zerojay' for suggesting this)
  • +
+

Version 3.6.16 (9th November 2021)

+
    +
  • Added platform 'moto' (Thank you to user 'Yserra' for suggesting it)
  • +
+

Version 3.6.15 (25th August 2021)

+
    +
  • Added 'chd' extension to 'atomiswave' platform (Thank you to user 'smeegoan' for reporting this)
  • +
  • Fixed bug that caused 'T000000' to be added multiple times when skipping entries in ES gamelists (Thank you to user 'sleve_mcdichael' for reporting this)
  • +
+

Version 3.6.14 (5th August 2021)

+
    +
  • Added 'windows 3.x' as alias to Mobygames scraping module (Thank you to user 'ecto' for reporting this)
  • +
+

Version 3.6.13 (2nd June 2021)

+
    +
  • Added 'mediaFolderHidden' EmulationStation specific config option that will set the media folder to 'PLATFORM/.media' when set to true. This can speed up EmulationStation initial loading when using slow storage such as a network file system (Thank you to user 'XenuIsWatching' for suggesting this)
  • +
+

Version 3.6.12 (15th May 2021)

+
    +
  • Removed 'Simple Mode' as it was broken and deprecated. Use RetroPie script instead
  • +
  • Removed A LOT of deprecated (and hidden) CLI options
  • +
+

Version 3.6.11 (6th May 2021)

+
    +
  • '--fromfile' now accepts both relative and absolute path to filename (Thank you to user 'sleve_mcdichael' for reporting this)
  • +
  • Added platforms 'atarijaguarcd', 'pcenginecd' and 'channelf' (Thank you to user 'XenuIsWatching' for providing info)
  • +
+

Version 3.6.10 (12th April 2021)

+
    +
  • Added 'naomi' as platform
  • +
  • Added 'msx2' as platform
  • +
  • Added 'atomiswave' as platform
  • +
+

Version 3.6.9 (1st April 2021)

+
    +
  • Probably fixed --flags unpack which broke with 3.6.8 due to deprecated Qt function calls
  • +
  • Probably fixed video conversion which broke with 3.6.8 due to deprecated Qt function calls
  • +
+

Version 3.6.8 (31st March 2021)

+
    +
  • Added 'cacheRefresh' config.ini option and added it to default config.ini for 'esgamelist' module (Thank you to user 'penx' for suggesting this)
  • +
  • 'ScreenScraper APIv2 returned invalid / empty Json' message now no longer includes message to attach the error json file unless it's more than 64 bytes long
  • +
  • Added 'ca' to region priority list (Thank you to user 'joaoluizcarvalho' for pointing this out)
  • +
  • Added 'de', 'fr' and 'es' to default language priority list
  • +
  • Added 'players' resource to '--cache report:missing' CLI option (Thank you to user 'peligwe' for reporting this)
  • +
  • Added 'theInFront' config.ini option and 'theinfront' CLI flag (Thank you to user 'sleve_mcdichael' for suggesting this)
  • +
  • 'unattend' and 'unattendSkip' can now be used in a scraping module section in config.ini (Thank you to user 'sleve_mcdichael' for reporting this)
  • +
  • Added Qt version pre-processor checks for deprecated Qt methods to avoid compilation warning on newer Qt installations
  • +
+

Version 3.6.7 (27th January 2021)

+
    +
  • Added '*.rvz' extension for 'wii' and 'gc' platforms (Thank you to user 'tcamargo' for reporting this)
  • +
  • Added '*.chd' extension for '3do' platform (Thank you to user 'Roudaku' for reporting this)
  • +
  • Added 'openbor' platform. Exists in RetroPie as experimental platform (as requested by user 'hahnmt')
  • +
  • 'neogeo' platform now uses 'box-2D' as cover instead of 'flyer' when scraping with the ScreenScraper module (Thank you to user 'retro81' for suggesting this)
  • +
+

Version 3.6.6 (21st December 2020)

+
    +
  • The 'esgamelist' module now looks for the 'gamelist.xml' file in the path set with '-g' instead of the path set with '-i' (Thank you to user 'c0d3h4x0r' for reporting this and helping me debug it)
  • +
+

Version 3.6.5 (18th December 2020)

+
    +
  • Now uses one shared networking instance, instead of one per thread
  • +
  • 'daphne' platform entries now added as '' entries instead of '' entries (Thank you to user 'clyde' for helping figure this out)
  • +
  • Added 'nocropblack' cli flag and 'cropBlack="BOOL"' config.ini option that allows user to disable cropping of black borders around screenshots when compositing the final gamelist artwork (Thank you to user 'WindyWinston' for suggesting this)
  • +
+

Version 3.6.1 (27th October 2020)

+
    +
  • Added 'amigacd32' as a separate platform just because (Thank you to user 'HoraceAndTheSpider' for resetting The Matrix)
  • +
+

Version 3.6.0 (25th October 2020)

+
    +
  • Changed 'SecsSinceEpich' to 'MSecsSinceEpich' to support pre-5.8 Qt versions (Thank you to user 'Hazza4569' for reporting this)
  • +
+

Version 3.5.9 (24th October 2020)

+
    +
  • Implemented the new IGDB v4 authentication method. IGDB will now work again, and requires free credentials. Read more about that here
  • +
  • Improved memory consumption when handing entries back to main thread
  • +
  • Added '.au3' file extension to id script exception list (Thank you to 'o0alucard0o' for reporting this)
  • +
+

Version 3.5.8 (16th August 2020)

+
    +
  • Now supports spaces in Attract-Mode media paths (Thank you to user 'o0alucard0o' for pointing this out)
  • +
+

Version 3.5.7 (31st July 2020)

+
    +
  • Now uses regular expressions when converting text to Pegasus format for better compatibility
  • +
  • Removed 'extensions' when generating Pegasus game list, as it isn't required
  • +
+

Version 3.5.6 (30th July 2020)

+
    +
  • Added 'neogeocd' platform (Thank you to 'igno2k' and 'meveric' for suggesting it)
  • +
  • 'screenscraper' module now supports region for screenshots (Thank you to 'cameronhimself' for pointing this out)
  • +
  • Low disk space check for cache is now calculated from cache folder location
  • +
  • Skyscraper initial output now shows frontend and extras when generating game lists
  • +
  • Changed 'gamelistFolder' config.ini option to 'gameListFolder' but kept old variable name for backwards compatibility
  • +
  • Added 'gameListBackup' config.ini option to allow for game list backups being created on each run (Thank you to 'igno2k' for suggesting it)
  • +
  • Attract-Mode now recognizes '$HOME' in emulator file
  • +
  • Refactored Attract-Mode media folder getters
  • +
  • Fixed rare faulty matches from 'esgamelist' module in cases where last part of 'path' was a match with filename (Thank you to 'akamming' for reporting this)
  • +
  • Exporting for Pegasus frontend now supports preservation of old game list key-value pairs (Thank you to user 'HugLifeTiZ' for suggesting this)
  • +
  • '--cache report' now also adheres so excludeFiles and includeFiles (Thank you to user 'spilinek' for pointing this out)
  • +
+

Version 3.5.5 (18th May 2020)

+
    +
  • Fixed bug in 'igdb' module where it still used old hardcoded key (Thank you to 'stlbluesfan' for reporting this)
  • +
+

Version 3.5.4 (14th May 2020)

+
    +
  • Hints file converted to xml (hints.txt -> hints.xml)
  • +
  • Moved documentation resources into 'docs' folder
  • +
  • Added 'videoPreferNormalized' config.ini option for use with the 'screenscraper' module
  • +
+

Version 3.5.3 (11th May 2020)

+
    +
  • MAJOR: Added video conversion support through config.ini option 'videoConvertCommand="COMMAND %i %o"' option. See docs for more details
  • +
  • Removed deprecated <cover> node and replaced it with <thumbnail> for ES gamelist generation (Thank you to 'yisraeldov' for letting me know)
  • +
  • Added simple error handling for media network requests
  • +
  • '--flags help' and '--cache help' no longer requires '-p PLATFORM' to be set
  • +
  • Changed 'worldofspectrum' scraping module to use https
  • +
+

Version 3.5.2 (29th April 2020)

+
    +
  • Hid deprecated options from '--help'. Deprecated options still work just aren't shown
  • +
  • Updated 'Simple Mode' to use the new '--flags' options
  • +
+

Version 3.5.1 (28th April 2020)

+
    +
  • Fixed minor bug in 'daphne' when it only has one rom in 'daphne/roms'
  • +
+

Version 3.5.0 (27th April 2020)

+
    +
  • For 'screenscraper', 'thegamesdb', 'mobygames' and 'arcadedb' Skyscraper will now no longer fetch the artworks if user has requested it to remain uncached (Thank you to user 'herbymachine' for the request)
  • +
  • Added '--skipexistingmedia' option to allow for faster gamelist generation in cases where some artwork already exists in the gamelist media folder (Thank you to 'jacrify' for suggesting this)
  • +
  • Added '--flags' command-line option that collects certain flag-like options into one
  • +
  • Added deprecated warnings to all options that are now moved to the '--flags' option
  • +
  • Added '*.cue' extension to '3do' platform (Thank you to 'therealdealneil' for the suggestion)
  • +
  • Implemented a hack for the 'daphne' platform that will add the game info to the 'game.daphne' folders instead of the 'game.zip' files (Thank you to 'AlCzervik' for pointing this out)
  • +
+

Version 3.4.10 (20th April 2020)

+
    +
  • Re-implemented double-check when saving artwork to cache to avoid 0-byte files on systems with a broken libpng (Thank you to 'jacrify' for letting me know)
  • +
+

Version 3.4.9 (17th April 2020)

+
    +
  • Added 'wiiu' platform (Thank you to 'minilandl' for suggesting it)
  • +
+

Version 3.4.8 (14th April 2020)

+
    +
  • Lowered sanity check limit of 1024 bytes for artwork from 'screenscraper' to 256 bytes (Thank you to user 'Paul-Colucci' for reporting this)
  • +
+

Version 3.4.7 (12th April 2020)

+
    +
  • If '--onlymissing' is set the initial first found rom check is ignored (Thank you to user 'Paul-Colucci' for reporting this)
  • +
+

Version 3.4.6 (9th April 2020)

+
    +
  • Implemented nameTemplate as config.ini option. See CONFIGINI.md doc for explanation (Thank you to user 'vg8020' for suggesting this)
  • +
+

Version 3.4.5 (7th April 2020)

+
    +
  • Minor fix to 'screenscraper' romnom query
  • +
+

Version 3.4.4 (7th April 2020)

+
    +
  • Remove '.bin' support for 'saturn' platform since '.cue' is already in there. This caused double-entries in gamelists (Thank you to user 'joaoluizcarvalho' for pointing this out)
  • +
  • Now handles faulty login (incorrectly entered in config.ini) better with 'screenscraper'
  • +
  • Fixed bug that caused 'pegasus' launch command to be entered incorrectly (Thank you to user 'Itetrel' for letting me know)
  • +
  • Added '--onlymissing' CLI option which allows users to skip files that already have any kind of data cached from any source
  • +
  • Now clearly states what happened when a request timed out when using 'screenscraper' (Thank you to 'seriema' for pointing this out)
  • +
+

Version 3.4.3 (14th March 2020)

+
    +
  • All arcade platforms now use 'flyer' from 'screenscraper' for cover artwork instead of 'box-2D' (Thank you to user 'aidy80s' for suggesting this)
  • +
+

Version 3.4.2 (26th February 2020)

+
    +
  • Improved filename versioning removal for optimal query generation
  • +
  • Updated 'mameMap.csv' with latest Mame v0.218 listings
  • +
  • Added support for 'wheel' artwork type in 'arcadedb' scraping module
  • +
  • Now supports symlinked paths inside of the input folder (Thank you to rhunjadi for pointing this out)
  • +
+

Version 3.4.1 (11th February 2020)

+
    +
  • Further optimized artwork space requirements. Now checks if original takes up less space than resized artwork, then forces use of original for those cases
  • +
  • The 'thegamesdb' module now also supports wheel and marquee for the games that have them (Thank you to 'tv21' for pointing this out)
  • +
  • Updated developer and publisher json list for 'thegamesdb'
  • +
+

Version 3.4.0 (4th February 2020)

+
    +
  • Disabled config.ini migration as most people should be migrated now
  • +
  • Added 'launcher' option to 'simple mode' when using 'pegasus' frontend
  • +
  • Added 'excludeFiles' config option that allows excluding certain files when scraping (Thank you to 'timothybrown' for suggesting this)
  • +
  • Added 'includeFiles' config option that allows only including certain files when scraping
  • +
  • Added '--excludefiles' cli option that allows excluding certain files when scraping
  • +
  • Added '--includefiles' cli option that allows only including certain files when scraping
  • +
  • If 'noresize' is set all images are now saved to cache in their original format and size instead of always converting them to PNG's (Thank you to 'krcroft' for requesting this)
  • +
  • Added 'jpgQuality' config option that sets the default JPG quality (0-100) when '--noresize' is NOT set. Screenshots and images with transparency are still saved as PNG's.
  • +
  • Upped 'screenscraper' request limiter to 1.2 seconds per request to avoid 'maximum threads per minute reached' error message
  • +
+

Version 3.3.8 (12th January 2020)

+
    +
  • Fixed bug in platform alias for 'pc (microsoft windows)' which was missing the last ')' bracket (Thank you to 'o0alucard0o' for reporting this)
  • +
+

Version 3.3.7 (8th January 2020)

+
    +
  • Added the option to add frontend specific config sections to set artwork xml and other relevant configuration parameters per frontend
  • +
+

Version 3.3.6 (5th January 2020)

+
    +
  • Skipped file now includes platform in the filename (Thanks to 'parker-hemphill' for suggesting this)
  • +
  • Now handles the '**' error message from ScreenScraper (Thank you to 'GiulioCentorame' for reporting this)
  • +
  • Updated 'thegamesdb' to include the 'v1' prefix (Thank you to 'Clyde' for letting me know)
  • +
+

Version 3.3.5 (15th December 2019)

+
    +
  • Updated 'screenscraper' API to support new system name json format (Thank you to 'Micket' for letting me know)
  • +
+

Version 3.3.4 (8th December 2019)

+
    +
  • 'igdb' module global key has been removed from Skyscraper. They have moved to free personal keys instead (this is good news!). To get one, register at https://api.igdb.com (Thank you to 'stlbluesfan' for letting me know)
  • +
  • Fixed disk space check so it also works for 'attractmode' frontend
  • +
  • Updated 'attractmode' to save overviews in correct location and format (Thanks to 'barolo' for letting me know, thanks to 'teeedubb' and 'barolo' for testing)
  • +
  • For 'attractmode' added semicolon detection for 'snaps'/'video' parsing for determining resource folders (Thanks to 'barolo' for pointing this out)
  • +
  • Removed rom limit on 'igdb' module
  • +
+

Version 3.3.3 (21st November 2019)

+
    +
  • Added support for the Nintendo 3DS platform as '3ds' (Thank you to 'barolo' for suggesting this)
  • +
  • Fixed system id for 'coleco' platform when using the 'screenscraper' module causing roms not to be recognized (Thank you to 'DMahlon' for reporting this)
  • +
+

Version 3.3.2 (4th November 2019)

+
    +
  • Added '.gcm' file extension to Gamecube platform (Thank you to 'mrsilver76' for suggesting this)
  • +
  • The '-e' option can now be used to optionally set the Pegasus launch command (Thank you to Sixdd6' for suggesting this)
  • +
  • Refined 'Found/Skipped' output a bit
  • +
+

Version 3.3.1 (17th October 2019)

+
    +
  • Added new '--cache edit:new=' option for efficient batch adding of a resource of a certain type
  • +
  • Improved ctrl+c handling
  • +
  • Protected cache write calls from ctrl+c obstruction
  • +
+

Version 3.3.0 (15th September 2019)

+
    +
  • MAJOR: File identification now uses new quick id method for up to 75% faster processing (Thank you to 'langest' for finally making me look into this)
  • +
  • Added 'pc98' platform (Thank you to 'leosmeira' for suggesting it)
  • +
  • Added 'pokemini' platform (Thank you to 'leosmeira' for suggesting it)
  • +
  • Renamed all 'sha1' file id's to 'id' as sha1 was misleading
  • +
  • Changed relevant defines to constexpr
  • +
  • ScreenScraper now always prioritizes the 'video-normalized' above 'video' (Thank you to 'JuanVCS' for suggesting this)
  • +
  • Fixed bug in ScreenScraper retry code which made it retry more than necessary
  • +
+

Version 3.2.8 (31st August 2019)

+
    +
  • --cache report now supports symlinks (Thank you to 'vanativo' for reporting this)
  • +
+

Version 3.2.7 (31st August 2019)

+
    +
  • The '--cache report' option now adhere to the '--nosubdirs' option (Thank you to 'vanativo' for reporting this)
  • +
+

Version 3.2.6 (24th August 2019)

+
    +
  • Fixed bug where entering a correct fronted in simple mode would loop the question (Thank you to 'thadboyd' for reporting this)
  • +
+

Version 3.2.5 (21st August 2019)

+
    +
  • EmulationStation module now converts to '' entry for all supported platform suffixes (Thank you to 'kgonepostl' for reporting this)
  • +
+

Version 3.2.4 (19th August 2019)

+
    +
  • Implemented JSON flaw workaround for the ScreenScraper module
  • +
+

Version 3.2.3 (13th August 2019)

+
    +
  • MAJOR: Added support for the 'Pegasus' frontend (set with '-f pegasus')
  • +
  • Now checks for, and removes, double bracket notes in final game list title
  • +
  • Fixed minor bug in the 3D gamebox effect renderer
  • +
  • Completely transparent images are no longer saved when compositing (Thank you to metallkopf for getting me to finally fix this)
  • +
  • Optimized the final game list assembling code to use game entry references instead of copies
  • +
  • Optimized all cache resource iterations to use const references instead of copies
  • +
  • Optimized the entire codebase by removing all Qt-centric foreach iterate-by-copy to use references instead
  • +
  • Fixed bug where 'screenscraper' would only look for ESRB age classification
  • +
  • Potential faulty JSON replies from Screenscraper are now saved to '/home/USER/.skyscraper/screenscraper_error.json' for easier debugging
  • +
+

Version 3.2.2 (3rd August 2019)

+
    +
  • Added 'bat' scripts to sha1 special handling list (please purge platforms using 'bat' files and rescrape)
  • +
  • Now discards 'ZZZ(notgame)' results from ScreenScraper
  • +
  • Fixed double-quote issue when reading titles from scummvm.ini
  • +
  • Made location of scummvm.ini configurable in config.ini
  • +
  • Fixed bug in roman and integer numeral conversion functions
  • +
  • Rewrote the entire 'screenscraper' module to use JSON instead of XML
  • +
+

Version 3.2.1 (15th July 2019)

+
    +
  • Fixed bug in 'mobygames' module that could cause an endless loop while getting cover
  • +
+

Version 3.2.0 (8th July 2019)

+
    +
  • Improved output of edit mode to show queue size and initial prioritized resources
  • +
  • Now clearly shows missing resources when adding resources in editing mode
  • +
  • Made '--cache vacuum' more verbose at verbosity level 2 or higher
  • +
  • Removed platform check for screenscraper module as it wasn't really necessary
  • +
  • Added 'videoSizeLimit' config variable (Thank you to 'silverfr0st' for suggesting this)
  • +
  • Now also processes brackets for skipped games (Thanks to 'hexive' for pointing this out)
  • +
  • Added 'arcadia' and 'astrocde' MESS platforms (Thank you to 'raphkoster' for suggesting these)
  • +
  • Added '--cache report:missing=' option (Thank you to JaymesRS for suggesting this)
  • +
  • Added '--fromfile' option allowing you to work on a subset of files taken from a file. Compatible with files created using the '--cache report' option
  • +
  • MobyGames module now includes 'platform' id query when relevant. Should improve result accuracy
  • +
  • Added region support for MobyGames cover artwork
  • +
  • Added A LOT of regions to region autodetection in general
  • +
  • Now no longer creates media folders when in resource gathering mode
  • +
  • Moved help texts for '--cache' options to '--cache help'
  • +
+

Version 3.1.4 (23rd March 2019)

+
    +
  • Refactored network communication class
  • +
  • Added request download size limit to work around possible bug in ScreenScraper API
  • +
+

Version 3.1.3 (12th March 2019)

+
    +
  • Improved 'screenscraper' error handling
  • +
+

Version 3.1.2 (10th March 2019)

+
    +
  • Added support for Moonlight '*.ml' extension for all platforms
  • +
  • Added 'ps2' platform for use with Moonlight and all other supported Ps2 extensions
  • +
  • Added 'steam' platform for use with Moonlight '*.ml' files
  • +
  • Fixed minor bug in CRC checksum calculator for 'screenscraper'
  • +
  • Improved workarounds for ScreenScraper XML errors
  • +
+

Version 3.1.1 (8th March 2019)

+
    +
  • Worked around a bug in the 'screenscraper' xml output improving stability
  • +
  • Added limiter to media getter functions for 'screenscraper' which should improve stability
  • +
+

Version 3.1.0 (2th March 2019)

+
    +
  • MAJOR: Added '--cache edit' command line option which allows viewing, editing and deleting cached resources for the roms in the queue. Narrow the queue down by providing file names on command line or by using '--startat ' and '--endat '
  • +
  • Added 'zx81' platform. Note! The only module that supports it is the 'screenscraper' module
  • +
+

Version 3.0.5 (26th February 2019)

+
    +
  • Fixed and improved ES metadata preservation code
  • +
+

Version 3.0.4 (18th February 2019)

+
    +
  • Improved Amiga CD32 disc image support to prep for the upcoming support in Amiberry (Thank you to 'HoraceAndTheSpider' for the heads up)
  • +
  • Added check for Qt5.4.0 and ignores QStorageInfo include if requirement not met (Thank you to 'ByteThis' for reporting this issue)
  • +
  • Improved work around to fix invalid XML returned by ScreenScraper
  • +
+

Version 3.0.3 (16th February 2019)

+
    +
  • Worked around a flaw in the returned screenscraper xml results which caused many games to not be recognized due to invalid xml being returned (Thank you to 'Brunnis' for reporting this)
  • +
  • Improved SIGINT handler to now allow unclean exit if user is VERY adament about it
  • +
+

Version 3.0.2 (14th February 2019)

+
    +
  • Upped the rom limit from 5 to 35 for the "igdb" module
  • +
  • Upped the rom limit from 25 to 35 for the "mobygames" module
  • +
  • Added media cache config options to module section
  • +
  • Add Sharp X1 platform as "x1"
  • +
  • Now exits nicely when running low on disk space
  • +
  • Added 'spaceCheck=' to config.ini
  • +
  • Fixed crash when using '--startat' and '--endat' where the '--endat' file name came before the '--startat' file name. Note! What 'ls' reports in alphabetical order is not always what Skyscraper see as it it locale specific. So be aware of this. A huge thanks to 'Gemba' for taking the time to investigate this bug thoroughly.
  • +
  • Fixed bug in game list metadata preservation when using relativePaths and '' entries (thank you to 'HumanRob' for reporting this)
  • +
  • Fixed game list entries skipping for 'relativePaths' and '' instances
  • +
  • Skyscraper now saves the cache and exits nicely on ctrl+c (SIGINT) (thanks to 'krcroft' for pointing this out)
  • +
  • The 'screenscraper' module now includes 'systemeid' in the query for better results
  • +
  • Now skips the game list assembling when in gathering mode
  • +
  • Now skips cache saving when in game list generation mode
  • +
  • Output now says whether it was a gathering run or a game list generation run
  • +
+

Version 3.0.1 (27th January 2019)

+
    +
  • Added note for '--query' option when user forgets to add a file name on command line
  • +
  • Fixed minor bugs in aliasMap, mameMap and whdLoad map lookups
  • +
+

Version 3.0.0 (23rd January 2019)

+
    +
  • Game list creation and media compositing now only processed when leaving out the "-s" option
  • +
  • Now converts textual "\n" in descriptions to newlines. Useful when importing textual data (Thank you to "scthirouin" for requesting it)
  • +
  • Added disk space check. Now aborts if disk has less than 200 megs left
  • +
  • Updated "igdb" API til APIv3000. Key now global for all users
  • +
  • Fixed ES game list release dates by adding "T000000" (Thank you to "michaelbaudino" for noticing)
  • +
  • Added "subdirs" to config.ini (Thank you to "otherguy" for noticing it was missing)
  • +
  • "--startat" and "--endat" now supports current workdir file paths
  • +
  • Changed "allowExtension" config variable to "addExtensions"
  • +
  • Changed "noHints" config variable to "hints"
  • +
  • Changed "noBrackets" config variable to "brackets"
  • +
  • Changed "noResize" config variable to "resize"
  • +
  • Removed "--updatedb" cli option as it was deprecated (use "--cache refresh" instead)
  • +
  • Removed "--nolocaldb" cli option as it is irrelevant after 3.0.0 usage changes
  • +
  • Changed "--allowext" cli option to "--addext"
  • +
  • Removed "localDb"/"cache" config sections and moved options to "main"
  • +
  • Removed "userCreds" option from main and platform sections in config.ini. They should be set in scraping module section instead
  • +
  • Removed "scraper" config.ini variable as it made no sense to ever set it like that
  • +
  • Changed "startat" config.ini option to "startAt" to conform with style of other variables
  • +
  • Changed "endat" config.ini option to "endAt" to conform with style of other variables
  • +
  • Fixed "--relative" option when combined with roms in subdirs
  • +
  • "--unattend" options now answers "y" to all "--cache purge:" prompts automatically
  • +
  • Added "regionPrios" and "langPrios" variables to config.ini for main and platform sections
  • +
  • Added auto-migration from old config.ini format to v3.0.0 format
  • +
  • Added auto-migration from "dbs" folder to "cache" folder
  • +
  • Optimized "--cache validate" A LOT
  • +
  • Now converts ES game list entries to entries if a .cue or .m3u file is found in a subfolder. This causes the information and artwork to be shown when selecting the folder in ES rather than the .cue or .m3u file inside the subfolder. This is very useful for psx scraping
  • +
+

Version 2.9.5 (22nd December 2018)

+
    +
  • MAJOR: Added option "--purgedb vacuum" which vacuums all resources not related to your current romset. Remember to make backups of your cache before using this
  • +
  • MAJOR: Added option "--purgedb all" that purges all resources for the selected platform. Remember to make backups of your cache before using this
  • +
  • MAJOR: Added "--symlink" option which forces cached videos to be symlinked to destination instead of being copied when scraping with the "localdb" scraping module
  • +
  • MAJOR: Added "esgamelist" emulationstation gamelist.xml scraping module. Contributed by "mgerhardy". Rewritten by me to better conform to Skyscraper design
  • +
  • MAJOR: Added aliasMap.csv that forces the use of a title alias when searching for specific filenames
  • +
  • Removed version bracket tag for Amiga lha files
  • +
  • Improved getCompareTitle for mame games and lha files
  • +
  • Code cleanup for sqrNotes
  • +
  • Added the "ti99" platform. (Thank you to "jhbeskow" for suggesting it)
  • +
+

Version 2.9.1 (15th December 2018)

+
    +
  • Fixed bug in "screenscraper" module when using --unpack where files exceeding limits would get checksummed as the "Point Blank" game. (Thank you to CookiePLMonster for reporting this issue)
  • +
  • Raised --unpack size limit to 8 megs
  • +
+

Version 2.9.0 (13th December 2018)

+
    +
  • MAJOR: Now looks up Amiga lha files in the Amiberry "whdload_db.xml" and retrieves data from "openretro.org" based on the uuid from the xml
  • +
  • Added search based fallback pass for Amiga when game isn't found via uuid
  • +
  • Added "" effect to compositor. Check here for more info (Thank you to "jakejm79" for suggesting this)
  • +
  • "mobygames" module now uses https
  • +
  • Fixed bug in "openretro" module where "developer" would potentially scrape wrong under certain circumstances
  • +
  • Improved "description" scraping for "openretro" module
  • +
  • Improved bracket tag handling for Amiga lha files A LOT
  • +
  • Fixed minor pass 2 bug when using search based sources
  • +
  • Added a pass for integer to roman conversion for search based sources (eg. "4" converts to "IV")
  • +
  • "the" matching now uses regular expression for better precision
  • +
  • Added "mame-*" platforms to "mameMap" name load list
  • +
+

Version 2.8.3 (8th December 2018)

+
    +
  • Added "region" and "lang" options to "[platform]" section of config.ini
  • +
  • Added "noResize" variable to "[localDb]" config.ini section
  • +
  • Fixed bug where "--query" would stop working when region was autodetected
  • +
+

Version 2.8.2 (7th December 2018)

+
    +
  • Now autodetects region from filename and prioritizes it when scraping with "screenscraper" unless user has forced region with "--region" option (thank you to "movisman" for suggesting this)
  • +
  • Fixed issue where "wheel" artwork would not be fetched for specified region even though it exists
  • +
  • Added ".cue" file extension to Amiga
  • +
+

Version 2.8.1 (25th November 2018)

+
    +
  • Now loads tgdb json files from "~/.skyscraper"
  • +
+

Version 2.8.0 (4th November 2018)

+
    +
  • Now tries to preserve EmulationStaton gamelist.xml metadata unless new incoming data exists
  • +
  • Now tries to preserve AttractMode romlist metadata unless new incoming data exists
  • +
  • Improved terminal output a bit with fflush and progress dots
  • +
  • Now looks for 'definitions.dat' in base import folder before trying import/[platform] folder
  • +
+

Version 2.7.8 (21st October 2018)

+
    +
  • Added CRC32 checksum attribute to 'screenscraper' pass
  • +
  • Changed the 'import' folder to also look into 'import/[platform]' if it exists so users can create imports per platform. Old method also works (Thanks to tacodog for suggesting this)
  • +
  • Added 'mpixels' attribute to compositor layers that allows users to scale images to a desired area size instead of using width and height (Thank you to Vynce for suggesting this)
  • +
+

Version 2.7.7 (10th October 2018)

+
    +
  • Fixed instances where '--query' would be applied to all roms if a faulty filename was given on command line (Thank you to Razile89 for reporting this)
  • +
+

Version 2.7.6 (7th October 2018)

+
    +
  • Added 'Found', 'Not found' and 'Total' to output
  • +
  • Re-added 'aga' Amiga detection for OpenRetro after refactoring
  • +
  • Added 'resource' attribute to 'output' nodes which enables user to directly export wheel as marquee by adding a node to artwork.xml
  • +
+

Version 2.7.5 (21st September 2018)

+
    +
  • Fixed a bug where 'brackets="false"' in config.ini would be flipped (Thanks to Vynce for reporting this)
  • +
  • Completely refactored pass procedures for cleaner code and to enable '--query' option
  • +
  • Added '--query' command line option. This option requires a single rom file to be passed on command line aswell, otherwise it will be ignored (Thank you to AnalogHero and Vynce for suggesting this)
  • +
  • Added scrapers to 'psx' and 'pc' platforms when using Simple Mode
  • +
+

Version 2.7.4 (11th September 2018)

+
    +
  • Added '--unpack' command line option and config.ini options which enables the calculation of the checksum of the file inside 7z and zip files instead of the base file itself. Only relevant for 'screenscraper' module and requires 'p7zip' installed. ($ sudo apt-get install p7zip-full for Debian base systems)
  • +
  • Added 'inputFolder' to [main] section of config.ini
  • +
  • Added 'gamelistFolder' to [main] section of config.ini
  • +
  • Added 'mediaFolder' to [main] section of config.ini
  • +
+

Version 2.7.3 (4th September 2018)

+
    +
  • Improved image cropping to now also crop black borders, but only for screenshots (Thank you to 'chipsnblip' for suggesting this)
  • +
  • Made 'import' base folder configurable in config.ini
  • +
  • Fixed bug in 'import' scraping module that caused dummy titles to be saved to localdb when scraping media resources
  • +
  • Changed 'curl' to 'wget' in update_skyscraper.sh script to avoid curl requirement
  • +
+

Version 2.7.2 (2nd September 2018)

+