diff --git a/README.md b/README.md index 3870d3c..b194101 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,3 @@ -### What is new -* Implemented S3FileChannel and adopted few places so that the FileSystem can be plugged in Apache Mina SFTP -* Upgraded AWS SDK to 1.11.125 -* Added propagation of all passed in env's - -### How to use in Apache MINA -```ruby -public FileSystemFactory createFileSystemFactory(String bucketName) throws IOException, URISyntaxException { - FileSystem fileSystem = FileSystems.newFileSystem(new URI("s3:///"), env, Thread.currentThread().getContextClassLoader()); - String bucketPath = fileSystem.getPath("/" + bucketName); - - return new VirtualFileSystemFactory(bucketPath); -} -``` --- - An **Amazon AWS S3** FileSystem Provider **JSR-203** for Java 7 (NIO2) Amazon Simple Storage Service provides a fully redundant data storage infrastructure for storing and retrieving any amount of data, at any time. @@ -22,27 +6,27 @@ This project provides a first API implementation, little optimized, but "complet [![Build Status](https://travis-ci.org/Upplication/Amazon-S3-FileSystem-NIO2.svg?branch=master)](https://travis-ci.org/Upplication/Amazon-S3-FileSystem-NIO2/builds) [![Coverage Status](https://coveralls.io/repos/Upplication/Amazon-S3-FileSystem-NIO2/badge.png?branch=master)](https://coveralls.io/r/Upplication/Amazon-S3-FileSystem-NIO2?branch=master) [![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.upplication/s3fs/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.upplication/s3fs) -## How to use +#### How to use -### Download from Maven Central +##### Download from Maven Central ```XML com.upplication s3fs - 1.5.4 + 2.0.0 ``` And add to your META-INF/services/java.nio.file.spi.FileSystemProvider (create if not exists yet) a new line like this: com.upplication.s3fs.S3FileSystemProvider. -### S3FileSystem and AmazonS3 settings +##### S3FileSystem and AmazonS3 settings All settings for S3FileSystem and for the underlying AmazonS3 connector library can be set through System properties or environment variables. Possible settings can be found in com.upplication.s3fs.AmazonS3Factory. -### Using service locator and system vars +#### Using service locator and system vars Check that s3fs_access_key and s3fs_secret_key system vars are present with the correct values to have full access to your amazon s3 bucket. @@ -52,11 +36,11 @@ Use this code to create the fileSystem and set to a concrete endpoint. FileSystems.newFileSystem("s3:///", new HashMap(), Thread.currentThread().getContextClassLoader()); ``` -### Using service locator and amazon.properties in the classpath +##### Using service locator and amazon.properties in the classpath Add to your resources folder the file amazon.properties with the content: -s3fs_access_key=access key -s3fs_secret_key=secret key +s3fs_access_key=access-key +s3fs_secret_key=secret-key Use this code to create the fileSystem and set to a concrete endpoint. @@ -64,7 +48,7 @@ Use this code to create the fileSystem and set to a concrete endpoint. FileSystems.newFileSystem("s3:///", new HashMap(), Thread.currentThread().getContextClassLoader()); ``` -### Using service locator and programatically authentication +##### Using service locator and programatically authentication Create a map with the authentication and use the fileSystem to create the fileSystem and set to a concrete endpoint. @@ -96,7 +80,7 @@ Complete settings lists: * s3fs_user_agent * s3fs_amazon_s3_factory -### Set endpoint to reduce data latency in your applications +##### Set endpoint to reduce data latency in your applications ```java // Northern Virginia or Pacific Northwest @@ -113,7 +97,59 @@ FileSystems.newFileSystem("s3://s3-eu-west-1.amazonaws.com/", env, Thread.curren For a complete list of available regions look at: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region -## Features: +##### How to use in Apache MINA + +```java +public FileSystemFactory createFileSystemFactory(String bucketName) throws IOException, URISyntaxException { + FileSystem fileSystem = FileSystems.newFileSystem(new URI("s3:///"), env, Thread.currentThread().getContextClassLoader()); + String bucketPath = fileSystem.getPath("/" + bucketName); + + return new VirtualFileSystemFactory(bucketPath); +} +``` + +##### How to use in Spring + +Add to classpath and configure: + +```java +@Configuration +public class AwsConfig { + + @Value("${upplication.aws.accessKey}") + private String accessKey; + + @Value("${upplication.aws.secretKey}") + private String secretKey; + + @Bean + public FileSystem s3FileSystem() throws IOException { + Map env = new HashMap<>(); + env.put(com.upplication.s3fs.AmazonS3Factory.ACCESS_KEY, accessKey); + env.put(com.upplication.s3fs.AmazonS3Factory.SECRET_KEY, secretKey); + + return FileSystems.newFileSystem(URI.create("s3:///"), env, Thread.currentThread().getContextClassLoader()); + } +} +``` + +Now you can inject in any spring component: + +```java +@Autowired +private FileSystem s3FileSystem; + +``` + +##### What is new 2.0.0 + +* Preserve URI with end slash #76 +* Removed META-INF/services/java.nio.file.spi.FileTypeDetector #78 +* Bucket are filestores and root directories for a bucket is the bucket itself. +* getFileName for a root Path is ```null``` +* Improved S3Path Tests + +#### Features: * Copy and create folders and files * Delete folders and files @@ -123,16 +159,16 @@ For a complete list of available regions look at: http://docs.aws.amazon.com/gen * List buckets for the client * Multi endpoint fileSystem -## Roadmap: +#### Roadmap: * Performance issue (slow querys with virtual folders, add multipart submit...) * Disallow upload binary files with same name as folders and vice versa -## Out of Roadmap: +#### Out of Roadmap: * Watchers -## How to contribute +#### How to contribute Clone the github repository: @@ -153,6 +189,6 @@ s3fs_access_key=your access key for test Thats all, now you can run the test with the command: `mvn test` or `mvn integration-test -Pintegration-tests` -## LICENSE: +#### LICENSE: Amazon S3 FileSystem NIO2 is released under the MIT License. diff --git a/pom.xml b/pom.xml index 3515a4d..734c10a 100644 --- a/pom.xml +++ b/pom.xml @@ -4,7 +4,7 @@ com.upplication s3fs jar - 1.5.4 + 2.0.0 s3fs S3 filesystem provider for Java 7 https://github.com/Upplication/Amazon-S3-FileSystem-NIO2 @@ -16,6 +16,12 @@ + + + jcustovic + Jan Čustović + jan.custovic@gmail.com + heikkipora Heikki Pora @@ -52,7 +58,7 @@ UTF-8 2.7.1 1.7.5 - 1.11.125 + 1.11.232 18.0 1.5 1.3.9 diff --git a/src/main/java/com/upplication/s3fs/S3FileStore.java b/src/main/java/com/upplication/s3fs/S3FileStore.java index 724578c..245e7f9 100644 --- a/src/main/java/com/upplication/s3fs/S3FileStore.java +++ b/src/main/java/com/upplication/s3fs/S3FileStore.java @@ -91,7 +91,7 @@ private Bucket getBucket(String bucketName) { } public S3Path getRootDirectory() { - return new S3Path(fileSystem, this, ImmutableList.of()); + return new S3Path(fileSystem, "/" + this.name()); } private AmazonS3 getClient() { diff --git a/src/main/java/com/upplication/s3fs/S3FileSystem.java b/src/main/java/com/upplication/s3fs/S3FileSystem.java index fe55e8a..96ed9e4 100644 --- a/src/main/java/com/upplication/s3fs/S3FileSystem.java +++ b/src/main/java/com/upplication/s3fs/S3FileSystem.java @@ -9,12 +9,10 @@ import java.nio.file.PathMatcher; import java.nio.file.WatchService; import java.nio.file.attribute.UserPrincipalLookupService; -import java.util.List; import java.util.Set; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.model.Bucket; -import com.google.common.base.Joiner; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; @@ -139,12 +137,6 @@ public String[] key2Parts(String keyParts) { return split; } - public String parts2Key(List parts) { - if (parts.isEmpty()) - return ""; - return Joiner.on(PATH_SEPARATOR).join(ImmutableList.copyOf(parts)); - } - @Override public int hashCode() { final int prime = 31; diff --git a/src/main/java/com/upplication/s3fs/S3FileSystemProvider.java b/src/main/java/com/upplication/s3fs/S3FileSystemProvider.java index 91f1a5c..e136cd3 100644 --- a/src/main/java/com/upplication/s3fs/S3FileSystemProvider.java +++ b/src/main/java/com/upplication/s3fs/S3FileSystemProvider.java @@ -363,7 +363,8 @@ public void createDirectory(Path dir, FileAttribute... attrs) throws IOExcept // create the object as directory ObjectMetadata metadata = new ObjectMetadata(); metadata.setContentLength(0); - s3Path.getFileSystem().getClient().putObject(bucketName, s3Path.getKey() + "/", new ByteArrayInputStream(new byte[0]), metadata); + String directoryKey = s3Path.getKey().endsWith("/") ? s3Path.getKey() : s3Path.getKey() + "/"; + s3Path.getFileSystem().getClient().putObject(bucketName, directoryKey, new ByteArrayInputStream(new byte[0]), metadata); } @Override diff --git a/src/main/java/com/upplication/s3fs/S3Iterator.java b/src/main/java/com/upplication/s3fs/S3Iterator.java index b7eb6c5..ad84593 100644 --- a/src/main/java/com/upplication/s3fs/S3Iterator.java +++ b/src/main/java/com/upplication/s3fs/S3Iterator.java @@ -39,7 +39,7 @@ public S3Iterator(S3Path path) { } public S3Iterator(S3Path path, boolean incremental) { - this(path.getFileStore(), path.getKey().length() == 0 ? "" : (path.getKey() + (incremental ? "" : "/")), incremental); + this(path.getFileStore(), path.getKey() + (!incremental && !path.getKey().isEmpty() && !path.getKey().endsWith("/") ? "/" : ""), incremental); } public S3Iterator(S3FileStore fileStore, String key, boolean incremental) { @@ -89,7 +89,7 @@ private void parseObjects() { final String objectSummaryKey = objectSummary.getKey(); String[] keyParts = fileSystem.key2Parts(objectSummaryKey); addParentPaths(keyParts); - S3Path path = new S3Path(fileSystem, fileStore, keyParts); + S3Path path = new S3Path(fileSystem, "/" + fileStore.name(), keyParts); if (!items.contains(path)) { items.add(path); } @@ -102,7 +102,7 @@ private void addParentPaths(String[] keyParts) { String[] subParts = Arrays.copyOf(keyParts, keyParts.length - 1); List parentPaths = new ArrayList<>(); while (subParts.length > 0) { - S3Path path = new S3Path(fileSystem, fileStore, subParts); + S3Path path = new S3Path(fileSystem, "/" + fileStore.name(), subParts); String prefix = current.getPrefix(); String parentKey = path.getKey(); @@ -131,7 +131,7 @@ private void addParentPaths(String[] keyParts) { private void parseObjectListing(String key, List listPath, ObjectListing current) { for (String commonPrefix : current.getCommonPrefixes()) { if (!commonPrefix.equals("/")) { - listPath.add(new S3Path(fileSystem, fileStore, fileSystem.key2Parts(commonPrefix))); + listPath.add(new S3Path(fileSystem, "/" + fileStore.name(), fileSystem.key2Parts(commonPrefix))); } } // TODO: figure our a way to efficiently preprocess commonPrefix basicFileAttributes @@ -140,7 +140,7 @@ private void parseObjectListing(String key, List listPath, ObjectListing // we only want the first level String immediateDescendantKey = getImmediateDescendant(key, objectSummaryKey); if (immediateDescendantKey != null) { - S3Path descendentPart = new S3Path(fileSystem, fileStore, fileSystem.key2Parts(immediateDescendantKey)); + S3Path descendentPart = new S3Path(fileSystem, "/" + fileStore.name(), fileSystem.key2Parts(immediateDescendantKey)); descendentPart.setFileAttributes(s3Utils.toS3FileAttributes(objectSummary, descendentPart.getKey())); if (!listPath.contains(descendentPart)) { listPath.add(descendentPart); diff --git a/src/main/java/com/upplication/s3fs/S3Path.java b/src/main/java/com/upplication/s3fs/S3Path.java index e6857bf..5dadc9a 100644 --- a/src/main/java/com/upplication/s3fs/S3Path.java +++ b/src/main/java/com/upplication/s3fs/S3Path.java @@ -1,32 +1,21 @@ package com.upplication.s3fs; -import static com.google.common.collect.Iterables.concat; -import static com.google.common.collect.Iterables.filter; -import static com.google.common.collect.Iterables.transform; -import static java.lang.String.format; +import com.google.common.base.*; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; +import com.upplication.s3fs.attribute.S3BasicFileAttributes; import java.io.File; import java.io.IOException; +import java.io.UnsupportedEncodingException; import java.net.URI; -import java.nio.file.LinkOption; -import java.nio.file.Path; -import java.nio.file.WatchEvent; -import java.nio.file.WatchKey; -import java.nio.file.WatchService; -import java.util.Arrays; +import java.net.URLDecoder; +import java.nio.file.*; import java.util.Iterator; import java.util.List; -import javax.annotation.Nullable; - -import com.google.common.base.Function; -import com.google.common.base.Joiner; -import com.google.common.base.Preconditions; -import com.google.common.base.Predicate; -import com.google.common.base.Splitter; -import com.google.common.collect.ImmutableList; -import com.google.common.collect.Lists; -import com.upplication.s3fs.attribute.S3BasicFileAttributes; +import static com.google.common.collect.Iterables.*; +import static java.lang.String.format; public class S3Path implements Path { @@ -35,10 +24,13 @@ public class S3Path implements Path { * S3FileStore which represents the Bucket this path resides in. */ private final S3FileStore fileStore; + /** - * Parts without bucket name. + * URI not encoded + * Is the key for AmazonS3 */ - private final List parts; + private String uri; + /** * actual filesystem */ @@ -49,23 +41,6 @@ public class S3Path implements Path { */ private S3BasicFileAttributes fileAttributes; - /** - * path must be a string of the form "/{bucket}", "/{bucket}/{key}" or just - * "{key}". - * Examples: - *
    - *
  • "/{bucket}//{value}" good, empty key paths are ignored
    • - *
  • "//{key}" error, missing bucket
    • - *
  • "/" error, missing bucket
    • - *
- * - * @param fileSystem - * @param path - */ - public S3Path(S3FileSystem fileSystem, String path) { - this(fileSystem, path, ""); - } - /** * Build an S3Path from path segments. '/' are stripped from each segment. * @@ -74,43 +49,64 @@ public S3Path(S3FileSystem fileSystem, String path) { * @param more directories and files */ public S3Path(S3FileSystem fileSystem, String first, String... more) { - String bucket = null; - List pathParts = Lists.newArrayList(Splitter.on(PATH_SEPARATOR).split(first)); - if (first.endsWith(PATH_SEPARATOR)) - pathParts.remove(pathParts.size() - 1); + Preconditions.checkArgument(first != null, "first path must be not null"); + Preconditions.checkArgument(!first.startsWith("//"), "first path doesnt start with '//'. Miss bucket"); + // see tests com.upplication.s3fs.Path.EndsWithTest#endsWithRelativeBlankAbsolute() + // Preconditions.checkArgument(!first.isEmpty(), "first path must be not empty"); + + boolean hasBucket = first.startsWith("/"); - if (first.startsWith(PATH_SEPARATOR)) { // absolute path - pathParts = pathParts.subList(1, pathParts.size()); - Preconditions.checkArgument(pathParts.size() >= 1, "path must start with bucket name"); - Preconditions.checkArgument(!pathParts.get(0).isEmpty(), "bucket name must be not empty"); - bucket = pathParts.get(0); - pathParts = pathParts.subList(1, pathParts.size()); - } - List moreSplitted = Lists.newArrayList(); - for (String part : more) - moreSplitted.addAll(Lists.newArrayList(Splitter.on(PATH_SEPARATOR).split(part))); + List pathsURI = Lists + .newArrayList(Splitter.on(PATH_SEPARATOR) + .omitEmptyStrings() + .split(first)); - pathParts.addAll(moreSplitted); - if (bucket != null) + if (hasBucket) { // absolute path + + Preconditions.checkArgument(pathsURI.size() >= 1, "path must start with bucket name"); + Preconditions.checkArgument(!pathsURI.get(0).isEmpty(), "bucket name must be not empty"); + String bucket = pathsURI.get(0); this.fileStore = new S3FileStore(fileSystem, bucket); - else + // the filestore is not part of the uri + pathsURI.remove(0); + } + else { + // relative uri this.fileStore = null; - this.parts = KeyParts.parse(pathParts); - this.fileSystem = fileSystem; - } + } + + StringBuilder uriBuilder = new StringBuilder(); + if (hasBucket) { + uriBuilder.append(PATH_SEPARATOR); + } + for (String path : pathsURI) { + uriBuilder.append(path + PATH_SEPARATOR); + } + if (more != null) { + for (String path : more) { + uriBuilder.append(path + PATH_SEPARATOR); + } + } + this.uri = normalizeURI(uriBuilder.toString()); + // remove last PATH_SEPARATOR + if (!first.isEmpty() && + // only first param and not ended with PATH_SEPARATOR + ((!first.endsWith(PATH_SEPARATOR) && (more == null || more.length == 0)) + // we have more param and not ended with PATH_SEPARATOR + || more != null && more.length > 0 && !more[more.length-1].endsWith(PATH_SEPARATOR))) { + this.uri = this.uri.substring(0, this.uri.length() - 1); + } - S3Path(S3FileSystem fileSystem, S3FileStore fileStore, Iterable keys) { - this.fileStore = fileStore; - this.parts = KeyParts.parse(keys); this.fileSystem = fileSystem; } - S3Path(S3FileSystem fileSystem, S3FileStore fileStore, String... keys) { - this.fileStore = fileStore; - this.parts = KeyParts.parse(keys); - this.fileSystem = fileSystem; + /** + * Remove duplicated slash + */ + private String normalizeURI(String uri) { + return uri.replace("//", "/"); } public S3FileStore getFileStore() { @@ -122,7 +118,20 @@ public S3FileStore getFileStore() { * note: the final slash need to be added to save a directory (Amazon s3 spec) */ public String getKey() { - return fileSystem.parts2Key(parts); + + String key = this.uri; + + if (key.startsWith("/")) { + key = key.substring(1, key.length()); + } + + // TODO: review this... :S + /* + if (key.endsWith("/")) { + key = key.substring(0, key.length()-1); + } + */ + return key; } @Override @@ -138,7 +147,7 @@ public boolean isAbsolute() { @Override public Path getRoot() { if (isAbsolute()) { - return new S3Path(fileSystem, fileStore, ImmutableList.of()); + return new S3Path(fileSystem, PATH_SEPARATOR + fileStore.name() + PATH_SEPARATOR); } return null; @@ -146,38 +155,110 @@ public Path getRoot() { @Override public Path getFileName() { - if (!parts.isEmpty()) - return new S3Path(fileSystem, null, parts.subList(parts.size() - 1, parts.size())); - return new S3Path(fileSystem, (S3FileStore) null, fileStore != null ? fileStore.name() : null); // bucket dont have fileName + List paths = uriToList(); + if (paths.isEmpty()) { + // get FileName of root directory is null + return null; + } + String filename = paths.get(paths.size()-1); + return new S3Path(fileSystem, filename); } @Override public Path getParent() { // bucket is not present in the parts - if (parts.isEmpty()) { + if (uri.isEmpty()) { return null; } - if (parts.size() == 1 && fileStore == null) { + String newUri = this.uri; + + if (this.uri.endsWith("/")) { + newUri = this.uri.substring(0, this.uri.length()-1); + } + int lastPathSeparatorPosition = newUri.lastIndexOf(PATH_SEPARATOR); + + if (lastPathSeparatorPosition == -1) { return null; } - return new S3Path(fileSystem, fileStore, parts.subList(0, parts.size() - 1)); + newUri = uri.substring(0, lastPathSeparatorPosition + 1); + + if (newUri.isEmpty()) + return null; + + String filestore = isAbsolute() ? PATH_SEPARATOR + fileStore.name() + PATH_SEPARATOR : ""; + + return new S3Path(fileSystem, filestore + newUri); } @Override public int getNameCount() { - return parts.size(); + return uriToList().size(); } @Override public Path getName(int index) { - return new S3Path(fileSystem, null, parts.subList(index, index + 1)); + + List paths = uriToList(); + + if (index < 0 || index >= paths.size()) { + throw new IllegalArgumentException("index out of range"); + } + + String path = paths.get(index); + StringBuilder pathsBuilder = new StringBuilder(); + if (isAbsolute() && index == 0) { + pathsBuilder.append(PATH_SEPARATOR + fileStore.name() + PATH_SEPARATOR); + } + pathsBuilder.append(path); + + if (index < paths.size() - 1) { + pathsBuilder.append(PATH_SEPARATOR); + } + + // if is the last path, check if end with path separator + if (index == paths.size() - 1 && this.uri.endsWith(PATH_SEPARATOR)) { + pathsBuilder.append(PATH_SEPARATOR); + } + + return new S3Path(fileSystem, pathsBuilder.toString()); + } + + private List uriToList() { + return Splitter.on(PATH_SEPARATOR).omitEmptyStrings().splitToList(this.uri); } + /** + * The bucket name not count + */ @Override public Path subpath(int beginIndex, int endIndex) { - return new S3Path(fileSystem, null, parts.subList(beginIndex, endIndex)); + + List paths = uriToList(); + + if (beginIndex < 0 || endIndex > paths.size()) { + throw new IllegalArgumentException("index out of range"); + } + + List pathSubList = paths.subList(beginIndex, endIndex); + StringBuilder pathsStringBuilder = new StringBuilder(); + + // build path string + + if (this.isAbsolute() && beginIndex == 0) { + pathsStringBuilder.append(PATH_SEPARATOR + fileStore.name() + PATH_SEPARATOR); + } + for (String path : pathSubList) { + pathsStringBuilder.append(path).append(PATH_SEPARATOR); + } + String pathsResult = pathsStringBuilder.toString(); + // if the uri doesnt have last PATH_SEPARATOR we must remove it. + if (endIndex == paths.size() && !this.uri.endsWith(PATH_SEPARATOR)) { + pathsResult = pathsResult.substring(0, pathsResult.length() - 1); + } + + return new S3Path(fileSystem, pathsResult); } @Override @@ -191,18 +272,27 @@ public boolean startsWith(Path other) { return false; } + if (this.isAbsolute() && !other.isAbsolute()) { + return false; + } + S3Path path = (S3Path) other; - if (path.parts.size() == 0 && path.fileStore == null && (this.parts.size() != 0 || this.fileStore != null)) { + if (this.isAbsolute() && other.isAbsolute() && + !this.fileStore.name().equals(path.fileStore.name())) { return false; } - if ((path.getFileStore() != null && !path.getFileStore().equals(this.getFileStore())) || (path.getFileStore() == null && this.getFileStore() != null)) { + if (path.uri.isEmpty() && !this.uri.isEmpty()) { return false; } - for (int i = 0; i < path.parts.size(); i++) { - if (!path.parts.get(i).equals(this.parts.get(i))) { + List pathsOther = path.uriToList(); + List paths = this.uriToList(); + + + for (int i = 0; i < pathsOther.size(); i++) { + if (!pathsOther.get(i).equals(paths.get(i))) { return false; } } @@ -237,11 +327,15 @@ public boolean endsWith(Path other) { // check subkeys - int i = path.parts.size() - 1; - int j = this.parts.size() - 1; + List pathsOther = path.uriToList(); + List paths = this.uriToList(); + + + int i = pathsOther.size() - 1; + int j = paths.size() - 1; for (; i >= 0 && j >= 0; ) { - if (!path.parts.get(i).equals(this.parts.get(j))) { + if (!pathsOther.get(i).equals(paths.get(j))) { return false; } i--; @@ -267,14 +361,17 @@ public Path resolve(Path other) { return other; } - ImmutableList.Builder builder = ImmutableList.builder(); - for (int i = 0; i < other.getNameCount(); i++) - builder.add(other.getName(i).toString()); - ImmutableList otherParts = builder.build(); - if (otherParts.isEmpty()) // other is relative and empty - return this; + S3Path otherS3Path = (S3Path) other; + StringBuilder pathBuilder = new StringBuilder(); - return new S3Path(fileSystem, fileStore, concat(parts, otherParts)); + if (this.isAbsolute()) { + pathBuilder.append(PATH_SEPARATOR + this.fileStore.name() + PATH_SEPARATOR); + } + pathBuilder.append(this.uri); + if (!otherS3Path.uri.isEmpty()) + pathBuilder.append(PATH_SEPARATOR + otherS3Path.uri); + + return new S3Path(this.fileSystem, pathBuilder.toString()); } @Override @@ -294,11 +391,27 @@ public Path resolveSibling(Path other) { return s3Path; } - if (s3Path.parts.isEmpty()) { // other is relative and empty + List othersPaths = s3Path.uriToList(); + + if (othersPaths.isEmpty()) { // other is relative and empty return parent; } - return new S3Path(fileSystem, fileStore, concat(parts.subList(0, parts.size() - 1), s3Path.parts)); + List paths = this.uriToList(); + + StringBuilder pathBuilder = new StringBuilder(); + String lastPath = othersPaths.get(othersPaths.size() - 1); + if (isAbsolute()) { + pathBuilder.append(PATH_SEPARATOR + fileStore.name() + PATH_SEPARATOR); + } + for (String path : concat(paths.subList(0, paths.size() - 1), othersPaths)) { + pathBuilder.append(path); + if (!lastPath.equals(path) || s3Path.uri.endsWith(PATH_SEPARATOR)) { + pathBuilder.append(PATH_SEPARATOR); + } + } + + return new S3Path(fileSystem, pathBuilder.toString()); } @Override @@ -318,28 +431,45 @@ public Path relativize(Path other) { Preconditions.checkArgument(isAbsolute(), "Path is already relative: %s", this); Preconditions.checkArgument(s3Path.isAbsolute(), "Cannot relativize against a relative path: %s", s3Path); Preconditions.checkArgument(fileStore.equals(s3Path.getFileStore()), "Cannot relativize paths with different buckets: '%s', '%s'", this, other); - Preconditions.checkArgument(parts.size() <= s3Path.parts.size(), "Cannot relativize against a parent path: '%s', '%s'", this, other); + // Preconditions.checkArgument(parts.size() <= s3Path.parts.size(), "Cannot relativize against a parent path: '%s', '%s'", this, other); - int startPart = 0; - for (int i = 0; i < this.parts.size(); i++) - if (this.parts.get(i).equals(s3Path.parts.get(i))) - startPart++; - return new S3Path(fileSystem, null, s3Path.parts.subList(startPart, s3Path.parts.size())); + String uriPath = decode(URI.create(encode(this.uri)).relativize(URI.create(encode(s3Path.uri)))); + return new S3Path(fileSystem, uriPath); } + /** + * Examples: + * + * Relative: + * -------- + * NO use fileSystem and not used fileStore. + * - path/file + * + * Absolute: + * -------- + * Use the fileSystem to get the host and the filestore to get the first path (in the future the filestore can be attached to the host) + * http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html + * - s3://AMAZONACCESSKEY@s3.amazonaws.com/bucket/path/file + * - s3://AMAZONACCESSKEY@bucket.s3.amazonaws.com/path/file + * - s3://s3-aws-region.amazonaws.com/bucket/path/file + * + * @return URI never null + */ @Override public URI toUri() { - if (fileStore == null) - return null; - StringBuilder builder = new StringBuilder(); - builder.append("s3://"); - builder.append(fileSystem.getKey()); - builder.append("/"); - builder.append(fileStore.name()); - builder.append(PATH_SEPARATOR); - builder.append(Joiner.on(PATH_SEPARATOR).join(parts)); - return URI.create(builder.toString()); + String uri = encode(this.uri); + // absolute + if (this.isAbsolute()) { + StringBuilder builder = new StringBuilder(); + builder.append(fileSystem.getKey()); + builder.append(PATH_SEPARATOR + fileStore.name() + PATH_SEPARATOR); + builder.append(uri); + return URI.create("s3://" + normalizeURI(builder.toString())); + } + else { + return URI.create(this.uri); + } } @Override @@ -373,10 +503,22 @@ public WatchKey register(WatchService watcher, WatchEvent.Kind... events) thr @Override public Iterator iterator() { + ImmutableList.Builder builder = ImmutableList.builder(); - for (String part : parts) { - builder.add(new S3Path(fileSystem, null, ImmutableList.of(part))); + if (isAbsolute()) { + builder.add(new S3Path(fileSystem, PATH_SEPARATOR + fileStore.name() + PATH_SEPARATOR)); + } + + List paths = uriToList(); + String lastPath = paths.get(paths.size()-1); + + for (String path : uriToList()) { + String pathFinal = path + PATH_SEPARATOR; + if (path.equals(lastPath) && !lastPath.endsWith(PATH_SEPARATOR)) { + pathFinal = pathFinal.substring(0, pathFinal.length() - 1); + } + builder.add(new S3Path(fileSystem, pathFinal)); } return builder.build().iterator(); @@ -389,20 +531,7 @@ public int compareTo(Path other) { @Override public String toString() { - StringBuilder builder = new StringBuilder(); - if (isAbsolute()) { - builder.append(PATH_SEPARATOR); - builder.append(fileStore.name()); - builder.append(PATH_SEPARATOR); - } - List parts2 = parts; - for (Iterator iterator = parts2.iterator(); iterator.hasNext(); ) { - String part = iterator.next(); - builder.append(part); - if (iterator.hasNext()) - builder.append(PATH_SEPARATOR); - } - return builder.toString(); + return toUri().toString(); } @Override @@ -412,10 +541,10 @@ public boolean equals(Object o) { if (o == null || getClass() != o.getClass()) return false; - S3Path paths = (S3Path) o; - if (fileStore != null ? !fileStore.equals(paths.fileStore) : paths.fileStore != null) + S3Path path = (S3Path) o; + if (fileStore != null ? !fileStore.equals(path.fileStore) : path.fileStore != null) return false; - if (!parts.equals(paths.parts)) + if (!uri.equals(path.uri)) return false; return true; } @@ -423,57 +552,42 @@ public boolean equals(Object o) { @Override public int hashCode() { int result = fileStore != null ? fileStore.name().hashCode() : 0; - result = 31 * result + parts.hashCode(); + result = 31 * result + uri.hashCode(); return result; } - public S3BasicFileAttributes getFileAttributes() { - return fileAttributes; - } - - public void setFileAttributes(S3BasicFileAttributes fileAttributes) { - this.fileAttributes = fileAttributes; - } - - // ~ helpers methods - - private static Function strip(final String... strs) { - return new Function() { - @Override - public String apply(String input) { - String res = input; - if (res != null) - for (String str : strs) - res = res.replace(str, ""); - return res; - } - }; - } - - private static Predicate notEmpty() { - return new Predicate() { - @Override - public boolean apply(@Nullable String input) { - return input != null && !input.isEmpty(); - } - }; + /** + * Encode special URI characters for path. + * @param uri String the uri path + * @return String + */ + private String encode(String uri) { + // remove special case URI starting with // + uri = uri.replace("//", "/"); + uri = uri.replaceAll(" ", "%20"); + return uri; } - /* - * delete redundant "/" and empty parts + /** + * Decode uri special characters + * + * @param uri URI mandatory + * @return String decoded */ - private abstract static class KeyParts { - private static ImmutableList parse(String[] parts) { - return ImmutableList.copyOf(filter(transform(Arrays.asList(parts), strip("/")), notEmpty())); + private String decode(URI uri) { + try { + return URLDecoder.decode(uri.toString(), "UTF-8"); } - - private static ImmutableList parse(List parts) { - return ImmutableList.copyOf(filter(transform(parts, strip("/")), notEmpty())); + catch (UnsupportedEncodingException e) { + throw new IllegalStateException("Error decoding key: " + this.uri, e); } + } - private static ImmutableList parse(Iterable parts) { - return ImmutableList.copyOf(filter(transform(parts, strip("/")), notEmpty())); - } + public S3BasicFileAttributes getFileAttributes() { + return fileAttributes; } + public void setFileAttributes(S3BasicFileAttributes fileAttributes) { + this.fileAttributes = fileAttributes; + } } \ No newline at end of file diff --git a/src/main/java/com/upplication/s3fs/S3SeekableByteChannel.java b/src/main/java/com/upplication/s3fs/S3SeekableByteChannel.java index 54678f6..3cb9dbf 100644 --- a/src/main/java/com/upplication/s3fs/S3SeekableByteChannel.java +++ b/src/main/java/com/upplication/s3fs/S3SeekableByteChannel.java @@ -103,7 +103,9 @@ protected void sync() throws IOException { try (InputStream stream = new BufferedInputStream(Files.newInputStream(tempFile))) { ObjectMetadata metadata = new ObjectMetadata(); metadata.setContentLength(Files.size(tempFile)); - metadata.setContentType(new Tika().detect(stream, path.getFileName().toString())); + if (path.getFileName() != null) { + metadata.setContentType(new Tika().detect(stream, path.getFileName().toString())); + } String bucket = path.getFileStore().name(); String key = path.getKey(); diff --git a/src/main/java/com/upplication/s3fs/util/S3Utils.java b/src/main/java/com/upplication/s3fs/util/S3Utils.java index 3c3926f..aff084f 100644 --- a/src/main/java/com/upplication/s3fs/util/S3Utils.java +++ b/src/main/java/com/upplication/s3fs/util/S3Utils.java @@ -32,6 +32,7 @@ public S3ObjectSummary getS3ObjectSummary(S3Path s3Path) throws NoSuchFileExcept String key = s3Path.getKey(); String bucketName = s3Path.getFileStore().name(); AmazonS3 client = s3Path.getFileSystem().getClient(); + // try to find the element with the current key (maybe with end slash or maybe not.) try { ObjectMetadata metadata = client.getObjectMetadata(bucketName, key); S3ObjectSummary result = new S3ObjectSummary(); @@ -48,11 +49,17 @@ public S3ObjectSummary getS3ObjectSummary(S3Path s3Path) throws NoSuchFileExcept throw e; } + // if not found (404 err) with the original key. + // try to find the elment as a directory. try { // is a virtual directory ListObjectsRequest request = new ListObjectsRequest(); request.setBucketName(bucketName); - request.setPrefix(key + "/"); + String keyFolder = key; + if (!keyFolder.endsWith("/")) { + keyFolder += "/"; + } + request.setPrefix(keyFolder); request.setMaxKeys(1); ObjectListing current = client.listObjects(request); if (!current.getObjectSummaries().isEmpty()) diff --git a/src/main/resources/META-INF/services/java.nio.file.spi.FileTypeDetector b/src/main/resources/META-INF/services/java.nio.file.spi.FileTypeDetector deleted file mode 100644 index f9c9b5b..0000000 --- a/src/main/resources/META-INF/services/java.nio.file.spi.FileTypeDetector +++ /dev/null @@ -1 +0,0 @@ -com.upplication.s3fs.util.FileTypeDetector \ No newline at end of file diff --git a/src/test/java/com/upplication/s3fs/FileSystemProvider/CreateDirectoryTest.java b/src/test/java/com/upplication/s3fs/FileSystemProvider/CreateDirectoryTest.java index a4f873a..392beb3 100644 --- a/src/test/java/com/upplication/s3fs/FileSystemProvider/CreateDirectoryTest.java +++ b/src/test/java/com/upplication/s3fs/FileSystemProvider/CreateDirectoryTest.java @@ -51,7 +51,7 @@ public void createDirectoryInNewBucket() throws IOException { S3Path root = createNewS3FileSystem().getPath("/newer-bucket"); Path resolve = root.resolve("folder"); Path path = Files.createDirectories(resolve); - assertEquals("/newer-bucket/folder", path.toAbsolutePath().toString()); + assertEquals("s3://s3.test.amazonaws.com/newer-bucket/folder", path.toAbsolutePath().toString()); // assert assertTrue(Files.exists(root)); assertTrue(Files.isDirectory(root)); diff --git a/src/test/java/com/upplication/s3fs/FileSystemProvider/DeleteTest.java b/src/test/java/com/upplication/s3fs/FileSystemProvider/DeleteTest.java index 7ee04ca..40e44b0 100644 --- a/src/test/java/com/upplication/s3fs/FileSystemProvider/DeleteTest.java +++ b/src/test/java/com/upplication/s3fs/FileSystemProvider/DeleteTest.java @@ -50,7 +50,7 @@ public void deleteEmptyDirectory() throws IOException { AmazonS3ClientMock client = AmazonS3MockFactory.getAmazonClientMock(); client.bucket("bucketA").dir("dir"); Path base = s3fsProvider.newFileSystem(URI.create("s3://endpoint1/"), - ImmutableMap.builder().put(ACCESS_KEY, "access key").put(SECRET_KEY, "secret key").build()) + ImmutableMap.builder().put(ACCESS_KEY, "access_key").put(SECRET_KEY, "secret_key").build()) .getPath("/bucketA/dir"); // act s3fsProvider.delete(base); diff --git a/src/test/java/com/upplication/s3fs/FileSystemProvider/GetPathTest.java b/src/test/java/com/upplication/s3fs/FileSystemProvider/GetPathTest.java index 0a1286e..4f58ad2 100644 --- a/src/test/java/com/upplication/s3fs/FileSystemProvider/GetPathTest.java +++ b/src/test/java/com/upplication/s3fs/FileSystemProvider/GetPathTest.java @@ -57,12 +57,12 @@ public void getAnotherPath() throws IOException { @Test(expected = IllegalArgumentException.class) public void getPathWithEndpointAndWithoutBucket() throws IOException { FileSystem fs = FileSystems.newFileSystem(URI.create("s3://endpoint1/"), null); - fs.provider().getPath(URI.create("s3://endpoint1//falta-bucket")); + fs.provider().getPath(URI.create("s3://endpoint1//missed-bucket")); } @Test(expected = IllegalArgumentException.class) public void getPathWithDefaultEndpointAndWithoutBucket() throws IOException { FileSystem fs = FileSystems.newFileSystem(URI.create("s3:///"), ImmutableMap.of()); - fs.provider().getPath(URI.create("s3:////falta-bucket")); + fs.provider().getPath(URI.create("s3:////missed-bucket")); } } \ No newline at end of file diff --git a/src/test/java/com/upplication/s3fs/FileSystemProvider/NewFileSystemTest.java b/src/test/java/com/upplication/s3fs/FileSystemProvider/NewFileSystemTest.java index 3aa49fe..ffad467 100644 --- a/src/test/java/com/upplication/s3fs/FileSystemProvider/NewFileSystemTest.java +++ b/src/test/java/com/upplication/s3fs/FileSystemProvider/NewFileSystemTest.java @@ -89,8 +89,8 @@ public void createsAuthenticatedByEnv() { @Test public void setEncodingByProperties() { Properties props = new Properties(); - props.setProperty(SECRET_KEY, "better secret key"); - props.setProperty(ACCESS_KEY, "better access key"); + props.setProperty(SECRET_KEY, "better_secret_key"); + props.setProperty(ACCESS_KEY, "better_access_key"); props.setProperty(CHARSET_KEY, "UTF-8"); doReturn(props).when(s3fsProvider).loadAmazonProperties(); URI uri = S3EndpointConstant.S3_GLOBAL_URI_TEST; @@ -98,21 +98,21 @@ public void setEncodingByProperties() { FileSystem fileSystem = s3fsProvider.newFileSystem(uri, ImmutableMap.of()); assertNotNull(fileSystem); - verify(s3fsProvider).createFileSystem(eq(uri), eq(buildFakeProps("better access key", "better secret key", "UTF-8"))); + verify(s3fsProvider).createFileSystem(eq(uri), eq(buildFakeProps("better_access_key", "better_secret_key", "UTF-8"))); } @Test public void createAuthenticatedByProperties() { Properties props = new Properties(); - props.setProperty(SECRET_KEY, "better secret key"); - props.setProperty(ACCESS_KEY, "better access key"); + props.setProperty(SECRET_KEY, "better_secret_key"); + props.setProperty(ACCESS_KEY, "better_access_key"); doReturn(props).when(s3fsProvider).loadAmazonProperties(); URI uri = S3EndpointConstant.S3_GLOBAL_URI_TEST; FileSystem fileSystem = s3fsProvider.newFileSystem(uri, ImmutableMap.of()); assertNotNull(fileSystem); - verify(s3fsProvider).createFileSystem(eq(uri), eq(buildFakeProps("better access key", "better secret key"))); + verify(s3fsProvider).createFileSystem(eq(uri), eq(buildFakeProps("better_access_key", "better_secret_key"))); } @Test @@ -149,8 +149,8 @@ public void createsAnonymous() { @Test public void createWithDefaultEndpoint() { Properties props = new Properties(); - props.setProperty(SECRET_KEY, "better secret key"); - props.setProperty(ACCESS_KEY, "better access key"); + props.setProperty(SECRET_KEY, "better_secret_key"); + props.setProperty(ACCESS_KEY, "better_access_key"); props.setProperty(CHARSET_KEY, "UTF-8"); doReturn(props).when(s3fsProvider).loadAmazonProperties(); URI uri = URI.create("s3:///"); @@ -158,13 +158,13 @@ public void createWithDefaultEndpoint() { FileSystem fileSystem = s3fsProvider.newFileSystem(uri, ImmutableMap.of()); assertNotNull(fileSystem); - verify(s3fsProvider).createFileSystem(eq(uri), eq(buildFakeProps("better access key", "better secret key", "UTF-8"))); + verify(s3fsProvider).createFileSystem(eq(uri), eq(buildFakeProps("better_access_key", "better_secret_key", "UTF-8"))); } @Test(expected = IllegalArgumentException.class) public void createWithOnlyAccessKey() { Properties props = new Properties(); - props.setProperty(ACCESS_KEY, "better access key"); + props.setProperty(ACCESS_KEY, "better_access_key"); doReturn(props).when(s3fsProvider).loadAmazonProperties(); s3fsProvider.newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, ImmutableMap.of()); } @@ -172,7 +172,7 @@ public void createWithOnlyAccessKey() { @Test(expected = IllegalArgumentException.class) public void createWithOnlySecretKey() { Properties props = new Properties(); - props.setProperty(SECRET_KEY, "better secret key"); + props.setProperty(SECRET_KEY, "better_secret_key"); doReturn(props).when(s3fsProvider).loadAmazonProperties(); s3fsProvider.newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, ImmutableMap.of()); } @@ -230,7 +230,7 @@ public void createTwoFileSystemThrowError() { private Map buildFakeEnv() { - return ImmutableMap.builder().put(ACCESS_KEY, "access key").put(SECRET_KEY, "secret key").build(); + return ImmutableMap.builder().put(ACCESS_KEY, "access_key").put(SECRET_KEY, "secret_key").build(); } private Properties buildFakeProps(String access_key, String secret_key, String encoding) { diff --git a/src/test/java/com/upplication/s3fs/FileSystemProvider/NewInputStreamTest.java b/src/test/java/com/upplication/s3fs/FileSystemProvider/NewInputStreamTest.java index 2f1a9fe..3dd71fa 100644 --- a/src/test/java/com/upplication/s3fs/FileSystemProvider/NewInputStreamTest.java +++ b/src/test/java/com/upplication/s3fs/FileSystemProvider/NewInputStreamTest.java @@ -93,7 +93,7 @@ public void inputStreamDirectory() throws IOException { } private Map buildFakeEnv() { - return ImmutableMap.builder().put(ACCESS_KEY, "access key").put(SECRET_KEY, "secret key").build(); + return ImmutableMap.builder().put(ACCESS_KEY, "accesskey").put(SECRET_KEY, "secretkey").build(); } /** diff --git a/src/test/java/com/upplication/s3fs/FileSystemProvider/NewOutputStreamTest.java b/src/test/java/com/upplication/s3fs/FileSystemProvider/NewOutputStreamTest.java index 104f489..41df1e5 100644 --- a/src/test/java/com/upplication/s3fs/FileSystemProvider/NewOutputStreamTest.java +++ b/src/test/java/com/upplication/s3fs/FileSystemProvider/NewOutputStreamTest.java @@ -136,7 +136,7 @@ private Path getS3Directory() throws IOException { // fixtures AmazonS3ClientMock client = AmazonS3MockFactory.getAmazonClientMock(); client.bucket("bucketA").dir("dir"); - return s3fsProvider.newFileSystem(URI.create("s3://endpoint1/"), ImmutableMap.builder().put(ACCESS_KEY, "access key").put(SECRET_KEY, "secret key").build()).getPath("/bucketA/dir"); + return s3fsProvider.newFileSystem(URI.create("s3://endpoint1/"), ImmutableMap.builder().put(ACCESS_KEY, "access_key").put(SECRET_KEY, "secret_key").build()).getPath("/bucketA/dir"); } /** diff --git a/src/test/java/com/upplication/s3fs/FilesIT.java b/src/test/java/com/upplication/s3fs/FilesIT.java index 6ee9be8..4460a79 100644 --- a/src/test/java/com/upplication/s3fs/FilesIT.java +++ b/src/test/java/com/upplication/s3fs/FilesIT.java @@ -193,10 +193,10 @@ public void directoryStreamFirstDirTest() throws IOException { @Test public void virtualDirectoryStreamTest() throws IOException { - String folder = UUID.randomUUID().toString(); + String folder = UUID.randomUUID().toString() + "/"; - String file1 = folder + "/file.html"; - String file2 = folder + "/file2.html"; + String file1 = folder + "file.html"; + String file2 = folder + "file2.html"; Path dir = fileSystemAmazon.getPath(bucket, folder); @@ -238,10 +238,10 @@ public void virtualDirectoryStreamTest() throws IOException { @Test public void virtualDirectoryStreamWithVirtualSubFolderTest() throws IOException { - String folder = UUID.randomUUID().toString(); + String folder = UUID.randomUUID().toString() + "/"; - String subfoler = folder + "/subfolder/file.html"; - String file2 = folder + "/file2.html"; + String subfoler = folder + "subfolder/file.html"; + String file2 = folder + "file2.html"; Path dir = fileSystemAmazon.getPath(bucket, folder); @@ -261,7 +261,7 @@ public void virtualDirectoryStreamWithVirtualSubFolderTest() throws IOException boolean file2Find = false; for (Path path : dirStream) { number++; - // solo recorre ficheros del primer nivel + // only the first level one assertEquals(dir, path.getParent()); switch (path.getFileName().toString()) { case "subfolder": @@ -494,7 +494,7 @@ public void seekableCloseTwice() throws IOException { @Test public void bucketIsDirectory() throws IOException { - Path path = fileSystemAmazon.getPath(bucket, "/"); + Path path = fileSystemAmazon.getPath(bucket); BasicFileAttributes attrs = Files.readAttributes(path, BasicFileAttributes.class); assertEquals(0, attrs.size()); assertEquals(null, attrs.creationTime()); diff --git a/src/test/java/com/upplication/s3fs/Path/EndsWithTest.java b/src/test/java/com/upplication/s3fs/Path/EndsWithTest.java new file mode 100644 index 0000000..bd38ee6 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/EndsWithTest.java @@ -0,0 +1,116 @@ +package com.upplication.s3fs.Path; + +import com.github.marschall.memoryfilesystem.MemoryFileSystemBuilder; +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystem; +import java.nio.file.FileSystems; +import java.nio.file.Path; +import java.nio.file.Paths; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +public class EndsWithTest extends S3UnitTestBase { + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Test + public void endsWithAbsoluteRelative() { + assertTrue(getPath("/bucket/file1").endsWith(getPath("file1"))); + } + + @Test + public void endsWithAbsoluteAbsolute() { + assertTrue(getPath("/bucket/file1").endsWith(getPath("/bucket/file1"))); + } + + @Test + public void endsWithRelativeRelative() { + assertTrue(getPath("file/file1").endsWith(getPath("file1"))); + } + + @Test + public void endsWithRelativeAbsolute() { + assertFalse(getPath("file/file1").endsWith(getPath("/bucket"))); + } + + @Test + public void endsWithDifferenteFileSystem() { + assertFalse(getPath("/bucket/file1").endsWith(Paths.get("/bucket/file1"))); + } + + @Test + public void endsWithBlankRelativeAbsolute() { + assertFalse(getPath("").endsWith(getPath("/bucket"))); + } + + @Test + public void endsWithBlankBlank() { + assertTrue(getPath("").endsWith(getPath(""))); + } + + @Test + public void endsWithRelativeBlankAbsolute() { + assertFalse(getPath("/bucket/file1").endsWith(getPath(""))); + } + + @Test + public void endsWithRelativeBlankRelative() { + assertFalse(getPath("file1").endsWith(getPath(""))); + } + + @Test + public void endsWithDifferent() { + assertFalse(getPath("/bucket/dir/dir/file1").endsWith(getPath("fail/dir/file1"))); + } + + @Test + public void endsWithDifferentProvider() throws IOException { + try (FileSystem linux = MemoryFileSystemBuilder.newLinux().build("linux")) { + Path fileLinux = linux.getPath("/file"); + + assertFalse(getPath("/bucket/file").endsWith(fileLinux)); + } + + try (FileSystem window = MemoryFileSystemBuilder.newWindows().build("window")) { + Path file = window.getPath("c:/file"); + + assertFalse(getPath("/c/file").endsWith(file)); + } + } + + @Test + public void endsWithString() { + // endsWithAbsoluteRelative(){ + assertTrue(getPath("/bucket/file1").endsWith("file1")); + // endsWithAbsoluteAbsolute + assertTrue(getPath("/bucket/file1").endsWith("/bucket/file1")); + // endsWithRelativeRelative + assertTrue(getPath("file/file1").endsWith("file1")); + // endsWithRelativeAbsolute + assertFalse(getPath("file/file1").endsWith("/bucket")); + // endsWithBlankRelativeAbsolute + assertFalse(getPath("").endsWith("/bucket")); + // endsWithBlankBlank + assertTrue(getPath("").endsWith("")); + // endsWithRelativeBlankAbsolute + assertFalse(getPath("/bucket/file1").endsWith("")); + // endsWithRelativeBlankRelative + assertFalse(getPath("file1").endsWith("")); + } +} diff --git a/src/test/java/com/upplication/s3fs/Path/EqualsTest.java b/src/test/java/com/upplication/s3fs/Path/EqualsTest.java new file mode 100644 index 0000000..5ca0cbf --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/EqualsTest.java @@ -0,0 +1,108 @@ +package com.upplication.s3fs.Path; + +import com.github.marschall.memoryfilesystem.MemoryFileSystemBuilder; +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.net.URI; +import java.nio.file.FileSystem; +import java.nio.file.FileSystems; +import java.nio.file.Path; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.*; + +public class EqualsTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void equals() { + Path path = getPath("/bucketA/dir/file"); + Path path2 = getPath("/bucketA/dir/file"); + + assertEquals(path, path2); + } + + @Test + public void equalsDir() { + Path path = getPath("/bucketA/dir/"); + Path path2 = getPath("/bucketA/dir/"); + + assertEquals(path, path2); + } + + @Test + public void equalsBucket() { + Path path = getPath("/bucketA/"); + Path path2 = getPath("/bucketA/"); + + assertEquals(path, path2); + } + + @Test + public void equalsBucketWithoutEndSlash() { + Path path = getPath("/bucketA/"); + Path path2 = getPath("/bucketA"); + + assertNotEquals(path, path2); + } + + @Test + public void notEquals() { + Path path = getPath("/bucketA/dir/file"); + Path path2 = getPath("/bucketA/dir/file2"); + + assertNotEquals(path, path2); + } + + @Test + public void notEqualsDirFile() { + Path path = getPath("/bucketA/dir/asd/"); + Path path2 = getPath("/bucketA/dir/asd"); + + assertNotEquals(path, path2); + } + + @Test + public void notEqualsNull() { + Path path = getPath("/bucketA/dir/file"); + + assertNotEquals(path, null); + } + + @Test + public void notEqualsDifferentProvider() throws IOException { + Path path = getPath("/c/dir/file"); + + try (FileSystem linux = MemoryFileSystemBuilder.newLinux().build("linux")) { + Path fileLinux = linux.getPath("/dir/file"); + + assertNotEquals(path, fileLinux); + } + + try (FileSystem window = MemoryFileSystemBuilder.newWindows().build("window")) { + Path file = window.getPath("c:/dir/file"); + + assertNotEquals(path, file); + } + + Path pathS3EmptyEndpoint = FileSystems.newFileSystem(URI.create("s3:///"), null).getPath("/bucketA/dir/"); + Path pathS3TestEndpoint = getPath("/bucketA/dir/"); + + assertNotEquals(pathS3EmptyEndpoint, pathS3TestEndpoint); + } + +} diff --git a/src/test/java/com/upplication/s3fs/Path/GetFilenameTest.java b/src/test/java/com/upplication/s3fs/Path/GetFilenameTest.java new file mode 100644 index 0000000..44a4e70 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/GetFilenameTest.java @@ -0,0 +1,53 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; +import java.nio.file.Path; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; + +public class GetFileNameTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void getFileName() { + Path path = getPath("/bucketA/file"); + Path name = path.getFileName(); + + assertEquals(getPath("file"), name); + } + + @Test + public void getAnotherFileName() { + Path path = getPath("/bucketA/dir/another-file"); + Path fileName = path.getFileName(); + Path dirName = path.getParent().getFileName(); + + assertEquals(getPath("another-file"), fileName); + assertEquals(getPath("dir"), dirName); + } + + @Test + public void getFileNameBucket() { + Path path = getPath("/bucket"); + Path name = path.getFileName(); + assertNull(name); + } +} diff --git a/src/test/java/com/upplication/s3fs/Path/GetKeyTest.java b/src/test/java/com/upplication/s3fs/Path/GetKeyTest.java new file mode 100644 index 0000000..86f07fb --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/GetKeyTest.java @@ -0,0 +1,68 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; + +public class GetKeyTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void getKeyBucket() { + S3Path path = getPath("/bucket"); + + assertEquals("", path.getKey()); + } + + @Test + public void getKeyFile() { + S3Path path = getPath("/bucket/file"); + assertEquals(path.getKey(), "file"); + } + + @Test + public void getKeyFolder() { + S3Path path = getPath("/bucket/folder/"); + + assertEquals(path.getKey(), "folder/"); + } + + @Test + public void getKeyParent() { + S3Path path = (S3Path) getPath("/bucket/folder/file").getParent(); + + assertEquals(path.getKey(), "folder/"); + } + + @Test + public void getKeyRoot() { + S3Path path = (S3Path) getPath("/bucket/folder/file").getRoot(); + + assertEquals(path.getKey(), ""); + } + + @Test + public void getKeyEncodingPath() { + S3Path path = getPath("/bucket/path with spaces/to/β ϐ"); + + assertEquals("path with spaces/to/β ϐ", path.getKey()); + + } +} diff --git a/src/test/java/com/upplication/s3fs/Path/GetNameTest.java b/src/test/java/com/upplication/s3fs/Path/GetNameTest.java new file mode 100644 index 0000000..8da4353 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/GetNameTest.java @@ -0,0 +1,54 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; + +public class GetNameTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test(expected = IllegalArgumentException.class) + public void getNameBucket() { + // TODO: this is ok? + S3Path path = getPath("/bucket"); + path.getName(0); + } + + @Test + public void getName0() { + S3Path path = getPath("/bucket/file"); + assertEquals(getPath("/bucket/file"), path.getName(0)); + } + + + @Test + public void getNames() { + S3Path path = getPath("/bucket/path/to/file"); + assertEquals(path.getName(0), getPath("/bucket/path/")); + assertEquals(path.getName(1), getPath("to/")); + assertEquals(path.getName(2), getPath("file")); + } + + @Test(expected = IllegalArgumentException.class) + public void getNameOutOfIndex() { + S3Path path = getPath("/bucket/path/to/file"); + path.getName(3); + } +} diff --git a/src/test/java/com/upplication/s3fs/Path/GetRootTest.java b/src/test/java/com/upplication/s3fs/Path/GetRootTest.java new file mode 100644 index 0000000..b221e66 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/GetRootTest.java @@ -0,0 +1,37 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; + +public class GetRootTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems.newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void getRootReturnBucket() { + assertEquals(getPath("/bucketA/"), getPath("/bucketA/dir/file").getRoot()); + } + + @Test + public void getRootRelativeReturnNull() { + assertNull(getPath("dir/file").getRoot()); + } + +} diff --git a/src/test/java/com/upplication/s3fs/Path/ItearatorTest.java b/src/test/java/com/upplication/s3fs/Path/ItearatorTest.java new file mode 100644 index 0000000..4243691 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/ItearatorTest.java @@ -0,0 +1,40 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; +import java.nio.file.Path; +import java.util.Iterator; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; + +public class ItearatorTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void iterator() { + Iterator iterator = getPath("/bucket/path/to/file").iterator(); + + assertEquals(getPath("/bucket/"), iterator.next()); + assertEquals(getPath("path/"), iterator.next()); + assertEquals(getPath("to/"), iterator.next()); + assertEquals(getPath("file"), iterator.next()); + } + + +} diff --git a/src/test/java/com/upplication/s3fs/Path/ResolveSiblingTest.java b/src/test/java/com/upplication/s3fs/Path/ResolveSiblingTest.java new file mode 100644 index 0000000..75af3dc --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/ResolveSiblingTest.java @@ -0,0 +1,75 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; + +public class ResolveSiblingTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + + @Test + public void resolveSibling() { + // absolute (non-root) vs... + assertEquals(getPath("/bucket/path/to/file").resolveSibling(getPath("other/child")), getPath("/bucket/path/to/other/child")); + assertEquals(getPath("/bucket/path/to/file").resolveSibling(getPath("/bucket2/other/child")), getPath("/bucket2/other/child")); + assertEquals(getPath("/bucket/path/to/file").resolveSibling(getPath("")), getPath("/bucket/path/to/")); + + // absolute (root) vs ... + assertEquals(getPath("/bucket").resolveSibling(getPath("other/child")), getPath("other/child")); + assertEquals(getPath("/bucket").resolveSibling(getPath("/bucket2/other/child")), getPath("/bucket2/other/child")); + assertEquals(getPath("/bucket").resolveSibling(getPath("")), getPath("")); + + // relative (empty) vs ... + assertEquals(getPath("").resolveSibling(getPath("other/child")), getPath("other/child")); + assertEquals(getPath("").resolveSibling(getPath("/bucket2/other/child")), getPath("/bucket2/other/child")); + assertEquals(getPath("").resolveSibling(getPath("")), getPath("")); + + // relative (non-empty) vs ... + assertEquals(getPath("path/to/file").resolveSibling(getPath("other/child")), getPath("path/to/other/child")); + assertEquals(getPath("path/to/file").resolveSibling(getPath("/bucket2/other/child")), getPath("/bucket2/other/child")); + assertEquals(getPath("path/to/file").resolveSibling(getPath("")), getPath("path/to/")); + } + + @Test + public void resolveSiblingString() { + // absolute (non-root) vs... + assertEquals(getPath("/bucket/path/to/file").resolveSibling("other/child"), getPath("/bucket/path/to/other/child")); + assertEquals(getPath("/bucket/path/to/file").resolveSibling("/bucket2/other/child"), getPath("/bucket2/other/child")); + assertEquals(getPath("/bucket/path/to/file").resolveSibling(""), getPath("/bucket/path/to/")); + + // absolute (root) vs ... + assertEquals(getPath("/bucket").resolveSibling("other/child"), getPath("other/child")); + assertEquals(getPath("/bucket").resolveSibling("/bucket2/other/child"), getPath("/bucket2/other/child")); + assertEquals(getPath("/bucket").resolveSibling(""), getPath("")); + + // relative (empty) vs ... + assertEquals(getPath("").resolveSibling("other/child"), getPath("other/child")); + assertEquals(getPath("").resolveSibling("/bucket2/other/child"), getPath("/bucket2/other/child")); + assertEquals(getPath("").resolveSibling(""), getPath("")); + + // relative (non-empty) vs ... + assertEquals(getPath("path/to/file").resolveSibling("other/child"), getPath("path/to/other/child")); + assertEquals(getPath("path/to/file").resolveSibling("/bucket2/other/child"), getPath("/bucket2/other/child")); + assertEquals(getPath("path/to/file").resolveSibling(""), getPath("path/to/")); + } + + +} diff --git a/src/test/java/com/upplication/s3fs/Path/ResolveTest.java b/src/test/java/com/upplication/s3fs/Path/ResolveTest.java new file mode 100644 index 0000000..2a560f9 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/ResolveTest.java @@ -0,0 +1,50 @@ +package com.upplication.s3fs.Path; + +import com.github.marschall.memoryfilesystem.MemoryFileSystemBuilder; +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; + +public class ResolveTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void resolve() { + assertEquals(getPath("/bucket/path/to/dir/child/xyz"), getPath("/bucket/path/to/dir/").resolve(getPath("child/xyz"))); + assertEquals(getPath("/bucket/path/to/dir/child/xyz"), getPath("/bucket/path/to/dir/").resolve("child/xyz")); + + assertEquals(getPath("/bucket/path/to/dir/child/xyz"), getPath("/bucket/path/to/dir").resolve(getPath("child/xyz"))); + assertEquals(getPath("/bucket/path/to/dir/child/xyz"), getPath("/bucket/path/to/dir").resolve("child/xyz")); + + assertEquals(getPath("/bucket/path/to/file"), getPath("/bucket/path/to/file").resolve(getPath(""))); + assertEquals(getPath("/bucket/path/to/file"), getPath("/bucket/path/to/file").resolve("")); + + assertEquals(getPath("path/to/file/child/xyz"), getPath("path/to/file").resolve(getPath("child/xyz"))); + assertEquals(getPath("path/to/file/child/xyz"), getPath("path/to/file").resolve("child/xyz")); + + assertEquals(getPath("path/to/file"), getPath("path/to/file").resolve(getPath(""))); + assertEquals(getPath("path/to/file"), getPath("path/to/file").resolve("")); + + assertEquals(getPath("/bucket2/other/child"), getPath("/bucket/path/to/file").resolve(getPath("/bucket2/other/child"))); + assertEquals(getPath("/bucket2/other/child"), getPath("/bucket/path/to/file").resolve("/bucket2/other/child")); + } + + +} diff --git a/src/test/java/com/upplication/s3fs/Path/S3PathTest.java b/src/test/java/com/upplication/s3fs/Path/S3PathTest.java new file mode 100644 index 0000000..fcf38b4 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/S3PathTest.java @@ -0,0 +1,232 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3FileSystem; +import com.upplication.s3fs.S3FileSystemProvider; +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; +import java.nio.file.Path; +import java.nio.file.WatchEvent; +import java.util.HashMap; + +import static org.junit.Assert.*; + +public class S3PathTest extends S3UnitTestBase { + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void createNoPath() { + S3Path path = forPath("/bucket"); + + assertEquals("bucket", path.getFileStore().name()); + assertEquals("", path.getKey()); + } + + @Test + public void createWithTrailingSlash() { + S3Path path = forPath("/bucket/"); + + assertEquals(path.getFileStore().name(), "bucket"); + assertEquals(path.getKey(), ""); + } + + @Test + public void createWithPath() { + S3Path path = forPath("/bucket/path/to/file"); + + assertEquals(path.getFileStore().name(), "bucket"); + assertEquals(path.getKey(), "path/to/file"); + } + + @Test + public void createWithPathAndTrailingSlash() { + S3Path path = forPath("/bucket/path/to/dir/"); + + assertEquals("bucket", path.getFileStore().name()); + assertEquals("path/to/dir/", path.getKey()); + + } + + @Test + public void createWithPathAndTrailingSlashDir() { + S3Path path = forPath("/bucket/path/to/dir/"); + + assertEquals("bucket", path.getFileStore().name()); + assertEquals("path/to/dir/", path.getKey()); + } + + @Test + public void createRelative() { + S3Path path = forPath("path/to/file"); + + assertNull(path.getFileStore()); + assertEquals(path.getKey(), "path/to/file"); + assertFalse(path.isAbsolute()); + } + + @Test + public void getParent() { + assertEquals(forPath("/bucket/path/to/"), forPath("/bucket/path/to/file").getParent()); + assertEquals(forPath("/bucket/path/to/"), forPath("/bucket/path/to/file/").getParent()); + assertNull(forPath("/bucket/").getParent()); + assertNull(forPath("/bucket").getParent()); + } + + @Test + public void nameCount() { + assertEquals(forPath("/bucket/path/to/file").getNameCount(), 3); + assertEquals(forPath("/bucket/").getNameCount(), 0); + } + + + @Test + public void relativize() { + Path path = forPath("/bucket/path/to/file"); + Path other = forPath("/bucket/path/to/file/hello"); + + assertEquals(forPath("hello"), path.relativize(other)); + + // another + + assertEquals(forPath("file/hello"), forPath("/bucket/path/to/").relativize(forPath("/bucket/path/to/file/hello"))); + + // empty + + assertEquals(forPath(""), forPath("/bucket/path/to/").relativize(forPath("/bucket/path/to/"))); + } + + + // register + + @Test(expected = UnsupportedOperationException.class) + public void registerWithEventsThrowException() throws IOException { + forPath("file1").register(null); + } + + @Test(expected = UnsupportedOperationException.class) + public void registerThrowException() throws IOException { + forPath("file1").register(null); + } + + @Test(expected = UnsupportedOperationException.class) + public void registerWithEventsAndModiferThrowException() throws IOException { + forPath("file1").register(null); + } + + // to file + + @Test(expected = UnsupportedOperationException.class) + public void toFile() { + forPath("file1").toFile(); + } + + // compares to + + @Test + public void compare() { + assertTrue(forPath("file1").compareTo(forPath("file1")) == 0); + assertTrue(forPath("/path/file1").compareTo(forPath("/path/file1")) == 0); + assertTrue(forPath("/A/file1").compareTo(forPath("/B/file1")) == -1); + assertTrue(forPath("/B/file1").compareTo(forPath("/A/file1")) == 1); + assertTrue(forPath("/AA/file1").compareTo(forPath("/A/file1")) > 0); + assertTrue(forPath("a").compareTo(forPath("aa")) < 0); + assertTrue(forPath("ab").compareTo(forPath("aa")) > 0); + } + + // toRealPath + + @Test + public void toRealPathThrowException() throws IOException { + Path path = forPath("/file1"); + Path other = path.toRealPath(); + assertEquals(path, other); + } + + // toAbsolutePath + + @SuppressWarnings("unused") + @Test(expected = IllegalStateException.class) + public void toAbsolutePathRelativePathThrowException() throws IOException { + forPath("file1").toAbsolutePath(); + } + + @Test + public void toAbsolutePath() { + Path path = forPath("/file1"); + Path other = path.toAbsolutePath(); + assertEquals(path, other); + } + + @Test + public void hashCodeHashMap() { + HashMap hashMap = new HashMap<>(); + hashMap.put(forPath("/bucket/a"), "a"); + hashMap.put(forPath("/bucket/a"), "b"); + + assertEquals(1, hashMap.size()); + assertEquals("b", hashMap.get(forPath("/bucket/a"))); + } + + @Test(expected = IllegalArgumentException.class) + public void preconditions() { + S3FileSystem fileSystem = new S3FileSystemProvider().getFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST); + new S3Path(fileSystem, "/"); + } + + @Test + public void constructors() { + S3FileSystem fileSystem = new S3FileSystemProvider().getFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST); + S3Path path = new S3Path(fileSystem, "/buckname"); + assertEquals("buckname", path.getFileStore().name()); + assertEquals("", path.getKey()); + assertNull(path.getParent()); + assertEquals("", path.getKey()); + path = new S3Path(fileSystem, "/buckname/"); + assertEquals("buckname", path.getFileStore().name()); + assertEquals("", path.getKey()); + path = new S3Path(fileSystem, "/buckname/file"); + assertEquals("buckname", path.getFileStore().name()); + assertEquals("file", path.getKey()); + path = new S3Path(fileSystem, "/buckname/dir/file"); + assertEquals("buckname", path.getFileStore().name()); + assertEquals("dir/file", path.getKey()); + path = new S3Path(fileSystem, "dir/file"); + assertNull(path.getFileStore()); + assertEquals("dir/file", path.getKey()); + assertEquals("dir/", path.getParent().toString()); + path = new S3Path(fileSystem, "bla"); + assertNull(path.getFileStore()); + assertEquals("bla", path.getKey()); + assertNull(path.getParent()); + path = new S3Path(fileSystem, ""); + assertNull(path.getFileStore()); + assertEquals("", path.getKey()); + assertNull(path.getParent()); + } + + @Test(expected = UnsupportedOperationException.class) + public void register() throws IOException { + S3Path path = forPath("/buck/file"); + path.register(null); + } + + @Test(expected = UnsupportedOperationException.class) + public void registerWatchService() throws IOException { + S3Path path = forPath("/buck/file"); + path.register(null, new WatchEvent.Kind[0], new WatchEvent.Modifier[0]); + } + + private static S3Path forPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST).getPath(path); + } +} \ No newline at end of file diff --git a/src/test/java/com/upplication/s3fs/Path/StartsWithTest.java b/src/test/java/com/upplication/s3fs/Path/StartsWithTest.java new file mode 100644 index 0000000..7ffc02f --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/StartsWithTest.java @@ -0,0 +1,128 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; +import java.nio.file.Paths; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +public class StartsWithTest extends S3UnitTestBase { + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Test + public void startsWithBucket() { + assertTrue(getPath("/bucket/file1").startsWith(getPath("/bucket/"))); + assertFalse(getPath("/bucket/file1").startsWith(getPath("/bucket"))); + } + + @Test + public void startsWithBlank() { + assertFalse(getPath("/bucket/file1").startsWith(getPath(""))); + } + + @Test + public void startsWithBlankRelative() { + assertFalse(getPath("file1").startsWith(getPath(""))); + } + + @Test + public void startsWithSlash() { + assertTrue(getPath("/bucket/file").startsWith(getPath("/bucket/"))); + } + + @Test + public void startsWithBlankBlank() { + assertTrue(getPath("").startsWith(getPath(""))); + } + + @Test + public void startsWithOnlyBuckets() { + assertTrue(getPath("/bucket").startsWith(getPath("/bucket"))); + } + + @Test + public void startsWithRelativeVsAbsolute() { + assertFalse(getPath("/bucket/file1").startsWith(getPath("file1"))); + } + + @Test + public void startsWithRelativeVsAbsoluteInBucket() { + assertFalse(getPath("/bucket/file1").startsWith(getPath("bucket"))); + } + + @Test + public void startsWithFalse() { + assertFalse(getPath("/bucket/file1").startsWith(getPath("/bucket/file1/file2"))); + assertTrue(getPath("/bucket/file1/file2").startsWith(getPath("/bucket/file1"))); + } + + @Test + public void startsWithNotNormalize() { + assertFalse(getPath("/bucket/file1/file2").startsWith(getPath("/bucket/file1/../"))); + } + + @Test + public void startsWithNormalize() { + // in this implementation not exists .. or . special paths + assertFalse(getPath("/bucket/file1/file2").startsWith(getPath("/bucket/file1/../").normalize())); + } + + @Test + public void startsWithRelative() { + assertTrue(getPath("file/file1").startsWith(getPath("file"))); + } + + @Test + public void startsWithDifferentProvider() { + assertFalse(getPath("/bucket/hello").startsWith(Paths.get("/bucket"))); + } + + @Test + public void startsWithString() { + assertTrue(getPath("/bucket/hello").startsWith("/bucket/hello")); + } + + @Test + public void startsWithStringRelative() { + assertTrue(getPath("subkey1/hello").startsWith("subkey1/hello")); + } + + @Test + public void startsWithStringOnlyBuckets() { + assertTrue(getPath("/bucket").startsWith("/bucket")); + } + + @Test + public void startsWithStringRelativeVsAbsolute() { + assertFalse(getPath("/bucket/file1").startsWith("file1")); + } + + @Test + public void startsWithStringFalse() { + assertFalse(getPath("/bucket/file1").startsWith("/bucket/file1/file2")); + assertTrue(getPath("/bucket/file1/file2").startsWith("/bucket/file1")); + } + + @Test + public void startsWithStringRelativeVsAbsoluteInBucket() { + assertFalse(getPath("/bucket/file1").startsWith("bucket")); + } + +} diff --git a/src/test/java/com/upplication/s3fs/Path/SubpathTest.java b/src/test/java/com/upplication/s3fs/Path/SubpathTest.java new file mode 100644 index 0000000..934d055 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/SubpathTest.java @@ -0,0 +1,46 @@ +package com.upplication.s3fs.Path; + +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.FileSystems; + +import static com.upplication.s3fs.util.S3EndpointConstant.S3_GLOBAL_URI_TEST; +import static org.junit.Assert.assertEquals; + +public class SubpathTest extends S3UnitTestBase { + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void subPath0() { + // TODO: this is ok? + assertEquals(getPath("/bucket/path/"), getPath("/bucket/path/to/file").subpath(0, 1)); + } + + @Test + public void subPath() { + assertEquals(getPath("/bucket/path/to/"), getPath("/bucket/path/to/file").subpath(0, 2)); + assertEquals(getPath("/bucket/path/to/file"), getPath("/bucket/path/to/file").subpath(0, 3)); + assertEquals(getPath("to/"), getPath("/bucket/path/to/file").subpath(1, 2)); + assertEquals(getPath("to/file"), getPath("/bucket/path/to/file").subpath(1, 3)); + assertEquals(getPath("file"), getPath("/bucket/path/to/file").subpath(2, 3)); + } + + @Test(expected = IllegalArgumentException.class) + public void subPathOutOfRange() { + getPath("/bucket/path/to/file").subpath(0, 4); + } +} diff --git a/src/test/java/com/upplication/s3fs/Path/ToUriTest.java b/src/test/java/com/upplication/s3fs/Path/ToUriTest.java new file mode 100644 index 0000000..209a795 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/Path/ToUriTest.java @@ -0,0 +1,110 @@ +package com.upplication.s3fs.Path; + +import com.google.common.collect.ImmutableMap; +import com.upplication.s3fs.S3FileSystem; +import com.upplication.s3fs.S3FileSystemProvider; +import com.upplication.s3fs.S3Path; +import com.upplication.s3fs.S3UnitTestBase; +import com.upplication.s3fs.util.S3EndpointConstant; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.net.URI; +import java.nio.file.FileSystem; +import java.nio.file.FileSystems; +import java.nio.file.Path; +import java.util.Map; + +import static com.upplication.s3fs.AmazonS3Factory.ACCESS_KEY; +import static com.upplication.s3fs.AmazonS3Factory.SECRET_KEY; +import static com.upplication.s3fs.util.S3EndpointConstant.*; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +public class ToUriTest extends S3UnitTestBase { + + @Before + public void setup() throws IOException { + FileSystems + .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); + } + + @Test + public void toUri() { + Path path = getPath("/bucket/path/to/file"); + URI uri = path.toUri(); + + // the scheme is s3 + assertEquals("s3", uri.getScheme()); + + // could get the correct fileSystem + FileSystem fs = FileSystems.getFileSystem(uri); + assertTrue(fs instanceof S3FileSystem); + // the host is the endpoint specified in fileSystem + assertEquals(((S3FileSystem) fs).getEndpoint(), uri.getHost()); + + // bucket name as first path + Path pathActual = fs.provider().getPath(uri); + + assertEquals(path, pathActual); + } + + @Test + public void toUriWithEndSlash() { + S3Path s3Path = getPath("/bucket/folder/"); + + assertEquals(S3_GLOBAL_URI_TEST + "bucket/folder/", s3Path.toUri().toString()); + } + + @Test + public void toUriWithNotEndSlash() { + S3Path s3Path = getPath("/bucket/file"); + + assertEquals(S3_GLOBAL_URI_TEST + "bucket/file", s3Path.toUri().toString()); + } + + @Test + public void toUriRelative() { + S3FileSystem fileSystem = new S3FileSystemProvider() + .getFileSystem(S3_GLOBAL_URI_TEST); + + S3Path path = new S3Path(fileSystem, "bla"); + assertEquals(URI.create("bla"), path.toUri()); + } + + @Test + public void toUriBucketWithoutEndSlash() { + S3Path s3Path = getPath("/bucket"); + + assertEquals(S3_GLOBAL_URI_TEST.resolve("/bucket/"), s3Path.toUri()); + } + + @Test + public void toUriWithCredentials() { + Map envs = ImmutableMap.builder().put(ACCESS_KEY, "access").put(SECRET_KEY, "secret").build(); + FileSystem fileSystem = new S3FileSystemProvider() + .newFileSystem(S3_GLOBAL_URI_TEST, envs); + + Path path = fileSystem.getPath("/bla/file"); + + assertEquals(URI.create("s3://access@s3.test.amazonaws.com/bla/file"), path.toUri()); + } + + @Test + public void toUriWithEndpoint() throws IOException { + try (FileSystem fs = FileSystems.newFileSystem(URI.create("s3://endpoint/"), null)) { + Path path = fs.getPath("/bucket/path/to/file"); + URI uri = path.toUri(); + // the scheme is s3 + assertEquals("s3", uri.getScheme()); + assertEquals("endpoint", uri.getHost()); + assertEquals("/bucket/path/to/file", uri.getPath()); + } + } + + private static S3Path getPath(String path) { + return (S3Path) FileSystems.getFileSystem(S3_GLOBAL_URI_TEST).getPath(path); + } +} diff --git a/src/test/java/com/upplication/s3fs/S3FileStoreTest.java b/src/test/java/com/upplication/s3fs/S3FileStoreTest.java index eb7de36..a50d348 100644 --- a/src/test/java/com/upplication/s3fs/S3FileStoreTest.java +++ b/src/test/java/com/upplication/s3fs/S3FileStoreTest.java @@ -2,10 +2,7 @@ import static com.upplication.s3fs.AmazonS3Factory.ACCESS_KEY; import static com.upplication.s3fs.AmazonS3Factory.SECRET_KEY; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; +import static org.junit.Assert.*; import static org.mockito.Matchers.any; import java.io.IOException; @@ -40,7 +37,7 @@ public void prepareFileStore() throws IOException { AmazonS3ClientMock client = AmazonS3MockFactory.getAmazonClientMock(); client.bucket("bucket").file("placeholder"); client.bucket("bucket2").file("placeholder"); - S3Path path = fileSystem.getPath("/bucket"); + S3Path path = fileSystem.getPath("/bucket/"); fileStore = path.getFileStore(); } @@ -91,8 +88,9 @@ public void getOwner() { @Test public void getRootDirectory() { S3Path rootDirectory = fileStore.getRootDirectory(); - assertEquals("bucket", rootDirectory.getFileName().toString()); - assertEquals("/bucket/", rootDirectory.toAbsolutePath().toString()); + assertNull("", rootDirectory.getFileName()); + assertTrue(rootDirectory.isAbsolute()); + assertEquals("s3://access-mocked@s3.test.amazonaws.com/bucket/", rootDirectory.toAbsolutePath().toString()); assertEquals("s3://access-mocked@s3.test.amazonaws.com/bucket/", rootDirectory.toUri().toString()); } diff --git a/src/test/java/com/upplication/s3fs/S3FileSystemTest.java b/src/test/java/com/upplication/s3fs/S3FileSystemTest.java index 57802ca..a3b5c7d 100644 --- a/src/test/java/com/upplication/s3fs/S3FileSystemTest.java +++ b/src/test/java/com/upplication/s3fs/S3FileSystemTest.java @@ -19,10 +19,12 @@ import java.util.Map; import java.util.Set; +import com.amazonaws.regions.Regions; +import com.amazonaws.services.s3.AmazonS3; +import com.amazonaws.services.s3.AmazonS3ClientBuilder; import org.junit.Before; import org.junit.Test; -import com.amazonaws.services.s3.AmazonS3Client; import com.google.common.collect.ImmutableMap; import com.upplication.s3fs.util.AmazonS3ClientMock; import com.upplication.s3fs.util.AmazonS3MockFactory; @@ -147,8 +149,7 @@ public void getFileStores() { } @Test - public void getRootDirectoriesReturnBuckets() { - + public void getRootDirectories() { Iterable paths = fs.getRootDirectories(); assertNotNull(paths); @@ -158,10 +159,12 @@ public void getRootDirectoriesReturnBuckets() { boolean bucketNameB = false; for (Path path : paths) { - String name = path.getFileName().toString(); - if (name.equals("bucketA")) { + S3Path s3Path = (S3Path)path; + String fileStore = s3Path.getFileStore().name(); + Path fileName = s3Path.getFileName(); + if (fileStore.equals("bucketA") && fileName == null) { bucketNameA = true; - } else if (name.equals("bucketB")) { + } else if (fileStore.equals("bucketB") && fileName == null) { bucketNameB = true; } size++; @@ -183,11 +186,6 @@ public void supportedFileAttributeViewsReturnBasic() { assertTrue(operations.contains("posix")); } - @Test - public void getRootDirectories() { - fs.getRootDirectories(); - } - @Test public void close() throws IOException { assertTrue(fs.isOpen()); @@ -290,13 +288,15 @@ public void parts2Key() { @Test public void urlWithSpecialCharacters() throws IOException { String fileName = "βeta.png"; - String expected = "https://bucket.s3.amazonaws.com/%CE%B2eta.png"; + String expected = "https://bucket.s3.eu-west-1.amazonaws.com/%CE%B2eta.png"; - AmazonS3Client amazonS3Client = new AmazonS3Client(); + AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard() + .withRegion(Regions.EU_WEST_1) + .build(); S3FileSystem s3FileSystem = new S3FileSystem(null, null, amazonS3Client, "mirror"); S3Path path = new S3Path(s3FileSystem, fileName); - String url = amazonS3Client.getResourceUrl("bucket", path.getKey()); + String url = amazonS3Client.getUrl("bucket", path.getKey()).toString(); assertEquals(expected, url); } @@ -304,13 +304,15 @@ public void urlWithSpecialCharacters() throws IOException { @Test public void urlWithSpaceCharacters() throws IOException { String fileName = "beta gaming.png"; - String expected = "https://bucket.s3.amazonaws.com/beta%20gaming.png"; + String expected = "https://bucket.s3.eu-west-1.amazonaws.com/beta%20gaming.png"; - AmazonS3Client amazonS3Client = new AmazonS3Client(); + AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard() + .withRegion(Regions.EU_WEST_1) + .build(); S3FileSystem s3FileSystem = new S3FileSystem(null, null, amazonS3Client, "mirror"); S3Path path = new S3Path(s3FileSystem, fileName); - String url = amazonS3Client.getResourceUrl("bucket", path.getKey()); + String url = amazonS3Client.getUrl("bucket", path.getKey()).toString(); assertEquals(expected, url); } diff --git a/src/test/java/com/upplication/s3fs/S3PathTest.java b/src/test/java/com/upplication/s3fs/S3PathTest.java deleted file mode 100644 index 4ed91db..0000000 --- a/src/test/java/com/upplication/s3fs/S3PathTest.java +++ /dev/null @@ -1,613 +0,0 @@ -package com.upplication.s3fs; - -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; - -import java.io.IOException; -import java.net.URI; -import java.nio.file.FileSystem; -import java.nio.file.FileSystems; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.nio.file.WatchEvent; -import java.util.HashMap; -import java.util.Iterator; - -import com.upplication.s3fs.util.S3EndpointConstant; -import org.junit.Before; -import org.junit.Test; - -import com.github.marschall.memoryfilesystem.MemoryFileSystemBuilder; - -public class S3PathTest extends S3UnitTestBase { - - @Before - public void setup() throws IOException { - FileSystems - .newFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST, null); - } - - @Test - public void createNoPath() { - S3Path path = forPath("/bucket"); - - assertEquals("bucket", path.getFileStore().name()); - assertEquals("", path.getKey()); - } - - @Test - public void createWithTrailingSlash() { - S3Path path = forPath("/bucket/"); - - assertEquals(path.getFileStore().name(), "bucket"); - assertEquals(path.getKey(), ""); - } - - @Test - public void createWithPath() { - S3Path path = forPath("/bucket/path/to/file"); - - assertEquals(path.getFileStore().name(), "bucket"); - assertEquals(path.getKey(), "path/to/file"); - } - - @Test - public void createWithPathAndTrailingSlash() { - S3Path path = forPath("/bucket/path/to/file/"); - - assertEquals("bucket", path.getFileStore().name()); - assertEquals("path/to/file", path.getKey()); - } - - @Test - public void createRelative() { - S3Path path = forPath("path/to/file"); - - assertNull(path.getFileStore()); - assertEquals(path.getKey(), "path/to/file"); - assertFalse(path.isAbsolute()); - } - - @Test - public void getParent() { - assertEquals(forPath("/bucket/path/to/"), forPath("/bucket/path/to/file").getParent()); - assertEquals(forPath("/bucket/path/to/"), forPath("/bucket/path/to/file/").getParent()); - assertNull(forPath("/bucket/").getParent()); - assertNull(forPath("/bucket").getParent()); - } - - @Test - public void nameCount() { - assertEquals(forPath("/bucket/path/to/file").getNameCount(), 3); - assertEquals(forPath("/bucket/").getNameCount(), 0); - } - - @Test - public void resolve() { - assertEquals(forPath("/bucket/path/to/dir/").resolve(forPath("child/xyz")), forPath("/bucket/path/to/dir/child/xyz")); - assertEquals(forPath("/bucket/path/to/dir").resolve(forPath("child/xyz")), forPath("/bucket/path/to/dir/child/xyz")); - assertEquals(forPath("/bucket/path/to/file").resolve(forPath("")), forPath("/bucket/path/to/file")); // TODO: should this be "path/to/dir/" - assertEquals(forPath("path/to/file").resolve(forPath("child/xyz")), forPath("path/to/file/child/xyz")); - assertEquals(forPath("path/to/file").resolve(forPath("")), forPath("path/to/file")); // TODO: should this be "path/to/dir/" - assertEquals(forPath("/bucket/path/to/file").resolve(forPath("/bucket2/other/child")), forPath("/bucket2/other/child")); - } - - @Test - public void name() { - assertEquals(forPath("/bucket/path/to/file").getName(0), forPath("path/")); - assertEquals(forPath("/bucket/path/to/file").getName(1), forPath("to/")); - assertEquals(forPath("/bucket/path/to/file").getName(2), forPath("file")); - } - - @Test - public void subPath() { - assertEquals(forPath("/bucket/path/to/file").subpath(0, 1), forPath("path/")); - assertEquals(forPath("/bucket/path/to/file").subpath(0, 2), forPath("path/to/")); - assertEquals(forPath("/bucket/path/to/file").subpath(0, 3), forPath("path/to/file")); - assertEquals(forPath("/bucket/path/to/file").subpath(1, 2), forPath("to/")); - assertEquals(forPath("/bucket/path/to/file").subpath(1, 3), forPath("to/file")); - assertEquals(forPath("/bucket/path/to/file").subpath(2, 3), forPath("file")); - } - - @Test - public void iterator() { - Iterator iterator = forPath("/bucket/path/to/file").iterator(); - - assertEquals(iterator.next(), forPath("path/")); - assertEquals(iterator.next(), forPath("to/")); - assertEquals(iterator.next(), forPath("file")); - } - - @Test - public void resolveSibling() { - // absolute (non-root) vs... - assertEquals(forPath("/bucket/path/to/file").resolveSibling(forPath("other/child")), forPath("/bucket/path/to/other/child")); - assertEquals(forPath("/bucket/path/to/file").resolveSibling(forPath("/bucket2/other/child")), forPath("/bucket2/other/child")); - assertEquals(forPath("/bucket/path/to/file").resolveSibling(forPath("")), forPath("/bucket/path/to/")); - - // absolute (root) vs ... - assertEquals(forPath("/bucket").resolveSibling(forPath("other/child")), forPath("other/child")); - assertEquals(forPath("/bucket").resolveSibling(forPath("/bucket2/other/child")), forPath("/bucket2/other/child")); - assertEquals(forPath("/bucket").resolveSibling(forPath("")), forPath("")); - - // relative (empty) vs ... - assertEquals(forPath("").resolveSibling(forPath("other/child")), forPath("other/child")); - assertEquals(forPath("").resolveSibling(forPath("/bucket2/other/child")), forPath("/bucket2/other/child")); - assertEquals(forPath("").resolveSibling(forPath("")), forPath("")); - - // relative (non-empty) vs ... - assertEquals(forPath("path/to/file").resolveSibling(forPath("other/child")), forPath("path/to/other/child")); - assertEquals(forPath("path/to/file").resolveSibling(forPath("/bucket2/other/child")), forPath("/bucket2/other/child")); - assertEquals(forPath("path/to/file").resolveSibling(forPath("")), forPath("path/to/")); - } - - @Test - public void resolveSiblingString() { - // absolute (non-root) vs... - assertEquals(forPath("/bucket/path/to/file").resolveSibling("other/child"), forPath("/bucket/path/to/other/child")); - assertEquals(forPath("/bucket/path/to/file").resolveSibling("/bucket2/other/child"), forPath("/bucket2/other/child")); - assertEquals(forPath("/bucket/path/to/file").resolveSibling(""), forPath("/bucket/path/to/")); - - // absolute (root) vs ... - assertEquals(forPath("/bucket").resolveSibling("other/child"), forPath("other/child")); - assertEquals(forPath("/bucket").resolveSibling("/bucket2/other/child"), forPath("/bucket2/other/child")); - assertEquals(forPath("/bucket").resolveSibling(""), forPath("")); - - // relative (empty) vs ... - assertEquals(forPath("").resolveSibling("other/child"), forPath("other/child")); - assertEquals(forPath("").resolveSibling("/bucket2/other/child"), forPath("/bucket2/other/child")); - assertEquals(forPath("").resolveSibling(""), forPath("")); - - // relative (non-empty) vs ... - assertEquals(forPath("path/to/file").resolveSibling("other/child"), forPath("path/to/other/child")); - assertEquals(forPath("path/to/file").resolveSibling("/bucket2/other/child"), forPath("/bucket2/other/child")); - assertEquals(forPath("path/to/file").resolveSibling(""), forPath("path/to/")); - } - - @Test - public void relativize() { - Path path = forPath("/bucket/path/to/file"); - Path other = forPath("/bucket/path/to/file/hello"); - - assertEquals(forPath("hello"), path.relativize(other)); - - // another - - assertEquals(forPath("file/hello"), forPath("/bucket/path/to/").relativize(forPath("/bucket/path/to/file/hello"))); - - // empty - - assertEquals(forPath(""), forPath("/bucket/path/to/").relativize(forPath("/bucket/path/to/"))); - } - - // to uri - - @Test - public void toUri() { - Path path = forPath("/bucket/path/to/file"); - URI uri = path.toUri(); - - // the scheme is s3 - assertEquals("s3", uri.getScheme()); - - // could get the correct fileSystem - FileSystem fs = FileSystems.getFileSystem(uri); - assertTrue(fs instanceof S3FileSystem); - // the host is the endpoint specified in fileSystem - assertEquals(((S3FileSystem) fs).getEndpoint(), uri.getHost()); - - // bucket name as first path - Path pathActual = fs.provider().getPath(uri); - - assertEquals(path, pathActual); - } - - @Test - public void toUriWithEndpoint() throws IOException { - try (FileSystem fs = FileSystems.newFileSystem(URI.create("s3://endpoint/"), null)) { - Path path = fs.getPath("/bucket/path/to/file"); - URI uri = path.toUri(); - // the scheme is s3 - assertEquals("s3", uri.getScheme()); - assertEquals("endpoint", uri.getHost()); - assertEquals("/bucket/path/to/file", uri.getPath()); - } - } - - // tests startsWith - - @Test - public void startsWith() { - assertTrue(forPath("/bucket/file1").startsWith(forPath("/bucket"))); - } - - @Test - public void startsWithBlank() { - assertFalse(forPath("/bucket/file1").startsWith(forPath(""))); - } - - @Test - public void startsWithBlankRelative() { - assertFalse(forPath("file1").startsWith(forPath(""))); - } - - @Test - public void startsWithBlankBlank() { - assertTrue(forPath("").startsWith(forPath(""))); - } - - @Test - public void startsWithOnlyBuckets() { - assertTrue(forPath("/bucket").startsWith(forPath("/bucket"))); - } - - @Test - public void startsWithRelativeVsAbsolute() { - assertFalse(forPath("/bucket/file1").startsWith(forPath("file1"))); - } - - @Test - public void startsWithRelativeVsAbsoluteInBucket() { - assertFalse(forPath("/bucket/file1").startsWith(forPath("bucket"))); - } - - @Test - public void startsWithFalse() { - assertFalse(forPath("/bucket/file1").startsWith(forPath("/bucket/file1/file2"))); - assertTrue(forPath("/bucket/file1/file2").startsWith(forPath("/bucket/file1"))); - } - - @Test - public void startsWithNotNormalize() { - assertFalse(forPath("/bucket/file1/file2").startsWith(forPath("/bucket/file1/../"))); - } - - @Test - public void startsWithNormalize() { - // in this implementation not exists .. or . special paths - assertFalse(forPath("/bucket/file1/file2").startsWith(forPath("/bucket/file1/../").normalize())); - } - - @Test - public void startsWithRelative() { - assertTrue(forPath("file/file1").startsWith(forPath("file"))); - } - - @Test - public void startsWithDifferentProvider() { - assertFalse(forPath("/bucket/hello").startsWith(Paths.get("/bucket"))); - } - - @Test - public void startsWithString() { - assertTrue(forPath("/bucket/hello").startsWith("/bucket/hello")); - } - - @Test - public void startsWithStringRelative() { - assertTrue(forPath("subkey1/hello").startsWith("subkey1/hello")); - } - - @Test - public void startsWithStringOnlyBuckets() { - assertTrue(forPath("/bucket").startsWith("/bucket")); - } - - @Test - public void startsWithStringRelativeVsAbsolute() { - assertFalse(forPath("/bucket/file1").startsWith("file1")); - } - - @Test - public void startsWithStringFalse() { - assertFalse(forPath("/bucket/file1").startsWith("/bucket/file1/file2")); - assertTrue(forPath("/bucket/file1/file2").startsWith("/bucket/file1")); - } - - @Test - public void startsWithStringRelativeVsAbsoluteInBucket() { - assertFalse(forPath("/bucket/file1").startsWith("bucket")); - } - - // ends with - - @Test - public void endsWithAbsoluteRelative() { - assertTrue(forPath("/bucket/file1").endsWith(forPath("file1"))); - } - - @Test - public void endsWithAbsoluteAbsolute() { - assertTrue(forPath("/bucket/file1").endsWith(forPath("/bucket/file1"))); - } - - @Test - public void endsWithRelativeRelative() { - assertTrue(forPath("file/file1").endsWith(forPath("file1"))); - } - - @Test - public void endsWithRelativeAbsolute() { - assertFalse(forPath("file/file1").endsWith(forPath("/bucket"))); - } - - @Test - public void endsWithDifferenteFileSystem() { - assertFalse(forPath("/bucket/file1").endsWith(Paths.get("/bucket/file1"))); - } - - @Test - public void endsWithBlankRelativeAbsolute() { - assertFalse(forPath("").endsWith(forPath("/bucket"))); - } - - @Test - public void endsWithBlankBlank() { - assertTrue(forPath("").endsWith(forPath(""))); - } - - @Test - public void endsWithRelativeBlankAbsolute() { - assertFalse(forPath("/bucket/file1").endsWith(forPath(""))); - } - - @Test - public void endsWithRelativeBlankRelative() { - assertFalse(forPath("file1").endsWith(forPath(""))); - } - - @Test - public void endsWithDifferent() { - assertFalse(forPath("/bucket/dir/dir/file1").endsWith(forPath("fail/dir/file1"))); - } - - @Test - public void endsWithDifferentProvider() throws IOException { - try (FileSystem linux = MemoryFileSystemBuilder.newLinux().build("linux")) { - Path fileLinux = linux.getPath("/file"); - - assertFalse(forPath("/bucket/file").endsWith(fileLinux)); - } - - try (FileSystem window = MemoryFileSystemBuilder.newWindows().build("window")) { - Path file = window.getPath("c:/file"); - - assertFalse(forPath("/c/file").endsWith(file)); - } - } - - @Test - public void endsWithString() { - // endsWithAbsoluteRelative(){ - assertTrue(forPath("/bucket/file1").endsWith("file1")); - // endsWithAbsoluteAbsolute - assertTrue(forPath("/bucket/file1").endsWith("/bucket/file1")); - // endsWithRelativeRelative - assertTrue(forPath("file/file1").endsWith("file1")); - // endsWithRelativeAbsolute - assertFalse(forPath("file/file1").endsWith("/bucket")); - // endsWithBlankRelativeAbsolute - assertFalse(forPath("").endsWith("/bucket")); - // endsWithBlankBlank - assertTrue(forPath("").endsWith("")); - // endsWithRelativeBlankAbsolute - assertFalse(forPath("/bucket/file1").endsWith("")); - // endsWithRelativeBlankRelative - assertFalse(forPath("file1").endsWith("")); - } - - // register - - @Test(expected = UnsupportedOperationException.class) - public void registerWithEventsThrowException() throws IOException { - forPath("file1").register(null); - } - - @Test(expected = UnsupportedOperationException.class) - public void registerThrowException() throws IOException { - forPath("file1").register(null); - } - - @Test(expected = UnsupportedOperationException.class) - public void registerWithEventsAndModiferThrowException() throws IOException { - forPath("file1").register(null); - } - - // to file - - @Test(expected = UnsupportedOperationException.class) - public void toFile() { - forPath("file1").toFile(); - } - - // compares to - - @Test - public void compare() { - assertTrue(forPath("file1").compareTo(forPath("file1")) == 0); - assertTrue(forPath("/path/file1").compareTo(forPath("/path/file1")) == 0); - assertTrue(forPath("/A/file1").compareTo(forPath("/B/file1")) == -1); - assertTrue(forPath("/B/file1").compareTo(forPath("/A/file1")) == 1); - assertTrue(forPath("/AA/file1").compareTo(forPath("/A/file1")) > 0); - assertTrue(forPath("a").compareTo(forPath("aa")) < 0); - assertTrue(forPath("ab").compareTo(forPath("aa")) > 0); - } - - // toRealPath - - @Test - public void toRealPathThrowException() throws IOException { - Path path = forPath("/file1"); - Path other = path.toRealPath(); - assertEquals(path, other); - } - - // toAbsolutePath - - @SuppressWarnings("unused") - @Test(expected = IllegalStateException.class) - public void toAbsolutePathRelativePathThrowException() throws IOException { - forPath("file1").toAbsolutePath(); - } - - @Test - public void toAbsolutePath() { - Path path = forPath("/file1"); - Path other = path.toAbsolutePath(); - assertEquals(path, other); - } - - // get root - - @Test - public void getRootReturnBucket() { - assertEquals(forPath("/bucketA"), forPath("/bucketA/dir/file").getRoot()); - } - - @Test - public void getRootRelativeReturnNull() { - assertNull(forPath("dir/file").getRoot()); - } - - // file name - - @Test - public void getFileName() { - Path path = forPath("/bucketA/file"); - Path name = path.getFileName(); - - assertEquals(forPath("file"), name); - } - - @Test - public void getAnotherFileName() { - Path path = forPath("/bucketA/dir/another-file"); - Path fileName = path.getFileName(); - Path dirName = path.getParent().getFileName(); - - assertEquals(forPath("another-file"), fileName); - assertEquals(forPath("dir"), dirName); - } - - @Test - public void getFileNameBucket() { - Path path = forPath("/bucket"); - Path name = path.getFileName(); - assertEquals(name.toString(), "bucket"); - } - - // equals - - @Test - public void equals() { - Path path = forPath("/bucketA/dir/file"); - Path path2 = forPath("/bucketA/dir/file"); - - assertTrue(path.equals(path2)); - } - - @Test - public void notEquals() { - Path path = forPath("/bucketA/dir/file"); - Path path2 = forPath("/bucketA/dir/file2"); - - assertFalse(path.equals(path2)); - } - - @Test - public void notEqualsNull() { - Path path = forPath("/bucketA/dir/file"); - - assertFalse(path.equals(null)); - } - - @Test - public void notEqualsDifferentProvider() throws IOException { - Path path = forPath("/c/dir/file"); - - try (FileSystem linux = MemoryFileSystemBuilder.newLinux().build("linux")) { - Path fileLinux = linux.getPath("/dir/file"); - - assertFalse(path.equals(fileLinux)); - } - - try (FileSystem window = MemoryFileSystemBuilder.newWindows().build("window")) { - Path file = window.getPath("c:/dir/file"); - - assertFalse(path.equals(file)); - } - } - - @Test - public void hashCodeHashMap() { - HashMap hashMap = new HashMap<>(); - hashMap.put(forPath("/bucket/a"), "a"); - hashMap.put(forPath("/bucket/a"), "b"); - - assertEquals(1, hashMap.size()); - assertEquals("b", hashMap.get(forPath("/bucket/a"))); - } - - @Test(expected = IllegalArgumentException.class) - public void preconditions() { - S3FileSystem fileSystem = new S3FileSystemProvider().getFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST); - new S3Path(fileSystem, "/"); - } - - @Test - public void constructors() { - S3FileSystem fileSystem = new S3FileSystemProvider().getFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST); - S3Path path = new S3Path(fileSystem, "/buckname"); - assertEquals("buckname", path.getFileStore().name()); - assertEquals("buckname", path.getFileName().toString()); - assertNull(path.getParent()); - assertEquals("", path.getKey()); - path = new S3Path(fileSystem, "/buckname/"); - assertEquals("buckname", path.getFileStore().name()); - assertEquals("buckname", path.getFileName().toString()); - assertEquals("", path.getKey()); - path = new S3Path(fileSystem, "/buckname/file"); - assertEquals("buckname", path.getFileStore().name()); - assertEquals("file", path.getFileName().toString()); - assertEquals("file", path.getKey()); - path = new S3Path(fileSystem, "/buckname/dir/file"); - assertEquals("buckname", path.getFileStore().name()); - assertEquals("file", path.getFileName().toString()); - assertEquals("dir/file", path.getKey()); - path = new S3Path(fileSystem, "dir/file"); - assertNull(path.getFileStore()); - assertEquals("file", path.getFileName().toString()); - assertEquals("dir/file", path.getKey()); - assertEquals("dir", path.getParent().toString()); - path = new S3Path(fileSystem, "bla"); - assertNull(path.getFileStore()); - assertEquals("bla", path.getFileName().toString()); - assertEquals("bla", path.getKey()); - assertNull(path.getParent()); - assertNull(path.toUri()); - path = new S3Path(fileSystem, ""); - assertNull(path.getFileStore()); - assertEquals("", path.getFileName().toString()); - assertEquals("", path.getKey()); - assertNull(path.getParent()); - } - - @Test(expected = UnsupportedOperationException.class) - public void register() throws IOException { - S3Path path = forPath("/buck/file"); - path.register(null); - } - - @Test(expected = UnsupportedOperationException.class) - public void registerWatchService() throws IOException { - S3Path path = forPath("/buck/file"); - path.register(null, new WatchEvent.Kind[0], new WatchEvent.Modifier[0]); - } - - private static S3Path forPath(String path) { - return (S3Path) FileSystems.getFileSystem(S3EndpointConstant.S3_GLOBAL_URI_TEST).getPath(path); - } -} \ No newline at end of file diff --git a/src/test/java/com/upplication/s3fs/spike/FileStoreTest.java b/src/test/java/com/upplication/s3fs/spike/FileStoreTest.java new file mode 100644 index 0000000..182fd1f --- /dev/null +++ b/src/test/java/com/upplication/s3fs/spike/FileStoreTest.java @@ -0,0 +1,61 @@ +package com.upplication.s3fs.spike; + +import com.github.marschall.memoryfilesystem.MemoryFileSystemBuilder; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import java.io.IOException; +import java.nio.file.*; + +public class FileStoreTest { + + FileSystem fs; + FileSystem fsWindows; + FileSystem fsMac; + + @Before + public void setup() throws IOException { + fs = MemoryFileSystemBuilder.newLinux().build("linux"); + fsWindows = MemoryFileSystemBuilder.newWindows().build("windows"); + fsMac = MemoryFileSystemBuilder.newMacOs().build("mac"); + } + + @After + public void close() throws IOException { + fs.close(); + fsWindows.close(); + fsMac.close(); + } + + @Test + public void getFileStore() { + System.out.println("Default:"); + System.out.println("-------"); + for (FileStore fileStore : FileSystems.getDefault().getFileStores()) { + System.out.println("- " + fileStore.name()); + } + System.out.println("\nLinux:"); + System.out.println("-----"); + for (FileStore fileStore : fs.getFileStores()) { + System.out.println("- " + fileStore.name()); + } + System.out.println("\nWindows:"); + System.out.println("-------"); + for (FileStore fileStore : fsWindows.getFileStores()) { + System.out.println("- " + fileStore.name()); + } + System.out.println("\nMac:"); + System.out.println("---"); + for (FileStore fileStore : fsMac.getFileStores()) { + System.out.println("- " + fileStore.name()); + } + } + + + // ~ helpers methods + + private Path get(String path) { + return fs.getPath(path); + } +} diff --git a/src/test/java/com/upplication/s3fs/spike/PathSpecTest.java b/src/test/java/com/upplication/s3fs/spike/PathSpecTest.java index ca217c8..94f0218 100644 --- a/src/test/java/com/upplication/s3fs/spike/PathSpecTest.java +++ b/src/test/java/com/upplication/s3fs/spike/PathSpecTest.java @@ -6,9 +6,7 @@ import static org.junit.Assert.assertTrue; import java.io.IOException; -import java.nio.file.FileSystem; -import java.nio.file.Path; -import java.nio.file.Paths; +import java.nio.file.*; import org.junit.After; import org.junit.Before; @@ -19,15 +17,18 @@ public class PathSpecTest { FileSystem fs; + FileSystem fsWindows; @Before public void setup() throws IOException { fs = MemoryFileSystemBuilder.newLinux().build("linux"); + fsWindows = MemoryFileSystemBuilder.newWindows().build("windows"); } @After public void close() throws IOException { fs.close(); + fsWindows.close(); } // first and more @@ -150,6 +151,16 @@ public void endsWithRelativeBlankRelative() { assertFalse(get("file1").endsWith(get(""))); } + @Test + public void getParentNull() { + assertNull(get("/").getParent()); + } + + @Test + public void getParentWindowsNull() { + assertNull(fsWindows.getPath("C://").getParent()); + } + // file name @Test @@ -163,6 +174,63 @@ public void getFileName() throws IOException { } } + + @Test + public void getFileNameRootIsNull() throws IOException { + Path fileNameRoot = fs.getRootDirectories().iterator().next().getFileName(); + assertNull(fileNameRoot); + } + + // root + + @Test + public void getRootReturnBucket() { + assertEquals(get("/"), get("/dir/dir/file").getRoot()); + } + + + @Test(expected = FileAlreadyExistsException.class) + public void fileWithSameNameAsDir() throws IOException { + Files.createFile(fs.getPath("/tmp")); + Files.createDirectory(fs.getPath("/tmp/")); + } + + @Test(expected = FileAlreadyExistsException.class) + public void dirWithSameNameAsFile() throws IOException { + Files.createDirectories(fs.getPath("/tmp/")); + Files.createFile(fs.getPath("/tmp")); + } + + @Test + public void createDirWithoutEndSlash() throws IOException { + Path dir = Files.createDirectory(fs.getPath("/tmp")); + Files.isDirectory(dir); + } + + @Test + public void getRootRelativeReturnNull() { + assertNull(get("dir/file").getRoot()); + } + + @Test + public void getRoot() { + System.out.println("Default:"); + System.out.println("-------"); + for (Path root : FileSystems.getDefault().getRootDirectories()) { + System.out.println("- " + root); + } + System.out.println("\nLinux:"); + System.out.println("-----"); + for (Path root : fs.getRootDirectories()) { + System.out.println("- " + root); + } + System.out.println("\nWindows:"); + System.out.println("-------"); + for (Path root : fsWindows.getRootDirectories()) { + System.out.println("- " + root); + } + } + // ~ helpers methods private Path get(String path) { diff --git a/src/test/java/com/upplication/s3fs/spike/URISpikeTest.java b/src/test/java/com/upplication/s3fs/spike/URISpikeTest.java new file mode 100644 index 0000000..b938915 --- /dev/null +++ b/src/test/java/com/upplication/s3fs/spike/URISpikeTest.java @@ -0,0 +1,52 @@ +package com.upplication.s3fs.spike; + +import com.google.common.base.Splitter; +import org.junit.Test; + +import java.net.MalformedURLException; +import java.net.URI; +import java.nio.file.FileSystem; +import java.nio.file.FileSystems; +import java.util.Arrays; +import java.util.List; + +public class URISpikeTest { + + private List s3Path_cases = Arrays.asList( + "/bucket//folder//folder2/file", + "///bucket//folder//folder2/file", + "//bucket//folder//folder2/file", + "////bucket//folder//folder2/file", + "folder//folder2/file", + "folder//..//./folder2/file/.", + "folder/folder2/folder3/"); + + @Test + public void test() throws MalformedURLException { + for (String s3case : s3Path_cases) { + System.out.println(URI.create(s3case).toString() + + " ==> " + URI.create(s3case).normalize().toString()); + + System.out.println("Host:" + URI.create(s3case).getHost()); + System.out.println("Raw query:" + URI.create(s3case).getRawQuery()); + System.out.println("Authority:" + URI.create(s3case).getAuthority()); + System.out.println("Query:" + URI.create(s3case).getQuery()); + System.out.println("Raw Path:" + URI.create(s3case).getRawPath()); + System.out.println("Path:" + URI.create(s3case).getPath()); + + // get bucket + + System.out.println("Bucket:" + Splitter.on("/").omitEmptyStrings().splitToList(URI.create(s3case).normalize().toString()).get(0)); + System.out.println("Parent:" + URI.create(s3case).resolve("..").toString()); + } + + + } + + @Test + public void uriWithSpaces() throws MalformedURLException { + URI uri = FileSystems.getDefault().getPath("/file with spaces").toUri(); + System.out.println(uri); + } + +} diff --git a/src/test/java/com/upplication/s3fs/util/AmazonS3MockFactory.java b/src/test/java/com/upplication/s3fs/util/AmazonS3MockFactory.java index fbf55a5..bfb48fd 100644 --- a/src/test/java/com/upplication/s3fs/util/AmazonS3MockFactory.java +++ b/src/test/java/com/upplication/s3fs/util/AmazonS3MockFactory.java @@ -39,7 +39,9 @@ public static AmazonS3ClientMock getAmazonClientMock() { private static FileSystem getFsMem() { if (fsMem == null) try { - fsMem = MemoryFileSystemBuilder.newLinux().build(UUID.randomUUID().toString()); + fsMem = MemoryFileSystemBuilder.newLinux() + .setCurrentWorkingDirectory("/") + .build(UUID.randomUUID().toString()); } catch (IOException e) { throw new RuntimeException(e); } diff --git a/src/test/java/com/upplication/s3fs/util/CopyDirVisitor.java b/src/test/java/com/upplication/s3fs/util/CopyDirVisitor.java index 0a7282b..2f8a464 100644 --- a/src/test/java/com/upplication/s3fs/util/CopyDirVisitor.java +++ b/src/test/java/com/upplication/s3fs/util/CopyDirVisitor.java @@ -28,11 +28,11 @@ public CopyDirVisitor(Path fromPath, Path toPath) { @Override public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException { - // permitimos resolver entre distintos providers + // we allow to work against different providers Path targetPath = appendPath(dir); if (!Files.exists(targetPath)) { - if (!targetPath.getFileName().toString().endsWith("/")) { + if (!targetPath.toString().endsWith("/")) { targetPath = targetPath.getParent().resolve(targetPath.getFileName().toString() + "/"); } Files.createDirectory(targetPath); diff --git a/src/test/java/com/upplication/s3fs/util/EnvironmentBuilder.java b/src/test/java/com/upplication/s3fs/util/EnvironmentBuilder.java index 2c48f53..012b861 100644 --- a/src/test/java/com/upplication/s3fs/util/EnvironmentBuilder.java +++ b/src/test/java/com/upplication/s3fs/util/EnvironmentBuilder.java @@ -54,12 +54,15 @@ public static String getBucket() { String bucketName = System.getenv(BUCKET_NAME_KEY); if (bucketName != null) { + if (!bucketName.endsWith("/")) bucketName += "/"; return bucketName; } final Properties props = new Properties(); try { props.load(EnvironmentBuilder.class.getResourceAsStream("/amazon-test.properties")); - return props.getProperty(BUCKET_NAME_KEY); + bucketName = props.getProperty(BUCKET_NAME_KEY); + if (bucketName != null && !bucketName.endsWith("/")) bucketName += "/"; + return bucketName; } catch (IOException e) { throw new RuntimeException("needed /amazon-test.properties in the classpath"); } diff --git a/src/test/resources/META-INF/services/java.nio.file.spi.FileTypeDetector b/src/test/resources/META-INF/services/java.nio.file.spi.FileTypeDetector deleted file mode 100644 index f9c9b5b..0000000 --- a/src/test/resources/META-INF/services/java.nio.file.spi.FileTypeDetector +++ /dev/null @@ -1 +0,0 @@ -com.upplication.s3fs.util.FileTypeDetector \ No newline at end of file