Skip to content

Commit

Permalink
Merge pull request #13 from NarrativeScience/pants-2.0.0-features
Browse files Browse the repository at this point in the history
QPT-32631: Pants 2.0.0 Support
  • Loading branch information
ns-bdesimone authored Dec 18, 2020
2 parents 6127024 + 7cf0f2e commit fe8f544
Show file tree
Hide file tree
Showing 10 changed files with 42 additions and 65 deletions.
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: 2.1

orbs:
ghpr: narrativescience/ghpr@0.0.5
ghpr: narrativescience/ghpr@1.1.1

commands:
install-deps:
Expand Down
22 changes: 10 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,15 +135,13 @@ type = library
; Flag denoting whether to generate a BUILD file.
; generate_build_file = true

; Flag denoting whether to generate a python_binary target for local.py. This is
; Flag denoting whether to generate a pex_binary target for local.py. This is
; essentially an extra entry point. It's only used for specific package types.
; generate_local_binary = false

; Flag denoting whether to include a python_binary target for pytest
; Flag denoting whether to include a pex_binary target for pytest
; generate_pytest_binary = false

; Flag denoting whether to include a coverage attribute on pytest targets
; include_test_coverage = true
```

### Package Types
Expand Down Expand Up @@ -186,7 +184,7 @@ python_library(
sources=["cli_deploy/**/*"],
tags={"apps", "code", "python"},
)
python_binary(
pex_binary(
name="deploy",
dependencies=[":lib"],
source="cli_deploy/cli.py",
Expand All @@ -195,7 +193,7 @@ python_binary(
```

- The `python_library` target is pretty much the same as an internal Python library package
- The `python_binary` target defines an explicit name. This is because when we go to build the PEX file, we want to define the filename. In this example, running `./pants binary apps/cli_deploy/src:deploy` will result in `dist/deploy.pex`.
- The `pex_binary` target defines an explicit name. This is because when we go to build the PEX file, we want to define the filename. In this example, running `./pants binary apps/cli_deploy/src:deploy` will result in `dist/deploy.pex`.
- The only dependency for the binary should be the library. The library will then include all the dependencies.
- `source` points to the entry point of the binary. This module should handle the `if __name__ == "__main__"` condition to kick off the script.

Expand All @@ -219,7 +217,7 @@ python_tests(
sources=["**/*.py"],
tags={"lib", "python", "tests", "unit"},
)
python_binary(
pex_binary(
name="unittest",
entry_point="unittest",
dependencies=[":lib/time_utils/tests/unit"]
Expand All @@ -228,11 +226,11 @@ python_binary(

- The `python_library` target is mostly here to define the unit tests dependencies in a single place so the other two targets can point to it
- The `python_tests` target lets us run pytest against the test files that match `**/*.py`
- The `python_binary` target lets us run the unittest module directly. We won't actually package up this target via `./pants binary`. Setting the entry_point to `"unittest"` is essentially the same as running `python -m unittest test_something.py` from the command line.
- The `pex_binary` target lets us run the unittest module directly. We won't actually package up this target via `./pants binary`. Setting the entry_point to `"unittest"` is essentially the same as running `python -m unittest test_something.py` from the command line.

#### `lambda_function`

The BUILD file for the Lambda handler contains a special-purpose build target: `python_awslambda`. This target is a wrapper around [lambdex](https://github.com/wickman/lambdex). It creates a PEX like the `python_binary` target (you can execute it) but it modifies the PEX to work with a Lambda Function. For example:
The BUILD file for the Lambda handler contains a special-purpose build target: `python_awslambda`. This target is a wrapper around [lambdex](https://github.com/wickman/lambdex). It creates a PEX like the `pex_binary` target (you can execute it) but it modifies the PEX to work with a Lambda Function. For example:

```python
python_library(
Expand All @@ -243,7 +241,7 @@ python_library(
"lib/logger/src",
],
)
python_binary(
pex_binary(
name="my-lambda-bin",
source="lambda_handler/lambda_handler.py",
dependencies=[":my-lambda-lib"],
Expand Down Expand Up @@ -272,7 +270,7 @@ python_library(
sources=["**/*"],
tags={"code", "db", "migration", "python"},
)
python_binary(name="alembic", entry_point="alembic.config", dependencies=[":lib"])
pex_binary(name="alembic", entry_point="alembic.config", dependencies=[":lib"])
python_app(
name="migrations-my-database-name",
archive="tar",
Expand Down Expand Up @@ -302,7 +300,7 @@ python_library(
sources=["**/*"],
tags={"integration", "python", "tests", "tests-integration"},
)
python_binary(
pex_binary(
source="behave_cli.py",
dependencies=[":lib"],
tags={"integration", "python", "tests", "tests-integration"},
Expand Down
2 changes: 1 addition & 1 deletion src/pypants/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""CLI for working with Python packages and BUILD files in a Pants monorepo"""

__version__ = "1.0.1"
__version__ = "2.0.1"
8 changes: 4 additions & 4 deletions src/pypants/build_targets/behave_.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
class PythonBehaveTestPackage(PythonTestPackage):
"""Represents a Python test package build target in Pants that uses behave"""

def _generate_python_binary_behave_wrapper_node(self) -> ast.Expr:
"""Generate an AST node for a python_binary Pants target that wraps behave"""
def _generate_pex_binary_behave_wrapper_node(self) -> ast.Expr:
"""Generate an AST node for a pex_binary Pants target that wraps behave"""
node = ast.Expr(
value=ast.Call(
func=ast.Name(id="python_binary"),
func=ast.Name(id="pex_binary"),
args=[],
keywords=[
ast.keyword(
Expand All @@ -31,7 +31,7 @@ def generate_build_file_ast_node(self) -> ast.Module:
node = ast.Module(
body=[
self._generate_python_library_ast_node(name="lib"),
self._generate_python_binary_behave_wrapper_node(),
self._generate_pex_binary_behave_wrapper_node(),
self._generate_python_library_resources_ast_node(),
]
)
Expand Down
21 changes: 12 additions & 9 deletions src/pypants/build_targets/binary.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,12 @@ def _parse_entry_point(self) -> Tuple[str, str]:
)
return binary_name, source_module_path

def _generate_python_binary_cli_ast_node(self) -> ast.Expr:
"""Generate an AST node for a python_binary Pants target"""
def _generate_pex_binary_cli_ast_node(self) -> ast.Expr:
"""Generate an AST node for a pex_binary Pants target"""
binary_name, source_module_path = self._parse_entry_point()
node = ast.Expr(
value=ast.Call(
func=ast.Name(id="python_binary"),
func=ast.Name(id="pex_binary"),
args=[],
keywords=[
ast.keyword(arg="name", value=ast.Str(binary_name)),
Expand All @@ -108,14 +108,17 @@ def _generate_python_binary_cli_ast_node(self) -> ast.Expr:
)
return node

def _generate_python_binary_local_ast_node(self) -> ast.Expr:
"""Generate an AST node for a python_binary Pants target that runs local.py"""
def _generate_pex_binary_local_ast_node(self) -> ast.Expr:
"""Generate an AST node for a pex_binary Pants target that runs local.py"""
node = ast.Expr(
value=ast.Call(
func=ast.Name(id="python_binary"),
func=ast.Name(id="pex_binary"),
args=[],
keywords=[
ast.keyword(arg="name", value=ast.Str("local")),
ast.keyword(
arg="name",
value=ast.Str(f"local-{self.package_name.rsplit('_', 1)[1]}"),
),
ast.keyword(
arg="dependencies",
value=ast.List(
Expand All @@ -138,10 +141,10 @@ def generate_build_file_ast_node(self) -> ast.Module:
self._generate_python_library_ast_node(
name="lib", globs_path=f"{self.package_name}/**/*.py"
),
self._generate_python_binary_cli_ast_node(),
self._generate_pex_binary_cli_ast_node(),
]
if self.config.generate_local_binary:
body.append(self._generate_python_binary_local_ast_node())
body.append(self._generate_pex_binary_local_ast_node())
if self.config.resource_glob_path:
body.append(self._generate_python_library_resources_ast_node())
node = ast.Module(body=body)
Expand Down
10 changes: 5 additions & 5 deletions src/pypants/build_targets/lambda_function.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ def dependency_target(self) -> str:
"""
return f"{self.key}:lib"

def _generate_python_binary_cli_ast_node(self) -> ast.Expr:
"""Generate an AST node for a python_binary Pants target that runs lambda_handler.py""" # noqa
def _generate_pex_binary_cli_ast_node(self) -> ast.Expr:
"""Generate an AST node for a pex_binary Pants target that runs lambda_handler.py""" # noqa
node = ast.Expr(
value=ast.Call(
func=ast.Name(id="python_binary"),
func=ast.Name(id="pex_binary"),
args=[],
keywords=[
ast.keyword(arg="name", value=ast.Str("bin")),
Expand Down Expand Up @@ -87,10 +87,10 @@ def generate_build_file_ast_node(self) -> ast.Module:
self._generate_python_library_ast_node(
name="lib", globs_path=f"{self.package_name}/**/*.py"
),
self._generate_python_binary_cli_ast_node(),
self._generate_pex_binary_cli_ast_node(),
self._generate_python_lambda_ast_node(),
]
if self.config.generate_local_binary:
body.append(self._generate_python_binary_local_ast_node())
body.append(self._generate_pex_binary_local_ast_node())
node = ast.Module(body=body)
return node
2 changes: 1 addition & 1 deletion src/pypants/build_targets/migration.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ def generate_build_file_ast_node(self) -> ast.Module:
node = ast.Module(
body=[
self._generate_python_library_ast_node(name="lib"),
self._generate_python_binary_wrapper_node(
self._generate_pex_binary_wrapper_node(
"alembic", entry_point="alembic.config", dependencies=[":lib"]
),
self._generate_migrations_python_app_ast_node(),
Expand Down
8 changes: 4 additions & 4 deletions src/pypants/build_targets/python_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -243,15 +243,15 @@ def _generate_python_library_ast_node(
)
return node

def _generate_python_binary_wrapper_node(
def _generate_pex_binary_wrapper_node(
self,
name: str,
entry_point: Optional[str] = None,
dependencies: Optional[List[str]] = None,
) -> ast.Expr:
"""Generate an AST node for a python_binary Pants target with an entry point
"""Generate an AST node for a pex_binary Pants target with an entry point
See: https://www.pantsbuild.org/python_readme.html#python_binary-entry_point
See: https://www.pantsbuild.org/python_readme.html#pex_binary-entry_point
Args:
name: Name of the binary, e.g. unittest
Expand All @@ -268,7 +268,7 @@ def _generate_python_binary_wrapper_node(

node = ast.Expr(
value=ast.Call(
func=ast.Name(id="python_binary"),
func=ast.Name(id="pex_binary"),
args=[],
keywords=[
ast.keyword(arg="name", value=ast.Str(name)),
Expand Down
20 changes: 2 additions & 18 deletions src/pypants/build_targets/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
import ast

from .python_package import PythonPackage
from .requirement import PythonRequirement


class PythonTestPackage(PythonPackage):
Expand All @@ -27,21 +26,6 @@ def _generate_python_tests_ast_node(self) -> ast.Expr:
ast.keyword(arg="sources", value=ast.List(elts=[ast.Str("**/*.py")])),
self._tags_keyword,
]
if self.config.include_test_coverage:
if self.code_target_package_name is None:
# Go through the python_library dependencies and get the package names
# Do not include 3rdparty packages
code_target_package_names = {
d.package_name
for d in self.dependencies
if not isinstance(d, PythonRequirement)
and d.config.include_test_coverage
}
# Build the coverage based on the dependency packages
elements = [ast.Str(pkg) for pkg in sorted(code_target_package_names)]
else:
elements = [ast.Str(self.code_target_package_name)]
keywords.append(ast.keyword(arg="coverage", value=ast.List(elts=elements)))
node = ast.Expr(
value=ast.Call(func=ast.Name(id="python_tests"), args=[], keywords=keywords)
)
Expand All @@ -54,13 +38,13 @@ def generate_build_file_ast_node(self) -> ast.Module:
name=self.rendered_package_name, include_extra_dependencies=True
),
self._generate_python_tests_ast_node(),
self._generate_python_binary_wrapper_node(
self._generate_pex_binary_wrapper_node(
"unittest", dependencies=[f":{self.rendered_package_name}"]
),
]
if self.config.generate_pytest_binary:
body.append(
self._generate_python_binary_wrapper_node(
self._generate_pex_binary_wrapper_node(
"pytest", dependencies=[f":{self.rendered_package_name}"]
)
)
Expand Down
12 changes: 2 additions & 10 deletions src/pypants/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ class Config:
generate_build_file = <bool>
generate_local_binary = <bool>
generate_pytest_binary = <bool>
include_test_coverage = <bool>
type = "library"|"binary"|...
Instantiating this class will load and initialize a config parser instance.
Expand Down Expand Up @@ -125,25 +124,18 @@ def generate_build_file(self) -> bool:

@property
def generate_local_binary(self) -> bool:
"""Flag denoting whether to generate a python_binary target for local.py"""
"""Flag denoting whether to generate a pex_binary target for local.py"""
return self._config.getboolean(
"package", "generate_local_binary", fallback=False
)

@property
def generate_pytest_binary(self) -> bool:
"""Flag denoting whether to include a python_binary target for pytest"""
"""Flag denoting whether to include a pex_binary target for pytest"""
return self._config.getboolean(
"package", "generate_pytest_binary", fallback=False
)

@property
def include_test_coverage(self) -> bool:
"""Flag denoting whether to include a coverage attribute on pytest targets"""
return self._config.getboolean(
"package", "include_test_coverage", fallback=True
)

@property
def type(self) -> Optional[str]:
"""Package type.
Expand Down

0 comments on commit fe8f544

Please sign in to comment.