-
-
Notifications
You must be signed in to change notification settings - Fork 88
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
43 changed files
with
61,542 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
.PHONY: create-env update-env | ||
|
||
REPO=$(shell basename $(CURDIR)) | ||
|
||
create-env: | ||
python3 -m venv .$(REPO); | ||
. .$(REPO)/bin/activate; \ | ||
pip3 install --upgrade -r requirements-dev.txt; \ | ||
python setup.py develop; | ||
|
||
update-env: | ||
. .$(REPO)/bin/activate; \ | ||
pip3 install --upgrade -r requirements-dev.txt; | ||
|
||
attach-kernel: | ||
python -m ipykernel install --user --name=$(REPO); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
# Python Package | ||
|
||
## Desenvolvimento Linux e Mac: | ||
|
||
Clone o repositório principal: | ||
|
||
```sh | ||
git clone https://github.com/basedosdados/mais.git | ||
``` | ||
Entre na pasta local do repositório usando `cd mais/` e suba o ambiente localmente: | ||
|
||
```sh | ||
make create-env | ||
. .mais/bin/activate | ||
cd python-package/ | ||
python setup.py develop | ||
``` | ||
|
||
### Desenvolva uma nova feature | ||
|
||
1. Abra uma branch com o nome issue-<X> | ||
2. Faça as modificações necessárias | ||
3. Suba o Pull Request apontando para a branch `python-next-minor` ou `python-next-patch`. | ||
Sendo, minor e patch referentes ao bump da versão: v1.5.7 --> v\<major>.\<minor>.\<patch>. | ||
4. O nome do PR deve seguir o padrão | ||
`[infra] <titulo explicativo>` | ||
|
||
|
||
### O que uma modificação precisa ter | ||
|
||
|
||
- Resolver o problema | ||
- Lista de modificações efetuadas | ||
1. Mudei a função X para fazer Y | ||
2. Troquei o nome da variavel Z | ||
- Referência aos issues atendidos | ||
- Documentação e Docstrings | ||
- Testes | ||
|
||
|
||
## Versionamento | ||
|
||
**Para publicar uma nova versão do pacote é preciso seguir os seguintes passos:** | ||
|
||
1. Fazer o pull da branch: | ||
|
||
```bash | ||
git pull origin [python-version] | ||
``` | ||
|
||
Onde `[python-version]` é a branch da nova versão do pacote. | ||
|
||
2. Se necessario adicionar novas dependências: | ||
```bash | ||
poetry add <package-name> | ||
``` | ||
|
||
3. Gerar novo `requirements-dev.txt` | ||
|
||
```bash | ||
poetry export -f requirements.txt --output requirements-dev.txt --without-hashes | ||
``` | ||
|
||
4. Editar `pyproject.toml`: | ||
|
||
O arquivo `pyproject.toml` contém, entre outras informações, a versão do pacote em python da **BD**. Segue excerto do arquivo: | ||
|
||
```toml | ||
description = "Organizar e facilitar o acesso a dados brasileiros através de tabelas públicas no BigQuery." | ||
homepage = "https://github.com/base-dos-dados/bases" | ||
license = "MIT" | ||
name = "basedosdados" | ||
packages = [ | ||
{include = "basedosdados"}, | ||
] | ||
readme = "README.md" | ||
repository = "https://github.com/base-dos-dados/bases" | ||
version = "1.6.1-beta.2" | ||
``` | ||
|
||
O campo `version` deve ser alterado para o número da versão sendo lançada. | ||
|
||
5. Editar `basedosdados/__init__.py`: | ||
|
||
O arquivo `basedosdados/__init__.py` contém a versão do pacote em python da **BD**. Exemplo: | ||
|
||
```python | ||
__version__ = "1.6.1-beta.2" | ||
``` | ||
|
||
O atributo `__version__` também deve ser alterado para o número da versão sendo lançada. | ||
|
||
6. Push para branch: | ||
|
||
```bash | ||
git push origin [python-version] | ||
``` | ||
|
||
7. Publicação do pacote no PyPI (exige usuário e senha): | ||
Para publicar o pacote no PyPI, use: | ||
|
||
Para publicar o pacote no PyPI, use: | ||
|
||
```bash | ||
poetry version [python-version] | ||
poetry publish --build | ||
``` | ||
8. Faz merge da branch para a master | ||
9. Faz release usando a UI do GitHub | ||
10. Atualizar versão do pacote usada internamente |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
""" | ||
Importing the module will automatically import the submodules. | ||
""" | ||
import os | ||
import sys | ||
|
||
from basedosdados._version import __version__ | ||
from basedosdados._warnings import show_warnings | ||
|
||
show_warnings() | ||
|
||
sys.path.append(f"{os.getcwd()}/python-package") | ||
|
||
from basedosdados.backend import Backend | ||
from basedosdados.constants import config, constants | ||
from basedosdados.download.download import download, read_sql, read_table | ||
from basedosdados.download.metadata import ( | ||
get_columns, | ||
get_datasets, | ||
get_tables, | ||
search, | ||
) | ||
from basedosdados.upload.connection import Connection | ||
from basedosdados.upload.dataset import Dataset | ||
from basedosdados.upload.storage import Storage | ||
from basedosdados.upload.table import Table |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
import importlib.metadata | ||
|
||
__version__ = importlib.metadata.version("basedosdados") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
""" | ||
Checks for warnings and general-purpose messages and displays them to the user. | ||
""" | ||
|
||
from typing import List, Tuple | ||
|
||
import requests | ||
from loguru import logger | ||
|
||
from basedosdados._version import __version__ | ||
|
||
|
||
def get_latest_version_number(): | ||
"""Get the latest version number from PyPI.""" | ||
try: | ||
response = requests.get( | ||
"https://pypi.python.org/pypi/basedosdados/json", timeout=5 | ||
) | ||
return response.json()["info"]["version"] | ||
except: # noqa | ||
logger.warning( | ||
"Could not check for updates. Please check your internet connection." | ||
) | ||
return None | ||
|
||
|
||
def compare_version_numbers(versionA: str, versionB: str) -> Tuple[int, str, str]: | ||
""" | ||
Compares two version numbers and returns the difference between them. | ||
Args: | ||
versionA (str): The first version number. | ||
versionB (str): The second version number. | ||
Returns: | ||
1 if versionA > versionB | ||
0 if versionA == versionB | ||
-1 if versionA < versionB | ||
""" | ||
|
||
def parse_version(version: str): | ||
version: List[str] = version.split(".") | ||
if len(version) == 1: | ||
version.append(0) | ||
version.append(0) | ||
elif len(version) == 2: | ||
version.append(0) | ||
elif len(version) > 3: | ||
version = version[:3] | ||
version[0] = int(version[0]) | ||
version[1] = int(version[1]) | ||
patch = "" | ||
for char in version[2]: | ||
if char.isdigit(): | ||
patch += char | ||
else: | ||
break | ||
version_type = "" | ||
if "b" in version[2]: | ||
version_type = "BETA" | ||
elif "a" in version[2]: | ||
version_type = "ALPHA" | ||
else: | ||
version_type = "GA" | ||
version[2] = int(patch) | ||
version.append(version_type) | ||
return version | ||
|
||
versionA = parse_version(versionA) | ||
versionB = parse_version(versionB) | ||
|
||
# Compare types. If both types are the same, compare the numbers. | ||
if versionA[3] == versionB[3]: | ||
for i in range(3): | ||
if versionA[i] > versionB[i]: | ||
return 1, versionA[3], versionB[3] | ||
elif versionA[i] < versionB[i]: | ||
return -1, versionA[3], versionB[3] | ||
return 0, versionA[3], versionB[3] | ||
# If the types are different, don't compare the numbers. | ||
return 0, versionA[3], versionB[3] | ||
|
||
|
||
def show_warnings(): | ||
"""Show warnings and general-purpose messages to the user.""" | ||
# Version warning | ||
try: | ||
latest_version = get_latest_version_number() | ||
if latest_version is not None: | ||
comparison = compare_version_numbers(__version__, latest_version) | ||
if comparison[0] == -1: | ||
logger.warning( | ||
f"You are using an outdated version of basedosdados ({__version__}). " | ||
f"Please upgrade to the latest version ({latest_version}) using " | ||
"'pip install --upgrade basedosdados'." | ||
) | ||
except: # noqa | ||
logger.warning( | ||
"Could not check for updates. Please check your internet connection." | ||
) | ||
# General-purpose warnings and messages | ||
try: | ||
response = requests.get( | ||
"https://basedosdados.github.io/notifications/data.json" | ||
) | ||
data = response.json() | ||
if "general" in data: | ||
if "messages" in data["general"]: | ||
for message in data["general"]["messages"]: | ||
logger.info(message) | ||
if "warnings" in data["general"]: | ||
for warning in data["general"]["warnings"]: | ||
logger.warning(warning) | ||
if "python" in data: | ||
if "messages" in data["python"]: | ||
for message in data["python"]["messages"]: | ||
logger.info(message) | ||
if "warnings" in data["python"]: | ||
for warning in data["python"]["warnings"]: | ||
logger.warning(warning) | ||
except: # noqa | ||
logger.warning( | ||
"Could not check for warnings and messages. Please check your internet connection." | ||
) |
Oops, something went wrong.