Here’s a structured documentation to create and schedule tasks and flows in Prefect based on your setup and examples.
-
Create Python Virtual Environment:
python3 -m venv prefect_env1
-
Activate the Virtual Environment:
- Windows:
.\prefect_env1\Scripts\activate
- macOS/Linux:
source prefect_env1/bin/activate
- Windows:
-
Install Prefect:
pip install prefect
-
Check Prefect Version:
prefect --version
-
Log In/Log Out of Prefect Cloud (if needed):
prefect cloud logout prefect cloud login
-
Start Prefect Server (Local):
prefect server start
-
Set Prefect API URL:
prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api
In Prefect, a task is a basic unit of work, often used to perform a specific operation or function.
from prefect import task
@task
def say_hello(name: str):
print(f"Hello, {name}!")
- Explanation: This task takes a
name
parameter and prints a greeting message. - Usage: Define multiple tasks as needed to perform individual operations within a flow.
A flow is a collection of tasks, defining the sequence in which the tasks should be executed.
from prefect import flow
@flow
def hello_goodbye_flow(name: str):
say_hello(name)
say_goodbye(name)
- Explanation: This flow is defined to call both
say_hello
andsay_goodbye
tasks sequentially, greeting and then saying goodbye to the name provided.
To allow flexibility in your workflows, you can parameterize your flows by adding arguments that can be set at runtime.
from prefect import flow, task
@task
def say_hello(name: str):
print(f"Hello, {name}!")
@task
def say_goodbye(name: str):
print(f"Goodbye, {name}!")
@flow
def hello_goodbye_flow(name: str):
say_hello(name)
say_goodbye(name)
- Usage: You can invoke
hello_goodbye_flow
with a specific name:hello_goodbye_flow(name="Jitendra")
Use the deployment setup with an interval to schedule flows.
if __name__ == "__main__":
hello_goodbye_flow.serve(
name="Demo Deployment",
tags=["Demo"],
parameters={"name": "Jitendra"},
interval=60 # runs every 60 seconds
)
- Explanation:
interval=60
schedules the flow to run every 60 seconds.
With Prefect, flows can be set up for periodic or event-based automation by configuring schedules. In this example, the flow is automated to run every 60 seconds as demonstrated above.