Skip to content

jitesh-raut/Prefect-New

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Here’s a structured documentation to create and schedule tasks and flows in Prefect based on your setup and examples.


Prefect Installation and Environment Setup

Create Python Virtual Environment & Install Prefect

  1. Create Python Virtual Environment:

    python3 -m venv prefect_env1
  2. Activate the Virtual Environment:

    • Windows:
      .\prefect_env1\Scripts\activate
    • macOS/Linux:
      source prefect_env1/bin/activate
  3. Install Prefect:

    pip install prefect
  4. Check Prefect Version:

    prefect --version
  5. Log In/Log Out of Prefect Cloud (if needed):

    prefect cloud logout
    prefect cloud login
  6. Start Prefect Server (Local):

    prefect server start
  7. Set Prefect API URL:

    prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api

1. Creating a Simple Prefect Task

In Prefect, a task is a basic unit of work, often used to perform a specific operation or function.

Example Task: say_hello

from prefect import task

@task
def say_hello(name: str):
    print(f"Hello, {name}!")
  • Explanation: This task takes a name parameter and prints a greeting message.
  • Usage: Define multiple tasks as needed to perform individual operations within a flow.

2. Creating a Simple Prefect Flow

A flow is a collection of tasks, defining the sequence in which the tasks should be executed.

Example Flow: hello_goodbye_flow

from prefect import flow

@flow
def hello_goodbye_flow(name: str):
    say_hello(name)
    say_goodbye(name)
  • Explanation: This flow is defined to call both say_hello and say_goodbye tasks sequentially, greeting and then saying goodbye to the name provided.

3. Creating and Scheduling a Parameterized Prefect Flow

To allow flexibility in your workflows, you can parameterize your flows by adding arguments that can be set at runtime.

Example: Parameterized Flow

from prefect import flow, task

@task
def say_hello(name: str):
    print(f"Hello, {name}!")

@task
def say_goodbye(name: str):
    print(f"Goodbye, {name}!")

@flow
def hello_goodbye_flow(name: str):
    say_hello(name)
    say_goodbye(name)
  • Usage: You can invoke hello_goodbye_flow with a specific name:
    hello_goodbye_flow(name="Jitendra")

Scheduling the Parameterized Flow

Use the deployment setup with an interval to schedule flows.

if __name__ == "__main__":
    hello_goodbye_flow.serve(
        name="Demo Deployment",
        tags=["Demo"],
        parameters={"name": "Jitendra"},
        interval=60  # runs every 60 seconds
    )
  • Explanation: interval=60 schedules the flow to run every 60 seconds.

4. Automating and Scheduling a Prefect Flow

With Prefect, flows can be set up for periodic or event-based automation by configuring schedules. In this example, the flow is automated to run every 60 seconds as demonstrated above.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published