Task Automation with Python

Description

The course empowers programming professionals to streamline and automate a wide range of workflows—whether in development, data processing, or system administration—by harnessing the power of Python. Beginning with environment setup and best practices for logging, students learn to manipulate files and folders, handle command-line arguments, apply regular expressions, and manage subprocesses. They then delve into concurrent and network programming, exploring asynchronous operations and API integrations to build more responsive, scalable automations. Optional modules introduce cutting-edge Generative AI tools such as GitHub Copilot and OpenAI’s APIs, as well as techniques for working with tabular data using Pandas. Finally, the course can be tailored to organizational needs with additional modules on distributed task automation frameworks like Airflow, Celery, or Faust. Additional modules may lengthen the course.

Objectives

• Understand task automation and the advantages of using Python for automating workflows.
• Set up a Python development environment with Visual Studio Code and code linting tools.
• Leverage GitHub Copilot and AI tools to assist with Python scripting and command-line tasks.
• Implement structured logging using Python’s logging module, including log levels, handlers, and formatting.
• Work with exit codes and error handling in Python automation scripts.
• Manage environment variables across operating systems and subprocesses.
• Perform file and folder operations, including working with formats like CSV, JSON, Excel, Word, and PDF.
• Use Python for subprocess orchestration, async programming, and API-driven automation.

Duration

14 hours of intensive training with live instruction delivered over two or three days to accommodate varied scheduling needs. Additional modules, may lengthen the course.

Request Information

Introduction

• What is Task Automation?
• Why Use Python for Task Automation?
• Python Automation Libraries
• Python Automation Tools
• Python Automation Best Practices
• Task Automation Examples and Ideas

Development Environment

• Configure Visual Studio Code for Python Script Programming
• Python Code Linting & Reformatting with Ruff & MyPy
• Debugging Python Scripts with Visual Studio Code
• Docker Desktop

Generative AI Tooling (optional)

• GitHub Copilot and Visual Studio Code Integration
• Using GitHub Copilot for Python Programming
• Using GitHub Copilot for Terminal Commands
• Leverage ChatGPT to Program Task Automation Scripts
• Customize the Course with Your Generative AI Programmer Tool (optional)

Logging

• What is Logging?
• Benefits of Logging
• Python Logging Module
• Log Levels
• Log Handlers
• Log Formatters
• Customize the Course with Your Logging Destination (optional)

Exit Codes

• What are Exit Codes?
• Return Exit Codes
• Capture Exit Codes
• Handle Exit Codes
• Error Handling, Exit Codes, and Logging

Environment Variables

• What are Environment Variables?
• Setting Environment Variables for a Script (Windows, macOS, and Linux)
• Reading Environment Variables
• Setting Environment Variables in Python for Child Processes

Files, Folders, and Operating System

• OS Module vs PathLib Module
• Current Working Directory
• File / Folder Paths
• Reading and Writing Files
• Asynchronous Reading and Writing Files

Document Processing

• Work with Various File Formats such as CSV, JSON, and YAML
• Read and Write Excel Files
• Convert from Markdown to Word
• Convert from Word to PDF
• Modify Word Files
• Modify PDF Files
• Customize the Course with Your Preferred File Formats (optional)

High-Level File Operations

• ShUtil Module
• Copy/Move/Delete Files
• Copy/Move/Delete Directory Tree
• Determine Disk Usage
• Working with File Archives
• Create, Remove, & List Directories

Command Line Arguments

• Arg Parse Module
• Click Package
• Positional Arguments
• Named Arguments
• Default Arguments
• Argument Data Types
• Argument Documentation

Regular Expressions

• Syntax overview
• Regular Expression Objects
• Searching and Matching
• Compilation Flags
• Groups and Named Groups
• Replacing Text
• Splitting Strings

Subprocess

• Run Method
• POpen Class
• Running Subprocesses
• Capture and Use Subprocess Output
• Exit Codes

Concurrent and Network Programming

• Orchestrating Subprocesses
• Access Data Sources Asynchronously
• Use SFTP to Download/Upload Files
• Download/Upload Files from/to S3 Buckets (optional)
• Download/Upload Files from/to Azure Blob Storage (optional)
• Calling OpenAI API with Python (optional)
• AI Embeddings, RAG, and PostgreSQL with Vector Extension (optional)
• Integration with GitHub Rest API and Web Hooks (optional)
• Task Automation in GitHub Actions via Docker (optional)
• Customize the Course with Your Preferred API Server Integrations (optional)

Conclusion

• Summary of Key Concepts
• Q&A
• Further Resources and Next Steps

Prerequisites

• Basic understanding of Python programming.
• Familiarity with command-line interfaces (CLI).
• Basic knowledge of file systems and operating system concepts.

Training Materials

All students receive comprehensive courseware covering all topics in the course. The instructor distributes courseware via GitHub. The courseware includes documentation and extensive code samples. Students practice the topics covered through challenging hands-on lab exercises. Students will need a free, personal GitHub account to access the courseware. All students will need a modern web browser such as Google Chrome. Student machines will need a text editor, such as VS Code, the latest LTS version of Node.js, the latest Python version, PanDoc, and OpenOffice. Students will need permission to install NPM packages and PyPi packages. Preconfigured student virtual machines can provided upon request. For some demostrations, students will need access to 3rd party services such as AWS, Azure, and GitHub. The need for these services will be discussed as part of planning the specifics of the class.