Python Scripting Techniques for Automation
Table of Contents
- Introduction to Automation with Python
- Technique 1: Automating System Administration Tasks
- Technique 2: Using the
subprocessModule for Command Execution - Technique 3: Reading and Writing Files with Python
- Technique 4: Working with CSV and Excel Files
- Technique 5: Automating Email Sending using
smtplib - Technique 6: Using
pandasfor Data Manipulation and Analysis - Technique 7: Automating GUI Tasks with
pyautoguiandtkinter - Technique 8: Using
scrapyfor Web Scraping - Technique 9: Automating System Monitoring with
psutil - Technique 10: Creating a Custom Script using a CLI
- Technique 11: Using
paramikofor Remote System Administration - Technique 12: Automating Database Operations with
sqlite3 - Technique 13: Using
schedulefor Scheduling Tasks - Technique 14: Automating Web Page Interactions with
selenium - Technique 15: Using
pytesseractfor Optical Character Recognition - Technique 16: Automating Network Tasks with
scapy - Conclusion and Further Reading
Introduction to Automation with Python
Python is a versatile language that can be used for automating various tasks, from simple file operations to complex system administration tasks. With its vast collection of libraries and modules, Python provides an ideal platform for automation.
In this article, we’ll explore 18 different Python scripting techniques that can be used for automation in various domains.
Technique 1: Automating System Administration Tasks
Python can be used to automate various system administration tasks such as user management, file system operations, and network configuration. The shutil, os, and subprocess modules are commonly used for these tasks.
“`python
import os
import shutil
Create a new directory
new_dir = ‘/path/to/new/dir’
if not os.path.exists(new_dir):
os.makedirs(new_dir)
Copy a file to the new directory
src_file = ‘/path/to/source/file.txt’
dst_file = os.path.join(new_dir, ‘file.txt’)
shutil.copy(src_file, dst_file)
“`
Technique 2: Using the subprocess Module for Command Execution
The subprocess module can be used to execute shell commands and capture their output.
“`python
import subprocess
Execute a command and capture its output
command = ‘ls -l’
output = subprocess.check_output(command, shell=True)
print(output.decode())
“`
Technique 3: Reading and Writing Files with Python
Python provides various libraries for reading and writing files in different formats such as CSV, Excel, JSON, and XML.
“`python
import csv
Read a CSV file
with open(‘data.csv’, ‘r’) as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
print(row)
Write to a CSV file
fieldnames = [‘name’, ‘age’]
with open(‘output.csv’, ‘w’) as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
writer.writerow({‘name’: ‘John’, ‘age’: 25})
“`
Technique 4: Working with CSV and Excel Files
Python provides various libraries such as pandas for working with CSV and Excel files.
“`python
import pandas as pd
Read an Excel file
df = pd.read_excel(‘data.xlsx’)
print(df)
Write to an Excel file
df.to_excel(‘output.xlsx’, index=False)
“`
Technique 5: Automating Email Sending using smtplib
Python’s smtplib library can be used to send emails.
“`python
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
Set up the mail server
server = smtplib.SMTP(‘smtp.gmail.com’, 587)
server.starttls()
server.login(‘your-email@gmail.com’, ‘your-password’)
Create a message
msg = MIMEMultipart()
msg[‘From’] = ‘your-email@gmail.com’
msg[‘To’] = ‘recipient-email@gmail.com’
msg[‘Subject’] = ‘Test Email’
Attach the message body
body = ‘Hello, this is a test email.’
msg.attach(MIMEText(body, ‘plain’))
Send the email
server.send_message(msg)
server.quit()
“`
Technique 6: Using pandas for Data Manipulation and Analysis
Python’s pandas library provides data structures and functions to efficiently handle structured data.
“`python
import pandas as pd
Create a DataFrame
df = pd.DataFrame({‘name’: [‘John’, ‘Mary’, ‘Jane’],
‘age’: [25, 31, 22]})
Filter the DataFrame
filtered_df = df[df[‘age’] > 24]
Group the DataFrame
grouped_df = df.groupby(‘name’)
Calculate statistics
print(grouped_df.size())
“`
Technique 7: Automating GUI Tasks with pyautogui and tkinter
Python’s pyautogui library can be used to automate GUI tasks, while tkinter is a built-in Python library for creating graphical user interfaces.
“`python
import pyautogui
Move the mouse cursor
pyautogui.moveTo(100, 100)
Click on an element
element = pyautogui.locateOnScreen(‘button.png’)
if element:
pyautogui.click(element)
Create a GUI window
root = tkinter.Tk()
label = tkinter.Label(root, text=’Hello, world!’)
label.pack()
root.mainloop()
“`
Technique 8: Using scrapy for Web Scraping
Python’s scrapy library is a powerful tool for web scraping.
“`python
import scrapy
class MySpider(scrapy.Spider):
name = ‘example’
start_urls = [‘https://example.com/’]
def parse(self, response):
yield {
'title': response.css('h1::text').get(),
}
“`
Technique 9: Automating System Monitoring with psutil
Python’s psutil library can be used to monitor system resources and processes.
“`python
import psutil
Get the current CPU usage
cpu_usage = psutil.cpu_percent()
print(f’CPU Usage: {cpu_usage}%’)
Get the memory information
mem_info = psutil.virtual_memory()
print(f’Memory: {mem_info.percent}%’)
“`
Technique 10: Creating a Custom Script using a CLI
You can create a custom script using a command-line interface (CLI) tool like argparse.
“`python
import argparse
Define the argument parser
parser = argparse.ArgumentParser(description=’My Script’)
Add arguments to the parser
parser.add_argument(‘–name’, type=str, help=’Name’)
parser.add_argument(‘–age’, type=int, help=’Age’)
Parse the command-line arguments
args = parser.parse_args()
print(f’Hello, {args.name}! You are {args.age} years old.’)
“`
Technique 11: Using paramiko for Remote System Administration
Python’s paramiko library can be used to perform remote system administration tasks.
“`python
import paramiko
Create a SSH client
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
Connect to the server
ssh_client.connect(‘server.com’, username=’username’, password=’password’)
Run a command on the remote server
stdin, stdout, stderr = ssh_client.exec_command(‘ls -l’)
print(stdout.readlines())
“`
Technique 12: Automating Database Operations with sqlite3
Python’s sqlite3 library can be used to interact with SQLite databases.
“`python
import sqlite3
Connect to the database
conn = sqlite3.connect(‘example.db’)
Create a cursor object
cur = conn.cursor()
Execute a SQL query
cur.execute(“SELECT * FROM example_table”)
rows = cur.fetchall()
print(rows)
Close the connection
conn.close()
“`
Technique 13: Using requests for API Interactions
Python’s requests library can be used to interact with web APIs.
“`python
import requests
Send a GET request
response = requests.get(‘https://example.com/api/data’)
print(response.json())
Send a POST request
data = {‘key’: ‘value’}
response = requests.post(‘https://example.com/api/endpoint’, json=data)
print(response.status_code)
“`
Technique 14: Using xml.etree.ElementTree for XML Parsing
Python’s xml.etree.ElementTree library can be used to parse and manipulate XML documents.
“`python
import xml.etree.ElementTree as ET
Parse an XML file
tree = ET.parse(‘example.xml’)
root = tree.getroot()
Access an element
print(root.tag)
print(root.attrib)
Create a new XML element
new_element = ET.Element(‘new-tag’)
Add the new element to the root
root.append(new_element)
Write the modified XML to a file
tree.write(‘modified_example.xml’)
“`
This concludes our tour of various Python libraries and techniques for automating tasks, web scraping, data analysis, and more. Each library has its own unique features and use cases, so be sure to explore them in depth to unlock their full potential!
About the Author
