Robotic Python Automation – Automating Working with Files with Python. (Cleaning, Sorting and Archiving)

Robotic Python Automation Files Cleaning Sorting and Archiving.

Sometimes we have folders where files are saved that get out of control. Sometimes we have Excel documents or Work files that pile up and we need an efficient way to sift and sort through them and either archive files or delete them. Using Python, we can build a script that automatically completes two tasks.

Looks for files of a certain name and archives them
Looks for files older than a certain data and deletes them

Setup

Create a new Jupyter Notebook and name it Document_Automation

Create a new folder on your C Drive “C:\Python\Automation” and seed it with 10 empty files that follow the naming conventin file_YYYYMMDD and log_YYYYMMDD. Create two folders called “Files” and “Logs”

Setup

The following command will list out all text files located in the folder you are searching for and place the date in to a list called “mylist”. The “*” is a wildcard that allows you to search for all character combinations that end with “.txt”. This can be updated to look for other file types

import glob
mylist = glob.glob("C:/Python/Automation/*.txt")
print(mylist)

Next we can try to create our folders where we will organize our files. Since we already created them, the try/catch will fail but if you had deleted them you will see them reappear. The OS and shutil need to be imported for these commands to work

import os, shutil
try: 
    os.mkdir("C:/Python/Automation/Files")
    os.mkdir("C:/Python/Automation/Files")
except OSError:
     print ("Folder Creation Failed")
else:
    print ("Folder Creation Success")

Start with a loop to go through your list

for i in mylist: 
     print(i)

Lets update the look to search for any “log” first. This can be done with a string startswith() command or using the “in” command with an If statement. Both work in the case but be careful if your file naming rules can change. We will use the latter example for the rest of the tutorial.

for i in mylist: 
    if (i.startswith('C:/Python/Automation\log_')):
        print(i)

for i in mylist: 
    if ("log" in i):
        print(i)

Now move the files found in the search to the destination folder using the Move command. We need to specify the source location and destination location so we use Replace string manipulation to create both of those strings to pass to the function. Once run, you will see the files moved to the respective folders.

for i in mylist: 
    if ("log" in i):
        #print(i)
        source = i
        destination = i.replace('C:/Python/Automation', 'C:/Python/Automation/Logs') 
        shutil.move(source, destination)

We can now do the same with the files titled “Files”

for i in mylist: 
    if ("file" in i):
        #print(i)
        source = i
        destination = i.replace('C:/Python/Automation', 'C:/Python/Automation/Files') 
        shutil.move(source, destination)

Cleaning out old Files

Finally, we want to delete any old files. We can do this using the remove and time functions in Python to identify the last modified time of the file. In this sample 1800 is 30 minutes in seconds. This can be adjusted to make the window longer or a few days. We need to import time and sys to be able to the correct commands. Uncomment the If statement to use parameters on the time duration.

import os, time, sys

path = r'C:/Python/Automation/Logs'

now = time.time()

for f in os.listdir(path):
    #if os.stat(os.path.join(path,f)).st_mtime < now - 7 * 86400:
    if os.stat(os.path.join(path,f)).st_mtime < now:
        print("Removing File")
        os.remove(os.path.join(path,f))

Check out my these Udemy Classes below to get started in Python

And Other articles here below.

Robotic Python Automation Files Cleaning Sorting and Archiving