Subset Folder Contents ArcPy

Question

I have about 1500 .txt files within a folder named 3410001ne => 3809962sw. I need to subset about 470 of these files to process with a Python script. Below is the section of code prior to my for loop which lists all of the files in a folder:

# Get the list of the text files to process
txt_list = arcpy.ListFiles("*.txt")

How can I subset the folder contents to include 470 of 1500 files ranging from 3609902sw => 3610032sw?

Answer 1

You could skip using ap.Listfiles all together and use a for loop with xrange...

for rt in xrange(3609902,3610032):
quads = ["%snw.txt"%rt,"%sne.txt"%rt,"%ssw.txt"%rt,"%sse.txt"%rt]
print quads
for quad in quads:
    if ap.Exists(quad):
        # Do whatever...

Answer 2

For starters, you could include sw in your wildcard statement (*sw.txt), which presumably would reduce your number of returned records substantially (assuming you have ne, nw, se, etc).

Second, now that you're working with a subset of files, use a conditional statement to widdle down your files to your exact needs.

Pseudo Code:

1. Set workspace (directory)
2. For loop with your txt_list var
3. Create a variable that stores the first 7 characters in the file name (ie. theNumbers = theFile[:6])
4. Cast the variable as an integer: int(theNumbers)
5. Test (conditional) that your variable is greater than or equal to 3609902 and less than or equal to 3610032
6. If YES, do whatever needs to be done

Answer 3

Based on explanation by Roy:

toprocess = [p for p in [int(filename[0:7]) for filename in txt_list] if p>=3609902 and p<=3610032]

Answer 4

you can do it with this way:

import os
import random

yourMainFolder = r'C:/out'
range1 = 3609902sw
range2 = 3610032sw

newAry = []

for a in (int(range1[:-2]) - 1, int(range2[:-2] + 1)):
    newAry.append(str(a) + range2[-2:])

for dirname, dirnames, filenames in os.walk('yourMainFolder'):
    for subdirname in dirnames:
        path = os.path.join(dirname, subdirname)
        if str(subdirname) in newAry:
            files = [f for f in os.listdir(path) if f.endswith('.txt')]

#print files[random.randint(0, 469)]

for a in range(len(files)):
    if a < 470:
         print files[a]

i hope it helps you...