Bash for Python: smooth directory tree

Question

On Unix-like systems, I use this script, which would help me port to Python for execution on Windows hosts:

---

#!/bin/bash SENTINEL_FILENAME='__sentinel__' SENTINEL_MD5_CHECKSUM='' SENTINEL_SHA_CHECKSUM='' function is_directory_to_be_flattened() { local -r directory_to_consider="$1" local -r sentinel_filepath="${directory_to_consider}/${SENTINEL_FILENAME}" if [ ! -f "${sentinel_filepath}" ]; then return 1 fi if [[ "$( md5 "${sentinel_filepath}" \ | awk '{ print $NF }' 2> /dev/null )" \ == "${SENTINEL_MD5_CHECKSUM}" && \ "$( shasum -a 512 "${sentinel_filepath}" \ | awk '{ print $1 }' 2> /dev/null )" \ == "${SENTINEL_SHA_CHECKSUM}" ]]; then return 0 else return 1 fi } function conditionally_flatten() { local -r directory_to_flatten="$1" local -r flatten_into_directory="$2" if is_directory_to_be_flattened "${directory_to_flatten}"; then if [ ! -d "${flatten_into_directory}" ]; then mkdir -v "${flatten_into_directory}" fi for file_to_move in $(find ${directory_to_flatten} -type f -maxdepth 1); do mv \ -v \ -n \ "${file_to_move}" \ "${flatten_into_directory}" done fi } function flatten_directory() { local -r directory_to_flatten="$1" local -r descend_depth="$2" local -r flattened_directory="${directory_to_flatten}/__flattened__" if [ ! -d "${directory_to_flatten}" ]; then printf "The argument '%s' does not seem to be a directory.\n" \ "${directory_to_flatten}" \ >&2 return fi find "${directory_to_flatten}" \ -type d \ -maxdepth "${descend_depth}" \ | \ while read directory_path; do conditionally_flatten \ "${directory_path}" \ "${flattened_directory}" done } n_arguments="$#" if [ "${n_arguments}" -eq 1 ]; then flatten_directory "$1" '1' # maybe use a constant, not a "magic #" here? else echo usage: "$0" /path/to/directory/to/flatten fi unset is_directory_to_be_flattened unset conditionally_flatten unset flatten_directory 

---

How do you port this to Win Python? I start in both Python and Bash scripts.

Feel free to update my implementation when you port it, if you are missing it too, with justification please. This is not a “Code Review”, but the “thumbs up / thumbs down” on my efforts at Bash could give me an idea of ​​whether I am improving myself or should I change the way I generally study ...

---

Here we are, my attempt in Python: (criticize it if necessary, this is the only way to recognize me!)

---

 #!/usr/bin/env python2.7 import sys import os import shutil SENTINEL_FILENAME='' SENTINEL_MD5_CHECKSUM='' SENTINEL_SHA_CHECKSUM='' DEFAULT_DEPTH = 1 FLATTED_DIRECTORY_NAME = '__flattened__' def is_directory_to_be_flattened(directory_to_consider): sentinel_location = os.path.join(directory_to_consider, SENTINEL_FILENAME) if not os.path.isfile(sentinel_location): return False import hashlib with open(sentinel_location) as sentinel_file: file_contents = sentinel_file.read() return (hashlib.md5(file_contents).hexdigest() == SENTINEL_MD5_CHECKSUM and hashlib.sha512(file_contents).hexdigest() == SENTINEL_SHA_CHECKSUM) def flatten(directory, depth, to_directory, do_files_here): if depth < 0: return contained_filenames = [f for f in os.listdir(directory)] if do_files_here: for filename in contained_filenames: if filename == SENTINEL_FILENAME: continue filepath = os.path.join(directory, filename) if not os.path.isfile(filepath): continue file_to = os.path.join(to_directory, filename) if not os.path.isdir(to_directory): os.makedirs(to_directory) if not os.path.isfile(file_to): print "Moving: '{}' -> '{}'".format(filepath, file_to) shutil.move(filepath, file_to) else: sys.stderr.write('Error: {} exists already.\n'.format(file_to)) next_depth = depth - 1 for subdirectory in (d for d in contained_filenames if os.path.isdir(d)): if is_directory_to_be_flattened(subdirectory): flatten(subdirectory, next_depth, to_directory, True) def flatten_directory(to_flatten, depth): to_directory = os.path.join(to_flatten, FLATTED_DIRECTORY_NAME) if not os.path.isdir(to_flatten): sys.stderr.write( 'The argument {} does not seem to be a directory.\n'.format( to_flatten)) return flatten(to_flatten, depth, to_directory, False) def main(): if len(sys.argv) == 2: flatten_directory(sys.argv[1], DEFAULT_DEPTH) else: print 'usage: {} /path/to/directory/to/flatten'.format(sys.argv[0]) if __name__ == '__main__': main() 

---

Although this is obvious from the code, the goal is:

• Start in the given directory
• Go down to a certain depth
• Consider the subdirectories and move all the files there if and only if:
  • The directory contains a "watch file" with the given file name
  • The sentinel file is actually a signal file, not just a file renamed under the same name
• Map the files in the __flattened__ directory in the directory in which the search was run

Answer 1

Most file functions in Python are located in the os module - there you will find os.rename (for renaming or moving Directoruy entries), os.listdir - which gives you a list of file names in the directory passed as the first arg, os.walk for recursively navigating through the directory structure, os.path.walk, do the same, but with the callback os.path.exists, os.path.isdir, os.mkdir, are others that may be convenient.

For a "quick and dirty" translation, you can also cehck "os.system". which allows you to execute a shell command just as it was entered into the shell, and os.popen - which allows you to access the stdin and stdout of the specified process. A more thorough translation, tough, will require the use of the anothe: "subprocess" module, which can give one full control over a shell command that runs as a subprocess (although if you need to find , for example, it will not be available in windows)

Another interesting topic is sys (sys.argv - the arguments passed to the script) and shutil (with things like copy, rmtree, etc.)

Your script does some error checking, and it’s trivial, given the above function names in “os” and base Python, to add them, but a short “just do it” script in Python might be simple:

 import os, sys dir_to_flatten = sys.argv[1] for dirpath, dirnames, filenames in os.walk(dir_to_flatten): for filename in filenames: try: os.rename(os.path.join(dirpath, filename), os.path.join(dir_to_flatten, filename)) except OSError: print ("Could not move %s " % os.path.join(dirpath, filename))