Read python variable in shell script?

Question

There are two variables in my python file:

week_date = "01/03/16-01/09/16" cust_id = "12345" 

how can i read this in a shell script that takes these 2 variables?

my current shell script requires manual editing of "dt" and "id". I want to read python variables in a shell script, so I can just edit the python options file and not many files.

shell file:

 #!/bin/sh dt="01/03/16-01/09/16" cust_id="12345" 

In the new python file, I can just import the python parameter file.

Answer 1

Consider something similar to the following:

 #!/bin/bash # ^^^^ NOT /bin/sh, which doesn't have process substitution available. python_script=' import sys d = {} # create a context for variables exec(open(sys.argv[1], "r").read()) in d # execute the Python code in that context for k in sys.argv[2:]: print "%s\0" % str(d[k]).split("\0")[0] # ...and extract your strings NUL-delimited ' read_python_vars() { local python_file=$1; shift local varname for varname; do IFS= read -r -d '' "${varname#*:}" done < <(python -c "$python_script" "$python_file" "${@%%:*}") } 

Then you can use this as:

 read_python_vars config.py week_date:dt cust_id:id echo "Customer id is $id; date range is $dt" 

... or, if you do not want to rename variables as you read them, simply:

 read_python_vars config.py week_date cust_id echo "Customer id is $cust_id; date range is $week_date" 

---

Benefits:

• Unlike a naive regex-based solution (which may have problems with some of the details of Python parsing - try teaching sed to process both source and regular strings, and both single and triple quotes, without turning them into a ball!) or a similar approach that used output using a new line from a Python subprocess, this will correctly handle any object for which str() gives a view without NUL characters that your shell script can use.

• Running content through the Python interpreter also means that you can determine the values ​​programmatically - for example, you may have Python code that asks your version control system for the last modification date of the corresponding content.

  Think of scenarios like this:

   start_date = '01/03/16' end_date = '01/09/16' week_date = '%s-%s' % (start_date, end_date) 

  ... using the Python interpreter for parsing Python means that you do not limit how people can update / modify your Python configuration file in the future.

Now let me say the reservations:

• If your Python code has side effects, these side effects will obviously take effect (just as if you had chosen the import file as a module in Python). Do not use this to retrieve a configuration from a file whose contents you do not trust.
• Python strings are Pascal-style: they can contain literal NULs. Strings in shell languages ​​are C-style: they end with the first NUL character. Thus, some variables can exist in Python, which cannot be represented in the shell without literary shielding. To prevent an object whose str() representation contains NULs flows to other destinations, this code breaks lines in the first NUL.

---

Now let's talk about implementation details.

• ${@%%:*} is the $@ extension that trims all contents after and includes the first : in each argument, thus passing only the names of Python variables to the interpreter. Similarly, ${varname#*:} is an extension that aligns everything to and including the first : on behalf of the variable passed to read . See the bash-hackers page for parameter extension .
• Using <(python ...) is the process substitution syntax: the expression <(...) evaluates the name of the file, which when read will produce the output of this command. Using < <(...) redirects the output from this file and, therefore, this command (the first < is a redirection, while the second is part of the token <( which starts the process replacement). Using this form for output in a while read allows avoid the error mentioned in BashFAQ # 24 ("I set the variables in a loop that is in the pipeline. Why do they disappear after the loop ends? Or, why can't I pass data to read?") .

• The IFS= read -r -d '' construct has a number of components, each of which makes the read behavior more true to the original content:

  • Clearing IFS throughout the command prevents trimming whitespace from the end of the variable contents.
  • Using -r prevents the use of the literal backslash of read itself rather than the output.
  • Using -d '' sets the first character of the empty string '' as a record separator. Since C strings have a NUL end and the shell uses C strings, this character is NUL. This ensures that the contents of the variables can contain any value other than NUL, including literal newlines.

  See BashFAQ # 001 ("How can I read a file (data stream, variable) line (and / or field by field)?") For more information about the process of reading write-oriented data from a string in bash.

Answer 2

Other answers provide a way to do exactly what you ask, but I think the idea is a little crazy. There, an easier way to satisfy both scenarios is to move these variables to the configuration file. You can even save a simple assignment format.

Create your own config: (ini-style)

 dt="01/03/16-01/09/16" cust_id="12345" 

In python:

 config_vars = {} with open('the/file/path', 'r') as f: for line in f: if '=' in line: k,v = line.split('=', 1) config_vars[k] = v week_date = config_vars['dt'] cust_id = config_vars['cust_id'] 

In bash:

 source "the/file/path" 

And you no longer need to do crazy parsing. Alternatively, you can simply use json for the configuration file, and then use the json module in python and jq in the shell for parsing.

Answer 3

I would do something like this. You can change it a bit for minor changes to include / exclude quotes, as I really haven't tested it for your scenario:

 #!/bin/sh exec <$python_filename while read line do match=`echo $line|grep "week_date ="` if [ $? -eq 0 ]; then dt=`echo $line|cut -d '"' -f 2` fi match=`echo $line|grep "cust_id ="` if [ $? -eq 0 ]; then cust_id=`echo $line|cut -d '"' -f 2` fi done