How to return an array in bash without using global variables?

Question

I have a function that creates an array, and I want to return the array to the caller:

create_array() { local my_list=("a", "b", "c") echo "${my_list[@]}" } my_algorithm() { local result=$(create_array) } 

With this, I get only the extended string. How can I "return" my_list without using anything global?

Answer 1

What happened to the globals?

Returned arrays are really impractical. There are many pitfalls.

However, here is one method that works if it is OK that the variable has the same name:

 $ f () { local a; a=(abc 'def ghi' jkl); declare -pa; } $ g () { local a; eval $(f); declare -pa; } $ f; declare -pa; echo; g; declare -pa declare -aa='([0]="abc" [1]="def ghi" [2]="jkl")' -bash: declare: a: not found declare -aa='([0]="abc" [1]="def ghi" [2]="jkl")' -bash: declare: a: not found 

The declare -p commands (except for one in f() ) are used to display the state of the array for demonstration purposes. In f() it is used as a mechanism to return an array.

If you need an array with a different name, you can do something like this:

 $ g () { local br; r=$(f); r="declare -ab=${r#*=}"; eval "$r"; declare -pa; declare -pb; } $ f; declare -pa; echo; g; declare -pa declare -aa='([0]="abc" [1]="def ghi" [2]="jkl")' -bash: declare: a: not found -bash: declare: a: not found declare -ab='([0]="abc" [1]="def ghi" [2]="jkl")' -bash: declare: a: not found 

Answer 2

In Bash version 4.3 and above, you can use nameref so that the caller can pass the name of the array, and the caller can use nameref to indirectly populate the named array.

 #!/usr/bin/env bash create_array() { local -n arr=$1 # use nameref for indirection arr=(one "two three" four) } use_array() { local my_array create_array my_array # call function to populate the array echo "inside use_array" declare -p my_array # test the array } use_array # call the main function 

Outputs:

 inside use_array declare -a my_array=([0]="one" [1]="two three" [2]="four") 

You can also force the function to update an existing array:

 update_array() { local -n arr=$1 # use nameref for indirection arr+=("two three" four) # update the array } use_array() { local my_array=(one) update_array my_array # call function to update the array } 

This is a more elegant and efficient approach since we do not need to substitute the $() commands to get the standard output of the called function. It also helps if the function returns more than one output - we can just use as many names as the number of outputs.

---

Here's what Bash Manual says about nameref:

You can assign a variable the nameref attribute using the -n option for declaration commands or local built-in commands (see the Bash Built-in Commands section) to create a link to a name or a link to another variable. This allows you to indirectly manipulate variables. Whenever a nameref is referenced, assigned, canceled, or modified by its attributes (other than using or changing the nameref attribute itself), the operation is actually performed on the variable indicated by the value of the nameref variables. Nameref is usually used in shell functions to refer to a variable whose name is passed as an argument to the function. For example, if the variable name is passed to the shell function as the first argument,

Declaring the function -n ref = $ 1 inside the function creates a variable nameref ref whose value is the name of the variable passed as the first argument. References and assignments for ref and changes to its attributes are considered as references, assignments and modification of attributes for a variable whose name was passed as $ 1.

Answer 3

Bash cannot pass data structures as return values. The return value must be the numerical output status between 0-255. However, you can use a command or process substitution to pass commands to an eval statement if you are so inclined.

It's rarely worth it, IMHO. If you must pass data structures to Bash, use a global variable - what they are for. If you do not want to do this for some reason, think about positional parameters.

Your example can be easily rewritten to use positional parameters instead of global variables:

 use_array () { for idx in "$@"; do echo "$idx" done } create_array () { local array=("a" "b" "c") use_array "${array[@]}" } 

All this creates a certain amount of unnecessary complexity. Bash functions usually work best when you treat them more like procedures with side effects and call them sequentially.

 # Gather values and store them in FOO. get_values_for_array () { :; } # Do something with the values in FOO. process_global_array_variable () { :; } # Call your functions. get_values_for_array process_global_array_variable 

If everything that bothers you is polluting your global namespace, you can also use unset builtin to remove the global variable after you are done with it. Using the original example, let my_list be global (by removing the local keyword) and add unset my_list to the end of my_algorithm to clear after itself.

Answer 4

Use the technique developed by Matt McClure: http://notes-matthewlmcclure.blogspot.com/2009/12/return-array-from-bash-function-v-2.html

By avoiding global variables, you can use this function in a pipe. Here is an example:

 #!/bin/bash makeJunk() { echo 'this is junk' echo '#more junk and "b@d" characters!' echo '!#$^%^&(*)_^&% ^$#@:"<>?/.,\\"'"'" } processJunk() { local -a arr=() # read each input and add it to arr while read -r line do arr[${#arr[@]}]='"'"$line"'" is junk'; done; # output the array as a string in the "declare" representation declare -p arr | sed -e 's/^declare -a [^=]*=//' } # processJunk returns the array in a flattened string ready for "declare" # Note that because of the pipe processJunk cannot return anything using # a global variable returned_string=`makeJunk | processJunk` # convert the returned string to an array named returned_array # declare correctly manages spaces and bad characters eval "declare -a returned_array=${returned_string}" for junk in "${returned_array[@]}" do echo "$junk" done 

Exit:

 "this is junk" is junk "#more junk and "b@d" characters!" is junk "!#$^%^&(*)_^&% ^$#@:"<>?/.,\\"'" is junk 

Answer 5

You are not so far from your original decision. You had several problems, you used a comma as a separator, and you were unable to grab the returned items into a list, try the following:

 my_algorithm() { local result=( $(create_array) ) } create_array() { local my_list=("a" "b" "c") echo "${my_list[@]}" } 

Given comments about embedded spaces, a few settings using IFS may solve the following:

 my_algorithm() { oldIFS="$IFS" IFS=',' local result=( $(create_array) ) IFS="$oldIFS" echo "Should be 'c d': ${result[1]}" } create_array() { IFS=',' local my_list=("ab" "cd" "ef") echo "${my_list[*]}" } 

Answer 6

Pure bash, a minimal and reliable solution based on the built-in -p declaration, no crazy global variables

This approach includes the following three steps:

• Convert the array with 'declare -p' and save the output in a variable.
  myVar="$( declare -p myArray )"
  The output of the declare -p statement can be used to recreate the array. For example, the output of declare -p myVar might look like this:
  declare -a myVar='([0]="1st field" [1]="2nd field" [2]="3rd field")'
• Use the built-in echo to pass a variable to a function or pass it back.
  • To keep whitspaces in the fields of the array when repeating this variable, IFS is temporarily set to a control character (for example, a vertical tab).
  • Only the right side of the declare statement should be specified in the variable - this can be achieved by expanding the form parameter $ {parameter # word}. Regarding the above example: ${myVar#*=}
• Finally, recreate the array to which it is passed using eval and the declare -a built-in functions.

Example 1 - return an array from a function

 #!/bin/bash # Example 1 - return an array from a function function my-fun () { # set up a new array with 3 fields - note the whitespaces in the # 2nd (2 spaces) and 3rd (2 tabs) field local myFunArray=( "1st field" "2nd field" "3rd field" ) # show its contents on stderr (must not be output to stdout!) echo "now in $FUNCNAME () - showing contents of myFunArray" >&2 echo "by the help of the 'declare -p' builtin:" >&2 declare -p myFunArray >&2 # return the array local myVar="$( declare -p myFunArray )" local IFS=$'\v'; echo "${myVar#*=}" # if the function would continue at this point, then IFS should be # restored to its default value: <space><tab><newline> IFS=' '$'\t'$'\n'; } # main # call the function and recreate the array that was originally # set up in the function eval declare -a myMainArray="$( my-fun )" # show the array contents echo "" echo "now in main part of the script - showing contents of myMainArray" echo "by the help of the 'declare -p' builtin:" declare -p myMainArray # end-of-file 

The result of example 1:

 now in my-fun () - showing contents of myFunArray by the help of the 'declare -p' builtin: declare -a myFunArray='([0]="1st field" [1]="2nd field" [2]="3rd field")' now in main part of the script - showing contents of myMainArray by the help of the 'declare -p' builtin: declare -a myMainArray='([0]="1st field" [1]="2nd field" [2]="3rd field")' 

---

Example 2 - pass an array to a function

 #!/bin/bash # Example 2 - pass an array to a function function my-fun () { # recreate the array that was originally set up in the main part of # the script eval declare -a myFunArray="$( echo "$1" )" # note that myFunArray is local - from the bash(1) man page: when used # in a function, declare makes each name local, as with the local # command, unless the '-g' option is used. # IFS has been changed in the main part of this script - now that we # have recreated the array it better to restore it to the its (local) # default value: <space><tab><newline> local IFS=' '$'\t'$'\n'; # show contents of the array echo "" echo "now in $FUNCNAME () - showing contents of myFunArray" echo "by the help of the 'declare -p' builtin:" declare -p myFunArray } # main # set up a new array with 3 fields - note the whitespaces in the # 2nd (2 spaces) and 3rd (2 tabs) field myMainArray=( "1st field" "2nd field" "3rd field" ) # show the array contents echo "now in the main part of the script - showing contents of myMainArray" echo "by the help of the 'declare -p' builtin:" declare -p myMainArray # call the function and pass the array to it myVar="$( declare -p myMainArray )" IFS=$'\v'; my-fun $( echo "${myVar#*=}" ) # if the script would continue at this point, then IFS should be restored # to its default value: <space><tab><newline> IFS=' '$'\t'$'\n'; # end-of-file 

Exit from example 2:

 now in the main part of the script - showing contents of myMainArray by the help of the 'declare -p' builtin: declare -a myMainArray='([0]="1st field" [1]="2nd field" [2]="3rd field")' now in my-fun () - showing contents of myFunArray by the help of the 'declare -p' builtin: declare -a myFunArray='([0]="1st field" [1]="2nd field" [2]="3rd field")' 

Answer 7

Useful example: return an array from a function

 function Query() { local _tmp='echo -n "$*" | mysql 2>> zz.err'; echo -e "$_tmp"; } function StrToArray() { IFS=$'\t'; set $1; for item; do echo $item; done; IFS=$oIFS; } sql="SELECT codi, bloc, requisit FROM requisits ORDER BY codi"; qry=$(Query $sql0); IFS=$'\n'; for row in $qry; do r=( $(StrToArray $row) ); echo ${r[0]} - ${r[1]} - ${r[2]}; done 

Answer 8

[ Note: the following was rejected as editing this answer for reasons that do not make sense to me (since editing was not intended for the author of the message!), Therefore I accept the proposal to make it a separate answer.

A simpler implementation of Steve Zobell’s adaptation to Matt McClure’s method uses the built-in (since version = 4 ) readarray proposed by RastaMatt to create an array view that can be converted to an array at runtime. (Note that readarray and mapfile call readarray the same code.) It still avoids global variables (allows the use of a function in the pipeline) and still processes nasty characters.

For some more fully developed (e.g., more modular) but still quite fun examples, see bash_pass_arrays_between_functions . Below are some easy-to-follow examples given here to avoid moderators discussing external links.

Cut the following block and paste it into the bash terminal to create /tmp/source.sh and /tmp/junk1.sh :

 FP='/tmp/source.sh' # path to file to be created for 'source'ing cat << 'EOF' > "${FP}" # suppress interpretation of variables in heredoc function make_junk { echo 'this is junk' echo '#more junk and "b@d" characters!' echo '!#$^%^&(*)_^&% ^$#@:"<>?/.,\\"'"'" } ### Use 'readarray' (aka 'mapfile', bash built-in) to read lines into an array. ### Handles blank lines, whitespace and even nastier characters. function lines_to_array_representation { local -a arr=() readarray -t arr # output array as string using 'declare representation (minus header) declare -p arr | sed -e 's/^declare -a [^=]*=//' } EOF FP1='/tmp/junk1.sh' # path to script to run cat << 'EOF' > "${FP1}" # suppress interpretation of variables in heredoc #!/usr/bin/env bash source '/tmp/source.sh' # to reuse its functions returned_string="$(make_junk | lines_to_array_representation)" eval "declare -a returned_array=${returned_string}" for elem in "${returned_array[@]}" ; do echo "${elem}" done EOF chmod u+x "${FP1}" # newline here ... just hit Enter ... 

Run /tmp/junk1.sh : the output should be

 this is junk #more junk and "b@d" characters! !#$^%^&(*)_^&% ^$#@:"<>?/.,\\"' 

Note that lines_to_array_representation also handles empty lines. Try pasting the following block into your bash terminal:

 FP2='/tmp/junk2.sh' # path to script to run cat << 'EOF' > "${FP2}" # suppress interpretation of variables in heredoc #!/usr/bin/env bash source '/tmp/source.sh' # to reuse its functions echo ''bash --version' the normal way:' echo '--------------------------------' bash --version echo # newline echo ''bash --version' via 'lines_to_array_representation':' echo '-----------------------------------------------------' bash_version="$(bash --version | lines_to_array_representation)" eval "declare -a returned_array=${bash_version}" for elem in "${returned_array[@]}" ; do echo "${elem}" done echo # newline echo 'But are they *really* the same? Ask 'diff':' echo '-------------------------------------------' echo 'You already know how to capture normal output (from 'bash --version'):' declare -r PATH_TO_NORMAL_OUTPUT="$(mktemp)" bash --version > "${PATH_TO_NORMAL_OUTPUT}" echo "normal output captured to file @ ${PATH_TO_NORMAL_OUTPUT}" ls -al "${PATH_TO_NORMAL_OUTPUT}" echo # newline echo 'Capturing L2AR takes a bit more work, but is not onerous.' echo "Look @ contents of the file you're about to run to see how it done." declare -r RAW_L2AR_OUTPUT="$(bash --version | lines_to_array_representation)" declare -r PATH_TO_COOKED_L2AR_OUTPUT="$(mktemp)" eval "declare -a returned_array=${RAW_L2AR_OUTPUT}" for elem in "${returned_array[@]}" ; do echo "${elem}" >> "${PATH_TO_COOKED_L2AR_OUTPUT}" done echo "output from lines_to_array_representation captured to file @ ${PATH_TO_COOKED_L2AR_OUTPUT}" ls -al "${PATH_TO_COOKED_L2AR_OUTPUT}" echo # newline echo 'So are they really the same? Per' echo "\'diff -uwB "${PATH_TO_NORMAL_OUTPUT}" "${PATH_TO_COOKED_L2AR_OUTPUT}" | wc -l\'" diff -uwB "${PATH_TO_NORMAL_OUTPUT}" "${PATH_TO_COOKED_L2AR_OUTPUT}" | wc -l echo '... they are the same!' EOF chmod u+x "${FP2}" # newline here ... just hit Enter ... 

Run /tmp/junk2.sh @ /tmp/junk2.sh . Your output should be similar to mine:

 'bash --version' the normal way: -------------------------------- GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. 'bash --version' via 'lines_to_array_representation': ----------------------------------------------------- GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu) Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. But are they *really* the same? Ask 'diff': ------------------------------------------- You already know how to capture normal output (from 'bash --version'): normal output captured to file @ /tmp/tmp.Ni1bgyPPEw -rw------- 1 me me 308 Jun 18 16:27 /tmp/tmp.Ni1bgyPPEw Capturing L2AR takes a bit more work, but is not onerous. Look @ contents of the file you're about to run to see how it done. output from lines_to_array_representation captured to file @ /tmp/tmp.1D6O2vckGz -rw------- 1 me me 308 Jun 18 16:27 /tmp/tmp.1D6O2vckGz So are they really the same? Per 'diff -uwB /tmp/tmp.Ni1bgyPPEw /tmp/tmp.1D6O2vckGz | wc -l' 0 ... they are the same! 

Answer 9

I tried various implementations, and none of the surviving arrays that had elements with spaces ... because they all had to use echo .

 # These implementations only work if no array items contain spaces. use_array() { eval echo '(' \"\${${1}\[\@\]}\" ')'; } use_array() { local _array="${1}[@]"; echo '(' "${!_array}" ')'; } 

Decision

Then I came across Denis Williamson's answer . I have included his method in the following functions so that they can: a) accept an arbitrary array and b) use to transfer, duplicate and add arrays.

 # Print array definition to use with assignments, for loops, etc. # varname: the name of an array variable. use_array() { local r=$( declare -p $1 ) r=${r#declare\ -a\ *=} # Strip keys so printed definition will be a simple list (like when using # "${array[@]}"). One side effect of having keys in the definition is # that when appending arrays (ie `a1+=$( use_array a2 )`), values at # matching indices merge instead of pushing all items onto array. echo ${r//\[[0-9]\]=} } # Same as use_array() but preserves keys. use_array_assoc() { local r=$( declare -p $1 ) echo ${r#declare\ -a\ *=} } 

Other functions can then return the array using subtle output or indirect arguments.

 # catchable output return_array_by_printing() { local returnme=( "one" "two" "two and a half" ) use_array returnme } eval test1=$( return_array_by_printing ) # indirect argument return_array_to_referenced_variable() { local returnme=( "one" "two" "two and a half" ) eval $1=$( use_array returnme ) } return_array_to_referenced_variable test2 # Now both test1 and test2 are arrays with three elements 

Answer 10

I needed similar functionality recently, so the following is a combination of the suggestions made by RashaMatt and Steve Zobell .

• echo of each element of the array / list as a separate line from the function
• use mapfile to read all array / list elements whose echo is performed by the function.

As far as I can see, strings are kept intact and spaces remain.

 #!bin/bash function create-array() { local somearray=("aaa" "bbb ccc" "d" "efgh") for elem in "${somearray[@]}" do echo "${elem}" done } mapfile -t resa <<< "$(create-array)" # quick output check declare -p resa 

A few more options ...

 #!/bin/bash function create-array-from-ls() { local somearray=("$(ls -1)") for elem in "${somearray[@]}" do echo "${elem}" done } function create-array-from-args() { local somearray=("$@") for elem in "${somearray[@]}" do echo "${elem}" done } mapfile -t resb <<< "$(create-array-from-ls)" mapfile -t resc <<< "$(create-array-from-args 'xxx' 'yy zz' 'tsu' )" sentenceA="create array from this sentence" sentenceB="keep this sentence" mapfile -t resd <<< "$(create-array-from-args ${sentenceA} )" mapfile -t rese <<< "$(create-array-from-args "$sentenceB" )" mapfile -t resf <<< "$(create-array-from-args "$sentenceB" "and" "this words" )" # quick output check declare -p resb declare -p resc declare -p resd declare -p rese declare -p resf 

Answer 11

I recently discovered a quirk in BASH that a function has direct access to variables declared in functions above in the call stack. I just started thinking about how to use this function (it promises both advantages and dangers), but one obvious application is the solution to the spirit of this problem.

I would also prefer to get the return value, rather than using a global variable when delegating the creation of an array. There are several reasons for my preferences, among which one should avoid a possible violation of the existing value and avoid leaving a value that may be invalid upon subsequent access. Although there are workarounds for these problems, the simplest is that the variable goes out of scope when the code exits with it.

My solution ensures that the array is available as needed and discarded when the function returns, and leaves the global variable of the same name unchanged.

 #!/bin/bash myarr=(global array elements) get_an_array() { myarr=( $( date +"%Y %m %d" ) ) } request_array() { declare -a myarr get_an_array "myarr" echo "New contents of local variable myarr:" printf "%s\n" "${myarr[@]}" } echo "Original contents of global variable myarr:" printf "%s\n" "${myarr[@]}" echo request_array echo echo "Confirm the global myarr was not touched:" printf "%s\n" "${myarr[@]}" 

Here is the output of this code:

When the request_array function calls get_an_array, get_an_array can directly set the myarr variable, which is local to request_array. Since myarr is created with declare , it is local request_array and thus goes out of scope when request_array is returned.

Although this solution does not literally return a value, I suggest that it be taken as a whole, it satisfies the promise of a true return value of the function.

Answer 12

Here is a solution without references to external arrays and IFS manipulations:

 # add one level of single quotes to args, eval to remove squote () { local a=("$@") a=("${a[@]//\'/\'\\\'\'}") # "'" => "'\''" a=("${a[@]/#/\'}") # add "'" prefix to each word a=("${a[@]/%/\'}") # add "'" suffix to each word echo "${a[@]}" } create_array () { local my_list=(a "b 'c'" "\\\"d ") squote "${my_list[@]}" } my_algorithm () { eval "local result=($(create_array))" # result=([0]="a" [1]="b 'c'" [2]=$'\\"d\n') } 

Answer 13

You can also do this by simply passing the array variable to the function and assigning the array values ​​to this var, then use this var outside the function. For example.

 create_array() { local __resultArgArray=$1 local my_list=("a" "b" "c") eval $__resultArgArray="("${my_list[@]}")" } my_algorithm() { create_array result echo "Total elements in the array: ${#result[@]}" for i in "${result[@]}" do echo $i done } my_algorithm 

Answer 14

If the source data is formatted with each element of the list on a separate line, then the built-in mapfile is a simple and elegant way to read the list into an array:

 $ list=$(ls -1 /usr/local) # one item per line $ mapfile -t arrayVar <<<"$list" # -t trims trailing newlines $ declare -p arrayVar | sed 's#\[#\n[#g' declare -a arrayVar='( [0]="bin" [1]="etc" [2]="games" [3]="include" [4]="lib" [5]="man" [6]="sbin" [7]="share" [8]="src")' 

Note that, as with read inline, you would normally not use the mapfile in the pipeline (or in the subshell), because the assigned array variable would not be available for subsequent statements (* if bash is disabled and shopt -s lastpipe ).

 $ help mapfile mapfile: mapfile [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum] [array] Read lines from the standard input into an indexed array variable. Read lines from the standard input into the indexed array variable ARRAY, or from file descriptor FD if the -u option is supplied. The variable MAPFILE is the default ARRAY. Options: -n count Copy at most COUNT lines. If COUNT is 0, all lines are copied. -O origin Begin assigning to ARRAY at index ORIGIN. The default index is 0. -s count Discard the first COUNT lines read. -t Remove a trailing newline from each line read. -u fd Read lines from file descriptor FD instead of the standard input. -C callback Evaluate CALLBACK each time QUANTUM lines are read. -c quantum Specify the number of lines read between each call to CALLBACK. Arguments: ARRAY Array variable name to use for file data. If -C is supplied without -c, the default quantum is 5000. When CALLBACK is evaluated, it is supplied the index of the next array element to be assigned and the line to be assigned to that element as additional arguments. If not supplied with an explicit origin, mapfile will clear ARRAY before assigning to it. Exit Status: Returns success unless an invalid option is given or ARRAY is readonly or not an indexed array. 

Answer 15

 my_algorithm() { create_array list for element in "${list[@]}" do echo "${element}" done } create_array() { local my_list=("1st one" "2nd two" "3rd three") eval "${1}=()" for element in "${my_list[@]}" do eval "${1}+=(\"${element}\")" done } my_algorithm 

Output signal

 1st one 2nd two 3rd three