Ansible - with_dict: dictionary - How to use variables defined in each dictionary, which depends on others

Question

Environment: Carrier 1.9.2, CentOS 6.5

I created a role to download JAVA artifact files (.tar.gz) for 3 different versions of JAVA from Artifactory. I am trying to use the Ansible with_dict function (instead of using with_items).

The following files have been created:

$ cat role / java / defaults / main.yml

--- java_versions: java7_60: version: 1.7.60 group_path: com/oracle/jdk classifier: linux-x64 ext: tar.gz dist_file: "jdk-{{ version }}-{{ classifier }}-{{ ext }}" # dist_file: "jdk-{{item.value.version }}-{{ item.value.classifier }}-{{ item.value.ext }}" dist_url: "{{ artifactory_url }}/{{ group_path }}/{{ version }}/{{ dist_file }}" # dist_url: "{{ artifactory_url }}/{{ item.value.group_path }}/{{ item.value.version }}/{{ dist_file }}" java7_67: version: 1.7.67 group_path: com/oracle/jdk classifier: linux-x64 ext: tar.gz dist_file: "jdk-{{item.value.version }}-{{ item.value.classifier }}-{{ item.value.ext }}" dist_url: "{{ artifactory_url }}/{{ item.value.group_path }}/{{ item.value.version }}/{{ dist_file }}" java8_45: version: 1.8.45 group_path: com/oracle/jdk classifier: linux-x64 ext: tar.gz dist_file: "jdk-{{item.value.version }}-{{ item.value.classifier }}-{{ item.value.ext }}" dist_url: "{{ artifactory_url }}/{{ item.value.group_path }}/{{ item.value.version }}/{{ dist_file }}" 

How to set or use dist_file or dist_url variables that depend on other variables defined in the same key (for example, in KEY java7_60)?

Right now, when I try to use either the current dist_file or dist_url variables OR , the commented lines the way they are set (i.e. using item.value.), It does not set the value of these 2 as desired, i.e. depending on other versions of the variables , group_path, classifier, ext, and artifactory_url (which are defined in another common defaults / main.yml file)).

I saw that to use with_dict: inside playbook / task I have to use {{item.value.variable_name}}, but how can I define a variable that depends on others within the same KEY section of the dictionary.

The error message I get when using the above dictionary in the following task:

$ cat role / java / tasks / main.yml :

 - name: Download Java/JDK Versions command: wget -q "{{ item.value.dist_url }}" chdir="{{ common_download_dir }}" creates="{{ common_download_dir }}/{{ item.value.dist_file }}" with_dict: "{{ java_versions }}" become_user: "{{ build_user }}" 

Error message using dist_file / dist_url (with current setting in roles / java / defaults / main.yml):

 TASK: [java | Download Java/JDK Versions] ************************************* failed: [server01.poc.jenkins] => (item={'key': 'java7_60', 'value': {'dist_file': u'jdk-{# version #}-{# classifier #}-{# ext #}', 'ext': 'tar.gz', 'version': '1.7.60', 'dist_url': u'{# artifactory_ur #}/{# group_path #}/{# version #}/{# dist_file #}', 'group_path': 'com/oracle/jdk', 'classifier': 'linux-x64'}}) => {"changed": true, "cmd": ["wget", "-q", "{# artifactory_url #}/{# group_path #}/{# version }/{# dist_file #}"], "delta": "0:00:00.006081", "end": "2015-11-23 12:50:18.383728", "item": {"key": "java7_60", "value": {"classifier": "linux-x64", "dist_file": "jdk-{# version #}-{# classifier #}-{# ext #}, "dist_url": "{# artifactory_url #}/{# group_path #}/{# version #}/{# dist_file #}", "ext": "tar.gz", "group_path": "com/oracle/jdk", "version": "1.7.60"}}, "rc": 4, "start": "2015-11-23 12:50:18.377647", "wrnings": ["Consider using get_url module rather than running wget"]} 

Error message using dist_file / dist_url (with the lines currently commented out in the roles /java/defaults/main.yml):

 TASK: [java | Download Java/JDK Versions] ************************************* failed: [server01.poc.jenkins] => (item={'key': 'java7_60', 'value': {'dist_file': u'jdk-{#item.value.version #}-{# item.value.classifier #}-{# item.value.ext #}', 'ext': 'tar.gz', 'version': '1.7.60' , 'dist_url': u'{# artifactory_url #}/{# item.value.group_path #}/{# item.value.version #}/{# dist_file #}', 'group_path': 'com/oracle/jdk', 'classifier': 'linux-x64'}}) => {"changed": true, "cmd": ["wget", "-q", "{# artifactory_url #}/{# item.value.group_path #}/{# item.value.version #}/{# dist_file #}"], "delta": "0:00:00.005900", "end": "2015-11-23 12:36:24.131327", "item": {"key": "java7_60", "value": {"cla ssifier": "linux-x64", "dist_file": "jdk-{#item.value.version #}-{# item.value.classifier #}-{# item.value.ext #}", "dist_url": "{# artifactory_url #}/{# item.value.group_path #}/{# item.value.version #}/{# dist_file #}", "ext": "tar.gz", "group_path": "com/oracle/jdk", "version": "1.7.60"}}, "rc": 4, "start": "2015-11-23 12:36:24.125427", "warnings": ["Consider using get_url module rather than running wget"]} 

Answer 1

I believe that the best way to do what you want (since I don't believe loop variables can be referenced by themselves) will have your task:

 - name: Download Java/JDK Versions command: wget -q "{{ artifactory_url }}/{{ item.value.group_path }}/{{ item.value.version }}/jdk-{{item.value.version }}-{{ item.value.classifier }}-{{ item.value.ext }}" chdir="{{ common_download_dir }}" creates="{{ common_download_dir }}/jdk-{{item.value.version }}-{{ item.value.classifier }}-{{ item.value.ext }}" with_dict: "{{ java_versions }}" become_user: "{{ build_user }}" 

essentially manually inserting variables into your task.

This is not very good if you need to use variables for several tasks, but in indispensable 1.x I do not think there is a way to make it beautiful. Ansible 2.0 has blocks with which you could combine several tasks together with a dict, and you can define variables for all tasks in this block.

Answer 2

I was interested, so I did a little work, and at the same time I met this similar answer . This is only part of the solution in your case.

It seems that Ansible will not allow you to refer to a variable from its own definition, which I think makes sense because it is not fully defined. Thus, something like this will not work, and will actually cause a somewhat confusing error when referring to a variable:

 --- myvar: param1: foo param2: "{{ myvar['foo'] }} bar" 

You can also see from your own example that Ansible will not allow you to use item constructors in variables to refer to other complex variables. This type makes sense to me, since it seems that Ansible resolves jinja2 constructs in variables at the time the variable is defined, and not at run time when it refers to the variable.

So, although this is not quite what you were looking for, I think that if you decompose your variable into two parts, you can make it work by doing something in this direction:

 --- artifactory_url: "http://path.to.jarfile" java_versions: java7_60: version: 1.7.60 group_path: com/oracle/jdk classifier: linux-x64 ext: tar.gz java_downloads: java7_60: dist_url: "{{ artifactory_url }}/{{ java_versions['java7_60']['group_path'] }}/{{ java_versions['java7_60']['version'] }}/jdk-{{ java_versions['java7_60']['version'] }}-{{ java_versions['java7_60']['classifier'] }}.{{ java_versions['java7_60']['ext'] }}" 

When you debug java_downloads this way, you get the full url you are looking for:

 TASK: [debug var=item] ******************************************************** ok: [localhost] => (item={'key': 'java7_60', 'value': {'dist_url': u'http://path.to.jarfile/com/oracle/jdk/1.7.60/jdk-1.7.60-linux-x64.tar.gz'}}) => { "item": { "key": "java7_60", "value": { "dist_url": "http://path.to.jarfile/com/oracle/jdk/1.7.60/jdk-1.7.60-linux-x64.tar.gz" } }, "var": { "item": { "key": "java7_60", "value": { "dist_url": "http://path.to.jarfile/com/oracle/jdk/1.7.60/jdk-1.7.60-linux-x64.tar.gz" } } } } 

Answer 3

Thanks to Bruce P. and Seper N., I did the following after making decisions / hints from their answer. Now my task is to work with several tools (jdk, mvn, gradle, maven, etc.) with several versions with minimal changes to the files, and without the need (definition of the 2nd dictionary).

What I've done:

In the top level / some common role default / main.yml I will have the following:

$ cat role / some_common_global_role / defaults / main.yml

 --- a_var: giga other_var: fifa dist_file: "{{ item.value.tool }}-{{item.value.version }}-{{ item.value.classifier }}.{{ item.value.ext }}" dist_url: "{{ artifactory_url }}/{{ item.value.group_path }}/{{ item.value.version }}/{{ dist_file }}" 

In $ cat role / java / defaults / main.yml

 --- tool: jdk java_versions: java7_60: version: 1.7.60 group_path: com/oracle/jdk classifier: linux-x64 ext: tar.gz java7_67: version: 1.7.67 group_path: com/oracle/jdk classifier: linux-x64 ext: tar.gz java8_45: version: 1.8.45 group_path: com/oracle/jdk classifier: linux-x64 ext: tar.gz 

In separate roles of the /default/main.yml tool, I will set the same dictionary and tool variable and it will load several tools / versions. In this case, your default general level role /main.yml will look like this:

 --- dist_file: "{{ tool }}-{{item.value.version }}-{{ item.value.classifier }}.{{ item.value.ext }}" dist_url: "{{ artifactory_url }}/{{ item.value.group_path }}/{{ item.value.version }}/{{ dist_file }}" 

If I use the above approach, I can define the "tool" variable at given levels //defaults/main.yml and it will be available for dist_file / dist_url.

Also, instead of using the command: wget -q "...", I can use get_url (module) in Ansible.

 - name: Download Java/JDK Versions # command: wget -q "{{ dist_url }}" # chdir="{{ common_download_dir }}" # creates="{{ common_download_dir }}/{{ dist_file }}" get_url: url="{{ dist_url }}" dest="{{ common_download_dir }}" become_user: "{{ build_user }}" with_dict: "{{ java_versions }}" 

and make sure that on the REMOTE machine you can execute the ping command on the Artifactory server (check ping .... or the /etc/resolv.conf file for search entries / names), and if everything looks good, then the solutions posted here WORKS!

I personally liked the solution for passing the tool variable (at the role / task level), which denies the use of the definition tool: at the dictionary level and does not require the definition of individual variables of the _dist_file / dist_url level.