Writing the contents of a makefile (> 131000 chars) to a file?

Question

How can I write the contents of a makefile to a file without invoking a shell command? The problem is that the contents of the variable are possible longer than the shell allows the command (i.e. longer than the characters MAX_ARG_STRLEN (131072)).

In particular, in the makefile, I have a variable containing a long list of file names to process (including their absolute templates for assembly outside the source). Now I need to write these file names to a (temporary) file, which I can then transfer to another command.

So far, we had a rule like ( $COLLATED_FILES is a variable containing paths):

 $(outdir)/collated-files.tely: $(COLLATED_FILES) $(LYS_TO_TELY) --name=$(outdir)/collated-files.tely --title="$(TITLE)" \ --author="$(AUTHOR)" $^ 

This breaks if COLLATED_FILES longer than 130,000 characters, we get an error message:

 make[2]: execvp: /bin/sh: Argument list too long 

As a solution, we are now trying to write the contents of the variable to a file and use this file in the $(LYS_TO_TELY) . Unfortunately, I have not yet found a way to do this without invoking the shell. My attempts include:

 $(outdir)/collated-files.list: $(COLLATED_FILES) echo "" > $@ $(foreach f,$^,echo $f >> $@ ;) 

But it also calls all echo once in the shell, so the shell command is just as long.

Is there a way to write the contents of $(COLLATED_FILES) to a file on disk without passing them on the command line to a shell command?

I also searched if I could pass the contents of the variable to the shell, but I could not find anything in that direction ...

Answer 1

Assuming you are using GNU Make, there is a file function!

https://www.gnu.org/software/make/manual/html_node/File-Function.html

 $(file op filename,text) 

where op is either > or >> .

This requires GNU Make 4.0 +

Answer 2

You can port any makefile code you use to create the COLLATED_FILES value for the trivial helper makefile, and then call make recursively from your makefile source file and use the trivial shell redirection to capture the stdout of the recursive make invocation - mainly using make as rudimentary word processing tool in this context. For example, create a make file called get_collated_files.mk with this content:

 COLLATED_FILES=abc COLLATED_FILES+=def COLLATED_FILES+=ghi # ... etc ... # Use $(info) to print the list to stdout. If you want each filename on a # separate line, use this instead: # # $(foreach name,$(COLLATED_FILES),$(info $(name))) $(info $(COLLATED_FILES)) all: ;@#shell no-op to quell make complaints 

Then in your original make file:

 collated-files.list: $(MAKE) -f get_collated_files.mk > $@ $(outdir)/collated-files.tely: collated-files.list $(LYS_TO_TELY) --name=$(outdir)/collated-files.tely --title="$(TITLE)" \ --author="$(AUTHOR)" --filelist=collated-files.list 

This will be much more efficient than using hundreds or thousands of individual echo windows to add a file one path at a time.

EDIT : The last option, if you really want to have each file name on a separate line, and you have a lot of control over how COLLATED_FILES defined:

 define NL endef COLLATED_FILES=abc COLLATED_FILES+=$(NL)def COLLATED_FILES+=$(NL)ghi $(info $(COLLATED_FILES)) all: ;@#no-op 

This approach allows you to reuse only one call to $(info) if this is important to you for any reason.

Answer 3

Here's the patch for gnu make, which allows you to directly write the variable to a file. It creates a new "writefile" function similar to the existing "info" function, except that it takes a file name argument and writes to the file:

https://savannah.gnu.org/bugs/?35384

Answer 4

It seems to me that you should rethink your assembly design - of course, a better way than letting the variable get this big one. But here is a way to do it:

 # Make sure this doesn't collide with any of your other targets. NAMES_TO_WRITE = $(addprefix write_,$(COLLATED_FILES)) collated-files.list: $(NAMES_TO_WRITE) write_blank: echo "" > collated-files.list .PHONY: $(NAMES_TO_WRITE) $(NAMES_TO_WRITE) : write_% : write_blank echo $* >> collated-files.list