Scons: How to deal with dynamic goals?

Question

I am trying to automate my work of converting a PDF file to png using scons . The tool used for my conversion is convert from ImageMagick .

Here's the raw command line:

• convert input.pdf temp/temp.png
• convert temp/*.png -append output.png

The first command will generate one PNG file for each page in the PDF file, so the goal of the first command is to dynamically list the files.

Here is the SConstruct file I'm working on:

 convert = Builder(action=[ Delete("${TARGET.dir}"), Mkdir("${TARGET.dir}"), "convert $SOURCE $TARGET"]) combine = Builder(action="convert $SOURCE -append $TARGET") env = Environment(BUILDERS={"Convert": convert, "Combine": combine}) pdf = env.PDF("input.tex") pngs = env.Convert("temp/temp.png", pdf) # I don't know how to specify target in this line png = env.Combine('output.png', pngs) Default(png) 

The code pngs = env.Convert("temp/temp.png", pdf) is actually wrong, since the goal is a few files that I don’t know how long before env.Convert is executed, so the final output.png contains only the first PDF file page.

Any hint is appreciated.

UPDATE:

I just found that I can use the convert input.pdf -append output.png to avoid a two-step conversion.

However, I am curious how to handle the scenario when the interim temporary list of files is unknown in advance and requires a dynamic target list.

Answer 1

If you want to know how to make the initial (convert and combine) situation that you proposed, I would suggest creating a builder with SCons Emitter . The emitter allows you to change the list of source and target files. This works well for generated files that do not exist with a clean build.

As you mentioned, the conversion step will generate several goals, the trick is that you need to “calculate” these goals in the emitter based on the source. For example, I recently created the wsdl2java constructor and was able to do a simple parsing of wsdl in the emitter to calculate all the target Java files to be created (source is wsdl).

Here's a general idea of ​​what build scripts look like:

 def convert_emitter(source, target, env): # both and source and target will be a list of nodes # in this case, the target will be empty, and you need # to calculate all of the generated targets based on the # source pdf file. You will need to open the source file # with standard python code. All of the targets will be # removed when cleaned (scons -c) target = [] # fill in accordingly return (target, source) # Optionally, you could supply a function for the action # which would have the same signature as the emitter convert = env.Builder(emitter=convert_emitter, action=[ Delete("temp"), Mkdir("temp"), "convert $SOURCE $TARGET"]) env.Append(BUILDERS={'Convert' : convert}) combine = env.Builder(action=convert_action, emitter=combine_emitter) env.Append(BUILDERS={'Combine' : combine}) pdf = env.PDF('input.tex') # You can omit the target in this call, as it will be filled-in by the emitter pngs = env.Convert(source=pdf) png = env.Combine(target='output.png', source=pngs) 

Answer 2

If you look at this, there is no need for the individual temp / * png to be saved - if that were the case, you should not put them in the temp directory, and in any case you will need to do a bit of work if you want determine which pages will be created.

So it seems more reasonable to take it as one step, it means that you have something like this

 png = env.Convert('output.png', 'input.pdf') 

where the action function for the conversion was something like this:

 Delete('temp'), Mkdir('temp'), 'convert $SOURCE temp/$TARGET', 'for i in temp/*png; do convert $TARGET temp/$i', Delete('temp') 

Although, frankly, you could better write this whole thing as the only script to be called, to make sure you sort the page correctly.

Answer 3

Depending on what suits you as “dynamic,” I think the correct answer is: impossible .

As long as the source by which you want to "dynamically" calculate the target set is present when SCons starts up, the @Brady solution should work fine. However, if the source itself is the target of any other team, this will not work. This is a fundamental limitation of SCons, since it makes the assumption that a set of construction goals can be statically determined from a basic set of source (non-intermediate) sources. It goes through and calculates the build / goal / dependency graph in one scan, and then runs it in the next. He does not have the ability to run any known part of the construction schedule, stops to think about some intermediate objects, to dynamically calculate the rest of the construction schedule, and then continue. I frankly love this ability in the work that I do with the Skinsons, but I'm afraid that this is just a fundamental limitation.

The best thing you can do is install the assembly so that the first time it starts, it stops when you build the PDF (if the PDF target does not exist when the script is being built). After the PDF file has been created, you can restart the assembly and configure it so that the rest of the assembly steps are based on the PDF created from the last run. It more or less works decently ... except for one problem. If the PDF ends with a change (and, for example, creating some new pages), you will have to restart the assembly twice to commit the changes to the PDF file, since any number of pages (etc.) will be based on the old version of PDF.

I would really like someone to prove that I am not right here, but this is the way of things.