How to use wait in parallel foreach?

Question

So, I feel most of the night trying to figure it out.

I was lucky to get acquainted with the parallel. Yesterday it works as if I want it to be done, except from one detail.

I have the following:

Parallel.ForEach(data, (d) => { try { MyMethod(d, measurements); } catch (Exception e) { // logg } }); 

In the "MyMethod" method, I have a lot of logic that executes, and most of them are fine, but I make api calls where I get the data, and I use the async task to do this, so that I can "wait" for the code to wait until this particular part is completed, then go:

  private async void MyMethod(PimData pimData, IEnumerable<ProductMeasurements> measurements) { try { // alot of logic but most relevant part await Task.WhenAll(ExecuteMeasurmentAndChartLogic(pimData.ProductNumber, entity)); await Task.WhenAll(resourceImportManager.HandleEntityImageFiles(pimData.ProductType + pimData.ProductSize,SwepImageType.Png, ResourceFileTypes.ThreeD, entity, LinkTypeId.ProductResource)); await Task.WhenAll(resourceImportManager.HandleEntityImageFiles(pimData.ProductSketch, SwepImageType.Png, ResourceFileTypes.Sketch, entity, LinkTypeId.ProductResource)); } catch (Exception e) { // logg } } 

The problems :

1 To begin, the cycle ends before the completion of all code

2 The second problem is that I get "Task canceled" in alo api calls

3 And the third, as mentioned above, the code does not wait for each method to be fully executed.

I cannot get it to execute everything in the ExecuteMeasurmentAndChartLogic () method before proceeding to the next step.

This gives me the following problems (more problems):

In this method, I create an element and add it to db, and this element needs additional information that I get from the api call, which is executed inside ExecuteMeasurmentAndChartLogic (), but the problem is that several elements become crazy and should wait for the rest of the data what i don't want.

SIDE-NOTE: I know that I broke the element and added to db before all the data is not the best, but I integrate into PIM, and the process for this is delicate

I want several threads to execute, but at the same time I want fuill logic to execute for each element before moving on to the next method.

Clarify:

Multiple Elements Performing

Each element conveys ALL the logic necessary for the transfer, before proceeding to the next part of the code, it is normal to do this with anticipation.

In the above code, the resourceImportManager () method is run until ExecuteMeasurmentAndChartLogic () is completed. what i don't want.

Instead of Parallel.ForEach, I used:

  Task task1 = Task.Factory.StartNew(() => MyMethod(data, measurements)); Task.WaitAll(task1); 

but it didn’t help much

Pretty new to this and could not figure out where I am doing it wrong.

EDIT: Updated issues with this

EDIT: this is what ExecuteMeasurmentAndChartLogic () looks like:

  public async Task ExecuteMeasurmentAndChartLogic(string productNumber, Entity entity) { try { GrafGeneratorManager grafManager = new GrafGeneratorManager(); var graphMeasurmentList = await MeasurmentHandler.GetMeasurments(productNumber); if (graphMeasurmentList.Count == 0) return; var chart = await grafManager.GenerateChart(500, 950, SystemColors.Window, ChartColorPalette.EarthTones, "legend", graphMeasurmentList); await AddChartsAndAddToXpc(chart, entity, productNumber); } catch (Exception e) { Console.WriteLine(e); } } 

EDIT: Prerequisites for this: I am calling the api to get a lot of data. For each element in this data, I need to make an api call and get the data that I apply to the element.

After reading the comments that alos made me think. I can possibly skip all my elements and make secondary logic for them and add the URL to the task list and complete a separate task that runs one at a time.

Wll save this update

Answer 1

Do not use Parralel.ForEach at all. Make your own method to return the task instead of void, collect the whole task and wait for how:

 Task.WaitAll(data.Select(d => MyMethod(d, someParam)).ToArray());