Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix optimizer writeStep final (if optimization ends for reaching the limit number of iterations) #2387

Closed
wants to merge 14 commits into from
Closed

fix optimizer writeStep final (if optimization ends for reaching the limit number of iterations) #2387

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d2f4ec3
Closes #2386
alfoa Oct 21, 2024
254f49a
regolded GA that did not print final solution (when the optimization …
alfoa Oct 21, 2024
90536b0
Update ravenframework/Steps/MultiRun.py
alfoa Oct 21, 2024
2b68e5c
if the printing is done, don't print
alfoa Oct 23, 2024
ff47a1a
Merge branch 'alfoa/fixOptimizerWriteStepFinal' of github.com:idahola…
alfoa Oct 23, 2024
39c5a39
xml
alfoa Oct 23, 2024
e8b15fd
Update tests/framework/Optimizers/GeneticAlgorithms/discrete/unconstr…
alfoa Oct 23, 2024
5fe5932
test pinned library
alfoa Oct 25, 2024
c1bdd2f
Merge branch 'alfoa/fixOptimizerWriteStepFinal' of github.com:idahola…
alfoa Oct 25, 2024
d9c9dcc
trying with 5.1.0
alfoa Oct 28, 2024
57d648c
trying version of ray and redis that worked on windows
alfoa Oct 31, 2024
50828f1
trying version of ray and redis that worked on windows
alfoa Oct 31, 2024
e63c1e2
changed approach to fix the issue. This is much cleaner and less conf…
alfoa Nov 6, 2024
3164cbc
Apply suggestions from code review
alfoa Nov 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions dependencies.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,9 @@ Note all install methods after "main" take
<nomkl os='linux' skip_check='True'/>
<cmake skip_check='True' optional='True'/>
<dask source="pip" pip_extra="[complete]"/>
<ray source="pip" pip_extra="[default]">2.6</ray>
<ray source="pip" pip_extra="[default]">2.6.3</ray>
alfoa marked this conversation as resolved.
Show resolved Hide resolved
<!-- redis is needed by ray, but on windows, this seems to need to be explicitly stated -->
<redis source="pip" os='windows'/>
<redis source="pip" os='windows'>5.2.0</redis>
alfoa marked this conversation as resolved.
Show resolved Hide resolved
<imageio source="pip">2.22</imageio>
<line_profiler optional='True'/>
<!-- <ete3 optional='True'/> -->
Expand Down
1 change: 1 addition & 0 deletions ravenframework/Models/Code.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -529,6 +529,7 @@ def evaluateSample(self, myInput, samplerType, kwargs):
sampleDirectory = os.path.join(os.getcwd(),metaData['subDirectory'])
localenv = dict(os.environ)
localenv['PWD'] = str(sampleDirectory)
os.makedirs(os.path.dirname(os.path.join(sampleDirectory,codeLogFile)), exist_ok=True)
outFileObject = open(os.path.join(sampleDirectory,codeLogFile), 'w', bufferSize)

found = False
Expand Down
2 changes: 2 additions & 0 deletions ravenframework/OutStreams/PrintInterfaces/FilePrint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,8 @@ def run(self):
if self.options['type'] == 'csv':
filename = dictOptions['filenameroot']
rlzIndex = self.indexPrinted.get(filename,0)
if rlzIndex and rlzIndex >= len(self.sourceData[index]):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if these two lines are needed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is needed when the printing of the dataset is not finished (e.g. point set) and this is triggered in the multi run step (right before exiting the step). We check here the index and the length in case the printing is finished before reaching the ned of the step

continue
dictOptions['firstIndex'] = rlzIndex
# clusterLabel lets the user print a point set as if it were a history, with input decided by clusterLabel
if 'clusterLabel' in self.options:
Expand Down
5 changes: 5 additions & 0 deletions ravenframework/Steps/MultiRun.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,11 @@ def _localTakeAstepRun(self, inDictionary):
# END while loop that runs the step iterations (collection and submission-for-DET)
# if any collected runs failed, let the sampler treat them appropriately, and any other closing-out actions
sampler.finalizeSampler(self.failedRuns)
for myLambda, outIndex in self._outputCollectionLambda:
if isinstance(outputs[outIndex], OutStreamEntity):
myLambda([None,outputs[outIndex]])
self.raiseAMessage(f'Finalized output "{inDictionary["Output"][outIndex].name}"')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alfoa The proposed changes can resolve the issue. However, it can be confusing since these lines are almost similar to collection part in the same function. Could you provide more details why previous collection can not collect the final solution? Is it possible to make some modifications inside the optimizer to enable it? I have two concerns for the proposed approach:

  1. Two collections in the same function, which make it very confusing. Either add more explanations or find a way to avoid it.
  2. It is also very confusing in the FilePrint.py, since the new added lines to check the rlzIndex seem very unnecessary. I see the changes make the final collection possible, but it is really hard to understand why these lines are needed unless the developers fully understand the collections in the steps.

Copy link
Collaborator Author

@alfoa alfoa Nov 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lines in the FilePrint are not unnecessary. They are required when the collection in triggered on data objects that are not "collected/created" by the Optimizers.

Basically, the "SolutionExport" in the Optimizers is "updated" with the final solution after the collection is triggered (at the begin of the processing of the "last job"). So the Outstream is not invoked before exiting the Multirun loop.

This modification was the "minimal viable" solution to trigger an out stream call at the end of the multi run.

Maybe another approach could be to split the calls to the output collection:

  • Right after a Job is finished for data objects (DataObjects/Databases)
  • After the call to the finalizeActualSampling for OutStreams


alfoa marked this conversation as resolved.
Show resolved Hide resolved

def _addNewRuns(self, sampler, model, inputs, outputs, jobHandler, inDictionary, verbose=True):
"""
Expand Down
3 changes: 3 additions & 0 deletions .../framework/Optimizers/GeneticAlgorithms/discrete/unconstrained/testGAMaxwoRepConvAHDp.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
The objective variable is ans. The problem in unconstrained, it is a maximization problem, and the sampling is from discrete variables without replacement.
The convergence criterion used is the (p-Average Hausdorff Distance between populations) AHDp.
</description>
<revisions>
<revision author="alfoa" date="2024-10-21">Modified writeSteps to ``final'' to make sure the Outstream is executed and the final solution is always written. Closes #2386</revision>
</revisions>
<analytic>
This test uses myLocalSum's analytic objective function.
</analytic>
Expand Down
381 changes: 201 additions & 180 deletions ...izers/GeneticAlgorithms/gold/continuous/unconstrained/ensemble_withGA_HDSM/opt_export.csv
View file
Open in desktop

Large diffs are not rendered by default.

17 changes: 10 additions & 7 deletions ...thms/gold/continuous/unconstrained/metaModelWithCodeAndFunctionsAndGenetic/opt_export.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
trajID,sigma-A,sigma-B,decay_A,decay_B,sum,age,batchId,fitness,iteration,accepted,AHDp,conv_AHDp
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while the previous csv contains new rows the the bottom of the file as expected, this file contains some differences thorughout the rows, any possible explanation here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the rows are swapped.

0,812.997460787,464.508307042,1.255650268e-08,0.000100012556503,3.15760954156,0,1,-3.15760954156,0,first,,False
0,759.507928453,912.545952926,8.14640085566e-08,0.000100081464009,2.90476176926,0,1,-2.90476176926,0,first,,False
0,912.545952926,812.997460787,1.255650268e-08,0.000100012556503,3.15873736407,0,2,-3.15873736407,1,accepted,299.428022512,False
0,812.997460787,464.508307042,1.255650268e-08,0.000100012556503,3.15760954156,0,1,-3.15760954156,0,first,,False
0,464.508307042,759.507928453,8.14640085566e-08,0.000100081464009,2.83929690449,0,2,-2.83929690449,1,accepted,299.428022512,False
0,912.545952926,812.997460787,1.255650268e-08,0.000100012556503,3.15873736407,0,2,-3.15873736407,1,accepted,299.428022512,False
0,794.044539517,912.545952926,5.18057476338e-08,0.000100051805748,2.98393998882,1,3,-2.98393998882,2,accepted,261.653404312,False
0,794.044539517,759.507928453,7.83557135608e-08,0.000100078355714,2.9174761674,1,3,-2.9174761674,2,accepted,261.653404312,False
0,912.545952926,464.508307042,8.14640085566e-08,0.000100081464009,2.93136915288,2,4,-2.93136915288,3,accepted,379.964041465,False
0,759.507928453,759.507928453,8.14640085566e-08,0.000100081464009,2.90491165751,2,4,-2.90491165751,3,accepted,379.964041465,False
0,464.508307042,912.545952926,8.14640085566e-08,0.000100081464009,2.83915821533,3,5,-2.83915821533,4,accepted,154.98621892,False
0,794.044539517,759.507928453,7.83557135608e-08,0.000100078355714,2.9174761674,3,5,-2.9174761674,4,accepted,154.98621892,False
0,912.545952926,464.508307042,8.14640085566e-08,0.000100081464009,2.93136915288,3,6,-2.93136915288,5,accepted,484.155218746,False
0,759.507928453,464.508307042,8.14640085566e-08,0.000100081464009,2.90520071484,3,6,-2.90520071484,5,accepted,484.155218746,False
0,912.545952926,464.508307042,8.14640085566e-08,0.000100081464009,2.93136915288,4,7,-2.93136915288,6,accepted,484.155218746,False
0,912.545952926,464.508307042,8.14640085566e-08,0.000100081464009,2.93136915288,3,6,-2.93136915288,5,accepted,484.155218746,False
0,759.507928453,464.508307042,8.14640085566e-08,0.000100081464009,2.90520071484,4,7,-2.90520071484,6,accepted,484.155218746,False
0,912.545952926,464.508307042,8.14640085566e-08,0.000100081464009,2.93136915288,4,7,-2.93136915288,6,accepted,484.155218746,False
0,794.044539517,759.507928453,5.18057476338e-08,0.000100051805748,2.98410104597,5,8,-2.98410104597,7,accepted,329.536232476,False
0,794.044539517,912.545952926,5.18057476338e-08,0.000100051805748,2.98393998882,5,8,-2.98393998882,7,accepted,329.536232476,False
0,794.044539517,759.507928453,5.18057476338e-08,0.000100051805748,2.98410104597,6,9,-2.98410104597,8,accepted,261.553081047,False
0,464.508307042,912.545952926,8.14640085566e-08,0.000100081464009,2.83915821533,6,9,-2.83915821533,8,accepted,261.553081047,False
0,794.044539517,759.507928453,5.18057476338e-08,0.000100051805748,2.98410104597,6,9,-2.98410104597,8,accepted,261.553081047,False
0,794.044539517,912.545952926,5.18057476338e-08,0.000100051805748,2.98393998882,4,10,-2.98393998882,9,accepted,329.536232476,False
0,794.044539517,912.545952926,5.18057476338e-08,0.000100051805748,2.98393998882,4,10,-2.98393998882,9,accepted,329.536232476,False
0,464.508307042,794.044539517,9.21291357633e-08,0.000100092129136,2.81925891819,5,11,-2.81925891819,10,accepted,118.501413408,False
Expand All @@ -29,11 +29,14 @@ trajID,sigma-A,sigma-B,decay_A,decay_B,sum,age,batchId,fitness,iteration,accepte
0,912.545952926,794.044539517,5.18057476338e-08,0.000100051805748,3.00110960171,2,14,-3.00110960171,13,accepted,448.037645884,False
0,912.545952926,794.044539517,5.18057476338e-08,0.000100051805748,3.00110960171,3,15,-3.00110960171,14,accepted,448.037645884,False
0,912.545952926,794.044539517,5.18057476338e-08,0.000100051805748,3.00110960171,3,15,-3.00110960171,14,accepted,448.037645884,False
0,794.044539517,464.508307042,9.21291357633e-08,0.000100092129136,2.89183632446,4,16,-2.89183632446,15,accepted,369.891914497,False
0,464.508307042,794.044539517,9.21291357633e-08,0.000100092129136,2.81925891819,4,16,-2.81925891819,15,accepted,369.891914497,False
0,794.044539517,464.508307042,9.21291357633e-08,0.000100092129136,2.89183632446,4,16,-2.89183632446,15,accepted,369.891914497,False
0,794.044539517,464.508307042,9.21291357633e-08,0.000100092129136,2.89183632446,3,17,-2.89183632446,16,accepted,466.03460926,False
0,794.044539517,464.508307042,9.21291357633e-08,0.000100092129136,2.89183632446,3,17,-2.89183632446,16,accepted,466.03460926,False
0,794.044539517,464.508307042,9.21291357633e-08,0.000100092129136,2.89183632446,4,18,-2.89183632446,17,accepted,466.03460926,False
0,794.044539517,464.508307042,9.21291357633e-08,0.000100092129136,2.89183632446,4,18,-2.89183632446,17,accepted,466.03460926,False
0,464.508307042,912.545952926,8.14640085566e-08,0.000100081464009,2.83915821533,5,19,-2.83915821533,18,accepted,94.054634152,False
0,464.508307042,794.044539517,9.21291357633e-08,0.000100092129136,2.81925891819,5,19,-2.81925891819,18,accepted,94.054634152,False
0,464.508307042,912.545952926,8.14640085566e-08,0.000100081464009,2.83915821533,5,19,-2.83915821533,18,accepted,94.054634152,False
0,464.508307042,794.044539517,9.21291357633e-08,0.000100092129136,2.81925891819,3,20,-2.81925891819,19,accepted,94.054634152,False
0,464.508307042,912.545952926,8.14640085566e-08,0.000100081464009,2.83915821533,3,20,-2.83915821533,19,accepted,94.054634152,False
0,464.508307042,794.044539517,9.21291357633e-08,0.000100092129136,2.81925891819,1,20,-2.81925891819,19,final,94.054634152,False