A simulation failed after 8 hours without any reason
Hi,
I ran a 3D simulation and it failed after more than 8 hours without providing any reason and I did not receive any results. The simulation took 136.7 CH taking most of my core hours. If a model is correct then I expect to receive final results. Could you help me with this?
Thanks,
Piotr
14 comments
Hi Piotr,
Could you share the files with us via google drive or similar?
Thanks,
Oliver
Hi Oliver,
You can download the files from https://drive.google.com/file/d/1fPTzVUgT2pKlKULzWNZibn-5Lpu6e7x_/view?usp=sharing
Thanks,
Piotr
Hi Piotr,
Can I just confirm, have any files been transferred to the storage? I'm looking for the flxprt and cosole log files.
If you have these files can you add them to the drive.
Thanks,
Oliver
Oliver,
No, the storage containes only input files which I shared with you. It did not create any output files.
Piotr
Oliver,
Any updates on this? Have you tried to run the model I sent you? Is there a possibility that the simulation failed due to insufficient memory allocation? I have the impression that the problem was caused by exporting the results to Matlab.
Could you also reimburse me CH that were deduced by this simulation?
Thanks,
Piotr
Hi Piotr,
Sorry about the delay, it's a simple model there isn't any issues with the setup.
What version of the application are you using? This can be found clicking about in the help tab. I've tested this on 1.30.11 and the matlab file is transfered to the storage. This version is avaliable to download from the portal can you update and retest this please
Can you provide the JobID and Job Name of the failed simulation that cost 136CH? Open Analyst click Job Status and locate the job in the job status window, double click to open and send me the information I need to confirm this before we reimburse the CH
Best regards,
Oliver
Oliver,
Thanks for the response. You are right that the model is very simple and it should not fail but it did.
I did one more test today with this model by reducing a simulation time from 0.05 s to 0.0005 s and it worked fine. The output folder was created in the storage with all the data. Hence, I am not sure if the problem was a memory allocation or something else.
I have the newest version of the application, 1.30.11.0.
The JobID and Job Name of the failed simulation are as below
Thanks,
Piotr
Here is also a summary of this Job
Core Hour Estimate: 101
Total Core Hours: 136.70
Status: FAILED - license acquisition
Piotr
Hi Oliver,
Any updates on this? Do you need any other information from my side?
Thanks,
Piotr
Hi Piotr,
Sorry about the delay. I can reimburse the CH lost due to the fail. The status message Failed - License Acquisition normally means the simulation has failed because you have run out of CH.
May I suggest you change the target HPC in the settings from GCP to AWS as we are seeing a slowdown on GCP. This can be changed in the cloud account configuration.
Oliver
Hi Oliver,
Thank you for reimbursing the CH. I really appreciate it.
I will change the settings to AWS as you suggest.
If the model failed due to not enough CH then it means that despite the ~100 CH estimated the total CH was ~150 CH or more.
Piotr
Hi Oliver,
I ran a simulation one more time. This time however, I used twice smaller model compared to the previous one to make sure that I have enough CH. (Core hour estimate was 45.6; Simulation Study Cost: 64.78 CH). I have changed the settings from GCP to AWS as you recommended. Unfortunately, the simulation failed again. Here is a report:
Fortunately the "console" file has been generated which says "Insufficient memory allocation for problem". Could you please confirm that a solution for this is to override the estimates for RAM assigment?
I will appreciate I could you also reimburse the CH that were taken for this simulation. The JobID and Job Name of the the simulation are:
Thank you for your help,
Piotr
Can you add the new files you are working with to the google drive linked above and I will look into this.
If I remember correctly the last model you were simulating was outputting a ton of data, the estimate isn't taking that into account which is why you will be seeing the insufficent memmory allocation errors.
Oliver
Here is a link to the input files https://drive.google.com/drive/folders/1jgvP4XjXaeetdWvTWcIgmJsFX9OqiG-8?usp=sharing
In this model I want to save velocity profiles in Z axis from ~7700 points to Matlab.
Thanks,
Piotr
Please sign in to leave a comment.
Didn't find what you were looking for?
New post