Script to run tormes on HPC clusters

[biobrad]

Hi all,
not an issue, but thought I would share a script I wrote that auto-creates the samples_metadata file from the files you have in your folder and also adds the reference genome file to the arguments.
It then sftps your files over to a HPC (it is configured for the one at my university so you would need to change it). It also creates a 'slurm' job submission file to run tormes. Finally, it also creates a script to retrieve the files once you receive notification that tormes has finished.

It is my first script so it is probably a bit under optimised, but it gets the job done and the parts that parse the file names into an array and then puts them into the samples_metadata text file may come in handy for other people (that was the part that took me the longest!!!)

Cheers!
Brad
tormesbot.txt

[nmquijada]

Hi Brad,

Really thanks for sharing your code. I think it will be helpful for other users using HPC and slurm. When we started the tormes project, networking with people by giving and receiving feedback was one of our main aims.
Have you considered creating a GitHub repository? If so, I would reference to your code in the tormes documentation. Otherwise I might refer to this issue.
Good that you opened the box of scripting! The first most important thing is that it gets the job done :)

Best,
Narciso

[biobrad]

Hi Narciso,

Done! I have created a repository here: https://github.com/stenotrophomonas/tormesbot/wiki/Tormesbot

Feel free to delete this thread from the issues section!

cheers mate
Brad

[nmquijada]

It is better later than never...
Welcome to the tormes documentation ;)

[biobrad]

Hi Narciso, Hahaha, thank you! Wow, you moved countries! That is a lot of disruption and distraction! I don't want to put any pressure on you, I just really like Tormes and think a lot more people probably use it than you know. I understand what you are saying about the spades pipeline. Wow, you really did do a lot of analysis of your results when running at different hardware settings! Speaking of updates, I am not sure that anyone would use it but my HPC script has gone through quite a few changes as well. I should throw in an update and also add the stand alone 'samples_metadata' script I extracted from it. Thank you again for such a great piece of software. It must be extremely difficult making sure it all stays together considering everything is a dependency on a dependency! Also thank you for saying that I could make amendments to the code. I have only been doing bioinformatics for 6 months now, at this stage I cannot think of anything I would do to make it better! ooo, actually.... contamination detection. I am waiting to get database access for confindr... depending on how that goes I might see if I can work it in somehow. I don't have much time left though until my classes start again so I will see how I go. I hope things are going well in the new lab and country. Cheers Brad

…

------ Original Message ------ From: "Narciso M. Quijada" <notifications@github.com<mailto:notifications@github.com>> To: "nmquijada/tormes" <tormes@noreply.github.com<mailto:tormes@noreply.github.com>> Cc: "biobrad" <pathetichumans@hotmail.com<mailto:pathetichumans@hotmail.com>>; "Author" <author@noreply.github.com<mailto:author@noreply.github.com>> Sent: 31/01/2020 10:25:39 PM Subject: Re: [nmquijada/tormes] Script to run tormes on HPC clusters (#14) It is better later than never... Welcome to the tormes documentation ;) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnmquijada%2Ftormes%2Fissues%2F14%3Femail_source%3Dnotifications%26email_token%3DANITBGTXFW2CYMV5TQOJSJDRAQGTXA5CNFSM4J6J4CK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKONTVY%23issuecomment-580704727&data=02%7C01%7C%7C80dc867a6a994b853e1f08d7a6447e27%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637160685408875789&sdata=Y%2Bsn82AR0OVxhTsqrFdFxFbr5d1B6r%2BmyCrNriOY9lk%3D&reserved=0>, or unsubscribe<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANITBGTISEPFQ26ZAHOJBX3RAQGTXANCNFSM4J6J4CKQ&data=02%7C01%7C%7C80dc867a6a994b853e1f08d7a6447e27%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637160685408885800&sdata=Ny8gv4ZyxLv6oh3XO5F1SRnon%2FAsswrEETZFGI4qmyU%3D&reserved=0>.