Proper timeout for jobs


#1

ci jobs don’t need to run for hours - at the least, allow for max-time cutoff of any running job.

per discussion with Lev Lazinskiy, I think it would be useful (at least for a University team I know of who is a paying customer) to set a max timeout for all jobs for a project. That is: since these aren’t computational pipelines, but simply integration suites (e2e tests, unit tests, etc.) there is absolutely no reason we ever want to see a job has managed to run for 2 hours. None the less, there are many cases where we find this happening.


#2