tobiasBora 2017-03-07 06:45:22
I would like a program which allows me to send tasks through ssh to lot's of machines where I'm not administrator in a "queued" way.
tobiasBora 2017-03-07 06:45:29
Maybe I'm not really clear
tobiasBora 2017-03-07 06:46:30
The idea is that I put in a file all my possible entries "42 \n100 \n160..."
tobiasBora 2017-03-07 06:46:44
In another file I put the list of all the ssh machines I would like to use
tobiasBora 2017-03-07 06:47:33
The I give to the program a command to run that takes the above entries as entry
Delf 2017-03-07 06:47:41
tobiasBora: I would use tmux and task-spooler
tobiasBora 2017-03-07 06:48:28
(in my case it's a blender command that do a render), and then broadcast the tasks to every nodes.
lkn 2017-03-07 06:52:18
tobiasBora, ansible
lkn 2017-03-07 06:52:18
ansible allows you to execute commands on multiple servers using ssh
tobiasBora 2017-03-07 06:52:18
lkn: Yes, but I want also queue, I can't find queues in ansible
lkn 2017-03-07 06:52:18
what do you mean by queue?
lkn 2017-03-07 06:52:18
with ansible you can provide playbooks, so called list of commands ( queues ? )
greycat 2017-03-07 06:52:45
I think he means he only wants to run 5 at a time, and when one finishes, launch another.
greycat 2017-03-07 06:53:11
Maybe he's load-balancing 100 jobs against 5 workstations or something. It would help if he were more specific.
decuser 2017-03-07 06:53:50
Just upgraded to stretch- everything is swell, but when i rebooted- no kde started. Wound up having to reinstall sddm to get it working.
decuser 2017-03-07 06:54:48
Which took some digging since I thought I was using kdm :)
tobiasBora 2017-03-07 06:55:16
lkn: Well, let s be a set of command, I'd like that all the s commands are run at least once, by one of the n servers. Is it what playbooks are ?
tobiasBora 2017-03-07 06:55:41
greycat: Yes, it's the idea
tobiasBora 2017-03-07 06:55:52
to explain what I want to do (it will be clearer):
tobiasBora 2017-03-07 06:56:27
I have a scene that I would like to render using blender. However every frame of the movie takes 30s to render, so it takes ages to have my whole movie.
greycat 2017-03-07 06:56:28
I don't use ansible, but my understanding is that it's more suited for "I want to run these 174 commands on each of my remote servers"
tobiasBora 2017-03-07 06:56:56
Blender provides a way to run a command like "do the render from frame X to frame Y".
tobiasBora 2017-03-07 06:57:18
So the idea is that I cut all the frame into bunch of 10 frames (about 5mn to run)
tobiasBora 2017-03-07 06:57:24
all the frames*
greycat 2017-03-07 06:57:38
tobiasBora: this is not something I've ever used before, but maybe dsh? I've heard people mention it.
decuser 2017-03-07 06:58:03
As a side note and q, i used apt-get autoremove - scary number of unused pkgs. is that normal and should I tremble with trepidation about using it. In the past it wreaked all kinds of havok with my sys.
tobiasBora 2017-03-07 06:58:06
Delf: I tried to understand task-spooler, but not sure to understand everything for now
decuser 2017-03-07 06:58:15
This time seems ok.
Delf 2017-03-07 06:59:03
tobiasBora: Did you read the manual? after installing task-spooler, you needs to do "man tsp" for whatever confusing reason
greycat 2017-03-07 06:59:11
Or if all else fails, you can cobble together something using GNU parallel or xargs -P
tobiasBora 2017-03-07 07:01:13
greycat: I don't see any queue thing inside dsh
tobiasBora 2017-03-07 07:02:58
Delf: I don't see any ssh access to task-spooler, so I have troubles to understand how it can be used by the nodes (by the way, the user (me) does not have a computer with a global address, he can be beside a NAT)
greycat 2017-03-07 07:03:24
You would include the ssh in your commands.
tobiasBora 2017-03-07 07:05:09
ahhhh
tobiasBora 2017-03-07 07:06:58
like if, instead of keeping one connection, I always create a new connection... It's not very optimised, but since my tasks are quite long to run it's a good idea !
greycat 2017-03-07 07:07:25
Yeah, this is a low-budget approach.
tobiasBora 2017-03-07 07:09:11
greycat: Delf: is it in the debian repo ? I can't find it
greycat 2017-03-07 07:10:22
What?
greycat 2017-03-07 07:11:14
so are dsh and parallel, and of course xargs is in the base system
Delf 2017-03-07 07:11:53
We wouldn't ever suggest something outside the Debian repo… would we?
greycat 2017-03-07 07:13:07
Sure.
tobiasBora 2017-03-07 07:15:57
Delf: I was talking about task-spooler
greycat 2017-03-07 07:16:25
,v task-spooler
judd 2017-03-07 07:16:26
Package: task-spooler on amd64 -- wheezy: 0.7.3-1; jessie: 0.7.4-1; stretch: 1.0-1; sid: 1.0-1
tobiasBora 2017-03-07 07:16:27
hum wait
tobiasBora 2017-03-07 07:16:50
hum sorry, don't know why I didn't find it ^^'
tobiasBora 2017-03-07 07:17:14
But their is a problem with task-spooler :
Delf 2017-03-07 07:17:24
tobiasBora: Spelling is a matter of life/death! :)
tobiasBora 2017-03-07 07:17:28
I need to provide at the beginning which server is supposed to do what
tobiasBora 2017-03-07 07:18:43
so it makes ts pretty useless because "the hard part" is to deal with the fact that all the tasks does not last the same amount of time, and also some nodes are slower than others
Delf 2017-03-07 07:20:12
tobiasBora: You want list of commands, passed to servers, and which server depends on which server happens to be not doing any work?
baphomet` 2017-03-07 07:30:58
How do I search for backdoors in the debian software?
jelly 2017-03-07 07:35:52
baphomet`: how do you search for backdoors anywhere?
abrotman 2017-03-07 07:38:00
walk around the back
`Kevin 2017-03-07 07:38:00
what a broad question :|
babilen 2017-03-07 07:41:06
Would be easy to fine one if the rules of the game allow one to open them also
babilen 2017-03-07 07:41:12
*find
Noetic 2017-03-07 07:42:14
any i3 users here?
jasonwc 2017-03-07 07:43:25
Noetic is in Core i3?
Noetic 2017-03-07 07:43:48
i3 wm, but its ok i found this irc chan, so the question I have I will direct there
