tutorials: add flux proxy command tutorial #200

chu11 · 2023-02-10T22:31:11Z

No description provided.

vsoch · 2023-02-10T22:34:57Z

tutorials/commands/index.rst

 - ``flux-mini-submit`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance"
+ - ``flux proxy`` (:ref:`flux-proxy-command`): "flux proxy basics"


Can we here give the reader a better sense of why they would want to use flux proxy, and how it differs from the "ssh across clusters" tutorial?

vsoch · 2023-02-10T22:36:49Z

tutorials/commands/flux-proxy-command.rst

+Flux Proxy Basics
+=================
+
+With Flux, it is very common to create a Flux :ref:`subinstances<subinstance>`.  However, it can be confusing how to interact with those subinstances.


So this is saying when I launch a flux job that controls some set of subinstances, I would use flux proxy to communicate with them? Or could I have flux controlling multiple clusters, and then flux proxy would give me access to a cluster? What is a subinstance (or what can it be) in concrete terms? Why would I want to communicate with it? E.g., what we might have here is "you might want to do this if X or Y or Z" is what you are doing.

Maybe a better intro here would be to talk about traditional batch jobs running a batch script, and how in Flux those "batch jobs" are actually running a full instance of Flux, and the flux proxy command can be used to execute processes, including an interactive shell, as if they were connected directly to the instance of Flux running your batch workload.

Also, flux proxy is not restricted to running against subinstances. You can also use it to connect to a system instance running on another cluster, a test instance running locally, or as you show below, a Flux instance running under a Slurm, LSF, or other resource manager job.

Maybe a better intro here would be to talk about traditional batch jobs running a batch script, and how in Flux those "batch jobs" are actually running a full instance of Flux, and the flux proxy command can be used to execute processes, including an interactive shell, as if they were connected directly to the instance of Flux running your batch workload.

The way I imagined it was that we'd eventually have this discussed more in a flux mini batch tutorial. Although I can probably go into it a little more than I have so the reader isn't quite as clueless.

vsoch · 2023-02-10T22:38:47Z

tutorials/commands/flux-proxy-command.rst

+
+.. code-block:: sh
+
+   #!/bin/sh


Wait where does flux mini batch come in? This looks like it's running 4x flux mini submit?

Is there a batch equivalent?

To answer your question @v, flux mini batch is the flux mini utility that submits a "batch job" which in Flux is a job that runs an instance of Flux on the assigned resources with a batch script as its "initial program"

okay - so you give it a script with other flux submit commands? As opposed to (what I've seen for other managers) a text file with some list of single commands (without the job manager directive) to run?

How does that (conceptually) compare to a job array?

Wait where does flux mini batch come in? This looks like it's running 4x flux mini submit?

I listed the script and then run the script in flux mini batch right below, I guess I should put a sentence in between, perhaps its not obvious when skimming.

vsoch · 2023-02-10T22:39:36Z

tutorials/commands/flux-proxy-command.rst

+
+.. code-block:: console
+
+   > flux mini batch -n4 ./subinstance-jobs.sh


I totally don't understand what is going on here, lol (I've never used batch before).

OK going to stop here and submit this early feedback because I can't follow along (sorry, not familiar with batch!)

lemme try to add a bit more details then. one issue is we are writing these tutorials out of order. I jumped on flux proxy mostly b/c you already wrote the ssh-across-clusters one. Really flux mini batch probably should have come after flux mini submit. which I will add to the todo list.

shouldn't we encourage the use of --cc in this case? Possibly using waitable jobs and flux job wait for more "standard" practice.

for some of these small examples I figure its easier to do this than have to explain --cc (leaving --cc to the #195).

Do we have a preference on flux job wait vs flux job status for something this small?

Good points. Given that, I'm fine with current example, was more of a question anyway.

grondo

Also added some quick comments, though I didn't get all the way through.

grondo · 2023-02-10T23:51:04Z

tutorials/commands/flux-proxy-command.rst

+Flux Proxy Basics
+=================
+
+With Flux, it is very common to create a Flux :ref:`subinstances<subinstance>`.  However, it can be confusing how to interact with those subinstances.


Maybe a better intro here would be to talk about traditional batch jobs running a batch script, and how in Flux those "batch jobs" are actually running a full instance of Flux, and the flux proxy command can be used to execute processes, including an interactive shell, as if they were connected directly to the instance of Flux running your batch workload.

Also, flux proxy is not restricted to running against subinstances. You can also use it to connect to a system instance running on another cluster, a test instance running locally, or as you show below, a Flux instance running under a Slurm, LSF, or other resource manager job.

grondo · 2023-02-10T23:53:28Z

tutorials/commands/flux-proxy-command.rst

+
+.. code-block:: sh
+
+   #!/bin/sh


To answer your question @v, flux mini batch is the flux mini utility that submits a "batch job" which in Flux is a job that runs an instance of Flux on the assigned resources with a batch script as its "initial program"

grondo · 2023-02-10T23:54:12Z

tutorials/commands/flux-proxy-command.rst

+
+.. code-block:: console
+
+   > flux mini batch -n4 ./subinstance-jobs.sh


shouldn't we encourage the use of --cc in this case? Possibly using waitable jobs and flux job wait for more "standard" practice.

grondo · 2023-02-10T23:55:14Z

tutorials/commands/flux-proxy-command.rst

+Connect to the Instance
+-----------------------
+
+The easiest way to operate with a subinstance is to use ``flux-proxy``.  You can pass it the subinstance jobid and the command you want to run against that subinstance.  Lets submit a job via ``flux mini submit`` and list it via ``flux jobs``.


I'd remove subinstance here, might be less confusing to just say jobid.

grondo · 2023-02-10T23:55:48Z

tutorials/commands/flux-proxy-command.rst

+          fh2fcK1 achu     sleep       R      1      1   4.152m opal186
+          fa6txwZ achu     sleep       R      1      1   4.157m opal186
+
+Notice that we're running ``flux jobs`` against the subinstance itself.  We don't have to specify the ``--recursive`` option to list the jobids anymore.


More interesting might be to submit more work to the instance (this is where flux job wait --all would come in handy)

grondo · 2023-02-10T23:56:58Z

tutorials/commands/flux-proxy-command.rst

+
+Notice that we're running ``flux jobs`` against the subinstance itself.  We don't have to specify the ``--recursive`` option to list the jobids anymore.
+
+You can also ssh to the node if you know the subinstance's Flux :ref:`URI<URI>`.  This may be useful if you want to proxy to the subinstance from outside of the cluster at a later time (see :ref:`SSH Across Clusters`<ssh-across-clusters>` for more information).


I think you mean here "you can also specify a native uri" instead of a target jobid?

grondo · 2023-02-10T23:57:30Z

tutorials/commands/flux-proxy-command.rst

+          fa6txwZ achu     sleep       R      1      1   6.385m opal186
+
+
+If you do not specify a command to run, you will be dropped into a shell that will forward all commands to the subinstance.


Suggested change

If you do not specify a command to run, you will be dropped into a shell that will forward all commands to the subinstance.

If you do not specify a command to run, you will be dropped into a shell that will forward all messages to the subinstance.

?

chu11 · 2023-02-13T15:21:54Z

Re-pushed, re-working the flow of the tutorial in a different way. Instead of starting with flux mini batch I start with flux mini alloc and try to add a little more description about subinstances along the way. I could certainly add a bit more, but I think its best to save that for an eventual flux mini alloc / flux mini batch tutorial that we can eventually point to.

Side note, pushed the dumb cleanups into #202, so its only the tutorial now

chu11 · 2023-02-17T15:19:21Z

re-pushed with some minor tweaks given discussion from other PRs. If there is still a little confusion of what is flux mini batch doing and why would I want to make a subinstance, I think that there will be future tutorials to deal with that.

vsoch · 2023-02-17T16:39:34Z

tutorials/commands/index.rst

@@ -7,6 +7,7 @@ Welcome to the Command Tutorials! These tutorials should help you to map specifi
 with your use case, and then see detailed usage. 

 - ``flux mini submit/flux mini run`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance"
+ - ``flux proxy`` (:ref:`flux-proxy-command`): "Send commands to other Flux instances"


Yeah I guess this is good for now - some of these are definitely more advanced.

chu11 · 2023-03-27T13:59:13Z

rebased, removing "flux mini" references

Problem: There's no flux proxy command tutorial. Add one.

vsoch reviewed Feb 10, 2023

View reviewed changes

grondo reviewed Feb 10, 2023

View reviewed changes

chu11 force-pushed the flux_proxy_tutorial branch from 226ee89 to 293cc90 Compare February 13, 2023 15:21

chu11 mentioned this pull request Feb 13, 2023

tutorials: add flux mini run tutorial/example #202

Merged

chu11 force-pushed the flux_proxy_tutorial branch from 293cc90 to 2efaf74 Compare February 13, 2023 18:53

chu11 mentioned this pull request Feb 13, 2023

Command Tutorials to do #193

Open

16 tasks

chu11 force-pushed the flux_proxy_tutorial branch from 2efaf74 to 9860222 Compare February 17, 2023 15:16

chu11 force-pushed the flux_proxy_tutorial branch from 9860222 to dc7ec31 Compare February 17, 2023 15:23

vsoch reviewed Feb 17, 2023

View reviewed changes

chu11 force-pushed the flux_proxy_tutorial branch 3 times, most recently from d167514 to 0078415 Compare March 27, 2023 13:57

chu11 force-pushed the flux_proxy_tutorial branch from 0078415 to 47103a0 Compare March 27, 2023 14:00

chu11 force-pushed the flux_proxy_tutorial branch from 47103a0 to 693b2b9 Compare June 7, 2023 18:51

tutorials: add flux proxy command tutorial

c6b16a7

Problem: There's no flux proxy command tutorial. Add one.

chu11 force-pushed the flux_proxy_tutorial branch from 693b2b9 to c6b16a7 Compare June 8, 2023 15:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tutorials: add flux proxy command tutorial #200

tutorials: add flux proxy command tutorial #200

chu11 commented Feb 10, 2023

vsoch Feb 10, 2023

vsoch Feb 10, 2023

grondo Feb 10, 2023

chu11 Feb 11, 2023

vsoch Feb 10, 2023

vsoch Feb 10, 2023

grondo Feb 10, 2023

vsoch Feb 11, 2023

vsoch Feb 11, 2023

chu11 Feb 11, 2023 •

edited

Loading

vsoch Feb 10, 2023

vsoch Feb 10, 2023

chu11 Feb 10, 2023

grondo Feb 10, 2023

chu11 Feb 11, 2023

grondo Feb 11, 2023 •

edited

Loading

grondo left a comment

grondo Feb 10, 2023

grondo Feb 10, 2023

grondo Feb 10, 2023

grondo Feb 10, 2023

grondo Feb 10, 2023

grondo Feb 10, 2023

grondo Feb 10, 2023

chu11 commented Feb 13, 2023 •

edited

Loading

chu11 commented Feb 17, 2023

vsoch Feb 17, 2023

chu11 commented Mar 27, 2023

		- ``flux-mini-submit`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance"
		- ``flux proxy`` (:ref:`flux-proxy-command`): "flux proxy basics"


		.. code-block:: console

		> flux mini batch -n4 ./subinstance-jobs.sh


		Notice that we're running ``flux jobs`` against the subinstance itself. We don't have to specify the ``--recursive`` option to list the jobids anymore.

		You can also ssh to the node if you know the subinstance's Flux :ref:`URI<URI>`. This may be useful if you want to proxy to the subinstance from outside of the cluster at a later time (see :ref:`SSH Across Clusters`<ssh-across-clusters>` for more information).

		fa6txwZ achu sleep R 1 1 6.385m opal186


		If you do not specify a command to run, you will be dropped into a shell that will forward all commands to the subinstance.

tutorials: add flux proxy command tutorial #200

Are you sure you want to change the base?

tutorials: add flux proxy command tutorial #200

Conversation

chu11 commented Feb 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chu11 Feb 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grondo Feb 11, 2023 • edited Loading

Choose a reason for hiding this comment

grondo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chu11 commented Feb 13, 2023 • edited Loading

chu11 commented Feb 17, 2023

Choose a reason for hiding this comment

chu11 commented Mar 27, 2023

chu11 Feb 11, 2023 •

edited

Loading

grondo Feb 11, 2023 •

edited

Loading

chu11 commented Feb 13, 2023 •

edited

Loading