High throughput I/O (Multiple Ethernet Connections) #177
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* All SCPError derrived exceptions now include a summary of the offending packet (most importantly x, y, p and command!). The SCPPacket object is also made accessible from the exc.packet attribute. * RC errors also now contain a human readable explanation of what the error codes mean. * Instead of having many RC error exception types, there is now one FatalReturnCodeError exception type. If differentiating between the types is important, the RC is included in the exc.return_code attribute. If required in the future, subclasses can (of course) be created for individual return codes without breaking backward compatibility. This change (strictly speaking) breaks backward compatibility as it renames exceptions of the RC-specific types. @mundya are you using these exceptions anywhere and is breaking compatibility here a problem?
Instead of raising an exception, if get_machine() encounters an SCP error while probing the cores/links of a chip it now simply reports that chip as dead. This scenario most commonly occurs when a chip dies (or becomes inaccessible) some time after the P2P routing tables have been intialised. This change means that get_machine() now returns a valid subset of the machine which is still accessible and is especially useful for post-morten diagnostics, e.g. using rig-ps. Finally, get_machine() now also has an x and y argument allowing the initial P2P table reading commands to be sent to non-(0, 0) chips. Again, this is potentially useful if (0, 0) has become isolated from many other chips and an alternative ethernet connected chip is used.
If rig-ps encounters a core which return SCP errors it now prints the error and contiinues rather than falling over immediately.
This commit adds support for discovering and using multiple Ethernet connections (when available) but does not (yet) feature any way to use these connections in parallel to improve performance. A limited performance improvement due to reduced average latency within the machine may be plausible, however. This commit lifts board geometry functions from commit b666e80 (part of the now defunct non-blocking-io branch) along with the basic principles for probing the machine for Ethernet connections.
This now matches [Python's file API](https://docs.python.org/3/library/io.html#io.BufferedIOBase.readinto), hurah!
mossblaser
changed the title
|
Some preliminary results using threads to support many streams of communication: things fairly quickly become CPU bound...
(Code not yet committed/pushed due to lack of formal tests and cleaning...) |
|
...as a point of comparison, running 24 instances of rig-scp in parallel achieves something slightly over 285 MBit/s without maxing out my CPU. Either way we're off my an order of magnitude... |
7 tasks
|
This PR has now been superseded by #224; closing. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When finished, this PR fixes #142. This is a branch off the fault-tollerant-rig-ps branch and is intended for merging after #176.
read buffer in advance.
buffer. For conventional reads, a buffer should be created and then
read_into called with that buffer.
used...).
processing the queues.