Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor kernel memory usage #183

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Refactor kernel memory usage #183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 75 additions & 58 deletions doc/memory
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
KnightOS Memory Layout
Kernel memory structures

Note that this describes the default settings, and that the actual layout of
memory is subject to change depending on how the kernel is configured. Userspace
programs mucking around in memory manually is very strongly frowned upon.
programs mucking around in memory manually is _very strongly_ frowned upon. All
kernel data structures are subject to change.

Overall Layout
Overall layout

Address Length Description

Expand All @@ -21,88 +22,84 @@ functions, including boot, filesystem access, context switching, etc.
Volatile Flash access is generally only used with interrupts disabled to prevent
other processes from ruining your day.

RAM Layout
RAM layout

Right now, KnightOS only uses 0x8000 (32K) of RAM, even on devices that may have
additional RAM available. Nothing of value is kept in RAM, and it may be safely
wiped during a reboot (and in fact, the kernel does so).

Note that RAM data structures are scheduled to change dramatically for kernel
0.7.0.

Address Length Description

0x8000 0x100 Thread table
0x8100 0x40 Library table
0x8140 0x40 Signal table
0x8180 0x40 File handle table
0x8200 0x100 Misc kernel state
0x8300 0x100 Kernel flash functions
0x8400 0x200 Volatile kernel garbage
0x8600 0x7A00 Userspace memory
0x8000 0x80 Miscellaneous kernel state
0x8080 0x80 Flash execution space
0x8100 0x100 Volatile kernel utility space (kernelGarbage)
0x8200 0x7D00 Heap

Misc kernel state holds several statically allocated kernel variables. Flash
execution space is reserved for the Flash driver to use to execute Flash
operations, which must be run from RAM. Volatile kernel utility space is a fixed
buffer used for various kernel operations where dynamic memory allocation cannot
fail. Finally, the heap is where `malloc` allocates memory from.

Userspace memory is where `malloc` allocates memory. Volatile kernel garbage is
used throughout the kernel whenever it just needs a little space to work, and is
accessed with interrupts disabled. Kernel flash functions are where the Flash
driver loads executable code to perform Flash manipulation from RAM. Misc kernel
state is used for various things like the currently executing thread.
The process table

Data Structures
The kernel supports preemptive multitasking, and to this end it keeps track of
running processes and their resources. The process table is where this state is
stored, and it is allocated in the heap and grows or shrinks and demands change.
There are some variables in misc kernel state that keep track of the process
table:

Each of the tables is a fixed-length array that holds several of the following
data structures.
process_table Pointer to process table in the heap
active_processes Number of active processes
current_process Index of the currently running process
last_pid The last PID allocated by the kernel
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These variables are currently called threadTable, activeThreads, currentThreadIndex, and lastThreadID. Part of this change is to rename all of the kernel internal memory to use snake_case. Also, note that threadTable is a fixed address currently - you'll have to stick two bytes of memory somewhere in the kernel state and update code that refers to threadTable to read that instead.


Threads
At boot, the kernel allocates a process table large enough to hold 8 processes.
As more processes are started, the process table may be `realloc`d to hold
more or less process state (though never less than 8). Each process state
structure is designed like so:

Address Length Description

0x0000 0x01 Thread ID
0x0000 0x01 Process ID (pid)
0x0001 0x02 Executable address
0x0003 0x02 Stack pointer
0x0003 0x02 Stack address (SP)
0x0005 0x01 Flags (bitfield)
1: May be suspended
2: Is suspended
3: Color enabled
0x0006 0x02 Reserved for future use

Libraries

Address Length Description

0x0000 0x01 Library ID
0x0001 0x02 Library address
0x0003 0x01 Dependent threads

Signals

Address Length Description

0x0000 0x01 Target thread ID
0x0001 0x01 Message type
0x0002 0x02 Payload

File Handles
0: May be suspended
1: Is suspended
2: Color enabled (84+CSE only)
0x0006 0x02 Pointer to file handle table
0x0008 0x02 Pointer to library handle table
0x000A 0x02 Pointer to signal table
0x000C 0x03 Reserved for future use
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the length of a process entry has changed.


Each process has its own stack, file descriptor table, library handle table, and
signal table. These are also allocated in the heap and expand or shrink to meet
the process's needs. By default, each process is allocated a file descriptor
table 4 entries long, a library handle table 2 entries long, and no signal
table (the pointer will equal zero). Each table begins with a byte giving the
number of entries present, then the table itself follows.

File handle

Address Length Description

0x0000 0x01 Flags/Owner
0x0000 0x01 Flags (bitfield)
0: Writable if 1
1: EoF if 1
2: Flushed if 1
3: Set if final block of the file*
4-7: Reserved for future use
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, flags were split across two different parts of the file handle entry, because I was lazy. This merges the flags into one byte (since we don't need owner any more) and rearranges them a bit with future changes in mind.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was owner needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To identify which thread owned the file. It was necessary when all file handles were in one table, but now that each process is going to have its own file handle table it's not necessary.

0x0001 0x02 Buffer address
0x0003 0x01 Stream pointer
0x0004 0x02 Section ID
0x0006 0x01 Length of final block
0x0007 0x01 File entry page
0x0008 0x02 File entry address
0x000A 0x03 File working size
0x000D 0x01 Writable stream flags
0x000E 0x02 Reserved for future use
0x000D 0x03 Reserved for future use

Flags/Owner is the thread ID of the file handle's owner. It takes the format of
FTExxxxx, where xxxxx is the thread ID. F is set to indicate that the stream is
pointing at the final block of the file. T is set if the thread is writable. E
is set if the file handle is set to EOF.

Writable stream flags are set to xxxxxxF, where F is set if the stream has been
flushed, and x is reserved for future use.
All kernel data structures are subject to change, but this one especially so.

The buffer address refers to the address of the 256-byte stream buffer, which
contains the contents of the current DAT block the stream points to. The stream
Expand All @@ -114,3 +111,23 @@ in the KFS docs.

The length of the final block is used to determine when EOF has been reached on
the final block.

Library handle

Address Length Description

0x0000 0x02 Library ID
0x0002 0x02 Library address

When loading a library, the kernel looks at libraries loaded by other processes
for one with the same library ID, and copies the entry into this process if
found. When a process is killed, the kernel looks to see if this library is in
use by any other processes, and unloads it if not.

Signal handler

Address Length Description

0x0000 0x02 Signal handler
0x0002 0x01 Signal number
0x0003 0x01 Reserved for future use
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signals will have to change to be more Unix-like, and some userspace programs may have to change to compensate for this (and some syscalls too). createSignal and readSignal will have to be removed and replaced with signal and kill, which should behave like their POSIX counterparts.

6 changes: 6 additions & 0 deletions src/00/flash.asm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ _: pop af
ld (0), a
jp .return
.ram_end:
.echo "writeFlashByte RAM footprint: 0x{0:X4}" .ram_end-.ram

;; writeFlashBuffer [Flash]
;; Writes several bytes of memory to Flash
Expand Down Expand Up @@ -195,6 +196,7 @@ _: pop af
jr nz, .loop
jp .return
.ram_end:
.echo "writeFlashBuffer RAM footprint: 0x{0:X4}" .ram_end-.ram

;; eraseSwapSector [Flash]
;; Erases the swap sector.
Expand Down Expand Up @@ -267,6 +269,7 @@ _: ld a, (0)
ld (0x4000), a
jp .return
.ram_end:
.echo "eraseFlashSector RAM footprint: 0x{0:X4}" .ram_end-.ram

;; eraseFlashPage [Flash]
;; Erases a single page of Flash.
Expand Down Expand Up @@ -488,6 +491,7 @@ _: pop af
jp .return
.ram_end:
#endif
.echo "copySectorToSwap RAM footprint: 0x{0:X4}" .ram_end-.ram

;; copyFlashExcept [Flash]
;; Copies one Flash page to another, but omits a certain range of bytes in increments of
Expand Down Expand Up @@ -690,6 +694,7 @@ _: inc hl
jr .continue_loop
.ram_end:
#endif
.echo "copyFlashExcept RAM footprint: 0x{0:X4}" .ram_end-.ram

;; copyFlashPage [Flash]
;; Copies one page of Flash to another.
Expand Down Expand Up @@ -841,3 +846,4 @@ _: inc hl
jp .return
.ram_end:
#endif
.echo "copyFlashPage RAM footprint: 0x{0:X4}" .ram_end-.ram