all: update courses for 2024

We have updated the course to remove general programming guidelines
and the linter part. We are adding content to focus on pprof and other
speed and memory management practices.

.gitignore

@@ -11,6 +11,7 @@
 
 # Output of the go coverage tool, specifically when used with LiteIDE
 *.out
+*.pprof
 
 # Dependency directories (remove the comment below to include it)
 # vendor/

README.md

@@ -20,6 +20,12 @@ These exercises are for people programming but who are new to the go programming
 - [Golang Bot](https://golangbot.com/learn-golang-series/)
 - [Learn Go](https://go.dev/learn/)
 
+## Exercises
+
+[Exercise 1](/exercise-1/README.md)
+[Exercise 2](/exercise-2/README.md)
+[Exercise 3](/exercise-3/README.md)
+
 ### Solutions
 
 There are examples of solutions for each exercise in the internal solutions directory. They are not the only possible solutions to the exercises but are references that can be used if you get stuck.

ex-3-memory-sketch/README.md → archived/ex-3-memory-sketch/README.md

@@ -64,6 +64,8 @@ _cum_: cumulative
 
 ## Resources: 
 - [Scheduler saga](https://www.youtube.com/watch?v=YHRO5WQGh0k)
+- [pprof for beginners]()
+- [pprof docs]()
 - [GC traces](https://www.ardanlabs.com/blog/2019/05/garbage-collection-in-go-part2-gctraces.html)
 - [how to pprof](https://dev.to/agamm/how-to-profile-go-with-pprof-in-30-seconds-592a)
 - [Two Go Programs, Three Different Profiling Techniques](https://www.youtube.com/watch?v=nok0aYiGiYA)

exercise-1/README.md

@@ -0,0 +1,94 @@
+# Exercise 3 - Draw Go Scheduler
+
+## Part 1 - scheduling a single process go app: 
+Using the free and opensource tool [Draw IO app](https://app.diagrams.net/) draw a diagram worker-pool app from the last exercise being scheduled by the Go Scheduler
+
+## Part 2 - Add pprof to your Go App:
+Using the provided `main.go` file, add pprof and explore the memory insights of a single process Go app.
+
+If you are completing this on your own, here are some helpful videos:
+* [pprof setup](https://youtu.be/vSdOAzrVvaU)
+* [pprof cli](https://youtu.be/Fuz3fNg30cU)
+* [pprof web ui](https://youtu.be/v6skRrlXsjY)
+
+### Step 1:
+add pprof server to your text parsing app.
+First add the pprof driver to your app.
+
+```go
+import _ "net/http/pprof"
+```
+
+_*NOTE*: the "\_" means that the import is added globally as a backend system. This is common for servers, db drivers, etc_
+
+### Step 2: 
+add a pprof server as it's own goroutine in your main function.
+
+```go
+// run pprof
+go func() {
+	http.ListenAndServe("localhost:6060", nil)
+}()
+```
+
+_*NOTE*: When you do a default `ListenAndServe()` to spin up your server, your pprof is open to the public internet. To add protections use a `mux.Server()` for a custom server and you basic security precautions._
+
+### Step 3: 
+install [graphviz](https://graphviz.org/download/) on your machine to get the visual insights.
+
+*Mac:* 
+```bash
+brew install graphviz
+```
+
+### Step 4: 
+run pprof while your worker-pool is executing
+
+```bash
+go tool pprof -http=:18080 http://localhost:6060/debug/pprof/profile?seconds=30
+```
+
+In the default graph each node is a function that your program is running. Size and color indicate how much cpu and time each function is taking.
+
+To acces the commandline tool tool run:
+
+``` bash
+go tool pprof http://localhost:6060/debug/pprof/allocs
+```
+
+in the command line tool you can search for functions like this
+
+```bash
+(pprof) list worker
+```
+
+The functions will provide insights in the following categories:
+
+* allocs: A sampling of all past heap memory allocations
+* heap: A sampling of heap memory allocations of live objects. 
+* profile: CPU profile.
+* goroutine: Stack traces of all current goroutines.
+* block: Stack traces that led to blocking on synchronization primitives
+* cmdline: The command line invocation of the current program
+* mutex: Stack traces of holders of contended mutexes
+* threadcreate: Stack traces that led to the creation of new OS threads
+* trace: A trace of execution of the current program.
+
+
+### Step 5:
+
+Take some time to expore pprof. Be able to answer the following questions:
+1. What function takes the most time?
+1. What function take the most cpu?
+1. What function takes the most memory?
+1. Are any funcitons inlined?
+
+
+## Resources: 
+- [Scheduler saga](https://www.youtube.com/watch?v=YHRO5WQGh0k)
+- [pprof for beginners](https://captainnobody1.medium.com/a-beginners-guide-to-pprof-optimizing-your-go-code-c0310e59c485)
+- [pprof talk](https://www.youtube.com/watch?v=HjzJ5r2D8ZM)
+- [pprof docs](https://github.com/google/pprof/tree/main/doc)
+- [GC traces](https://www.ardanlabs.com/blog/2019/05/garbage-collection-in-go-part2-gctraces.html)
+- [how to pprof](https://dev.to/agamm/how-to-profile-go-with-pprof-in-30-seconds-592a)
+- [Two Go Programs, Three Different Profiling Techniques](https://www.youtube.com/watch?v=nok0aYiGiYA)

exercise-1/Untitled Diagram.drawio

@@ -0,0 +1,72 @@
+<mxfile host="app.diagrams.net" modified="2022-05-03T22:09:40.332Z" agent="5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36" etag="N_4qUCoIJt7H02KS_Qbp" version="17.4.6" type="github">
+  <diagram id="kgpKYQtTHZ0yAKxKKP6v" name="Page-1">
+    <mxGraphModel dx="1234" dy="733" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
+      <root>
+        <mxCell id="0" />
+        <mxCell id="1" parent="0" />
+        <mxCell id="3nuBFxr9cyL0pnOWT2aG-1" value="Worker Pool App Scheduler" style="shape=table;childLayout=tableLayout;startSize=40;collapsible=0;recursiveResize=0;expand=0;fillColor=none;fontStyle=1" parent="1" vertex="1">
+          <mxGeometry x="100" y="70" width="650" height="370" as="geometry" />
+        </mxCell>
+        <mxCell id="3nuBFxr9cyL0pnOWT2aG-3" value="Thread1" style="shape=tableRow;horizontal=0;swimlaneHead=0;swimlaneBody=0;top=0;left=0;bottom=0;right=0;fillColor=none;points=[[0,0.5],[1,0.5]];portConstraint=eastwest;startSize=40;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" parent="3nuBFxr9cyL0pnOWT2aG-1" vertex="1">
+          <mxGeometry y="40" width="650" height="90" as="geometry" />
+        </mxCell>
+        <mxCell id="3nuBFxr9cyL0pnOWT2aG-4" value="" style="swimlane;swimlaneHead=0;swimlaneBody=0;connectable=0;fillColor=none;startSize=0;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" parent="3nuBFxr9cyL0pnOWT2aG-3" vertex="1">
+          <mxGeometry x="40" width="610" height="90" as="geometry">
+            <mxRectangle width="610" height="90" as="alternateBounds" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-14" value="goroutine - Main" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="3nuBFxr9cyL0pnOWT2aG-4">
+          <mxGeometry x="10" y="15" width="240" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-21" value="GC" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="3nuBFxr9cyL0pnOWT2aG-4">
+          <mxGeometry x="260" y="15" width="60" height="305" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-22" value="goroutine - Main" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="3nuBFxr9cyL0pnOWT2aG-4">
+          <mxGeometry x="330" y="15" width="270" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="3nuBFxr9cyL0pnOWT2aG-15" value="Thread 2" style="shape=tableRow;horizontal=0;swimlaneHead=0;swimlaneBody=0;top=0;left=0;bottom=0;right=0;fillColor=none;points=[[0,0.5],[1,0.5]];portConstraint=eastwest;startSize=40;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" parent="3nuBFxr9cyL0pnOWT2aG-1" vertex="1">
+          <mxGeometry y="130" width="650" height="80" as="geometry" />
+        </mxCell>
+        <mxCell id="3nuBFxr9cyL0pnOWT2aG-16" value="" style="swimlane;swimlaneHead=0;swimlaneBody=0;connectable=0;fillColor=none;startSize=0;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" parent="3nuBFxr9cyL0pnOWT2aG-15" vertex="1">
+          <mxGeometry x="40" width="610" height="80" as="geometry">
+            <mxRectangle width="610" height="80" as="alternateBounds" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-15" value="goroutine - Worker 1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="3nuBFxr9cyL0pnOWT2aG-16">
+          <mxGeometry x="10" y="10" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-16" value="goroutine - Worker 4" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="3nuBFxr9cyL0pnOWT2aG-16">
+          <mxGeometry x="130" y="10" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-6" value="Thread 3" style="shape=tableRow;horizontal=0;swimlaneHead=0;swimlaneBody=0;top=0;left=0;bottom=0;right=0;fillColor=none;points=[[0,0.5],[1,0.5]];portConstraint=eastwest;startSize=40;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" vertex="1" parent="3nuBFxr9cyL0pnOWT2aG-1">
+          <mxGeometry y="210" width="650" height="80" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-7" style="swimlane;swimlaneHead=0;swimlaneBody=0;connectable=0;fillColor=none;startSize=0;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" vertex="1" parent="we4XuuItNtd-6PvZgcAC-6">
+          <mxGeometry x="40" width="610" height="80" as="geometry">
+            <mxRectangle width="610" height="80" as="alternateBounds" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-20" value="goroutine - Worker 2" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="we4XuuItNtd-6PvZgcAC-7">
+          <mxGeometry x="10" y="10" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-17" value="goroutine - Worker 5" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="we4XuuItNtd-6PvZgcAC-7">
+          <mxGeometry x="130" y="10" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-2" value="Thread 4" style="shape=tableRow;horizontal=0;swimlaneHead=0;swimlaneBody=0;top=0;left=0;bottom=0;right=0;fillColor=none;points=[[0,0.5],[1,0.5]];portConstraint=eastwest;startSize=40;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" vertex="1" parent="3nuBFxr9cyL0pnOWT2aG-1">
+          <mxGeometry y="290" width="650" height="80" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-3" style="swimlane;swimlaneHead=0;swimlaneBody=0;connectable=0;fillColor=none;startSize=0;collapsible=0;recursiveResize=0;expand=0;fontStyle=1;" vertex="1" parent="we4XuuItNtd-6PvZgcAC-2">
+          <mxGeometry x="40" width="610" height="80" as="geometry">
+            <mxRectangle width="610" height="80" as="alternateBounds" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-19" value="goroutine - Worker 3" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="we4XuuItNtd-6PvZgcAC-3">
+          <mxGeometry x="10" y="10" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="we4XuuItNtd-6PvZgcAC-18" value="goroutine - Worker 6" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="we4XuuItNtd-6PvZgcAC-3">
+          <mxGeometry x="130" y="10" width="120" height="60" as="geometry" />
+        </mxCell>
+      </root>
+    </mxGraphModel>
+  </diagram>
+</mxfile>

exercise-1/main.go

@@ -0,0 +1,42 @@
+package main
+
+import (
+	"fmt"
+	"math/rand"
+	"os"
+	"strings"
+	"time"
+)
+
+// getWords gets a slice of messages to process
+func getWords() []string {
+	file, _ := os.ReadFile("datums/melville-moby_dick.txt")
+	words := strings.Split(string(file), " ")
+	return words
+}
+
+func detectWords() int {
+	msgs := getWords()
+	var numWordsDetected int
+	for _, word := range msgs {
+		// golang is too powerful, so we have to slow it down to run pprof
+		// this 'sleep' is to simlutate work.
+		length := time.Duration(rand.Int63n(50))
+		time.Sleep(length * time.Millisecond)
+		// this condition returns words like whale, whaling, whales
+		if strings.Contains(word, "whal") {
+			numWordsDetected++
+		}
+	}
+	return numWordsDetected
+
+}
+
+func main() {
+
+	startTime := time.Now()
+	// start the workers in the background and wait for data on the channel
+	// we already know the number of workers, we can increase the WaitGroup once
+	numWords := detectWords()
+	fmt.Printf("Number of words: %d\nTime to process file: %2f seconds", numWords, time.Since(startTime).Seconds())
+}

exercise-1/solution/.golangci-lint.yml

@@ -0,0 +1,29 @@
+run:
+  timeout: 5m
+  issues-exit-code: 2
+  tests: false
+  modules-download-mode: readonly
+  go: '1.18' 
+  linters:
+  enable:
+    - bodyclose
+    - deadcode
+    - dogsled
+    - errcheck
+    - goconst
+    - gocyclo
+    - gofmt
+    - gosimple
+    - govet
+    - importas
+    - ineffassign
+    - misspell
+    - revive
+    - rowserrcheck
+    - sqlclosecheck
+    - staticcheck
+    - structcheck
+    - stylecheck
+    - typecheck
+    - unused
+    - varcheck