leodido.dev - GoLeo's take on security, eBPF, Linux, kernel, and whatever tech he meetsZola2020-11-18T00:00:00+00:00https://leodido.dev/tags/go/atom.xmlgo-conventionalcommits2020-11-18T00:00:00+00:002020-11-18T00:00:00+00:00https://leodido.dev/projects/go-conventionalcommits/<blockquote>
<p>Fu powers to parse your commits!</p>
</blockquote>
<p>This repository provides a library to parse your commit messages according to the Conventional Commits v1.0 specification.</p>
<h2 id="installation">Installation<a class="zola-anchor" href="#installation" aria-label="Anchor link for: installation">§</a>
</h2>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">go get github.com/leodido/go-conventionalcommits
</code></pre>
<h2 id="docs">Docs<a class="zola-anchor" href="#docs" aria-label="Anchor link for: docs">§</a>
</h2>
<p>The <a rel="noopener noreferrer" target="_blank" href="https://github.com/leodido/go-conventionalcommits/tree/develop/parser/docs/">parser/docs</a> directory contains <code>.dot</code> and <code>.png</code> files representing the finite-state machines (FSMs) implementing the parser.</p>
<h2 id="usage">Usage<a class="zola-anchor" href="#usage" aria-label="Anchor link for: usage">§</a>
</h2>
<h3 id="parse">Parse<a class="zola-anchor" href="#parse" aria-label="Anchor link for: parse">§</a>
</h3>
<p>Your code base uses only single line commit messages like this one?</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">feat: awesomeness
</code></pre>
<p>No problem at all since the body and the footer parts are not mandatory:</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">m, _ := parser.NewMachine().Parse([]byte(`feat: awesomeness`))
</code></pre>
<h3 id="full-conventional-commit-messages">Full conventional commit messages<a class="zola-anchor" href="#full-conventional-commit-messages" aria-label="Anchor link for: full-conventional-commit-messages">§</a>
</h3>
<p>Imagine you have a commit message like this:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">docs: correct minor typos
see the issue for details
on docs edits.
Reviewed-by: Z
Refs #133
</code></pre>
<p>Go with this:</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">opts := []conventionalcommits.MachineOption{
WithTypes(conventionalcommits.TypesConventional),
}
res, err := parser.NewMachine(opts...).Parse(i)
</code></pre>
<p>Or, more simpler:</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">res, err := parser.NewMachine(WithTypes(conventionalcommits.TypesConventional)).Parse(i)
</code></pre>
<h3 id="types">Types<a class="zola-anchor" href="#types" aria-label="Anchor link for: types">§</a>
</h3>
<p>This library provides support for different types sets:</p>
<ul>
<li><strong>minimal</strong>: fix, feat</li>
<li><strong>conventional</strong>: build, ci, chore, docs, feat, fix, perf, refactor, revert, style, test</li>
<li><strong>falco</strong>: build, ci, chore, docs, feat, fix, perf, new, revert, update, test, rule</li>
</ul>
<p>At the moment, those types are at build time. Which means users can’t configure them at runtime.</p>
<p>Anyway, there’s also a <strong>free-form</strong> types set that accepts any combination of printable characters (before the separator after which the commit description starts) as a valid type.</p>
<p>You can choose the type set passing the <code>WithTypes(conventionalcommits.TypesConventional)</code> option as shown above.</p>
<h3 id="options">Options<a class="zola-anchor" href="#options" aria-label="Anchor link for: options">§</a>
</h3>
<p>A parser behaviour is configurable by using options.</p>
<p>You can set them calling a function on the parser machine.</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">p := parser.NewMachine()
p.WithBestEffort()
res, err := p.Parse(i)
</code></pre>
<p>Or you can provide options to <code>NewMachine(...)</code> directly.</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">p := parser.NewMachine(WithBestEffort())
res, err := p.Parse(i)
</code></pre>
<h3 id="best-effort">Best effort<a class="zola-anchor" href="#best-effort" aria-label="Anchor link for: best-effort">§</a>
</h3>
<p>The best effort mode will make the parser return what it found until the point it errored out,
if it found (at least) a valid type and a valid description.</p>
<p>Let’s make an example.</p>
<p>Suppose this input commit message:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">fix: description
a blank line is mandatory to start the body part of the commit message!
</code></pre>
<p>The input does not respect the Conventional Commits v1 specification because it lacks a blank line after the description (before the body).</p>
<p>Anyways, if the parser you’re using has the best effort mode enabled, you can still obtain some structured data since at least a valid type and description have been found!</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">res, err := parser.NewMachine(WithBestEffort()).Parse(i)
</code></pre>
<p>The result will contain a <code>ConventionalCommit</code> struct instance with the <code>Type</code> and the <code>Description</code> fields populated and ignore the rest after the error column.</p>
<p>The parser will still return the error (with the position information), so that you can eventually use it.</p>
<h2 id="performances">Performances<a class="zola-anchor" href="#performances" aria-label="Anchor link for: performances">§</a>
</h2>
<p>To run the benchmark suite execute the following command.</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">make bench
</code></pre>
<p>All the parsers have the best effort mode on.</p>
<p>On my machine<sup><a href="https://leodido.dev/projects/go-conventionalcommits/#mymachine">1</a></sup>, these are the results for the <code>slim</code> parser with the default - ie., <code>minimal</code>, commit message types.</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">[ok]_minimal______________________________________-12 4876018 242 ns/op 147 B/op 5 allocs/op
[ok]_minimal_with_scope___________________________-12 4258562 284 ns/op 163 B/op 6 allocs/op
[ok]_minimal_breaking_with_scope__________________-12 4176747 288 ns/op 163 B/op 6 allocs/op
[ok]_full_with_50_characters_long_description_____-12 1661618 700 ns/op 288 B/op 10 allocs/op
[no]_empty________________________________________-12 4059327 292 ns/op 112 B/op 3 allocs/op
[no]_type_but_missing_colon_______________________-12 2701904 444 ns/op 200 B/op 6 allocs/op
[no]_type_but_missing_description_________________-12 2207985 539 ns/op 288 B/op 8 allocs/op
[no]_type_and_scope_but_missing_description_______-12 1969390 605 ns/op 312 B/op 10 allocs/op
[no]_breaking_with_type_and_scope_but_missing_desc-12 1978302 606 ns/op 312 B/op 10 allocs/op
[~~]_newline_in_description_______________________-12 2115649 563 ns/op 232 B/op 11 allocs/op
[no]_missing_whitespace_in_description____________-12 1997863 595 ns/op 312 B/op 10 allocs/op
</code></pre>
<p>Using another set of commit message types, for example the <code>conventional</code> one, does not have any noticeable impact on performances, as you can see below.</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">[ok]_minimal______________________________________-12 5297486 228 ns/op 147 B/op 5 allocs/op
[ok]_minimal_with_scope___________________________-12 4498694 267 ns/op 163 B/op 6 allocs/op
[ok]_minimal_breaking_with_scope__________________-12 4431040 273 ns/op 163 B/op 6 allocs/op
[ok]_full_with_50_characters_long_description_____-12 1750111 692 ns/op 288 B/op 10 allocs/op
[no]_empty________________________________________-12 3996532 294 ns/op 112 B/op 3 allocs/op
[no]_type_but_missing_colon_______________________-12 2657913 451 ns/op 200 B/op 6 allocs/op
[no]_type_but_missing_description_________________-12 2172524 553 ns/op 288 B/op 8 allocs/op
[no]_type_and_scope_but_missing_description_______-12 1880526 637 ns/op 312 B/op 10 allocs/op
[no]_breaking_with_type_and_scope_but_missing_desc-12 1879779 635 ns/op 312 B/op 10 allocs/op
[~~]_newline_in_description_______________________-12 2023514 592 ns/op 232 B/op 11 allocs/op
[no]_missing_whitespace_in_description____________-12 1883124 623 ns/op 312 B/op 10 allocs/op
</code></pre>
<p>As you may notice, this library is very fast at what it does.</p>
<p>Parsing a commit goes from taking about the same amount of time (~299ns) the half-life of polonium-212 takes<sup><a href="https://leodido.dev/projects/go-conventionalcommits/#nanosecondwiki">2</a></sup> to less than a microsecond.</p>
<ul>
<li><a name="mymachine">[1]</a>: Intel Core i7-8850H CPU @ 2.60GHz</li>
<li><a name="nanosecondwiki">[2]</a>: <a rel="noopener noreferrer" target="_blank" href="https://en.wikipedia.org/wiki/nanosecond">https://en.wikipedia.org/wiki/nanosecond</a></li>
</ul>
kubectl-trace2018-11-21T00:00:00+00:002018-11-21T00:00:00+00:00https://leodido.dev/projects/kubectl-trace/
<figure class="center">
<img src="https://raw.githubusercontent.com/iovisor/kubectl-trace/master/docs/logo/logo-sm.png" alt="kubectl-trace Logo" />
</figure>
<p><code>kubectl trace</code> is a kubectl plugin that allows you to schedule the execution
of <a rel="noopener noreferrer" target="_blank" href="https://github.com/iovisor/bpftrace">bpftrace</a> programs in your Kubernetes cluster.</p>
<p><img src="https://raw.githubusercontent.com/iovisor/kubectl-trace/master/docs/img/intro.png" alt="Screenshot showing the read.bt program for kubectl-trace" /></p>
<h2 id="installing">Installing<a class="zola-anchor" href="#installing" aria-label="Anchor link for: installing">§</a>
</h2>
<h3 id="krew">Krew<a class="zola-anchor" href="#krew" aria-label="Anchor link for: krew">§</a>
</h3>
<p>You can install <code>kubectl trace</code> using the <a rel="noopener noreferrer" target="_blank" href="https://github.com/kubernetes-sigs/krew">Krew</a>, the package manager for kubectl plugins.</p>
<p>Once you have <a rel="noopener noreferrer" target="_blank" href="https://krew.sigs.k8s.io/docs/user-guide/setup/install/">Krew installed</a> just run:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">kubectl krew install trace
</code></pre>
<p>You’re ready to go!</p>
<h3 id="pre-built-binaries">Pre-built binaries<a class="zola-anchor" href="#pre-built-binaries" aria-label="Anchor link for: pre-built-binaries">§</a>
</h3>
<p>See the <a rel="noopener noreferrer" target="_blank" href="https://github.com/iovisor/kubectl-trace/releases">release</a> page for the full list of pre-built assets.</p>
<p>The commands here show <code>amd64</code> versions, <code>386</code> versions are available in the releases page.</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">curl -L -o kubectl-trace.tar.gz https://github.com/iovisor/kubectl-trace/releases/download/v0.1.0-rc.1/kubectl-trace_0.1.0-rc.1_linux_amd64.tar.gz
tar -xvf kubectl-trace.tar.gz
mv kubectl-trace /usr/local/bin/kubectl-trace
</code></pre>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">curl -L -o kubectl-trace.tar.gz https://github.com/iovisor/kubectl-trace/releases/download/v0.1.0-rc.1/kubectl-trace_0.1.0-rc.1_darwin_amd64.tar.gz
tar -xvf kubectl-trace.tar.gz
mv kubectl-trace /usr/local/bin/kubectl-trace
</code></pre>
<p>In PowerShell v5+</p>
<pre data-lang="powershell" class="language-powershell "><code class="language-powershell" data-lang="powershell">$url = "https://github.com/iovisor/kubectl-trace/releases/download/v0.1.0-rc.1/kubectl-trace_0.1.0-rc.1_windows_amd64.zip"
$output = "$PSScriptRoot\kubectl-trace.zip"
Invoke-WebRequest -Uri $url -OutFile $output
Expand-Archive "$PSScriptRoot\kubectl-trace.zip" -DestinationPath "$PSScriptRoot\kubectl-trace"
</code></pre>
<h3 id="source">Source<a class="zola-anchor" href="#source" aria-label="Anchor link for: source">§</a>
</h3>
<p>Using go modules, you can build kubectl-trace at any git tag:</p>
<pre><code>GO111MODULE=on go get github.com/iovisor/kubectl-trace/cmd/kubectl-trace@latest
</code></pre>
<p>This will download and compile <code>kubectl-trace</code> so that you can use it as a kubectl plugin with <code>kubectl trace</code>, note that you will need to be on a recent version of go which supports go modules.</p>
<p>To keep track of the ref you used to build, you can add an ldflag at build time to set this to match the ref provided to go modules:</p>
<pre><code>> GO111MODULE=on go get -ldflags='-X github.com/iovisor/kubectl-trace/pkg/version.gitCommit=v0.1.2' github.com/iovisor/kubectl-trace/cmd/kubectl-trace@v0.1.2
> $GOHOME/bin/kubectl-trace version
git commit: v0.1.2
build date: 2021-08-10 12:38:37.921341766 -0400 EDT m=+0.034327432
</code></pre>
<h3 id="packages">Packages<a class="zola-anchor" href="#packages" aria-label="Anchor link for: packages">§</a>
</h3>
<p>You can’t find the package for your distro of choice?
You are very welcome and encouraged to create it and then <a rel="noopener noreferrer" target="_blank" href="https://github.com/iovisor/kubectl-trace/issues/new">open an issue</a> to inform us for review.</p>
<h4 id="arch-aur">Arch - AUR<a class="zola-anchor" href="#arch-aur" aria-label="Anchor link for: arch-aur">§</a>
</h4>
<p>The official <a rel="noopener noreferrer" target="_blank" href="https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=kubectl-trace-git">PKGBUILD</a> is on AUR.</p>
<p>If you use <code>yay</code> to manage AUR packages you can do:</p>
<pre><code>yay -S kubectl-trace-git
</code></pre>
<h2 id="architecture">Architecture<a class="zola-anchor" href="#architecture" aria-label="Anchor link for: architecture">§</a>
</h2>
<p>See <a rel="noopener noreferrer" target="_blank" href="https://github.com/iovisor/kubectl-trace/tree/master/docs/architecture.md">architecture.md</a></p>
<h2 id="usage">Usage<a class="zola-anchor" href="#usage" aria-label="Anchor link for: usage">§</a>
</h2>
<p>You don’t need to setup anything on your cluster before using it, please don’t use it already
on a production system, just because this isn’t yet 100% ready.</p>
<h3 id="run-a-program-from-string-literal">Run a program from string literal<a class="zola-anchor" href="#run-a-program-from-string-literal" aria-label="Anchor link for: run-a-program-from-string-literal">§</a>
</h3>
<p>In this case we are running a program that probes a tracepoint
on the node <code>ip-180-12-0-152.ec2.internal</code>.</p>
<pre><code>kubectl trace run ip-180-12-0-152.ec2.internal -e "tracepoint:syscalls:sys_enter_* { @[probe] = count(); }"
</code></pre>
<h3 id="run-a-program-from-file">Run a program from file<a class="zola-anchor" href="#run-a-program-from-file" aria-label="Anchor link for: run-a-program-from-file">§</a>
</h3>
<p>Here we run a program named <code>read.bt</code> against the node <code>ip-180-12-0-152.ec2.internal</code></p>
<pre><code>kubectl trace run ip-180-12-0-152.ec2.internal -f read.bt
</code></pre>
<h3 id="run-a-program-against-a-pod">Run a program against a Pod<a class="zola-anchor" href="#run-a-program-against-a-pod" aria-label="Anchor link for: run-a-program-against-a-pod">§</a>
</h3>
<p><img src="https://raw.githubusercontent.com/iovisor/kubectl-trace/master/docs/img/pod.png" alt="Screenshot showing the read.bt program for kubectl-trace" /></p>
<p>That pod has a Go program in it that is at <code>/caturday</code>, that program has a function called <code>main.counterValue</code> in it that returns an integer
every time it is called.</p>
<p>The purpose of this program is to load an <code>uretprobe</code> on the <code>/caturday</code> binary so that every time the <code>main.counterValue</code> function is called
we get the return value out.</p>
<p>Since <code>kubectl trace</code> for pods is just an helper to resolve the context of a container’s Pod, you will always be in the root namespaces
but in this case you will have a variable <code>$container_pid</code> containing the pid of the root process in that container on the root pid namespace.</p>
<p>What you do then is that you get the <code>/caturday</code> binary via <code>/proc/$container_pid/exe</code>, like this:</p>
<pre><code>kubectl trace run -e 'uretprobe:/proc/$container_pid/exe:"main.counterValue" { printf("%d\n", retval) }' pod/caturday-566d99889-8glv9 -a -n caturday
</code></pre>
<h3 id="running-against-a-pod-vs-against-a-node">Running against a Pod vs against a Node<a class="zola-anchor" href="#running-against-a-pod-vs-against-a-node" aria-label="Anchor link for: running-against-a-pod-vs-against-a-node">§</a>
</h3>
<p>In general, you run kprobes/kretprobes, tracepoints, software, hardware and profile events against nodes using the <code>node/node-name</code> syntax or just use the
node name, node is the default.</p>
<p>When you want to actually probe an userspace program with an uprobe/uretprobe or use an user-level static tracepoint (usdt) your best
bet is to run it against a pod using the <code>pod/pod-name</code> syntax.</p>
<p>It’s always important to remember that running a program against a pod, as of now, is just a facilitator to find the process id for the binary you want to probe
on the root process namespace.</p>
<p>You could do the same thing when running in a Node by knowing the pid of your process yourself after entering in the node via another medium, e.g: ssh.</p>
<p>So, running against a pod <strong>doesn’t mean</strong> that your bpftrace program will be contained in that pod but just that it will pass to your program some
knowledge of the context of a container, in this case only the root process id is supported via the <code>$container_pid</code> variable.</p>
<h3 id="using-a-custom-service-account">Using a custom service account<a class="zola-anchor" href="#using-a-custom-service-account" aria-label="Anchor link for: using-a-custom-service-account">§</a>
</h3>
<p>By default <code>kubectl trace</code> will use the <code>default</code> service account in the target namespace (that is also <code>default</code>), to schedule the pods needed for your bpftrace program.</p>
<p>If you need to pass a service account you can use the <code>--serviceaccount</code> flag.</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">kubectl trace run --serviceaccount=kubectltrace ip-180-12-0-152.ec2.internal -f read.bt
</code></pre>
<h3 id="executing-in-a-cluster-using-pod-security-policies">Executing in a cluster using Pod Security Policies<a class="zola-anchor" href="#executing-in-a-cluster-using-pod-security-policies" aria-label="Anchor link for: executing-in-a-cluster-using-pod-security-policies">§</a>
</h3>
<p>If your cluster has pod security policies you will need to make so that <code>kubectl trace</code> can
use a service account that can run privileged containers.</p>
<p>That service account, then will need to be in a group that uses the proper privileged <code>PodSecurityPolicy</code>.</p>
<p>First, create the service account that you will use with <code>kubectl trace</code>,
you can use a different namespace other than <code>default</code>, just remember to pass that namespace to the <code>run</code> command when you will use <code>kubectl trace</code>:</p>
<pre data-lang="yaml" class="language-yaml "><code class="language-yaml" data-lang="yaml">apiVersion: v1
kind: ServiceAccount
metadata:
name: kubectltrace
namespace: default
</code></pre>
<p>Now that we have a <code>kubectltrace</code> service account let’s create a Pod Security Policy:</p>
<pre data-lang="yaml" class="language-yaml "><code class="language-yaml" data-lang="yaml">apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: kubectltrace
spec:
fsGroup:
rule: RunAsAny
privileged: true
runAsUser:
rule: RunAsAny
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- '*'
allowedCapabilities:
- '*'
hostPID: true
hostIPC: true
hostNetwork: true
hostPorts:
- min: 1
max: 65536
</code></pre>
<p>Ok, this <code>PodSecurityPolicy</code> will allow users assigned to it to run privileged containers,
<code>kubectl trace</code> needs that because of the extended privileges eBPF programs need to run with
to trace your kernel and programs running in it.</p>
<p>Now with a <code>ClusterRoleBinding</code> you bind the <code>ClusterRole</code> with the <code>ServiceAccount</code>, so that
they can work together with the <code>PodSecurityPolicy</code> we just created.</p>
<p>You can change the <code>namespace: default</code> here if you created the service account in a namespace other than <code>default</code>.</p>
<pre data-lang="yaml" class="language-yaml "><code class="language-yaml" data-lang="yaml">apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubectltrace-psp
rules:
- apiGroups:
- policy
resources:
- podsecuritypolicies
resourceNames:
- kubectltrace
verbs:
- use
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kubectltrace-psp
subjects:
- kind: ServiceAccount
name: kubectltrace
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubectltrace-psp
</code></pre>
<p>OK! Now that we are all set we can just run the program by specifying the service account
we just created and it will use our pod security policy!</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">kubectl trace run --serviceaccount=kubectltrace ip-180-12-0-152.ec2.internal -f read.bt
</code></pre>
<p>If you used a different namespace other than default for your service account, you will want to specify the namespace too, like this:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">kubectl trace run --namespace=mynamespace --serviceaccount=kubectltrace ip-180-12-0-152.ec2.internal -f read.bt
</code></pre>
<h3 id="using-a-patch-to-customize-the-trace-job">Using a patch to customize the trace job<a class="zola-anchor" href="#using-a-patch-to-customize-the-trace-job" aria-label="Anchor link for: using-a-patch-to-customize-the-trace-job">§</a>
</h3>
<p>There may be times when you need to customize the job descriptor that kubectl-trace generates. You can provide a patch file that will modify any of the job’s attributes before it executes on the cluster.</p>
<p>The <code>--patch</code> and <code>--patch-type</code> arguments to the <code>run</code> command specify your patch file’s location and merge strategy:</p>
<ul>
<li><code>--patch</code> - sets the path to a YAML or JSON file containing your patch.</li>
<li><code>--patch-type</code> - sets the strategy that will be used to modify the job descriptor.</li>
</ul>
<p>The supported patch strategies are the same as those used by Kubernetes to support <a rel="noopener noreferrer" target="_blank" href="https://v1-17.docs.kubernetes.io/docs/tasks/run-application/update-api-object-kubectl-patch/#use-a-json-merge-patch-to-update-a-deployment">in-place API object updates</a>.</p>
<p>These 3 patch strategies are:</p>
<ul>
<li><code>json</code> - Sets the <a rel="noopener noreferrer" target="_blank" href="http://jsonpatch.com/">JSON patch</a> strategy (see <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc6902">RFC 6209</a>).</li>
<li><code>merge</code> - Sets the <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc7396">JSON merge patch</a> strategy.</li>
<li><code>strategic</code> - JSON strategic merge patch is like the “JSON merge patch” but with different array handling (see <a rel="noopener noreferrer" target="_blank" href="https://v1-17.docs.kubernetes.io/docs/tasks/run-application/update-api-object-kubectl-patch/#use-a-json-merge-patch-to-update-a-deployment">Kubernetes strategic merge</a> for more).</li>
</ul>
<p>A cluster administrator may have set strict resource limits that conflict with the defaults used by <code>kubectl-trace</code>, preventing your job from executing. With a patch you can adjust a job’s resource limits to match your cluster’s config.</p>
<p>Below is an example of a YAML patch which uses the <code>json</code> strategy (“JSON patch”). This strategy consists of a list of operations (add, replace, remove), a path which references a location in the document, and an optional value (to add or replace).</p>
<p>The patch below replaces the first container’s resources section, in order to increase both the request and limit values for cpu and memory:</p>
<pre data-lang="yaml" class="language-yaml "><code class="language-yaml" data-lang="yaml"># mypatch.yaml
- op: replace
path: /spec/template/spec/containers/0/resources
value:
limits:
cpu: 2
memory: 500Mi
requests:
cpu: 2
memory: 500Mi
</code></pre>
<p>We can now run the job using our patch:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">kubectl trace run ip-180-12-0-152.ec2.internal -f read.bt --patch mypatch.yaml --patch-type json
</code></pre>
<p>The following JSON format patch adds a <code>BPFTRACE_STRLEN</code> environment variable to the first container. The variable increases <code>bpftrace</code>’s string length limit from 64 to 128:</p>
<pre data-lang="json" class="language-json "><code class="language-json" data-lang="json">[
{
"op": "add",
"path": "/spec/template/spec/containers/0/env",
"value": [{ "name": "BPFTRACE_STRLEN", "value": "128" }]
}
]
</code></pre>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">kubectl trace run ip-180-12-0-152.ec2.internal -f read.bt --patch mypatch.json --patch-type json
</code></pre>
<h3 id="more-bpftrace-programs">More bpftrace programs<a class="zola-anchor" href="#more-bpftrace-programs" aria-label="Anchor link for: more-bpftrace-programs">§</a>
</h3>
<p>Need more programs? Look <a rel="noopener noreferrer" target="_blank" href="https://github.com/iovisor/bpftrace/tree/master/tools">here</a>.</p>
<h2 id="contributing">Contributing<a class="zola-anchor" href="#contributing" aria-label="Anchor link for: contributing">§</a>
</h2>
<p>Already pumped up to commit some code? Here are some resources to join the
discussions in the <a rel="noopener noreferrer" target="_blank" href="https://www.iovisor.org/">IOVisor</a> community and see
what you want to work on.</p>
<ul>
<li><em>Mailing List:</em> http://lists.iovisor.org/mailman/listinfo/iovisor-dev</li>
<li><em>IRC:</em> #iovisor at irc.oftc.net</li>
<li><em>Slack</em> #kubectl-trace in the <a rel="noopener noreferrer" target="_blank" href="http://kubernetes.slack.com/">Kubernetes Slack</a></li>
<li><em>Kubectl Trace Issue Tracker:</em> <a rel="noopener noreferrer" target="_blank" href="https://github.com/iovisor/kubectl-trace/issues">Github Issues</a></li>
</ul>
<p>Special thanks to <a rel="noopener noreferrer" target="_blank" href="https://github.com/RamonGilabert">Ramon Gilabert</a> for the logo.</p>
go-syslog2018-03-28T00:00:00+00:002018-03-28T00:00:00+00:00https://leodido.dev/projects/go-syslog/<p>To wrap up, this package provides:</p>
<ul>
<li>an <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/rfc5424">RFC5424-compliant parser and builder</a></li>
<li>an <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/rfc3164">RFC3164-compliant parser</a> - ie., BSD-syslog messages</li>
<li>a parser that works on streams for syslog with <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc5425#section-4.3">octet counting</a> framing technique, see <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/octetcounting">octetcounting</a></li>
<li>a parser that works on streams for syslog with <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc6587#section-3.4.2">non-transparent</a> framing technique, see <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/nontransparent">nontransparent</a></li>
</ul>
<p>This library provides the pieces to parse Syslog messages transported following various RFCs.</p>
<p>For example:</p>
<ul>
<li>TLS with octet count (<a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc5425">RFC5425</a>)</li>
<li>TCP with non-transparent framing or with octet count (<a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc6587">RFC 6587</a>)</li>
<li>UDP carrying one message per packet (<a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc5426">RFC5426</a>)</li>
</ul>
<h2 id="installation">Installation<a class="zola-anchor" href="#installation" aria-label="Anchor link for: installation">§</a>
</h2>
<pre><code>go get github.com/influxdata/go-syslog/v3
</code></pre>
<h2 id="docs">Docs<a class="zola-anchor" href="#docs" aria-label="Anchor link for: docs">§</a>
</h2>
<p>The <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/docs/">docs</a> directory contains <code>.dot</code> files representing the finite-state machines (FSMs) implementing the syslog parsers and transports.</p>
<h2 id="usage">Usage<a class="zola-anchor" href="#usage" aria-label="Anchor link for: usage">§</a>
</h2>
<p>Suppose you want to parse a given sequence of bytes as a RFC5424 message.</p>
<p><em>Notice that the same interface applies for RFC3164. But you can always take a look at the <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/rfc3164/example_test.go">examples file</a>.</em></p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">i := []byte(`<165>4 2018-10-11T22:14:15.003Z mymach.it e - 1 [ex@32473 iut="3"] An application event log entry...`)
p := rfc5424.NewParser()
m, e := p.Parse(i)
</code></pre>
<p>This results in <code>m</code> being equal to:</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">// (*rfc5424.SyslogMessage)({
// Base: (syslog.Base) {
// Facility: (*uint8)(20),
// Severity: (*uint8)(5),
// Priority: (*uint8)(165),
// Timestamp: (*time.Time)(2018-10-11 22:14:15.003 +0000 UTC),
// Hostname: (*string)((len=9) "mymach.it"),
// Appname: (*string)((len=1) "e"),
// ProcID: (*string)(<nil>),
// MsgID: (*string)((len=1) "1"),
// Message: (*string)((len=33) "An application event log entry...")
// },
// Version: (uint16) 4,
// StructuredData: (*map[string]map[string]string)((len=1) {
// (string) (len=8) "ex@32473": (map[string]string) (len=1) {
// (string) (len=3) "iut": (string) (len=1) "3"
// }
// })
// })
</code></pre>
<p>And <code>e</code> being equal to <code>nil</code> since the <code>i</code> byte slice contains a perfectly valid RFC5424 message.</p>
<h3 id="best-effort-mode">Best effort mode<a class="zola-anchor" href="#best-effort-mode" aria-label="Anchor link for: best-effort-mode">§</a>
</h3>
<p>RFC5424 parser has the ability to perform partial matches (until it can).</p>
<p>With this mode enabled, when the parsing process errors out it returns the message collected until that position, and the error that caused the parser to stop.</p>
<p>Notice that in this modality the output is returned <em>iff</em> it represents a minimally valid message - ie., a message containing almost a priority field in <code>[1,191]</code> within angular brackets, followed by a version in <code>]0,999]</code> (in the case of RFC5424).</p>
<p>Let’s look at an example.</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">i := []byte("<1>1 A - - - - - -")
p := NewParser(WithBestEffort())
m, e := p.Parse(i)
</code></pre>
<p>This results in <code>m</code> being equal to the following <code>SyslogMessage</code> instance.</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">// (*rfc5424.SyslogMessage)({
// Base: (syslog.Base) {
// Facility: (*uint8)(0),
// Severity: (*uint8)(1),
// Priority: (*uint8)(1),
// Timestamp: (*time.Time)(<nil>),
// Hostname: (*string)(<nil>),
// Appname: (*string)(<nil>),
// ProcID: (*string)(<nil>),
// MsgID: (*string)(<nil>),
// Message: (*string)(<nil>)
// },
// Version: (uint16) 1,
// StructuredData: (*map[string]map[string]string)(<nil>)
// })
</code></pre>
<p>And, at the same time, in <code>e</code> reporting the error that actually stopped the parser.</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">// expecting a RFC3339MICRO timestamp or a nil value [col 5]
</code></pre>
<p>Both <code>m</code> and <code>e</code> have a value since at the column the parser stopped it already was able to construct a minimally valid RFC5424 <code>SyslogMessage</code>.</p>
<h3 id="builder">Builder<a class="zola-anchor" href="#builder" aria-label="Anchor link for: builder">§</a>
</h3>
<p>This library also provides a builder to construct valid syslog messages.</p>
<p>Notice that its API ignores input values that does not match the grammar.</p>
<p>Let’s have a look to an example.</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">msg := &rfc5424.SyslogMessage{}
msg.SetTimestamp("not a RFC3339MICRO timestamp")
msg.Valid() // Not yet a valid message (try msg.Valid())
msg.SetPriority(191)
msg.SetVersion(1)
msg.Valid() // Now it is minimally valid
</code></pre>
<p>Printing <code>msg</code> you will verify it contains a <code>nil</code> timestamp (since an invalid one has been given).</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">// (*rfc5424.SyslogMessage)({
// Base: (syslog.Base) {
// Facility: (*uint8)(23),
// Severity: (*uint8)(7),
// Priority: (*uint8)(191),
// Timestamp: (*time.Time)(<nil>),
// Hostname: (*string)(<nil>),
// Appname: (*string)(<nil>),
// ProcID: (*string)(<nil>),
// MsgID: (*string)(<nil>),
// Message: (*string)(<nil>)
// },
// Version: (uint16) 1,
// StructuredData: (*map[string]map[string]string)(<nil>)
// })
</code></pre>
<p>Finally you can serialize the message into a string.</p>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">str, _ := msg.String()
// <191>1 - - - - - -
</code></pre>
<h2 id="message-transfer">Message transfer<a class="zola-anchor" href="#message-transfer" aria-label="Anchor link for: message-transfer">§</a>
</h2>
<p>Excluding encapsulating one message for packet in packet protocols there are two ways to transfer syslog messages over streams.</p>
<p>The older - ie., the <strong>non-transparent</strong> framing - and the newer one - ie., the <strong>octet counting</strong> framing - which is reliable and has not been seen to cause problems noted with the non-transparent one.</p>
<p>This library provide stream parsers for both.</p>
<h3 id="octet-counting">Octet counting<a class="zola-anchor" href="#octet-counting" aria-label="Anchor link for: octet-counting">§</a>
</h3>
<p>In short, <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc5425#section-4.3">RFC5425</a> and <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc6587">RFC6587</a>, aside from the protocol considerations, describe a <strong>transparent framing</strong> technique for Syslog messages that uses the <strong>octect counting</strong> technique - ie., the message length of the incoming message.</p>
<p>Each Syslog message is sent with a prefix representing the number of bytes it is made of.</p>
<p>The <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/octetcounting">octecounting package</a> parses messages stream following such rule.</p>
<p>To quickly understand how to use it please have a look at the <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/octetcounting/example_test.go">example file</a>.</p>
<h3 id="non-transparent">Non transparent<a class="zola-anchor" href="#non-transparent" aria-label="Anchor link for: non-transparent">§</a>
</h3>
<p>The <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc6587#section-3.4.2">RFC6587</a> also describes the <strong>non-transparent framing</strong> transport of syslog messages.</p>
<p>In such case the messages are separated by a trailer, usually a line feed.</p>
<p>The <a href="./nontransparent">nontransparent package</a> parses message stream following such <a rel="noopener noreferrer" target="_blank" href="https://tools.ietf.org/html/rfc6587#section-3.4.2">technique</a>.</p>
<p>To quickly understand how to use it please have a look at the <a rel="noopener noreferrer" target="_blank" href="https://github.com/influxdata/go-syslog/tree/develop/nontransparent/example_test.go">example file</a>.</p>
<p>Things we do not support:</p>
<ul>
<li>trailers other than <code>LF</code> or <code>NUL</code></li>
<li>trailers which length is greater than 1 byte</li>
<li>trailer change on a frame-by-frame basis</li>
</ul>
<h2 id="performances">Performances<a class="zola-anchor" href="#performances" aria-label="Anchor link for: performances">§</a>
</h2>
<p>To run the benchmark execute the following command.</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">make bench
</code></pre>
<p>On my machine<sup><a href="https://leodido.dev/projects/go-syslog/#mymachine">1</a></sup> these are the results obtained paring RFC5424 syslog messages with best effort mode on.</p>
<pre><code>[no]_empty_input__________________________________ 4524100 274 ns/op 272 B/op 4 allocs/op
[no]_multiple_syslog_messages_on_multiple_lines___ 3039513 361 ns/op 288 B/op 8 allocs/op
[no]_impossible_timestamp_________________________ 1244562 951 ns/op 512 B/op 11 allocs/op
[no]_malformed_structured_data____________________ 2389249 512 ns/op 512 B/op 9 allocs/op
[no]_with_duplicated_structured_data_id___________ 1000000 1183 ns/op 712 B/op 17 allocs/op
[ok]_minimal______________________________________ 6876235 178 ns/op 227 B/op 5 allocs/op
[ok]_average_message______________________________ 730473 1653 ns/op 1520 B/op 24 allocs/op
[ok]_complicated_message__________________________ 908776 1344 ns/op 1264 B/op 24 allocs/op
[ok]_very_long_message____________________________ 392737 3114 ns/op 2448 B/op 25 allocs/op
[ok]_all_max_length_and_complete__________________ 510740 2431 ns/op 1872 B/op 28 allocs/op
[ok]_all_max_length_except_structured_data_and_mes 755124 1593 ns/op 867 B/op 13 allocs/op
[ok]_minimal_with_message_containing_newline______ 6142984 199 ns/op 230 B/op 6 allocs/op
[ok]_w/o_procid,_w/o_structured_data,_with_message 1670286 732 ns/op 348 B/op 10 allocs/op
[ok]_minimal_with_UTF-8_message___________________ 3013480 407 ns/op 339 B/op 6 allocs/op
[ok]_minimal_with_UTF-8_message_starting_with_BOM_ 2926410 423 ns/op 355 B/op 6 allocs/op
[ok]_with_structured_data_id,_w/o_structured_data_ 1558971 814 ns/op 570 B/op 11 allocs/op
[ok]_with_multiple_structured_data________________ 1000000 1243 ns/op 1205 B/op 16 allocs/op
[ok]_with_escaped_backslash_within_structured_data 1000000 1025 ns/op 896 B/op 17 allocs/op
[ok]_with_UTF-8_structured_data_param_value,_with_ 1000000 1241 ns/op 1034 B/op 19 allocs/op
</code></pre>
<p>As you can see it takes:</p>
<ul>
<li>
<p>~250ns to parse the smallest legal message</p>
</li>
<li>
<p>less than 2µs to parse an average legal message</p>
</li>
<li>
<p>~3µs to parse a very long legal message</p>
</li>
</ul>
<p>Other RFC5424 implementations, like this <a rel="noopener noreferrer" target="_blank" href="https://github.com/roguelazer/rust-syslog-rfc5424">one</a> in Rust, spend 8µs to parse an average legal message.</p>
<p><em>TBD: comparison against other Go parsers</em>.</p>
<ul>
<li><a name="mymachine">[1]</a>: Intel Core i7-8850H CPU @ 2.60GHz</li>
</ul>
go-urn2017-11-21T00:00:00+00:002017-11-21T00:00:00+00:00https://leodido.dev/projects/go-urn/<h2 id="installation">Installation<a class="zola-anchor" href="#installation" aria-label="Anchor link for: installation">§</a>
</h2>
<pre><code>go get github.com/leodido/go-urn
</code></pre>
<h2 id="performances">Performances<a class="zola-anchor" href="#performances" aria-label="Anchor link for: performances">§</a>
</h2>
<p>This implementation results to be really fast.</p>
<p>Usually below ½ microsecond on my machine<sup><a href="https://leodido.dev/projects/go-urn/#mymachine">1</a></sup>.</p>
<p>Notice it also performs, while parsing:</p>
<ol>
<li>fine-grained and informative erroring</li>
<li>specific-string normalization</li>
</ol>
<pre><code>ok/00/urn:a:b______________________________________/-4 20000000 265 ns/op 182 B/op 6 allocs/op
ok/01/URN:foo:a123,456_____________________________/-4 30000000 296 ns/op 200 B/op 6 allocs/op
ok/02/urn:foo:a123%2c456___________________________/-4 20000000 331 ns/op 208 B/op 6 allocs/op
ok/03/urn:ietf:params:scim:schemas:core:2.0:User___/-4 20000000 430 ns/op 280 B/op 6 allocs/op
ok/04/urn:ietf:params:scim:schemas:extension:enterp/-4 20000000 411 ns/op 312 B/op 6 allocs/op
ok/05/urn:ietf:params:scim:schemas:extension:enterp/-4 20000000 472 ns/op 344 B/op 6 allocs/op
ok/06/urn:burnout:nss______________________________/-4 30000000 257 ns/op 192 B/op 6 allocs/op
ok/07/urn:abcdefghilmnopqrstuvzabcdefghilm:x_______/-4 20000000 375 ns/op 213 B/op 6 allocs/op
ok/08/urn:urnurnurn:urn____________________________/-4 30000000 265 ns/op 197 B/op 6 allocs/op
ok/09/urn:ciao:@!=%2c(xyz)+a,b.*@g=$_'_____________/-4 20000000 307 ns/op 248 B/op 6 allocs/op
ok/10/URN:x:abc%1dz%2f%3az_________________________/-4 30000000 259 ns/op 212 B/op 6 allocs/op
no/11/URN:-xxx:x___________________________________/-4 20000000 445 ns/op 320 B/op 6 allocs/op
no/12/urn::colon:nss_______________________________/-4 20000000 461 ns/op 320 B/op 6 allocs/op
no/13/urn:abcdefghilmnopqrstuvzabcdefghilmn:specifi/-4 10000000 660 ns/op 320 B/op 6 allocs/op
no/14/URN:a!?:x____________________________________/-4 20000000 507 ns/op 320 B/op 6 allocs/op
no/15/urn:urn:NSS__________________________________/-4 20000000 429 ns/op 288 B/op 6 allocs/op
no/16/urn:white_space:NSS__________________________/-4 20000000 482 ns/op 320 B/op 6 allocs/op
no/17/urn:concat:no_spaces_________________________/-4 20000000 539 ns/op 328 B/op 7 allocs/op
no/18/urn:a:/______________________________________/-4 20000000 470 ns/op 320 B/op 7 allocs/op
no/19/urn:UrN:NSS__________________________________/-4 20000000 399 ns/op 288 B/op 6 allocs/op
</code></pre>
<ul>
<li><a name="mymachine">[1]</a>: Intel Core i7-7600U CPU @ 2.80GHz</li>
</ul>
<h2 id="example">Example<a class="zola-anchor" href="#example" aria-label="Anchor link for: example">§</a>
</h2>
<pre data-lang="go" class="language-go "><code class="language-go" data-lang="go">package main
import (
"fmt"
"github.com/leodido/go-urn"
)
func main() {
var uid = "URN:foo:a123,456"
u, ok := urn.Parse([]byte(uid))
if !ok {
panic("error parsing urn")
}
fmt.Println(u.ID)
fmt.Println(u.SS)
// Output:
// foo
// a123,456
}
</code></pre>