Rebase all SUSE changes onto upstream v0.76.6#147
Open
djoreilly wants to merge 142 commits into
Open
Conversation
This allows using a magefiles/ directory for target definitions.
When building in a multiarchitecture build system, it's helpful to have the same commands for every architecture. This commit makes the Linux and LinuxBare targets default to the host architecture and creates new LinuxAmd64 and LinuxBareAmd64 targets to target that architecture specifically.
With the journald code added, systemd-dev is a build dependency.
We don't need to build the GUI or the binary on every architecture before running the testcases.
Originally by Nikolay Borisov <nborisov@suse.com>
Newer Linux systems use the systemd journal facility for logging instead of the classic syslogd logging daemon. While it's possible to configure journald to forward events to e.g. rsyslogd, doing so requires a configuration change on the endpoint that would only be necessary to accommodate the syslog parser. This commit adds the ability to parse the systemd journal using the native libsystem-journal.so shared library. It adds watcher and scanner parsers identical to the syslog versions. One difference is that since the underlying library handles opening and tracking the journal files, there is no facility to subscribe to specific journal files. The support is also limited to the system journal, but that should be sufficient for Velociraptor. It is expected that queries against these parsers filter on systemd unit (or other fields) as desired rather than implementing those filters in the parser itself.
This adds an additional query to the Linux.Syslog.SSHLogin artifact to iterate over the systemd journal. The default systemd unit is sshd.service but it can be configured differently. The Grok query is reused, making the syslog-specific portions of the string optional.
This adds an additional query to the Linux.Events.SSHLogin artifact to iterate over the systemd journal. The default systemd unit is sshd.service but it can be configured differently. The Grok query is reused, making the syslog-specific portions of the string optional. This also adds a MONITORING_DAILY report since the default query only shows the _first_ 100 events, which is likely not what anybody wants.
The SSHLogin artifacts as shipped with Velociraptor are inconsistent with one another. These artifacts resolve the inconsistencies without breaking existing VRR installs.
When events are received by the journal from the kernel they do not have a _SYSTEMD_UNIT key. This results in noisy error messages in the log. We can avoid this by requiring that _TRANSPORT != 'kernel'.
We're about to add several plugins that operate using BPF. This package adds some helpers that will be shared among them. We use the libbpfgo module to implement the BPF interface. The libbpfgo project wants to build its own libbpf and statically link with it. In order to build the embedded libbpf, we need to incorporate libbpfgo as a submodule. There are external requirements to build the BPF programs: - bpftool (unless the builder provides a pregenerated vmlinux.h) - clang >= 10 - llvm-strip Since these external requirements are new, the BPF plugins will only be enabled if the above requirements are met. If they are unmet, a message will be printed informing the user about the missing tools and the build will continue. The plugins can be made mandatory by setting the environment variable: BUILD_BPF_PLUGINS=1. Cross-builds of BPF modules are unsupported with this iteration. Co-authored-by: Jeff Mahoney <jeffm@suse.com>
The plugin utilizes the ebpf infrastructure to load a small program which hooks into tcp_connect/tcp_accept code paths of the kernel and print information about the connection source/dest ports/ip addresses, PID/UID of the process initiating the connection, ip protocol version, timestamp, and the direction of the connection. The Linux networking subsystem supports an xt_AUDIT target that, on the surface, looks like it would serve a similar purpose but it provides no information about user or pid. Co-authored-by: Jeff Mahoney <jeffm@suse.com>
It works by creating a raw socket and attaching an ebpf program to filter only incoming dns packets. That way replies can be parsed. Signed-off-by: Nikolay Borisov <nborisov@suse.com> Co-authored-by: Jeff Mahoney <jeffm@suse.com>
…butes It hooks into do_vfs_ioctl and also performs some of the checks that the generic code would perform in order to ensure we are triggering the ebpf kprobe on a fs which indeed supports the given operations. While at it remove the vmlinux.h files that comes with tcpsnoop and instead rely on the one provided by libbpf. Co-authored-by: Jeff Mahoney <jeffm@suse.com>
The test builds use Ubuntu containers and Ubuntu tries _very_ hard to match the kernel-related tools with the kernel version. It's much easier to just provide vmlinux.h.
Add artifact collects user group changes from auditd events, by monitoring updates made to either /etc/group or /etc/nsswitch.conf.
The file-based ring buffer implementation works well for generic data buffering, but there's no need to take a byte stream, turn it into an ordered dict and then marshal it to json to use it. This commit adds EnqueueSerialized and LeaseSerialized methods to allow direct queuing of bytestreams.
…nterface The previous commit allowed the FileBasedRingBuffer to use direct byte sequences. This adds the listener to use it.
In 2020, the upstream go-libaudit project moved to v2, changing the module path to github.com/elastic/go-libaudit/v2. Upcoming changes to the audit plugin to improve scalability and allow individual rule deletion require some changes that are pending upstream acceptance: - Reporting rules that were missing when the rule is deleted - Not assuming that no syscalls means all syscalls when building rules - Support for removing individual rules - Adding a new AuditMessage.Parse method to allow the caller to manage the lifecycle of the AuditMessage - Message lifecycle For now, we have a replace rule in effect that points to the github.com/jeffmahoney/go-libaudit/v2 repository until those changes land. Side effects: watch_auditd and parse_auditd must now be Linux-only due to the use of unix.SignalName in go-libaudit/v2. I doubt this will have much of an impact since parsing Linux kernel audit logs in an artifact on non-Linux systems seems pretty unlikely.
The audit plugin doesn't share anything among invocations, which makes sense given the historical model of preconfiguring of the rules, subscribing to the firehose, and sorting out the events later. The problem with that architecture is that if anything external happens with the audit subsystem -- even as simple as restarting the audit daemon -- the audit rules are lost and no more events will be generated. By specifying which rules a particular artifact depends on in the arguments, we can be certain that the rules required will always be in effect, can log when they are not, and restore them automatically. Since multiple invocations can ultimately require the same audit rules, management of those rules must be centralized and reference counted so they are only removed when the last plugin using a rule exits. There are a few changes required to make this go: - We need a separate command-and-control audit client to do the configuration, checking, and fixing - We need a goroutine that periodically checks to ensure that the rules are correct. This is implemented as something that sits idle waiting for configuration events and then batches them until 500ms has passed since receipt of the last one. The reason for this is that e.g. "auditctl -D" will generate an event for every single rule removed and we don't need to check the rules every time. - Each invocation will receive a copy of each event and also log messages that apply to the invocation as needed. - The lifetime of the shared services is tied to the listeners. Once the last listener exits, the shared services shutdown. If new invocations come in, a new instance will be started up. - In order to not conflict with auditd, we must use the multicast interface for receiving messages. The kernel does *not* queue and retry multicast audit messages so we need to be very careful to avoid dropping events. - In order to avoid dropping events on very busy systems, we increase the default socket receive buffer to 512k and allow it to scale up as needed in response to ENOBUFS from the kernel. - We use a separate buffer for each event received so that the reception of the messages and processing of events can be decoupled. - These buffers are kept in a pool to avoid constant allocation and freeing. - We use a FileBytesRingBuffer to queue events so that we don't end up pinning lots of memory if events get backlogged. - To aid in debugging and monitoring, there are statistics gathered. A global variable can be used to enable printing the statistics to the client terminal every 5 seconds.
Now that the audit plugin monitors the rules, we can specify the required rules in the plugin call instead of calling auditctl.
The cronsnoop plugin uses fsnotify to create events when cron jobs are added/removed/changed. Co-authored-by: Jeff Mahoney <jeffm@suse.com>
This needs to be a separate source since the Windows shell artifacts and Linux.Sys.Bash present the same information in different ways.
This commit adds the rpm() vql plugin that will return all installed RPM packages on the system. Tags can be specified using the tags parameter. Alternate paths to the directory containing the database files can be specified using the paths parameter.
When we use Velociraptor on systems with a big fork/exec workload, use system call auditing, and hash the executable, we end up with pinned CPUs for obvious reasons. There's no value in hashing the same file over and over if we know it hasn't changed. This commit adds an LRU cache for the hash results. It uses the dev number, inode number, mtime, and ctime to ensure that the file hasn't been modified. A race does exist betewen stat and open, but for this use case a bigger race exists between exec and the hash being requested.
To simplify and be consistent with all other plugins which leave the hashing to VQL in the artifact. Also the VQL hash function has caching and our other artifacts also give the SHA1.
Deferred calls were used to close files from a loop, but these won't run until the function returns. Move the whole event parsing into its own function and use os.Stat() instead. Also there is no need to convert the timestamp to a string.
Some errors logs where missing context making it difficult to know which plugin or where in the plugin the error came from.
Also there is no need to convert time to a string or have json tags.
When the client event table changes, Velociraptor cancels each plugin's context and expects it to shut itself down. Then it starts them again if they are still needed in the new table. The bpf based plugins were not stopping and Velociraptor was starting new goroutines for them, leaving the old ones running and leaking resources forever. This commit changes the bpf plugins so they now exit when the context is cancelled. Velociraptor does not wait until the plugins are fully stopped before starting new instances and the concurrent loading and unloading of BPF modules is problematic. To ensure only one BPF module load or unload operation is happening at any one time, this commit introduces a publish/subscribe service.
This plugin uses eBPF and has less performance overhead than using the audit subsystem to capture execve system calls. Also it does not drop events under heavy load conditions. It requires linux 5.8+ for eBPF ringbuffer support - artifacts should check that before calling it. It uses tracepoint/sched/sched_process_exec because tracepoint/syscalls/sys_enter_execve is not reliable bpftrace/bpftrace#2523 (comment)
The execsnoop plugin can be used if it's in the binary and the kernel is 5.8+ required for eBPF ringbuffer support. Otherwise the audit plugin will be used as before.
If more than one instance of the audit plugin is called with watch rules for the same path but each using different keys, e.g.: SELECT * FROM audit(rules=["-w /etc/passwd -k key1"]) WHERE "key1" IN Tags SELECT * FROM audit(rules=["-w /etc/passwd -k key2"]) WHERE "key2" IN Tags Then it turns out that each event will only be sent to one of the instances. Fix by using the same key for the same paths across instances. The audit plugin reference counts audit rules and duplicates are not a problem.
Update our fork of go-rpmdb to fix a panic when modernc.org/libc tries to read a badly formatted /etc/services: https://gitlab.com/cznic/libc/-/issues/25
This artifact sends events from configured client event artifacts to
the local rsyslog listening with the imuxsock input module on the
specified unix domain socket.
The plugin uses a file backed ring buffer when needed and should be able
to survive short outages of the rsyslog daemon (like restarts) without
losing events - up to 1GB of events.
Tested with this rsyslog config:
input(type="imuxsock" socket="/tmp/velo-socket" ruleset="velo-ruleset")
ruleset(name="velo-ruleset") {
action(type="omfile" file="/tmp/velo.log")
}
The SHA256 is sufficient for Treat Intelligence identification. Also name the column consistenly as "SHA256".
Updates each event query to include the machine-id for SENS-149. A materialized call is used to read /etc/machine-id, so it's only done once per query.
Sometimes grok() fails to parse the output of systemctl and the logs get spammed with "symbol not found in scope" error messages. Fix by not referencing the expected symbols in the serviceDetails() function as there is no need. They are referenced from a dict in the main SELECT and get the value of null if they don't exist without any error messages.
This reduces the binary size by ~15MB.
Prior to v0.74 the VQL array() function would flatten a subquery. This is documented and some of our artifacts depended on this behaviour. https://github.com/Velocidex/velociraptor/blob/v0.7.0/docs/references/vql.yaml#L110 In v0.74 this was changed and now a different syntax is needed to get the same Velocidex/velociraptor@7652599 Our artifacts need to work with a mix of clients versions, some have the old behaviour and some have the new, and they all get the same artifacts from the server. Unfortunately the version of array() was not bumped with this breaking change, so version() can't be used in an if() statement. This fix uses python with the starl() plugin instead of array(), as a common syntax that works with all versions could not be found.
We were parsing the output of `systemctl show` with grok(), but the order of properties in the output appears to be random and can change with systemd version. Fix by simply adding the key/value pairs to a dict.
Our original version depended on the upstream Linux.Ssh.AuthorizedKeys artifact, but it changed significantly and ours no longer works with it. Fix by copy/pasting the parts that parse the authorized_keys files. Having no dependency is less brittle in the long term.
This also updates github.com/ProtonMail/gopenpgp
Our build rpm spec runs a sed command to change the VERSION constant to a format semver can't parse and the build fails. As we don't need these offline dependencies we can just skip creating them.
djoreilly
force-pushed
the
sensor-base-0.76.6
branch
5 times, most recently
from
41a6561 to
ed56a51
Compare
- update ubuntu and go versions - don't specify clang version - remove windows
djoreilly
force-pushed
the
sensor-base-0.76.6
branch
from
ed56a51 to
7be3879
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebase commits from sensor-base-0.7.0 onto upstream v0.76.6