1 .. SPDX-License-Identifier: GPL-2.0
7 DAMON provides below interfaces for different users.
9 - *DAMON user space tool.*
10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as
11 system administrators who want a just-working human-friendly interface.
12 Using this, users can use the DAMON’s major features in a human-friendly way.
13 It may not be highly tuned for special cases, though. It supports both
14 virtual and physical address spaces monitoring. For more detail, please
15 refer to its `usage document
16 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_.
18 :ref:`This <sysfs_interface>` is for privileged user space programmers who
19 want more optimized use of DAMON. Using this, users can use DAMON’s major
20 features by reading from and writing to special sysfs files. Therefore,
21 you can write and use your personalized DAMON sysfs wrapper programs that
22 reads/writes the sysfs files instead of you. The `DAMON user space tool
23 <https://github.com/awslabs/damo>`_ is one example of such programs. It
24 supports both virtual and physical address spaces monitoring. Note that this
25 interface provides only simple :ref:`statistics <damos_stats>` for the
26 monitoring results. For detailed monitoring results, DAMON provides a
27 :ref:`tracepoint <tracepoint>`.
28 - *debugfs interface.*
29 :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface
30 <sysfs_interface>`. This will be removed after next LTS kernel is released,
31 so users should move to the :ref:`sysfs interface <sysfs_interface>`.
32 - *Kernel Space Programming Interface.*
33 :doc:`This </vm/damon/api>` is for kernel space programmers. Using this,
34 users can utilize every feature of DAMON most flexibly and efficiently by
35 writing kernel space DAMON application programs for you. You can even extend
36 DAMON for various address spaces. For detail, please refer to the interface
37 :doc:`document </vm/damon/api>`.
44 DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It
45 creates multiple directories and files under its sysfs directory,
46 ``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading
47 from the files under the directory.
49 For a short example, users can monitor the virtual address space of a given
52 # cd /sys/kernel/mm/damon/admin/
53 # echo 1 > kdamonds/nr && echo 1 > kdamonds/0/contexts/nr
54 # echo vaddr > kdamonds/0/contexts/0/operations
55 # echo 1 > kdamonds/0/contexts/0/targets/nr
56 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid
57 # echo on > kdamonds/0/state
62 The files hierarchy of DAMON sysfs interface is shown below. In the below
63 figure, parents-children relations are represented with indentations, each
64 directory is having ``/`` suffix, and files in each directory are separated by
67 /sys/kernel/mm/damon/admin
68 │ kdamonds/nr_kdamonds
70 │ │ │ contexts/nr_contexts
71 │ │ │ │ 0/avail_operations,operations
72 │ │ │ │ │ monitoring_attrs/
73 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
74 │ │ │ │ │ │ nr_regions/min,max
75 │ │ │ │ │ targets/nr_targets
76 │ │ │ │ │ │ 0/pid_target
77 │ │ │ │ │ │ │ regions/nr_regions
78 │ │ │ │ │ │ │ │ 0/start,end
81 │ │ │ │ │ schemes/nr_schemes
83 │ │ │ │ │ │ │ access_pattern/
84 │ │ │ │ │ │ │ │ sz/min,max
85 │ │ │ │ │ │ │ │ nr_accesses/min,max
86 │ │ │ │ │ │ │ │ age/min,max
87 │ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms
88 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
89 │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low
90 │ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds
98 The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
99 has one directory named ``admin``. The directory contains the files for
100 privileged user space programs' control of DAMON. User space tools or deamons
101 having the root permission could use this directory.
106 The monitoring-related information including request specifications and results
107 are called DAMON context. DAMON executes each context with a kernel thread
108 called kdamond, and multiple kdamonds could run in parallel.
110 Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
111 controlling the kdamonds exist. In the beginning, this directory has only one
112 file, ``nr_kdamonds``. Writing a number (``N``) to the file creates the number
113 of child directories named ``0`` to ``N-1``. Each directory represents each
119 In each kdamond directory, two files (``state`` and ``pid``) and one directory
120 (``contexts``) exist.
122 Reading ``state`` returns ``on`` if the kdamond is currently running, or
123 ``off`` if it is not running. Writing ``on`` or ``off`` makes the kdamond be
124 in the state. Writing ``update_schemes_stats`` to ``state`` file updates the
125 contents of stats files for each DAMON-based operation scheme of the kdamond.
126 For details of the stats, please refer to :ref:`stats section
127 <sysfs_schemes_stats>`.
129 If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
131 ``contexts`` directory contains files for controlling the monitoring contexts
132 that this kdamond will execute.
134 kdamonds/<N>/contexts/
135 ----------------------
137 In the beginning, this directory has only one file, ``nr_contexts``. Writing a
138 number (``N``) to the file creates the number of child directories named as
139 ``0`` to ``N-1``. Each directory represents each monitoring context. At the
140 moment, only one context per kdamond is supported, so only ``0`` or ``1`` can
141 be written to the file.
146 In each context directory, two files (``avail_operations`` and ``operations``)
147 and three directories (``monitoring_attrs``, ``targets``, and ``schemes``)
150 DAMON supports multiple types of monitoring operations, including those for
151 virtual address space and the physical address space. You can get the list of
152 available monitoring operations set on the currently running kernel by reading
153 ``avail_operations`` file. Based on the kernel configuration, the file will
154 list some or all of below keywords.
156 - vaddr: Monitor virtual address spaces of specific processes
157 - fvaddr: Monitor fixed virtual address ranges
158 - paddr: Monitor the physical address space of the system
160 Please refer to :ref:`regions sysfs directory <sysfs_regions>` for detailed
161 differences between the operations sets in terms of the monitoring target
164 You can set and get what type of monitoring operations DAMON will use for the
165 context by writing one of the keywords listed in ``avail_operations`` file and
166 reading from the ``operations`` file.
168 contexts/<N>/monitoring_attrs/
169 ------------------------------
171 Files for specifying attributes of the monitoring including required quality
172 and efficiency of the monitoring are in ``monitoring_attrs`` directory.
173 Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this
176 Under ``intervals`` directory, three files for DAMON's sampling interval
177 (``sample_us``), aggregation interval (``aggr_us``), and update interval
178 (``update_us``) exist. You can set and get the values in micro-seconds by
179 writing to and reading from the files.
181 Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
182 of DAMON's monitoring regions (``min`` and ``max``, respectively), which
183 controls the monitoring overhead, exist. You can set and get the values by
184 writing to and rading from the files.
186 For more details about the intervals and monitoring regions range, please refer
187 to the Design document (:doc:`/vm/damon/design`).
189 contexts/<N>/targets/
190 ---------------------
192 In the beginning, this directory has only one file, ``nr_targets``. Writing a
193 number (``N``) to the file creates the number of child directories named ``0``
194 to ``N-1``. Each directory represents each monitoring target.
199 In each target directory, one file (``pid_target``) and one directory
202 If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
203 be a process. You can specify the process to DAMON by writing the pid of the
204 process to the ``pid_target`` file.
211 When ``vaddr`` monitoring operations set is being used (``vaddr`` is written to
212 the ``contexts/<N>/operations`` file), DAMON automatically sets and updates the
213 monitoring target regions so that entire memory mappings of target processes
214 can be covered. However, users could want to set the initial monitoring region
215 to specific address ranges.
217 In contrast, DAMON do not automatically sets and updates the monitoring target
218 regions when ``fvaddr`` or ``paddr`` monitoring operations sets are being used
219 (``fvaddr`` or ``paddr`` have written to the ``contexts/<N>/operations``).
220 Therefore, users should set the monitoring target regions by themselves in the
223 For such cases, users can explicitly set the initial monitoring target regions
224 as they want, by writing proper values to the files under this directory.
226 In the beginning, this directory has only one file, ``nr_regions``. Writing a
227 number (``N``) to the file creates the number of child directories named ``0``
228 to ``N-1``. Each directory represents each initial monitoring target region.
233 In each region directory, you will find two files (``start`` and ``end``). You
234 can set and get the start and end addresses of the initial monitoring target
235 region by writing to and reading from the files, respectively.
237 contexts/<N>/schemes/
238 ---------------------
240 For usual DAMON-based data access aware memory management optimizations, users
241 would normally want the system to apply a memory management action to a memory
242 region of a specific access pattern. DAMON receives such formalized operation
243 schemes from the user and applies those to the target memory regions. Users
244 can get and set the schemes by reading from and writing to files under this
247 In the beginning, this directory has only one file, ``nr_schemes``. Writing a
248 number (``N``) to the file creates the number of child directories named ``0``
249 to ``N-1``. Each directory represents each DAMON-based operation scheme.
254 In each scheme directory, four directories (``access_pattern``, ``quotas``,
255 ``watermarks``, and ``stats``) and one file (``action``) exist.
257 The ``action`` file is for setting and getting what action you want to apply to
258 memory regions having specific access pattern of the interest. The keywords
259 that can be written to and read from the file and their meaning are as below.
261 - ``willneed``: Call ``madvise()`` for the region with ``MADV_WILLNEED``
262 - ``cold``: Call ``madvise()`` for the region with ``MADV_COLD``
263 - ``pageout``: Call ``madvise()`` for the region with ``MADV_PAGEOUT``
264 - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``
265 - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``
266 - ``stat``: Do nothing but count the statistics
268 schemes/<N>/access_pattern/
269 ---------------------------
271 The target access pattern of each DAMON-based operation scheme is constructed
272 with three ranges including the size of the region in bytes, number of
273 monitored accesses per aggregate interval, and number of aggregated intervals
274 for the age of the region.
276 Under the ``access_pattern`` directory, three directories (``sz``,
277 ``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
278 exist. You can set and get the access pattern for the given scheme by writing
279 to and reading from the ``min`` and ``max`` files under ``sz``,
280 ``nr_accesses``, and ``age`` directories, respectively.
285 Optimal ``target access pattern`` for each ``action`` is workload dependent, so
286 not easy to find. Worse yet, setting a scheme of some action too aggressive
287 can cause severe overhead. To avoid such overhead, users can limit time and
288 size quota for each scheme. In detail, users can ask DAMON to try to use only
289 up to specific time (``time quota``) for applying the action, and to apply the
290 action to only up to specific amount (``size quota``) of memory regions having
291 the target access pattern within a given time interval (``reset interval``).
293 When the quota limit is expected to be exceeded, DAMON prioritizes found memory
294 regions of the ``target access pattern`` based on their size, access frequency,
295 and age. For personalized prioritization, users can set the weights for the
298 Under ``quotas`` directory, three files (``ms``, ``bytes``,
299 ``reset_interval_ms``) and one directory (``weights``) having three files
300 (``sz_permil``, ``nr_accesses_permil``, and ``age_permil``) in it exist.
302 You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
303 ``reset interval`` in milliseconds by writing the values to the three files,
304 respectively. You can also set the prioritization weights for size, access
305 frequency, and age in per-thousand unit by writing the values to the three
306 files under the ``weights`` directory.
308 schemes/<N>/watermarks/
309 -----------------------
311 To allow easy activation and deactivation of each scheme based on system
312 status, DAMON provides a feature called watermarks. The feature receives five
313 values called ``metric``, ``interval``, ``high``, ``mid``, and ``low``. The
314 ``metric`` is the system metric such as free memory ratio that can be measured.
315 If the metric value of the system is higher than the value in ``high`` or lower
316 than ``low`` at the memoent, the scheme is deactivated. If the value is lower
317 than ``mid``, the scheme is activated.
319 Under the watermarks directory, five files (``metric``, ``interval_us``,
320 ``high``, ``mid``, and ``low``) for setting each value exist. You can set and
321 get the five values by writing to the files, respectively.
323 Keywords and meanings of those that can be written to the ``metric`` file are
326 - none: Ignore the watermarks
327 - free_mem_rate: System's free memory rate (per thousand)
329 The ``interval`` should written in microseconds unit.
331 .. _sysfs_schemes_stats:
336 DAMON counts the total number and bytes of regions that each scheme is tried to
337 be applied, the two numbers for the regions that each scheme is successfully
338 applied, and the total number of the quota limit exceeds. This statistics can
339 be used for online analysis or tuning of the schemes.
341 The statistics can be retrieved by reading the files under ``stats`` directory
342 (``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and
343 ``qt_exceeds``), respectively. The files are not updated in real time, so you
344 should ask DAMON sysfs interface to updte the content of the files for the
345 stats by writing a special keyword, ``update_schemes_stats`` to the relevant
346 ``kdamonds/<N>/state`` file.
351 Below commands applies a scheme saying "If a memory region of size in [4KiB,
352 8KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
353 interval in [10, 20], page out the region. For the paging out, use only up to
354 10ms per second, and also don't page out more than 1GiB per second. Under the
355 limitation, page out memory regions having longer age first. Also, check the
356 free memory rate of the system every 5 seconds, start the monitoring and paging
357 out when the free memory rate becomes lower than 50%, but stop it if the free
358 memory rate becomes larger than 60%, or lower than 30%". ::
360 # cd <sysfs>/kernel/mm/damon/admin
361 # # populate directories
362 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts;
363 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes
364 # cd kdamonds/0/contexts/0/schemes/0
365 # # set the basic access pattern and the action
366 # echo 4096 > access_patterns/sz/min
367 # echo 8192 > access_patterns/sz/max
368 # echo 0 > access_patterns/nr_accesses/min
369 # echo 5 > access_patterns/nr_accesses/max
370 # echo 10 > access_patterns/age/min
371 # echo 20 > access_patterns/age/max
372 # echo pageout > action
374 # echo 10 > quotas/ms
375 # echo $((1024*1024*1024)) > quotas/bytes
376 # echo 1000 > quotas/reset_interval_ms
378 # echo free_mem_rate > watermarks/metric
379 # echo 5000000 > watermarks/interval_us
380 # echo 600 > watermarks/high
381 # echo 500 > watermarks/mid
382 # echo 300 > watermarks/low
384 Please note that it's highly recommended to use user space tools like `damo
385 <https://github.com/awslabs/damo>`_ rather than manually reading and writing
386 the files as above. Above is only for an example.
388 .. _debugfs_interface:
393 DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``,
394 ``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and
395 ``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``.
401 Users can get and set the ``sampling interval``, ``aggregation interval``,
402 ``update interval``, and min/max number of monitoring target regions by
403 reading from and writing to the ``attrs`` file. To know about the monitoring
404 attributes in detail, please refer to the :doc:`/vm/damon/design`. For
405 example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and
406 1000, and then check it again::
409 # echo 5000 100000 1000000 10 1000 > attrs
411 5000 100000 1000000 10 1000
417 Some types of address spaces supports multiple monitoring target. For example,
418 the virtual memory address spaces monitoring can have multiple processes as the
419 monitoring targets. Users can set the targets by writing relevant id values of
420 the targets to, and get the ids of the current targets by reading from the
421 ``target_ids`` file. In case of the virtual address spaces monitoring, the
422 values should be pids of the monitoring target processes. For example, below
423 commands set processes having pids 42 and 4242 as the monitoring targets and
427 # echo 42 4242 > target_ids
431 Users can also monitor the physical memory address space of the system by
432 writing a special keyword, "``paddr\n``" to the file. Because physical address
433 space monitoring doesn't support multiple targets, reading the file will show a
434 fake value, ``42``, as below::
437 # echo paddr > target_ids
441 Note that setting the target ids doesn't start the monitoring.
444 Initial Monitoring Target Regions
445 ---------------------------------
447 In case of the virtual address space monitoring, DAMON automatically sets and
448 updates the monitoring target regions so that entire memory mappings of target
449 processes can be covered. However, users can want to limit the monitoring
450 region to specific address ranges, such as the heap, the stack, or specific
451 file-mapped area. Or, some users can know the initial access pattern of their
452 workloads and therefore want to set optimal initial regions for the 'adaptive
455 In contrast, DAMON do not automatically sets and updates the monitoring target
456 regions in case of physical memory monitoring. Therefore, users should set the
457 monitoring target regions by themselves.
459 In such cases, users can explicitly set the initial monitoring target regions
460 as they want, by writing proper values to the ``init_regions`` file. Each line
461 of the input should represent one region in below form.::
463 <target idx> <start address> <end address>
465 The ``target idx`` should be the index of the target in ``target_ids`` file,
466 starting from ``0``, and the regions should be passed in address order. For
467 example, below commands will set a couple of address ranges, ``1-100`` and
468 ``100-200`` as the initial monitoring target region of pid 42, which is the
469 first one (index ``0``) in ``target_ids``, and another couple of address
470 ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one
471 (index ``1``) in ``target_ids``.::
479 1 50 100" > init_regions
481 Note that this sets the initial monitoring target regions only. In case of
482 virtual memory monitoring, DAMON will automatically updates the boundary of the
483 regions after one ``update interval``. Therefore, users should set the
484 ``update interval`` large enough in this case, if they don't want the
491 For usual DAMON-based data access aware memory management optimizations, users
492 would simply want the system to apply a memory management action to a memory
493 region of a specific access pattern. DAMON receives such formalized operation
494 schemes from the user and applies those to the target processes.
496 Users can get and set the schemes by reading from and writing to ``schemes``
497 debugfs file. Reading the file also shows the statistics of each scheme. To
498 the file, each of the schemes should be represented in each line in below
501 <target access pattern> <action> <quota> <watermarks>
503 You can disable schemes by simply writing an empty string to the file.
505 Target Access Pattern
506 ~~~~~~~~~~~~~~~~~~~~~
508 The ``<target access pattern>`` is constructed with three ranges in below
511 min-size max-size min-acc max-acc min-age max-age
513 Specifically, bytes for the size of regions (``min-size`` and ``max-size``),
514 number of monitored accesses per aggregate interval for access frequency
515 (``min-acc`` and ``max-acc``), number of aggregate intervals for the age of
516 regions (``min-age`` and ``max-age``) are specified. Note that the ranges are
522 The ``<action>`` is a predefined integer for memory management actions, which
523 DAMON will apply to the regions having the target access pattern. The
524 supported numbers and their meanings are as below.
526 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``
527 - 1: Call ``madvise()`` for the region with ``MADV_COLD``
528 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT``
529 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``
530 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``
531 - 5: Do nothing but count the statistics
536 Optimal ``target access pattern`` for each ``action`` is workload dependent, so
537 not easy to find. Worse yet, setting a scheme of some action too aggressive
538 can cause severe overhead. To avoid such overhead, users can limit time and
539 size quota for the scheme via the ``<quota>`` in below form::
541 <ms> <sz> <reset interval> <priority weights>
543 This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying
544 the action to memory regions of the ``target access pattern`` within the
545 ``<reset interval>`` milliseconds, and to apply the action to only up to
546 ``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both
547 ``<ms>`` and ``<sz>`` zero disables the quota limits.
549 When the quota limit is expected to be exceeded, DAMON prioritizes found memory
550 regions of the ``target access pattern`` based on their size, access frequency,
551 and age. For personalized prioritization, users can set the weights for the
552 three properties in ``<priority weights>`` in below form::
554 <size weight> <access frequency weight> <age weight>
559 Some schemes would need to run based on current value of the system's specific
560 metrics like free memory ratio. For such cases, users can specify watermarks
563 <metric> <check interval> <high mark> <middle mark> <low mark>
565 ``<metric>`` is a predefined integer for the metric to be checked. The
566 supported numbers and their meanings are as below.
568 - 0: Ignore the watermarks
569 - 1: System's free memory rate (per thousand)
571 The value of the metric is checked every ``<check interval>`` microseconds.
573 If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the
574 scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme
582 It also counts the total number and bytes of regions that each scheme is tried
583 to be applied, the two numbers for the regions that each scheme is successfully
584 applied, and the total number of the quota limit exceeds. This statistics can
585 be used for online analysis or tuning of the schemes.
587 The statistics can be shown by reading the ``schemes`` file. Reading the file
588 will show each scheme you entered in each line, and the five numbers for the
589 statistics will be added at the end of each line.
594 Below commands applies a scheme saying "If a memory region of size in [4KiB,
595 8KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
596 interval in [10, 20], page out the region. For the paging out, use only up to
597 10ms per second, and also don't page out more than 1GiB per second. Under the
598 limitation, page out memory regions having longer age first. Also, check the
599 free memory rate of the system every 5 seconds, start the monitoring and paging
600 out when the free memory rate becomes lower than 50%, but stop it if the free
601 memory rate becomes larger than 60%, or lower than 30%".::
604 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action
605 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas
606 # scheme+=" 0 0 100" # prioritization weights
607 # scheme+=" 1 5000000 600 500 300" # watermarks
608 # echo "$scheme" > schemes
614 Setting the files as described above doesn't incur effect unless you explicitly
615 start the monitoring. You can start, stop, and check the current status of the
616 monitoring by writing to and reading from the ``monitor_on`` file. Writing
617 ``on`` to the file starts the monitoring of the targets with the attributes.
618 Writing ``off`` to the file stops those. DAMON also stops if every target
619 process is terminated. Below example commands turn on, off, and check the
623 # echo on > monitor_on
624 # echo off > monitor_on
628 Please note that you cannot write to the above-mentioned debugfs files while
629 the monitoring is turned on. If you write to the files while DAMON is running,
630 an error code such as ``-EBUSY`` will be returned.
633 Monitoring Thread PID
634 ---------------------
636 DAMON does requested monitoring with a kernel thread called ``kdamond``. You
637 can get the pid of the thread by reading the ``kdamond_pid`` file. When the
638 monitoring is turned off, reading the file returns ``none``. ::
645 # echo on > monitor_on
650 Using Multiple Monitoring Threads
651 ---------------------------------
653 One ``kdamond`` thread is created for each monitoring context. You can create
654 and remove monitoring contexts for multiple ``kdamond`` required use case using
655 the ``mk_contexts`` and ``rm_contexts`` files.
657 Writing the name of the new context to the ``mk_contexts`` file creates a
658 directory of the name on the DAMON debugfs directory. The directory will have
659 DAMON debugfs files for the context. ::
663 # ls: cannot access 'foo': No such file or directory
664 # echo foo > mk_contexts
666 # attrs init_regions kdamond_pid schemes target_ids
668 If the context is not needed anymore, you can remove it and the corresponding
669 directory by putting the name of the context to the ``rm_contexts`` file. ::
671 # echo foo > rm_contexts
673 # ls: cannot access 'foo': No such file or directory
675 Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the
681 Tracepoint for Monitoring Results
682 =================================
684 DAMON provides the monitoring results via a tracepoint,
685 ``damon:damon_aggregated``. While the monitoring is turned on, you could
686 record the tracepoint events and show results using tracepoint supporting tools
687 like ``perf``. For example::
689 # echo on > monitor_on
690 # perf record -e damon:damon_aggregated &
692 # kill 9 $(pidof perf)
693 # echo off > monitor_on