Merge in the x86 changes to apply a fix.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
--- /dev/null
+What: /sys/firmware/efi/fw_vendor
+Date: December 2013
+Contact: Dave Young <dyoung@redhat.com>
+Description: It shows the physical address of firmware vendor field in the
+ EFI system table.
+Users: Kexec
+
+What: /sys/firmware/efi/runtime
+Date: December 2013
+Contact: Dave Young <dyoung@redhat.com>
+Description: It shows the physical address of runtime service table entry in
+ the EFI system table.
+Users: Kexec
+
+What: /sys/firmware/efi/config_table
+Date: December 2013
+Contact: Dave Young <dyoung@redhat.com>
+Description: It shows the physical address of config table entry in the EFI
+ system table.
+Users: Kexec
--- /dev/null
+What: /sys/firmware/efi/runtime-map/
+Date: December 2013
+Contact: Dave Young <dyoung@redhat.com>
+Description: Switching efi runtime services to virtual mode requires
+ that all efi memory ranges which have the runtime attribute
+ bit set to be mapped to virtual addresses.
+
+ The efi runtime services can only be switched to virtual
+ mode once without rebooting. The kexec kernel must maintain
+ the same physical to virtual address mappings as the first
+ kernel. The mappings are exported to sysfs so userspace tools
+ can reassemble them and pass them into the kexec kernel.
+
+ /sys/firmware/efi/runtime-map/ is the directory the kernel
+ exports that information in.
+
+ subdirectories are named with the number of the memory range:
+
+ /sys/firmware/efi/runtime-map/0
+ /sys/firmware/efi/runtime-map/1
+ /sys/firmware/efi/runtime-map/2
+ /sys/firmware/efi/runtime-map/3
+ ...
+
+ Each subdirectory contains five files:
+
+ attribute : The attributes of the memory range.
+ num_pages : The size of the memory range in pages.
+ phys_addr : The physical address of the memory range.
+ type : The type of the memory range.
+ virt_addr : The virtual address of the memory range.
+
+ Above values are all hexadecimal numbers with the '0x' prefix.
+Users: Kexec
--- /dev/null
+What: /sys/kernel/boot_params
+Date: December 2013
+Contact: Dave Young <dyoung@redhat.com>
+Description: The /sys/kernel/boot_params directory contains two
+ files: "data" and "version" and one subdirectory "setup_data".
+ It is used to export the kernel boot parameters of an x86
+ platform to userspace for kexec and debugging purpose.
+
+ If there's no setup_data in boot_params the subdirectory will
+ not be created.
+
+ "data" file is the binary representation of struct boot_params.
+
+ "version" file is the string representation of boot
+ protocol version.
+
+ "setup_data" subdirectory contains the setup_data data
+ structure in boot_params. setup_data is maintained in kernel
+ as a link list. In "setup_data" subdirectory there's one
+ subdirectory for each link list node named with the number
+ of the list nodes. The list node subdirectory contains two
+ files "type" and "data". "type" file is the string
+ representation of setup_data type. "data" file is the binary
+ representation of setup_data payload.
+
+ The whole boot_params directory structure is like below:
+ /sys/kernel/boot_params
+ |__ data
+ |__ setup_data
+ | |__ 0
+ | | |__ data
+ | | |__ type
+ | |__ 1
+ | |__ data
+ | |__ type
+ |__ version
+
+Users: Kexec
The output of "cat rcu/rcu_sched/rcu_pending" looks as follows:
- 0!np=26111 qsp=29 rpq=5386 cbr=1 cng=570 gpc=3674 gps=577 nn=15903
- 1!np=28913 qsp=35 rpq=6097 cbr=1 cng=448 gpc=3700 gps=554 nn=18113
- 2!np=32740 qsp=37 rpq=6202 cbr=0 cng=476 gpc=4627 gps=546 nn=20889
- 3 np=23679 qsp=22 rpq=5044 cbr=1 cng=415 gpc=3403 gps=347 nn=14469
- 4!np=30714 qsp=4 rpq=5574 cbr=0 cng=528 gpc=3931 gps=639 nn=20042
- 5 np=28910 qsp=2 rpq=5246 cbr=0 cng=428 gpc=4105 gps=709 nn=18422
- 6!np=38648 qsp=5 rpq=7076 cbr=0 cng=840 gpc=4072 gps=961 nn=25699
- 7 np=37275 qsp=2 rpq=6873 cbr=0 cng=868 gpc=3416 gps=971 nn=25147
+ 0!np=26111 qsp=29 rpq=5386 cbr=1 cng=570 gpc=3674 gps=577 nn=15903 ndw=0
+ 1!np=28913 qsp=35 rpq=6097 cbr=1 cng=448 gpc=3700 gps=554 nn=18113 ndw=0
+ 2!np=32740 qsp=37 rpq=6202 cbr=0 cng=476 gpc=4627 gps=546 nn=20889 ndw=0
+ 3 np=23679 qsp=22 rpq=5044 cbr=1 cng=415 gpc=3403 gps=347 nn=14469 ndw=0
+ 4!np=30714 qsp=4 rpq=5574 cbr=0 cng=528 gpc=3931 gps=639 nn=20042 ndw=0
+ 5 np=28910 qsp=2 rpq=5246 cbr=0 cng=428 gpc=4105 gps=709 nn=18422 ndw=0
+ 6!np=38648 qsp=5 rpq=7076 cbr=0 cng=840 gpc=4072 gps=961 nn=25699 ndw=0
+ 7 np=37275 qsp=2 rpq=6873 cbr=0 cng=868 gpc=3416 gps=971 nn=25147 ndw=0
The fields are as follows:
o "gps" is the number of times that a new grace period had started,
but this CPU was not yet aware of it.
+o "ndw" is the number of times that a wakeup of an rcuo
+ callback-offload kthread had to be deferred in order to avoid
+ deadlock.
+
o "nn" is the number of times that this CPU needed nothing.
balk: nt=0 egt=6541 bt=0 nb=0 ny=126 nos=0
This information is output only for rcu_preempt. Each two-line entry
-corresponds to a leaf rcu_node strcuture. The fields are as follows:
+corresponds to a leaf rcu_node structure. The fields are as follows:
o "n:m" is the CPU-number range for the corresponding two-line
entry. In the sample output above, the first entry covers
injection. Before this, please specify all necessary error
parameters.
+- flags
+ Present for kernel version 3.13 and above. Used to specify which
+ of param{1..4} are valid and should be used by BIOS during injection.
+ Value is a bitmask as specified in ACPI5.0 spec for the
+ SET_ERROR_TYPE_WITH_ADDRESS data structure:
+ Bit 0 - Processor APIC field valid (see param3 below)
+ Bit 1 - Memory address and mask valid (param1 and param2)
+ Bit 2 - PCIe (seg,bus,dev,fn) valid (param4 below)
+ If set to zero, legacy behaviour is used where the type of injection
+ specifies just one bit set, and param1 is multiplexed.
+
- param1
This file is used to set the first error parameter value. Effect of
parameter depends on error_type specified. For example, if error
type is memory related type, the param1 should be a valid physical
- memory address.
+ memory address. [Unless "flag" is set - see above]
- param2
This file is used to set the second error parameter value. Effect of
address mask. Linux requires page or narrower granularity, say,
0xfffffffffffff000.
+- param3
+ Used when the 0x1 bit is set in "flag" to specify the APIC id
+
+- param4
+ Used when the 0x4 bit is set in "flag" to specify target PCIe device
+
- notrigger
The EINJ mechanism is a two step process. First inject the error, then
perform some actions to trigger it. Setting "notrigger" to 1 skips the
spin_lock(&producer_lock);
unsigned long head = buffer->head;
+ /* The spin_unlock() and next spin_lock() provide needed ordering. */
unsigned long tail = ACCESS_ONCE(buffer->tail);
if (CIRC_SPACE(head, tail, buffer->size) >= 1) {
produce_item(item);
- smp_wmb(); /* commit the item before incrementing the head */
-
- buffer->head = (head + 1) & (buffer->size - 1);
+ smp_store_release(buffer->head,
+ (head + 1) & (buffer->size - 1));
/* wake_up() will make sure that the head is committed before
* waking anyone up */
before the head index makes it available to the consumer and then instructs the
CPU that the revised head index must be written before the consumer is woken.
-Note that wake_up() doesn't have to be the exact mechanism used, but whatever
-is used must guarantee a (write) memory barrier between the update of the head
-index and the change of state of the consumer, if a change of state occurs.
+Note that wake_up() does not guarantee any sort of barrier unless something
+is actually awakened. We therefore cannot rely on it for ordering. However,
+there is always one element of the array left empty. Therefore, the
+producer must produce two elements before it could possibly corrupt the
+element currently being read by the consumer. Therefore, the unlock-lock
+pair between consecutive invocations of the consumer provides the necessary
+ordering between the read of the index indicating that the consumer has
+vacated a given element and the write by the producer to that same element.
THE CONSUMER
spin_lock(&consumer_lock);
- unsigned long head = ACCESS_ONCE(buffer->head);
+ /* Read index before reading contents at that index. */
+ unsigned long head = smp_load_acquire(buffer->head);
unsigned long tail = buffer->tail;
if (CIRC_CNT(head, tail, buffer->size) >= 1) {
- /* read index before reading contents at that index */
- smp_read_barrier_depends();
/* extract one item from the buffer */
struct item *item = buffer[tail];
consume_item(item);
- smp_mb(); /* finish reading descriptor before incrementing tail */
-
- buffer->tail = (tail + 1) & (buffer->size - 1);
+ /* Finish reading descriptor before incrementing tail. */
+ smp_store_release(buffer->tail,
+ (tail + 1) & (buffer->size - 1));
}
spin_unlock(&consumer_lock);
the new item, and then it shall make sure the CPU has finished reading the item
before it writes the new tail pointer, which will erase the item.
-
-Note the use of ACCESS_ONCE() in both algorithms to read the opposition index.
-This prevents the compiler from discarding and reloading its cached value -
-which some compilers will do across smp_read_barrier_depends(). This isn't
-strictly needed if you can be sure that the opposition index will _only_ be
-used the once.
+Note the use of ACCESS_ONCE() and smp_load_acquire() to read the
+opposition index. This prevents the compiler from discarding and
+reloading its cached value - which some compilers will do across
+smp_read_barrier_depends(). This isn't strictly needed if you can
+be sure that the opposition index will _only_ be used the once.
+The smp_load_acquire() additionally forces the CPU to order against
+subsequent memory references. Similarly, smp_store_release() is used
+in both algorithms to write the thread's index. This documents the
+fact that we are writing to something that can be read concurrently,
+prevents the compiler from tearing the store, and enforces ordering
+against previous accesses.
===============
--- /dev/null
+Allwinner SoCs High Speed Timer Controller
+
+Required properties:
+
+- compatible : should be "allwinner,sun5i-a13-hstimer" or
+ "allwinner,sun7i-a20-hstimer"
+- reg : Specifies base physical address and size of the registers.
+- interrupts : The interrupts of these timers (2 for the sun5i IP, 4 for the sun7i
+ one)
+- clocks: phandle to the source clock (usually the AHB clock)
+
+Example:
+
+timer@01c60000 {
+ compatible = "allwinner,sun7i-a20-hstimer";
+ reg = <0x01c60000 0x1000>;
+ interrupts = <0 51 1>,
+ <0 52 1>,
+ <0 53 1>,
+ <0 54 1>;
+ clocks = <&ahb1_gates 19>;
+};
disable= [IPV6]
See Documentation/networking/ipv6.txt.
+ disable_cpu_apicid= [X86,APIC,SMP]
+ Format: <int>
+ The number of initial APIC ID for the
+ corresponding CPU to be disabled at boot,
+ mostly used for the kdump 2nd kernel to
+ disable BSP to wake up multiple CPUs without
+ causing system reset or hang due to sending
+ INIT from AP to BSP.
+
disable_ddw [PPC/PSERIES]
Disable Dynamic DMA Window support. Use this if
to workaround buggy firmware.
The xen output can only be used by Xen PV guests.
+ edac_report= [HW,EDAC] Control how to report EDAC event
+ Format: {"on" | "off" | "force"}
+ on: enable EDAC to report H/W event. May be overridden
+ by other higher priority error reporting module.
+ off: disable H/W event reporting through EDAC.
+ force: enforce the use of EDAC to report H/W event.
+ default: on.
+
ekgdboc= [X86,KGDB] Allow early kernel console debugging
ekgdboc=kbd
edd= [EDD]
Format: {"off" | "on" | "skip[mbr]"}
+ efi= [EFI]
+ Format: { "old_map" }
+ old_map [X86-64]: switch to the old ioremap-based EFI
+ runtime services mapping. 32-bit still uses this one by
+ default.
+
efi_no_storage_paranoia [EFI; X86]
Using this parameter you can use more than 50% of
your efi variable storage. Use this parameter only if
noapic [SMP,APIC] Tells the kernel to not make use of any
IOAPICs that may be present in the system.
+ nokaslr [X86]
+ Disable kernel base offset ASLR (Address Space
+ Layout Randomization) if built into the kernel.
+
noautogroup Disable scheduler automatic task group creation.
nobats [PPC] Do not use BATs for mapping kernel lowmem
for RCU-preempt, and "s" for RCU-sched, and "N"
is the CPU number. This reduces OS jitter on the
offloaded CPUs, which can be useful for HPC and
-
real-time workloads. It can also improve energy
efficiency for asymmetric multiprocessors.
periodically wake up to do the polling.
rcutree.blimit= [KNL]
- Set maximum number of finished RCU callbacks to process
- in one batch.
+ Set maximum number of finished RCU callbacks to
+ process in one batch.
rcutree.rcu_fanout_leaf= [KNL]
Increase the number of CPUs assigned to each
value is one, and maximum value is HZ.
rcutree.qhimark= [KNL]
- Set threshold of queued
- RCU callbacks over which batch limiting is disabled.
+ Set threshold of queued RCU callbacks beyond which
+ batch limiting is disabled.
rcutree.qlowmark= [KNL]
Set threshold of queued RCU callbacks below which
--- /dev/null
+/*?
+ * Text: "Cryptographic device %x failed and was set offline\n"
+ * Severity: Error
+ * Parameter:
+ * @1: device index
+ * Description:
+ * A cryptographic device failed to process a cryptographic request.
+ * The cryptographic device driver could not correct the error and
+ * set the device offline. The application that issued the
+ * request received an indication that the request has failed.
+ * User action:
+ * Use the lszcrypt command to confirm that the cryptographic
+ * hardware is still configured to your LPAR or z/VM guest virtual
+ * machine. If the device is available to your Linux instance the
+ * command output contains a line that begins with 'card<device index>',
+ * where <device index> is the two-digit decimal number in the message text.
+ * After ensuring that the device is available, use the chzcrypt command to
+ * set it online again.
+ * If the error persists, contact your support organization.
+ */
(*) On any given CPU, dependent memory accesses will be issued in order, with
respect to itself. This means that for:
- Q = P; D = *Q;
+ ACCESS_ONCE(Q) = P; smp_read_barrier_depends(); D = ACCESS_ONCE(*Q);
the CPU will issue the following memory operations:
Q = LOAD P, D = LOAD *Q
- and always in that order.
+ and always in that order. On most systems, smp_read_barrier_depends()
+ does nothing, but it is required for DEC Alpha. The ACCESS_ONCE()
+ is required to prevent compiler mischief. Please note that you
+ should normally use something like rcu_dereference() instead of
+ open-coding smp_read_barrier_depends().
(*) Overlapping loads and stores within a particular CPU will appear to be
ordered within that CPU. This means that for:
- a = *X; *X = b;
+ a = ACCESS_ONCE(*X); ACCESS_ONCE(*X) = b;
the CPU will only issue the following sequence of memory operations:
And for:
- *X = c; d = *X;
+ ACCESS_ONCE(*X) = c; d = ACCESS_ONCE(*X);
the CPU will only issue:
And there are a number of things that _must_ or _must_not_ be assumed:
+ (*) It _must_not_ be assumed that the compiler will do what you want with
+ memory references that are not protected by ACCESS_ONCE(). Without
+ ACCESS_ONCE(), the compiler is within its rights to do all sorts
+ of "creative" transformations, which are covered in the Compiler
+ Barrier section.
+
(*) It _must_not_ be assumed that independent loads and stores will be issued
in the order given. This means that for:
And a couple of implicit varieties:
- (5) LOCK operations.
+ (5) ACQUIRE operations.
This acts as a one-way permeable barrier. It guarantees that all memory
- operations after the LOCK operation will appear to happen after the LOCK
- operation with respect to the other components of the system.
+ operations after the ACQUIRE operation will appear to happen after the
+ ACQUIRE operation with respect to the other components of the system.
+ ACQUIRE operations include LOCK operations and smp_load_acquire()
+ operations.
- Memory operations that occur before a LOCK operation may appear to happen
- after it completes.
+ Memory operations that occur before an ACQUIRE operation may appear to
+ happen after it completes.
- A LOCK operation should almost always be paired with an UNLOCK operation.
+ An ACQUIRE operation should almost always be paired with a RELEASE
+ operation.
- (6) UNLOCK operations.
+ (6) RELEASE operations.
This also acts as a one-way permeable barrier. It guarantees that all
- memory operations before the UNLOCK operation will appear to happen before
- the UNLOCK operation with respect to the other components of the system.
+ memory operations before the RELEASE operation will appear to happen
+ before the RELEASE operation with respect to the other components of the
+ system. RELEASE operations include UNLOCK operations and
+ smp_store_release() operations.
- Memory operations that occur after an UNLOCK operation may appear to
+ Memory operations that occur after a RELEASE operation may appear to
happen before it completes.
- LOCK and UNLOCK operations are guaranteed to appear with respect to each
- other strictly in the order specified.
+ The use of ACQUIRE and RELEASE operations generally precludes the need
+ for other sorts of memory barrier (but note the exceptions mentioned in
+ the subsection "MMIO write barrier"). In addition, a RELEASE+ACQUIRE
+ pair is -not- guaranteed to act as a full memory barrier. However, after
+ an ACQUIRE on a given variable, all memory accesses preceding any prior
+ RELEASE on that same variable are guaranteed to be visible. In other
+ words, within a given variable's critical section, all accesses of all
+ previous critical sections for that variable are guaranteed to have
+ completed.
- The use of LOCK and UNLOCK operations generally precludes the need for
- other sorts of memory barrier (but note the exceptions mentioned in the
- subsection "MMIO write barrier").
+ This means that ACQUIRE acts as a minimal "acquire" operation and
+ RELEASE acts as a minimal "release" operation.
Memory barriers are only required where there's a possibility of interaction
it's not always obvious that they're needed. To illustrate, consider the
following sequence of events:
- CPU 1 CPU 2
- =============== ===============
+ CPU 1 CPU 2
+ =============== ===============
{ A == 1, B == 2, C = 3, P == &A, Q == &C }
B = 4;
<write barrier>
- P = &B
- Q = P;
- D = *Q;
+ ACCESS_ONCE(P) = &B
+ Q = ACCESS_ONCE(P);
+ D = *Q;
There's a clear data dependency here, and it would seem that by the end of the
sequence, Q must be either &A or &B, and that:
To deal with this, a data dependency barrier or better must be inserted
between the address load and the data load:
- CPU 1 CPU 2
- =============== ===============
+ CPU 1 CPU 2
+ =============== ===============
{ A == 1, B == 2, C = 3, P == &A, Q == &C }
B = 4;
<write barrier>
- P = &B
- Q = P;
- <data dependency barrier>
- D = *Q;
+ ACCESS_ONCE(P) = &B
+ Q = ACCESS_ONCE(P);
+ <data dependency barrier>
+ D = *Q;
This enforces the occurrence of one of the two implications, and prevents the
third possibility from arising.
but the old value of the variable B (2).
-Another example of where data dependency barriers might by required is where a
+Another example of where data dependency barriers might be required is where a
number is read from memory and then used to calculate the index for an array
access:
- CPU 1 CPU 2
- =============== ===============
+ CPU 1 CPU 2
+ =============== ===============
{ M[0] == 1, M[1] == 2, M[3] = 3, P == 0, Q == 3 }
M[1] = 4;
<write barrier>
- P = 1
- Q = P;
- <data dependency barrier>
- D = M[Q];
+ ACCESS_ONCE(P) = 1
+ Q = ACCESS_ONCE(P);
+ <data dependency barrier>
+ D = M[Q];
-The data dependency barrier is very important to the RCU system, for example.
-See rcu_dereference() in include/linux/rcupdate.h. This permits the current
-target of an RCU'd pointer to be replaced with a new modified target, without
-the replacement target appearing to be incompletely initialised.
+The data dependency barrier is very important to the RCU system,
+for example. See rcu_assign_pointer() and rcu_dereference() in
+include/linux/rcupdate.h. This permits the current target of an RCU'd
+pointer to be replaced with a new modified target, without the replacement
+target appearing to be incompletely initialised.
See also the subsection on "Cache Coherency" for a more thorough example.
dependency barrier to make it work correctly. Consider the following bit of
code:
- q = &a;
- if (p) {
- <data dependency barrier>
- q = &b;
+ q = ACCESS_ONCE(a);
+ if (q) {
+ <data dependency barrier> /* BUG: No data dependency!!! */
+ p = ACCESS_ONCE(b);
}
- x = *q;
This will not have the desired effect because there is no actual data
-dependency, but rather a control dependency that the CPU may short-circuit by
-attempting to predict the outcome in advance. In such a case what's actually
-required is:
+dependency, but rather a control dependency that the CPU may short-circuit
+by attempting to predict the outcome in advance, so that other CPUs see
+the load from b as having happened before the load from a. In such a
+case what's actually required is:
- q = &a;
- if (p) {
+ q = ACCESS_ONCE(a);
+ if (q) {
<read barrier>
- q = &b;
+ p = ACCESS_ONCE(b);
+ }
+
+However, stores are not speculated. This means that ordering -is- provided
+in the following example:
+
+ q = ACCESS_ONCE(a);
+ if (ACCESS_ONCE(q)) {
+ ACCESS_ONCE(b) = p;
+ }
+
+Please note that ACCESS_ONCE() is not optional! Without the ACCESS_ONCE(),
+the compiler is within its rights to transform this example:
+
+ q = a;
+ if (q) {
+ b = p; /* BUG: Compiler can reorder!!! */
+ do_something();
+ } else {
+ b = p; /* BUG: Compiler can reorder!!! */
+ do_something_else();
+ }
+
+into this, which of course defeats the ordering:
+
+ b = p;
+ q = a;
+ if (q)
+ do_something();
+ else
+ do_something_else();
+
+Worse yet, if the compiler is able to prove (say) that the value of
+variable 'a' is always non-zero, it would be well within its rights
+to optimize the original example by eliminating the "if" statement
+as follows:
+
+ q = a;
+ b = p; /* BUG: Compiler can reorder!!! */
+ do_something();
+
+The solution is again ACCESS_ONCE(), which preserves the ordering between
+the load from variable 'a' and the store to variable 'b':
+
+ q = ACCESS_ONCE(a);
+ if (q) {
+ ACCESS_ONCE(b) = p;
+ do_something();
+ } else {
+ ACCESS_ONCE(b) = p;
+ do_something_else();
+ }
+
+You could also use barrier() to prevent the compiler from moving
+the stores to variable 'b', but barrier() would not prevent the
+compiler from proving to itself that a==1 always, so ACCESS_ONCE()
+is also needed.
+
+It is important to note that control dependencies absolutely require a
+a conditional. For example, the following "optimized" version of
+the above example breaks ordering:
+
+ q = ACCESS_ONCE(a);
+ ACCESS_ONCE(b) = p; /* BUG: No ordering vs. load from a!!! */
+ if (q) {
+ /* ACCESS_ONCE(b) = p; -- moved up, BUG!!! */
+ do_something();
+ } else {
+ /* ACCESS_ONCE(b) = p; -- moved up, BUG!!! */
+ do_something_else();
}
- x = *q;
+
+It is of course legal for the prior load to be part of the conditional,
+for example, as follows:
+
+ if (ACCESS_ONCE(a) > 0) {
+ ACCESS_ONCE(b) = q / 2;
+ do_something();
+ } else {
+ ACCESS_ONCE(b) = q / 3;
+ do_something_else();
+ }
+
+This will again ensure that the load from variable 'a' is ordered before the
+stores to variable 'b'.
+
+In addition, you need to be careful what you do with the local variable 'q',
+otherwise the compiler might be able to guess the value and again remove
+the needed conditional. For example:
+
+ q = ACCESS_ONCE(a);
+ if (q % MAX) {
+ ACCESS_ONCE(b) = p;
+ do_something();
+ } else {
+ ACCESS_ONCE(b) = p;
+ do_something_else();
+ }
+
+If MAX is defined to be 1, then the compiler knows that (q % MAX) is
+equal to zero, in which case the compiler is within its rights to
+transform the above code into the following:
+
+ q = ACCESS_ONCE(a);
+ ACCESS_ONCE(b) = p;
+ do_something_else();
+
+This transformation loses the ordering between the load from variable 'a'
+and the store to variable 'b'. If you are relying on this ordering, you
+should do something like the following:
+
+ q = ACCESS_ONCE(a);
+ BUILD_BUG_ON(MAX <= 1); /* Order load from a with store to b. */
+ if (q % MAX) {
+ ACCESS_ONCE(b) = p;
+ do_something();
+ } else {
+ ACCESS_ONCE(b) = p;
+ do_something_else();
+ }
+
+Finally, control dependencies do -not- provide transitivity. This is
+demonstrated by two related examples:
+
+ CPU 0 CPU 1
+ ===================== =====================
+ r1 = ACCESS_ONCE(x); r2 = ACCESS_ONCE(y);
+ if (r1 >= 0) if (r2 >= 0)
+ ACCESS_ONCE(y) = 1; ACCESS_ONCE(x) = 1;
+
+ assert(!(r1 == 1 && r2 == 1));
+
+The above two-CPU example will never trigger the assert(). However,
+if control dependencies guaranteed transitivity (which they do not),
+then adding the following two CPUs would guarantee a related assertion:
+
+ CPU 2 CPU 3
+ ===================== =====================
+ ACCESS_ONCE(x) = 2; ACCESS_ONCE(y) = 2;
+
+ assert(!(r1 == 2 && r2 == 2 && x == 1 && y == 1)); /* FAILS!!! */
+
+But because control dependencies do -not- provide transitivity, the
+above assertion can fail after the combined four-CPU example completes.
+If you need the four-CPU example to provide ordering, you will need
+smp_mb() between the loads and stores in the CPU 0 and CPU 1 code fragments.
+
+In summary:
+
+ (*) Control dependencies can order prior loads against later stores.
+ However, they do -not- guarantee any other sort of ordering:
+ Not prior loads against later loads, nor prior stores against
+ later anything. If you need these other forms of ordering,
+ use smb_rmb(), smp_wmb(), or, in the case of prior stores and
+ later loads, smp_mb().
+
+ (*) Control dependencies require at least one run-time conditional
+ between the prior load and the subsequent store. If the compiler
+ is able to optimize the conditional away, it will have also
+ optimized away the ordering. Careful use of ACCESS_ONCE() can
+ help to preserve the needed conditional.
+
+ (*) Control dependencies require that the compiler avoid reordering the
+ dependency into nonexistence. Careful use of ACCESS_ONCE() or
+ barrier() can help to preserve your control dependency. Please
+ see the Compiler Barrier section for more information.
+
+ (*) Control dependencies do -not- provide transitivity. If you
+ need transitivity, use smp_mb().
SMP BARRIER PAIRING
barrier or a data dependency barrier should always be paired with at least an
write barrier, though, again, a general barrier is viable:
- CPU 1 CPU 2
- =============== ===============
- a = 1;
+ CPU 1 CPU 2
+ =============== ===============
+ ACCESS_ONCE(a) = 1;
<write barrier>
- b = 2; x = b;
- <read barrier>
- y = a;
+ ACCESS_ONCE(b) = 2; x = ACCESS_ONCE(b);
+ <read barrier>
+ y = ACCESS_ONCE(a);
Or:
- CPU 1 CPU 2
- =============== ===============================
+ CPU 1 CPU 2
+ =============== ===============================
a = 1;
<write barrier>
- b = &a; x = b;
- <data dependency barrier>
- y = *x;
+ ACCESS_ONCE(b) = &a; x = ACCESS_ONCE(b);
+ <data dependency barrier>
+ y = *x;
Basically, the read barrier always has to be there, even though it can be of
the "weaker" type.
match the loads after the read barrier or the data dependency barrier, and vice
versa:
- CPU 1 CPU 2
- =============== ===============
- a = 1; }---- --->{ v = c
- b = 2; } \ / { w = d
- <write barrier> \ <read barrier>
- c = 3; } / \ { x = a;
- d = 4; }---- --->{ y = b;
+ CPU 1 CPU 2
+ =================== ===================
+ ACCESS_ONCE(a) = 1; }---- --->{ v = ACCESS_ONCE(c);
+ ACCESS_ONCE(b) = 2; } \ / { w = ACCESS_ONCE(d);
+ <write barrier> \ <read barrier>
+ ACCESS_ONCE(c) = 3; } / \ { x = ACCESS_ONCE(a);
+ ACCESS_ONCE(d) = 4; }---- --->{ y = ACCESS_ONCE(b);
EXAMPLES OF MEMORY BARRIER SEQUENCES
Consider:
- CPU 1 CPU 2
+ CPU 1 CPU 2
======================= =======================
- LOAD B
- DIVIDE } Divide instructions generally
- DIVIDE } take a long time to perform
- LOAD A
+ LOAD B
+ DIVIDE } Divide instructions generally
+ DIVIDE } take a long time to perform
+ LOAD A
Which might appear as this:
Placing a read barrier or a data dependency barrier just before the second
load:
- CPU 1 CPU 2
+ CPU 1 CPU 2
======================= =======================
- LOAD B
- DIVIDE
- DIVIDE
+ LOAD B
+ DIVIDE
+ DIVIDE
<read barrier>
- LOAD A
+ LOAD A
will force any value speculatively obtained to be reconsidered to an extent
dependent on the type of barrier used. If there was no change made to the
barrier();
-This is a general barrier - lesser varieties of compiler barrier do not exist.
+This is a general barrier -- there are no read-read or write-write variants
+of barrier(). However, ACCESS_ONCE() can be thought of as a weak form
+for barrier() that affects only the specific accesses flagged by the
+ACCESS_ONCE().
+
+The barrier() function has the following effects:
+
+ (*) Prevents the compiler from reordering accesses following the
+ barrier() to precede any accesses preceding the barrier().
+ One example use for this property is to ease communication between
+ interrupt-handler code and the code that was interrupted.
+
+ (*) Within a loop, forces the compiler to load the variables used
+ in that loop's conditional on each pass through that loop.
+
+The ACCESS_ONCE() function can prevent any number of optimizations that,
+while perfectly safe in single-threaded code, can be fatal in concurrent
+code. Here are some examples of these sorts of optimizations:
+
+ (*) The compiler is within its rights to merge successive loads from
+ the same variable. Such merging can cause the compiler to "optimize"
+ the following code:
+
+ while (tmp = a)
+ do_something_with(tmp);
+
+ into the following code, which, although in some sense legitimate
+ for single-threaded code, is almost certainly not what the developer
+ intended:
+
+ if (tmp = a)
+ for (;;)
+ do_something_with(tmp);
+
+ Use ACCESS_ONCE() to prevent the compiler from doing this to you:
+
+ while (tmp = ACCESS_ONCE(a))
+ do_something_with(tmp);
+
+ (*) The compiler is within its rights to reload a variable, for example,
+ in cases where high register pressure prevents the compiler from
+ keeping all data of interest in registers. The compiler might
+ therefore optimize the variable 'tmp' out of our previous example:
+
+ while (tmp = a)
+ do_something_with(tmp);
+
+ This could result in the following code, which is perfectly safe in
+ single-threaded code, but can be fatal in concurrent code:
+
+ while (a)
+ do_something_with(a);
+
+ For example, the optimized version of this code could result in
+ passing a zero to do_something_with() in the case where the variable
+ a was modified by some other CPU between the "while" statement and
+ the call to do_something_with().
+
+ Again, use ACCESS_ONCE() to prevent the compiler from doing this:
+
+ while (tmp = ACCESS_ONCE(a))
+ do_something_with(tmp);
+
+ Note that if the compiler runs short of registers, it might save
+ tmp onto the stack. The overhead of this saving and later restoring
+ is why compilers reload variables. Doing so is perfectly safe for
+ single-threaded code, so you need to tell the compiler about cases
+ where it is not safe.
+
+ (*) The compiler is within its rights to omit a load entirely if it knows
+ what the value will be. For example, if the compiler can prove that
+ the value of variable 'a' is always zero, it can optimize this code:
+
+ while (tmp = a)
+ do_something_with(tmp);
-The compiler barrier has no direct effect on the CPU, which may then reorder
-things however it wishes.
+ Into this:
+
+ do { } while (0);
+
+ This transformation is a win for single-threaded code because it gets
+ rid of a load and a branch. The problem is that the compiler will
+ carry out its proof assuming that the current CPU is the only one
+ updating variable 'a'. If variable 'a' is shared, then the compiler's
+ proof will be erroneous. Use ACCESS_ONCE() to tell the compiler
+ that it doesn't know as much as it thinks it does:
+
+ while (tmp = ACCESS_ONCE(a))
+ do_something_with(tmp);
+
+ But please note that the compiler is also closely watching what you
+ do with the value after the ACCESS_ONCE(). For example, suppose you
+ do the following and MAX is a preprocessor macro with the value 1:
+
+ while ((tmp = ACCESS_ONCE(a)) % MAX)
+ do_something_with(tmp);
+
+ Then the compiler knows that the result of the "%" operator applied
+ to MAX will always be zero, again allowing the compiler to optimize
+ the code into near-nonexistence. (It will still load from the
+ variable 'a'.)
+
+ (*) Similarly, the compiler is within its rights to omit a store entirely
+ if it knows that the variable already has the value being stored.
+ Again, the compiler assumes that the current CPU is the only one
+ storing into the variable, which can cause the compiler to do the
+ wrong thing for shared variables. For example, suppose you have
+ the following:
+
+ a = 0;
+ /* Code that does not store to variable a. */
+ a = 0;
+
+ The compiler sees that the value of variable 'a' is already zero, so
+ it might well omit the second store. This would come as a fatal
+ surprise if some other CPU might have stored to variable 'a' in the
+ meantime.
+
+ Use ACCESS_ONCE() to prevent the compiler from making this sort of
+ wrong guess:
+
+ ACCESS_ONCE(a) = 0;
+ /* Code that does not store to variable a. */
+ ACCESS_ONCE(a) = 0;
+
+ (*) The compiler is within its rights to reorder memory accesses unless
+ you tell it not to. For example, consider the following interaction
+ between process-level code and an interrupt handler:
+
+ void process_level(void)
+ {
+ msg = get_message();
+ flag = true;
+ }
+
+ void interrupt_handler(void)
+ {
+ if (flag)
+ process_message(msg);
+ }
+
+ There is nothing to prevent the the compiler from transforming
+ process_level() to the following, in fact, this might well be a
+ win for single-threaded code:
+
+ void process_level(void)
+ {
+ flag = true;
+ msg = get_message();
+ }
+
+ If the interrupt occurs between these two statement, then
+ interrupt_handler() might be passed a garbled msg. Use ACCESS_ONCE()
+ to prevent this as follows:
+
+ void process_level(void)
+ {
+ ACCESS_ONCE(msg) = get_message();
+ ACCESS_ONCE(flag) = true;
+ }
+
+ void interrupt_handler(void)
+ {
+ if (ACCESS_ONCE(flag))
+ process_message(ACCESS_ONCE(msg));
+ }
+
+ Note that the ACCESS_ONCE() wrappers in interrupt_handler()
+ are needed if this interrupt handler can itself be interrupted
+ by something that also accesses 'flag' and 'msg', for example,
+ a nested interrupt or an NMI. Otherwise, ACCESS_ONCE() is not
+ needed in interrupt_handler() other than for documentation purposes.
+ (Note also that nested interrupts do not typically occur in modern
+ Linux kernels, in fact, if an interrupt handler returns with
+ interrupts enabled, you will get a WARN_ONCE() splat.)
+
+ You should assume that the compiler can move ACCESS_ONCE() past
+ code not containing ACCESS_ONCE(), barrier(), or similar primitives.
+
+ This effect could also be achieved using barrier(), but ACCESS_ONCE()
+ is more selective: With ACCESS_ONCE(), the compiler need only forget
+ the contents of the indicated memory locations, while with barrier()
+ the compiler must discard the value of all memory locations that
+ it has currented cached in any machine registers. Of course,
+ the compiler must also respect the order in which the ACCESS_ONCE()s
+ occur, though the CPU of course need not do so.
+
+ (*) The compiler is within its rights to invent stores to a variable,
+ as in the following example:
+
+ if (a)
+ b = a;
+ else
+ b = 42;
+
+ The compiler might save a branch by optimizing this as follows:
+
+ b = 42;
+ if (a)
+ b = a;
+
+ In single-threaded code, this is not only safe, but also saves
+ a branch. Unfortunately, in concurrent code, this optimization
+ could cause some other CPU to see a spurious value of 42 -- even
+ if variable 'a' was never zero -- when loading variable 'b'.
+ Use ACCESS_ONCE() to prevent this as follows:
+
+ if (a)
+ ACCESS_ONCE(b) = a;
+ else
+ ACCESS_ONCE(b) = 42;
+
+ The compiler can also invent loads. These are usually less
+ damaging, but they can result in cache-line bouncing and thus in
+ poor performance and scalability. Use ACCESS_ONCE() to prevent
+ invented loads.
+
+ (*) For aligned memory locations whose size allows them to be accessed
+ with a single memory-reference instruction, prevents "load tearing"
+ and "store tearing," in which a single large access is replaced by
+ multiple smaller accesses. For example, given an architecture having
+ 16-bit store instructions with 7-bit immediate fields, the compiler
+ might be tempted to use two 16-bit store-immediate instructions to
+ implement the following 32-bit store:
+
+ p = 0x00010002;
+
+ Please note that GCC really does use this sort of optimization,
+ which is not surprising given that it would likely take more
+ than two instructions to build the constant and then store it.
+ This optimization can therefore be a win in single-threaded code.
+ In fact, a recent bug (since fixed) caused GCC to incorrectly use
+ this optimization in a volatile store. In the absence of such bugs,
+ use of ACCESS_ONCE() prevents store tearing in the following example:
+
+ ACCESS_ONCE(p) = 0x00010002;
+
+ Use of packed structures can also result in load and store tearing,
+ as in this example:
+
+ struct __attribute__((__packed__)) foo {
+ short a;
+ int b;
+ short c;
+ };
+ struct foo foo1, foo2;
+ ...
+
+ foo2.a = foo1.a;
+ foo2.b = foo1.b;
+ foo2.c = foo1.c;
+
+ Because there are no ACCESS_ONCE() wrappers and no volatile markings,
+ the compiler would be well within its rights to implement these three
+ assignment statements as a pair of 32-bit loads followed by a pair
+ of 32-bit stores. This would result in load tearing on 'foo1.b'
+ and store tearing on 'foo2.b'. ACCESS_ONCE() again prevents tearing
+ in this example:
+
+ foo2.a = foo1.a;
+ ACCESS_ONCE(foo2.b) = ACCESS_ONCE(foo1.b);
+ foo2.c = foo1.c;
+
+All that aside, it is never necessary to use ACCESS_ONCE() on a variable
+that has been marked volatile. For example, because 'jiffies' is marked
+volatile, it is never necessary to say ACCESS_ONCE(jiffies). The reason
+for this is that ACCESS_ONCE() is implemented as a volatile cast, which
+has no effect when its argument is already marked volatile.
+
+Please note that these compiler barriers have no direct effect on the CPU,
+which may then reorder things however it wishes.
CPU MEMORY BARRIERS
clear_bit( ... );
This prevents memory operations before the clear leaking to after it. See
- the subsection on "Locking Functions" with reference to UNLOCK operation
+ the subsection on "Locking Functions" with reference to RELEASE operation
implications.
See Documentation/atomic_ops.txt for more information. See the "Atomic
of arch specific code.
-LOCKING FUNCTIONS
------------------
+ACQUIRING FUNCTIONS
+-------------------
The Linux kernel has a number of locking constructs:
(*) R/W semaphores
(*) RCU
-In all cases there are variants on "LOCK" operations and "UNLOCK" operations
+In all cases there are variants on "ACQUIRE" operations and "RELEASE" operations
for each construct. These operations all imply certain barriers:
- (1) LOCK operation implication:
+ (1) ACQUIRE operation implication:
- Memory operations issued after the LOCK will be completed after the LOCK
- operation has completed.
+ Memory operations issued after the ACQUIRE will be completed after the
+ ACQUIRE operation has completed.
- Memory operations issued before the LOCK may be completed after the LOCK
- operation has completed.
+ Memory operations issued before the ACQUIRE may be completed after the
+ ACQUIRE operation has completed. An smp_mb__before_spinlock(), combined
+ with a following ACQUIRE, orders prior loads against subsequent stores and
+ stores and prior stores against subsequent stores. Note that this is
+ weaker than smp_mb()! The smp_mb__before_spinlock() primitive is free on
+ many architectures.
- (2) UNLOCK operation implication:
+ (2) RELEASE operation implication:
- Memory operations issued before the UNLOCK will be completed before the
- UNLOCK operation has completed.
+ Memory operations issued before the RELEASE will be completed before the
+ RELEASE operation has completed.
- Memory operations issued after the UNLOCK may be completed before the
- UNLOCK operation has completed.
+ Memory operations issued after the RELEASE may be completed before the
+ RELEASE operation has completed.
- (3) LOCK vs LOCK implication:
+ (3) ACQUIRE vs ACQUIRE implication:
- All LOCK operations issued before another LOCK operation will be completed
- before that LOCK operation.
+ All ACQUIRE operations issued before another ACQUIRE operation will be
+ completed before that ACQUIRE operation.
- (4) LOCK vs UNLOCK implication:
+ (4) ACQUIRE vs RELEASE implication:
- All LOCK operations issued before an UNLOCK operation will be completed
- before the UNLOCK operation.
+ All ACQUIRE operations issued before a RELEASE operation will be
+ completed before the RELEASE operation.
- All UNLOCK operations issued before a LOCK operation will be completed
- before the LOCK operation.
+ (5) Failed conditional ACQUIRE implication:
- (5) Failed conditional LOCK implication:
-
- Certain variants of the LOCK operation may fail, either due to being
- unable to get the lock immediately, or due to receiving an unblocked
+ Certain locking variants of the ACQUIRE operation may fail, either due to
+ being unable to get the lock immediately, or due to receiving an unblocked
signal whilst asleep waiting for the lock to become available. Failed
locks do not imply any sort of barrier.
-Therefore, from (1), (2) and (4) an UNLOCK followed by an unconditional LOCK is
-equivalent to a full barrier, but a LOCK followed by an UNLOCK is not.
-
-[!] Note: one of the consequences of LOCKs and UNLOCKs being only one-way
- barriers is that the effects of instructions outside of a critical section
- may seep into the inside of the critical section.
+[!] Note: one of the consequences of lock ACQUIREs and RELEASEs being only
+one-way barriers is that the effects of instructions outside of a critical
+section may seep into the inside of the critical section.
-A LOCK followed by an UNLOCK may not be assumed to be full memory barrier
-because it is possible for an access preceding the LOCK to happen after the
-LOCK, and an access following the UNLOCK to happen before the UNLOCK, and the
-two accesses can themselves then cross:
+An ACQUIRE followed by a RELEASE may not be assumed to be full memory barrier
+because it is possible for an access preceding the ACQUIRE to happen after the
+ACQUIRE, and an access following the RELEASE to happen before the RELEASE, and
+the two accesses can themselves then cross:
*A = a;
- LOCK
- UNLOCK
+ ACQUIRE M
+ RELEASE M
*B = b;
may occur as:
- LOCK, STORE *B, STORE *A, UNLOCK
+ ACQUIRE M, STORE *B, STORE *A, RELEASE M
+
+This same reordering can of course occur if the lock's ACQUIRE and RELEASE are
+to the same lock variable, but only from the perspective of another CPU not
+holding that lock.
+
+In short, a RELEASE followed by an ACQUIRE may -not- be assumed to be a full
+memory barrier because it is possible for a preceding RELEASE to pass a
+later ACQUIRE from the viewpoint of the CPU, but not from the viewpoint
+of the compiler. Note that deadlocks cannot be introduced by this
+interchange because if such a deadlock threatened, the RELEASE would
+simply complete.
+
+If it is necessary for a RELEASE-ACQUIRE pair to produce a full barrier, the
+ACQUIRE can be followed by an smp_mb__after_unlock_lock() invocation. This
+will produce a full barrier if either (a) the RELEASE and the ACQUIRE are
+executed by the same CPU or task, or (b) the RELEASE and ACQUIRE act on the
+same variable. The smp_mb__after_unlock_lock() primitive is free on many
+architectures. Without smp_mb__after_unlock_lock(), the critical sections
+corresponding to the RELEASE and the ACQUIRE can cross:
+
+ *A = a;
+ RELEASE M
+ ACQUIRE N
+ *B = b;
+
+could occur as:
+
+ ACQUIRE N, STORE *B, STORE *A, RELEASE M
+
+With smp_mb__after_unlock_lock(), they cannot, so that:
+
+ *A = a;
+ RELEASE M
+ ACQUIRE N
+ smp_mb__after_unlock_lock();
+ *B = b;
+
+will always occur as either of the following:
+
+ STORE *A, RELEASE, ACQUIRE, STORE *B
+ STORE *A, ACQUIRE, RELEASE, STORE *B
+
+If the RELEASE and ACQUIRE were instead both operating on the same lock
+variable, only the first of these two alternatives can occur.
Locks and semaphores may not provide any guarantee of ordering on UP compiled
systems, and so cannot be counted on in such a situation to actually achieve
*A = a;
*B = b;
- LOCK
+ ACQUIRE
*C = c;
*D = d;
- UNLOCK
+ RELEASE
*E = e;
*F = f;
The following sequence of events is acceptable:
- LOCK, {*F,*A}, *E, {*C,*D}, *B, UNLOCK
+ ACQUIRE, {*F,*A}, *E, {*C,*D}, *B, RELEASE
[+] Note that {*F,*A} indicates a combined access.
But none of the following are:
- {*F,*A}, *B, LOCK, *C, *D, UNLOCK, *E
- *A, *B, *C, LOCK, *D, UNLOCK, *E, *F
- *A, *B, LOCK, *C, UNLOCK, *D, *E, *F
- *B, LOCK, *C, *D, UNLOCK, {*F,*A}, *E
+ {*F,*A}, *B, ACQUIRE, *C, *D, RELEASE, *E
+ *A, *B, *C, ACQUIRE, *D, RELEASE, *E, *F
+ *A, *B, ACQUIRE, *C, RELEASE, *D, *E, *F
+ *B, ACQUIRE, *C, *D, RELEASE, {*F,*A}, *E
INTERRUPT DISABLING FUNCTIONS
-----------------------------
-Functions that disable interrupts (LOCK equivalent) and enable interrupts
-(UNLOCK equivalent) will act as compiler barriers only. So if memory or I/O
+Functions that disable interrupts (ACQUIRE equivalent) and enable interrupts
+(RELEASE equivalent) will act as compiler barriers only. So if memory or I/O
barriers are required in such a situation, they must be provided from some
other means.
(*) schedule() and similar imply full memory barriers.
-=================================
-INTER-CPU LOCKING BARRIER EFFECTS
-=================================
+===================================
+INTER-CPU ACQUIRING BARRIER EFFECTS
+===================================
On SMP systems locking primitives give a more substantial form of barrier: one
that does affect memory access ordering on other CPUs, within the context of
conflict on any particular lock.
-LOCKS VS MEMORY ACCESSES
-------------------------
+ACQUIRES VS MEMORY ACCESSES
+---------------------------
Consider the following: the system has a pair of spinlocks (M) and (Q), and
three CPUs; then should the following sequence of events occur:
CPU 1 CPU 2
=============================== ===============================
- *A = a; *E = e;
- LOCK M LOCK Q
- *B = b; *F = f;
- *C = c; *G = g;
- UNLOCK M UNLOCK Q
- *D = d; *H = h;
+ ACCESS_ONCE(*A) = a; ACCESS_ONCE(*E) = e;
+ ACQUIRE M ACQUIRE Q
+ ACCESS_ONCE(*B) = b; ACCESS_ONCE(*F) = f;
+ ACCESS_ONCE(*C) = c; ACCESS_ONCE(*G) = g;
+ RELEASE M RELEASE Q
+ ACCESS_ONCE(*D) = d; ACCESS_ONCE(*H) = h;
Then there is no guarantee as to what order CPU 3 will see the accesses to *A
through *H occur in, other than the constraints imposed by the separate locks
on the separate CPUs. It might, for example, see:
- *E, LOCK M, LOCK Q, *G, *C, *F, *A, *B, UNLOCK Q, *D, *H, UNLOCK M
+ *E, ACQUIRE M, ACQUIRE Q, *G, *C, *F, *A, *B, RELEASE Q, *D, *H, RELEASE M
But it won't see any of:
- *B, *C or *D preceding LOCK M
- *A, *B or *C following UNLOCK M
- *F, *G or *H preceding LOCK Q
- *E, *F or *G following UNLOCK Q
+ *B, *C or *D preceding ACQUIRE M
+ *A, *B or *C following RELEASE M
+ *F, *G or *H preceding ACQUIRE Q
+ *E, *F or *G following RELEASE Q
However, if the following occurs:
CPU 1 CPU 2
=============================== ===============================
- *A = a;
- LOCK M [1]
- *B = b;
- *C = c;
- UNLOCK M [1]
- *D = d; *E = e;
- LOCK M [2]
- *F = f;
- *G = g;
- UNLOCK M [2]
- *H = h;
+ ACCESS_ONCE(*A) = a;
+ ACQUIRE M [1]
+ ACCESS_ONCE(*B) = b;
+ ACCESS_ONCE(*C) = c;
+ RELEASE M [1]
+ ACCESS_ONCE(*D) = d; ACCESS_ONCE(*E) = e;
+ ACQUIRE M [2]
+ smp_mb__after_unlock_lock();
+ ACCESS_ONCE(*F) = f;
+ ACCESS_ONCE(*G) = g;
+ RELEASE M [2]
+ ACCESS_ONCE(*H) = h;
CPU 3 might see:
- *E, LOCK M [1], *C, *B, *A, UNLOCK M [1],
- LOCK M [2], *H, *F, *G, UNLOCK M [2], *D
+ *E, ACQUIRE M [1], *C, *B, *A, RELEASE M [1],
+ ACQUIRE M [2], *H, *F, *G, RELEASE M [2], *D
But assuming CPU 1 gets the lock first, CPU 3 won't see any of:
- *B, *C, *D, *F, *G or *H preceding LOCK M [1]
- *A, *B or *C following UNLOCK M [1]
- *F, *G or *H preceding LOCK M [2]
- *A, *B, *C, *E, *F or *G following UNLOCK M [2]
+ *B, *C, *D, *F, *G or *H preceding ACQUIRE M [1]
+ *A, *B or *C following RELEASE M [1]
+ *F, *G or *H preceding ACQUIRE M [2]
+ *A, *B, *C, *E, *F or *G following RELEASE M [2]
+Note that the smp_mb__after_unlock_lock() is critically important
+here: Without it CPU 3 might see some of the above orderings.
+Without smp_mb__after_unlock_lock(), the accesses are not guaranteed
+to be seen in order unless CPU 3 holds lock M.
-LOCKS VS I/O ACCESSES
----------------------
+
+ACQUIRES VS I/O ACCESSES
+------------------------
Under certain circumstances (especially involving NUMA), I/O accesses within
two spinlocked sections on two different CPUs may be seen as interleaved by the
xchg();
cmpxchg();
- atomic_xchg();
- atomic_cmpxchg();
- atomic_inc_return();
- atomic_dec_return();
- atomic_add_return();
- atomic_sub_return();
- atomic_inc_and_test();
- atomic_dec_and_test();
- atomic_sub_and_test();
- atomic_add_negative();
- atomic_add_unless(); /* when succeeds (returns 1) */
+ atomic_xchg(); atomic_long_xchg();
+ atomic_cmpxchg(); atomic_long_cmpxchg();
+ atomic_inc_return(); atomic_long_inc_return();
+ atomic_dec_return(); atomic_long_dec_return();
+ atomic_add_return(); atomic_long_add_return();
+ atomic_sub_return(); atomic_long_sub_return();
+ atomic_inc_and_test(); atomic_long_inc_and_test();
+ atomic_dec_and_test(); atomic_long_dec_and_test();
+ atomic_sub_and_test(); atomic_long_sub_and_test();
+ atomic_add_negative(); atomic_long_add_negative();
test_and_set_bit();
test_and_clear_bit();
test_and_change_bit();
-These are used for such things as implementing LOCK-class and UNLOCK-class
+ /* when succeeds (returns 1) */
+ atomic_add_unless(); atomic_long_add_unless();
+
+These are used for such things as implementing ACQUIRE-class and RELEASE-class
operations and adjusting reference counters towards object destruction, and as
such the implicit memory barrier effects are necessary.
The following operations are potential problems as they do _not_ imply memory
-barriers, but might be used for implementing such things as UNLOCK-class
+barriers, but might be used for implementing such things as RELEASE-class
operations:
atomic_set();
clear_bit_unlock();
__clear_bit_unlock();
-These implement LOCK-class and UNLOCK-class operations. These should be used in
+These implement ACQUIRE-class and RELEASE-class operations. These should be used in
preference to other operations when implementing locking primitives, because
their implementations can be optimised on many architectures.
space should suffice for PCI.
[*] NOTE! attempting to load from the same location as was written to may
- cause a malfunction - consider the 16550 Rx/Tx serial registers for
- example.
+ cause a malfunction - consider the 16550 Rx/Tx serial registers for
+ example.
Used with prefetchable I/O memory, an mmiowb() barrier may be required to
force stores to be ordered.
:
+--------+ +--------+ : +--------+ +-----------+
| | | | : | | | | +--------+
- | CPU | | Memory | : | CPU | | | | |
- | Core |--->| Access |----->| Cache |<-->| | | |
+ | CPU | | Memory | : | CPU | | | | |
+ | Core |--->| Access |----->| Cache |<-->| | | |
| | | Queue | : | | | |--->| Memory |
- | | | | : | | | | | |
- +--------+ +--------+ : +--------+ | | | |
+ | | | | : | | | | | |
+ +--------+ +--------+ : +--------+ | | | |
: | Cache | +--------+
: | Coherency |
: | Mechanism | +--------+
+--------+ +--------+ : +--------+ | | | |
| | | | : | | | | | |
| CPU | | Memory | : | CPU | | |--->| Device |
- | Core |--->| Access |----->| Cache |<-->| | | |
- | | | Queue | : | | | | | |
+ | Core |--->| Access |----->| Cache |<-->| | | |
+ | | | Queue | : | | | | | |
| | | | : | | | | +--------+
+--------+ +--------+ : +--------+ +-----------+
:
p = &v; q = p;
<D:request p>
<B:modify p=&v> <D:commit p=&v>
- <D:read p>
+ <D:read p>
x = *q;
<C:read *q> Reads from v before v updated in cache
<C:unbusy>
p = &v; q = p;
<D:request p>
<B:modify p=&v> <D:commit p=&v>
- <D:read p>
+ <D:read p>
smp_read_barrier_depends()
<C:unbusy>
<C:commit v=2>
operations in exactly the order specified, so that if the CPU is, for example,
given the following piece of code to execute:
- a = *A;
- *B = b;
- c = *C;
- d = *D;
- *E = e;
+ a = ACCESS_ONCE(*A);
+ ACCESS_ONCE(*B) = b;
+ c = ACCESS_ONCE(*C);
+ d = ACCESS_ONCE(*D);
+ ACCESS_ONCE(*E) = e;
they would then expect that the CPU will complete the memory operation for each
instruction before moving on to the next one, leading to a definite sequence of
_own_ accesses appear to be correctly ordered, without the need for a memory
barrier. For instance with the following code:
- U = *A;
- *A = V;
- *A = W;
- X = *A;
- *A = Y;
- Z = *A;
+ U = ACCESS_ONCE(*A);
+ ACCESS_ONCE(*A) = V;
+ ACCESS_ONCE(*A) = W;
+ X = ACCESS_ONCE(*A);
+ ACCESS_ONCE(*A) = Y;
+ Z = ACCESS_ONCE(*A);
and assuming no intervention by an external influence, it can be assumed that
the final result will appear to be:
in that order, but, without intervention, the sequence may have almost any
combination of elements combined or discarded, provided the program's view of
-the world remains consistent.
+the world remains consistent. Note that ACCESS_ONCE() is -not- optional
+in the above example, as there are architectures where a given CPU might
+interchange successive loads to the same location. On such architectures,
+ACCESS_ONCE() does whatever is necessary to prevent this, for example, on
+Itanium the volatile casts used by ACCESS_ONCE() cause GCC to emit the
+special ld.acq and st.rel instructions that prevent such reordering.
The compiler may also combine, discard or defer elements of the sequence before
the CPU even sees them.
*A = W;
-since, without a write barrier, it can be assumed that the effect of the
-storage of V to *A is lost. Similarly:
+since, without either a write barrier or an ACCESS_ONCE(), it can be
+assumed that the effect of the storage of V to *A is lost. Similarly:
*A = Y;
Z = *A;
-may, without a memory barrier, be reduced to:
+may, without a memory barrier or an ACCESS_ONCE(), be reduced to:
*A = Y;
Z = Y;
1) set the 'list_op_pending' word to the address of the 'lock entry'
to be removed,
2) remove the lock entry for this lock from the 'head' list,
- 2) release the futex lock, and
- 2) clear the 'lock_op_pending' word.
+ 3) release the futex lock, and
+ 4) clear the 'lock_op_pending' word.
On exit, the kernel will consider the address stored in
'list_op_pending' and the address of each 'lock word' found by walking
numa_balancing_scan_size_mb is how many megabytes worth of pages are
scanned for a given scan.
-numa_balancing_settle_count is how many scan periods must complete before
-the schedule balancer stops pushing the task towards a preferred node. This
-gives the scheduler a chance to place the task on an alternative node if the
-preferred node is overloaded.
-
numa_balancing_migrate_deferred is how many page migrations get skipped
unconditionally, after a page migration is skipped because a page is shared
with other tasks. This reduces page migration overhead, and determines
- If 1, the kernel supports the 64-bit EFI handoff entry point
given at handover_offset + 0x200.
+ Bit 4 (read): XLF_EFI_KEXEC
+ - If 1, the kernel supports kexec EFI boot with EFI runtime support.
+
Field name: cmdline_size
Type: read
Offset/size: 0x238/4
Current X86-64 implementations only support 40 bits of address space,
but we support up to 46 bits. This expands into MBZ space in the page tables.
+->trampoline_pgd:
+
+We map EFI runtime services in the aforementioned PGD in the virtual
+range of 64Gb (arbitrarily set, can be raised if needed)
+
+0xffffffef00000000 - 0xffffffff00000000
+
-Andi Kleen, Jul 2004
-------------
linux/include/linux/zorro.h
-linux/include/asm-{m68k,ppc}/zorro.h
-linux/include/linux/zorro_ids.h
+linux/include/uapi/linux/zorro.h
+linux/include/uapi/linux/zorro_ids.h
+linux/arch/m68k/include/asm/zorro.h
linux/drivers/zorro
/proc/bus/zorro
F: include/linux/lguest*.h
F: tools/lguest/
+LIBLOCKDEP
+M: Sasha Levin <sasha.levin@oracle.com>
+S: Maintained
+F: tools/lib/lockdep/
+
LINUX FOR IBM pSERIES (RS/6000)
M: Paul Mackerras <paulus@au.ibm.com>
W: http://www.ibm.com/linux/ltc/projects/ppc
F: Documentation/RCU/torture.txt
F: kernel/rcu/torture.c
+RCUTORTURE TEST FRAMEWORK
+M: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
+S: Supported
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
+F: tools/testing/selftests/rcutorture
+
RDC R-321X SoC
M: Florian Fainelli <florian@openwrt.org>
S: Maintained
VIRTIO CONSOLE DRIVER
M: Amit Shah <amit.shah@redhat.com>
+L: virtio-dev@lists.oasis-open.org
L: virtualization@lists.linux-foundation.org
S: Maintained
F: drivers/char/virtio_console.c
VIRTIO CORE, NET AND BLOCK DRIVERS
M: Rusty Russell <rusty@rustcorp.com.au>
M: "Michael S. Tsirkin" <mst@redhat.com>
+L: virtio-dev@lists.oasis-open.org
L: virtualization@lists.linux-foundation.org
S: Maintained
F: drivers/virtio/
VIRTIO HOST (VHOST)
M: "Michael S. Tsirkin" <mst@redhat.com>
L: kvm@vger.kernel.org
+L: virtio-dev@lists.oasis-open.org
L: virtualization@lists.linux-foundation.org
L: netdev@vger.kernel.org
S: Maintained
VERSION = 3
PATCHLEVEL = 13
SUBLEVEL = 0
-EXTRAVERSION = -rc8
+EXTRAVERSION =
NAME = One Giant Leap for Frogkind
# *DOCUMENTATION*
KBUILD_CFLAGS += $(call cc-option,-Wframe-larger-than=${CONFIG_FRAME_WARN})
endif
-# Force gcc to behave correct even for buggy distributions
-ifndef CONFIG_CC_STACKPROTECTOR
-KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector)
+# Handle stack protector mode.
+ifdef CONFIG_CC_STACKPROTECTOR_REGULAR
+ stackp-flag := -fstack-protector
+ ifeq ($(call cc-option, $(stackp-flag)),)
+ $(warning Cannot use CONFIG_CC_STACKPROTECTOR: \
+ -fstack-protector not supported by compiler))
+ endif
+else ifdef CONFIG_CC_STACKPROTECTOR_STRONG
+ stackp-flag := -fstack-protector-strong
+ ifeq ($(call cc-option, $(stackp-flag)),)
+ $(warning Cannot use CONFIG_CC_STACKPROTECTOR_STRONG: \
+ -fstack-protector-strong not supported by compiler)
+ endif
+else
+ # Force off for distro compilers that enable stack protector by default.
+ stackp-flag := $(call cc-option, -fno-stack-protector)
endif
+KBUILD_CFLAGS += $(stackp-flag)
# This warning generated too much noise in a regular build.
# Use make W=1 to enable this warning (see scripts/Makefile.build)
See Documentation/prctl/seccomp_filter.txt for details.
+config HAVE_CC_STACKPROTECTOR
+ bool
+ help
+ An arch should select this symbol if:
+ - its compiler supports the -fstack-protector option
+ - it has implemented a stack canary (e.g. __stack_chk_guard)
+
+config CC_STACKPROTECTOR
+ def_bool n
+ help
+ Set when a stack-protector mode is enabled, so that the build
+ can enable kernel-side support for the GCC feature.
+
+choice
+ prompt "Stack Protector buffer overflow detection"
+ depends on HAVE_CC_STACKPROTECTOR
+ default CC_STACKPROTECTOR_NONE
+ help
+ This option turns on the "stack-protector" GCC feature. This
+ feature puts, at the beginning of functions, a canary value on
+ the stack just before the return address, and validates
+ the value just before actually returning. Stack based buffer
+ overflows (that need to overwrite this return address) now also
+ overwrite the canary, which gets detected and the attack is then
+ neutralized via a kernel panic.
+
+config CC_STACKPROTECTOR_NONE
+ bool "None"
+ help
+ Disable "stack-protector" GCC feature.
+
+config CC_STACKPROTECTOR_REGULAR
+ bool "Regular"
+ select CC_STACKPROTECTOR
+ help
+ Functions will have the stack-protector canary logic added if they
+ have an 8-byte or larger character array on the stack.
+
+ This feature requires gcc version 4.2 or above, or a distribution
+ gcc with the feature backported ("-fstack-protector").
+
+ On an x86 "defconfig" build, this feature adds canary checks to
+ about 3% of all kernel functions, which increases kernel code size
+ by about 0.3%.
+
+config CC_STACKPROTECTOR_STRONG
+ bool "Strong"
+ select CC_STACKPROTECTOR
+ help
+ Functions will have the stack-protector canary logic added in any
+ of the following conditions:
+
+ - local variable's address used as part of the right hand side of an
+ assignment or function argument
+ - local variable is an array (or union containing an array),
+ regardless of array type or length
+ - uses register local variables
+
+ This feature requires gcc version 4.9 or above, or a distribution
+ gcc with the feature backported ("-fstack-protector-strong").
+
+ On an x86 "defconfig" build, this feature adds canary checks to
+ about 20% of all kernel functions, which increases the kernel code
+ size by about 2%.
+
+endchoice
+
config HAVE_CONTEXT_TRACKING
bool
help
#include <asm/compiler.h>
-#define mb() \
-__asm__ __volatile__("mb": : :"memory")
+#define mb() __asm__ __volatile__("mb": : :"memory")
+#define rmb() __asm__ __volatile__("mb": : :"memory")
+#define wmb() __asm__ __volatile__("wmb": : :"memory")
-#define rmb() \
-__asm__ __volatile__("mb": : :"memory")
-
-#define wmb() \
-__asm__ __volatile__("wmb": : :"memory")
-
-#define read_barrier_depends() \
-__asm__ __volatile__("mb": : :"memory")
+#define read_barrier_depends() __asm__ __volatile__("mb": : :"memory")
#ifdef CONFIG_SMP
#define __ASM_SMP_MB "\tmb\n"
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
#else
#define __ASM_SMP_MB
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while (0)
#endif
-#define set_mb(var, value) \
-do { var = value; mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* __BARRIER_H */
generic-y += auxvec.h
+generic-y += barrier.h
generic-y += bugs.h
generic-y += bitsperlong.h
generic-y += clkdev.h
#endif /* !CONFIG_ARC_HAS_LLSC */
+#define smp_mb__before_atomic_dec() barrier()
+#define smp_mb__after_atomic_dec() barrier()
+#define smp_mb__before_atomic_inc() barrier()
+#define smp_mb__after_atomic_inc() barrier()
+
/**
* __atomic_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
#define smp_wmb() barrier()
#endif
-#define smp_mb__before_atomic_dec() barrier()
-#define smp_mb__after_atomic_dec() barrier()
-#define smp_mb__before_atomic_inc() barrier()
-#define smp_mb__after_atomic_inc() barrier()
-
#define smp_read_barrier_depends() do { } while (0)
#endif
select HAVE_BPF_JIT
select HAVE_CONTEXT_TRACKING
select HAVE_C_RECORDMCOUNT
+ select HAVE_CC_STACKPROTECTOR
select HAVE_DEBUG_KMEMLEAK
select HAVE_DMA_API_DEBUG
select HAVE_DMA_ATTRS
and the task is only allowed to execute a few safe syscalls
defined by each seccomp mode.
-config CC_STACKPROTECTOR
- bool "Enable -fstack-protector buffer overflow detection (EXPERIMENTAL)"
- help
- This option turns on the -fstack-protector GCC feature. This
- feature puts, at the beginning of functions, a canary value on
- the stack just before the return address, and validates
- the value just before actually returning. Stack based buffer
- overflows (that need to overwrite this return address) now also
- overwrite the canary, which gets detected and the attack is then
- neutralized via a kernel panic.
- This feature requires gcc version 4.2 or above.
-
config SWIOTLB
def_bool y
KBUILD_CFLAGS +=-fno-omit-frame-pointer -mapcs -mno-sched-prolog
endif
-ifeq ($(CONFIG_CC_STACKPROTECTOR),y)
-KBUILD_CFLAGS +=-fstack-protector
-endif
-
ifeq ($(CONFIG_CPU_BIG_ENDIAN),y)
KBUILD_CPPFLAGS += -mbig-endian
AS += -EB
error("Attempting division by 0!");
}
+unsigned long __stack_chk_guard;
+
+void __stack_chk_guard_setup(void)
+{
+ __stack_chk_guard = 0x000a0dff;
+}
+
+void __stack_chk_fail(void)
+{
+ error("stack-protector: Kernel stack is corrupted\n");
+}
+
extern int do_decompress(u8 *input, int len, u8 *output, void (*error)(char *x));
{
int ret;
+ __stack_chk_guard_setup();
+
output_data = (unsigned char *)output_start;
free_mem_ptr = free_mem_ptr_p;
free_mem_end_ptr = free_mem_ptr_end_p;
clock-frequency = <100000>;
status = "disabled";
};
+
+ timer@01c60000 {
+ compatible = "allwinner,sun5i-a13-hstimer";
+ reg = <0x01c60000 0x1000>;
+ interrupts = <82>, <83>;
+ clocks = <&ahb_gates 28>;
+ };
};
};
clock-frequency = <100000>;
status = "disabled";
};
+
+ timer@01c60000 {
+ compatible = "allwinner,sun5i-a13-hstimer";
+ reg = <0x01c60000 0x1000>;
+ interrupts = <82>, <83>;
+ clocks = <&ahb_gates 28>;
+ };
};
};
status = "disabled";
};
+ hstimer@01c60000 {
+ compatible = "allwinner,sun7i-a20-hstimer";
+ reg = <0x01c60000 0x1000>;
+ interrupts = <0 81 1>,
+ <0 82 1>,
+ <0 83 1>,
+ <0 84 1>;
+ clocks = <&ahb_gates 28>;
+ };
+
gic: interrupt-controller@01c81000 {
compatible = "arm,cortex-a7-gic", "arm,cortex-a15-gic";
reg = <0x01c81000 0x1000>,
#define smp_wmb() dmb(ishst)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#define read_barrier_depends() do { } while(0)
#define smp_read_barrier_depends() do { } while(0)
#include <uapi/asm/unistd.h>
-#define __NR_syscalls (380)
+#define __NR_syscalls (384)
#define __ARM_NR_cmpxchg (__ARM_NR_BASE+0x00fff0)
#define __ARCH_WANT_STAT64
#define __NR_process_vm_writev (__NR_SYSCALL_BASE+377)
#define __NR_kcmp (__NR_SYSCALL_BASE+378)
#define __NR_finit_module (__NR_SYSCALL_BASE+379)
+#define __NR_sched_setattr (__NR_SYSCALL_BASE+380)
+#define __NR_sched_getattr (__NR_SYSCALL_BASE+381)
/*
* This may need to be greater than __NR_last_syscall+1 in order to
CALL(sys_process_vm_writev)
CALL(sys_kcmp)
CALL(sys_finit_module)
+/* 380 */ CALL(sys_sched_setattr)
+ CALL(sys_sched_getattr)
#ifndef syscalls_counted
.equ syscalls_padding, ((NR_syscalls + 3) & ~3) - NR_syscalls
#define syscalls_counted
bool arch_match_cpu_phys_id(int cpu, u64 phys_id)
{
- return (phys_id & MPIDR_HWID_BITMASK) == cpu_logical_map(cpu);
+ return phys_id == cpu_logical_map(cpu);
}
static const void * __init arch_get_next_mach(const char *const **match)
s64 period = hwc->sample_period;
int ret = 0;
- /* The period may have been changed by PERF_EVENT_IOC_PERIOD */
- if (unlikely(period != hwc->last_period))
- left = period - (hwc->last_period - left);
-
if (unlikely(left <= -period)) {
left = period;
local64_set(&hwc->period_left, left);
static int cpu_pmu_device_probe(struct platform_device *pdev)
{
const struct of_device_id *of_id;
- int (*init_fn)(struct arm_pmu *);
+ const int (*init_fn)(struct arm_pmu *);
struct device_node *node = pdev->dev.of_node;
struct arm_pmu *pmu;
int ret = -ENODEV;
instr2 = __mem_to_opcode_thumb16(instr2);
instr = __opcode_thumb32_compose(instr, instr2);
}
- } else if (get_user(instr, (u32 __user *)pc)) {
+ } else {
+ if (get_user(instr, (u32 __user *)pc))
+ goto die_sig;
instr = __mem_to_opcode_arm(instr);
- goto die_sig;
}
if (call_undef_hook(regs, instr) == 0)
static void highbank_l2x0_disable(void)
{
+ outer_flush_all();
/* Disable PL310 L2 Cache controller */
highbank_smc1(0x102, 0x0);
}
static void omap4_l2x0_disable(void)
{
+ outer_flush_all();
/* Disable PL310 L2 Cache controller */
omap_smc1(0x102, 0x0);
}
select PINCTRL_SUNXI
select SPARSE_IRQ
select SUN4I_TIMER
+ select SUN5I_HSTIMER
#ifdef CONFIG_ZONE_DMA
if (mdesc->dma_zone_size) {
arm_dma_zone_size = mdesc->dma_zone_size;
- arm_dma_limit = __pv_phys_offset + arm_dma_zone_size - 1;
+ arm_dma_limit = PHYS_OFFSET + arm_dma_zone_size - 1;
} else
arm_dma_limit = 0xffffffff;
arm_dma_pfn_limit = arm_dma_limit >> PAGE_SHIFT;
emit(ARM_MUL(r_A, r_A, r_X), ctx);
break;
case BPF_S_ALU_DIV_K:
- /* current k == reciprocal_value(userspace k) */
+ if (k == 1)
+ break;
emit_mov_i(r_scratch, k, ctx);
- /* A = top 32 bits of the product */
- emit(ARM_UMULL(r_scratch, r_A, r_A, r_scratch), ctx);
+ emit_udiv(r_A, r_A, r_scratch, ctx);
break;
case BPF_S_ALU_DIV_X:
update_on_xread(ctx);
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#else
+
#define smp_mb() asm volatile("dmb ish" : : : "memory")
#define smp_rmb() asm volatile("dmb ishld" : : : "memory")
#define smp_wmb() asm volatile("dmb ishst" : : : "memory")
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("stlr %w1, %0" \
+ : "=Q" (*p) : "r" (v) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("stlr %1, %0" \
+ : "=Q" (*p) : "r" (v) : "memory"); \
+ break; \
+ } \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("ldar %w0, %1" \
+ : "=r" (___p1) : "Q" (*p) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("ldar %0, %1" \
+ : "=r" (___p1) : "Q" (*p) : "memory"); \
+ break; \
+ } \
+ ___p1; \
+})
+
#endif
#define read_barrier_depends() do { } while(0)
extern void __iounmap(volatile void __iomem *addr);
extern void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size);
-#define PROT_DEFAULT (pgprot_default | PTE_DIRTY)
+#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_DIRTY)
#define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_ATTRINDX(MT_DEVICE_nGnRE))
#define PROT_NORMAL_NC (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL_NC))
#define PROT_NORMAL (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL))
#ifndef __ASM_AVR32_BARRIER_H
#define __ASM_AVR32_BARRIER_H
-#define nop() asm volatile("nop")
-
-#define mb() asm volatile("" : : : "memory")
-#define rmb() mb()
-#define wmb() asm volatile("sync 0" : : : "memory")
-#define read_barrier_depends() do { } while(0)
-#define set_mb(var, value) do { var = value; mb(); } while(0)
+/*
+ * Weirdest thing ever.. no full barrier, but it has a write barrier!
+ */
+#define wmb() asm volatile("sync 0" : : : "memory")
#ifdef CONFIG_SMP
# error "The AVR32 port does not support SMP"
-#else
-# define smp_mb() barrier()
-# define smp_rmb() barrier()
-# define smp_wmb() barrier()
-# define smp_read_barrier_depends() do { } while(0)
#endif
+#include <asm-generic/barrier.h>
#endif /* __ASM_AVR32_BARRIER_H */
# define rmb() do { barrier(); smp_check_barrier(); } while (0)
# define wmb() do { barrier(); smp_mark_barrier(); } while (0)
# define read_barrier_depends() do { barrier(); smp_check_barrier(); } while (0)
-#else
-# define mb() barrier()
-# define rmb() barrier()
-# define wmb() barrier()
-# define read_barrier_depends() do { } while (0)
#endif
-#else /* !CONFIG_SMP */
-
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define read_barrier_depends() do { } while (0)
-
#endif /* !CONFIG_SMP */
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#define smp_read_barrier_depends() read_barrier_depends()
+#include <asm-generic/barrier.h>
#endif /* _BLACKFIN_BARRIER_H */
header-y += arch-v32/
+generic-y += barrier.h
generic-y += clkdev.h
generic-y += exec.h
generic-y += kvm_para.h
+++ /dev/null
-#ifndef __ASM_CRIS_BARRIER_H
-#define __ASM_CRIS_BARRIER_H
-
-#define nop() __asm__ __volatile__ ("nop");
-
-#define barrier() __asm__ __volatile__("": : :"memory")
-#define mb() barrier()
-#define rmb() mb()
-#define wmb() mb()
-#define read_barrier_depends() do { } while(0)
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while(0)
-#endif
-
-#endif /* __ASM_CRIS_BARRIER_H */
#define mb() asm volatile ("membar" : : :"memory")
#define rmb() asm volatile ("membar" : : :"memory")
#define wmb() asm volatile ("membar" : : :"memory")
-#define read_barrier_depends() do { } while (0)
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do {} while(0)
-#define set_mb(var, value) \
- do { var = (value); barrier(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* _ASM_BARRIER_H */
header-y += ucontext.h
generic-y += auxvec.h
+generic-y += barrier.h
generic-y += bug.h
generic-y += bugs.h
generic-y += clkdev.h
#define atomic_sub_and_test(i, v) (atomic_sub_return(i, (v)) == 0)
#define atomic_add_negative(i, v) (atomic_add_return(i, (v)) < 0)
-
#define atomic_inc_return(v) (atomic_add_return(1, v))
#define atomic_dec_return(v) (atomic_sub_return(1, v))
+#define smp_mb__before_atomic_dec() barrier()
+#define smp_mb__after_atomic_dec() barrier()
+#define smp_mb__before_atomic_inc() barrier()
+#define smp_mb__after_atomic_inc() barrier()
+
#endif
#define smp_read_barrier_depends() barrier()
#define smp_wmb() barrier()
#define smp_mb() barrier()
-#define smp_mb__before_atomic_dec() barrier()
-#define smp_mb__after_atomic_dec() barrier()
-#define smp_mb__before_atomic_inc() barrier()
-#define smp_mb__after_atomic_inc() barrier()
/* Set a value and use a memory barrier. Used by the scheduler somewhere. */
#define set_mb(var, value) \
over full virtualization. However, when run without a hypervisor
the kernel is theoretically slower and slightly larger.
-
-source "arch/ia64/xen/Kconfig"
-
endif
choice
SGI-SN2 For SGI Altix systems
SGI-UV For SGI UV systems
Ski-simulator For the HP simulator <http://www.hpl.hp.com/research/linux/ski/>
- Xen-domU For xen domU system
If you don't know what to do, choose "generic".
bool "Ski-simulator"
select SWIOTLB
-config IA64_XEN_GUEST
- bool "Xen guest"
- select SWIOTLB
- depends on XEN
- help
- Build a kernel that runs on Xen guest domain. At this moment only
- 16KB page size in supported.
-
endchoice
choice
core-$(CONFIG_IA64_GENERIC) += arch/ia64/dig/
core-$(CONFIG_IA64_HP_ZX1) += arch/ia64/dig/
core-$(CONFIG_IA64_HP_ZX1_SWIOTLB) += arch/ia64/dig/
-core-$(CONFIG_IA64_XEN_GUEST) += arch/ia64/dig/
core-$(CONFIG_IA64_SGI_SN2) += arch/ia64/sn/
core-$(CONFIG_IA64_SGI_UV) += arch/ia64/uv/
core-$(CONFIG_KVM) += arch/ia64/kvm/
-core-$(CONFIG_XEN) += arch/ia64/xen/
drivers-$(CONFIG_PCI) += arch/ia64/pci/
drivers-$(CONFIG_IA64_HP_SIM) += arch/ia64/hp/sim/
+++ /dev/null
-CONFIG_EXPERIMENTAL=y
-CONFIG_SYSVIPC=y
-CONFIG_POSIX_MQUEUE=y
-CONFIG_IKCONFIG=y
-CONFIG_IKCONFIG_PROC=y
-CONFIG_LOG_BUF_SHIFT=20
-CONFIG_SYSFS_DEPRECATED_V2=y
-CONFIG_BLK_DEV_INITRD=y
-CONFIG_KALLSYMS_ALL=y
-CONFIG_MODULES=y
-CONFIG_MODULE_UNLOAD=y
-CONFIG_MODVERSIONS=y
-CONFIG_MODULE_SRCVERSION_ALL=y
-# CONFIG_BLK_DEV_BSG is not set
-CONFIG_PARAVIRT_GUEST=y
-CONFIG_IA64_XEN_GUEST=y
-CONFIG_MCKINLEY=y
-CONFIG_IA64_CYCLONE=y
-CONFIG_SMP=y
-CONFIG_NR_CPUS=16
-CONFIG_HOTPLUG_CPU=y
-CONFIG_PERMIT_BSP_REMOVE=y
-CONFIG_FORCE_CPEI_RETARGET=y
-CONFIG_IA64_MCA_RECOVERY=y
-CONFIG_PERFMON=y
-CONFIG_IA64_PALINFO=y
-CONFIG_KEXEC=y
-CONFIG_EFI_VARS=y
-CONFIG_BINFMT_MISC=m
-CONFIG_ACPI_PROCFS=y
-CONFIG_ACPI_BUTTON=m
-CONFIG_ACPI_FAN=m
-CONFIG_ACPI_PROCESSOR=m
-CONFIG_ACPI_CONTAINER=m
-CONFIG_HOTPLUG_PCI=y
-CONFIG_HOTPLUG_PCI_ACPI=m
-CONFIG_PACKET=y
-CONFIG_UNIX=y
-CONFIG_INET=y
-CONFIG_IP_MULTICAST=y
-CONFIG_ARPD=y
-CONFIG_SYN_COOKIES=y
-# CONFIG_INET_LRO is not set
-# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
-CONFIG_BLK_DEV_LOOP=m
-CONFIG_BLK_DEV_CRYPTOLOOP=m
-CONFIG_BLK_DEV_NBD=m
-CONFIG_BLK_DEV_RAM=y
-CONFIG_IDE=y
-CONFIG_BLK_DEV_IDECD=y
-CONFIG_BLK_DEV_GENERIC=y
-CONFIG_BLK_DEV_CMD64X=y
-CONFIG_BLK_DEV_PIIX=y
-CONFIG_SCSI=y
-CONFIG_BLK_DEV_SD=y
-CONFIG_CHR_DEV_ST=m
-CONFIG_BLK_DEV_SR=m
-CONFIG_CHR_DEV_SG=m
-CONFIG_SCSI_SYM53C8XX_2=y
-CONFIG_SCSI_QLOGIC_1280=y
-CONFIG_MD=y
-CONFIG_BLK_DEV_MD=m
-CONFIG_MD_LINEAR=m
-CONFIG_MD_RAID0=m
-CONFIG_MD_RAID1=m
-CONFIG_MD_MULTIPATH=m
-CONFIG_BLK_DEV_DM=m
-CONFIG_DM_CRYPT=m
-CONFIG_DM_SNAPSHOT=m
-CONFIG_DM_MIRROR=m
-CONFIG_DM_ZERO=m
-CONFIG_FUSION=y
-CONFIG_FUSION_SPI=y
-CONFIG_FUSION_FC=y
-CONFIG_FUSION_CTL=y
-CONFIG_NETDEVICES=y
-CONFIG_DUMMY=m
-CONFIG_NET_ETHERNET=y
-CONFIG_NET_TULIP=y
-CONFIG_TULIP=m
-CONFIG_NET_PCI=y
-CONFIG_NET_VENDOR_INTEL=y
-CONFIG_E100=m
-CONFIG_E1000=y
-CONFIG_TIGON3=y
-CONFIG_NETCONSOLE=y
-# CONFIG_SERIO_SERPORT is not set
-CONFIG_GAMEPORT=m
-CONFIG_SERIAL_NONSTANDARD=y
-CONFIG_SERIAL_8250=y
-CONFIG_SERIAL_8250_CONSOLE=y
-CONFIG_SERIAL_8250_NR_UARTS=6
-CONFIG_SERIAL_8250_EXTENDED=y
-CONFIG_SERIAL_8250_SHARE_IRQ=y
-# CONFIG_HW_RANDOM is not set
-CONFIG_EFI_RTC=y
-CONFIG_RAW_DRIVER=m
-CONFIG_HPET=y
-CONFIG_AGP=m
-CONFIG_DRM=m
-CONFIG_DRM_TDFX=m
-CONFIG_DRM_R128=m
-CONFIG_DRM_RADEON=m
-CONFIG_DRM_MGA=m
-CONFIG_DRM_SIS=m
-CONFIG_HID_GYRATION=y
-CONFIG_HID_NTRIG=y
-CONFIG_HID_PANTHERLORD=y
-CONFIG_HID_PETALYNX=y
-CONFIG_HID_SAMSUNG=y
-CONFIG_HID_SONY=y
-CONFIG_HID_SUNPLUS=y
-CONFIG_HID_TOPSEED=y
-CONFIG_USB=y
-CONFIG_USB_DEVICEFS=y
-CONFIG_USB_EHCI_HCD=m
-CONFIG_USB_OHCI_HCD=m
-CONFIG_USB_UHCI_HCD=y
-CONFIG_USB_STORAGE=m
-CONFIG_EXT2_FS=y
-CONFIG_EXT2_FS_XATTR=y
-CONFIG_EXT2_FS_POSIX_ACL=y
-CONFIG_EXT2_FS_SECURITY=y
-CONFIG_EXT3_FS=y
-CONFIG_EXT3_FS_POSIX_ACL=y
-CONFIG_EXT3_FS_SECURITY=y
-CONFIG_REISERFS_FS=y
-CONFIG_REISERFS_FS_XATTR=y
-CONFIG_REISERFS_FS_POSIX_ACL=y
-CONFIG_REISERFS_FS_SECURITY=y
-CONFIG_XFS_FS=y
-CONFIG_AUTOFS_FS=y
-CONFIG_AUTOFS4_FS=y
-CONFIG_ISO9660_FS=m
-CONFIG_JOLIET=y
-CONFIG_UDF_FS=m
-CONFIG_VFAT_FS=y
-CONFIG_NTFS_FS=m
-CONFIG_PROC_KCORE=y
-CONFIG_TMPFS=y
-CONFIG_HUGETLBFS=y
-CONFIG_NFS_FS=m
-CONFIG_NFS_V3=y
-CONFIG_NFS_V4=y
-CONFIG_NFSD=m
-CONFIG_NFSD_V4=y
-CONFIG_SMB_FS=m
-CONFIG_SMB_NLS_DEFAULT=y
-CONFIG_CIFS=m
-CONFIG_PARTITION_ADVANCED=y
-CONFIG_SGI_PARTITION=y
-CONFIG_EFI_PARTITION=y
-CONFIG_NLS_CODEPAGE_437=y
-CONFIG_NLS_CODEPAGE_737=m
-CONFIG_NLS_CODEPAGE_775=m
-CONFIG_NLS_CODEPAGE_850=m
-CONFIG_NLS_CODEPAGE_852=m
-CONFIG_NLS_CODEPAGE_855=m
-CONFIG_NLS_CODEPAGE_857=m
-CONFIG_NLS_CODEPAGE_860=m
-CONFIG_NLS_CODEPAGE_861=m
-CONFIG_NLS_CODEPAGE_862=m
-CONFIG_NLS_CODEPAGE_863=m
-CONFIG_NLS_CODEPAGE_864=m
-CONFIG_NLS_CODEPAGE_865=m
-CONFIG_NLS_CODEPAGE_866=m
-CONFIG_NLS_CODEPAGE_869=m
-CONFIG_NLS_CODEPAGE_936=m
-CONFIG_NLS_CODEPAGE_950=m
-CONFIG_NLS_CODEPAGE_932=m
-CONFIG_NLS_CODEPAGE_949=m
-CONFIG_NLS_CODEPAGE_874=m
-CONFIG_NLS_ISO8859_8=m
-CONFIG_NLS_CODEPAGE_1250=m
-CONFIG_NLS_CODEPAGE_1251=m
-CONFIG_NLS_ISO8859_1=y
-CONFIG_NLS_ISO8859_2=m
-CONFIG_NLS_ISO8859_3=m
-CONFIG_NLS_ISO8859_4=m
-CONFIG_NLS_ISO8859_5=m
-CONFIG_NLS_ISO8859_6=m
-CONFIG_NLS_ISO8859_7=m
-CONFIG_NLS_ISO8859_9=m
-CONFIG_NLS_ISO8859_13=m
-CONFIG_NLS_ISO8859_14=m
-CONFIG_NLS_ISO8859_15=m
-CONFIG_NLS_KOI8_R=m
-CONFIG_NLS_KOI8_U=m
-CONFIG_NLS_UTF8=m
-CONFIG_MAGIC_SYSRQ=y
-CONFIG_DEBUG_KERNEL=y
-CONFIG_DEBUG_MUTEXES=y
-# CONFIG_RCU_CPU_STALL_DETECTOR is not set
-CONFIG_IA64_GRANULE_16MB=y
-CONFIG_CRYPTO_ECB=m
-CONFIG_CRYPTO_PCBC=m
-CONFIG_CRYPTO_MD5=y
-# CONFIG_CRYPTO_ANSI_CPRNG is not set
return "uv";
# elif defined (CONFIG_IA64_DIG)
return "dig";
-# elif defined (CONFIG_IA64_XEN_GUEST)
- return "xen";
# elif defined(CONFIG_IA64_DIG_VTD)
return "dig_vtd";
# else
# define smp_rmb() rmb()
# define smp_wmb() wmb()
# define smp_read_barrier_depends() read_barrier_depends()
+
#else
+
# define smp_mb() barrier()
# define smp_rmb() barrier()
# define smp_wmb() barrier()
# define smp_read_barrier_depends() do { } while(0)
+
#endif
+/*
+ * IA64 GCC turns volatile stores into st.rel and volatile loads into ld.acq no
+ * need for asm trickery!
+ */
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
/*
* XXX check on this ---I suspect what Linus really wants here is
* acquire vs release semantics but we can't discuss this stuff with
# include <asm/machvec_sn2.h>
# elif defined (CONFIG_IA64_SGI_UV)
# include <asm/machvec_uv.h>
-# elif defined (CONFIG_IA64_XEN_GUEST)
-# include <asm/machvec_xen.h>
# elif defined (CONFIG_IA64_GENERIC)
# ifdef MACHVEC_PLATFORM_HEADER
+++ /dev/null
-#ifndef _ASM_IA64_MACHVEC_XEN_h
-#define _ASM_IA64_MACHVEC_XEN_h
-
-extern ia64_mv_setup_t dig_setup;
-extern ia64_mv_cpu_init_t xen_cpu_init;
-extern ia64_mv_irq_init_t xen_irq_init;
-extern ia64_mv_send_ipi_t xen_platform_send_ipi;
-
-/*
- * This stuff has dual use!
- *
- * For a generic kernel, the macros are used to initialize the
- * platform's machvec structure. When compiling a non-generic kernel,
- * the macros are used directly.
- */
-#define ia64_platform_name "xen"
-#define platform_setup dig_setup
-#define platform_cpu_init xen_cpu_init
-#define platform_irq_init xen_irq_init
-#define platform_send_ipi xen_platform_send_ipi
-
-#endif /* _ASM_IA64_MACHVEC_XEN_h */
* - crash dumping code reserved region
* - Kernel memory map built from EFI memory map
* - ELF core header
- * - xen start info if CONFIG_XEN
*
* More could be added if necessary
*/
#ifdef CONFIG_PARAVIRT_GUEST
#define PARAVIRT_HYPERVISOR_TYPE_DEFAULT 0
-#define PARAVIRT_HYPERVISOR_TYPE_XEN 1
#ifndef __ASSEMBLY__
/*
* These structs MUST NOT be changed.
* They are the ABI between hypervisor and guest OS.
- * Both Xen and KVM are using this.
+ * KVM is using this.
*
* pvclock_vcpu_time_info holds the system time and the tsc timestamp
* of the last update. So the guest can use the tsc delta to get a
+++ /dev/null
-#ifndef _ASM_IA64_SYNC_BITOPS_H
-#define _ASM_IA64_SYNC_BITOPS_H
-
-/*
- * Copyright (C) 2008 Isaku Yamahata <yamahata at valinux co jp>
- *
- * Based on synch_bitops.h which Dan Magenhaimer wrote.
- *
- * bit operations which provide guaranteed strong synchronisation
- * when communicating with Xen or other guest OSes running on other CPUs.
- */
-
-static inline void sync_set_bit(int nr, volatile void *addr)
-{
- set_bit(nr, addr);
-}
-
-static inline void sync_clear_bit(int nr, volatile void *addr)
-{
- clear_bit(nr, addr);
-}
-
-static inline void sync_change_bit(int nr, volatile void *addr)
-{
- change_bit(nr, addr);
-}
-
-static inline int sync_test_and_set_bit(int nr, volatile void *addr)
-{
- return test_and_set_bit(nr, addr);
-}
-
-static inline int sync_test_and_clear_bit(int nr, volatile void *addr)
-{
- return test_and_clear_bit(nr, addr);
-}
-
-static inline int sync_test_and_change_bit(int nr, volatile void *addr)
-{
- return test_and_change_bit(nr, addr);
-}
-
-static inline int sync_test_bit(int nr, const volatile void *addr)
-{
- return test_bit(nr, addr);
-}
-
-#define sync_cmpxchg(ptr, old, new) \
- ((__typeof__(*(ptr)))cmpxchg_acq((ptr), (old), (new)))
-
-#endif /* _ASM_IA64_SYNC_BITOPS_H */
+++ /dev/null
-/******************************************************************************
- * arch/ia64/include/asm/xen/events.h
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-#ifndef _ASM_IA64_XEN_EVENTS_H
-#define _ASM_IA64_XEN_EVENTS_H
-
-enum ipi_vector {
- XEN_RESCHEDULE_VECTOR,
- XEN_IPI_VECTOR,
- XEN_CMCP_VECTOR,
- XEN_CPEP_VECTOR,
-
- XEN_NR_IPIS,
-};
-
-static inline int xen_irqs_disabled(struct pt_regs *regs)
-{
- return !(ia64_psr(regs)->i);
-}
-
-#define irq_ctx_init(cpu) do { } while (0)
-
-#endif /* _ASM_IA64_XEN_EVENTS_H */
+++ /dev/null
-/******************************************************************************
- * hypercall.h
- *
- * Linux-specific hypervisor handling.
- *
- * Copyright (c) 2002-2004, K A Fraser
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version 2
- * as published by the Free Software Foundation; or, when distributed
- * separately from the Linux kernel or incorporated into other
- * software packages, subject to the following license:
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this source file (the "Software"), to deal in the Software without
- * restriction, including without limitation the rights to use, copy, modify,
- * merge, publish, distribute, sublicense, and/or sell copies of the Software,
- * and to permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
- * IN THE SOFTWARE.
- */
-
-#ifndef _ASM_IA64_XEN_HYPERCALL_H
-#define _ASM_IA64_XEN_HYPERCALL_H
-
-#include <xen/interface/xen.h>
-#include <xen/interface/physdev.h>
-#include <xen/interface/sched.h>
-#include <asm/xen/xcom_hcall.h>
-struct xencomm_handle;
-extern unsigned long __hypercall(unsigned long a1, unsigned long a2,
- unsigned long a3, unsigned long a4,
- unsigned long a5, unsigned long cmd);
-
-/*
- * Assembler stubs for hyper-calls.
- */
-
-#define _hypercall0(type, name) \
-({ \
- long __res; \
- __res = __hypercall(0, 0, 0, 0, 0, __HYPERVISOR_##name);\
- (type)__res; \
-})
-
-#define _hypercall1(type, name, a1) \
-({ \
- long __res; \
- __res = __hypercall((unsigned long)a1, \
- 0, 0, 0, 0, __HYPERVISOR_##name); \
- (type)__res; \
-})
-
-#define _hypercall2(type, name, a1, a2) \
-({ \
- long __res; \
- __res = __hypercall((unsigned long)a1, \
- (unsigned long)a2, \
- 0, 0, 0, __HYPERVISOR_##name); \
- (type)__res; \
-})
-
-#define _hypercall3(type, name, a1, a2, a3) \
-({ \
- long __res; \
- __res = __hypercall((unsigned long)a1, \
- (unsigned long)a2, \
- (unsigned long)a3, \
- 0, 0, __HYPERVISOR_##name); \
- (type)__res; \
-})
-
-#define _hypercall4(type, name, a1, a2, a3, a4) \
-({ \
- long __res; \
- __res = __hypercall((unsigned long)a1, \
- (unsigned long)a2, \
- (unsigned long)a3, \
- (unsigned long)a4, \
- 0, __HYPERVISOR_##name); \
- (type)__res; \
-})
-
-#define _hypercall5(type, name, a1, a2, a3, a4, a5) \
-({ \
- long __res; \
- __res = __hypercall((unsigned long)a1, \
- (unsigned long)a2, \
- (unsigned long)a3, \
- (unsigned long)a4, \
- (unsigned long)a5, \
- __HYPERVISOR_##name); \
- (type)__res; \
-})
-
-
-static inline int
-xencomm_arch_hypercall_sched_op(int cmd, struct xencomm_handle *arg)
-{
- return _hypercall2(int, sched_op, cmd, arg);
-}
-
-static inline long
-HYPERVISOR_set_timer_op(u64 timeout)
-{
- unsigned long timeout_hi = (unsigned long)(timeout >> 32);
- unsigned long timeout_lo = (unsigned long)timeout;
- return _hypercall2(long, set_timer_op, timeout_lo, timeout_hi);
-}
-
-static inline int
-xencomm_arch_hypercall_multicall(struct xencomm_handle *call_list,
- int nr_calls)
-{
- return _hypercall2(int, multicall, call_list, nr_calls);
-}
-
-static inline int
-xencomm_arch_hypercall_memory_op(unsigned int cmd, struct xencomm_handle *arg)
-{
- return _hypercall2(int, memory_op, cmd, arg);
-}
-
-static inline int
-xencomm_arch_hypercall_event_channel_op(int cmd, struct xencomm_handle *arg)
-{
- return _hypercall2(int, event_channel_op, cmd, arg);
-}
-
-static inline int
-xencomm_arch_hypercall_xen_version(int cmd, struct xencomm_handle *arg)
-{
- return _hypercall2(int, xen_version, cmd, arg);
-}
-
-static inline int
-xencomm_arch_hypercall_console_io(int cmd, int count,
- struct xencomm_handle *str)
-{
- return _hypercall3(int, console_io, cmd, count, str);
-}
-
-static inline int
-xencomm_arch_hypercall_physdev_op(int cmd, struct xencomm_handle *arg)
-{
- return _hypercall2(int, physdev_op, cmd, arg);
-}
-
-static inline int
-xencomm_arch_hypercall_grant_table_op(unsigned int cmd,
- struct xencomm_handle *uop,
- unsigned int count)
-{
- return _hypercall3(int, grant_table_op, cmd, uop, count);
-}
-
-int HYPERVISOR_grant_table_op(unsigned int cmd, void *uop, unsigned int count);
-
-extern int xencomm_arch_hypercall_suspend(struct xencomm_handle *arg);
-
-static inline int
-xencomm_arch_hypercall_callback_op(int cmd, struct xencomm_handle *arg)
-{
- return _hypercall2(int, callback_op, cmd, arg);
-}
-
-static inline long
-xencomm_arch_hypercall_vcpu_op(int cmd, int cpu, void *arg)
-{
- return _hypercall3(long, vcpu_op, cmd, cpu, arg);
-}
-
-static inline int
-HYPERVISOR_physdev_op(int cmd, void *arg)
-{
- switch (cmd) {
- case PHYSDEVOP_eoi:
- return _hypercall1(int, ia64_fast_eoi,
- ((struct physdev_eoi *)arg)->irq);
- default:
- return xencomm_hypercall_physdev_op(cmd, arg);
- }
-}
-
-static inline long
-xencomm_arch_hypercall_opt_feature(struct xencomm_handle *arg)
-{
- return _hypercall1(long, opt_feature, arg);
-}
-
-/* for balloon driver */
-#define HYPERVISOR_update_va_mapping(va, new_val, flags) (0)
-
-/* Use xencomm to do hypercalls. */
-#define HYPERVISOR_sched_op xencomm_hypercall_sched_op
-#define HYPERVISOR_event_channel_op xencomm_hypercall_event_channel_op
-#define HYPERVISOR_callback_op xencomm_hypercall_callback_op
-#define HYPERVISOR_multicall xencomm_hypercall_multicall
-#define HYPERVISOR_xen_version xencomm_hypercall_xen_version
-#define HYPERVISOR_console_io xencomm_hypercall_console_io
-#define HYPERVISOR_memory_op xencomm_hypercall_memory_op
-#define HYPERVISOR_suspend xencomm_hypercall_suspend
-#define HYPERVISOR_vcpu_op xencomm_hypercall_vcpu_op
-#define HYPERVISOR_opt_feature xencomm_hypercall_opt_feature
-
-/* to compile gnttab_copy_grant_page() in drivers/xen/core/gnttab.c */
-#define HYPERVISOR_mmu_update(req, count, success_count, domid) ({ BUG(); 0; })
-
-static inline int
-HYPERVISOR_shutdown(
- unsigned int reason)
-{
- struct sched_shutdown sched_shutdown = {
- .reason = reason
- };
-
- int rc = HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
-
- return rc;
-}
-
-/* for netfront.c, netback.c */
-#define MULTI_UVMFLAGS_INDEX 0 /* XXX any value */
-
-static inline void
-MULTI_update_va_mapping(
- struct multicall_entry *mcl, unsigned long va,
- pte_t new_val, unsigned long flags)
-{
- mcl->op = __HYPERVISOR_update_va_mapping;
- mcl->result = 0;
-}
-
-static inline void
-MULTI_grant_table_op(struct multicall_entry *mcl, unsigned int cmd,
- void *uop, unsigned int count)
-{
- mcl->op = __HYPERVISOR_grant_table_op;
- mcl->args[0] = cmd;
- mcl->args[1] = (unsigned long)uop;
- mcl->args[2] = count;
-}
-
-static inline void
-MULTI_mmu_update(struct multicall_entry *mcl, struct mmu_update *req,
- int count, int *success_count, domid_t domid)
-{
- mcl->op = __HYPERVISOR_mmu_update;
- mcl->args[0] = (unsigned long)req;
- mcl->args[1] = count;
- mcl->args[2] = (unsigned long)success_count;
- mcl->args[3] = domid;
-}
-
-#endif /* _ASM_IA64_XEN_HYPERCALL_H */
+++ /dev/null
-/******************************************************************************
- * hypervisor.h
- *
- * Linux-specific hypervisor handling.
- *
- * Copyright (c) 2002-2004, K A Fraser
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version 2
- * as published by the Free Software Foundation; or, when distributed
- * separately from the Linux kernel or incorporated into other
- * software packages, subject to the following license:
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this source file (the "Software"), to deal in the Software without
- * restriction, including without limitation the rights to use, copy, modify,
- * merge, publish, distribute, sublicense, and/or sell copies of the Software,
- * and to permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
- * IN THE SOFTWARE.
- */
-
-#ifndef _ASM_IA64_XEN_HYPERVISOR_H
-#define _ASM_IA64_XEN_HYPERVISOR_H
-
-#include <linux/err.h>
-#include <xen/interface/xen.h>
-#include <xen/interface/version.h> /* to compile feature.c */
-#include <xen/features.h> /* to comiple xen-netfront.c */
-#include <xen/xen.h>
-#include <asm/xen/hypercall.h>
-
-#ifdef CONFIG_XEN
-extern struct shared_info *HYPERVISOR_shared_info;
-extern struct start_info *xen_start_info;
-
-void __init xen_setup_vcpu_info_placement(void);
-void force_evtchn_callback(void);
-
-/* for drivers/xen/balloon/balloon.c */
-#ifdef CONFIG_XEN_SCRUB_PAGES
-#define scrub_pages(_p, _n) memset((void *)(_p), 0, (_n) << PAGE_SHIFT)
-#else
-#define scrub_pages(_p, _n) ((void)0)
-#endif
-
-/* For setup_arch() in arch/ia64/kernel/setup.c */
-void xen_ia64_enable_opt_feature(void);
-#endif
-
-#endif /* _ASM_IA64_XEN_HYPERVISOR_H */
+++ /dev/null
-/******************************************************************************
- * arch/ia64/include/asm/xen/inst.h
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <asm/xen/privop.h>
-
-#define ia64_ivt xen_ivt
-#define DO_SAVE_MIN XEN_DO_SAVE_MIN
-
-#define __paravirt_switch_to xen_switch_to
-#define __paravirt_leave_syscall xen_leave_syscall
-#define __paravirt_work_processed_syscall xen_work_processed_syscall
-#define __paravirt_leave_kernel xen_leave_kernel
-#define __paravirt_pending_syscall_end xen_work_pending_syscall_end
-#define __paravirt_work_processed_syscall_target \
- xen_work_processed_syscall
-
-#define paravirt_fsyscall_table xen_fsyscall_table
-#define paravirt_fsys_bubble_down xen_fsys_bubble_down
-
-#define MOV_FROM_IFA(reg) \
- movl reg = XSI_IFA; \
- ;; \
- ld8 reg = [reg]
-
-#define MOV_FROM_ITIR(reg) \
- movl reg = XSI_ITIR; \
- ;; \
- ld8 reg = [reg]
-
-#define MOV_FROM_ISR(reg) \
- movl reg = XSI_ISR; \
- ;; \
- ld8 reg = [reg]
-
-#define MOV_FROM_IHA(reg) \
- movl reg = XSI_IHA; \
- ;; \
- ld8 reg = [reg]
-
-#define MOV_FROM_IPSR(pred, reg) \
-(pred) movl reg = XSI_IPSR; \
- ;; \
-(pred) ld8 reg = [reg]
-
-#define MOV_FROM_IIM(reg) \
- movl reg = XSI_IIM; \
- ;; \
- ld8 reg = [reg]
-
-#define MOV_FROM_IIP(reg) \
- movl reg = XSI_IIP; \
- ;; \
- ld8 reg = [reg]
-
-.macro __MOV_FROM_IVR reg, clob
- .ifc "\reg", "r8"
- XEN_HYPER_GET_IVR
- .exitm
- .endif
- .ifc "\clob", "r8"
- XEN_HYPER_GET_IVR
- ;;
- mov \reg = r8
- .exitm
- .endif
-
- mov \clob = r8
- ;;
- XEN_HYPER_GET_IVR
- ;;
- mov \reg = r8
- ;;
- mov r8 = \clob
-.endm
-#define MOV_FROM_IVR(reg, clob) __MOV_FROM_IVR reg, clob
-
-.macro __MOV_FROM_PSR pred, reg, clob
- .ifc "\reg", "r8"
- (\pred) XEN_HYPER_GET_PSR;
- .exitm
- .endif
- .ifc "\clob", "r8"
- (\pred) XEN_HYPER_GET_PSR
- ;;
- (\pred) mov \reg = r8
- .exitm
- .endif
-
- (\pred) mov \clob = r8
- (\pred) XEN_HYPER_GET_PSR
- ;;
- (\pred) mov \reg = r8
- (\pred) mov r8 = \clob
-.endm
-#define MOV_FROM_PSR(pred, reg, clob) __MOV_FROM_PSR pred, reg, clob
-
-/* assuming ar.itc is read with interrupt disabled. */
-#define MOV_FROM_ITC(pred, pred_clob, reg, clob) \
-(pred) movl clob = XSI_ITC_OFFSET; \
- ;; \
-(pred) ld8 clob = [clob]; \
-(pred) mov reg = ar.itc; \
- ;; \
-(pred) add reg = reg, clob; \
- ;; \
-(pred) movl clob = XSI_ITC_LAST; \
- ;; \
-(pred) ld8 clob = [clob]; \
- ;; \
-(pred) cmp.geu.unc pred_clob, p0 = clob, reg; \
- ;; \
-(pred_clob) add reg = 1, clob; \
- ;; \
-(pred) movl clob = XSI_ITC_LAST; \
- ;; \
-(pred) st8 [clob] = reg
-
-
-#define MOV_TO_IFA(reg, clob) \
- movl clob = XSI_IFA; \
- ;; \
- st8 [clob] = reg \
-
-#define MOV_TO_ITIR(pred, reg, clob) \
-(pred) movl clob = XSI_ITIR; \
- ;; \
-(pred) st8 [clob] = reg
-
-#define MOV_TO_IHA(pred, reg, clob) \
-(pred) movl clob = XSI_IHA; \
- ;; \
-(pred) st8 [clob] = reg
-
-#define MOV_TO_IPSR(pred, reg, clob) \
-(pred) movl clob = XSI_IPSR; \
- ;; \
-(pred) st8 [clob] = reg; \
- ;;
-
-#define MOV_TO_IFS(pred, reg, clob) \
-(pred) movl clob = XSI_IFS; \
- ;; \
-(pred) st8 [clob] = reg; \
- ;;
-
-#define MOV_TO_IIP(reg, clob) \
- movl clob = XSI_IIP; \
- ;; \
- st8 [clob] = reg
-
-.macro ____MOV_TO_KR kr, reg, clob0, clob1
- .ifc "\clob0", "r9"
- .error "clob0 \clob0 must not be r9"
- .endif
- .ifc "\clob1", "r8"
- .error "clob1 \clob1 must not be r8"
- .endif
-
- .ifnc "\reg", "r9"
- .ifnc "\clob1", "r9"
- mov \clob1 = r9
- .endif
- mov r9 = \reg
- .endif
- .ifnc "\clob0", "r8"
- mov \clob0 = r8
- .endif
- mov r8 = \kr
- ;;
- XEN_HYPER_SET_KR
-
- .ifnc "\reg", "r9"
- .ifnc "\clob1", "r9"
- mov r9 = \clob1
- .endif
- .endif
- .ifnc "\clob0", "r8"
- mov r8 = \clob0
- .endif
-.endm
-
-.macro __MOV_TO_KR kr, reg, clob0, clob1
- .ifc "\clob0", "r9"
- ____MOV_TO_KR \kr, \reg, \clob1, \clob0
- .exitm
- .endif
- .ifc "\clob1", "r8"
- ____MOV_TO_KR \kr, \reg, \clob1, \clob0
- .exitm
- .endif
-
- ____MOV_TO_KR \kr, \reg, \clob0, \clob1
-.endm
-
-#define MOV_TO_KR(kr, reg, clob0, clob1) \
- __MOV_TO_KR IA64_KR_ ## kr, reg, clob0, clob1
-
-
-.macro __ITC_I pred, reg, clob
- .ifc "\reg", "r8"
- (\pred) XEN_HYPER_ITC_I
- .exitm
- .endif
- .ifc "\clob", "r8"
- (\pred) mov r8 = \reg
- ;;
- (\pred) XEN_HYPER_ITC_I
- .exitm
- .endif
-
- (\pred) mov \clob = r8
- (\pred) mov r8 = \reg
- ;;
- (\pred) XEN_HYPER_ITC_I
- ;;
- (\pred) mov r8 = \clob
- ;;
-.endm
-#define ITC_I(pred, reg, clob) __ITC_I pred, reg, clob
-
-.macro __ITC_D pred, reg, clob
- .ifc "\reg", "r8"
- (\pred) XEN_HYPER_ITC_D
- ;;
- .exitm
- .endif
- .ifc "\clob", "r8"
- (\pred) mov r8 = \reg
- ;;
- (\pred) XEN_HYPER_ITC_D
- ;;
- .exitm
- .endif
-
- (\pred) mov \clob = r8
- (\pred) mov r8 = \reg
- ;;
- (\pred) XEN_HYPER_ITC_D
- ;;
- (\pred) mov r8 = \clob
- ;;
-.endm
-#define ITC_D(pred, reg, clob) __ITC_D pred, reg, clob
-
-.macro __ITC_I_AND_D pred_i, pred_d, reg, clob
- .ifc "\reg", "r8"
- (\pred_i)XEN_HYPER_ITC_I
- ;;
- (\pred_d)XEN_HYPER_ITC_D
- ;;
- .exitm
- .endif
- .ifc "\clob", "r8"
- mov r8 = \reg
- ;;
- (\pred_i)XEN_HYPER_ITC_I
- ;;
- (\pred_d)XEN_HYPER_ITC_D
- ;;
- .exitm
- .endif
-
- mov \clob = r8
- mov r8 = \reg
- ;;
- (\pred_i)XEN_HYPER_ITC_I
- ;;
- (\pred_d)XEN_HYPER_ITC_D
- ;;
- mov r8 = \clob
- ;;
-.endm
-#define ITC_I_AND_D(pred_i, pred_d, reg, clob) \
- __ITC_I_AND_D pred_i, pred_d, reg, clob
-
-.macro __THASH pred, reg0, reg1, clob
- .ifc "\reg0", "r8"
- (\pred) mov r8 = \reg1
- (\pred) XEN_HYPER_THASH
- .exitm
- .endc
- .ifc "\reg1", "r8"
- (\pred) XEN_HYPER_THASH
- ;;
- (\pred) mov \reg0 = r8
- ;;
- .exitm
- .endif
- .ifc "\clob", "r8"
- (\pred) mov r8 = \reg1
- (\pred) XEN_HYPER_THASH
- ;;
- (\pred) mov \reg0 = r8
- ;;
- .exitm
- .endif
-
- (\pred) mov \clob = r8
- (\pred) mov r8 = \reg1
- (\pred) XEN_HYPER_THASH
- ;;
- (\pred) mov \reg0 = r8
- (\pred) mov r8 = \clob
- ;;
-.endm
-#define THASH(pred, reg0, reg1, clob) __THASH pred, reg0, reg1, clob
-
-#define SSM_PSR_IC_AND_DEFAULT_BITS_AND_SRLZ_I(clob0, clob1) \
- mov clob0 = 1; \
- movl clob1 = XSI_PSR_IC; \
- ;; \
- st4 [clob1] = clob0 \
- ;;
-
-#define SSM_PSR_IC_AND_SRLZ_D(clob0, clob1) \
- ;; \
- srlz.d; \
- mov clob1 = 1; \
- movl clob0 = XSI_PSR_IC; \
- ;; \
- st4 [clob0] = clob1
-
-#define RSM_PSR_IC(clob) \
- movl clob = XSI_PSR_IC; \
- ;; \
- st4 [clob] = r0; \
- ;;
-
-/* pred will be clobbered */
-#define MASK_TO_PEND_OFS (-1)
-#define SSM_PSR_I(pred, pred_clob, clob) \
-(pred) movl clob = XSI_PSR_I_ADDR \
- ;; \
-(pred) ld8 clob = [clob] \
- ;; \
- /* if (pred) vpsr.i = 1 */ \
- /* if (pred) (vcpu->vcpu_info->evtchn_upcall_mask)=0 */ \
-(pred) st1 [clob] = r0, MASK_TO_PEND_OFS \
- ;; \
- /* if (vcpu->vcpu_info->evtchn_upcall_pending) */ \
-(pred) ld1 clob = [clob] \
- ;; \
-(pred) cmp.ne.unc pred_clob, p0 = clob, r0 \
- ;; \
-(pred_clob)XEN_HYPER_SSM_I /* do areal ssm psr.i */
-
-#define RSM_PSR_I(pred, clob0, clob1) \
- movl clob0 = XSI_PSR_I_ADDR; \
- mov clob1 = 1; \
- ;; \
- ld8 clob0 = [clob0]; \
- ;; \
-(pred) st1 [clob0] = clob1
-
-#define RSM_PSR_I_IC(clob0, clob1, clob2) \
- movl clob0 = XSI_PSR_I_ADDR; \
- movl clob1 = XSI_PSR_IC; \
- ;; \
- ld8 clob0 = [clob0]; \
- mov clob2 = 1; \
- ;; \
- /* note: clears both vpsr.i and vpsr.ic! */ \
- st1 [clob0] = clob2; \
- st4 [clob1] = r0; \
- ;;
-
-#define RSM_PSR_DT \
- XEN_HYPER_RSM_PSR_DT
-
-#define RSM_PSR_BE_I(clob0, clob1) \
- RSM_PSR_I(p0, clob0, clob1); \
- rum psr.be
-
-#define SSM_PSR_DT_AND_SRLZ_I \
- XEN_HYPER_SSM_PSR_DT
-
-#define BSW_0(clob0, clob1, clob2) \
- ;; \
- /* r16-r31 all now hold bank1 values */ \
- mov clob2 = ar.unat; \
- movl clob0 = XSI_BANK1_R16; \
- movl clob1 = XSI_BANK1_R16 + 8; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r16, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r17, 16; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r18, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r19, 16; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r20, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r21, 16; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r22, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r23, 16; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r24, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r25, 16; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r26, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r27, 16; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r28, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r29, 16; \
- ;; \
-.mem.offset 0, 0; st8.spill [clob0] = r30, 16; \
-.mem.offset 8, 0; st8.spill [clob1] = r31, 16; \
- ;; \
- mov clob1 = ar.unat; \
- movl clob0 = XSI_B1NAT; \
- ;; \
- st8 [clob0] = clob1; \
- mov ar.unat = clob2; \
- movl clob0 = XSI_BANKNUM; \
- ;; \
- st4 [clob0] = r0
-
-
- /* FIXME: THIS CODE IS NOT NaT SAFE! */
-#define XEN_BSW_1(clob) \
- mov clob = ar.unat; \
- movl r30 = XSI_B1NAT; \
- ;; \
- ld8 r30 = [r30]; \
- mov r31 = 1; \
- ;; \
- mov ar.unat = r30; \
- movl r30 = XSI_BANKNUM; \
- ;; \
- st4 [r30] = r31; \
- movl r30 = XSI_BANK1_R16; \
- movl r31 = XSI_BANK1_R16+8; \
- ;; \
- ld8.fill r16 = [r30], 16; \
- ld8.fill r17 = [r31], 16; \
- ;; \
- ld8.fill r18 = [r30], 16; \
- ld8.fill r19 = [r31], 16; \
- ;; \
- ld8.fill r20 = [r30], 16; \
- ld8.fill r21 = [r31], 16; \
- ;; \
- ld8.fill r22 = [r30], 16; \
- ld8.fill r23 = [r31], 16; \
- ;; \
- ld8.fill r24 = [r30], 16; \
- ld8.fill r25 = [r31], 16; \
- ;; \
- ld8.fill r26 = [r30], 16; \
- ld8.fill r27 = [r31], 16; \
- ;; \
- ld8.fill r28 = [r30], 16; \
- ld8.fill r29 = [r31], 16; \
- ;; \
- ld8.fill r30 = [r30]; \
- ld8.fill r31 = [r31]; \
- ;; \
- mov ar.unat = clob
-
-#define BSW_1(clob0, clob1) XEN_BSW_1(clob1)
-
-
-#define COVER \
- XEN_HYPER_COVER
-
-#define RFI \
- XEN_HYPER_RFI; \
- dv_serialize_data
+++ /dev/null
-/******************************************************************************
- * arch-ia64/hypervisor-if.h
- *
- * Guest OS interface to IA64 Xen.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to
- * deal in the Software without restriction, including without limitation the
- * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
- * sell copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- * DEALINGS IN THE SOFTWARE.
- *
- * Copyright by those who contributed. (in alphabetical order)
- *
- * Anthony Xu <anthony.xu@intel.com>
- * Eddie Dong <eddie.dong@intel.com>
- * Fred Yang <fred.yang@intel.com>
- * Kevin Tian <kevin.tian@intel.com>
- * Alex Williamson <alex.williamson@hp.com>
- * Chris Wright <chrisw@sous-sol.org>
- * Christian Limpach <Christian.Limpach@cl.cam.ac.uk>
- * Dietmar Hahn <dietmar.hahn@fujitsu-siemens.com>
- * Hollis Blanchard <hollisb@us.ibm.com>
- * Isaku Yamahata <yamahata@valinux.co.jp>
- * Jan Beulich <jbeulich@novell.com>
- * John Levon <john.levon@sun.com>
- * Kazuhiro Suzuki <kaz@jp.fujitsu.com>
- * Keir Fraser <keir.fraser@citrix.com>
- * Kouya Shimura <kouya@jp.fujitsu.com>
- * Masaki Kanno <kanno.masaki@jp.fujitsu.com>
- * Matt Chapman <matthewc@hp.com>
- * Matthew Chapman <matthewc@hp.com>
- * Samuel Thibault <samuel.thibault@eu.citrix.com>
- * Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
- * Tristan Gingold <tgingold@free.fr>
- * Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
- * Yutaka Ezaki <yutaka.ezaki@jp.fujitsu.com>
- * Zhang Xin <xing.z.zhang@intel.com>
- * Zhang xiantao <xiantao.zhang@intel.com>
- * dan.magenheimer@hp.com
- * ian.pratt@cl.cam.ac.uk
- * michael.fetterman@cl.cam.ac.uk
- */
-
-#ifndef _ASM_IA64_XEN_INTERFACE_H
-#define _ASM_IA64_XEN_INTERFACE_H
-
-#define __DEFINE_GUEST_HANDLE(name, type) \
- typedef struct { type *p; } __guest_handle_ ## name
-
-#define DEFINE_GUEST_HANDLE_STRUCT(name) \
- __DEFINE_GUEST_HANDLE(name, struct name)
-#define DEFINE_GUEST_HANDLE(name) __DEFINE_GUEST_HANDLE(name, name)
-#define GUEST_HANDLE(name) __guest_handle_ ## name
-#define GUEST_HANDLE_64(name) GUEST_HANDLE(name)
-#define set_xen_guest_handle(hnd, val) do { (hnd).p = val; } while (0)
-
-#ifndef __ASSEMBLY__
-/* Explicitly size integers that represent pfns in the public interface
- * with Xen so that we could have one ABI that works for 32 and 64 bit
- * guests. */
-typedef unsigned long xen_pfn_t;
-typedef unsigned long xen_ulong_t;
-/* Guest handles for primitive C types. */
-__DEFINE_GUEST_HANDLE(uchar, unsigned char);
-__DEFINE_GUEST_HANDLE(uint, unsigned int);
-__DEFINE_GUEST_HANDLE(ulong, unsigned long);
-
-DEFINE_GUEST_HANDLE(char);
-DEFINE_GUEST_HANDLE(int);
-DEFINE_GUEST_HANDLE(long);
-DEFINE_GUEST_HANDLE(void);
-DEFINE_GUEST_HANDLE(uint64_t);
-DEFINE_GUEST_HANDLE(uint32_t);
-
-DEFINE_GUEST_HANDLE(xen_pfn_t);
-#define PRI_xen_pfn "lx"
-#endif
-
-/* Arch specific VIRQs definition */
-#define VIRQ_ITC VIRQ_ARCH_0 /* V. Virtual itc timer */
-#define VIRQ_MCA_CMC VIRQ_ARCH_1 /* MCA cmc interrupt */
-#define VIRQ_MCA_CPE VIRQ_ARCH_2 /* MCA cpe interrupt */
-
-/* Maximum number of virtual CPUs in multi-processor guests. */
-/* keep sizeof(struct shared_page) <= PAGE_SIZE.
- * this is checked in arch/ia64/xen/hypervisor.c. */
-#define MAX_VIRT_CPUS 64
-
-#ifndef __ASSEMBLY__
-
-#define INVALID_MFN (~0UL)
-
-union vac {
- unsigned long value;
- struct {
- int a_int:1;
- int a_from_int_cr:1;
- int a_to_int_cr:1;
- int a_from_psr:1;
- int a_from_cpuid:1;
- int a_cover:1;
- int a_bsw:1;
- long reserved:57;
- };
-};
-
-union vdc {
- unsigned long value;
- struct {
- int d_vmsw:1;
- int d_extint:1;
- int d_ibr_dbr:1;
- int d_pmc:1;
- int d_to_pmd:1;
- int d_itm:1;
- long reserved:58;
- };
-};
-
-struct mapped_regs {
- union vac vac;
- union vdc vdc;
- unsigned long virt_env_vaddr;
- unsigned long reserved1[29];
- unsigned long vhpi;
- unsigned long reserved2[95];
- union {
- unsigned long vgr[16];
- unsigned long bank1_regs[16]; /* bank1 regs (r16-r31)
- when bank0 active */
- };
- union {
- unsigned long vbgr[16];
- unsigned long bank0_regs[16]; /* bank0 regs (r16-r31)
- when bank1 active */
- };
- unsigned long vnat;
- unsigned long vbnat;
- unsigned long vcpuid[5];
- unsigned long reserved3[11];
- unsigned long vpsr;
- unsigned long vpr;
- unsigned long reserved4[76];
- union {
- unsigned long vcr[128];
- struct {
- unsigned long dcr; /* CR0 */
- unsigned long itm;
- unsigned long iva;
- unsigned long rsv1[5];
- unsigned long pta; /* CR8 */
- unsigned long rsv2[7];
- unsigned long ipsr; /* CR16 */
- unsigned long isr;
- unsigned long rsv3;
- unsigned long iip;
- unsigned long ifa;
- unsigned long itir;
- unsigned long iipa;
- unsigned long ifs;
- unsigned long iim; /* CR24 */
- unsigned long iha;
- unsigned long rsv4[38];
- unsigned long lid; /* CR64 */
- unsigned long ivr;
- unsigned long tpr;
- unsigned long eoi;
- unsigned long irr[4];
- unsigned long itv; /* CR72 */
- unsigned long pmv;
- unsigned long cmcv;
- unsigned long rsv5[5];
- unsigned long lrr0; /* CR80 */
- unsigned long lrr1;
- unsigned long rsv6[46];
- };
- };
- union {
- unsigned long reserved5[128];
- struct {
- unsigned long precover_ifs;
- unsigned long unat; /* not sure if this is needed
- until NaT arch is done */
- int interrupt_collection_enabled; /* virtual psr.ic */
-
- /* virtual interrupt deliverable flag is
- * evtchn_upcall_mask in shared info area now.
- * interrupt_mask_addr is the address
- * of evtchn_upcall_mask for current vcpu
- */
- unsigned char *interrupt_mask_addr;
- int pending_interruption;
- unsigned char vpsr_pp;
- unsigned char vpsr_dfh;
- unsigned char hpsr_dfh;
- unsigned char hpsr_mfh;
- unsigned long reserved5_1[4];
- int metaphysical_mode; /* 1 = use metaphys mapping
- 0 = use virtual */
- int banknum; /* 0 or 1, which virtual
- register bank is active */
- unsigned long rrs[8]; /* region registers */
- unsigned long krs[8]; /* kernel registers */
- unsigned long tmp[16]; /* temp registers
- (e.g. for hyperprivops) */
-
- /* itc paravirtualization
- * vAR.ITC = mAR.ITC + itc_offset
- * itc_last is one which was lastly passed to
- * the guest OS in order to prevent it from
- * going backwords.
- */
- unsigned long itc_offset;
- unsigned long itc_last;
- };
- };
-};
-
-struct arch_vcpu_info {
- /* nothing */
-};
-
-/*
- * This structure is used for magic page in domain pseudo physical address
- * space and the result of XENMEM_machine_memory_map.
- * As the XENMEM_machine_memory_map result,
- * xen_memory_map::nr_entries indicates the size in bytes
- * including struct xen_ia64_memmap_info. Not the number of entries.
- */
-struct xen_ia64_memmap_info {
- uint64_t efi_memmap_size; /* size of EFI memory map */
- uint64_t efi_memdesc_size; /* size of an EFI memory map
- * descriptor */
- uint32_t efi_memdesc_version; /* memory descriptor version */
- void *memdesc[0]; /* array of efi_memory_desc_t */
-};
-
-struct arch_shared_info {
- /* PFN of the start_info page. */
- unsigned long start_info_pfn;
-
- /* Interrupt vector for event channel. */
- int evtchn_vector;
-
- /* PFN of memmap_info page */
- unsigned int memmap_info_num_pages; /* currently only = 1 case is
- supported. */
- unsigned long memmap_info_pfn;
-
- uint64_t pad[31];
-};
-
-struct xen_callback {
- unsigned long ip;
-};
-typedef struct xen_callback xen_callback_t;
-
-#endif /* !__ASSEMBLY__ */
-
-#include <asm/pvclock-abi.h>
-
-/* Size of the shared_info area (this is not related to page size). */
-#define XSI_SHIFT 14
-#define XSI_SIZE (1 << XSI_SHIFT)
-/* Log size of mapped_regs area (64 KB - only 4KB is used). */
-#define XMAPPEDREGS_SHIFT 12
-#define XMAPPEDREGS_SIZE (1 << XMAPPEDREGS_SHIFT)
-/* Offset of XASI (Xen arch shared info) wrt XSI_BASE. */
-#define XMAPPEDREGS_OFS XSI_SIZE
-
-/* Hyperprivops. */
-#define HYPERPRIVOP_START 0x1
-#define HYPERPRIVOP_RFI (HYPERPRIVOP_START + 0x0)
-#define HYPERPRIVOP_RSM_DT (HYPERPRIVOP_START + 0x1)
-#define HYPERPRIVOP_SSM_DT (HYPERPRIVOP_START + 0x2)
-#define HYPERPRIVOP_COVER (HYPERPRIVOP_START + 0x3)
-#define HYPERPRIVOP_ITC_D (HYPERPRIVOP_START + 0x4)
-#define HYPERPRIVOP_ITC_I (HYPERPRIVOP_START + 0x5)
-#define HYPERPRIVOP_SSM_I (HYPERPRIVOP_START + 0x6)
-#define HYPERPRIVOP_GET_IVR (HYPERPRIVOP_START + 0x7)
-#define HYPERPRIVOP_GET_TPR (HYPERPRIVOP_START + 0x8)
-#define HYPERPRIVOP_SET_TPR (HYPERPRIVOP_START + 0x9)
-#define HYPERPRIVOP_EOI (HYPERPRIVOP_START + 0xa)
-#define HYPERPRIVOP_SET_ITM (HYPERPRIVOP_START + 0xb)
-#define HYPERPRIVOP_THASH (HYPERPRIVOP_START + 0xc)
-#define HYPERPRIVOP_PTC_GA (HYPERPRIVOP_START + 0xd)
-#define HYPERPRIVOP_ITR_D (HYPERPRIVOP_START + 0xe)
-#define HYPERPRIVOP_GET_RR (HYPERPRIVOP_START + 0xf)
-#define HYPERPRIVOP_SET_RR (HYPERPRIVOP_START + 0x10)
-#define HYPERPRIVOP_SET_KR (HYPERPRIVOP_START + 0x11)
-#define HYPERPRIVOP_FC (HYPERPRIVOP_START + 0x12)
-#define HYPERPRIVOP_GET_CPUID (HYPERPRIVOP_START + 0x13)
-#define HYPERPRIVOP_GET_PMD (HYPERPRIVOP_START + 0x14)
-#define HYPERPRIVOP_GET_EFLAG (HYPERPRIVOP_START + 0x15)
-#define HYPERPRIVOP_SET_EFLAG (HYPERPRIVOP_START + 0x16)
-#define HYPERPRIVOP_RSM_BE (HYPERPRIVOP_START + 0x17)
-#define HYPERPRIVOP_GET_PSR (HYPERPRIVOP_START + 0x18)
-#define HYPERPRIVOP_SET_RR0_TO_RR4 (HYPERPRIVOP_START + 0x19)
-#define HYPERPRIVOP_MAX (0x1a)
-
-/* Fast and light hypercalls. */
-#define __HYPERVISOR_ia64_fast_eoi __HYPERVISOR_arch_1
-
-/* Xencomm macros. */
-#define XENCOMM_INLINE_MASK 0xf800000000000000UL
-#define XENCOMM_INLINE_FLAG 0x8000000000000000UL
-
-#ifndef __ASSEMBLY__
-
-/*
- * Optimization features.
- * The hypervisor may do some special optimizations for guests. This hypercall
- * can be used to switch on/of these special optimizations.
- */
-#define __HYPERVISOR_opt_feature 0x700UL
-
-#define XEN_IA64_OPTF_OFF 0x0
-#define XEN_IA64_OPTF_ON 0x1
-
-/*
- * If this feature is switched on, the hypervisor inserts the
- * tlb entries without calling the guests traphandler.
- * This is useful in guests using region 7 for identity mapping
- * like the linux kernel does.
- */
-#define XEN_IA64_OPTF_IDENT_MAP_REG7 1
-
-/* Identity mapping of region 4 addresses in HVM. */
-#define XEN_IA64_OPTF_IDENT_MAP_REG4 2
-
-/* Identity mapping of region 5 addresses in HVM. */
-#define XEN_IA64_OPTF_IDENT_MAP_REG5 3
-
-#define XEN_IA64_OPTF_IDENT_MAP_NOT_SET (0)
-
-struct xen_ia64_opt_feature {
- unsigned long cmd; /* Which feature */
- unsigned char on; /* Switch feature on/off */
- union {
- struct {
- /* The page protection bit mask of the pte.
- * This will be or'ed with the pte. */
- unsigned long pgprot;
- unsigned long key; /* A protection key for itir.*/
- };
- };
-};
-
-#endif /* __ASSEMBLY__ */
-
-#endif /* _ASM_IA64_XEN_INTERFACE_H */
+++ /dev/null
-/******************************************************************************
- * arch/ia64/include/asm/xen/irq.h
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#ifndef _ASM_IA64_XEN_IRQ_H
-#define _ASM_IA64_XEN_IRQ_H
-
-/*
- * The flat IRQ space is divided into two regions:
- * 1. A one-to-one mapping of real physical IRQs. This space is only used
- * if we have physical device-access privilege. This region is at the
- * start of the IRQ space so that existing device drivers do not need
- * to be modified to translate physical IRQ numbers into our IRQ space.
- * 3. A dynamic mapping of inter-domain and Xen-sourced virtual IRQs. These
- * are bound using the provided bind/unbind functions.
- */
-
-#define XEN_PIRQ_BASE 0
-#define XEN_NR_PIRQS 256
-
-#define XEN_DYNIRQ_BASE (XEN_PIRQ_BASE + XEN_NR_PIRQS)
-#define XEN_NR_DYNIRQS (NR_CPUS * 8)
-
-#define XEN_NR_IRQS (XEN_NR_PIRQS + XEN_NR_DYNIRQS)
-
-#endif /* _ASM_IA64_XEN_IRQ_H */
+++ /dev/null
-
-#ifdef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE
-/* read ar.itc in advance, and use it before leaving bank 0 */
-#define XEN_ACCOUNT_GET_STAMP \
- MOV_FROM_ITC(pUStk, p6, r20, r2);
-#else
-#define XEN_ACCOUNT_GET_STAMP
-#endif
-
-/*
- * DO_SAVE_MIN switches to the kernel stacks (if necessary) and saves
- * the minimum state necessary that allows us to turn psr.ic back
- * on.
- *
- * Assumed state upon entry:
- * psr.ic: off
- * r31: contains saved predicates (pr)
- *
- * Upon exit, the state is as follows:
- * psr.ic: off
- * r2 = points to &pt_regs.r16
- * r8 = contents of ar.ccv
- * r9 = contents of ar.csd
- * r10 = contents of ar.ssd
- * r11 = FPSR_DEFAULT
- * r12 = kernel sp (kernel virtual address)
- * r13 = points to current task_struct (kernel virtual address)
- * p15 = TRUE if psr.i is set in cr.ipsr
- * predicate registers (other than p2, p3, and p15), b6, r3, r14, r15:
- * preserved
- * CONFIG_XEN note: p6/p7 are not preserved
- *
- * Note that psr.ic is NOT turned on by this macro. This is so that
- * we can pass interruption state as arguments to a handler.
- */
-#define XEN_DO_SAVE_MIN(__COVER,SAVE_IFS,EXTRA,WORKAROUND) \
- mov r16=IA64_KR(CURRENT); /* M */ \
- mov r27=ar.rsc; /* M */ \
- mov r20=r1; /* A */ \
- mov r25=ar.unat; /* M */ \
- MOV_FROM_IPSR(p0,r29); /* M */ \
- MOV_FROM_IIP(r28); /* M */ \
- mov r21=ar.fpsr; /* M */ \
- mov r26=ar.pfs; /* I */ \
- __COVER; /* B;; (or nothing) */ \
- adds r16=IA64_TASK_THREAD_ON_USTACK_OFFSET,r16; \
- ;; \
- ld1 r17=[r16]; /* load current->thread.on_ustack flag */ \
- st1 [r16]=r0; /* clear current->thread.on_ustack flag */ \
- adds r1=-IA64_TASK_THREAD_ON_USTACK_OFFSET,r16 \
- /* switch from user to kernel RBS: */ \
- ;; \
- invala; /* M */ \
- /* SAVE_IFS;*/ /* see xen special handling below */ \
- cmp.eq pKStk,pUStk=r0,r17; /* are we in kernel mode already? */ \
- ;; \
-(pUStk) mov ar.rsc=0; /* set enforced lazy mode, pl 0, little-endian, loadrs=0 */ \
- ;; \
-(pUStk) mov.m r24=ar.rnat; \
-(pUStk) addl r22=IA64_RBS_OFFSET,r1; /* compute base of RBS */ \
-(pKStk) mov r1=sp; /* get sp */ \
- ;; \
-(pUStk) lfetch.fault.excl.nt1 [r22]; \
-(pUStk) addl r1=IA64_STK_OFFSET-IA64_PT_REGS_SIZE,r1; /* compute base of memory stack */ \
-(pUStk) mov r23=ar.bspstore; /* save ar.bspstore */ \
- ;; \
-(pUStk) mov ar.bspstore=r22; /* switch to kernel RBS */ \
-(pKStk) addl r1=-IA64_PT_REGS_SIZE,r1; /* if in kernel mode, use sp (r12) */ \
- ;; \
-(pUStk) mov r18=ar.bsp; \
-(pUStk) mov ar.rsc=0x3; /* set eager mode, pl 0, little-endian, loadrs=0 */ \
- adds r17=2*L1_CACHE_BYTES,r1; /* really: biggest cache-line size */ \
- adds r16=PT(CR_IPSR),r1; \
- ;; \
- lfetch.fault.excl.nt1 [r17],L1_CACHE_BYTES; \
- st8 [r16]=r29; /* save cr.ipsr */ \
- ;; \
- lfetch.fault.excl.nt1 [r17]; \
- tbit.nz p15,p0=r29,IA64_PSR_I_BIT; \
- mov r29=b0 \
- ;; \
- WORKAROUND; \
- adds r16=PT(R8),r1; /* initialize first base pointer */ \
- adds r17=PT(R9),r1; /* initialize second base pointer */ \
-(pKStk) mov r18=r0; /* make sure r18 isn't NaT */ \
- ;; \
-.mem.offset 0,0; st8.spill [r16]=r8,16; \
-.mem.offset 8,0; st8.spill [r17]=r9,16; \
- ;; \
-.mem.offset 0,0; st8.spill [r16]=r10,24; \
- movl r8=XSI_PRECOVER_IFS; \
-.mem.offset 8,0; st8.spill [r17]=r11,24; \
- ;; \
- /* xen special handling for possibly lazy cover */ \
- /* SAVE_MIN case in dispatch_ia32_handler: mov r30=r0 */ \
- ld8 r30=[r8]; \
-(pUStk) sub r18=r18,r22; /* r18=RSE.ndirty*8 */ \
- st8 [r16]=r28,16; /* save cr.iip */ \
- ;; \
- st8 [r17]=r30,16; /* save cr.ifs */ \
- mov r8=ar.ccv; \
- mov r9=ar.csd; \
- mov r10=ar.ssd; \
- movl r11=FPSR_DEFAULT; /* L-unit */ \
- ;; \
- st8 [r16]=r25,16; /* save ar.unat */ \
- st8 [r17]=r26,16; /* save ar.pfs */ \
- shl r18=r18,16; /* compute ar.rsc to be used for "loadrs" */ \
- ;; \
- st8 [r16]=r27,16; /* save ar.rsc */ \
-(pUStk) st8 [r17]=r24,16; /* save ar.rnat */ \
-(pKStk) adds r17=16,r17; /* skip over ar_rnat field */ \
- ;; /* avoid RAW on r16 & r17 */ \
-(pUStk) st8 [r16]=r23,16; /* save ar.bspstore */ \
- st8 [r17]=r31,16; /* save predicates */ \
-(pKStk) adds r16=16,r16; /* skip over ar_bspstore field */ \
- ;; \
- st8 [r16]=r29,16; /* save b0 */ \
- st8 [r17]=r18,16; /* save ar.rsc value for "loadrs" */ \
- cmp.eq pNonSys,pSys=r0,r0 /* initialize pSys=0, pNonSys=1 */ \
- ;; \
-.mem.offset 0,0; st8.spill [r16]=r20,16; /* save original r1 */ \
-.mem.offset 8,0; st8.spill [r17]=r12,16; \
- adds r12=-16,r1; /* switch to kernel memory stack (with 16 bytes of scratch) */ \
- ;; \
-.mem.offset 0,0; st8.spill [r16]=r13,16; \
-.mem.offset 8,0; st8.spill [r17]=r21,16; /* save ar.fpsr */ \
- mov r13=IA64_KR(CURRENT); /* establish `current' */ \
- ;; \
-.mem.offset 0,0; st8.spill [r16]=r15,16; \
-.mem.offset 8,0; st8.spill [r17]=r14,16; \
- ;; \
-.mem.offset 0,0; st8.spill [r16]=r2,16; \
-.mem.offset 8,0; st8.spill [r17]=r3,16; \
- XEN_ACCOUNT_GET_STAMP \
- adds r2=IA64_PT_REGS_R16_OFFSET,r1; \
- ;; \
- EXTRA; \
- movl r1=__gp; /* establish kernel global pointer */ \
- ;; \
- ACCOUNT_SYS_ENTER \
- BSW_1(r3,r14); /* switch back to bank 1 (must be last in insn group) */ \
- ;;
+++ /dev/null
-#ifndef _ASM_IA64_XEN_PAGE_COHERENT_H
-#define _ASM_IA64_XEN_PAGE_COHERENT_H
-
-#include <asm/page.h>
-#include <linux/dma-attrs.h>
-#include <linux/dma-mapping.h>
-
-static inline void *xen_alloc_coherent_pages(struct device *hwdev, size_t size,
- dma_addr_t *dma_handle, gfp_t flags,
- struct dma_attrs *attrs)
-{
- void *vstart = (void*)__get_free_pages(flags, get_order(size));
- *dma_handle = virt_to_phys(vstart);
- return vstart;
-}
-
-static inline void xen_free_coherent_pages(struct device *hwdev, size_t size,
- void *cpu_addr, dma_addr_t dma_handle,
- struct dma_attrs *attrs)
-{
- free_pages((unsigned long) cpu_addr, get_order(size));
-}
-
-static inline void xen_dma_map_page(struct device *hwdev, struct page *page,
- unsigned long offset, size_t size, enum dma_data_direction dir,
- struct dma_attrs *attrs) { }
-
-static inline void xen_dma_unmap_page(struct device *hwdev, dma_addr_t handle,
- size_t size, enum dma_data_direction dir,
- struct dma_attrs *attrs) { }
-
-static inline void xen_dma_sync_single_for_cpu(struct device *hwdev,
- dma_addr_t handle, size_t size, enum dma_data_direction dir) { }
-
-static inline void xen_dma_sync_single_for_device(struct device *hwdev,
- dma_addr_t handle, size_t size, enum dma_data_direction dir) { }
-
-#endif /* _ASM_IA64_XEN_PAGE_COHERENT_H */
+++ /dev/null
-/******************************************************************************
- * arch/ia64/include/asm/xen/page.h
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#ifndef _ASM_IA64_XEN_PAGE_H
-#define _ASM_IA64_XEN_PAGE_H
-
-#define INVALID_P2M_ENTRY (~0UL)
-
-static inline unsigned long mfn_to_pfn(unsigned long mfn)
-{
- return mfn;
-}
-
-static inline unsigned long pfn_to_mfn(unsigned long pfn)
-{
- return pfn;
-}
-
-#define phys_to_machine_mapping_valid(_x) (1)
-
-static inline void *mfn_to_virt(unsigned long mfn)
-{
- return __va(mfn << PAGE_SHIFT);
-}
-
-static inline unsigned long virt_to_mfn(void *virt)
-{
- return __pa(virt) >> PAGE_SHIFT;
-}
-
-/* for tpmfront.c */
-static inline unsigned long virt_to_machine(void *virt)
-{
- return __pa(virt);
-}
-
-static inline void set_phys_to_machine(unsigned long pfn, unsigned long mfn)
-{
- /* nothing */
-}
-
-#define pte_mfn(_x) pte_pfn(_x)
-#define mfn_pte(_x, _y) __pte_ma(0) /* unmodified use */
-#define __pte_ma(_x) ((pte_t) {(_x)}) /* unmodified use */
-
-#endif /* _ASM_IA64_XEN_PAGE_H */
+++ /dev/null
-/******************************************************************************
- * arch/ia64/include/asm/xen/patchlist.h
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#define __paravirt_start_gate_fsyscall_patchlist \
- __xen_start_gate_fsyscall_patchlist
-#define __paravirt_end_gate_fsyscall_patchlist \
- __xen_end_gate_fsyscall_patchlist
-#define __paravirt_start_gate_brl_fsys_bubble_down_patchlist \
- __xen_start_gate_brl_fsys_bubble_down_patchlist
-#define __paravirt_end_gate_brl_fsys_bubble_down_patchlist \
- __xen_end_gate_brl_fsys_bubble_down_patchlist
-#define __paravirt_start_gate_vtop_patchlist \
- __xen_start_gate_vtop_patchlist
-#define __paravirt_end_gate_vtop_patchlist \
- __xen_end_gate_vtop_patchlist
-#define __paravirt_start_gate_mckinley_e9_patchlist \
- __xen_start_gate_mckinley_e9_patchlist
-#define __paravirt_end_gate_mckinley_e9_patchlist \
- __xen_end_gate_mckinley_e9_patchlist
+++ /dev/null
-#ifndef _ASM_IA64_XEN_PRIVOP_H
-#define _ASM_IA64_XEN_PRIVOP_H
-
-/*
- * Copyright (C) 2005 Hewlett-Packard Co
- * Dan Magenheimer <dan.magenheimer@hp.com>
- *
- * Paravirtualizations of privileged operations for Xen/ia64
- *
- *
- * inline privop and paravirt_alt support
- * Copyright (c) 2007 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- */
-
-#ifndef __ASSEMBLY__
-#include <linux/types.h> /* arch-ia64.h requires uint64_t */
-#endif
-#include <asm/xen/interface.h>
-
-/* At 1 MB, before per-cpu space but still addressable using addl instead
- of movl. */
-#define XSI_BASE 0xfffffffffff00000
-
-/* Address of mapped regs. */
-#define XMAPPEDREGS_BASE (XSI_BASE + XSI_SIZE)
-
-#ifdef __ASSEMBLY__
-#define XEN_HYPER_RFI break HYPERPRIVOP_RFI
-#define XEN_HYPER_RSM_PSR_DT break HYPERPRIVOP_RSM_DT
-#define XEN_HYPER_SSM_PSR_DT break HYPERPRIVOP_SSM_DT
-#define XEN_HYPER_COVER break HYPERPRIVOP_COVER
-#define XEN_HYPER_ITC_D break HYPERPRIVOP_ITC_D
-#define XEN_HYPER_ITC_I break HYPERPRIVOP_ITC_I
-#define XEN_HYPER_SSM_I break HYPERPRIVOP_SSM_I
-#define XEN_HYPER_GET_IVR break HYPERPRIVOP_GET_IVR
-#define XEN_HYPER_THASH break HYPERPRIVOP_THASH
-#define XEN_HYPER_ITR_D break HYPERPRIVOP_ITR_D
-#define XEN_HYPER_SET_KR break HYPERPRIVOP_SET_KR
-#define XEN_HYPER_GET_PSR break HYPERPRIVOP_GET_PSR
-#define XEN_HYPER_SET_RR0_TO_RR4 break HYPERPRIVOP_SET_RR0_TO_RR4
-
-#define XSI_IFS (XSI_BASE + XSI_IFS_OFS)
-#define XSI_PRECOVER_IFS (XSI_BASE + XSI_PRECOVER_IFS_OFS)
-#define XSI_IFA (XSI_BASE + XSI_IFA_OFS)
-#define XSI_ISR (XSI_BASE + XSI_ISR_OFS)
-#define XSI_IIM (XSI_BASE + XSI_IIM_OFS)
-#define XSI_ITIR (XSI_BASE + XSI_ITIR_OFS)
-#define XSI_PSR_I_ADDR (XSI_BASE + XSI_PSR_I_ADDR_OFS)
-#define XSI_PSR_IC (XSI_BASE + XSI_PSR_IC_OFS)
-#define XSI_IPSR (XSI_BASE + XSI_IPSR_OFS)
-#define XSI_IIP (XSI_BASE + XSI_IIP_OFS)
-#define XSI_B1NAT (XSI_BASE + XSI_B1NATS_OFS)
-#define XSI_BANK1_R16 (XSI_BASE + XSI_BANK1_R16_OFS)
-#define XSI_BANKNUM (XSI_BASE + XSI_BANKNUM_OFS)
-#define XSI_IHA (XSI_BASE + XSI_IHA_OFS)
-#define XSI_ITC_OFFSET (XSI_BASE + XSI_ITC_OFFSET_OFS)
-#define XSI_ITC_LAST (XSI_BASE + XSI_ITC_LAST_OFS)
-#endif
-
-#ifndef __ASSEMBLY__
-
-/************************************************/
-/* Instructions paravirtualized for correctness */
-/************************************************/
-
-/* "fc" and "thash" are privilege-sensitive instructions, meaning they
- * may have different semantics depending on whether they are executed
- * at PL0 vs PL!=0. When paravirtualized, these instructions mustn't
- * be allowed to execute directly, lest incorrect semantics result. */
-extern void xen_fc(void *addr);
-extern unsigned long xen_thash(unsigned long addr);
-
-/* Note that "ttag" and "cover" are also privilege-sensitive; "ttag"
- * is not currently used (though it may be in a long-format VHPT system!)
- * and the semantics of cover only change if psr.ic is off which is very
- * rare (and currently non-existent outside of assembly code */
-
-/* There are also privilege-sensitive registers. These registers are
- * readable at any privilege level but only writable at PL0. */
-extern unsigned long xen_get_cpuid(int index);
-extern unsigned long xen_get_pmd(int index);
-
-#ifndef ASM_SUPPORTED
-extern unsigned long xen_get_eflag(void); /* see xen_ia64_getreg */
-extern void xen_set_eflag(unsigned long); /* see xen_ia64_setreg */
-#endif
-
-/************************************************/
-/* Instructions paravirtualized for performance */
-/************************************************/
-
-/* Xen uses memory-mapped virtual privileged registers for access to many
- * performance-sensitive privileged registers. Some, like the processor
- * status register (psr), are broken up into multiple memory locations.
- * Others, like "pend", are abstractions based on privileged registers.
- * "Pend" is guaranteed to be set if reading cr.ivr would return a
- * (non-spurious) interrupt. */
-#define XEN_MAPPEDREGS ((struct mapped_regs *)XMAPPEDREGS_BASE)
-
-#define XSI_PSR_I \
- (*XEN_MAPPEDREGS->interrupt_mask_addr)
-#define xen_get_virtual_psr_i() \
- (!XSI_PSR_I)
-#define xen_set_virtual_psr_i(_val) \
- ({ XSI_PSR_I = (uint8_t)(_val) ? 0 : 1; })
-#define xen_set_virtual_psr_ic(_val) \
- ({ XEN_MAPPEDREGS->interrupt_collection_enabled = _val ? 1 : 0; })
-#define xen_get_virtual_pend() \
- (*(((uint8_t *)XEN_MAPPEDREGS->interrupt_mask_addr) - 1))
-
-#ifndef ASM_SUPPORTED
-/* Although all privileged operations can be left to trap and will
- * be properly handled by Xen, some are frequent enough that we use
- * hyperprivops for performance. */
-extern unsigned long xen_get_psr(void);
-extern unsigned long xen_get_ivr(void);
-extern unsigned long xen_get_tpr(void);
-extern void xen_hyper_ssm_i(void);
-extern void xen_set_itm(unsigned long);
-extern void xen_set_tpr(unsigned long);
-extern void xen_eoi(unsigned long);
-extern unsigned long xen_get_rr(unsigned long index);
-extern void xen_set_rr(unsigned long index, unsigned long val);
-extern void xen_set_rr0_to_rr4(unsigned long val0, unsigned long val1,
- unsigned long val2, unsigned long val3,
- unsigned long val4);
-extern void xen_set_kr(unsigned long index, unsigned long val);
-extern void xen_ptcga(unsigned long addr, unsigned long size);
-#endif /* !ASM_SUPPORTED */
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* _ASM_IA64_XEN_PRIVOP_H */
+++ /dev/null
-/*
- * Copyright (C) 2006 Tristan Gingold <tristan.gingold@bull.net>, Bull SAS
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- */
-
-#ifndef _ASM_IA64_XEN_XCOM_HCALL_H
-#define _ASM_IA64_XEN_XCOM_HCALL_H
-
-/* These function creates inline or mini descriptor for the parameters and
- calls the corresponding xencomm_arch_hypercall_X.
- Architectures should defines HYPERVISOR_xxx as xencomm_hypercall_xxx unless
- they want to use their own wrapper. */
-extern int xencomm_hypercall_console_io(int cmd, int count, char *str);
-
-extern int xencomm_hypercall_event_channel_op(int cmd, void *op);
-
-extern int xencomm_hypercall_xen_version(int cmd, void *arg);
-
-extern int xencomm_hypercall_physdev_op(int cmd, void *op);
-
-extern int xencomm_hypercall_grant_table_op(unsigned int cmd, void *op,
- unsigned int count);
-
-extern int xencomm_hypercall_sched_op(int cmd, void *arg);
-
-extern int xencomm_hypercall_multicall(void *call_list, int nr_calls);
-
-extern int xencomm_hypercall_callback_op(int cmd, void *arg);
-
-extern int xencomm_hypercall_memory_op(unsigned int cmd, void *arg);
-
-extern int xencomm_hypercall_suspend(unsigned long srec);
-
-extern long xencomm_hypercall_vcpu_op(int cmd, int cpu, void *arg);
-
-extern long xencomm_hypercall_opt_feature(void *arg);
-
-#endif /* _ASM_IA64_XEN_XCOM_HCALL_H */
+++ /dev/null
-/*
- * Copyright (C) 2006 Hollis Blanchard <hollisb@us.ibm.com>, IBM Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- */
-
-#ifndef _ASM_IA64_XEN_XENCOMM_H
-#define _ASM_IA64_XEN_XENCOMM_H
-
-#include <xen/xencomm.h>
-#include <asm/pgtable.h>
-
-/* Must be called before any hypercall. */
-extern void xencomm_initialize(void);
-extern int xencomm_is_initialized(void);
-
-/* Check if virtual contiguity means physical contiguity
- * where the passed address is a pointer value in virtual address.
- * On ia64, identity mapping area in region 7 or the piece of region 5
- * that is mapped by itr[IA64_TR_KERNEL]/dtr[IA64_TR_KERNEL]
- */
-static inline int xencomm_is_phys_contiguous(unsigned long addr)
-{
- return (PAGE_OFFSET <= addr &&
- addr < (PAGE_OFFSET + (1UL << IA64_MAX_PHYS_BITS))) ||
- (KERNEL_START <= addr &&
- addr < KERNEL_START + KERNEL_TR_PAGE_SIZE);
-}
-
-#endif /* _ASM_IA64_XEN_XENCOMM_H */
*/
#define __IA64_BREAK_SYSCALL 0x100000
-/*
- * Xen specific break numbers:
- */
-#define __IA64_XEN_HYPERCALL 0x1000
-/* [__IA64_XEN_HYPERPRIVOP_START, __IA64_XEN_HYPERPRIVOP_MAX] is used
- for xen hyperprivops */
-#define __IA64_XEN_HYPERPRIVOP_START 0x1
-#define __IA64_XEN_HYPERPRIVOP_MAX 0x1a
-
#endif /* _ASM_IA64_BREAK_H */
#include <asm/numa.h>
#include <asm/sal.h>
#include <asm/cyclone.h>
-#include <asm/xen/hypervisor.h>
#define BAD_MADT_ENTRY(entry, end) ( \
(!entry) || (unsigned long)entry + sizeof(*entry) > end || \
return "uv";
else
return "sn2";
- } else if (xen_pv_domain() && !strcmp(hdr->oem_id, "XEN")) {
- return "xen";
}
#ifdef CONFIG_INTEL_IOMMU
#include <asm/sigcontext.h>
#include <asm/mca.h>
-#include <asm/xen/interface.h>
-#include <asm/xen/hypervisor.h>
-
#include "../kernel/sigframe.h"
#include "../kernel/fsyscall_gtod_data.h"
DEFINE(IA64_ITC_LASTCYCLE_OFFSET,
offsetof (struct itc_jitter_data_t, itc_lastcycle));
-#ifdef CONFIG_XEN
- BLANK();
-
- DEFINE(XEN_NATIVE_ASM, XEN_NATIVE);
- DEFINE(XEN_PV_DOMAIN_ASM, XEN_PV_DOMAIN);
-
-#define DEFINE_MAPPED_REG_OFS(sym, field) \
- DEFINE(sym, (XMAPPEDREGS_OFS + offsetof(struct mapped_regs, field)))
-
- DEFINE_MAPPED_REG_OFS(XSI_PSR_I_ADDR_OFS, interrupt_mask_addr);
- DEFINE_MAPPED_REG_OFS(XSI_IPSR_OFS, ipsr);
- DEFINE_MAPPED_REG_OFS(XSI_IIP_OFS, iip);
- DEFINE_MAPPED_REG_OFS(XSI_IFS_OFS, ifs);
- DEFINE_MAPPED_REG_OFS(XSI_PRECOVER_IFS_OFS, precover_ifs);
- DEFINE_MAPPED_REG_OFS(XSI_ISR_OFS, isr);
- DEFINE_MAPPED_REG_OFS(XSI_IFA_OFS, ifa);
- DEFINE_MAPPED_REG_OFS(XSI_IIPA_OFS, iipa);
- DEFINE_MAPPED_REG_OFS(XSI_IIM_OFS, iim);
- DEFINE_MAPPED_REG_OFS(XSI_IHA_OFS, iha);
- DEFINE_MAPPED_REG_OFS(XSI_ITIR_OFS, itir);
- DEFINE_MAPPED_REG_OFS(XSI_PSR_IC_OFS, interrupt_collection_enabled);
- DEFINE_MAPPED_REG_OFS(XSI_BANKNUM_OFS, banknum);
- DEFINE_MAPPED_REG_OFS(XSI_BANK0_R16_OFS, bank0_regs[0]);
- DEFINE_MAPPED_REG_OFS(XSI_BANK1_R16_OFS, bank1_regs[0]);
- DEFINE_MAPPED_REG_OFS(XSI_B0NATS_OFS, vbnat);
- DEFINE_MAPPED_REG_OFS(XSI_B1NATS_OFS, vnat);
- DEFINE_MAPPED_REG_OFS(XSI_ITC_OFFSET_OFS, itc_offset);
- DEFINE_MAPPED_REG_OFS(XSI_ITC_LAST_OFS, itc_last);
-#endif /* CONFIG_XEN */
}
default_setup_hook = 0 // Currently nothing needs to be done.
- .weak xen_setup_hook
-
.global hypervisor_type
hypervisor_type:
data8 PARAVIRT_HYPERVISOR_TYPE_DEFAULT
hypervisor_setup_hooks:
data8 default_setup_hook
- data8 xen_setup_hook
num_hypervisor_hooks = (. - hypervisor_setup_hooks) / 8
.previous
#include <linux/kbuild.h>
#include <linux/threads.h>
#include <asm/native/irq.h>
-#include <asm/xen/irq.h>
void foo(void)
{
union paravirt_nr_irqs_max {
char ia64_native_nr_irqs[IA64_NATIVE_NR_IRQS];
-#ifdef CONFIG_XEN
- char xen_nr_irqs[XEN_NR_IRQS];
-#endif
};
DEFINE(NR_IRQS, sizeof (union paravirt_nr_irqs_max));
#ifdef __IA64_ASM_PARAVIRTUALIZED_PVCHECK
#include <asm/native/pvchk_inst.h>
-#elif defined(__IA64_ASM_PARAVIRTUALIZED_XEN)
-#include <asm/xen/inst.h>
-#include <asm/xen/minstate.h>
#else
#include <asm/native/inst.h>
#endif
*
*/
-#if defined(__IA64_GATE_PARAVIRTUALIZED_XEN)
-#include <asm/xen/patchlist.h>
-#else
#include <asm/native/patchlist.h>
-#endif
__start_gate_section = .;
*(.data..gate)
__stop_gate_section = .;
-#ifdef CONFIG_XEN
- . = ALIGN(PAGE_SIZE);
- __xen_start_gate_section = .;
- *(.data..gate.xen)
- __xen_stop_gate_section = .;
-#endif
}
/*
* make sure the gate page doesn't expose
+++ /dev/null
-#
-# This Kconfig describes xen/ia64 options
-#
-
-config XEN
- bool "Xen hypervisor support"
- default y
- depends on PARAVIRT && MCKINLEY && IA64_PAGE_SIZE_16KB
- select XEN_XENCOMM
- select NO_IDLE_HZ
- # followings are required to save/restore.
- select ARCH_SUSPEND_POSSIBLE
- select SUSPEND
- select PM_SLEEP
- help
- Enable Xen hypervisor support. Resulting kernel runs
- both as a guest OS on Xen and natively on hardware.
-
-config XEN_XENCOMM
- depends on XEN
- bool
-
-config NO_IDLE_HZ
- depends on XEN
- bool
+++ /dev/null
-#
-# Makefile for Xen components
-#
-
-obj-y := hypercall.o xenivt.o xensetup.o xen_pv_ops.o irq_xen.o \
- hypervisor.o xencomm.o xcom_hcall.o grant-table.o time.o suspend.o \
- gate-data.o
-
-obj-$(CONFIG_IA64_GENERIC) += machvec.o
-
-# The gate DSO image is built using a special linker script.
-include $(srctree)/arch/ia64/kernel/Makefile.gate
-
-# tell compiled for xen
-CPPFLAGS_gate.lds += -D__IA64_GATE_PARAVIRTUALIZED_XEN
-AFLAGS_gate.o += -D__IA64_ASM_PARAVIRTUALIZED_XEN -D__IA64_GATE_PARAVIRTUALIZED_XEN
-
-# use same file of native.
-$(obj)/gate.o: $(src)/../kernel/gate.S FORCE
- $(call if_changed_dep,as_o_S)
-$(obj)/gate.lds: $(src)/../kernel/gate.lds.S FORCE
- $(call if_changed_dep,cpp_lds_S)
-
-
-AFLAGS_xenivt.o += -D__IA64_ASM_PARAVIRTUALIZED_XEN
-
-# xen multi compile
-ASM_PARAVIRT_MULTI_COMPILE_SRCS = ivt.S entry.S fsys.S
-ASM_PARAVIRT_OBJS = $(addprefix xen-,$(ASM_PARAVIRT_MULTI_COMPILE_SRCS:.S=.o))
-obj-y += $(ASM_PARAVIRT_OBJS)
-define paravirtualized_xen
-AFLAGS_$(1) += -D__IA64_ASM_PARAVIRTUALIZED_XEN
-endef
-$(foreach o,$(ASM_PARAVIRT_OBJS),$(eval $(call paravirtualized_xen,$(o))))
-
-$(obj)/xen-%.o: $(src)/../kernel/%.S FORCE
- $(call if_changed_dep,as_o_S)
+++ /dev/null
- .section .data..gate.xen, "aw"
-
- .incbin "arch/ia64/xen/gate.so"
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/grant-table.c
- *
- * Copyright (c) 2006 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <linux/module.h>
-#include <linux/vmalloc.h>
-#include <linux/slab.h>
-#include <linux/mm.h>
-
-#include <xen/interface/xen.h>
-#include <xen/interface/memory.h>
-#include <xen/grant_table.h>
-
-#include <asm/xen/hypervisor.h>
-
-/****************************************************************************
- * grant table hack
- * cmd: GNTTABOP_xxx
- */
-
-int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes,
- unsigned long max_nr_gframes,
- struct grant_entry **__shared)
-{
- *__shared = __va(frames[0] << PAGE_SHIFT);
- return 0;
-}
-
-void arch_gnttab_unmap_shared(struct grant_entry *shared,
- unsigned long nr_gframes)
-{
- /* nothing */
-}
-
-static void
-gnttab_map_grant_ref_pre(struct gnttab_map_grant_ref *uop)
-{
- uint32_t flags;
-
- flags = uop->flags;
-
- if (flags & GNTMAP_host_map) {
- if (flags & GNTMAP_application_map) {
- printk(KERN_DEBUG
- "GNTMAP_application_map is not supported yet: "
- "flags 0x%x\n", flags);
- BUG();
- }
- if (flags & GNTMAP_contains_pte) {
- printk(KERN_DEBUG
- "GNTMAP_contains_pte is not supported yet: "
- "flags 0x%x\n", flags);
- BUG();
- }
- } else if (flags & GNTMAP_device_map) {
- printk("GNTMAP_device_map is not supported yet 0x%x\n", flags);
- BUG(); /* not yet. actually this flag is not used. */
- } else {
- BUG();
- }
-}
-
-int
-HYPERVISOR_grant_table_op(unsigned int cmd, void *uop, unsigned int count)
-{
- if (cmd == GNTTABOP_map_grant_ref) {
- unsigned int i;
- for (i = 0; i < count; i++) {
- gnttab_map_grant_ref_pre(
- (struct gnttab_map_grant_ref *)uop + i);
- }
- }
- return xencomm_hypercall_grant_table_op(cmd, uop, count);
-}
-
-EXPORT_SYMBOL(HYPERVISOR_grant_table_op);
+++ /dev/null
-/*
- * Support routines for Xen hypercalls
- *
- * Copyright (C) 2005 Dan Magenheimer <dan.magenheimer@hp.com>
- * Copyright (C) 2008 Yaozu (Eddie) Dong <eddie.dong@intel.com>
- */
-
-#include <asm/asmmacro.h>
-#include <asm/intrinsics.h>
-#include <asm/xen/privop.h>
-
-#ifdef __INTEL_COMPILER
-/*
- * Hypercalls without parameter.
- */
-#define __HCALL0(name,hcall) \
- GLOBAL_ENTRY(name); \
- break hcall; \
- br.ret.sptk.many rp; \
- END(name)
-
-/*
- * Hypercalls with 1 parameter.
- */
-#define __HCALL1(name,hcall) \
- GLOBAL_ENTRY(name); \
- mov r8=r32; \
- break hcall; \
- br.ret.sptk.many rp; \
- END(name)
-
-/*
- * Hypercalls with 2 parameters.
- */
-#define __HCALL2(name,hcall) \
- GLOBAL_ENTRY(name); \
- mov r8=r32; \
- mov r9=r33; \
- break hcall; \
- br.ret.sptk.many rp; \
- END(name)
-
-__HCALL0(xen_get_psr, HYPERPRIVOP_GET_PSR)
-__HCALL0(xen_get_ivr, HYPERPRIVOP_GET_IVR)
-__HCALL0(xen_get_tpr, HYPERPRIVOP_GET_TPR)
-__HCALL0(xen_hyper_ssm_i, HYPERPRIVOP_SSM_I)
-
-__HCALL1(xen_set_tpr, HYPERPRIVOP_SET_TPR)
-__HCALL1(xen_eoi, HYPERPRIVOP_EOI)
-__HCALL1(xen_thash, HYPERPRIVOP_THASH)
-__HCALL1(xen_set_itm, HYPERPRIVOP_SET_ITM)
-__HCALL1(xen_get_rr, HYPERPRIVOP_GET_RR)
-__HCALL1(xen_fc, HYPERPRIVOP_FC)
-__HCALL1(xen_get_cpuid, HYPERPRIVOP_GET_CPUID)
-__HCALL1(xen_get_pmd, HYPERPRIVOP_GET_PMD)
-
-__HCALL2(xen_ptcga, HYPERPRIVOP_PTC_GA)
-__HCALL2(xen_set_rr, HYPERPRIVOP_SET_RR)
-__HCALL2(xen_set_kr, HYPERPRIVOP_SET_KR)
-
-GLOBAL_ENTRY(xen_set_rr0_to_rr4)
- mov r8=r32
- mov r9=r33
- mov r10=r34
- mov r11=r35
- mov r14=r36
- XEN_HYPER_SET_RR0_TO_RR4
- br.ret.sptk.many rp
- ;;
-END(xen_set_rr0_to_rr4)
-#endif
-
-GLOBAL_ENTRY(xen_send_ipi)
- mov r14=r32
- mov r15=r33
- mov r2=0x400
- break 0x1000
- ;;
- br.ret.sptk.many rp
- ;;
-END(xen_send_ipi)
-
-GLOBAL_ENTRY(__hypercall)
- mov r2=r37
- break 0x1000
- br.ret.sptk.many b0
- ;;
-END(__hypercall)
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/hypervisor.c
- *
- * Copyright (c) 2006 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <linux/efi.h>
-#include <linux/export.h>
-#include <asm/xen/hypervisor.h>
-#include <asm/xen/privop.h>
-
-#include "irq_xen.h"
-
-struct shared_info *HYPERVISOR_shared_info __read_mostly =
- (struct shared_info *)XSI_BASE;
-EXPORT_SYMBOL(HYPERVISOR_shared_info);
-
-DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);
-
-struct start_info *xen_start_info;
-EXPORT_SYMBOL(xen_start_info);
-
-EXPORT_SYMBOL(xen_domain_type);
-
-EXPORT_SYMBOL(__hypercall);
-
-/* Stolen from arch/x86/xen/enlighten.c */
-/*
- * Flag to determine whether vcpu info placement is available on all
- * VCPUs. We assume it is to start with, and then set it to zero on
- * the first failure. This is because it can succeed on some VCPUs
- * and not others, since it can involve hypervisor memory allocation,
- * or because the guest failed to guarantee all the appropriate
- * constraints on all VCPUs (ie buffer can't cross a page boundary).
- *
- * Note that any particular CPU may be using a placed vcpu structure,
- * but we can only optimise if the all are.
- *
- * 0: not available, 1: available
- */
-
-static void __init xen_vcpu_setup(int cpu)
-{
- /*
- * WARNING:
- * before changing MAX_VIRT_CPUS,
- * check that shared_info fits on a page
- */
- BUILD_BUG_ON(sizeof(struct shared_info) > PAGE_SIZE);
- per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu];
-}
-
-void __init xen_setup_vcpu_info_placement(void)
-{
- int cpu;
-
- for_each_possible_cpu(cpu)
- xen_vcpu_setup(cpu);
-}
-
-void
-xen_cpu_init(void)
-{
- xen_smp_intr_init();
-}
-
-/**************************************************************************
- * opt feature
- */
-void
-xen_ia64_enable_opt_feature(void)
-{
- /* Enable region 7 identity map optimizations in Xen */
- struct xen_ia64_opt_feature optf;
-
- optf.cmd = XEN_IA64_OPTF_IDENT_MAP_REG7;
- optf.on = XEN_IA64_OPTF_ON;
- optf.pgprot = pgprot_val(PAGE_KERNEL);
- optf.key = 0; /* No key on linux. */
- HYPERVISOR_opt_feature(&optf);
-}
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/irq_xen.c
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <linux/cpu.h>
-
-#include <xen/interface/xen.h>
-#include <xen/interface/callback.h>
-#include <xen/events.h>
-
-#include <asm/xen/privop.h>
-
-#include "irq_xen.h"
-
-/***************************************************************************
- * pv_irq_ops
- * irq operations
- */
-
-static int
-xen_assign_irq_vector(int irq)
-{
- struct physdev_irq irq_op;
-
- irq_op.irq = irq;
- if (HYPERVISOR_physdev_op(PHYSDEVOP_alloc_irq_vector, &irq_op))
- return -ENOSPC;
-
- return irq_op.vector;
-}
-
-static void
-xen_free_irq_vector(int vector)
-{
- struct physdev_irq irq_op;
-
- if (vector < IA64_FIRST_DEVICE_VECTOR ||
- vector > IA64_LAST_DEVICE_VECTOR)
- return;
-
- irq_op.vector = vector;
- if (HYPERVISOR_physdev_op(PHYSDEVOP_free_irq_vector, &irq_op))
- printk(KERN_WARNING "%s: xen_free_irq_vector fail vector=%d\n",
- __func__, vector);
-}
-
-
-static DEFINE_PER_CPU(int, xen_timer_irq) = -1;
-static DEFINE_PER_CPU(int, xen_ipi_irq) = -1;
-static DEFINE_PER_CPU(int, xen_resched_irq) = -1;
-static DEFINE_PER_CPU(int, xen_cmc_irq) = -1;
-static DEFINE_PER_CPU(int, xen_cmcp_irq) = -1;
-static DEFINE_PER_CPU(int, xen_cpep_irq) = -1;
-#define NAME_SIZE 15
-static DEFINE_PER_CPU(char[NAME_SIZE], xen_timer_name);
-static DEFINE_PER_CPU(char[NAME_SIZE], xen_ipi_name);
-static DEFINE_PER_CPU(char[NAME_SIZE], xen_resched_name);
-static DEFINE_PER_CPU(char[NAME_SIZE], xen_cmc_name);
-static DEFINE_PER_CPU(char[NAME_SIZE], xen_cmcp_name);
-static DEFINE_PER_CPU(char[NAME_SIZE], xen_cpep_name);
-#undef NAME_SIZE
-
-struct saved_irq {
- unsigned int irq;
- struct irqaction *action;
-};
-/* 16 should be far optimistic value, since only several percpu irqs
- * are registered early.
- */
-#define MAX_LATE_IRQ 16
-static struct saved_irq saved_percpu_irqs[MAX_LATE_IRQ];
-static unsigned short late_irq_cnt;
-static unsigned short saved_irq_cnt;
-static int xen_slab_ready;
-
-#ifdef CONFIG_SMP
-#include <linux/sched.h>
-
-/* Dummy stub. Though we may check XEN_RESCHEDULE_VECTOR before __do_IRQ,
- * it ends up to issue several memory accesses upon percpu data and
- * thus adds unnecessary traffic to other paths.
- */
-static irqreturn_t
-xen_dummy_handler(int irq, void *dev_id)
-{
- return IRQ_HANDLED;
-}
-
-static irqreturn_t
-xen_resched_handler(int irq, void *dev_id)
-{
- scheduler_ipi();
- return IRQ_HANDLED;
-}
-
-static struct irqaction xen_ipi_irqaction = {
- .handler = handle_IPI,
- .flags = IRQF_DISABLED,
- .name = "IPI"
-};
-
-static struct irqaction xen_resched_irqaction = {
- .handler = xen_resched_handler,
- .flags = IRQF_DISABLED,
- .name = "resched"
-};
-
-static struct irqaction xen_tlb_irqaction = {
- .handler = xen_dummy_handler,
- .flags = IRQF_DISABLED,
- .name = "tlb_flush"
-};
-#endif
-
-/*
- * This is xen version percpu irq registration, which needs bind
- * to xen specific evtchn sub-system. One trick here is that xen
- * evtchn binding interface depends on kmalloc because related
- * port needs to be freed at device/cpu down. So we cache the
- * registration on BSP before slab is ready and then deal them
- * at later point. For rest instances happening after slab ready,
- * we hook them to xen evtchn immediately.
- *
- * FIXME: MCA is not supported by far, and thus "nomca" boot param is
- * required.
- */
-static void
-__xen_register_percpu_irq(unsigned int cpu, unsigned int vec,
- struct irqaction *action, int save)
-{
- int irq = 0;
-
- if (xen_slab_ready) {
- switch (vec) {
- case IA64_TIMER_VECTOR:
- snprintf(per_cpu(xen_timer_name, cpu),
- sizeof(per_cpu(xen_timer_name, cpu)),
- "%s%d", action->name, cpu);
- irq = bind_virq_to_irqhandler(VIRQ_ITC, cpu,
- action->handler, action->flags,
- per_cpu(xen_timer_name, cpu), action->dev_id);
- per_cpu(xen_timer_irq, cpu) = irq;
- break;
- case IA64_IPI_RESCHEDULE:
- snprintf(per_cpu(xen_resched_name, cpu),
- sizeof(per_cpu(xen_resched_name, cpu)),
- "%s%d", action->name, cpu);
- irq = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR, cpu,
- action->handler, action->flags,
- per_cpu(xen_resched_name, cpu), action->dev_id);
- per_cpu(xen_resched_irq, cpu) = irq;
- break;
- case IA64_IPI_VECTOR:
- snprintf(per_cpu(xen_ipi_name, cpu),
- sizeof(per_cpu(xen_ipi_name, cpu)),
- "%s%d", action->name, cpu);
- irq = bind_ipi_to_irqhandler(XEN_IPI_VECTOR, cpu,
- action->handler, action->flags,
- per_cpu(xen_ipi_name, cpu), action->dev_id);
- per_cpu(xen_ipi_irq, cpu) = irq;
- break;
- case IA64_CMC_VECTOR:
- snprintf(per_cpu(xen_cmc_name, cpu),
- sizeof(per_cpu(xen_cmc_name, cpu)),
- "%s%d", action->name, cpu);
- irq = bind_virq_to_irqhandler(VIRQ_MCA_CMC, cpu,
- action->handler,
- action->flags,
- per_cpu(xen_cmc_name, cpu),
- action->dev_id);
- per_cpu(xen_cmc_irq, cpu) = irq;
- break;
- case IA64_CMCP_VECTOR:
- snprintf(per_cpu(xen_cmcp_name, cpu),
- sizeof(per_cpu(xen_cmcp_name, cpu)),
- "%s%d", action->name, cpu);
- irq = bind_ipi_to_irqhandler(XEN_CMCP_VECTOR, cpu,
- action->handler,
- action->flags,
- per_cpu(xen_cmcp_name, cpu),
- action->dev_id);
- per_cpu(xen_cmcp_irq, cpu) = irq;
- break;
- case IA64_CPEP_VECTOR:
- snprintf(per_cpu(xen_cpep_name, cpu),
- sizeof(per_cpu(xen_cpep_name, cpu)),
- "%s%d", action->name, cpu);
- irq = bind_ipi_to_irqhandler(XEN_CPEP_VECTOR, cpu,
- action->handler,
- action->flags,
- per_cpu(xen_cpep_name, cpu),
- action->dev_id);
- per_cpu(xen_cpep_irq, cpu) = irq;
- break;
- case IA64_CPE_VECTOR:
- case IA64_MCA_RENDEZ_VECTOR:
- case IA64_PERFMON_VECTOR:
- case IA64_MCA_WAKEUP_VECTOR:
- case IA64_SPURIOUS_INT_VECTOR:
- /* No need to complain, these aren't supported. */
- break;
- default:
- printk(KERN_WARNING "Percpu irq %d is unsupported "
- "by xen!\n", vec);
- break;
- }
- BUG_ON(irq < 0);
-
- if (irq > 0) {
- /*
- * Mark percpu. Without this, migrate_irqs() will
- * mark the interrupt for migrations and trigger it
- * on cpu hotplug.
- */
- irq_set_status_flags(irq, IRQ_PER_CPU);
- }
- }
-
- /* For BSP, we cache registered percpu irqs, and then re-walk
- * them when initializing APs
- */
- if (!cpu && save) {
- BUG_ON(saved_irq_cnt == MAX_LATE_IRQ);
- saved_percpu_irqs[saved_irq_cnt].irq = vec;
- saved_percpu_irqs[saved_irq_cnt].action = action;
- saved_irq_cnt++;
- if (!xen_slab_ready)
- late_irq_cnt++;
- }
-}
-
-static void
-xen_register_percpu_irq(ia64_vector vec, struct irqaction *action)
-{
- __xen_register_percpu_irq(smp_processor_id(), vec, action, 1);
-}
-
-static void
-xen_bind_early_percpu_irq(void)
-{
- int i;
-
- xen_slab_ready = 1;
- /* There's no race when accessing this cached array, since only
- * BSP will face with such step shortly
- */
- for (i = 0; i < late_irq_cnt; i++)
- __xen_register_percpu_irq(smp_processor_id(),
- saved_percpu_irqs[i].irq,
- saved_percpu_irqs[i].action, 0);
-}
-
-/* FIXME: There's no obvious point to check whether slab is ready. So
- * a hack is used here by utilizing a late time hook.
- */
-
-#ifdef CONFIG_HOTPLUG_CPU
-static int unbind_evtchn_callback(struct notifier_block *nfb,
- unsigned long action, void *hcpu)
-{
- unsigned int cpu = (unsigned long)hcpu;
-
- if (action == CPU_DEAD) {
- /* Unregister evtchn. */
- if (per_cpu(xen_cpep_irq, cpu) >= 0) {
- unbind_from_irqhandler(per_cpu(xen_cpep_irq, cpu),
- NULL);
- per_cpu(xen_cpep_irq, cpu) = -1;
- }
- if (per_cpu(xen_cmcp_irq, cpu) >= 0) {
- unbind_from_irqhandler(per_cpu(xen_cmcp_irq, cpu),
- NULL);
- per_cpu(xen_cmcp_irq, cpu) = -1;
- }
- if (per_cpu(xen_cmc_irq, cpu) >= 0) {
- unbind_from_irqhandler(per_cpu(xen_cmc_irq, cpu), NULL);
- per_cpu(xen_cmc_irq, cpu) = -1;
- }
- if (per_cpu(xen_ipi_irq, cpu) >= 0) {
- unbind_from_irqhandler(per_cpu(xen_ipi_irq, cpu), NULL);
- per_cpu(xen_ipi_irq, cpu) = -1;
- }
- if (per_cpu(xen_resched_irq, cpu) >= 0) {
- unbind_from_irqhandler(per_cpu(xen_resched_irq, cpu),
- NULL);
- per_cpu(xen_resched_irq, cpu) = -1;
- }
- if (per_cpu(xen_timer_irq, cpu) >= 0) {
- unbind_from_irqhandler(per_cpu(xen_timer_irq, cpu),
- NULL);
- per_cpu(xen_timer_irq, cpu) = -1;
- }
- }
- return NOTIFY_OK;
-}
-
-static struct notifier_block unbind_evtchn_notifier = {
- .notifier_call = unbind_evtchn_callback,
- .priority = 0
-};
-#endif
-
-void xen_smp_intr_init_early(unsigned int cpu)
-{
-#ifdef CONFIG_SMP
- unsigned int i;
-
- for (i = 0; i < saved_irq_cnt; i++)
- __xen_register_percpu_irq(cpu, saved_percpu_irqs[i].irq,
- saved_percpu_irqs[i].action, 0);
-#endif
-}
-
-void xen_smp_intr_init(void)
-{
-#ifdef CONFIG_SMP
- unsigned int cpu = smp_processor_id();
- struct callback_register event = {
- .type = CALLBACKTYPE_event,
- .address = { .ip = (unsigned long)&xen_event_callback },
- };
-
- if (cpu == 0) {
- /* Initialization was already done for boot cpu. */
-#ifdef CONFIG_HOTPLUG_CPU
- /* Register the notifier only once. */
- register_cpu_notifier(&unbind_evtchn_notifier);
-#endif
- return;
- }
-
- /* This should be piggyback when setup vcpu guest context */
- BUG_ON(HYPERVISOR_callback_op(CALLBACKOP_register, &event));
-#endif /* CONFIG_SMP */
-}
-
-void __init
-xen_irq_init(void)
-{
- struct callback_register event = {
- .type = CALLBACKTYPE_event,
- .address = { .ip = (unsigned long)&xen_event_callback },
- };
-
- xen_init_IRQ();
- BUG_ON(HYPERVISOR_callback_op(CALLBACKOP_register, &event));
- late_time_init = xen_bind_early_percpu_irq;
-}
-
-void
-xen_platform_send_ipi(int cpu, int vector, int delivery_mode, int redirect)
-{
-#ifdef CONFIG_SMP
- /* TODO: we need to call vcpu_up here */
- if (unlikely(vector == ap_wakeup_vector)) {
- /* XXX
- * This should be in __cpu_up(cpu) in ia64 smpboot.c
- * like x86. But don't want to modify it,
- * keep it untouched.
- */
- xen_smp_intr_init_early(cpu);
-
- xen_send_ipi(cpu, vector);
- /* vcpu_prepare_and_up(cpu); */
- return;
- }
-#endif
-
- switch (vector) {
- case IA64_IPI_VECTOR:
- xen_send_IPI_one(cpu, XEN_IPI_VECTOR);
- break;
- case IA64_IPI_RESCHEDULE:
- xen_send_IPI_one(cpu, XEN_RESCHEDULE_VECTOR);
- break;
- case IA64_CMCP_VECTOR:
- xen_send_IPI_one(cpu, XEN_CMCP_VECTOR);
- break;
- case IA64_CPEP_VECTOR:
- xen_send_IPI_one(cpu, XEN_CPEP_VECTOR);
- break;
- case IA64_TIMER_VECTOR: {
- /* this is used only once by check_sal_cache_flush()
- at boot time */
- static int used = 0;
- if (!used) {
- xen_send_ipi(cpu, IA64_TIMER_VECTOR);
- used = 1;
- break;
- }
- /* fallthrough */
- }
- default:
- printk(KERN_WARNING "Unsupported IPI type 0x%x\n",
- vector);
- notify_remote_via_irq(0); /* defaults to 0 irq */
- break;
- }
-}
-
-static void __init
-xen_register_ipi(void)
-{
-#ifdef CONFIG_SMP
- register_percpu_irq(IA64_IPI_VECTOR, &xen_ipi_irqaction);
- register_percpu_irq(IA64_IPI_RESCHEDULE, &xen_resched_irqaction);
- register_percpu_irq(IA64_IPI_LOCAL_TLB_FLUSH, &xen_tlb_irqaction);
-#endif
-}
-
-static void
-xen_resend_irq(unsigned int vector)
-{
- (void)resend_irq_on_evtchn(vector);
-}
-
-const struct pv_irq_ops xen_irq_ops __initconst = {
- .register_ipi = xen_register_ipi,
-
- .assign_irq_vector = xen_assign_irq_vector,
- .free_irq_vector = xen_free_irq_vector,
- .register_percpu_irq = xen_register_percpu_irq,
-
- .resend_irq = xen_resend_irq,
-};
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/irq_xen.h
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#ifndef IRQ_XEN_H
-#define IRQ_XEN_H
-
-extern void (*late_time_init)(void);
-extern char xen_event_callback;
-void __init xen_init_IRQ(void);
-
-extern const struct pv_irq_ops xen_irq_ops __initconst;
-extern void xen_smp_intr_init(void);
-extern void xen_send_ipi(int cpu, int vec);
-
-#endif /* IRQ_XEN_H */
+++ /dev/null
-#define MACHVEC_PLATFORM_NAME xen
-#define MACHVEC_PLATFORM_HEADER <asm/machvec_xen.h>
-#include <asm/machvec_init.h>
-
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/suspend.c
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- * suspend/resume
- */
-
-#include <xen/xen-ops.h>
-#include <asm/xen/hypervisor.h>
-#include "time.h"
-
-void
-xen_mm_pin_all(void)
-{
- /* nothing */
-}
-
-void
-xen_mm_unpin_all(void)
-{
- /* nothing */
-}
-
-void
-xen_arch_pre_suspend()
-{
- /* nothing */
-}
-
-void
-xen_arch_post_suspend(int suspend_cancelled)
-{
- if (suspend_cancelled)
- return;
-
- xen_ia64_enable_opt_feature();
- /* add more if necessary */
-}
-
-void xen_arch_resume(void)
-{
- xen_timer_resume_on_aps();
-}
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/time.c
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <linux/delay.h>
-#include <linux/kernel_stat.h>
-#include <linux/posix-timers.h>
-#include <linux/irq.h>
-#include <linux/clocksource.h>
-
-#include <asm/timex.h>
-
-#include <asm/xen/hypervisor.h>
-
-#include <xen/interface/vcpu.h>
-
-#include "../kernel/fsyscall_gtod_data.h"
-
-static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate);
-static DEFINE_PER_CPU(unsigned long, xen_stolen_time);
-static DEFINE_PER_CPU(unsigned long, xen_blocked_time);
-
-/* taken from i386/kernel/time-xen.c */
-static void xen_init_missing_ticks_accounting(int cpu)
-{
- struct vcpu_register_runstate_memory_area area;
- struct vcpu_runstate_info *runstate = &per_cpu(xen_runstate, cpu);
- int rc;
-
- memset(runstate, 0, sizeof(*runstate));
-
- area.addr.v = runstate;
- rc = HYPERVISOR_vcpu_op(VCPUOP_register_runstate_memory_area, cpu,
- &area);
- WARN_ON(rc && rc != -ENOSYS);
-
- per_cpu(xen_blocked_time, cpu) = runstate->time[RUNSTATE_blocked];
- per_cpu(xen_stolen_time, cpu) = runstate->time[RUNSTATE_runnable]
- + runstate->time[RUNSTATE_offline];
-}
-
-/*
- * Runstate accounting
- */
-/* stolen from arch/x86/xen/time.c */
-static void get_runstate_snapshot(struct vcpu_runstate_info *res)
-{
- u64 state_time;
- struct vcpu_runstate_info *state;
-
- BUG_ON(preemptible());
-
- state = &__get_cpu_var(xen_runstate);
-
- /*
- * The runstate info is always updated by the hypervisor on
- * the current CPU, so there's no need to use anything
- * stronger than a compiler barrier when fetching it.
- */
- do {
- state_time = state->state_entry_time;
- rmb();
- *res = *state;
- rmb();
- } while (state->state_entry_time != state_time);
-}
-
-#define NS_PER_TICK (1000000000LL/HZ)
-
-static unsigned long
-consider_steal_time(unsigned long new_itm)
-{
- unsigned long stolen, blocked;
- unsigned long delta_itm = 0, stolentick = 0;
- int cpu = smp_processor_id();
- struct vcpu_runstate_info runstate;
- struct task_struct *p = current;
-
- get_runstate_snapshot(&runstate);
-
- /*
- * Check for vcpu migration effect
- * In this case, itc value is reversed.
- * This causes huge stolen value.
- * This function just checks and reject this effect.
- */
- if (!time_after_eq(runstate.time[RUNSTATE_blocked],
- per_cpu(xen_blocked_time, cpu)))
- blocked = 0;
-
- if (!time_after_eq(runstate.time[RUNSTATE_runnable] +
- runstate.time[RUNSTATE_offline],
- per_cpu(xen_stolen_time, cpu)))
- stolen = 0;
-
- if (!time_after(delta_itm + new_itm, ia64_get_itc()))
- stolentick = ia64_get_itc() - new_itm;
-
- do_div(stolentick, NS_PER_TICK);
- stolentick++;
-
- do_div(stolen, NS_PER_TICK);
-
- if (stolen > stolentick)
- stolen = stolentick;
-
- stolentick -= stolen;
- do_div(blocked, NS_PER_TICK);
-
- if (blocked > stolentick)
- blocked = stolentick;
-
- if (stolen > 0 || blocked > 0) {
- account_steal_ticks(stolen);
- account_idle_ticks(blocked);
- run_local_timers();
-
- rcu_check_callbacks(cpu, user_mode(get_irq_regs()));
-
- scheduler_tick();
- run_posix_cpu_timers(p);
- delta_itm += local_cpu_data->itm_delta * (stolen + blocked);
-
- if (cpu == time_keeper_id)
- xtime_update(stolen + blocked);
-
- local_cpu_data->itm_next = delta_itm + new_itm;
-
- per_cpu(xen_stolen_time, cpu) += NS_PER_TICK * stolen;
- per_cpu(xen_blocked_time, cpu) += NS_PER_TICK * blocked;
- }
- return delta_itm;
-}
-
-static int xen_do_steal_accounting(unsigned long *new_itm)
-{
- unsigned long delta_itm;
- delta_itm = consider_steal_time(*new_itm);
- *new_itm += delta_itm;
- if (time_after(*new_itm, ia64_get_itc()) && delta_itm)
- return 1;
-
- return 0;
-}
-
-static void xen_itc_jitter_data_reset(void)
-{
- u64 lcycle, ret;
-
- do {
- lcycle = itc_jitter_data.itc_lastcycle;
- ret = cmpxchg(&itc_jitter_data.itc_lastcycle, lcycle, 0);
- } while (unlikely(ret != lcycle));
-}
-
-/* based on xen_sched_clock() in arch/x86/xen/time.c. */
-/*
- * This relies on HAVE_UNSTABLE_SCHED_CLOCK. If it can't be defined,
- * something similar logic should be implemented here.
- */
-/*
- * Xen sched_clock implementation. Returns the number of unstolen
- * nanoseconds, which is nanoseconds the VCPU spent in RUNNING+BLOCKED
- * states.
- */
-static unsigned long long xen_sched_clock(void)
-{
- struct vcpu_runstate_info runstate;
-
- unsigned long long now;
- unsigned long long offset;
- unsigned long long ret;
-
- /*
- * Ideally sched_clock should be called on a per-cpu basis
- * anyway, so preempt should already be disabled, but that's
- * not current practice at the moment.
- */
- preempt_disable();
-
- /*
- * both ia64_native_sched_clock() and xen's runstate are
- * based on mAR.ITC. So difference of them makes sense.
- */
- now = ia64_native_sched_clock();
-
- get_runstate_snapshot(&runstate);
-
- WARN_ON(runstate.state != RUNSTATE_running);
-
- offset = 0;
- if (now > runstate.state_entry_time)
- offset = now - runstate.state_entry_time;
- ret = runstate.time[RUNSTATE_blocked] +
- runstate.time[RUNSTATE_running] +
- offset;
-
- preempt_enable();
-
- return ret;
-}
-
-struct pv_time_ops xen_time_ops __initdata = {
- .init_missing_ticks_accounting = xen_init_missing_ticks_accounting,
- .do_steal_accounting = xen_do_steal_accounting,
- .clocksource_resume = xen_itc_jitter_data_reset,
- .sched_clock = xen_sched_clock,
-};
-
-/* Called after suspend, to resume time. */
-static void xen_local_tick_resume(void)
-{
- /* Just trigger a tick. */
- ia64_cpu_local_tick();
- touch_softlockup_watchdog();
-}
-
-void
-xen_timer_resume(void)
-{
- unsigned int cpu;
-
- xen_local_tick_resume();
-
- for_each_online_cpu(cpu)
- xen_init_missing_ticks_accounting(cpu);
-}
-
-static void ia64_cpu_local_tick_fn(void *unused)
-{
- xen_local_tick_resume();
- xen_init_missing_ticks_accounting(smp_processor_id());
-}
-
-void
-xen_timer_resume_on_aps(void)
-{
- smp_call_function(&ia64_cpu_local_tick_fn, NULL, 1);
-}
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/time.h
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-extern struct pv_time_ops xen_time_ops __initdata;
-void xen_timer_resume_on_aps(void);
+++ /dev/null
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
- *
- * Tristan Gingold <tristan.gingold@bull.net>
- *
- * Copyright (c) 2007
- * Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- * consolidate mini and inline version.
- */
-
-#include <linux/module.h>
-#include <xen/interface/xen.h>
-#include <xen/interface/memory.h>
-#include <xen/interface/grant_table.h>
-#include <xen/interface/callback.h>
-#include <xen/interface/vcpu.h>
-#include <asm/xen/hypervisor.h>
-#include <asm/xen/xencomm.h>
-
-/* Xencomm notes:
- * This file defines hypercalls to be used by xencomm. The hypercalls simply
- * create inlines or mini descriptors for pointers and then call the raw arch
- * hypercall xencomm_arch_hypercall_XXX
- *
- * If the arch wants to directly use these hypercalls, simply define macros
- * in asm/xen/hypercall.h, eg:
- * #define HYPERVISOR_sched_op xencomm_hypercall_sched_op
- *
- * The arch may also define HYPERVISOR_xxx as a function and do more operations
- * before/after doing the hypercall.
- *
- * Note: because only inline or mini descriptors are created these functions
- * must only be called with in kernel memory parameters.
- */
-
-int
-xencomm_hypercall_console_io(int cmd, int count, char *str)
-{
- /* xen early printk uses console io hypercall before
- * xencomm initialization. In that case, we just ignore it.
- */
- if (!xencomm_is_initialized())
- return 0;
-
- return xencomm_arch_hypercall_console_io
- (cmd, count, xencomm_map_no_alloc(str, count));
-}
-EXPORT_SYMBOL_GPL(xencomm_hypercall_console_io);
-
-int
-xencomm_hypercall_event_channel_op(int cmd, void *op)
-{
- struct xencomm_handle *desc;
- desc = xencomm_map_no_alloc(op, sizeof(struct evtchn_op));
- if (desc == NULL)
- return -EINVAL;
-
- return xencomm_arch_hypercall_event_channel_op(cmd, desc);
-}
-EXPORT_SYMBOL_GPL(xencomm_hypercall_event_channel_op);
-
-int
-xencomm_hypercall_xen_version(int cmd, void *arg)
-{
- struct xencomm_handle *desc;
- unsigned int argsize;
-
- switch (cmd) {
- case XENVER_version:
- /* do not actually pass an argument */
- return xencomm_arch_hypercall_xen_version(cmd, 0);
- case XENVER_extraversion:
- argsize = sizeof(struct xen_extraversion);
- break;
- case XENVER_compile_info:
- argsize = sizeof(struct xen_compile_info);
- break;
- case XENVER_capabilities:
- argsize = sizeof(struct xen_capabilities_info);
- break;
- case XENVER_changeset:
- argsize = sizeof(struct xen_changeset_info);
- break;
- case XENVER_platform_parameters:
- argsize = sizeof(struct xen_platform_parameters);
- break;
- case XENVER_get_features:
- argsize = (arg == NULL) ? 0 : sizeof(struct xen_feature_info);
- break;
-
- default:
- printk(KERN_DEBUG
- "%s: unknown version op %d\n", __func__, cmd);
- return -ENOSYS;
- }
-
- desc = xencomm_map_no_alloc(arg, argsize);
- if (desc == NULL)
- return -EINVAL;
-
- return xencomm_arch_hypercall_xen_version(cmd, desc);
-}
-EXPORT_SYMBOL_GPL(xencomm_hypercall_xen_version);
-
-int
-xencomm_hypercall_physdev_op(int cmd, void *op)
-{
- unsigned int argsize;
-
- switch (cmd) {
- case PHYSDEVOP_apic_read:
- case PHYSDEVOP_apic_write:
- argsize = sizeof(struct physdev_apic);
- break;
- case PHYSDEVOP_alloc_irq_vector:
- case PHYSDEVOP_free_irq_vector:
- argsize = sizeof(struct physdev_irq);
- break;
- case PHYSDEVOP_irq_status_query:
- argsize = sizeof(struct physdev_irq_status_query);
- break;
-
- default:
- printk(KERN_DEBUG
- "%s: unknown physdev op %d\n", __func__, cmd);
- return -ENOSYS;
- }
-
- return xencomm_arch_hypercall_physdev_op
- (cmd, xencomm_map_no_alloc(op, argsize));
-}
-
-static int
-xencommize_grant_table_op(struct xencomm_mini **xc_area,
- unsigned int cmd, void *op, unsigned int count,
- struct xencomm_handle **desc)
-{
- struct xencomm_handle *desc1;
- unsigned int argsize;
-
- switch (cmd) {
- case GNTTABOP_map_grant_ref:
- argsize = sizeof(struct gnttab_map_grant_ref);
- break;
- case GNTTABOP_unmap_grant_ref:
- argsize = sizeof(struct gnttab_unmap_grant_ref);
- break;
- case GNTTABOP_setup_table:
- {
- struct gnttab_setup_table *setup = op;
-
- argsize = sizeof(*setup);
-
- if (count != 1)
- return -EINVAL;
- desc1 = __xencomm_map_no_alloc
- (xen_guest_handle(setup->frame_list),
- setup->nr_frames *
- sizeof(*xen_guest_handle(setup->frame_list)),
- *xc_area);
- if (desc1 == NULL)
- return -EINVAL;
- (*xc_area)++;
- set_xen_guest_handle(setup->frame_list, (void *)desc1);
- break;
- }
- case GNTTABOP_dump_table:
- argsize = sizeof(struct gnttab_dump_table);
- break;
- case GNTTABOP_transfer:
- argsize = sizeof(struct gnttab_transfer);
- break;
- case GNTTABOP_copy:
- argsize = sizeof(struct gnttab_copy);
- break;
- case GNTTABOP_query_size:
- argsize = sizeof(struct gnttab_query_size);
- break;
- default:
- printk(KERN_DEBUG "%s: unknown hypercall grant table op %d\n",
- __func__, cmd);
- BUG();
- }
-
- *desc = __xencomm_map_no_alloc(op, count * argsize, *xc_area);
- if (*desc == NULL)
- return -EINVAL;
- (*xc_area)++;
-
- return 0;
-}
-
-int
-xencomm_hypercall_grant_table_op(unsigned int cmd, void *op,
- unsigned int count)
-{
- int rc;
- struct xencomm_handle *desc;
- XENCOMM_MINI_ALIGNED(xc_area, 2);
-
- rc = xencommize_grant_table_op(&xc_area, cmd, op, count, &desc);
- if (rc)
- return rc;
-
- return xencomm_arch_hypercall_grant_table_op(cmd, desc, count);
-}
-EXPORT_SYMBOL_GPL(xencomm_hypercall_grant_table_op);
-
-int
-xencomm_hypercall_sched_op(int cmd, void *arg)
-{
- struct xencomm_handle *desc;
- unsigned int argsize;
-
- switch (cmd) {
- case SCHEDOP_yield:
- case SCHEDOP_block:
- argsize = 0;
- break;
- case SCHEDOP_shutdown:
- argsize = sizeof(struct sched_shutdown);
- break;
- case SCHEDOP_poll:
- {
- struct sched_poll *poll = arg;
- struct xencomm_handle *ports;
-
- argsize = sizeof(struct sched_poll);
- ports = xencomm_map_no_alloc(xen_guest_handle(poll->ports),
- sizeof(*xen_guest_handle(poll->ports)));
-
- set_xen_guest_handle(poll->ports, (void *)ports);
- break;
- }
- default:
- printk(KERN_DEBUG "%s: unknown sched op %d\n", __func__, cmd);
- return -ENOSYS;
- }
-
- desc = xencomm_map_no_alloc(arg, argsize);
- if (desc == NULL)
- return -EINVAL;
-
- return xencomm_arch_hypercall_sched_op(cmd, desc);
-}
-EXPORT_SYMBOL_GPL(xencomm_hypercall_sched_op);
-
-int
-xencomm_hypercall_multicall(void *call_list, int nr_calls)
-{
- int rc;
- int i;
- struct multicall_entry *mce;
- struct xencomm_handle *desc;
- XENCOMM_MINI_ALIGNED(xc_area, nr_calls * 2);
-
- for (i = 0; i < nr_calls; i++) {
- mce = (struct multicall_entry *)call_list + i;
-
- switch (mce->op) {
- case __HYPERVISOR_update_va_mapping:
- case __HYPERVISOR_mmu_update:
- /* No-op on ia64. */
- break;
- case __HYPERVISOR_grant_table_op:
- rc = xencommize_grant_table_op
- (&xc_area,
- mce->args[0], (void *)mce->args[1],
- mce->args[2], &desc);
- if (rc)
- return rc;
- mce->args[1] = (unsigned long)desc;
- break;
- case __HYPERVISOR_memory_op:
- default:
- printk(KERN_DEBUG
- "%s: unhandled multicall op entry op %lu\n",
- __func__, mce->op);
- return -ENOSYS;
- }
- }
-
- desc = xencomm_map_no_alloc(call_list,
- nr_calls * sizeof(struct multicall_entry));
- if (desc == NULL)
- return -EINVAL;
-
- return xencomm_arch_hypercall_multicall(desc, nr_calls);
-}
-EXPORT_SYMBOL_GPL(xencomm_hypercall_multicall);
-
-int
-xencomm_hypercall_callback_op(int cmd, void *arg)
-{
- unsigned int argsize;
- switch (cmd) {
- case CALLBACKOP_register:
- argsize = sizeof(struct callback_register);
- break;
- case CALLBACKOP_unregister:
- argsize = sizeof(struct callback_unregister);
- break;
- default:
- printk(KERN_DEBUG
- "%s: unknown callback op %d\n", __func__, cmd);
- return -ENOSYS;
- }
-
- return xencomm_arch_hypercall_callback_op
- (cmd, xencomm_map_no_alloc(arg, argsize));
-}
-
-static int
-xencommize_memory_reservation(struct xencomm_mini *xc_area,
- struct xen_memory_reservation *mop)
-{
- struct xencomm_handle *desc;
-
- desc = __xencomm_map_no_alloc(xen_guest_handle(mop->extent_start),
- mop->nr_extents *
- sizeof(*xen_guest_handle(mop->extent_start)),
- xc_area);
- if (desc == NULL)
- return -EINVAL;
-
- set_xen_guest_handle(mop->extent_start, (void *)desc);
- return 0;
-}
-
-int
-xencomm_hypercall_memory_op(unsigned int cmd, void *arg)
-{
- GUEST_HANDLE(xen_pfn_t) extent_start_va[2] = { {NULL}, {NULL} };
- struct xen_memory_reservation *xmr = NULL;
- int rc;
- struct xencomm_handle *desc;
- unsigned int argsize;
- XENCOMM_MINI_ALIGNED(xc_area, 2);
-
- switch (cmd) {
- case XENMEM_increase_reservation:
- case XENMEM_decrease_reservation:
- case XENMEM_populate_physmap:
- xmr = (struct xen_memory_reservation *)arg;
- set_xen_guest_handle(extent_start_va[0],
- xen_guest_handle(xmr->extent_start));
-
- argsize = sizeof(*xmr);
- rc = xencommize_memory_reservation(xc_area, xmr);
- if (rc)
- return rc;
- xc_area++;
- break;
-
- case XENMEM_maximum_ram_page:
- argsize = 0;
- break;
-
- case XENMEM_add_to_physmap:
- argsize = sizeof(struct xen_add_to_physmap);
- break;
-
- default:
- printk(KERN_DEBUG "%s: unknown memory op %d\n", __func__, cmd);
- return -ENOSYS;
- }
-
- desc = xencomm_map_no_alloc(arg, argsize);
- if (desc == NULL)
- return -EINVAL;
-
- rc = xencomm_arch_hypercall_memory_op(cmd, desc);
-
- switch (cmd) {
- case XENMEM_increase_reservation:
- case XENMEM_decrease_reservation:
- case XENMEM_populate_physmap:
- set_xen_guest_handle(xmr->extent_start,
- xen_guest_handle(extent_start_va[0]));
- break;
- }
-
- return rc;
-}
-EXPORT_SYMBOL_GPL(xencomm_hypercall_memory_op);
-
-int
-xencomm_hypercall_suspend(unsigned long srec)
-{
- struct sched_shutdown arg;
-
- arg.reason = SHUTDOWN_suspend;
-
- return xencomm_arch_hypercall_sched_op(
- SCHEDOP_shutdown, xencomm_map_no_alloc(&arg, sizeof(arg)));
-}
-
-long
-xencomm_hypercall_vcpu_op(int cmd, int cpu, void *arg)
-{
- unsigned int argsize;
- switch (cmd) {
- case VCPUOP_register_runstate_memory_area: {
- struct vcpu_register_runstate_memory_area *area =
- (struct vcpu_register_runstate_memory_area *)arg;
- argsize = sizeof(*arg);
- set_xen_guest_handle(area->addr.h,
- (void *)xencomm_map_no_alloc(area->addr.v,
- sizeof(area->addr.v)));
- break;
- }
-
- default:
- printk(KERN_DEBUG "%s: unknown vcpu op %d\n", __func__, cmd);
- return -ENOSYS;
- }
-
- return xencomm_arch_hypercall_vcpu_op(cmd, cpu,
- xencomm_map_no_alloc(arg, argsize));
-}
-
-long
-xencomm_hypercall_opt_feature(void *arg)
-{
- return xencomm_arch_hypercall_opt_feature(
- xencomm_map_no_alloc(arg,
- sizeof(struct xen_ia64_opt_feature)));
-}
+++ /dev/null
-/******************************************************************************
- * arch/ia64/xen/xen_pv_ops.c
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <linux/console.h>
-#include <linux/irq.h>
-#include <linux/kernel.h>
-#include <linux/pm.h>
-#include <linux/unistd.h>
-
-#include <asm/xen/hypervisor.h>
-#include <asm/xen/xencomm.h>
-#include <asm/xen/privop.h>
-
-#include "irq_xen.h"
-#include "time.h"
-
-/***************************************************************************
- * general info
- */
-static struct pv_info xen_info __initdata = {
- .kernel_rpl = 2, /* or 1: determin at runtime */
- .paravirt_enabled = 1,
- .name = "Xen/ia64",
-};
-
-#define IA64_RSC_PL_SHIFT 2
-#define IA64_RSC_PL_BIT_SIZE 2
-#define IA64_RSC_PL_MASK \
- (((1UL << IA64_RSC_PL_BIT_SIZE) - 1) << IA64_RSC_PL_SHIFT)
-
-static void __init
-xen_info_init(void)
-{
- /* Xenified Linux/ia64 may run on pl = 1 or 2.
- * determin at run time. */
- unsigned long rsc = ia64_getreg(_IA64_REG_AR_RSC);
- unsigned int rpl = (rsc & IA64_RSC_PL_MASK) >> IA64_RSC_PL_SHIFT;
- xen_info.kernel_rpl = rpl;
-}
-
-/***************************************************************************
- * pv_init_ops
- * initialization hooks.
- */
-
-static void
-xen_panic_hypercall(struct unw_frame_info *info, void *arg)
-{
- current->thread.ksp = (__u64)info->sw - 16;
- HYPERVISOR_shutdown(SHUTDOWN_crash);
- /* we're never actually going to get here... */
-}
-
-static int
-xen_panic_event(struct notifier_block *this, unsigned long event, void *ptr)
-{
- unw_init_running(xen_panic_hypercall, NULL);
- /* we're never actually going to get here... */
- return NOTIFY_DONE;
-}
-
-static struct notifier_block xen_panic_block = {
- xen_panic_event, NULL, 0 /* try to go last */
-};
-
-static void xen_pm_power_off(void)
-{
- local_irq_disable();
- HYPERVISOR_shutdown(SHUTDOWN_poweroff);
-}
-
-static void __init
-xen_banner(void)
-{
- printk(KERN_INFO
- "Running on Xen! pl = %d start_info_pfn=0x%lx nr_pages=%ld "
- "flags=0x%x\n",
- xen_info.kernel_rpl,
- HYPERVISOR_shared_info->arch.start_info_pfn,
- xen_start_info->nr_pages, xen_start_info->flags);
-}
-
-static int __init
-xen_reserve_memory(struct rsvd_region *region)
-{
- region->start = (unsigned long)__va(
- (HYPERVISOR_shared_info->arch.start_info_pfn << PAGE_SHIFT));
- region->end = region->start + PAGE_SIZE;
- return 1;
-}
-
-static void __init
-xen_arch_setup_early(void)
-{
- struct shared_info *s;
- BUG_ON(!xen_pv_domain());
-
- s = HYPERVISOR_shared_info;
- xen_start_info = __va(s->arch.start_info_pfn << PAGE_SHIFT);
-
- /* Must be done before any hypercall. */
- xencomm_initialize();
-
- xen_setup_features();
- /* Register a call for panic conditions. */
- atomic_notifier_chain_register(&panic_notifier_list,
- &xen_panic_block);
- pm_power_off = xen_pm_power_off;
-
- xen_ia64_enable_opt_feature();
-}
-
-static void __init
-xen_arch_setup_console(char **cmdline_p)
-{
- add_preferred_console("xenboot", 0, NULL);
- add_preferred_console("tty", 0, NULL);
- /* use hvc_xen */
- add_preferred_console("hvc", 0, NULL);
-
-#if !defined(CONFIG_VT) || !defined(CONFIG_DUMMY_CONSOLE)
- conswitchp = NULL;
-#endif
-}
-
-static int __init
-xen_arch_setup_nomca(void)
-{
- return 1;
-}
-
-static void __init
-xen_post_smp_prepare_boot_cpu(void)
-{
- xen_setup_vcpu_info_placement();
-}
-
-#ifdef ASM_SUPPORTED
-static unsigned long __init_or_module
-xen_patch_bundle(void *sbundle, void *ebundle, unsigned long type);
-#endif
-static void __init
-xen_patch_branch(unsigned long tag, unsigned long type);
-
-static const struct pv_init_ops xen_init_ops __initconst = {
- .banner = xen_banner,
-
- .reserve_memory = xen_reserve_memory,
-
- .arch_setup_early = xen_arch_setup_early,
- .arch_setup_console = xen_arch_setup_console,
- .arch_setup_nomca = xen_arch_setup_nomca,
-
- .post_smp_prepare_boot_cpu = xen_post_smp_prepare_boot_cpu,
-#ifdef ASM_SUPPORTED
- .patch_bundle = xen_patch_bundle,
-#endif
- .patch_branch = xen_patch_branch,
-};
-
-/***************************************************************************
- * pv_fsys_data
- * addresses for fsys
- */
-
-extern unsigned long xen_fsyscall_table[NR_syscalls];
-extern char xen_fsys_bubble_down[];
-struct pv_fsys_data xen_fsys_data __initdata = {
- .fsyscall_table = (unsigned long *)xen_fsyscall_table,
- .fsys_bubble_down = (void *)xen_fsys_bubble_down,
-};
-
-/***************************************************************************
- * pv_patchdata
- * patchdata addresses
- */
-
-#define DECLARE(name) \
- extern unsigned long __xen_start_gate_##name##_patchlist[]; \
- extern unsigned long __xen_end_gate_##name##_patchlist[]
-
-DECLARE(fsyscall);
-DECLARE(brl_fsys_bubble_down);
-DECLARE(vtop);
-DECLARE(mckinley_e9);
-
-extern unsigned long __xen_start_gate_section[];
-
-#define ASSIGN(name) \
- .start_##name##_patchlist = \
- (unsigned long)__xen_start_gate_##name##_patchlist, \
- .end_##name##_patchlist = \
- (unsigned long)__xen_end_gate_##name##_patchlist
-
-static struct pv_patchdata xen_patchdata __initdata = {
- ASSIGN(fsyscall),
- ASSIGN(brl_fsys_bubble_down),
- ASSIGN(vtop),
- ASSIGN(mckinley_e9),
-
- .gate_section = (void*)__xen_start_gate_section,
-};
-
-/***************************************************************************
- * pv_cpu_ops
- * intrinsics hooks.
- */
-
-#ifndef ASM_SUPPORTED
-static void
-xen_set_itm_with_offset(unsigned long val)
-{
- /* ia64_cpu_local_tick() calls this with interrupt enabled. */
- /* WARN_ON(!irqs_disabled()); */
- xen_set_itm(val - XEN_MAPPEDREGS->itc_offset);
-}
-
-static unsigned long
-xen_get_itm_with_offset(void)
-{
- /* unused at this moment */
- printk(KERN_DEBUG "%s is called.\n", __func__);
-
- WARN_ON(!irqs_disabled());
- return ia64_native_getreg(_IA64_REG_CR_ITM) +
- XEN_MAPPEDREGS->itc_offset;
-}
-
-/* ia64_set_itc() is only called by
- * cpu_init() with ia64_set_itc(0) and ia64_sync_itc().
- * So XEN_MAPPEDRESG->itc_offset cal be considered as almost constant.
- */
-static void
-xen_set_itc(unsigned long val)
-{
- unsigned long mitc;
-
- WARN_ON(!irqs_disabled());
- mitc = ia64_native_getreg(_IA64_REG_AR_ITC);
- XEN_MAPPEDREGS->itc_offset = val - mitc;
- XEN_MAPPEDREGS->itc_last = val;
-}
-
-static unsigned long
-xen_get_itc(void)
-{
- unsigned long res;
- unsigned long itc_offset;
- unsigned long itc_last;
- unsigned long ret_itc_last;
-
- itc_offset = XEN_MAPPEDREGS->itc_offset;
- do {
- itc_last = XEN_MAPPEDREGS->itc_last;
- res = ia64_native_getreg(_IA64_REG_AR_ITC);
- res += itc_offset;
- if (itc_last >= res)
- res = itc_last + 1;
- ret_itc_last = cmpxchg(&XEN_MAPPEDREGS->itc_last,
- itc_last, res);
- } while (unlikely(ret_itc_last != itc_last));
- return res;
-
-#if 0
- /* ia64_itc_udelay() calls ia64_get_itc() with interrupt enabled.
- Should it be paravirtualized instead? */
- WARN_ON(!irqs_disabled());
- itc_offset = XEN_MAPPEDREGS->itc_offset;
- itc_last = XEN_MAPPEDREGS->itc_last;
- res = ia64_native_getreg(_IA64_REG_AR_ITC);
- res += itc_offset;
- if (itc_last >= res)
- res = itc_last + 1;
- XEN_MAPPEDREGS->itc_last = res;
- return res;
-#endif
-}
-
-static void xen_setreg(int regnum, unsigned long val)
-{
- switch (regnum) {
- case _IA64_REG_AR_KR0 ... _IA64_REG_AR_KR7:
- xen_set_kr(regnum - _IA64_REG_AR_KR0, val);
- break;
- case _IA64_REG_AR_ITC:
- xen_set_itc(val);
- break;
- case _IA64_REG_CR_TPR:
- xen_set_tpr(val);
- break;
- case _IA64_REG_CR_ITM:
- xen_set_itm_with_offset(val);
- break;
- case _IA64_REG_CR_EOI:
- xen_eoi(val);
- break;
- default:
- ia64_native_setreg_func(regnum, val);
- break;
- }
-}
-
-static unsigned long xen_getreg(int regnum)
-{
- unsigned long res;
-
- switch (regnum) {
- case _IA64_REG_PSR:
- res = xen_get_psr();
- break;
- case _IA64_REG_AR_ITC:
- res = xen_get_itc();
- break;
- case _IA64_REG_CR_ITM:
- res = xen_get_itm_with_offset();
- break;
- case _IA64_REG_CR_IVR:
- res = xen_get_ivr();
- break;
- case _IA64_REG_CR_TPR:
- res = xen_get_tpr();
- break;
- default:
- res = ia64_native_getreg_func(regnum);
- break;
- }
- return res;
-}
-
-/* turning on interrupts is a bit more complicated.. write to the
- * memory-mapped virtual psr.i bit first (to avoid race condition),
- * then if any interrupts were pending, we have to execute a hyperprivop
- * to ensure the pending interrupt gets delivered; else we're done! */
-static void
-xen_ssm_i(void)
-{
- int old = xen_get_virtual_psr_i();
- xen_set_virtual_psr_i(1);
- barrier();
- if (!old && xen_get_virtual_pend())
- xen_hyper_ssm_i();
-}
-
-/* turning off interrupts can be paravirtualized simply by writing
- * to a memory-mapped virtual psr.i bit (implemented as a 16-bit bool) */
-static void
-xen_rsm_i(void)
-{
- xen_set_virtual_psr_i(0);
- barrier();
-}
-
-static unsigned long
-xen_get_psr_i(void)
-{
- return xen_get_virtual_psr_i() ? IA64_PSR_I : 0;
-}
-
-static void
-xen_intrin_local_irq_restore(unsigned long mask)
-{
- if (mask & IA64_PSR_I)
- xen_ssm_i();
- else
- xen_rsm_i();
-}
-#else
-#define __DEFINE_FUNC(name, code) \
- extern const char xen_ ## name ## _direct_start[]; \
- extern const char xen_ ## name ## _direct_end[]; \
- asm (".align 32\n" \
- ".proc xen_" #name "\n" \
- "xen_" #name ":\n" \
- "xen_" #name "_direct_start:\n" \
- code \
- "xen_" #name "_direct_end:\n" \
- "br.cond.sptk.many b6\n" \
- ".endp xen_" #name "\n")
-
-#define DEFINE_VOID_FUNC0(name, code) \
- extern void \
- xen_ ## name (void); \
- __DEFINE_FUNC(name, code)
-
-#define DEFINE_VOID_FUNC1(name, code) \
- extern void \
- xen_ ## name (unsigned long arg); \
- __DEFINE_FUNC(name, code)
-
-#define DEFINE_VOID_FUNC1_VOID(name, code) \
- extern void \
- xen_ ## name (void *arg); \
- __DEFINE_FUNC(name, code)
-
-#define DEFINE_VOID_FUNC2(name, code) \
- extern void \
- xen_ ## name (unsigned long arg0, \
- unsigned long arg1); \
- __DEFINE_FUNC(name, code)
-
-#define DEFINE_FUNC0(name, code) \
- extern unsigned long \
- xen_ ## name (void); \
- __DEFINE_FUNC(name, code)
-
-#define DEFINE_FUNC1(name, type, code) \
- extern unsigned long \
- xen_ ## name (type arg); \
- __DEFINE_FUNC(name, code)
-
-#define XEN_PSR_I_ADDR_ADDR (XSI_BASE + XSI_PSR_I_ADDR_OFS)
-
-/*
- * static void xen_set_itm_with_offset(unsigned long val)
- * xen_set_itm(val - XEN_MAPPEDREGS->itc_offset);
- */
-/* 2 bundles */
-DEFINE_VOID_FUNC1(set_itm_with_offset,
- "mov r2 = " __stringify(XSI_BASE) " + "
- __stringify(XSI_ITC_OFFSET_OFS) "\n"
- ";;\n"
- "ld8 r3 = [r2]\n"
- ";;\n"
- "sub r8 = r8, r3\n"
- "break " __stringify(HYPERPRIVOP_SET_ITM) "\n");
-
-/*
- * static unsigned long xen_get_itm_with_offset(void)
- * return ia64_native_getreg(_IA64_REG_CR_ITM) + XEN_MAPPEDREGS->itc_offset;
- */
-/* 2 bundles */
-DEFINE_FUNC0(get_itm_with_offset,
- "mov r2 = " __stringify(XSI_BASE) " + "
- __stringify(XSI_ITC_OFFSET_OFS) "\n"
- ";;\n"
- "ld8 r3 = [r2]\n"
- "mov r8 = cr.itm\n"
- ";;\n"
- "add r8 = r8, r2\n");
-
-/*
- * static void xen_set_itc(unsigned long val)
- * unsigned long mitc;
- *
- * WARN_ON(!irqs_disabled());
- * mitc = ia64_native_getreg(_IA64_REG_AR_ITC);
- * XEN_MAPPEDREGS->itc_offset = val - mitc;
- * XEN_MAPPEDREGS->itc_last = val;
- */
-/* 2 bundles */
-DEFINE_VOID_FUNC1(set_itc,
- "mov r2 = " __stringify(XSI_BASE) " + "
- __stringify(XSI_ITC_LAST_OFS) "\n"
- "mov r3 = ar.itc\n"
- ";;\n"
- "sub r3 = r8, r3\n"
- "st8 [r2] = r8, "
- __stringify(XSI_ITC_LAST_OFS) " - "
- __stringify(XSI_ITC_OFFSET_OFS) "\n"
- ";;\n"
- "st8 [r2] = r3\n");
-
-/*
- * static unsigned long xen_get_itc(void)
- * unsigned long res;
- * unsigned long itc_offset;
- * unsigned long itc_last;
- * unsigned long ret_itc_last;
- *
- * itc_offset = XEN_MAPPEDREGS->itc_offset;
- * do {
- * itc_last = XEN_MAPPEDREGS->itc_last;
- * res = ia64_native_getreg(_IA64_REG_AR_ITC);
- * res += itc_offset;
- * if (itc_last >= res)
- * res = itc_last + 1;
- * ret_itc_last = cmpxchg(&XEN_MAPPEDREGS->itc_last,
- * itc_last, res);
- * } while (unlikely(ret_itc_last != itc_last));
- * return res;
- */
-/* 5 bundles */
-DEFINE_FUNC0(get_itc,
- "mov r2 = " __stringify(XSI_BASE) " + "
- __stringify(XSI_ITC_OFFSET_OFS) "\n"
- ";;\n"
- "ld8 r9 = [r2], " __stringify(XSI_ITC_LAST_OFS) " - "
- __stringify(XSI_ITC_OFFSET_OFS) "\n"
- /* r9 = itc_offset */
- /* r2 = XSI_ITC_OFFSET */
- "888:\n"
- "mov r8 = ar.itc\n" /* res = ar.itc */
- ";;\n"
- "ld8 r3 = [r2]\n" /* r3 = itc_last */
- "add r8 = r8, r9\n" /* res = ar.itc + itc_offset */
- ";;\n"
- "cmp.gtu p6, p0 = r3, r8\n"
- ";;\n"
- "(p6) add r8 = 1, r3\n" /* if (itc_last > res) itc_last + 1 */
- ";;\n"
- "mov ar.ccv = r8\n"
- ";;\n"
- "cmpxchg8.acq r10 = [r2], r8, ar.ccv\n"
- ";;\n"
- "cmp.ne p6, p0 = r10, r3\n"
- "(p6) hint @pause\n"
- "(p6) br.cond.spnt 888b\n");
-
-DEFINE_VOID_FUNC1_VOID(fc,
- "break " __stringify(HYPERPRIVOP_FC) "\n");
-
-/*
- * psr_i_addr_addr = XEN_PSR_I_ADDR_ADDR
- * masked_addr = *psr_i_addr_addr
- * pending_intr_addr = masked_addr - 1
- * if (val & IA64_PSR_I) {
- * masked = *masked_addr
- * *masked_addr = 0:xen_set_virtual_psr_i(1)
- * compiler barrier
- * if (masked) {
- * uint8_t pending = *pending_intr_addr;
- * if (pending)
- * XEN_HYPER_SSM_I
- * }
- * } else {
- * *masked_addr = 1:xen_set_virtual_psr_i(0)
- * }
- */
-/* 6 bundles */
-DEFINE_VOID_FUNC1(intrin_local_irq_restore,
- /* r8 = input value: 0 or IA64_PSR_I
- * p6 = (flags & IA64_PSR_I)
- * = if clause
- * p7 = !(flags & IA64_PSR_I)
- * = else clause
- */
- "cmp.ne p6, p7 = r8, r0\n"
- "mov r9 = " __stringify(XEN_PSR_I_ADDR_ADDR) "\n"
- ";;\n"
- /* r9 = XEN_PSR_I_ADDR */
- "ld8 r9 = [r9]\n"
- ";;\n"
-
- /* r10 = masked previous value */
- "(p6) ld1.acq r10 = [r9]\n"
- ";;\n"
-
- /* p8 = !masked interrupt masked previously? */
- "(p6) cmp.ne.unc p8, p0 = r10, r0\n"
-
- /* p7 = else clause */
- "(p7) mov r11 = 1\n"
- ";;\n"
- /* masked = 1 */
- "(p7) st1.rel [r9] = r11\n"
-
- /* p6 = if clause */
- /* masked = 0
- * r9 = masked_addr - 1
- * = pending_intr_addr
- */
- "(p8) st1.rel [r9] = r0, -1\n"
- ";;\n"
- /* r8 = pending_intr */
- "(p8) ld1.acq r11 = [r9]\n"
- ";;\n"
- /* p9 = interrupt pending? */
- "(p8) cmp.ne.unc p9, p10 = r11, r0\n"
- ";;\n"
- "(p10) mf\n"
- /* issue hypercall to trigger interrupt */
- "(p9) break " __stringify(HYPERPRIVOP_SSM_I) "\n");
-
-DEFINE_VOID_FUNC2(ptcga,
- "break " __stringify(HYPERPRIVOP_PTC_GA) "\n");
-DEFINE_VOID_FUNC2(set_rr,
- "break " __stringify(HYPERPRIVOP_SET_RR) "\n");
-
-/*
- * tmp = XEN_MAPPEDREGS->interrupt_mask_addr = XEN_PSR_I_ADDR_ADDR;
- * tmp = *tmp
- * tmp = *tmp;
- * psr_i = tmp? 0: IA64_PSR_I;
- */
-/* 4 bundles */
-DEFINE_FUNC0(get_psr_i,
- "mov r9 = " __stringify(XEN_PSR_I_ADDR_ADDR) "\n"
- ";;\n"
- "ld8 r9 = [r9]\n" /* r9 = XEN_PSR_I_ADDR */
- "mov r8 = 0\n" /* psr_i = 0 */
- ";;\n"
- "ld1.acq r9 = [r9]\n" /* r9 = XEN_PSR_I */
- ";;\n"
- "cmp.eq.unc p6, p0 = r9, r0\n" /* p6 = (XEN_PSR_I != 0) */
- ";;\n"
- "(p6) mov r8 = " __stringify(1 << IA64_PSR_I_BIT) "\n");
-
-DEFINE_FUNC1(thash, unsigned long,
- "break " __stringify(HYPERPRIVOP_THASH) "\n");
-DEFINE_FUNC1(get_cpuid, int,
- "break " __stringify(HYPERPRIVOP_GET_CPUID) "\n");
-DEFINE_FUNC1(get_pmd, int,
- "break " __stringify(HYPERPRIVOP_GET_PMD) "\n");
-DEFINE_FUNC1(get_rr, unsigned long,
- "break " __stringify(HYPERPRIVOP_GET_RR) "\n");
-
-/*
- * void xen_privop_ssm_i(void)
- *
- * int masked = !xen_get_virtual_psr_i();
- * // masked = *(*XEN_MAPPEDREGS->interrupt_mask_addr)
- * xen_set_virtual_psr_i(1)
- * // *(*XEN_MAPPEDREGS->interrupt_mask_addr) = 0
- * // compiler barrier
- * if (masked) {
- * uint8_t* pend_int_addr =
- * (uint8_t*)(*XEN_MAPPEDREGS->interrupt_mask_addr) - 1;
- * uint8_t pending = *pend_int_addr;
- * if (pending)
- * XEN_HYPER_SSM_I
- * }
- */
-/* 4 bundles */
-DEFINE_VOID_FUNC0(ssm_i,
- "mov r8 = " __stringify(XEN_PSR_I_ADDR_ADDR) "\n"
- ";;\n"
- "ld8 r8 = [r8]\n" /* r8 = XEN_PSR_I_ADDR */
- ";;\n"
- "ld1.acq r9 = [r8]\n" /* r9 = XEN_PSR_I */
- ";;\n"
- "st1.rel [r8] = r0, -1\n" /* psr_i = 0. enable interrupt
- * r8 = XEN_PSR_I_ADDR - 1
- * = pend_int_addr
- */
- "cmp.eq.unc p0, p6 = r9, r0\n"/* p6 = !XEN_PSR_I
- * previously interrupt
- * masked?
- */
- ";;\n"
- "(p6) ld1.acq r8 = [r8]\n" /* r8 = xen_pend_int */
- ";;\n"
- "(p6) cmp.eq.unc p6, p7 = r8, r0\n" /*interrupt pending?*/
- ";;\n"
- /* issue hypercall to get interrupt */
- "(p7) break " __stringify(HYPERPRIVOP_SSM_I) "\n"
- ";;\n");
-
-/*
- * psr_i_addr_addr = XEN_MAPPEDREGS->interrupt_mask_addr
- * = XEN_PSR_I_ADDR_ADDR;
- * psr_i_addr = *psr_i_addr_addr;
- * *psr_i_addr = 1;
- */
-/* 2 bundles */
-DEFINE_VOID_FUNC0(rsm_i,
- "mov r8 = " __stringify(XEN_PSR_I_ADDR_ADDR) "\n"
- /* r8 = XEN_PSR_I_ADDR */
- "mov r9 = 1\n"
- ";;\n"
- "ld8 r8 = [r8]\n" /* r8 = XEN_PSR_I */
- ";;\n"
- "st1.rel [r8] = r9\n"); /* XEN_PSR_I = 1 */
-
-extern void
-xen_set_rr0_to_rr4(unsigned long val0, unsigned long val1,
- unsigned long val2, unsigned long val3,
- unsigned long val4);
-__DEFINE_FUNC(set_rr0_to_rr4,
- "break " __stringify(HYPERPRIVOP_SET_RR0_TO_RR4) "\n");
-
-
-extern unsigned long xen_getreg(int regnum);
-#define __DEFINE_GET_REG(id, privop) \
- "mov r2 = " __stringify(_IA64_REG_ ## id) "\n" \
- ";;\n" \
- "cmp.eq p6, p0 = r2, r8\n" \
- ";;\n" \
- "(p6) break " __stringify(HYPERPRIVOP_GET_ ## privop) "\n" \
- "(p6) br.cond.sptk.many b6\n" \
- ";;\n"
-
-__DEFINE_FUNC(getreg,
- __DEFINE_GET_REG(PSR, PSR)
-
- /* get_itc */
- "mov r2 = " __stringify(_IA64_REG_AR_ITC) "\n"
- ";;\n"
- "cmp.eq p6, p0 = r2, r8\n"
- ";;\n"
- "(p6) br.cond.spnt xen_get_itc\n"
- ";;\n"
-
- /* get itm */
- "mov r2 = " __stringify(_IA64_REG_CR_ITM) "\n"
- ";;\n"
- "cmp.eq p6, p0 = r2, r8\n"
- ";;\n"
- "(p6) br.cond.spnt xen_get_itm_with_offset\n"
- ";;\n"
-
- __DEFINE_GET_REG(CR_IVR, IVR)
- __DEFINE_GET_REG(CR_TPR, TPR)
-
- /* fall back */
- "movl r2 = ia64_native_getreg_func\n"
- ";;\n"
- "mov b7 = r2\n"
- ";;\n"
- "br.cond.sptk.many b7\n");
-
-extern void xen_setreg(int regnum, unsigned long val);
-#define __DEFINE_SET_REG(id, privop) \
- "mov r2 = " __stringify(_IA64_REG_ ## id) "\n" \
- ";;\n" \
- "cmp.eq p6, p0 = r2, r9\n" \
- ";;\n" \
- "(p6) break " __stringify(HYPERPRIVOP_ ## privop) "\n" \
- "(p6) br.cond.sptk.many b6\n" \
- ";;\n"
-
-__DEFINE_FUNC(setreg,
- /* kr0 .. kr 7*/
- /*
- * if (_IA64_REG_AR_KR0 <= regnum &&
- * regnum <= _IA64_REG_AR_KR7) {
- * register __index asm ("r8") = regnum - _IA64_REG_AR_KR0
- * register __val asm ("r9") = val
- * "break HYPERPRIVOP_SET_KR"
- * }
- */
- "mov r17 = r9\n"
- "mov r2 = " __stringify(_IA64_REG_AR_KR0) "\n"
- ";;\n"
- "cmp.ge p6, p0 = r9, r2\n"
- "sub r17 = r17, r2\n"
- ";;\n"
- "(p6) cmp.ge.unc p7, p0 = "
- __stringify(_IA64_REG_AR_KR7) " - " __stringify(_IA64_REG_AR_KR0)
- ", r17\n"
- ";;\n"
- "(p7) mov r9 = r8\n"
- ";;\n"
- "(p7) mov r8 = r17\n"
- "(p7) break " __stringify(HYPERPRIVOP_SET_KR) "\n"
-
- /* set itm */
- "mov r2 = " __stringify(_IA64_REG_CR_ITM) "\n"
- ";;\n"
- "cmp.eq p6, p0 = r2, r8\n"
- ";;\n"
- "(p6) br.cond.spnt xen_set_itm_with_offset\n"
-
- /* set itc */
- "mov r2 = " __stringify(_IA64_REG_AR_ITC) "\n"
- ";;\n"
- "cmp.eq p6, p0 = r2, r8\n"
- ";;\n"
- "(p6) br.cond.spnt xen_set_itc\n"
-
- __DEFINE_SET_REG(CR_TPR, SET_TPR)
- __DEFINE_SET_REG(CR_EOI, EOI)
-
- /* fall back */
- "movl r2 = ia64_native_setreg_func\n"
- ";;\n"
- "mov b7 = r2\n"
- ";;\n"
- "br.cond.sptk.many b7\n");
-#endif
-
-static const struct pv_cpu_ops xen_cpu_ops __initconst = {
- .fc = xen_fc,
- .thash = xen_thash,
- .get_cpuid = xen_get_cpuid,
- .get_pmd = xen_get_pmd,
- .getreg = xen_getreg,
- .setreg = xen_setreg,
- .ptcga = xen_ptcga,
- .get_rr = xen_get_rr,
- .set_rr = xen_set_rr,
- .set_rr0_to_rr4 = xen_set_rr0_to_rr4,
- .ssm_i = xen_ssm_i,
- .rsm_i = xen_rsm_i,
- .get_psr_i = xen_get_psr_i,
- .intrin_local_irq_restore
- = xen_intrin_local_irq_restore,
-};
-
-/******************************************************************************
- * replacement of hand written assembly codes.
- */
-
-extern char xen_switch_to;
-extern char xen_leave_syscall;
-extern char xen_work_processed_syscall;
-extern char xen_leave_kernel;
-
-const struct pv_cpu_asm_switch xen_cpu_asm_switch = {
- .switch_to = (unsigned long)&xen_switch_to,
- .leave_syscall = (unsigned long)&xen_leave_syscall,
- .work_processed_syscall = (unsigned long)&xen_work_processed_syscall,
- .leave_kernel = (unsigned long)&xen_leave_kernel,
-};
-
-/***************************************************************************
- * pv_iosapic_ops
- * iosapic read/write hooks.
- */
-static void
-xen_pcat_compat_init(void)
-{
- /* nothing */
-}
-
-static struct irq_chip*
-xen_iosapic_get_irq_chip(unsigned long trigger)
-{
- return NULL;
-}
-
-static unsigned int
-xen_iosapic_read(char __iomem *iosapic, unsigned int reg)
-{
- struct physdev_apic apic_op;
- int ret;
-
- apic_op.apic_physbase = (unsigned long)iosapic -
- __IA64_UNCACHED_OFFSET;
- apic_op.reg = reg;
- ret = HYPERVISOR_physdev_op(PHYSDEVOP_apic_read, &apic_op);
- if (ret)
- return ret;
- return apic_op.value;
-}
-
-static void
-xen_iosapic_write(char __iomem *iosapic, unsigned int reg, u32 val)
-{
- struct physdev_apic apic_op;
-
- apic_op.apic_physbase = (unsigned long)iosapic -
- __IA64_UNCACHED_OFFSET;
- apic_op.reg = reg;
- apic_op.value = val;
- HYPERVISOR_physdev_op(PHYSDEVOP_apic_write, &apic_op);
-}
-
-static struct pv_iosapic_ops xen_iosapic_ops __initdata = {
- .pcat_compat_init = xen_pcat_compat_init,
- .__get_irq_chip = xen_iosapic_get_irq_chip,
-
- .__read = xen_iosapic_read,
- .__write = xen_iosapic_write,
-};
-
-/***************************************************************************
- * pv_ops initialization
- */
-
-void __init
-xen_setup_pv_ops(void)
-{
- xen_info_init();
- pv_info = xen_info;
- pv_init_ops = xen_init_ops;
- pv_fsys_data = xen_fsys_data;
- pv_patchdata = xen_patchdata;
- pv_cpu_ops = xen_cpu_ops;
- pv_iosapic_ops = xen_iosapic_ops;
- pv_irq_ops = xen_irq_ops;
- pv_time_ops = xen_time_ops;
-
- paravirt_cpu_asm_init(&xen_cpu_asm_switch);
-}
-
-#ifdef ASM_SUPPORTED
-/***************************************************************************
- * binary pacthing
- * pv_init_ops.patch_bundle
- */
-
-#define DEFINE_FUNC_GETREG(name, privop) \
- DEFINE_FUNC0(get_ ## name, \
- "break "__stringify(HYPERPRIVOP_GET_ ## privop) "\n")
-
-DEFINE_FUNC_GETREG(psr, PSR);
-DEFINE_FUNC_GETREG(eflag, EFLAG);
-DEFINE_FUNC_GETREG(ivr, IVR);
-DEFINE_FUNC_GETREG(tpr, TPR);
-
-#define DEFINE_FUNC_SET_KR(n) \
- DEFINE_VOID_FUNC0(set_kr ## n, \
- ";;\n" \
- "mov r9 = r8\n" \
- "mov r8 = " #n "\n" \
- "break " __stringify(HYPERPRIVOP_SET_KR) "\n")
-
-DEFINE_FUNC_SET_KR(0);
-DEFINE_FUNC_SET_KR(1);
-DEFINE_FUNC_SET_KR(2);
-DEFINE_FUNC_SET_KR(3);
-DEFINE_FUNC_SET_KR(4);
-DEFINE_FUNC_SET_KR(5);
-DEFINE_FUNC_SET_KR(6);
-DEFINE_FUNC_SET_KR(7);
-
-#define __DEFINE_FUNC_SETREG(name, privop) \
- DEFINE_VOID_FUNC0(name, \
- "break "__stringify(HYPERPRIVOP_ ## privop) "\n")
-
-#define DEFINE_FUNC_SETREG(name, privop) \
- __DEFINE_FUNC_SETREG(set_ ## name, SET_ ## privop)
-
-DEFINE_FUNC_SETREG(eflag, EFLAG);
-DEFINE_FUNC_SETREG(tpr, TPR);
-__DEFINE_FUNC_SETREG(eoi, EOI);
-
-extern const char xen_check_events[];
-extern const char __xen_intrin_local_irq_restore_direct_start[];
-extern const char __xen_intrin_local_irq_restore_direct_end[];
-extern const unsigned long __xen_intrin_local_irq_restore_direct_reloc;
-
-asm (
- ".align 32\n"
- ".proc xen_check_events\n"
- "xen_check_events:\n"
- /* masked = 0
- * r9 = masked_addr - 1
- * = pending_intr_addr
- */
- "st1.rel [r9] = r0, -1\n"
- ";;\n"
- /* r8 = pending_intr */
- "ld1.acq r11 = [r9]\n"
- ";;\n"
- /* p9 = interrupt pending? */
- "cmp.ne p9, p10 = r11, r0\n"
- ";;\n"
- "(p10) mf\n"
- /* issue hypercall to trigger interrupt */
- "(p9) break " __stringify(HYPERPRIVOP_SSM_I) "\n"
- "br.cond.sptk.many b6\n"
- ".endp xen_check_events\n"
- "\n"
- ".align 32\n"
- ".proc __xen_intrin_local_irq_restore_direct\n"
- "__xen_intrin_local_irq_restore_direct:\n"
- "__xen_intrin_local_irq_restore_direct_start:\n"
- "1:\n"
- "{\n"
- "cmp.ne p6, p7 = r8, r0\n"
- "mov r17 = ip\n" /* get ip to calc return address */
- "mov r9 = "__stringify(XEN_PSR_I_ADDR_ADDR) "\n"
- ";;\n"
- "}\n"
- "{\n"
- /* r9 = XEN_PSR_I_ADDR */
- "ld8 r9 = [r9]\n"
- ";;\n"
- /* r10 = masked previous value */
- "(p6) ld1.acq r10 = [r9]\n"
- "adds r17 = 1f - 1b, r17\n" /* calculate return address */
- ";;\n"
- "}\n"
- "{\n"
- /* p8 = !masked interrupt masked previously? */
- "(p6) cmp.ne.unc p8, p0 = r10, r0\n"
- "\n"
- /* p7 = else clause */
- "(p7) mov r11 = 1\n"
- ";;\n"
- "(p8) mov b6 = r17\n" /* set return address */
- "}\n"
- "{\n"
- /* masked = 1 */
- "(p7) st1.rel [r9] = r11\n"
- "\n"
- "[99:]\n"
- "(p8) brl.cond.dptk.few xen_check_events\n"
- "}\n"
- /* pv calling stub is 5 bundles. fill nop to adjust return address */
- "{\n"
- "nop 0\n"
- "nop 0\n"
- "nop 0\n"
- "}\n"
- "1:\n"
- "__xen_intrin_local_irq_restore_direct_end:\n"
- ".endp __xen_intrin_local_irq_restore_direct\n"
- "\n"
- ".align 8\n"
- "__xen_intrin_local_irq_restore_direct_reloc:\n"
- "data8 99b\n"
-);
-
-static struct paravirt_patch_bundle_elem xen_patch_bundle_elems[]
-__initdata_or_module =
-{
-#define XEN_PATCH_BUNDLE_ELEM(name, type) \
- { \
- (void*)xen_ ## name ## _direct_start, \
- (void*)xen_ ## name ## _direct_end, \
- PARAVIRT_PATCH_TYPE_ ## type, \
- }
-
- XEN_PATCH_BUNDLE_ELEM(fc, FC),
- XEN_PATCH_BUNDLE_ELEM(thash, THASH),
- XEN_PATCH_BUNDLE_ELEM(get_cpuid, GET_CPUID),
- XEN_PATCH_BUNDLE_ELEM(get_pmd, GET_PMD),
- XEN_PATCH_BUNDLE_ELEM(ptcga, PTCGA),
- XEN_PATCH_BUNDLE_ELEM(get_rr, GET_RR),
- XEN_PATCH_BUNDLE_ELEM(set_rr, SET_RR),
- XEN_PATCH_BUNDLE_ELEM(set_rr0_to_rr4, SET_RR0_TO_RR4),
- XEN_PATCH_BUNDLE_ELEM(ssm_i, SSM_I),
- XEN_PATCH_BUNDLE_ELEM(rsm_i, RSM_I),
- XEN_PATCH_BUNDLE_ELEM(get_psr_i, GET_PSR_I),
- {
- (void*)__xen_intrin_local_irq_restore_direct_start,
- (void*)__xen_intrin_local_irq_restore_direct_end,
- PARAVIRT_PATCH_TYPE_INTRIN_LOCAL_IRQ_RESTORE,
- },
-
-#define XEN_PATCH_BUNDLE_ELEM_GETREG(name, reg) \
- { \
- xen_get_ ## name ## _direct_start, \
- xen_get_ ## name ## _direct_end, \
- PARAVIRT_PATCH_TYPE_GETREG + _IA64_REG_ ## reg, \
- }
-
- XEN_PATCH_BUNDLE_ELEM_GETREG(psr, PSR),
- XEN_PATCH_BUNDLE_ELEM_GETREG(eflag, AR_EFLAG),
-
- XEN_PATCH_BUNDLE_ELEM_GETREG(ivr, CR_IVR),
- XEN_PATCH_BUNDLE_ELEM_GETREG(tpr, CR_TPR),
-
- XEN_PATCH_BUNDLE_ELEM_GETREG(itc, AR_ITC),
- XEN_PATCH_BUNDLE_ELEM_GETREG(itm_with_offset, CR_ITM),
-
-
-#define __XEN_PATCH_BUNDLE_ELEM_SETREG(name, reg) \
- { \
- xen_ ## name ## _direct_start, \
- xen_ ## name ## _direct_end, \
- PARAVIRT_PATCH_TYPE_SETREG + _IA64_REG_ ## reg, \
- }
-
-#define XEN_PATCH_BUNDLE_ELEM_SETREG(name, reg) \
- __XEN_PATCH_BUNDLE_ELEM_SETREG(set_ ## name, reg)
-
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr0, AR_KR0),
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr1, AR_KR1),
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr2, AR_KR2),
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr3, AR_KR3),
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr4, AR_KR4),
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr5, AR_KR5),
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr6, AR_KR6),
- XEN_PATCH_BUNDLE_ELEM_SETREG(kr7, AR_KR7),
-
- XEN_PATCH_BUNDLE_ELEM_SETREG(eflag, AR_EFLAG),
- XEN_PATCH_BUNDLE_ELEM_SETREG(tpr, CR_TPR),
- __XEN_PATCH_BUNDLE_ELEM_SETREG(eoi, CR_EOI),
-
- XEN_PATCH_BUNDLE_ELEM_SETREG(itc, AR_ITC),
- XEN_PATCH_BUNDLE_ELEM_SETREG(itm_with_offset, CR_ITM),
-};
-
-static unsigned long __init_or_module
-xen_patch_bundle(void *sbundle, void *ebundle, unsigned long type)
-{
- const unsigned long nelems = sizeof(xen_patch_bundle_elems) /
- sizeof(xen_patch_bundle_elems[0]);
- unsigned long used;
- const struct paravirt_patch_bundle_elem *found;
-
- used = __paravirt_patch_apply_bundle(sbundle, ebundle, type,
- xen_patch_bundle_elems, nelems,
- &found);
-
- if (found == NULL)
- /* fallback */
- return ia64_native_patch_bundle(sbundle, ebundle, type);
- if (used == 0)
- return used;
-
- /* relocation */
- switch (type) {
- case PARAVIRT_PATCH_TYPE_INTRIN_LOCAL_IRQ_RESTORE: {
- unsigned long reloc =
- __xen_intrin_local_irq_restore_direct_reloc;
- unsigned long reloc_offset = reloc - (unsigned long)
- __xen_intrin_local_irq_restore_direct_start;
- unsigned long tag = (unsigned long)sbundle + reloc_offset;
- paravirt_patch_reloc_brl(tag, xen_check_events);
- break;
- }
- default:
- /* nothing */
- break;
- }
- return used;
-}
-#endif /* ASM_SUPPOTED */
-
-const struct paravirt_patch_branch_target xen_branch_target[]
-__initconst = {
-#define PARAVIRT_BR_TARGET(name, type) \
- { \
- &xen_ ## name, \
- PARAVIRT_PATCH_TYPE_BR_ ## type, \
- }
- PARAVIRT_BR_TARGET(switch_to, SWITCH_TO),
- PARAVIRT_BR_TARGET(leave_syscall, LEAVE_SYSCALL),
- PARAVIRT_BR_TARGET(work_processed_syscall, WORK_PROCESSED_SYSCALL),
- PARAVIRT_BR_TARGET(leave_kernel, LEAVE_KERNEL),
-};
-
-static void __init
-xen_patch_branch(unsigned long tag, unsigned long type)
-{
- __paravirt_patch_apply_branch(tag, type, xen_branch_target,
- ARRAY_SIZE(xen_branch_target));
-}
+++ /dev/null
-/*
- * Copyright (C) 2006 Hollis Blanchard <hollisb@us.ibm.com>, IBM Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- */
-
-#include <linux/mm.h>
-#include <linux/err.h>
-
-static unsigned long kernel_virtual_offset;
-static int is_xencomm_initialized;
-
-/* for xen early printk. It uses console io hypercall which uses xencomm.
- * However early printk may use it before xencomm initialization.
- */
-int
-xencomm_is_initialized(void)
-{
- return is_xencomm_initialized;
-}
-
-void
-xencomm_initialize(void)
-{
- kernel_virtual_offset = KERNEL_START - ia64_tpa(KERNEL_START);
- is_xencomm_initialized = 1;
-}
-
-/* Translate virtual address to physical address. */
-unsigned long
-xencomm_vtop(unsigned long vaddr)
-{
- struct page *page;
- struct vm_area_struct *vma;
-
- if (vaddr == 0)
- return 0UL;
-
- if (REGION_NUMBER(vaddr) == 5) {
- pgd_t *pgd;
- pud_t *pud;
- pmd_t *pmd;
- pte_t *ptep;
-
- /* On ia64, TASK_SIZE refers to current. It is not initialized
- during boot.
- Furthermore the kernel is relocatable and __pa() doesn't
- work on addresses. */
- if (vaddr >= KERNEL_START
- && vaddr < (KERNEL_START + KERNEL_TR_PAGE_SIZE))
- return vaddr - kernel_virtual_offset;
-
- /* In kernel area -- virtually mapped. */
- pgd = pgd_offset_k(vaddr);
- if (pgd_none(*pgd) || pgd_bad(*pgd))
- return ~0UL;
-
- pud = pud_offset(pgd, vaddr);
- if (pud_none(*pud) || pud_bad(*pud))
- return ~0UL;
-
- pmd = pmd_offset(pud, vaddr);
- if (pmd_none(*pmd) || pmd_bad(*pmd))
- return ~0UL;
-
- ptep = pte_offset_kernel(pmd, vaddr);
- if (!ptep)
- return ~0UL;
-
- return (pte_val(*ptep) & _PFN_MASK) | (vaddr & ~PAGE_MASK);
- }
-
- if (vaddr > TASK_SIZE) {
- /* percpu variables */
- if (REGION_NUMBER(vaddr) == 7 &&
- REGION_OFFSET(vaddr) >= (1ULL << IA64_MAX_PHYS_BITS))
- ia64_tpa(vaddr);
-
- /* kernel address */
- return __pa(vaddr);
- }
-
- /* XXX double-check (lack of) locking */
- vma = find_extend_vma(current->mm, vaddr);
- if (!vma)
- return ~0UL;
-
- /* We assume the page is modified. */
- page = follow_page(vma, vaddr, FOLL_WRITE | FOLL_TOUCH);
- if (IS_ERR_OR_NULL(page))
- return ~0UL;
-
- return (page_to_pfn(page) << PAGE_SHIFT) | (vaddr & ~PAGE_MASK);
-}
+++ /dev/null
-/*
- * arch/ia64/xen/ivt.S
- *
- * Copyright (C) 2005 Hewlett-Packard Co
- * Dan Magenheimer <dan.magenheimer@hp.com>
- *
- * Copyright (c) 2008 Isaku Yamahata <yamahata at valinux co jp>
- * VA Linux Systems Japan K.K.
- * pv_ops.
- */
-
-#include <asm/asmmacro.h>
-#include <asm/kregs.h>
-#include <asm/pgtable.h>
-
-#include "../kernel/minstate.h"
-
- .section .text,"ax"
-GLOBAL_ENTRY(xen_event_callback)
- mov r31=pr // prepare to save predicates
- ;;
- SAVE_MIN_WITH_COVER // uses r31; defines r2 and r3
- ;;
- movl r3=XSI_PSR_IC
- mov r14=1
- ;;
- st4 [r3]=r14
- ;;
- adds r3=8,r2 // set up second base pointer for SAVE_REST
- srlz.i // ensure everybody knows psr.ic is back on
- ;;
- SAVE_REST
- ;;
-1:
- alloc r14=ar.pfs,0,0,1,0 // must be first in an insn group
- add out0=16,sp // pass pointer to pt_regs as first arg
- ;;
- br.call.sptk.many b0=xen_evtchn_do_upcall
- ;;
- movl r20=XSI_PSR_I_ADDR
- ;;
- ld8 r20=[r20]
- ;;
- adds r20=-1,r20 // vcpu_info->evtchn_upcall_pending
- ;;
- ld1 r20=[r20]
- ;;
- cmp.ne p6,p0=r20,r0 // if there are pending events,
- (p6) br.spnt.few 1b // call evtchn_do_upcall again.
- br.sptk.many xen_leave_kernel // we know ia64_leave_kernel is
- // paravirtualized as xen_leave_kernel
-END(xen_event_callback)
+++ /dev/null
-/*
- * Support routines for Xen
- *
- * Copyright (C) 2005 Dan Magenheimer <dan.magenheimer@hp.com>
- */
-
-#include <asm/processor.h>
-#include <asm/asmmacro.h>
-#include <asm/pgtable.h>
-#include <asm/paravirt.h>
-#include <asm/xen/privop.h>
-#include <linux/elfnote.h>
-#include <linux/init.h>
-#include <xen/interface/elfnote.h>
-
- .section .data..read_mostly
- .align 8
- .global xen_domain_type
-xen_domain_type:
- data4 XEN_NATIVE_ASM
- .previous
-
- __INIT
-ENTRY(startup_xen)
- // Calculate load offset.
- // The constant, LOAD_OFFSET, can't be used because the boot
- // loader doesn't always load to the LMA specified by the vmlinux.lds.
- mov r9=ip // must be the first instruction to make sure
- // that r9 = the physical address of startup_xen.
- // Usually r9 = startup_xen - LOAD_OFFSET
- movl r8=startup_xen
- ;;
- sub r9=r9,r8 // Usually r9 = -LOAD_OFFSET.
-
- mov r10=PARAVIRT_HYPERVISOR_TYPE_XEN
- movl r11=_start
- ;;
- add r11=r11,r9
- movl r8=hypervisor_type
- ;;
- add r8=r8,r9
- mov b0=r11
- ;;
- st8 [r8]=r10
- br.cond.sptk.many b0
- ;;
-END(startup_xen)
-
- ELFNOTE(Xen, XEN_ELFNOTE_GUEST_OS, .asciz "linux")
- ELFNOTE(Xen, XEN_ELFNOTE_GUEST_VERSION, .asciz "2.6")
- ELFNOTE(Xen, XEN_ELFNOTE_XEN_VERSION, .asciz "xen-3.0")
- ELFNOTE(Xen, XEN_ELFNOTE_ENTRY, data8.ua startup_xen - LOAD_OFFSET)
-
-#define isBP p3 // are we the Bootstrap Processor?
-
-GLOBAL_ENTRY(xen_setup_hook)
- mov r8=XEN_PV_DOMAIN_ASM
-(isBP) movl r9=xen_domain_type;;
-(isBP) st4 [r9]=r8
- movl r10=xen_ivt;;
-
- mov cr.iva=r10
-
- /* Set xsi base. */
-#define FW_HYPERCALL_SET_SHARED_INFO_VA 0x600
-(isBP) mov r2=FW_HYPERCALL_SET_SHARED_INFO_VA
-(isBP) movl r28=XSI_BASE;;
-(isBP) break 0x1000;;
-
- /* setup pv_ops */
-(isBP) mov r4=rp
- ;;
-(isBP) br.call.sptk.many rp=xen_setup_pv_ops
- ;;
-(isBP) mov rp=r4
- ;;
-
- br.ret.sptk.many rp
- ;;
-END(xen_setup_hook)
#define nop() __asm__ __volatile__ ("nop" : : )
-/*
- * Memory barrier.
- *
- * mb() prevents loads and stores being reordered across this point.
- * rmb() prevents loads being reordered across this point.
- * wmb() prevents stores being reordered across this point.
- */
-#define mb() barrier()
-#define rmb() mb()
-#define wmb() mb()
-
-/**
- * read_barrier_depends - Flush all pending reads that subsequents reads
- * depend on.
- *
- * No data-dependent reads from memory-like regions are ever reordered
- * over this barrier. All reads preceding this primitive are guaranteed
- * to access memory (but not necessarily other CPUs' caches) before any
- * reads following this primitive that depend on the data return by
- * any of the preceding reads. This primitive is much lighter weight than
- * rmb() on most CPUs, and is never heavier weight than is
- * rmb().
- *
- * These ordering constraints are respected by both the local CPU
- * and the compiler.
- *
- * Ordering is not guaranteed by anything other than these primitives,
- * not even by data dependencies. See the documentation for
- * memory_barrier() for examples and URLs to more information.
- *
- * For example, the following code would force ordering (the initial
- * value of "a" is zero, "b" is one, and "p" is "&a"):
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * b = 2;
- * memory_barrier();
- * p = &b; q = p;
- * read_barrier_depends();
- * d = *q;
- * </programlisting>
- *
- *
- * because the read of "*q" depends on the read of "p" and these
- * two reads are separated by a read_barrier_depends(). However,
- * the following code, with the same initial values for "a" and "b":
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * a = 2;
- * memory_barrier();
- * b = 3; y = b;
- * read_barrier_depends();
- * x = a;
- * </programlisting>
- *
- * does not enforce ordering, since there is no data dependency between
- * the read of "a" and the read of "b". Therefore, on some CPUs, such
- * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
- * in cases like this where there are no data dependencies.
- **/
-
-#define read_barrier_depends() do { } while (0)
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#define set_mb(var, value) do { (void) xchg(&var, value); } while (0)
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while (0)
-#define set_mb(var, value) do { var = value; barrier(); } while (0)
-#endif
+#include <asm-generic/barrier.h>
#endif /* _ASM_M32R_BARRIER_H */
bool
depends on MMU && !MMU_MOTOROLA && !MMU_COLDFIRE
+config KEXEC
+ bool "kexec system call"
+ depends on M68KCLASSIC
+ help
+ kexec is a system call that implements the ability to shutdown your
+ current kernel, and to start another kernel. It is like a reboot
+ but it is independent of the system firmware. And like a reboot
+ you can start any kernel with it, not just Linux.
+
+ The name comes from the similarity to the exec system call.
+
+ It is an ongoing process to be certain the hardware in a machine
+ is properly shutdown, so do not be surprised if this code does not
+ initially work for you. As of this writing the exact hardware
+ interface is strongly in flux, so no good recommendation can be
+ made.
+
+config BOOTINFO_PROC
+ bool "Export bootinfo in procfs"
+ depends on KEXEC && M68KCLASSIC
+ help
+ Say Y to export the bootinfo used to boot the kernel in a
+ "bootinfo" file in procfs. This is useful with kexec.
+
menu "Platform setup"
source arch/m68k/Kconfig.cpu
atomic_sub(size, &chipavail);
pr_debug("amiga_chip_alloc_res: returning %pR\n", res);
- return (void *)ZTWO_VADDR(res->start);
+ return ZTWO_VADDR(res->start);
}
void amiga_chip_free(void *ptr)
#include <linux/keyboard.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-amiga.h>
+#include <asm/byteorder.h>
#include <asm/setup.h>
#include <asm/pgtable.h>
#include <asm/amigahw.h>
* Parse an Amiga-specific record in the bootinfo
*/
-int amiga_parse_bootinfo(const struct bi_record *record)
+int __init amiga_parse_bootinfo(const struct bi_record *record)
{
int unknown = 0;
- const unsigned long *data = record->data;
+ const void *data = record->data;
- switch (record->tag) {
+ switch (be16_to_cpu(record->tag)) {
case BI_AMIGA_MODEL:
- amiga_model = *data;
+ amiga_model = be32_to_cpup(data);
break;
case BI_AMIGA_ECLOCK:
- amiga_eclock = *data;
+ amiga_eclock = be32_to_cpup(data);
break;
case BI_AMIGA_CHIPSET:
- amiga_chipset = *data;
+ amiga_chipset = be32_to_cpup(data);
break;
case BI_AMIGA_CHIP_SIZE:
- amiga_chip_size = *(const int *)data;
+ amiga_chip_size = be32_to_cpup(data);
break;
case BI_AMIGA_VBLANK:
- amiga_vblank = *(const unsigned char *)data;
+ amiga_vblank = *(const __u8 *)data;
break;
case BI_AMIGA_PSFREQ:
- amiga_psfreq = *(const unsigned char *)data;
+ amiga_psfreq = *(const __u8 *)data;
break;
case BI_AMIGA_AUTOCON:
#ifdef CONFIG_ZORRO
if (zorro_num_autocon < ZORRO_NUM_AUTO) {
- const struct ConfigDev *cd = (struct ConfigDev *)data;
- struct zorro_dev *dev = &zorro_autocon[zorro_num_autocon++];
+ const struct ConfigDev *cd = data;
+ struct zorro_dev_init *dev = &zorro_autocon_init[zorro_num_autocon++];
dev->rom = cd->cd_Rom;
- dev->slotaddr = cd->cd_SlotAddr;
- dev->slotsize = cd->cd_SlotSize;
- dev->resource.start = (unsigned long)cd->cd_BoardAddr;
- dev->resource.end = dev->resource.start + cd->cd_BoardSize - 1;
+ dev->slotaddr = be16_to_cpu(cd->cd_SlotAddr);
+ dev->slotsize = be16_to_cpu(cd->cd_SlotSize);
+ dev->boardaddr = be32_to_cpu(cd->cd_BoardAddr);
+ dev->boardsize = be32_to_cpu(cd->cd_BoardSize);
} else
printk("amiga_parse_bootinfo: too many AutoConfig devices\n");
#endif /* CONFIG_ZORRO */
#undef AMIGAHW_ANNOUNCE
}
+
+static unsigned long amiga_random_get_entropy(void)
+{
+ /* VPOSR/VHPOSR provide at least 17 bits of data changing at 1.79 MHz */
+ return *(unsigned long *)&amiga_custom.vposr;
+}
+
+
/*
* Setup the Amiga configuration info
*/
mach_heartbeat = amiga_heartbeat;
#endif
+ mach_random_get_entropy = amiga_random_get_entropy;
+
/* Fill in the clock value (based on the 700 kHz E-Clock) */
amiga_colorclock = 5*amiga_eclock; /* 3.5 MHz */
static int __init amiga_savekmsg_setup(char *arg)
{
+ bool registered;
+
if (!MACH_IS_AMIGA || strcmp(arg, "mem"))
return 0;
/* Just steal the block, the chipram allocator isn't functional yet */
amiga_chip_size -= SAVEKMSG_MAXMEM;
- savekmsg = (void *)ZTWO_VADDR(CHIP_PHYSADDR + amiga_chip_size);
+ savekmsg = ZTWO_VADDR(CHIP_PHYSADDR + amiga_chip_size);
savekmsg->magic1 = SAVEKMSG_MAGIC1;
savekmsg->magic2 = SAVEKMSG_MAGIC2;
savekmsg->magicptr = ZTWO_PADDR(savekmsg);
savekmsg->size = 0;
+ registered = !!amiga_console_driver.write;
amiga_console_driver.write = amiga_mem_console_write;
- register_console(&amiga_console_driver);
+ if (!registered)
+ register_console(&amiga_console_driver);
return 0;
}
static int __init amiga_debug_setup(char *arg)
{
- if (MACH_IS_AMIGA && !strcmp(arg, "ser")) {
- /* no initialization required (?) */
- amiga_console_driver.write = amiga_serial_console_write;
+ bool registered;
+
+ if (!MACH_IS_AMIGA || strcmp(arg, "ser"))
+ return 0;
+
+ /* no initialization required (?) */
+ registered = !!amiga_console_driver.write;
+ amiga_console_driver.write = amiga_serial_console_write;
+ if (!registered)
register_console(&amiga_console_driver);
- }
return 0;
}
#include <asm/amigahw.h>
#include <asm/amigayle.h>
+#include <asm/byteorder.h>
#ifdef CONFIG_ZORRO
{
unsigned int i;
- for (i = 0; i < zorro_num_autocon; i++)
- if (zorro_autocon[i].rom.er_Manufacturer == ZORRO_MANUF(id) &&
- zorro_autocon[i].rom.er_Product == ZORRO_PROD(id))
+ for (i = 0; i < zorro_num_autocon; i++) {
+ const struct ExpansionRom *rom = &zorro_autocon_init[i].rom;
+ if (be16_to_cpu(rom->er_Manufacturer) == ZORRO_MANUF(id) &&
+ rom->er_Product == ZORRO_PROD(id))
return 1;
+ }
return 0;
}
+#include <linux/init.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <asm/setup.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-apollo.h>
+#include <asm/byteorder.h>
#include <asm/pgtable.h>
#include <asm/apollohw.h>
#include <asm/irq.h>
[APOLLO_DN4500-APOLLO_DN3000] = "DN4500 (Roadrunner)"
};
-int apollo_parse_bootinfo(const struct bi_record *record) {
-
+int __init apollo_parse_bootinfo(const struct bi_record *record)
+{
int unknown = 0;
- const unsigned long *data = record->data;
+ const void *data = record->data;
- switch(record->tag) {
- case BI_APOLLO_MODEL:
- apollo_model=*data;
- break;
+ switch (be16_to_cpu(record->tag)) {
+ case BI_APOLLO_MODEL:
+ apollo_model = be32_to_cpup(data);
+ break;
- default:
- unknown=1;
+ default:
+ unknown=1;
}
return unknown;
}
-void dn_setup_model(void) {
-
-
+static void __init dn_setup_model(void)
+{
printk("Apollo hardware found: ");
printk("[%s]\n", apollo_models[apollo_model - APOLLO_DN3000]);
m68k_setup_irq_controller(&atari_mfptimer_chip, handle_simple_irq,
IRQ_MFP_TIMER1, 8);
+ irq_set_status_flags(IRQ_MFP_TIMER1, IRQ_IS_POLLED);
+ irq_set_status_flags(IRQ_MFP_TIMER2, IRQ_IS_POLLED);
+
/* prepare timer D data for use as poll interrupt */
/* set Timer D data Register - needs to be > 0 */
st_mfp.tim_dt_d = 254; /* < 100 Hz */
#include <linux/module.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-atari.h>
+#include <asm/byteorder.h>
#include <asm/setup.h>
#include <asm/atarihw.h>
#include <asm/atariints.h>
int __init atari_parse_bootinfo(const struct bi_record *record)
{
int unknown = 0;
- const u_long *data = record->data;
+ const void *data = record->data;
- switch (record->tag) {
+ switch (be16_to_cpu(record->tag)) {
case BI_ATARI_MCH_COOKIE:
- atari_mch_cookie = *data;
+ atari_mch_cookie = be32_to_cpup(data);
break;
case BI_ATARI_MCH_TYPE:
- atari_mch_type = *data;
+ atari_mch_type = be32_to_cpup(data);
break;
default:
unknown = 1;
static int __init atari_debug_setup(char *arg)
{
+ bool registered;
+
if (!MACH_IS_ATARI)
return 0;
/* defaults to ser2 for a Falcon and ser1 otherwise */
arg = MACH_IS_FALCON ? "ser2" : "ser1";
+ registered = !!atari_console_driver.write;
if (!strcmp(arg, "ser1")) {
/* ST-MFP Modem1 serial port */
atari_init_mfp_port(B9600|CS8);
sound_ym.wd_data = sound_ym.rd_data_reg_sel | 0x20; /* strobe H */
atari_console_driver.write = atari_par_console_write;
}
- if (atari_console_driver.write)
+ if (atari_console_driver.write && !registered)
register_console(&atari_console_driver);
return 0;
#include <linux/bcd.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-vme.h>
+#include <asm/byteorder.h>
#include <asm/pgtable.h>
#include <asm/setup.h>
#include <asm/irq.h>
static irq_handler_t tick_handler;
-int bvme6000_parse_bootinfo(const struct bi_record *bi)
+int __init bvme6000_parse_bootinfo(const struct bi_record *bi)
{
- if (bi->tag == BI_VME_TYPE)
+ if (be16_to_cpu(bi->tag) == BI_VME_TYPE)
return 0;
else
return 1;
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_IDE=y
CONFIG_IDE_GD_ATAPI=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
# CONFIG_NET_VENDOR_3COM is not set
CONFIG_A2065=y
CONFIG_ARIADNE=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_CIRRUS is not set
-# CONFIG_NET_VENDOR_FUJITSU is not set
# CONFIG_NET_VENDOR_HP is not set
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_MARVELL is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_SMSC is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_INPUT_MISC=y
CONFIG_INPUT_M68K_BEEP=m
# CONFIG_SERIO is not set
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
CONFIG_PRINTER=m
CONFIG_PROC_HARDWARE=y
CONFIG_AMIGA_BUILTIN_SERIAL=y
CONFIG_SERIAL_CONSOLE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_TGT=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_NATSEMI is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
# CONFIG_MOUSE_PS2 is not set
CONFIG_MOUSE_SERIAL=m
CONFIG_SERIO=m
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_HEARTBEAT=y
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_IDE=y
CONFIG_IDE_GD_ATAPI=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_EQUALIZER=m
-CONFIG_MII=y
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
CONFIG_ATARILANCE=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_MICREL is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_INPUT_MISC=y
CONFIG_INPUT_M68K_BEEP=m
# CONFIG_SERIO is not set
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
CONFIG_PRINTER=m
CONFIG_NFCON=y
CONFIG_NFETH=y
CONFIG_ATARI_DSP56K=m
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_TGT=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
CONFIG_BVME6000_NET=y
# CONFIG_NET_VENDOR_NATSEMI is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_RTC_DRV_GENERIC=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_TGT=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
CONFIG_HPLANCE=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_NATSEMI is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_INPUT_MISC=y
CONFIG_HP_SDC_RTC=m
CONFIG_SERIO_SERPORT=m
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
# CONFIG_HW_RANDOM is not set
CONFIG_RTC_DRV_GENERIC=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_DEV_APPLETALK=m
CONFIG_IPDDP=m
CONFIG_IPDDP_ENCAP=y
-CONFIG_IPDDP_DECAP=y
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_IDE=y
CONFIG_IDE_GD_ATAPI=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
CONFIG_MACMACE=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
CONFIG_MAC89x0=y
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_SMSC is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_INPUT_MISC=y
CONFIG_INPUT_M68K_BEEP=m
CONFIG_SERIO=m
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
CONFIG_SERIAL_PMACZILOG=y
CONFIG_RTC_DRV_GENERIC=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
+CONFIG_EARLY_PRINTK=y
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_DEV_APPLETALK=m
CONFIG_IPDDP=m
CONFIG_IPDDP_ENCAP=y
-CONFIG_IPDDP_DECAP=y
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_IDE=y
CONFIG_IDE_GD_ATAPI=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_EQUALIZER=m
-CONFIG_MII=y
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_MVME147_NET=y
CONFIG_SUN3LANCE=y
CONFIG_MACMACE=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
CONFIG_MAC89x0=y
-# CONFIG_NET_VENDOR_FUJITSU is not set
# CONFIG_NET_VENDOR_HP is not set
CONFIG_BVME6000_NET=y
CONFIG_MVME16x_NET=y
CONFIG_ZORRO8390=y
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PLIP=m
CONFIG_PPP=m
CONFIG_INPUT_M68K_BEEP=m
CONFIG_HP_SDC_RTC=m
CONFIG_SERIO_Q40KBD=y
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
CONFIG_SERIAL_PMACZILOG=y
CONFIG_ATARI_DSP56K=m
CONFIG_AMIGA_BUILTIN_SERIAL=y
CONFIG_SERIAL_CONSOLE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
+CONFIG_EARLY_PRINTK=y
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_TGT=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
CONFIG_MVME147_NET=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_NATSEMI is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_RTC_DRV_GENERIC=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_TGT=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
CONFIG_MVME16x_NET=y
# CONFIG_NET_VENDOR_NATSEMI is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_RTC_DRV_GENERIC=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
+CONFIG_EARLY_PRINTK=y
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_IDE=y
CONFIG_IDE_GD_ATAPI=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_VETH=m
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_NET_VENDOR_AMD is not set
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_CIRRUS is not set
-# CONFIG_NET_VENDOR_FUJITSU is not set
# CONFIG_NET_VENDOR_HP is not set
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_MARVELL is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_SMSC is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PLIP=m
CONFIG_PPP=m
CONFIG_INPUT_MISC=y
CONFIG_INPUT_M68K_BEEP=m
CONFIG_SERIO_Q40KBD=y
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
CONFIG_PRINTER=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_HEARTBEAT=y
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_TGT=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
CONFIG_SUN3LANCE=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
CONFIG_SUN3_82586=y
# CONFIG_NET_VENDOR_MARVELL is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
# CONFIG_NET_VENDOR_SUN is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_KEYBOARD_SUNKBD=y
# CONFIG_MOUSE_PS2 is not set
CONFIG_MOUSE_SERIAL=m
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
# CONFIG_HW_RANDOM is not set
CONFIG_RTC_DRV_GENERIC=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IPGRE=m
-CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
# CONFIG_INET_LRO is not set
CONFIG_INET_DIAG=m
CONFIG_INET_UDP_DIAG=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
+CONFIG_IPV6_VTI=m
CONFIG_IPV6_GRE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
+CONFIG_NF_TABLES=m
+CONFIG_NFT_EXTHDR=m
+CONFIG_NFT_META=m
+CONFIG_NFT_CT=m
+CONFIG_NFT_RBTREE=m
+CONFIG_NFT_HASH=m
+CONFIG_NFT_COUNTER=m
+CONFIG_NFT_LOG=m
+CONFIG_NFT_LIMIT=m
+CONFIG_NFT_NAT=m
+CONFIG_NFT_COMPAT=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
+CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
+CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
+CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
+CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_NF_CONNTRACK_IPV4=m
+CONFIG_NF_TABLES_IPV4=m
+CONFIG_NFT_REJECT_IPV4=m
+CONFIG_NFT_CHAIN_ROUTE_IPV4=m
+CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_NF_TABLES_ARP=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
+CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT_IPV4=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_NF_CONNTRACK_IPV6=m
+CONFIG_NF_TABLES_IPV6=m
+CONFIG_NFT_CHAIN_ROUTE_IPV6=m
+CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
+CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_NF_NAT_IPV6=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
+CONFIG_NF_TABLES_BRIDGE=m
CONFIG_IP_DCCP=m
# CONFIG_IP_DCCP_CCID3 is not set
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_RDS_TCP=m
CONFIG_L2TP=m
CONFIG_ATALK=m
+CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
CONFIG_BATMAN_ADV_DAT=y
+CONFIG_BATMAN_ADV_NC=y
+CONFIG_NETLINK_DIAG=m
+CONFIG_NET_MPLS_GSO=m
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
# CONFIG_FW_LOADER_USER_HELPER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_ATA_OVER_ETH=m
+CONFIG_DUMMY_IRQ=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_TGT=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
+CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_VXLAN=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_VETH=m
CONFIG_SUN3LANCE=y
+# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_CADENCE is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_NATSEMI is not set
# CONFIG_NET_VENDOR_SEEQ is not set
# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_KEYBOARD_SUNKBD=y
# CONFIG_MOUSE_PS2 is not set
CONFIG_MOUSE_SERIAL=m
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_DEVKMEM is not set
# CONFIG_HW_RANDOM is not set
CONFIG_RTC_DRV_GENERIC=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_PROC_HARDWARE=y
-CONFIG_EXT2_FS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-# CONFIG_EXT3_FS_XATTR is not set
CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_JFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V4=y
+CONFIG_NFS_V4=m
CONFIG_NFS_SWAP=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_STRING_HELPERS=m
CONFIG_ENCRYPTED_KEYS=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_USER=m
-CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_LZ4=m
+CONFIG_CRYPTO_LZ4HC=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
* the GNU General Public License (GPL), incorporated herein by reference.
*/
+#include <linux/init.h>
#include <linux/types.h>
#include <linux/console.h>
#include <linux/string.h>
nf_call(id);
}
-void nf_init(void)
+void __init nf_init(void)
{
unsigned long id, version;
char buf[256];
#include <linux/console.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-hp300.h>
+#include <asm/byteorder.h>
#include <asm/machdep.h>
#include <asm/blinken.h>
#include <asm/io.h> /* readb() and writeb() */
int __init hp300_parse_bootinfo(const struct bi_record *record)
{
int unknown = 0;
- const unsigned long *data = record->data;
+ const void *data = record->data;
- switch (record->tag) {
+ switch (be16_to_cpu(record->tag)) {
case BI_HP300_MODEL:
- hp300_model = *data;
+ hp300_model = be32_to_cpup(data);
break;
case BI_HP300_UART_SCODE:
- hp300_uart_scode = *data;
+ hp300_uart_scode = be32_to_cpup(data);
break;
case BI_HP300_UART_ADDR:
#include <linux/ioport.h>
- /*
- * Different Amiga models
- */
-
-#define AMI_UNKNOWN (0)
-#define AMI_500 (1)
-#define AMI_500PLUS (2)
-#define AMI_600 (3)
-#define AMI_1000 (4)
-#define AMI_1200 (5)
-#define AMI_2000 (6)
-#define AMI_2500 (7)
-#define AMI_3000 (8)
-#define AMI_3000T (9)
-#define AMI_3000PLUS (10)
-#define AMI_4000 (11)
-#define AMI_4000T (12)
-#define AMI_CDTV (13)
-#define AMI_CD32 (14)
-#define AMI_DRACO (15)
+#include <asm/bootinfo-amiga.h>
/*
extern unsigned long amiga_chipset;
-#define CS_STONEAGE (0)
-#define CS_OCS (1)
-#define CS_ECS (2)
-#define CS_AGA (3)
-
/*
* Miscellaneous
#define zTwoBase (0x80000000)
#define ZTWO_PADDR(x) (((unsigned long)(x))-zTwoBase)
-#define ZTWO_VADDR(x) (((unsigned long)(x))+zTwoBase)
+#define ZTWO_VADDR(x) ((void __iomem *)(((unsigned long)(x))+zTwoBase))
#define CUSTOM_PHYSADDR (0xdff000)
#define amiga_custom ((*(volatile struct CUSTOM *)(zTwoBase+CUSTOM_PHYSADDR)))
#include <linux/types.h>
-/*
- apollo models
-*/
+#include <asm/bootinfo-apollo.h>
+
extern u_long apollo_model;
-#define APOLLO_UNKNOWN (0)
-#define APOLLO_DN3000 (1)
-#define APOLLO_DN3010 (2)
-#define APOLLO_DN3500 (3)
-#define APOLLO_DN4000 (4)
-#define APOLLO_DN4500 (5)
/*
see scn2681 data sheet for more info.
#define _LINUX_ATARIHW_H_
#include <linux/types.h>
-#include <asm/bootinfo.h>
+#include <asm/bootinfo-atari.h>
#include <asm/raw_io.h>
extern u_long atari_mch_cookie;
#ifndef _M68K_BARRIER_H
#define _M68K_BARRIER_H
-/*
- * Force strict CPU ordering.
- * Not really required on m68k...
- */
#define nop() do { asm volatile ("nop"); barrier(); } while (0)
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define read_barrier_depends() ((void)0)
-#define set_mb(var, value) ({ (var) = (value); wmb(); })
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() ((void)0)
+#include <asm-generic/barrier.h>
#endif /* _M68K_BARRIER_H */
** This file is subject to the terms and conditions of the GNU General Public
** License. See the file COPYING in the main directory of this archive
** for more details.
-**
-** Created 09/29/92 by Greg Harp
-**
-** 5/2/94 Roman Hodek:
-** Added bi_atari part of the machine dependent union bi_un; for now it
-** contains just a model field to distinguish between TT and Falcon.
-** 26/7/96 Roman Zippel:
-** Renamed to setup.h; added some useful macros to allow gcc some
-** optimizations if possible.
-** 5/10/96 Geert Uytterhoeven:
-** Redesign of the boot information structure; renamed to bootinfo.h again
-** 27/11/96 Geert Uytterhoeven:
-** Backwards compatibility with bootinfo interface version 1.0
*/
#ifndef _M68K_BOOTINFO_H
#define _M68K_BOOTINFO_H
+#include <uapi/asm/bootinfo.h>
- /*
- * Bootinfo definitions
- *
- * This is an easily parsable and extendable structure containing all
- * information to be passed from the bootstrap to the kernel.
- *
- * This way I hope to keep all future changes back/forewards compatible.
- * Thus, keep your fingers crossed...
- *
- * This structure is copied right after the kernel bss by the bootstrap
- * routine.
- */
#ifndef __ASSEMBLY__
-struct bi_record {
- unsigned short tag; /* tag ID */
- unsigned short size; /* size of record (in bytes) */
- unsigned long data[0]; /* data */
-};
-
-#endif /* __ASSEMBLY__ */
-
-
- /*
- * Tag Definitions
- *
- * Machine independent tags start counting from 0x0000
- * Machine dependent tags start counting from 0x8000
- */
-
-#define BI_LAST 0x0000 /* last record (sentinel) */
-#define BI_MACHTYPE 0x0001 /* machine type (u_long) */
-#define BI_CPUTYPE 0x0002 /* cpu type (u_long) */
-#define BI_FPUTYPE 0x0003 /* fpu type (u_long) */
-#define BI_MMUTYPE 0x0004 /* mmu type (u_long) */
-#define BI_MEMCHUNK 0x0005 /* memory chunk address and size */
- /* (struct mem_info) */
-#define BI_RAMDISK 0x0006 /* ramdisk address and size */
- /* (struct mem_info) */
-#define BI_COMMAND_LINE 0x0007 /* kernel command line parameters */
- /* (string) */
-
- /*
- * Amiga-specific tags
- */
-
-#define BI_AMIGA_MODEL 0x8000 /* model (u_long) */
-#define BI_AMIGA_AUTOCON 0x8001 /* AutoConfig device */
- /* (struct ConfigDev) */
-#define BI_AMIGA_CHIP_SIZE 0x8002 /* size of Chip RAM (u_long) */
-#define BI_AMIGA_VBLANK 0x8003 /* VBLANK frequency (u_char) */
-#define BI_AMIGA_PSFREQ 0x8004 /* power supply frequency (u_char) */
-#define BI_AMIGA_ECLOCK 0x8005 /* EClock frequency (u_long) */
-#define BI_AMIGA_CHIPSET 0x8006 /* native chipset present (u_long) */
-#define BI_AMIGA_SERPER 0x8007 /* serial port period (u_short) */
-
- /*
- * Atari-specific tags
- */
-
-#define BI_ATARI_MCH_COOKIE 0x8000 /* _MCH cookie from TOS (u_long) */
-#define BI_ATARI_MCH_TYPE 0x8001 /* special machine type (u_long) */
- /* (values are ATARI_MACH_* defines */
-
-/* mch_cookie values (upper word) */
-#define ATARI_MCH_ST 0
-#define ATARI_MCH_STE 1
-#define ATARI_MCH_TT 2
-#define ATARI_MCH_FALCON 3
-
-/* mch_type values */
-#define ATARI_MACH_NORMAL 0 /* no special machine type */
-#define ATARI_MACH_MEDUSA 1 /* Medusa 040 */
-#define ATARI_MACH_HADES 2 /* Hades 040 or 060 */
-#define ATARI_MACH_AB40 3 /* Afterburner040 on Falcon */
-
- /*
- * VME-specific tags
- */
-
-#define BI_VME_TYPE 0x8000 /* VME sub-architecture (u_long) */
-#define BI_VME_BRDINFO 0x8001 /* VME board information (struct) */
-
-/* BI_VME_TYPE codes */
-#define VME_TYPE_TP34V 0x0034 /* Tadpole TP34V */
-#define VME_TYPE_MVME147 0x0147 /* Motorola MVME147 */
-#define VME_TYPE_MVME162 0x0162 /* Motorola MVME162 */
-#define VME_TYPE_MVME166 0x0166 /* Motorola MVME166 */
-#define VME_TYPE_MVME167 0x0167 /* Motorola MVME167 */
-#define VME_TYPE_MVME172 0x0172 /* Motorola MVME172 */
-#define VME_TYPE_MVME177 0x0177 /* Motorola MVME177 */
-#define VME_TYPE_BVME4000 0x4000 /* BVM Ltd. BVME4000 */
-#define VME_TYPE_BVME6000 0x6000 /* BVM Ltd. BVME6000 */
-
-/* BI_VME_BRDINFO is a 32 byte struct as returned by the Bug code on
- * Motorola VME boards. Contains board number, Bug version, board
- * configuration options, etc. See include/asm/mvme16xhw.h for details.
- */
-
-
- /*
- * Macintosh-specific tags (all u_long)
- */
-
-#define BI_MAC_MODEL 0x8000 /* Mac Gestalt ID (model type) */
-#define BI_MAC_VADDR 0x8001 /* Mac video base address */
-#define BI_MAC_VDEPTH 0x8002 /* Mac video depth */
-#define BI_MAC_VROW 0x8003 /* Mac video rowbytes */
-#define BI_MAC_VDIM 0x8004 /* Mac video dimensions */
-#define BI_MAC_VLOGICAL 0x8005 /* Mac video logical base */
-#define BI_MAC_SCCBASE 0x8006 /* Mac SCC base address */
-#define BI_MAC_BTIME 0x8007 /* Mac boot time */
-#define BI_MAC_GMTBIAS 0x8008 /* Mac GMT timezone offset */
-#define BI_MAC_MEMSIZE 0x8009 /* Mac RAM size (sanity check) */
-#define BI_MAC_CPUID 0x800a /* Mac CPU type (sanity check) */
-#define BI_MAC_ROMBASE 0x800b /* Mac system ROM base address */
-
- /*
- * Macintosh hardware profile data - unused, see macintosh.h for
- * reasonable type values
- */
-
-#define BI_MAC_VIA1BASE 0x8010 /* Mac VIA1 base address (always present) */
-#define BI_MAC_VIA2BASE 0x8011 /* Mac VIA2 base address (type varies) */
-#define BI_MAC_VIA2TYPE 0x8012 /* Mac VIA2 type (VIA, RBV, OSS) */
-#define BI_MAC_ADBTYPE 0x8013 /* Mac ADB interface type */
-#define BI_MAC_ASCBASE 0x8014 /* Mac Apple Sound Chip base address */
-#define BI_MAC_SCSI5380 0x8015 /* Mac NCR 5380 SCSI (base address, multi) */
-#define BI_MAC_SCSIDMA 0x8016 /* Mac SCSI DMA (base address) */
-#define BI_MAC_SCSI5396 0x8017 /* Mac NCR 53C96 SCSI (base address, multi) */
-#define BI_MAC_IDETYPE 0x8018 /* Mac IDE interface type */
-#define BI_MAC_IDEBASE 0x8019 /* Mac IDE interface base address */
-#define BI_MAC_NUBUS 0x801a /* Mac Nubus type (none, regular, pseudo) */
-#define BI_MAC_SLOTMASK 0x801b /* Mac Nubus slots present */
-#define BI_MAC_SCCTYPE 0x801c /* Mac SCC serial type (normal, IOP) */
-#define BI_MAC_ETHTYPE 0x801d /* Mac builtin ethernet type (Sonic, MACE */
-#define BI_MAC_ETHBASE 0x801e /* Mac builtin ethernet base address */
-#define BI_MAC_PMU 0x801f /* Mac power management / poweroff hardware */
-#define BI_MAC_IOP_SWIM 0x8020 /* Mac SWIM floppy IOP */
-#define BI_MAC_IOP_ADB 0x8021 /* Mac ADB IOP */
-
- /*
- * Mac: compatibility with old booter data format (temporarily)
- * Fields unused with the new bootinfo can be deleted now; instead of
- * adding new fields the struct might be splitted into a hardware address
- * part and a hardware type part
- */
-
-#ifndef __ASSEMBLY__
-
-struct mac_booter_data
-{
- unsigned long videoaddr;
- unsigned long videorow;
- unsigned long videodepth;
- unsigned long dimensions;
- unsigned long args;
- unsigned long boottime;
- unsigned long gmtbias;
- unsigned long bootver;
- unsigned long videological;
- unsigned long sccbase;
- unsigned long id;
- unsigned long memsize;
- unsigned long serialmf;
- unsigned long serialhsk;
- unsigned long serialgpi;
- unsigned long printmf;
- unsigned long printhsk;
- unsigned long printgpi;
- unsigned long cpuid;
- unsigned long rombase;
- unsigned long adbdelay;
- unsigned long timedbra;
-};
-
-extern struct mac_booter_data
- mac_bi_data;
-
+#ifdef CONFIG_BOOTINFO_PROC
+extern void save_bootinfo(const struct bi_record *bi);
+#else
+static inline void save_bootinfo(const struct bi_record *bi) {}
#endif
- /*
- * Apollo-specific tags
- */
-
-#define BI_APOLLO_MODEL 0x8000 /* model (u_long) */
-
- /*
- * HP300-specific tags
- */
-
-#define BI_HP300_MODEL 0x8000 /* model (u_long) */
-#define BI_HP300_UART_SCODE 0x8001 /* UART select code (u_long) */
-#define BI_HP300_UART_ADDR 0x8002 /* phys. addr of UART (u_long) */
-
- /*
- * Stuff for bootinfo interface versioning
- *
- * At the start of kernel code, a 'struct bootversion' is located.
- * bootstrap checks for a matching version of the interface before booting
- * a kernel, to avoid user confusion if kernel and bootstrap don't work
- * together :-)
- *
- * If incompatible changes are made to the bootinfo interface, the major
- * number below should be stepped (and the minor reset to 0) for the
- * appropriate machine. If a change is backward-compatible, the minor
- * should be stepped. "Backwards-compatible" means that booting will work,
- * but certain features may not.
- */
-
-#define BOOTINFOV_MAGIC 0x4249561A /* 'BIV^Z' */
-#define MK_BI_VERSION(major,minor) (((major)<<16)+(minor))
-#define BI_VERSION_MAJOR(v) (((v) >> 16) & 0xffff)
-#define BI_VERSION_MINOR(v) ((v) & 0xffff)
-
-#ifndef __ASSEMBLY__
-
-struct bootversion {
- unsigned short branch;
- unsigned long magic;
- struct {
- unsigned long machtype;
- unsigned long version;
- } machversions[0];
-};
-
#endif /* __ASSEMBLY__ */
-#define AMIGA_BOOTI_VERSION MK_BI_VERSION( 2, 0 )
-#define ATARI_BOOTI_VERSION MK_BI_VERSION( 2, 1 )
-#define MAC_BOOTI_VERSION MK_BI_VERSION( 2, 0 )
-#define MVME147_BOOTI_VERSION MK_BI_VERSION( 2, 0 )
-#define MVME16x_BOOTI_VERSION MK_BI_VERSION( 2, 0 )
-#define BVME6000_BOOTI_VERSION MK_BI_VERSION( 2, 0 )
-#define Q40_BOOTI_VERSION MK_BI_VERSION( 2, 0 )
-#define HP300_BOOTI_VERSION MK_BI_VERSION( 2, 0 )
-
-#ifdef BOOTINFO_COMPAT_1_0
-
- /*
- * Backwards compatibility with bootinfo interface version 1.0
- */
-
-#define COMPAT_AMIGA_BOOTI_VERSION MK_BI_VERSION( 1, 0 )
-#define COMPAT_ATARI_BOOTI_VERSION MK_BI_VERSION( 1, 0 )
-#define COMPAT_MAC_BOOTI_VERSION MK_BI_VERSION( 1, 0 )
-
-#include <linux/zorro.h>
-
-#define COMPAT_NUM_AUTO 16
-
-struct compat_bi_Amiga {
- int model;
- int num_autocon;
- struct ConfigDev autocon[COMPAT_NUM_AUTO];
- unsigned long chip_size;
- unsigned char vblank;
- unsigned char psfreq;
- unsigned long eclock;
- unsigned long chipset;
- unsigned long hw_present;
-};
-
-struct compat_bi_Atari {
- unsigned long hw_present;
- unsigned long mch_cookie;
-};
-
-#ifndef __ASSEMBLY__
-
-struct compat_bi_Macintosh
-{
- unsigned long videoaddr;
- unsigned long videorow;
- unsigned long videodepth;
- unsigned long dimensions;
- unsigned long args;
- unsigned long boottime;
- unsigned long gmtbias;
- unsigned long bootver;
- unsigned long videological;
- unsigned long sccbase;
- unsigned long id;
- unsigned long memsize;
- unsigned long serialmf;
- unsigned long serialhsk;
- unsigned long serialgpi;
- unsigned long printmf;
- unsigned long printhsk;
- unsigned long printgpi;
- unsigned long cpuid;
- unsigned long rombase;
- unsigned long adbdelay;
- unsigned long timedbra;
-};
-
-#endif
-
-struct compat_mem_info {
- unsigned long addr;
- unsigned long size;
-};
-
-#define COMPAT_NUM_MEMINFO 4
-
-#define COMPAT_CPUB_68020 0
-#define COMPAT_CPUB_68030 1
-#define COMPAT_CPUB_68040 2
-#define COMPAT_CPUB_68060 3
-#define COMPAT_FPUB_68881 5
-#define COMPAT_FPUB_68882 6
-#define COMPAT_FPUB_68040 7
-#define COMPAT_FPUB_68060 8
-
-#define COMPAT_CPU_68020 (1<<COMPAT_CPUB_68020)
-#define COMPAT_CPU_68030 (1<<COMPAT_CPUB_68030)
-#define COMPAT_CPU_68040 (1<<COMPAT_CPUB_68040)
-#define COMPAT_CPU_68060 (1<<COMPAT_CPUB_68060)
-#define COMPAT_CPU_MASK (31)
-#define COMPAT_FPU_68881 (1<<COMPAT_FPUB_68881)
-#define COMPAT_FPU_68882 (1<<COMPAT_FPUB_68882)
-#define COMPAT_FPU_68040 (1<<COMPAT_FPUB_68040)
-#define COMPAT_FPU_68060 (1<<COMPAT_FPUB_68060)
-#define COMPAT_FPU_MASK (0xfe0)
-
-#define COMPAT_CL_SIZE (256)
-
-struct compat_bootinfo {
- unsigned long machtype;
- unsigned long cputype;
- struct compat_mem_info memory[COMPAT_NUM_MEMINFO];
- int num_memory;
- unsigned long ramdisk_size;
- unsigned long ramdisk_addr;
- char command_line[COMPAT_CL_SIZE];
- union {
- struct compat_bi_Amiga bi_ami;
- struct compat_bi_Atari bi_ata;
- struct compat_bi_Macintosh bi_mac;
- } bi_un;
-};
-
-#define bi_amiga bi_un.bi_ami
-#define bi_atari bi_un.bi_ata
-#define bi_mac bi_un.bi_mac
-
-#endif /* BOOTINFO_COMPAT_1_0 */
-
#endif /* _M68K_BOOTINFO_H */
#ifndef _M68K_HP300HW_H
#define _M68K_HP300HW_H
-extern unsigned long hp300_model;
+#include <asm/bootinfo-hp300.h>
-/* This information was taken from NetBSD */
-#define HP_320 (0) /* 16MHz 68020+HP MMU+16K external cache */
-#define HP_330 (1) /* 16MHz 68020+68851 MMU */
-#define HP_340 (2) /* 16MHz 68030 */
-#define HP_345 (3) /* 50MHz 68030+32K external cache */
-#define HP_350 (4) /* 25MHz 68020+HP MMU+32K external cache */
-#define HP_360 (5) /* 25MHz 68030 */
-#define HP_370 (6) /* 33MHz 68030+64K external cache */
-#define HP_375 (7) /* 50MHz 68030+32K external cache */
-#define HP_380 (8) /* 25MHz 68040 */
-#define HP_385 (9) /* 33MHz 68040 */
-#define HP_400 (10) /* 50MHz 68030+32K external cache */
-#define HP_425T (11) /* 25MHz 68040 - model 425t */
-#define HP_425S (12) /* 25MHz 68040 - model 425s */
-#define HP_425E (13) /* 25MHz 68040 - model 425e */
-#define HP_433T (14) /* 33MHz 68040 - model 433t */
-#define HP_433S (15) /* 33MHz 68040 - model 433s */
+extern unsigned long hp300_model;
#endif /* _M68K_HP300HW_H */
--- /dev/null
+#ifndef _ASM_M68K_KEXEC_H
+#define _ASM_M68K_KEXEC_H
+
+#ifdef CONFIG_KEXEC
+
+/* Maximum physical address we can use pages from */
+#define KEXEC_SOURCE_MEMORY_LIMIT (-1UL)
+/* Maximum address we can reach in physical address mode */
+#define KEXEC_DESTINATION_MEMORY_LIMIT (-1UL)
+/* Maximum address we can use for the control code buffer */
+#define KEXEC_CONTROL_MEMORY_LIMIT (-1UL)
+
+#define KEXEC_CONTROL_PAGE_SIZE 4096
+
+#define KEXEC_ARCH KEXEC_ARCH_68K
+
+#ifndef __ASSEMBLY__
+
+static inline void crash_setup_regs(struct pt_regs *newregs,
+ struct pt_regs *oldregs)
+{
+ /* Dummy implementation for now */
+}
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* CONFIG_KEXEC */
+
+#endif /* _ASM_M68K_KEXEC_H */
extern volatile __u8 *via1,*via2;
extern int rbv_present,via_alt_mapping;
+struct irq_desc;
+
extern void via_register_interrupts(void);
extern void via_irq_enable(int);
extern void via_irq_disable(int);
#include <linux/seq_file.h>
#include <linux/interrupt.h>
+#include <asm/bootinfo-mac.h>
+
+
/*
* Apple Macintoshisms
*/
#define MAC_FLOPPY_SWIM_IOP 3
#define MAC_FLOPPY_AV 4
-/*
- * Gestalt numbers
- */
+extern struct mac_model *macintosh_config;
-#define MAC_MODEL_II 6
-#define MAC_MODEL_IIX 7
-#define MAC_MODEL_IICX 8
-#define MAC_MODEL_SE30 9
-#define MAC_MODEL_IICI 11
-#define MAC_MODEL_IIFX 13 /* And well numbered it is too */
-#define MAC_MODEL_IISI 18
-#define MAC_MODEL_LC 19
-#define MAC_MODEL_Q900 20
-#define MAC_MODEL_PB170 21
-#define MAC_MODEL_Q700 22
-#define MAC_MODEL_CLII 23 /* aka: P200 */
-#define MAC_MODEL_PB140 25
-#define MAC_MODEL_Q950 26 /* aka: WGS95 */
-#define MAC_MODEL_LCIII 27 /* aka: P450 */
-#define MAC_MODEL_PB210 29
-#define MAC_MODEL_C650 30
-#define MAC_MODEL_PB230 32
-#define MAC_MODEL_PB180 33
-#define MAC_MODEL_PB160 34
-#define MAC_MODEL_Q800 35 /* aka: WGS80 */
-#define MAC_MODEL_Q650 36
-#define MAC_MODEL_LCII 37 /* aka: P400/405/410/430 */
-#define MAC_MODEL_PB250 38
-#define MAC_MODEL_IIVI 44
-#define MAC_MODEL_P600 45 /* aka: P600CD */
-#define MAC_MODEL_IIVX 48
-#define MAC_MODEL_CCL 49 /* aka: P250 */
-#define MAC_MODEL_PB165C 50
-#define MAC_MODEL_C610 52 /* aka: WGS60 */
-#define MAC_MODEL_Q610 53
-#define MAC_MODEL_PB145 54 /* aka: PB145B */
-#define MAC_MODEL_P520 56 /* aka: LC520 */
-#define MAC_MODEL_C660 60
-#define MAC_MODEL_P460 62 /* aka: LCIII+, P466/P467 */
-#define MAC_MODEL_PB180C 71
-#define MAC_MODEL_PB520 72 /* aka: PB520C, PB540, PB540C, PB550C */
-#define MAC_MODEL_PB270C 77
-#define MAC_MODEL_Q840 78
-#define MAC_MODEL_P550 80 /* aka: LC550, P560 */
-#define MAC_MODEL_CCLII 83 /* aka: P275 */
-#define MAC_MODEL_PB165 84
-#define MAC_MODEL_PB190 85 /* aka: PB190CS */
-#define MAC_MODEL_TV 88
-#define MAC_MODEL_P475 89 /* aka: LC475, P476 */
-#define MAC_MODEL_P475F 90 /* aka: P475 w/ FPU (no LC040) */
-#define MAC_MODEL_P575 92 /* aka: LC575, P577/P578 */
-#define MAC_MODEL_Q605 94
-#define MAC_MODEL_Q605_ACC 95 /* Q605 accelerated to 33 MHz */
-#define MAC_MODEL_Q630 98 /* aka: LC630, P630/631/635/636/637/638/640 */
-#define MAC_MODEL_P588 99 /* aka: LC580, P580 */
-#define MAC_MODEL_PB280 102
-#define MAC_MODEL_PB280C 103
-#define MAC_MODEL_PB150 115
-extern struct mac_model *macintosh_config;
+ /*
+ * Internal representation of the Mac hardware, filled in from bootinfo
+ */
+
+struct mac_booter_data
+{
+ unsigned long videoaddr;
+ unsigned long videorow;
+ unsigned long videodepth;
+ unsigned long dimensions;
+ unsigned long boottime;
+ unsigned long gmtbias;
+ unsigned long videological;
+ unsigned long sccbase;
+ unsigned long id;
+ unsigned long memsize;
+ unsigned long cpuid;
+ unsigned long rombase;
+};
+
+extern struct mac_booter_data mac_bi_data;
#endif
#include <asm/atarihw.h>
-#define RTC_PORT(x) (TT_RTC_BAS + 2*(x))
+#define ATARI_RTC_PORT(x) (TT_RTC_BAS + 2*(x))
#define RTC_ALWAYS_BCD 0
#define CMOS_READ(addr) ({ \
-atari_outb_p((addr),RTC_PORT(0)); \
-atari_inb_p(RTC_PORT(1)); \
+atari_outb_p((addr), ATARI_RTC_PORT(0)); \
+atari_inb_p(ATARI_RTC_PORT(1)); \
})
#define CMOS_WRITE(val, addr) ({ \
-atari_outb_p((addr),RTC_PORT(0)); \
-atari_outb_p((val),RTC_PORT(1)); \
+atari_outb_p((addr), ATARI_RTC_PORT(0)); \
+atari_outb_p((val), ATARI_RTC_PORT(1)); \
})
#endif /* CONFIG_ATARI */
#include <asm/irq.h>
-/* Board ID data structure - pointer to this retrieved from Bug by head.S */
-
-/* Note, bytes 12 and 13 are board no in BCD (0162,0166,0167,0177,etc) */
-
-extern long mvme_bdid_ptr;
-
-typedef struct {
- char bdid[4];
- u_char rev, mth, day, yr;
- u_short size, reserved;
- u_short brdno;
- char brdsuffix[2];
- u_long options;
- u_short clun, dlun, ctype, dnum;
- u_long option2;
-} t_bdid, *p_bdid;
-
typedef struct {
u_char ack_icr,
#ifndef _M68K_SETUP_H
#define _M68K_SETUP_H
+#include <uapi/asm/bootinfo.h>
#include <uapi/asm/setup.h>
#define NUM_MEMINFO 4
#ifndef __ASSEMBLY__
-struct mem_info {
+struct m68k_mem_info {
unsigned long addr; /* physical address of memory chunk */
unsigned long size; /* length of memory chunk (in bytes) */
};
extern int m68k_num_memory; /* # of memory blocks found (and used) */
extern int m68k_realnum_memory; /* real # of memory blocks found */
-extern struct mem_info m68k_memory[NUM_MEMINFO];/* memory description */
+extern struct m68k_mem_info m68k_memory[NUM_MEMINFO];/* memory description */
#endif
#endif /* _M68K_SETUP_H */
return 0;
}
+extern unsigned long (*mach_random_get_entropy)(void);
+
+static inline unsigned long random_get_entropy(void)
+{
+ if (mach_random_get_entropy)
+ return mach_random_get_entropy();
+ return 0;
+}
+#define random_get_entropy random_get_entropy
+
#endif
generic-y += termios.h
header-y += a.out.h
+header-y += bootinfo.h
+header-y += bootinfo-amiga.h
+header-y += bootinfo-apollo.h
+header-y += bootinfo-atari.h
+header-y += bootinfo-hp300.h
+header-y += bootinfo-mac.h
+header-y += bootinfo-q40.h
+header-y += bootinfo-vme.h
header-y += byteorder.h
header-y += cachectl.h
header-y += fcntl.h
--- /dev/null
+/*
+** asm/bootinfo-amiga.h -- Amiga-specific boot information definitions
+*/
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_AMIGA_H
+#define _UAPI_ASM_M68K_BOOTINFO_AMIGA_H
+
+
+ /*
+ * Amiga-specific tags
+ */
+
+#define BI_AMIGA_MODEL 0x8000 /* model (__be32) */
+#define BI_AMIGA_AUTOCON 0x8001 /* AutoConfig device */
+ /* (AmigaOS struct ConfigDev) */
+#define BI_AMIGA_CHIP_SIZE 0x8002 /* size of Chip RAM (__be32) */
+#define BI_AMIGA_VBLANK 0x8003 /* VBLANK frequency (__u8) */
+#define BI_AMIGA_PSFREQ 0x8004 /* power supply frequency (__u8) */
+#define BI_AMIGA_ECLOCK 0x8005 /* EClock frequency (__be32) */
+#define BI_AMIGA_CHIPSET 0x8006 /* native chipset present (__be32) */
+#define BI_AMIGA_SERPER 0x8007 /* serial port period (__be16) */
+
+
+ /*
+ * Amiga models (BI_AMIGA_MODEL)
+ */
+
+#define AMI_UNKNOWN 0
+#define AMI_500 1
+#define AMI_500PLUS 2
+#define AMI_600 3
+#define AMI_1000 4
+#define AMI_1200 5
+#define AMI_2000 6
+#define AMI_2500 7
+#define AMI_3000 8
+#define AMI_3000T 9
+#define AMI_3000PLUS 10
+#define AMI_4000 11
+#define AMI_4000T 12
+#define AMI_CDTV 13
+#define AMI_CD32 14
+#define AMI_DRACO 15
+
+
+ /*
+ * Amiga chipsets (BI_AMIGA_CHIPSET)
+ */
+
+#define CS_STONEAGE 0
+#define CS_OCS 1
+#define CS_ECS 2
+#define CS_AGA 3
+
+
+ /*
+ * Latest Amiga bootinfo version
+ */
+
+#define AMIGA_BOOTI_VERSION MK_BI_VERSION(2, 0)
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_AMIGA_H */
--- /dev/null
+/*
+** asm/bootinfo-apollo.h -- Apollo-specific boot information definitions
+*/
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_APOLLO_H
+#define _UAPI_ASM_M68K_BOOTINFO_APOLLO_H
+
+
+ /*
+ * Apollo-specific tags
+ */
+
+#define BI_APOLLO_MODEL 0x8000 /* model (__be32) */
+
+
+ /*
+ * Apollo models (BI_APOLLO_MODEL)
+ */
+
+#define APOLLO_UNKNOWN 0
+#define APOLLO_DN3000 1
+#define APOLLO_DN3010 2
+#define APOLLO_DN3500 3
+#define APOLLO_DN4000 4
+#define APOLLO_DN4500 5
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_APOLLO_H */
--- /dev/null
+/*
+** asm/bootinfo-atari.h -- Atari-specific boot information definitions
+*/
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_ATARI_H
+#define _UAPI_ASM_M68K_BOOTINFO_ATARI_H
+
+
+ /*
+ * Atari-specific tags
+ */
+
+#define BI_ATARI_MCH_COOKIE 0x8000 /* _MCH cookie from TOS (__be32) */
+#define BI_ATARI_MCH_TYPE 0x8001 /* special machine type (__be32) */
+
+
+ /*
+ * mch_cookie values (upper word of BI_ATARI_MCH_COOKIE)
+ */
+
+#define ATARI_MCH_ST 0
+#define ATARI_MCH_STE 1
+#define ATARI_MCH_TT 2
+#define ATARI_MCH_FALCON 3
+
+
+ /*
+ * Atari machine types (BI_ATARI_MCH_TYPE)
+ */
+
+#define ATARI_MACH_NORMAL 0 /* no special machine type */
+#define ATARI_MACH_MEDUSA 1 /* Medusa 040 */
+#define ATARI_MACH_HADES 2 /* Hades 040 or 060 */
+#define ATARI_MACH_AB40 3 /* Afterburner040 on Falcon */
+
+
+ /*
+ * Latest Atari bootinfo version
+ */
+
+#define ATARI_BOOTI_VERSION MK_BI_VERSION(2, 1)
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_ATARI_H */
--- /dev/null
+/*
+** asm/bootinfo-hp300.h -- HP9000/300-specific boot information definitions
+*/
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_HP300_H
+#define _UAPI_ASM_M68K_BOOTINFO_HP300_H
+
+
+ /*
+ * HP9000/300-specific tags
+ */
+
+#define BI_HP300_MODEL 0x8000 /* model (__be32) */
+#define BI_HP300_UART_SCODE 0x8001 /* UART select code (__be32) */
+#define BI_HP300_UART_ADDR 0x8002 /* phys. addr of UART (__be32) */
+
+
+ /*
+ * HP9000/300 and /400 models (BI_HP300_MODEL)
+ *
+ * This information was taken from NetBSD
+ */
+
+#define HP_320 0 /* 16MHz 68020+HP MMU+16K external cache */
+#define HP_330 1 /* 16MHz 68020+68851 MMU */
+#define HP_340 2 /* 16MHz 68030 */
+#define HP_345 3 /* 50MHz 68030+32K external cache */
+#define HP_350 4 /* 25MHz 68020+HP MMU+32K external cache */
+#define HP_360 5 /* 25MHz 68030 */
+#define HP_370 6 /* 33MHz 68030+64K external cache */
+#define HP_375 7 /* 50MHz 68030+32K external cache */
+#define HP_380 8 /* 25MHz 68040 */
+#define HP_385 9 /* 33MHz 68040 */
+
+#define HP_400 10 /* 50MHz 68030+32K external cache */
+#define HP_425T 11 /* 25MHz 68040 - model 425t */
+#define HP_425S 12 /* 25MHz 68040 - model 425s */
+#define HP_425E 13 /* 25MHz 68040 - model 425e */
+#define HP_433T 14 /* 33MHz 68040 - model 433t */
+#define HP_433S 15 /* 33MHz 68040 - model 433s */
+
+
+ /*
+ * Latest HP9000/300 bootinfo version
+ */
+
+#define HP300_BOOTI_VERSION MK_BI_VERSION(2, 0)
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_HP300_H */
--- /dev/null
+/*
+** asm/bootinfo-mac.h -- Macintosh-specific boot information definitions
+*/
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_MAC_H
+#define _UAPI_ASM_M68K_BOOTINFO_MAC_H
+
+
+ /*
+ * Macintosh-specific tags (all __be32)
+ */
+
+#define BI_MAC_MODEL 0x8000 /* Mac Gestalt ID (model type) */
+#define BI_MAC_VADDR 0x8001 /* Mac video base address */
+#define BI_MAC_VDEPTH 0x8002 /* Mac video depth */
+#define BI_MAC_VROW 0x8003 /* Mac video rowbytes */
+#define BI_MAC_VDIM 0x8004 /* Mac video dimensions */
+#define BI_MAC_VLOGICAL 0x8005 /* Mac video logical base */
+#define BI_MAC_SCCBASE 0x8006 /* Mac SCC base address */
+#define BI_MAC_BTIME 0x8007 /* Mac boot time */
+#define BI_MAC_GMTBIAS 0x8008 /* Mac GMT timezone offset */
+#define BI_MAC_MEMSIZE 0x8009 /* Mac RAM size (sanity check) */
+#define BI_MAC_CPUID 0x800a /* Mac CPU type (sanity check) */
+#define BI_MAC_ROMBASE 0x800b /* Mac system ROM base address */
+
+
+ /*
+ * Macintosh hardware profile data - unused, see macintosh.h for
+ * reasonable type values
+ */
+
+#define BI_MAC_VIA1BASE 0x8010 /* Mac VIA1 base address (always present) */
+#define BI_MAC_VIA2BASE 0x8011 /* Mac VIA2 base address (type varies) */
+#define BI_MAC_VIA2TYPE 0x8012 /* Mac VIA2 type (VIA, RBV, OSS) */
+#define BI_MAC_ADBTYPE 0x8013 /* Mac ADB interface type */
+#define BI_MAC_ASCBASE 0x8014 /* Mac Apple Sound Chip base address */
+#define BI_MAC_SCSI5380 0x8015 /* Mac NCR 5380 SCSI (base address, multi) */
+#define BI_MAC_SCSIDMA 0x8016 /* Mac SCSI DMA (base address) */
+#define BI_MAC_SCSI5396 0x8017 /* Mac NCR 53C96 SCSI (base address, multi) */
+#define BI_MAC_IDETYPE 0x8018 /* Mac IDE interface type */
+#define BI_MAC_IDEBASE 0x8019 /* Mac IDE interface base address */
+#define BI_MAC_NUBUS 0x801a /* Mac Nubus type (none, regular, pseudo) */
+#define BI_MAC_SLOTMASK 0x801b /* Mac Nubus slots present */
+#define BI_MAC_SCCTYPE 0x801c /* Mac SCC serial type (normal, IOP) */
+#define BI_MAC_ETHTYPE 0x801d /* Mac builtin ethernet type (Sonic, MACE */
+#define BI_MAC_ETHBASE 0x801e /* Mac builtin ethernet base address */
+#define BI_MAC_PMU 0x801f /* Mac power management / poweroff hardware */
+#define BI_MAC_IOP_SWIM 0x8020 /* Mac SWIM floppy IOP */
+#define BI_MAC_IOP_ADB 0x8021 /* Mac ADB IOP */
+
+
+ /*
+ * Macintosh Gestalt numbers (BI_MAC_MODEL)
+ */
+
+#define MAC_MODEL_II 6
+#define MAC_MODEL_IIX 7
+#define MAC_MODEL_IICX 8
+#define MAC_MODEL_SE30 9
+#define MAC_MODEL_IICI 11
+#define MAC_MODEL_IIFX 13 /* And well numbered it is too */
+#define MAC_MODEL_IISI 18
+#define MAC_MODEL_LC 19
+#define MAC_MODEL_Q900 20
+#define MAC_MODEL_PB170 21
+#define MAC_MODEL_Q700 22
+#define MAC_MODEL_CLII 23 /* aka: P200 */
+#define MAC_MODEL_PB140 25
+#define MAC_MODEL_Q950 26 /* aka: WGS95 */
+#define MAC_MODEL_LCIII 27 /* aka: P450 */
+#define MAC_MODEL_PB210 29
+#define MAC_MODEL_C650 30
+#define MAC_MODEL_PB230 32
+#define MAC_MODEL_PB180 33
+#define MAC_MODEL_PB160 34
+#define MAC_MODEL_Q800 35 /* aka: WGS80 */
+#define MAC_MODEL_Q650 36
+#define MAC_MODEL_LCII 37 /* aka: P400/405/410/430 */
+#define MAC_MODEL_PB250 38
+#define MAC_MODEL_IIVI 44
+#define MAC_MODEL_P600 45 /* aka: P600CD */
+#define MAC_MODEL_IIVX 48
+#define MAC_MODEL_CCL 49 /* aka: P250 */
+#define MAC_MODEL_PB165C 50
+#define MAC_MODEL_C610 52 /* aka: WGS60 */
+#define MAC_MODEL_Q610 53
+#define MAC_MODEL_PB145 54 /* aka: PB145B */
+#define MAC_MODEL_P520 56 /* aka: LC520 */
+#define MAC_MODEL_C660 60
+#define MAC_MODEL_P460 62 /* aka: LCIII+, P466/P467 */
+#define MAC_MODEL_PB180C 71
+#define MAC_MODEL_PB520 72 /* aka: PB520C, PB540, PB540C, PB550C */
+#define MAC_MODEL_PB270C 77
+#define MAC_MODEL_Q840 78
+#define MAC_MODEL_P550 80 /* aka: LC550, P560 */
+#define MAC_MODEL_CCLII 83 /* aka: P275 */
+#define MAC_MODEL_PB165 84
+#define MAC_MODEL_PB190 85 /* aka: PB190CS */
+#define MAC_MODEL_TV 88
+#define MAC_MODEL_P475 89 /* aka: LC475, P476 */
+#define MAC_MODEL_P475F 90 /* aka: P475 w/ FPU (no LC040) */
+#define MAC_MODEL_P575 92 /* aka: LC575, P577/P578 */
+#define MAC_MODEL_Q605 94
+#define MAC_MODEL_Q605_ACC 95 /* Q605 accelerated to 33 MHz */
+#define MAC_MODEL_Q630 98 /* aka: LC630, P630/631/635/636/637/638/640 */
+#define MAC_MODEL_P588 99 /* aka: LC580, P580 */
+#define MAC_MODEL_PB280 102
+#define MAC_MODEL_PB280C 103
+#define MAC_MODEL_PB150 115
+
+
+ /*
+ * Latest Macintosh bootinfo version
+ */
+
+#define MAC_BOOTI_VERSION MK_BI_VERSION(2, 0)
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_MAC_H */
--- /dev/null
+/*
+** asm/bootinfo-q40.h -- Q40-specific boot information definitions
+*/
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_Q40_H
+#define _UAPI_ASM_M68K_BOOTINFO_Q40_H
+
+
+ /*
+ * Latest Q40 bootinfo version
+ */
+
+#define Q40_BOOTI_VERSION MK_BI_VERSION(2, 0)
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_Q40_H */
--- /dev/null
+/*
+** asm/bootinfo-vme.h -- VME-specific boot information definitions
+*/
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_VME_H
+#define _UAPI_ASM_M68K_BOOTINFO_VME_H
+
+
+#include <linux/types.h>
+
+
+ /*
+ * VME-specific tags
+ */
+
+#define BI_VME_TYPE 0x8000 /* VME sub-architecture (__be32) */
+#define BI_VME_BRDINFO 0x8001 /* VME board information (struct) */
+
+
+ /*
+ * VME models (BI_VME_TYPE)
+ */
+
+#define VME_TYPE_TP34V 0x0034 /* Tadpole TP34V */
+#define VME_TYPE_MVME147 0x0147 /* Motorola MVME147 */
+#define VME_TYPE_MVME162 0x0162 /* Motorola MVME162 */
+#define VME_TYPE_MVME166 0x0166 /* Motorola MVME166 */
+#define VME_TYPE_MVME167 0x0167 /* Motorola MVME167 */
+#define VME_TYPE_MVME172 0x0172 /* Motorola MVME172 */
+#define VME_TYPE_MVME177 0x0177 /* Motorola MVME177 */
+#define VME_TYPE_BVME4000 0x4000 /* BVM Ltd. BVME4000 */
+#define VME_TYPE_BVME6000 0x6000 /* BVM Ltd. BVME6000 */
+
+
+#ifndef __ASSEMBLY__
+
+/*
+ * Board ID data structure - pointer to this retrieved from Bug by head.S
+ *
+ * BI_VME_BRDINFO is a 32 byte struct as returned by the Bug code on
+ * Motorola VME boards. Contains board number, Bug version, board
+ * configuration options, etc.
+ *
+ * Note, bytes 12 and 13 are board no in BCD (0162,0166,0167,0177,etc)
+ */
+
+typedef struct {
+ char bdid[4];
+ __u8 rev, mth, day, yr;
+ __be16 size, reserved;
+ __be16 brdno;
+ char brdsuffix[2];
+ __be32 options;
+ __be16 clun, dlun, ctype, dnum;
+ __be32 option2;
+} t_bdid, *p_bdid;
+
+#endif /* __ASSEMBLY__ */
+
+
+ /*
+ * Latest VME bootinfo versions
+ */
+
+#define MVME147_BOOTI_VERSION MK_BI_VERSION(2, 0)
+#define MVME16x_BOOTI_VERSION MK_BI_VERSION(2, 0)
+#define BVME6000_BOOTI_VERSION MK_BI_VERSION(2, 0)
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_VME_H */
--- /dev/null
+/*
+ * asm/bootinfo.h -- Definition of the Linux/m68k boot information structure
+ *
+ * Copyright 1992 by Greg Harp
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file COPYING in the main directory of this archive
+ * for more details.
+ */
+
+#ifndef _UAPI_ASM_M68K_BOOTINFO_H
+#define _UAPI_ASM_M68K_BOOTINFO_H
+
+
+#include <linux/types.h>
+
+
+#ifndef __ASSEMBLY__
+
+ /*
+ * Bootinfo definitions
+ *
+ * This is an easily parsable and extendable structure containing all
+ * information to be passed from the bootstrap to the kernel.
+ *
+ * This way I hope to keep all future changes back/forewards compatible.
+ * Thus, keep your fingers crossed...
+ *
+ * This structure is copied right after the kernel by the bootstrap
+ * routine.
+ */
+
+struct bi_record {
+ __be16 tag; /* tag ID */
+ __be16 size; /* size of record (in bytes) */
+ __be32 data[0]; /* data */
+};
+
+
+struct mem_info {
+ __be32 addr; /* physical address of memory chunk */
+ __be32 size; /* length of memory chunk (in bytes) */
+};
+
+#endif /* __ASSEMBLY__ */
+
+
+ /*
+ * Tag Definitions
+ *
+ * Machine independent tags start counting from 0x0000
+ * Machine dependent tags start counting from 0x8000
+ */
+
+#define BI_LAST 0x0000 /* last record (sentinel) */
+#define BI_MACHTYPE 0x0001 /* machine type (__be32) */
+#define BI_CPUTYPE 0x0002 /* cpu type (__be32) */
+#define BI_FPUTYPE 0x0003 /* fpu type (__be32) */
+#define BI_MMUTYPE 0x0004 /* mmu type (__be32) */
+#define BI_MEMCHUNK 0x0005 /* memory chunk address and size */
+ /* (struct mem_info) */
+#define BI_RAMDISK 0x0006 /* ramdisk address and size */
+ /* (struct mem_info) */
+#define BI_COMMAND_LINE 0x0007 /* kernel command line parameters */
+ /* (string) */
+
+
+ /*
+ * Linux/m68k Architectures (BI_MACHTYPE)
+ */
+
+#define MACH_AMIGA 1
+#define MACH_ATARI 2
+#define MACH_MAC 3
+#define MACH_APOLLO 4
+#define MACH_SUN3 5
+#define MACH_MVME147 6
+#define MACH_MVME16x 7
+#define MACH_BVME6000 8
+#define MACH_HP300 9
+#define MACH_Q40 10
+#define MACH_SUN3X 11
+#define MACH_M54XX 12
+
+
+ /*
+ * CPU, FPU and MMU types (BI_CPUTYPE, BI_FPUTYPE, BI_MMUTYPE)
+ *
+ * Note: we may rely on the following equalities:
+ *
+ * CPU_68020 == MMU_68851
+ * CPU_68030 == MMU_68030
+ * CPU_68040 == FPU_68040 == MMU_68040
+ * CPU_68060 == FPU_68060 == MMU_68060
+ */
+
+#define CPUB_68020 0
+#define CPUB_68030 1
+#define CPUB_68040 2
+#define CPUB_68060 3
+#define CPUB_COLDFIRE 4
+
+#define CPU_68020 (1 << CPUB_68020)
+#define CPU_68030 (1 << CPUB_68030)
+#define CPU_68040 (1 << CPUB_68040)
+#define CPU_68060 (1 << CPUB_68060)
+#define CPU_COLDFIRE (1 << CPUB_COLDFIRE)
+
+#define FPUB_68881 0
+#define FPUB_68882 1
+#define FPUB_68040 2 /* Internal FPU */
+#define FPUB_68060 3 /* Internal FPU */
+#define FPUB_SUNFPA 4 /* Sun-3 FPA */
+#define FPUB_COLDFIRE 5 /* ColdFire FPU */
+
+#define FPU_68881 (1 << FPUB_68881)
+#define FPU_68882 (1 << FPUB_68882)
+#define FPU_68040 (1 << FPUB_68040)
+#define FPU_68060 (1 << FPUB_68060)
+#define FPU_SUNFPA (1 << FPUB_SUNFPA)
+#define FPU_COLDFIRE (1 << FPUB_COLDFIRE)
+
+#define MMUB_68851 0
+#define MMUB_68030 1 /* Internal MMU */
+#define MMUB_68040 2 /* Internal MMU */
+#define MMUB_68060 3 /* Internal MMU */
+#define MMUB_APOLLO 4 /* Custom Apollo */
+#define MMUB_SUN3 5 /* Custom Sun-3 */
+#define MMUB_COLDFIRE 6 /* Internal MMU */
+
+#define MMU_68851 (1 << MMUB_68851)
+#define MMU_68030 (1 << MMUB_68030)
+#define MMU_68040 (1 << MMUB_68040)
+#define MMU_68060 (1 << MMUB_68060)
+#define MMU_SUN3 (1 << MMUB_SUN3)
+#define MMU_APOLLO (1 << MMUB_APOLLO)
+#define MMU_COLDFIRE (1 << MMUB_COLDFIRE)
+
+
+ /*
+ * Stuff for bootinfo interface versioning
+ *
+ * At the start of kernel code, a 'struct bootversion' is located.
+ * bootstrap checks for a matching version of the interface before booting
+ * a kernel, to avoid user confusion if kernel and bootstrap don't work
+ * together :-)
+ *
+ * If incompatible changes are made to the bootinfo interface, the major
+ * number below should be stepped (and the minor reset to 0) for the
+ * appropriate machine. If a change is backward-compatible, the minor
+ * should be stepped. "Backwards-compatible" means that booting will work,
+ * but certain features may not.
+ */
+
+#define BOOTINFOV_MAGIC 0x4249561A /* 'BIV^Z' */
+#define MK_BI_VERSION(major, minor) (((major) << 16) + (minor))
+#define BI_VERSION_MAJOR(v) (((v) >> 16) & 0xffff)
+#define BI_VERSION_MINOR(v) ((v) & 0xffff)
+
+#ifndef __ASSEMBLY__
+
+struct bootversion {
+ __be16 branch;
+ __be32 magic;
+ struct {
+ __be32 machtype;
+ __be32 version;
+ } machversions[0];
+} __packed;
+
+#endif /* __ASSEMBLY__ */
+
+
+#endif /* _UAPI_ASM_M68K_BOOTINFO_H */
** This file is subject to the terms and conditions of the GNU General Public
** License. See the file COPYING in the main directory of this archive
** for more details.
-**
-** Created 09/29/92 by Greg Harp
-**
-** 5/2/94 Roman Hodek:
-** Added bi_atari part of the machine dependent union bi_un; for now it
-** contains just a model field to distinguish between TT and Falcon.
-** 26/7/96 Roman Zippel:
-** Renamed to setup.h; added some useful macros to allow gcc some
-** optimizations if possible.
-** 5/10/96 Geert Uytterhoeven:
-** Redesign of the boot information structure; moved boot information
-** structure to bootinfo.h
*/
#ifndef _UAPI_M68K_SETUP_H
#define _UAPI_M68K_SETUP_H
-
-
- /*
- * Linux/m68k Architectures
- */
-
-#define MACH_AMIGA 1
-#define MACH_ATARI 2
-#define MACH_MAC 3
-#define MACH_APOLLO 4
-#define MACH_SUN3 5
-#define MACH_MVME147 6
-#define MACH_MVME16x 7
-#define MACH_BVME6000 8
-#define MACH_HP300 9
-#define MACH_Q40 10
-#define MACH_SUN3X 11
-#define MACH_M54XX 12
-
#define COMMAND_LINE_SIZE 256
-
-
- /*
- * CPU, FPU and MMU types
- *
- * Note: we may rely on the following equalities:
- *
- * CPU_68020 == MMU_68851
- * CPU_68030 == MMU_68030
- * CPU_68040 == FPU_68040 == MMU_68040
- * CPU_68060 == FPU_68060 == MMU_68060
- */
-
-#define CPUB_68020 0
-#define CPUB_68030 1
-#define CPUB_68040 2
-#define CPUB_68060 3
-#define CPUB_COLDFIRE 4
-
-#define CPU_68020 (1<<CPUB_68020)
-#define CPU_68030 (1<<CPUB_68030)
-#define CPU_68040 (1<<CPUB_68040)
-#define CPU_68060 (1<<CPUB_68060)
-#define CPU_COLDFIRE (1<<CPUB_COLDFIRE)
-
-#define FPUB_68881 0
-#define FPUB_68882 1
-#define FPUB_68040 2 /* Internal FPU */
-#define FPUB_68060 3 /* Internal FPU */
-#define FPUB_SUNFPA 4 /* Sun-3 FPA */
-#define FPUB_COLDFIRE 5 /* ColdFire FPU */
-
-#define FPU_68881 (1<<FPUB_68881)
-#define FPU_68882 (1<<FPUB_68882)
-#define FPU_68040 (1<<FPUB_68040)
-#define FPU_68060 (1<<FPUB_68060)
-#define FPU_SUNFPA (1<<FPUB_SUNFPA)
-#define FPU_COLDFIRE (1<<FPUB_COLDFIRE)
-
-#define MMUB_68851 0
-#define MMUB_68030 1 /* Internal MMU */
-#define MMUB_68040 2 /* Internal MMU */
-#define MMUB_68060 3 /* Internal MMU */
-#define MMUB_APOLLO 4 /* Custom Apollo */
-#define MMUB_SUN3 5 /* Custom Sun-3 */
-#define MMUB_COLDFIRE 6 /* Internal MMU */
-
-#define MMU_68851 (1<<MMUB_68851)
-#define MMU_68030 (1<<MMUB_68030)
-#define MMU_68040 (1<<MMUB_68040)
-#define MMU_68060 (1<<MMUB_68060)
-#define MMU_SUN3 (1<<MMUB_SUN3)
-#define MMU_APOLLO (1<<MMUB_APOLLO)
-#define MMU_COLDFIRE (1<<MMUB_COLDFIRE)
-
-
#endif /* _UAPI_M68K_SETUP_H */
obj-$(CONFIG_HAS_DMA) += dma.o
+obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o
+obj-$(CONFIG_BOOTINFO_PROC) += bootinfo_proc.o
+
DEFINE(CIABBASE, &ciab);
DEFINE(C_PRA, offsetof(struct CIA, pra));
DEFINE(ZTWOBASE, zTwoBase);
+
+ /* enum m68k_fixup_type */
+ DEFINE(M68K_FIXUP_MEMOFFSET, m68k_fixup_memoffset);
#endif
return 0;
--- /dev/null
+/*
+ * Based on arch/arm/kernel/atags_proc.c
+ */
+
+#include <linux/fs.h>
+#include <linux/init.h>
+#include <linux/printk.h>
+#include <linux/proc_fs.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include <asm/bootinfo.h>
+#include <asm/byteorder.h>
+
+
+static char bootinfo_tmp[1536] __initdata;
+
+static void *bootinfo_copy;
+static size_t bootinfo_size;
+
+static ssize_t bootinfo_read(struct file *file, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ return simple_read_from_buffer(buf, count, ppos, bootinfo_copy,
+ bootinfo_size);
+}
+
+static const struct file_operations bootinfo_fops = {
+ .read = bootinfo_read,
+ .llseek = default_llseek,
+};
+
+void __init save_bootinfo(const struct bi_record *bi)
+{
+ const void *start = bi;
+ size_t size = sizeof(bi->tag);
+
+ while (be16_to_cpu(bi->tag) != BI_LAST) {
+ uint16_t n = be16_to_cpu(bi->size);
+ size += n;
+ bi = (struct bi_record *)((unsigned long)bi + n);
+ }
+
+ if (size > sizeof(bootinfo_tmp)) {
+ pr_err("Cannot save %zu bytes of bootinfo\n", size);
+ return;
+ }
+
+ pr_info("Saving %zu bytes of bootinfo\n", size);
+ memcpy(bootinfo_tmp, start, size);
+ bootinfo_size = size;
+}
+
+static int __init init_bootinfo_procfs(void)
+{
+ /*
+ * This cannot go into save_bootinfo() because kmalloc and proc don't
+ * work yet when it is called.
+ */
+ struct proc_dir_entry *pde;
+
+ if (!bootinfo_size)
+ return -EINVAL;
+
+ bootinfo_copy = kmalloc(bootinfo_size, GFP_KERNEL);
+ if (!bootinfo_copy)
+ return -ENOMEM;
+
+ memcpy(bootinfo_copy, bootinfo_tmp, bootinfo_size);
+
+ pde = proc_create_data("bootinfo", 0400, NULL, &bootinfo_fops, NULL);
+ if (!pde) {
+ kfree(bootinfo_copy);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+arch_initcall(init_bootinfo_procfs);
** 98/04/25 Phil Blundell: added HP300 support
** 1998/08/30 David Kilzer: Added support for font_desc structures
** for linux-2.1.115
-** 9/02/11 Richard Zidlicky: added Q40 support (initial vesion 99/01/01)
+** 1999/02/11 Richard Zidlicky: added Q40 support (initial version 99/01/01)
** 2004/05/13 Kars de Jong: Finalised HP300 support
**
** This file is subject to the terms and conditions of the GNU General Public
#include <linux/linkage.h>
#include <linux/init.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-amiga.h>
+#include <asm/bootinfo-atari.h>
+#include <asm/bootinfo-hp300.h>
+#include <asm/bootinfo-mac.h>
+#include <asm/bootinfo-q40.h>
+#include <asm/bootinfo-vme.h>
#include <asm/setup.h>
#include <asm/entry.h>
#include <asm/pgtable.h>
/*
* Find a tag record in the bootinfo structure
- * The bootinfo structure is located right after the kernel bss
+ * The bootinfo structure is located right after the kernel
* Returns: d0: size (-1 if not found)
* a0: data pointer (end-of-records if not found)
*/
#if defined(MAC_USE_SCC_A) || defined(MAC_USE_SCC_B)
movel %pc@(L(mac_sccbase)),%a0
- /* Reset SCC device */
+ /* Reset SCC register pointer */
+ moveb %a0@(mac_scc_cha_a_ctrl_offset),%d0
+ /* Reset SCC device: write register pointer then register value */
moveb #9,%a0@(mac_scc_cha_a_ctrl_offset)
moveb #0xc0,%a0@(mac_scc_cha_a_ctrl_offset)
/* Wait for 5 PCLK cycles, which is about 68 CPU cycles */
#endif
#if defined(CONFIG_MAC)
-L(mac_booter_data):
- .long 0
L(mac_videobase):
.long 0
L(mac_videodepth):
--- /dev/null
+/*
+ * machine_kexec.c - handle transition of Linux booting another kernel
+ */
+#include <linux/compiler.h>
+#include <linux/kexec.h>
+#include <linux/mm.h>
+#include <linux/delay.h>
+
+#include <asm/cacheflush.h>
+#include <asm/page.h>
+#include <asm/setup.h>
+
+extern const unsigned char relocate_new_kernel[];
+extern const size_t relocate_new_kernel_size;
+
+int machine_kexec_prepare(struct kimage *kimage)
+{
+ return 0;
+}
+
+void machine_kexec_cleanup(struct kimage *kimage)
+{
+}
+
+void machine_shutdown(void)
+{
+}
+
+void machine_crash_shutdown(struct pt_regs *regs)
+{
+}
+
+typedef void (*relocate_kernel_t)(unsigned long ptr,
+ unsigned long start,
+ unsigned long cpu_mmu_flags) __noreturn;
+
+void machine_kexec(struct kimage *image)
+{
+ void *reboot_code_buffer;
+ unsigned long cpu_mmu_flags;
+
+ reboot_code_buffer = page_address(image->control_code_page);
+
+ memcpy(reboot_code_buffer, relocate_new_kernel,
+ relocate_new_kernel_size);
+
+ /*
+ * we do not want to be bothered.
+ */
+ local_irq_disable();
+
+ pr_info("Will call new kernel at 0x%08lx. Bye...\n", image->start);
+ __flush_cache_all();
+ cpu_mmu_flags = m68k_cputype | m68k_mmutype << 8;
+ ((relocate_kernel_t) reboot_code_buffer)(image->head & PAGE_MASK,
+ image->start,
+ cpu_mmu_flags);
+}
--- /dev/null
+#include <linux/linkage.h>
+
+#include <asm/asm-offsets.h>
+#include <asm/page.h>
+#include <asm/setup.h>
+
+
+#define MMU_BASE 8 /* MMU flags base in cpu_mmu_flags */
+
+.text
+
+ENTRY(relocate_new_kernel)
+ movel %sp@(4),%a0 /* a0 = ptr */
+ movel %sp@(8),%a1 /* a1 = start */
+ movel %sp@(12),%d1 /* d1 = cpu_mmu_flags */
+ movew #PAGE_MASK,%d2 /* d2 = PAGE_MASK */
+
+ /* Disable MMU */
+
+ btst #MMU_BASE + MMUB_68851,%d1
+ jeq 3f
+
+1: /* 68851 or 68030 */
+
+ lea %pc@(.Lcopy),%a4
+2: addl #0x00000000,%a4 /* virt_to_phys() */
+
+ .section ".m68k_fixup","aw"
+ .long M68K_FIXUP_MEMOFFSET, 2b+2
+ .previous
+
+ .chip 68030
+ pmove %tc,%d0 /* Disable MMU */
+ bclr #7,%d0
+ pmove %d0,%tc
+ jmp %a4@ /* Jump to physical .Lcopy */
+ .chip 68k
+
+3:
+ btst #MMU_BASE + MMUB_68030,%d1
+ jne 1b
+
+ btst #MMU_BASE + MMUB_68040,%d1
+ jeq 6f
+
+4: /* 68040 or 68060 */
+
+ lea %pc@(.Lcont040),%a4
+5: addl #0x00000000,%a4 /* virt_to_phys() */
+
+ .section ".m68k_fixup","aw"
+ .long M68K_FIXUP_MEMOFFSET, 5b+2
+ .previous
+
+ movel %a4,%d0
+ andl #0xff000000,%d0
+ orw #0xe020,%d0 /* Map 16 MiB, enable, cacheable */
+ .chip 68040
+ movec %d0,%itt0
+ movec %d0,%dtt0
+ .chip 68k
+ jmp %a4@ /* Jump to physical .Lcont040 */
+
+.Lcont040:
+ moveq #0,%d0
+ .chip 68040
+ movec %d0,%tc /* Disable MMU */
+ movec %d0,%itt0
+ movec %d0,%itt1
+ movec %d0,%dtt0
+ movec %d0,%dtt1
+ .chip 68k
+ jra .Lcopy
+
+6:
+ btst #MMU_BASE + MMUB_68060,%d1
+ jne 4b
+
+.Lcopy:
+ movel %a0@+,%d0 /* d0 = entry = *ptr */
+ jeq .Lflush
+
+ btst #2,%d0 /* entry & IND_DONE? */
+ jne .Lflush
+
+ btst #1,%d0 /* entry & IND_INDIRECTION? */
+ jeq 1f
+ andw %d2,%d0
+ movel %d0,%a0 /* ptr = entry & PAGE_MASK */
+ jra .Lcopy
+
+1:
+ btst #0,%d0 /* entry & IND_DESTINATION? */
+ jeq 2f
+ andw %d2,%d0
+ movel %d0,%a2 /* a2 = dst = entry & PAGE_MASK */
+ jra .Lcopy
+
+2:
+ btst #3,%d0 /* entry & IND_SOURCE? */
+ jeq .Lcopy
+
+ andw %d2,%d0
+ movel %d0,%a3 /* a3 = src = entry & PAGE_MASK */
+ movew #PAGE_SIZE/32 - 1,%d0 /* d0 = PAGE_SIZE/32 - 1 */
+3:
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ movel %a3@+,%a2@+ /* *dst++ = *src++ */
+ dbf %d0, 3b
+ jra .Lcopy
+
+.Lflush:
+ /* Flush all caches */
+
+ btst #CPUB_68020,%d1
+ jeq 2f
+
+1: /* 68020 or 68030 */
+ .chip 68030
+ movec %cacr,%d0
+ orw #0x808,%d0
+ movec %d0,%cacr
+ .chip 68k
+ jra .Lreincarnate
+
+2:
+ btst #CPUB_68030,%d1
+ jne 1b
+
+ btst #CPUB_68040,%d1
+ jeq 4f
+
+3: /* 68040 or 68060 */
+ .chip 68040
+ nop
+ cpusha %bc
+ nop
+ cinva %bc
+ nop
+ .chip 68k
+ jra .Lreincarnate
+
+4:
+ btst #CPUB_68060,%d1
+ jne 3b
+
+.Lreincarnate:
+ jmp %a1@
+
+relocate_new_kernel_end:
+
+ENTRY(relocate_new_kernel_size)
+ .long relocate_new_kernel_end - relocate_new_kernel
#include <linux/initrd.h>
#include <asm/bootinfo.h>
+#include <asm/byteorder.h>
#include <asm/sections.h>
#include <asm/setup.h>
#include <asm/fpu.h>
int m68k_realnum_memory;
EXPORT_SYMBOL(m68k_realnum_memory);
unsigned long m68k_memoffset;
-struct mem_info m68k_memory[NUM_MEMINFO];
+struct m68k_mem_info m68k_memory[NUM_MEMINFO];
EXPORT_SYMBOL(m68k_memory);
-struct mem_info m68k_ramdisk;
+static struct m68k_mem_info m68k_ramdisk __initdata;
-static char m68k_command_line[CL_SIZE];
+static char m68k_command_line[CL_SIZE] __initdata;
void (*mach_sched_init) (irq_handler_t handler) __initdata = NULL;
/* machine dependent irq functions */
static void __init m68k_parse_bootinfo(const struct bi_record *record)
{
- while (record->tag != BI_LAST) {
+ uint16_t tag;
+
+ save_bootinfo(record);
+
+ while ((tag = be16_to_cpu(record->tag)) != BI_LAST) {
int unknown = 0;
- const unsigned long *data = record->data;
+ const void *data = record->data;
+ uint16_t size = be16_to_cpu(record->size);
- switch (record->tag) {
+ switch (tag) {
case BI_MACHTYPE:
case BI_CPUTYPE:
case BI_FPUTYPE:
case BI_MEMCHUNK:
if (m68k_num_memory < NUM_MEMINFO) {
- m68k_memory[m68k_num_memory].addr = data[0];
- m68k_memory[m68k_num_memory].size = data[1];
+ const struct mem_info *m = data;
+ m68k_memory[m68k_num_memory].addr =
+ be32_to_cpu(m->addr);
+ m68k_memory[m68k_num_memory].size =
+ be32_to_cpu(m->size);
m68k_num_memory++;
} else
- printk("m68k_parse_bootinfo: too many memory chunks\n");
+ pr_warn("%s: too many memory chunks\n",
+ __func__);
break;
case BI_RAMDISK:
- m68k_ramdisk.addr = data[0];
- m68k_ramdisk.size = data[1];
+ {
+ const struct mem_info *m = data;
+ m68k_ramdisk.addr = be32_to_cpu(m->addr);
+ m68k_ramdisk.size = be32_to_cpu(m->size);
+ }
break;
case BI_COMMAND_LINE:
- strlcpy(m68k_command_line, (const char *)data,
+ strlcpy(m68k_command_line, data,
sizeof(m68k_command_line));
break;
unknown = 1;
}
if (unknown)
- printk("m68k_parse_bootinfo: unknown tag 0x%04x ignored\n",
- record->tag);
- record = (struct bi_record *)((unsigned long)record +
- record->size);
+ pr_warn("%s: unknown tag 0x%04x ignored\n", __func__,
+ tag);
+ record = (struct bi_record *)((unsigned long)record + size);
}
m68k_realnum_memory = m68k_num_memory;
#ifdef CONFIG_SINGLE_MEMORY_CHUNK
if (m68k_num_memory > 1) {
- printk("Ignoring last %i chunks of physical memory\n",
- (m68k_num_memory - 1));
+ pr_warn("%s: ignoring last %i chunks of physical memory\n",
+ __func__, (m68k_num_memory - 1));
m68k_num_memory = 1;
}
#endif
int i;
#endif
- /* The bootinfo is located right after the kernel bss */
+ /* The bootinfo is located right after the kernel */
if (!CPU_IS_COLDFIRE)
m68k_parse_bootinfo((const struct bi_record *)_end);
asm (".chip 68060; movec %%pcr,%0; .chip 68k"
: "=d" (pcr));
if (((pcr >> 8) & 0xff) <= 5) {
- printk("Enabling workaround for errata I14\n");
+ pr_warn("Enabling workaround for errata I14\n");
asm (".chip 68060; movec %0,%%pcr; .chip 68k"
: : "d" (pcr | 0x20));
}
panic("No configuration setup");
}
+ paging_init();
+
#ifdef CONFIG_NATFEAT
nf_init();
#endif
- paging_init();
-
#ifndef CONFIG_SUN3
for (i = 1; i < m68k_num_memory; i++)
free_bootmem_node(NODE_DATA(i), m68k_memory[i].addr,
BOOTMEM_DEFAULT);
initrd_start = (unsigned long)phys_to_virt(m68k_ramdisk.addr);
initrd_end = initrd_start + m68k_ramdisk.size;
- printk("initrd: %08lx - %08lx\n", initrd_start, initrd_end);
+ pr_info("initrd: %08lx - %08lx\n", initrd_start, initrd_end);
}
#endif
{
#ifndef CONFIG_M68KFPU_EMU
if (m68k_fputype == 0) {
- printk(KERN_EMERG "*** YOU DO NOT HAVE A FLOATING POINT UNIT, "
+ pr_emerg("*** YOU DO NOT HAVE A FLOATING POINT UNIT, "
"WHICH IS REQUIRED BY LINUX/M68K ***\n");
- printk(KERN_EMERG "Upgrade your hardware or join the FPU "
+ pr_emerg("Upgrade your hardware or join the FPU "
"emulation project\n");
panic("no FPU");
}
#include <linux/timex.h>
#include <linux/profile.h>
+
+unsigned long (*mach_random_get_entropy)(void);
+
+
/*
* timer_interrupt() needs to keep up the real-time clock,
* as well as call the "xtime_update()" routine every clocktick
{
unsigned long fslw = fp->un.fmt4.pc; /* is really FSLW for access error */
-#ifdef DEBUG
- printk("fslw=%#lx, fa=%#lx\n", fslw, fp->un.fmt4.effaddr);
-#endif
+ pr_debug("fslw=%#lx, fa=%#lx\n", fslw, fp->un.fmt4.effaddr);
if (fslw & MMU060_BPE) {
/* branch prediction error -> clear branch cache */
}
if (fslw & MMU060_W)
errorcode |= 2;
-#ifdef DEBUG
- printk("errorcode = %d\n", errorcode );
-#endif
+ pr_debug("errorcode = %ld\n", errorcode);
do_page_fault(&fp->ptregs, addr, errorcode);
} else if (fslw & (MMU060_SEE)){
/* Software Emulation Error.
send_fault_sig(&fp->ptregs);
} else if (!(fslw & (MMU060_RE|MMU060_WE)) ||
send_fault_sig(&fp->ptregs) > 0) {
- printk("pc=%#lx, fa=%#lx\n", fp->ptregs.pc, fp->un.fmt4.effaddr);
- printk( "68060 access error, fslw=%lx\n", fslw );
+ pr_err("pc=%#lx, fa=%#lx\n", fp->ptregs.pc,
+ fp->un.fmt4.effaddr);
+ pr_err("68060 access error, fslw=%lx\n", fslw);
trap_c( fp );
}
}
set_fs(old_fs);
-#ifdef DEBUG
- printk("do_040writeback1, res=%d\n",res);
-#endif
+ pr_debug("do_040writeback1, res=%d\n", res);
return res;
}
int res = 0;
#if 0
if (fp->un.fmt7.wb1s & WBV_040)
- printk("access_error040: cannot handle 1st writeback. oops.\n");
+ pr_err("access_error040: cannot handle 1st writeback. oops.\n");
#endif
if ((fp->un.fmt7.wb2s & WBV_040) &&
unsigned short ssw = fp->un.fmt7.ssw;
unsigned long mmusr;
-#ifdef DEBUG
- printk("ssw=%#x, fa=%#lx\n", ssw, fp->un.fmt7.faddr);
- printk("wb1s=%#x, wb2s=%#x, wb3s=%#x\n", fp->un.fmt7.wb1s,
+ pr_debug("ssw=%#x, fa=%#lx\n", ssw, fp->un.fmt7.faddr);
+ pr_debug("wb1s=%#x, wb2s=%#x, wb3s=%#x\n", fp->un.fmt7.wb1s,
fp->un.fmt7.wb2s, fp->un.fmt7.wb3s);
- printk ("wb2a=%lx, wb3a=%lx, wb2d=%lx, wb3d=%lx\n",
+ pr_debug("wb2a=%lx, wb3a=%lx, wb2d=%lx, wb3d=%lx\n",
fp->un.fmt7.wb2a, fp->un.fmt7.wb3a,
fp->un.fmt7.wb2d, fp->un.fmt7.wb3d);
-#endif
if (ssw & ATC_040) {
unsigned long addr = fp->un.fmt7.faddr;
/* MMU error, get the MMUSR info for this access */
mmusr = probe040(!(ssw & RW_040), addr, ssw);
-#ifdef DEBUG
- printk("mmusr = %lx\n", mmusr);
-#endif
+ pr_debug("mmusr = %lx\n", mmusr);
errorcode = 1;
if (!(mmusr & MMU_R_040)) {
/* clear the invalid atc entry */
errorcode |= 2;
if (do_page_fault(&fp->ptregs, addr, errorcode)) {
-#ifdef DEBUG
- printk("do_page_fault() !=0\n");
-#endif
+ pr_debug("do_page_fault() !=0\n");
if (user_mode(&fp->ptregs)){
/* delay writebacks after signal delivery */
-#ifdef DEBUG
- printk(".. was usermode - return\n");
-#endif
+ pr_debug(".. was usermode - return\n");
return;
}
/* disable writeback into user space from kernel
* the writeback won't do good)
*/
disable_wb:
-#ifdef DEBUG
- printk(".. disabling wb2\n");
-#endif
+ pr_debug(".. disabling wb2\n");
if (fp->un.fmt7.wb2a == fp->un.fmt7.faddr)
fp->un.fmt7.wb2s &= ~WBV_040;
if (fp->un.fmt7.wb3a == fp->un.fmt7.faddr)
current->thread.signo = SIGBUS;
current->thread.faddr = fp->un.fmt7.faddr;
if (send_fault_sig(&fp->ptregs) >= 0)
- printk("68040 bus error (ssw=%x, faddr=%lx)\n", ssw,
+ pr_err("68040 bus error (ssw=%x, faddr=%lx)\n", ssw,
fp->un.fmt7.faddr);
goto disable_wb;
}
unsigned short ssw = fp->un.fmtb.ssw;
extern unsigned long _sun3_map_test_start, _sun3_map_test_end;
-#ifdef DEBUG
if (ssw & (FC | FB))
- printk ("Instruction fault at %#010lx\n",
+ pr_debug("Instruction fault at %#010lx\n",
ssw & FC ?
fp->ptregs.format == 0xa ? fp->ptregs.pc + 2 : fp->un.fmtb.baddr - 2
:
fp->ptregs.format == 0xa ? fp->ptregs.pc + 4 : fp->un.fmtb.baddr);
if (ssw & DF)
- printk ("Data %s fault at %#010lx in %s (pc=%#lx)\n",
+ pr_debug("Data %s fault at %#010lx in %s (pc=%#lx)\n",
ssw & RW ? "read" : "write",
fp->un.fmtb.daddr,
space_names[ssw & DFC], fp->ptregs.pc);
-#endif
/*
* Check if this page should be demand-mapped. This needs to go before
return;
/* instruction fault or kernel data fault! */
if (ssw & (FC | FB))
- printk ("Instruction fault at %#010lx\n",
+ pr_err("Instruction fault at %#010lx\n",
fp->ptregs.pc);
if (ssw & DF) {
/* was this fault incurred testing bus mappings? */
return;
}
- printk ("Data %s fault at %#010lx in %s (pc=%#lx)\n",
+ pr_err("Data %s fault at %#010lx in %s (pc=%#lx)\n",
ssw & RW ? "read" : "write",
fp->un.fmtb.daddr,
space_names[ssw & DFC], fp->ptregs.pc);
}
- printk ("BAD KERNEL BUSERR\n");
+ pr_err("BAD KERNEL BUSERR\n");
die_if_kernel("Oops", &fp->ptregs,0);
force_sig(SIGKILL, current);
else if (buserr_type & SUN3_BUSERR_INVALID)
errorcode = 0x00;
else {
-#ifdef DEBUG
- printk ("*** unexpected busfault type=%#04x\n", buserr_type);
- printk ("invalid %s access at %#lx from pc %#lx\n",
- !(ssw & RW) ? "write" : "read", addr,
- fp->ptregs.pc);
-#endif
+ pr_debug("*** unexpected busfault type=%#04x\n",
+ buserr_type);
+ pr_debug("invalid %s access at %#lx from pc %#lx\n",
+ !(ssw & RW) ? "write" : "read", addr,
+ fp->ptregs.pc);
die_if_kernel ("Oops", &fp->ptregs, buserr_type);
force_sig (SIGBUS, current);
return;
if (!mmu_emu_handle_fault(addr, 1, 0))
do_page_fault (&fp->ptregs, addr, 0);
} else {
-#ifdef DEBUG
- printk ("protection fault on insn access (segv).\n");
-#endif
+ pr_debug("protection fault on insn access (segv).\n");
force_sig (SIGSEGV, current);
}
}
unsigned short ssw = fp->un.fmtb.ssw;
#ifdef DEBUG
unsigned long desc;
+#endif
- printk ("pid = %x ", current->pid);
- printk ("SSW=%#06x ", ssw);
+ pr_debug("pid = %x ", current->pid);
+ pr_debug("SSW=%#06x ", ssw);
if (ssw & (FC | FB))
- printk ("Instruction fault at %#010lx\n",
+ pr_debug("Instruction fault at %#010lx\n",
ssw & FC ?
fp->ptregs.format == 0xa ? fp->ptregs.pc + 2 : fp->un.fmtb.baddr - 2
:
fp->ptregs.format == 0xa ? fp->ptregs.pc + 4 : fp->un.fmtb.baddr);
if (ssw & DF)
- printk ("Data %s fault at %#010lx in %s (pc=%#lx)\n",
+ pr_debug("Data %s fault at %#010lx in %s (pc=%#lx)\n",
ssw & RW ? "read" : "write",
fp->un.fmtb.daddr,
space_names[ssw & DFC], fp->ptregs.pc);
-#endif
/* ++andreas: If a data fault and an instruction fault happen
at the same time map in both pages. */
"pmove %%psr,%1"
: "=a&" (desc), "=m" (temp)
: "a" (addr), "d" (ssw));
+ pr_debug("mmusr is %#x for addr %#lx in task %p\n",
+ temp, addr, current);
+ pr_debug("descriptor address is 0x%p, contents %#lx\n",
+ __va(desc), *(unsigned long *)__va(desc));
#else
asm volatile ("ptestr %2,%1@,#7\n\t"
"pmove %%psr,%0"
: "=m" (temp) : "a" (addr), "d" (ssw));
#endif
mmusr = temp;
-
-#ifdef DEBUG
- printk("mmusr is %#x for addr %#lx in task %p\n",
- mmusr, addr, current);
- printk("descriptor address is %#lx, contents %#lx\n",
- __va(desc), *(unsigned long *)__va(desc));
-#endif
-
errorcode = (mmusr & MMU_I) ? 0 : 1;
if (!(ssw & RW) || (ssw & RM))
errorcode |= 2;
if (mmusr & (MMU_I | MMU_WP)) {
if (ssw & 4) {
- printk("Data %s fault at %#010lx in %s (pc=%#lx)\n",
+ pr_err("Data %s fault at %#010lx in %s (pc=%#lx)\n",
ssw & RW ? "read" : "write",
fp->un.fmtb.daddr,
space_names[ssw & DFC], fp->ptregs.pc);
} else if (!(mmusr & MMU_I)) {
/* probably a 020 cas fault */
if (!(ssw & RM) && send_fault_sig(&fp->ptregs) > 0)
- printk("unexpected bus error (%#x,%#x)\n", ssw, mmusr);
+ pr_err("unexpected bus error (%#x,%#x)\n", ssw,
+ mmusr);
} else if (mmusr & (MMU_B|MMU_L|MMU_S)) {
- printk("invalid %s access at %#lx from pc %#lx\n",
+ pr_err("invalid %s access at %#lx from pc %#lx\n",
!(ssw & RW) ? "write" : "read", addr,
fp->ptregs.pc);
die_if_kernel("Oops",&fp->ptregs,mmusr);
static volatile long tlong;
#endif
- printk("weird %s access at %#lx from pc %#lx (ssw is %#x)\n",
+ pr_err("weird %s access at %#lx from pc %#lx (ssw is %#x)\n",
!(ssw & RW) ? "write" : "read", addr,
fp->ptregs.pc, ssw);
asm volatile ("ptestr #1,%1@,#0\n\t"
: "a" (addr));
mmusr = temp;
- printk ("level 0 mmusr is %#x\n", mmusr);
+ pr_err("level 0 mmusr is %#x\n", mmusr);
#if 0
asm volatile ("pmove %%tt0,%0"
: "=m" (tlong));
- printk("tt0 is %#lx, ", tlong);
+ pr_debug("tt0 is %#lx, ", tlong);
asm volatile ("pmove %%tt1,%0"
: "=m" (tlong));
- printk("tt1 is %#lx\n", tlong);
-#endif
-#ifdef DEBUG
- printk("Unknown SIGSEGV - 1\n");
+ pr_debug("tt1 is %#lx\n", tlong);
#endif
+ pr_debug("Unknown SIGSEGV - 1\n");
die_if_kernel("Oops",&fp->ptregs,mmusr);
force_sig(SIGSEGV, current);
return;
return;
if (fp->ptregs.sr & PS_S) {
- printk("Instruction fault at %#010lx\n",
- fp->ptregs.pc);
+ pr_err("Instruction fault at %#010lx\n", fp->ptregs.pc);
buserr:
- printk ("BAD KERNEL BUSERR\n");
+ pr_err("BAD KERNEL BUSERR\n");
die_if_kernel("Oops",&fp->ptregs,0);
force_sig(SIGKILL, current);
return;
"pmove %%psr,%1"
: "=a&" (desc), "=m" (temp)
: "a" (addr));
+ pr_debug("mmusr is %#x for addr %#lx in task %p\n",
+ temp, addr, current);
+ pr_debug("descriptor address is 0x%p, contents %#lx\n",
+ __va(desc), *(unsigned long *)__va(desc));
#else
asm volatile ("ptestr #1,%1@,#7\n\t"
"pmove %%psr,%0"
: "=m" (temp) : "a" (addr));
#endif
mmusr = temp;
-
-#ifdef DEBUG
- printk ("mmusr is %#x for addr %#lx in task %p\n",
- mmusr, addr, current);
- printk ("descriptor address is %#lx, contents %#lx\n",
- __va(desc), *(unsigned long *)__va(desc));
-#endif
-
if (mmusr & MMU_I)
do_page_fault (&fp->ptregs, addr, 0);
else if (mmusr & (MMU_B|MMU_L|MMU_S)) {
- printk ("invalid insn access at %#lx from pc %#lx\n",
+ pr_err("invalid insn access at %#lx from pc %#lx\n",
addr, fp->ptregs.pc);
-#ifdef DEBUG
- printk("Unknown SIGSEGV - 2\n");
-#endif
+ pr_debug("Unknown SIGSEGV - 2\n");
die_if_kernel("Oops",&fp->ptregs,mmusr);
force_sig(SIGSEGV, current);
return;
if (user_mode(&fp->ptregs))
current->thread.esp0 = (unsigned long) fp;
-#ifdef DEBUG
- printk ("*** Bus Error *** Format is %x\n", fp->ptregs.format);
-#endif
+ pr_debug("*** Bus Error *** Format is %x\n", fp->ptregs.format);
#if defined(CONFIG_COLDFIRE) && defined(CONFIG_MMU)
if (CPU_IS_COLDFIRE) {
#endif
default:
die_if_kernel("bad frame format",&fp->ptregs,0);
-#ifdef DEBUG
- printk("Unknown SIGSEGV - 4\n");
-#endif
+ pr_debug("Unknown SIGSEGV - 4\n");
force_sig(SIGSEGV, current);
}
}
unsigned long addr;
int i;
- printk("Call Trace:");
+ pr_info("Call Trace:");
addr = (unsigned long)stack + THREAD_SIZE - 1;
endstack = (unsigned long *)(addr & -THREAD_SIZE);
i = 0;
if (__kernel_text_address(addr)) {
#ifndef CONFIG_KALLSYMS
if (i % 5 == 0)
- printk("\n ");
+ pr_cont("\n ");
#endif
- printk(" [<%08lx>] %pS\n", addr, (void *)addr);
+ pr_cont(" [<%08lx>] %pS\n", addr, (void *)addr);
i++;
}
}
- printk("\n");
+ pr_cont("\n");
}
void show_registers(struct pt_regs *regs)
int i;
print_modules();
- printk("PC: [<%08lx>] %pS\n", regs->pc, (void *)regs->pc);
- printk("SR: %04x SP: %p a2: %08lx\n", regs->sr, regs, regs->a2);
- printk("d0: %08lx d1: %08lx d2: %08lx d3: %08lx\n",
+ pr_info("PC: [<%08lx>] %pS\n", regs->pc, (void *)regs->pc);
+ pr_info("SR: %04x SP: %p a2: %08lx\n", regs->sr, regs, regs->a2);
+ pr_info("d0: %08lx d1: %08lx d2: %08lx d3: %08lx\n",
regs->d0, regs->d1, regs->d2, regs->d3);
- printk("d4: %08lx d5: %08lx a0: %08lx a1: %08lx\n",
+ pr_info("d4: %08lx d5: %08lx a0: %08lx a1: %08lx\n",
regs->d4, regs->d5, regs->a0, regs->a1);
- printk("Process %s (pid: %d, task=%p)\n",
+ pr_info("Process %s (pid: %d, task=%p)\n",
current->comm, task_pid_nr(current), current);
addr = (unsigned long)&fp->un;
- printk("Frame format=%X ", regs->format);
+ pr_info("Frame format=%X ", regs->format);
switch (regs->format) {
case 0x2:
- printk("instr addr=%08lx\n", fp->un.fmt2.iaddr);
+ pr_cont("instr addr=%08lx\n", fp->un.fmt2.iaddr);
addr += sizeof(fp->un.fmt2);
break;
case 0x3:
- printk("eff addr=%08lx\n", fp->un.fmt3.effaddr);
+ pr_cont("eff addr=%08lx\n", fp->un.fmt3.effaddr);
addr += sizeof(fp->un.fmt3);
break;
case 0x4:
- printk((CPU_IS_060 ? "fault addr=%08lx fslw=%08lx\n"
- : "eff addr=%08lx pc=%08lx\n"),
- fp->un.fmt4.effaddr, fp->un.fmt4.pc);
+ if (CPU_IS_060)
+ pr_cont("fault addr=%08lx fslw=%08lx\n",
+ fp->un.fmt4.effaddr, fp->un.fmt4.pc);
+ else
+ pr_cont("eff addr=%08lx pc=%08lx\n",
+ fp->un.fmt4.effaddr, fp->un.fmt4.pc);
addr += sizeof(fp->un.fmt4);
break;
case 0x7:
- printk("eff addr=%08lx ssw=%04x faddr=%08lx\n",
+ pr_cont("eff addr=%08lx ssw=%04x faddr=%08lx\n",
fp->un.fmt7.effaddr, fp->un.fmt7.ssw, fp->un.fmt7.faddr);
- printk("wb 1 stat/addr/data: %04x %08lx %08lx\n",
+ pr_info("wb 1 stat/addr/data: %04x %08lx %08lx\n",
fp->un.fmt7.wb1s, fp->un.fmt7.wb1a, fp->un.fmt7.wb1dpd0);
- printk("wb 2 stat/addr/data: %04x %08lx %08lx\n",
+ pr_info("wb 2 stat/addr/data: %04x %08lx %08lx\n",
fp->un.fmt7.wb2s, fp->un.fmt7.wb2a, fp->un.fmt7.wb2d);
- printk("wb 3 stat/addr/data: %04x %08lx %08lx\n",
+ pr_info("wb 3 stat/addr/data: %04x %08lx %08lx\n",
fp->un.fmt7.wb3s, fp->un.fmt7.wb3a, fp->un.fmt7.wb3d);
- printk("push data: %08lx %08lx %08lx %08lx\n",
+ pr_info("push data: %08lx %08lx %08lx %08lx\n",
fp->un.fmt7.wb1dpd0, fp->un.fmt7.pd1, fp->un.fmt7.pd2,
fp->un.fmt7.pd3);
addr += sizeof(fp->un.fmt7);
break;
case 0x9:
- printk("instr addr=%08lx\n", fp->un.fmt9.iaddr);
+ pr_cont("instr addr=%08lx\n", fp->un.fmt9.iaddr);
addr += sizeof(fp->un.fmt9);
break;
case 0xa:
- printk("ssw=%04x isc=%04x isb=%04x daddr=%08lx dobuf=%08lx\n",
+ pr_cont("ssw=%04x isc=%04x isb=%04x daddr=%08lx dobuf=%08lx\n",
fp->un.fmta.ssw, fp->un.fmta.isc, fp->un.fmta.isb,
fp->un.fmta.daddr, fp->un.fmta.dobuf);
addr += sizeof(fp->un.fmta);
break;
case 0xb:
- printk("ssw=%04x isc=%04x isb=%04x daddr=%08lx dobuf=%08lx\n",
+ pr_cont("ssw=%04x isc=%04x isb=%04x daddr=%08lx dobuf=%08lx\n",
fp->un.fmtb.ssw, fp->un.fmtb.isc, fp->un.fmtb.isb,
fp->un.fmtb.daddr, fp->un.fmtb.dobuf);
- printk("baddr=%08lx dibuf=%08lx ver=%x\n",
+ pr_info("baddr=%08lx dibuf=%08lx ver=%x\n",
fp->un.fmtb.baddr, fp->un.fmtb.dibuf, fp->un.fmtb.ver);
addr += sizeof(fp->un.fmtb);
break;
default:
- printk("\n");
+ pr_cont("\n");
}
show_stack(NULL, (unsigned long *)addr);
- printk("Code:");
+ pr_info("Code:");
set_fs(KERNEL_DS);
cp = (u16 *)regs->pc;
for (i = -8; i < 16; i++) {
if (get_user(c, cp + i) && i >= 0) {
- printk(" Bad PC value.");
+ pr_cont(" Bad PC value.");
break;
}
- printk(i ? " %04x" : " <%04x>", c);
+ if (i)
+ pr_cont(" %04x", c);
+ else
+ pr_cont(" <%04x>", c);
}
set_fs(old_fs);
- printk ("\n");
+ pr_cont("\n");
}
void show_stack(struct task_struct *task, unsigned long *stack)
}
endstack = (unsigned long *)(((unsigned long)stack + THREAD_SIZE - 1) & -THREAD_SIZE);
- printk("Stack from %08lx:", (unsigned long)stack);
+ pr_info("Stack from %08lx:", (unsigned long)stack);
p = stack;
for (i = 0; i < kstack_depth_to_print; i++) {
if (p + 1 > endstack)
break;
if (i % 8 == 0)
- printk("\n ");
- printk(" %08lx", *p++);
+ pr_cont("\n ");
+ pr_cont(" %08lx", *p++);
}
- printk("\n");
+ pr_cont("\n");
show_trace(stack);
}
console_verbose();
if (vector < ARRAY_SIZE(vec_names))
- printk ("*** %s *** FORMAT=%X\n",
+ pr_err("*** %s *** FORMAT=%X\n",
vec_names[vector],
fp->ptregs.format);
else
- printk ("*** Exception %d *** FORMAT=%X\n",
+ pr_err("*** Exception %d *** FORMAT=%X\n",
vector, fp->ptregs.format);
if (vector == VEC_ADDRERR && CPU_IS_020_OR_030) {
unsigned short ssw = fp->un.fmtb.ssw;
- printk ("SSW=%#06x ", ssw);
+ pr_err("SSW=%#06x ", ssw);
if (ssw & RC)
- printk ("Pipe stage C instruction fault at %#010lx\n",
+ pr_err("Pipe stage C instruction fault at %#010lx\n",
(fp->ptregs.format) == 0xA ?
fp->ptregs.pc + 2 : fp->un.fmtb.baddr - 2);
if (ssw & RB)
- printk ("Pipe stage B instruction fault at %#010lx\n",
+ pr_err("Pipe stage B instruction fault at %#010lx\n",
(fp->ptregs.format) == 0xA ?
fp->ptregs.pc + 4 : fp->un.fmtb.baddr);
if (ssw & DF)
- printk ("Data %s fault at %#010lx in %s (pc=%#lx)\n",
+ pr_err("Data %s fault at %#010lx in %s (pc=%#lx)\n",
ssw & RW ? "read" : "write",
fp->un.fmtb.daddr, space_names[ssw & DFC],
fp->ptregs.pc);
}
- printk ("Current process id is %d\n", task_pid_nr(current));
+ pr_err("Current process id is %d\n", task_pid_nr(current));
die_if_kernel("BAD KERNEL TRAP", &fp->ptregs, 0);
}
return;
console_verbose();
- printk("%s: %08x\n",str,nr);
+ pr_crit("%s: %08x\n", str, nr);
show_registers(fp);
add_taint(TAINT_DIE, LOCKDEP_NOW_UNRELIABLE);
do_exit(SIGSEGV);
#include <linux/adb.h>
#include <linux/cuda.h>
-#define BOOTINFO_COMPAT_1_0
#include <asm/setup.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-mac.h>
+#include <asm/byteorder.h>
#include <asm/io.h>
#include <asm/irq.h>
int __init mac_parse_bootinfo(const struct bi_record *record)
{
int unknown = 0;
- const u_long *data = record->data;
+ const void *data = record->data;
- switch (record->tag) {
+ switch (be16_to_cpu(record->tag)) {
case BI_MAC_MODEL:
- mac_bi_data.id = *data;
+ mac_bi_data.id = be32_to_cpup(data);
break;
case BI_MAC_VADDR:
- mac_bi_data.videoaddr = *data;
+ mac_bi_data.videoaddr = be32_to_cpup(data);
break;
case BI_MAC_VDEPTH:
- mac_bi_data.videodepth = *data;
+ mac_bi_data.videodepth = be32_to_cpup(data);
break;
case BI_MAC_VROW:
- mac_bi_data.videorow = *data;
+ mac_bi_data.videorow = be32_to_cpup(data);
break;
case BI_MAC_VDIM:
- mac_bi_data.dimensions = *data;
+ mac_bi_data.dimensions = be32_to_cpup(data);
break;
case BI_MAC_VLOGICAL:
- mac_bi_data.videological = VIDEOMEMBASE + (*data & ~VIDEOMEMMASK);
- mac_orig_videoaddr = *data;
+ mac_orig_videoaddr = be32_to_cpup(data);
+ mac_bi_data.videological =
+ VIDEOMEMBASE + (mac_orig_videoaddr & ~VIDEOMEMMASK);
break;
case BI_MAC_SCCBASE:
- mac_bi_data.sccbase = *data;
+ mac_bi_data.sccbase = be32_to_cpup(data);
break;
case BI_MAC_BTIME:
- mac_bi_data.boottime = *data;
+ mac_bi_data.boottime = be32_to_cpup(data);
break;
case BI_MAC_GMTBIAS:
- mac_bi_data.gmtbias = *data;
+ mac_bi_data.gmtbias = be32_to_cpup(data);
break;
case BI_MAC_MEMSIZE:
- mac_bi_data.memsize = *data;
+ mac_bi_data.memsize = be32_to_cpup(data);
break;
case BI_MAC_CPUID:
- mac_bi_data.cpuid = *data;
+ mac_bi_data.cpuid = be32_to_cpup(data);
break;
case BI_MAC_ROMBASE:
- mac_bi_data.rombase = *data;
+ mac_bi_data.rombase = be32_to_cpup(data);
break;
default:
unknown = 1;
#include <linux/init.h>
#include <linux/interrupt.h>
-#include <asm/bootinfo.h>
#include <asm/macintosh.h>
#include <asm/macints.h>
#include <asm/mac_iop.h>
/*#define DEBUG_IOP*/
-/* Set to non-zero if the IOPs are present. Set by iop_init() */
+/* Non-zero if the IOPs are present */
-int iop_scc_present,iop_ism_present;
+int iop_scc_present, iop_ism_present;
/* structure for tracking channel listeners */
#include <asm/mac_via.h>
#include <asm/mac_oss.h>
-#define BOOTINFO_COMPAT_1_0
-#include <asm/bootinfo.h>
#include <asm/machdep.h>
/* Offset between Unix time (1970-based) and Mac time (1904-based) */
#include <linux/init.h>
#include <linux/irq.h>
-#include <asm/bootinfo.h>
#include <asm/macintosh.h>
#include <asm/macints.h>
#include <asm/mac_via.h>
#include <linux/irq.h>
#include <asm/traps.h>
-#include <asm/bootinfo.h>
#include <asm/macintosh.h>
#include <asm/macints.h>
#include <asm/mac_psc.h>
* expanded to cover what I think are the other 7 channels.
*/
-static void psc_dma_die_die_die(void)
+static __init void psc_dma_die_die_die(void)
{
int i;
#include <linux/module.h>
#include <linux/irq.h>
-#include <asm/bootinfo.h>
#include <asm/macintosh.h>
#include <asm/macints.h>
#include <asm/mac_via.h>
siginfo.si_signo = current->thread.signo;
siginfo.si_code = current->thread.code;
siginfo.si_addr = (void *)current->thread.faddr;
-#ifdef DEBUG
- printk("send_fault_sig: %p,%d,%d\n", siginfo.si_addr, siginfo.si_signo, siginfo.si_code);
-#endif
+ pr_debug("send_fault_sig: %p,%d,%d\n", siginfo.si_addr,
+ siginfo.si_signo, siginfo.si_code);
if (user_mode(regs)) {
force_sig_info(siginfo.si_signo,
* terminate things with extreme prejudice.
*/
if ((unsigned long)siginfo.si_addr < PAGE_SIZE)
- printk(KERN_ALERT "Unable to handle kernel NULL pointer dereference");
+ pr_alert("Unable to handle kernel NULL pointer dereference");
else
- printk(KERN_ALERT "Unable to handle kernel access");
- printk(" at virtual address %p\n", siginfo.si_addr);
+ pr_alert("Unable to handle kernel access");
+ pr_cont(" at virtual address %p\n", siginfo.si_addr);
die_if_kernel("Oops", regs, 0 /*error_code*/);
do_exit(SIGKILL);
}
int fault;
unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
-#ifdef DEBUG
- printk ("do page fault:\nregs->sr=%#x, regs->pc=%#lx, address=%#lx, %ld, %p\n",
- regs->sr, regs->pc, address, error_code,
- current->mm->pgd);
-#endif
+ pr_debug("do page fault:\nregs->sr=%#x, regs->pc=%#lx, address=%#lx, %ld, %p\n",
+ regs->sr, regs->pc, address, error_code, mm ? mm->pgd : NULL);
/*
* If we're in an interrupt or have no user
* we can handle it..
*/
good_area:
-#ifdef DEBUG
- printk("do_page_fault: good_area\n");
-#endif
+ pr_debug("do_page_fault: good_area\n");
switch (error_code & 3) {
default: /* 3: write, present */
/* fall through */
*/
fault = handle_mm_fault(mm, vma, address, flags);
-#ifdef DEBUG
- printk("handle_mm_fault returns %d\n",fault);
-#endif
+ pr_debug("handle_mm_fault returns %d\n", fault);
if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current))
return 0;
void __init m68k_setup_node(int node)
{
#ifndef CONFIG_SINGLE_MEMORY_CHUNK
- struct mem_info *info = m68k_memory + node;
+ struct m68k_mem_info *info = m68k_memory + node;
int i, end;
i = (unsigned long)phys_to_virt(info->addr) >> __virt_to_node_shift();
/*
* For 040/060 we can use the virtual memory area like other architectures,
- * but for 020/030 we want to use early termination page descriptor and we
+ * but for 020/030 we want to use early termination page descriptors and we
* can't mix this with normal page descriptors, so we have to copy that code
- * (mm/vmalloc.c) and return appriorate aligned addresses.
+ * (mm/vmalloc.c) and return appropriately aligned addresses.
*/
#ifdef CPU_M68040_OR_M68060_ONLY
EXPORT_SYMBOL(__ioremap);
/*
- * Unmap a ioremap()ed region again
+ * Unmap an ioremap()ed region again
*/
void iounmap(void __iomem *addr)
{
/*
* __iounmap unmaps nearly everything, so be careful
- * it doesn't free currently pointer/page tables anymore but it
- * wans't used anyway and might be added later.
+ * Currently it doesn't free pointer/page tables anymore but this
+ * wasn't used anyway and might be added later.
*/
void __iounmap(void *addr, unsigned long size)
{
printk("Fix your bootloader or use a memfile to make use of this area!\n");
m68k_num_memory--;
memmove(m68k_memory + i, m68k_memory + i + 1,
- (m68k_num_memory - i) * sizeof(struct mem_info));
+ (m68k_num_memory - i) * sizeof(struct m68k_mem_info));
continue;
}
addr = m68k_memory[i].addr + m68k_memory[i].size;
#include <linux/interrupt.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-vme.h>
+#include <asm/byteorder.h>
#include <asm/pgtable.h>
#include <asm/setup.h>
#include <asm/irq.h>
irq_handler_t tick_handler;
-int mvme147_parse_bootinfo(const struct bi_record *bi)
+int __init mvme147_parse_bootinfo(const struct bi_record *bi)
{
- if (bi->tag == BI_VME_TYPE || bi->tag == BI_VME_BRDINFO)
+ uint16_t tag = be16_to_cpu(bi->tag);
+ if (tag == BI_VME_TYPE || tag == BI_VME_BRDINFO)
return 0;
else
return 1;
#include <linux/module.h>
#include <asm/bootinfo.h>
+#include <asm/bootinfo-vme.h>
+#include <asm/byteorder.h>
#include <asm/pgtable.h>
#include <asm/setup.h>
#include <asm/irq.h>
EXPORT_SYMBOL(mvme16x_config);
-int mvme16x_parse_bootinfo(const struct bi_record *bi)
+int __init mvme16x_parse_bootinfo(const struct bi_record *bi)
{
- if (bi->tag == BI_VME_TYPE || bi->tag == BI_VME_BRDINFO)
+ uint16_t tag = be16_to_cpu(bi->tag);
+ if (tag == BI_VME_TYPE || tag == BI_VME_BRDINFO)
return 0;
else
return 1;
suf[3] = '\0';
suf[0] = suf[1] ? '-' : '\0';
- sprintf(model, "Motorola MVME%x%s", p->brdno, suf);
+ sprintf(model, "Motorola MVME%x%s", be16_to_cpu(p->brdno), suf);
}
static void mvme16x_get_hardware_list(struct seq_file *m)
{
- p_bdid p = &mvme_bdid;
+ uint16_t brdno = be16_to_cpu(mvme_bdid.brdno);
- if (p->brdno == 0x0162 || p->brdno == 0x0172)
+ if (brdno == 0x0162 || brdno == 0x0172)
{
unsigned char rev = *(unsigned char *)MVME162_VERSION_REG;
{
p_bdid p = &mvme_bdid;
char id[40];
+ uint16_t brdno = be16_to_cpu(p->brdno);
mach_max_dma_address = 0xffffffff;
mach_sched_init = mvme16x_sched_init;
}
/* Board type is only set by newer versions of vmelilo/tftplilo */
if (vme_brdtype == 0)
- vme_brdtype = p->brdno;
+ vme_brdtype = brdno;
mvme16x_get_model(id);
printk ("\nBRD_ID: %s BUG %x.%x %02x/%02x/%02x\n", id, p->rev>>4,
p->rev&0xf, p->yr, p->mth, p->day);
- if (p->brdno == 0x0162 || p->brdno == 0x172)
+ if (brdno == 0x0162 || brdno == 0x172)
{
unsigned char rev = *(unsigned char *)MVME162_VERSION_REG;
mvme16x_config = rev | MVME16x_CONFIG_GOT_SCCA;
- printk ("MVME%x Hardware status:\n", p->brdno);
+ printk ("MVME%x Hardware status:\n", brdno);
printk (" CPU Type 68%s040\n",
rev & MVME16x_CONFIG_GOT_FPU ? "" : "LC");
printk (" CPU clock %dMHz\n",
static irqreturn_t mvme16x_abort_int (int irq, void *dev_id)
{
- p_bdid p = &mvme_bdid;
unsigned long *new = (unsigned long *)vectors;
unsigned long *old = (unsigned long *)0xffe00000;
volatile unsigned char uc, *ucp;
+ uint16_t brdno = be16_to_cpu(mvme_bdid.brdno);
- if (p->brdno == 0x0162 || p->brdno == 0x172)
+ if (brdno == 0x0162 || brdno == 0x172)
{
ucp = (volatile unsigned char *)0xfff42043;
uc = *ucp | 8;
*(new+9) = *(old+9); /* Trace */
*(new+47) = *(old+47); /* Trap #15 */
- if (p->brdno == 0x0162 || p->brdno == 0x172)
+ if (brdno == 0x0162 || brdno == 0x172)
*(new+0x5e) = *(old+0x5e); /* ABORT switch */
else
*(new+0x6e) = *(old+0x6e); /* ABORT switch */
void mvme16x_sched_init (irq_handler_t timer_routine)
{
- p_bdid p = &mvme_bdid;
+ uint16_t brdno = be16_to_cpu(mvme_bdid.brdno);
int irq;
tick_handler = timer_routine;
"timer", mvme16x_timer_int))
panic ("Couldn't register timer int");
- if (p->brdno == 0x0162 || p->brdno == 0x172)
+ if (brdno == 0x0162 || brdno == 0x172)
irq = MVME162_IRQ_ABORT;
else
irq = MVME167_IRQ_ABORT;
0x3f8,0x2f8,0x3e8,0x2e8,0
};
-static void q40_disable_irqs(void)
+static void __init q40_disable_irqs(void)
{
unsigned i, j;
}
-int q40_parse_bootinfo(const struct bi_record *rec)
+int __init q40_parse_bootinfo(const struct bi_record *rec)
{
return 1;
}
*
*/
+#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/bootmem.h>
}
-void sun3_dvma_init(void)
+void __init sun3_dvma_init(void)
{
-
memset(ptelist, 0, sizeof(ptelist));
-
-
}
** Started 1/16/98 @ 2:22 am
*/
+#include <linux/init.h>
#include <linux/mman.h>
#include <linux/mm.h>
#include <linux/kernel.h>
/*
* Initialise the MMU emulator.
*/
-void mmu_emu_init(unsigned long bootmem_end)
+void __init mmu_emu_init(unsigned long bootmem_end)
{
unsigned long seg, num;
int i,j;
* Contains common routines for sun3/sun3x DVMA management.
*/
+#include <linux/bootmem.h>
+#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/gfp.h>
extern void sun3_dvma_init(void);
#endif
-static unsigned long iommu_use[IOMMU_TOTAL_ENTRIES];
+static unsigned long *iommu_use;
#define dvma_index(baddr) ((baddr - DVMA_START) >> DVMA_PAGE_SHIFT)
}
-void dvma_init(void)
+void __init dvma_init(void)
{
struct hole *hole;
list_add(&(hole->list), &hole_list);
- memset(iommu_use, 0, sizeof(iommu_use));
+ iommu_use = alloc_bootmem(IOMMU_TOTAL_ENTRIES * sizeof(unsigned long));
dvma_unmap_iommu(DVMA_START, DVMA_SIZE);
#include <asm/page.h>
#include <asm/pgtable.h>
-#include <asm/bootinfo.h>
#include <asm/setup.h>
#include <asm/traps.h>
#include <asm/sun3xprom.h>
#define smp_read_barrier_depends() do { } while (0)
#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_METAG_BARRIER_H */
enum ipi_msg_type {
IPI_CALL_FUNC,
- IPI_CALL_FUNC_SINGLE,
IPI_RESCHEDULE,
};
extern void arch_send_call_function_single_ipi(int cpu);
extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
-#define arch_send_call_function_ipi_mask arch_send_call_function_ipi_mask
asmlinkage void secondary_start_kernel(void);
pgd = pgd_offset(&init_mm, CONSISTENT_START);
pud = pud_alloc(&init_mm, pgd, CONSISTENT_START);
pmd = pmd_alloc(&init_mm, pud, CONSISTENT_START);
- if (!pmd) {
- pr_err("%s: no pmd tables\n", __func__);
- ret = -ENOMEM;
- break;
- }
WARN_ON(!pmd_none(*pmd));
pte = pte_alloc_kernel(pmd, CONSISTENT_START);
/*
* "thread" is assumed to be a valid Meta hardware thread ID.
*/
-int boot_secondary(unsigned int thread, struct task_struct *idle)
+static int boot_secondary(unsigned int thread, struct task_struct *idle)
{
u32 val;
void arch_send_call_function_single_ipi(int cpu)
{
- send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC_SINGLE);
+ send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC);
}
void show_ipi_list(struct seq_file *p)
*
* Bit 0 - Inter-processor function call
*/
-static int do_IPI(struct pt_regs *regs)
+static int do_IPI(void)
{
unsigned int cpu = smp_processor_id();
struct ipi_data *ipi = &per_cpu(ipi_data, cpu);
- struct pt_regs *old_regs = set_irq_regs(regs);
unsigned long msgs, nextmsg;
int handled = 0;
generic_smp_call_function_interrupt();
break;
- case IPI_CALL_FUNC_SINGLE:
- generic_smp_call_function_single_interrupt();
- break;
-
default:
pr_crit("CPU%u: Unknown IPI message 0x%lx\n",
cpu, nextmsg);
}
}
- set_irq_regs(old_regs);
-
return handled;
}
static TBIRES ipi_handler(TBIRES State, int SigNum, int Triggers,
int Inst, PTBI pTBI, int *handled)
{
- *handled = do_IPI((struct pt_regs *)State.Sig.pCtx);
+ *handled = do_IPI();
return State;
}
DEFINE_PER_CPU(struct cpuinfo_metag, cpu_data);
cpumask_t cpu_core_map[NR_CPUS];
+EXPORT_SYMBOL(cpu_core_map);
static cpumask_t cpu_coregroup_map(unsigned int cpu)
{
+generic-y += barrier.h
generic-y += clkdev.h
generic-y += exec.h
generic-y += trace_clock.h
+++ /dev/null
-/*
- * Copyright (C) 2006 Atmark Techno, Inc.
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- */
-
-#ifndef _ASM_MICROBLAZE_BARRIER_H
-#define _ASM_MICROBLAZE_BARRIER_H
-
-#define nop() asm volatile ("nop")
-
-#define smp_read_barrier_depends() do {} while (0)
-#define read_barrier_depends() do {} while (0)
-
-#define mb() barrier()
-#define rmb() mb()
-#define wmb() mb()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#define set_wmb(var, value) do { var = value; wmb(); } while (0)
-
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-
-#endif /* _ASM_MICROBLAZE_BARRIER_H */
select MODULES_USE_ELF_RELA if MODULES && 64BIT
select CLONE_BACKWARDS
select HAVE_DEBUG_STACKOVERFLOW
+ select HAVE_CC_STACKPROTECTOR
menu "Machine selection"
If unsure, say Y. Only embedded should say N here.
-config CC_STACKPROTECTOR
- bool "Enable -fstack-protector buffer overflow detection (EXPERIMENTAL)"
- help
- This option turns on the -fstack-protector GCC feature. This
- feature puts, at the beginning of functions, a canary value on
- the stack just before the return address, and validates
- the value just before actually returning. Stack based buffer
- overflows (that need to overwrite this return address) now also
- overwrite the canary, which gets detected and the attack is then
- neutralized via a kernel panic.
-
- This feature requires gcc version 4.2 or above.
-
config USE_OF
bool
select OF
LDFLAGS += -m $(ld-emul)
-ifdef CONFIG_CC_STACKPROTECTOR
- KBUILD_CFLAGS += -fstack-protector
-endif
-
ifdef CONFIG_MIPS
CHECKFLAGS += $(shell $(CC) $(KBUILD_CFLAGS) -dM -E -x c /dev/null | \
egrep -vw '__GNUC_(|MINOR_|PATCHLEVEL_)_' | \
_machine_restart = ar7_machine_restart;
_machine_halt = ar7_machine_halt;
pm_power_off = ar7_machine_power_off;
- panic_timeout = 3;
io_base = (unsigned long)ioremap(AR7_REGS_BASE, 0x10000);
if (!io_base)
iomem_resource.start = EMMA2RH_IO_BASE;
iomem_resource.end = EMMA2RH_ROM_BASE - 1;
- /* Reboot on panic */
- panic_timeout = 180;
-
markeins_sio_setup();
}
#define nudge_writes() mb()
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* __ASM_BARRIER_H */
/*
* Loongson2-specific cacheops
*/
-#define Hit_Invalidate_I_Loongson23 0x00
+#define Hit_Invalidate_I_Loongson2 0x00
#endif /* __ASM_CACHEOPS_H */
__iflush_prologue
switch (boot_cpu_type()) {
case CPU_LOONGSON2:
- cache_op(Hit_Invalidate_I_Loongson23, addr);
+ cache_op(Hit_Invalidate_I_Loongson2, addr);
break;
default:
{
switch (boot_cpu_type()) {
case CPU_LOONGSON2:
- protected_cache_op(Hit_Invalidate_I_Loongson23, addr);
+ protected_cache_op(Hit_Invalidate_I_Loongson2, addr);
break;
default:
"i" (op));
/* build blast_xxx, blast_xxx_page, blast_xxx_page_indexed */
-#define __BUILD_BLAST_CACHE(pfx, desc, indexop, hitop, lsize) \
-static inline void blast_##pfx##cache##lsize(void) \
+#define __BUILD_BLAST_CACHE(pfx, desc, indexop, hitop, lsize, extra) \
+static inline void extra##blast_##pfx##cache##lsize(void) \
{ \
unsigned long start = INDEX_BASE; \
unsigned long end = start + current_cpu_data.desc.waysize; \
__##pfx##flush_epilogue \
} \
\
-static inline void blast_##pfx##cache##lsize##_page(unsigned long page) \
+static inline void extra##blast_##pfx##cache##lsize##_page(unsigned long page) \
{ \
unsigned long start = page; \
unsigned long end = page + PAGE_SIZE; \
__##pfx##flush_epilogue \
} \
\
-static inline void blast_##pfx##cache##lsize##_page_indexed(unsigned long page) \
+static inline void extra##blast_##pfx##cache##lsize##_page_indexed(unsigned long page) \
{ \
unsigned long indexmask = current_cpu_data.desc.waysize - 1; \
unsigned long start = INDEX_BASE + (page & indexmask); \
__##pfx##flush_epilogue \
}
-__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 16)
-__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 16)
-__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 16)
-__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 32)
-__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 32)
-__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 32)
-__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 64)
-__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 64)
-__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 64)
-__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 128)
-
-__BUILD_BLAST_CACHE(inv_d, dcache, Index_Writeback_Inv_D, Hit_Invalidate_D, 16)
-__BUILD_BLAST_CACHE(inv_d, dcache, Index_Writeback_Inv_D, Hit_Invalidate_D, 32)
-__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 16)
-__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 32)
-__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 64)
-__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 128)
+__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 16, )
+__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 16, )
+__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 16, )
+__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 32, )
+__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 32, )
+__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I_Loongson2, 32, loongson2_)
+__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 32, )
+__BUILD_BLAST_CACHE(d, dcache, Index_Writeback_Inv_D, Hit_Writeback_Inv_D, 64, )
+__BUILD_BLAST_CACHE(i, icache, Index_Invalidate_I, Hit_Invalidate_I, 64, )
+__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 64, )
+__BUILD_BLAST_CACHE(s, scache, Index_Writeback_Inv_SD, Hit_Writeback_Inv_SD, 128, )
+
+__BUILD_BLAST_CACHE(inv_d, dcache, Index_Writeback_Inv_D, Hit_Invalidate_D, 16, )
+__BUILD_BLAST_CACHE(inv_d, dcache, Index_Writeback_Inv_D, Hit_Invalidate_D, 32, )
+__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 16, )
+__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 32, )
+__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 64, )
+__BUILD_BLAST_CACHE(inv_s, scache, Index_Writeback_Inv_SD, Hit_Invalidate_SD, 128, )
/* build blast_xxx_range, protected_blast_xxx_range */
#define __BUILD_BLAST_CACHE_RANGE(pfx, desc, hitop, prot, extra) \
__BUILD_BLAST_CACHE_RANGE(d, dcache, Hit_Writeback_Inv_D, protected_, )
__BUILD_BLAST_CACHE_RANGE(s, scache, Hit_Writeback_Inv_SD, protected_, )
__BUILD_BLAST_CACHE_RANGE(i, icache, Hit_Invalidate_I, protected_, )
-__BUILD_BLAST_CACHE_RANGE(i, icache, Hit_Invalidate_I_Loongson23, \
- protected_, loongson23_)
+__BUILD_BLAST_CACHE_RANGE(i, icache, Hit_Invalidate_I_Loongson2, \
+ protected_, loongson2_)
__BUILD_BLAST_CACHE_RANGE(d, dcache, Hit_Writeback_Inv_D, , )
__BUILD_BLAST_CACHE_RANGE(s, scache, Hit_Writeback_Inv_SD, , )
/* blast_inv_dcache_range */
r4k_blast_icache_page = (void *)cache_noop;
else if (ic_lsize == 16)
r4k_blast_icache_page = blast_icache16_page;
+ else if (ic_lsize == 32 && current_cpu_type() == CPU_LOONGSON2)
+ r4k_blast_icache_page = loongson2_blast_icache32_page;
else if (ic_lsize == 32)
r4k_blast_icache_page = blast_icache32_page;
else if (ic_lsize == 64)
else if (TX49XX_ICACHE_INDEX_INV_WAR)
r4k_blast_icache_page_indexed =
tx49_blast_icache32_page_indexed;
+ else if (current_cpu_type() == CPU_LOONGSON2)
+ r4k_blast_icache_page_indexed =
+ loongson2_blast_icache32_page_indexed;
else
r4k_blast_icache_page_indexed =
blast_icache32_page_indexed;
r4k_blast_icache = blast_r4600_v1_icache32;
else if (TX49XX_ICACHE_INDEX_INV_WAR)
r4k_blast_icache = tx49_blast_icache32;
+ else if (current_cpu_type() == CPU_LOONGSON2)
+ r4k_blast_icache = loongson2_blast_icache32;
else
r4k_blast_icache = blast_icache32;
} else if (ic_lsize == 64)
else {
switch (boot_cpu_type()) {
case CPU_LOONGSON2:
- protected_blast_icache_range(start, end);
+ protected_loongson2_blast_icache_range(start, end);
break;
default:
- protected_loongson23_blast_icache_range(start, end);
+ protected_blast_icache_range(start, end);
break;
}
}
void __init plat_mem_setup(void)
{
- panic_timeout = 5;
_machine_restart = (void (*)(char *))nlm_linux_exit;
_machine_halt = nlm_linux_exit;
pm_power_off = nlm_linux_exit;
void __init plat_mem_setup(void)
{
- panic_timeout = 5;
_machine_restart = (void (*)(char *))nlm_linux_exit;
_machine_halt = nlm_linux_exit;
pm_power_off = nlm_linux_exit;
#error invalid SiByte board configuration
#endif
- panic_timeout = 5; /* For debug. */
-
board_be_handler = swarm_be_handler;
if (xicor_probe())
+generic-y += barrier.h
generic-y += clkdev.h
generic-y += exec.h
generic-y += trace_clock.h
+++ /dev/null
-/* MN10300 memory barrier definitions
- *
- * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public Licence
- * as published by the Free Software Foundation; either version
- * 2 of the Licence, or (at your option) any later version.
- */
-#ifndef _ASM_BARRIER_H
-#define _ASM_BARRIER_H
-
-#define nop() asm volatile ("nop")
-
-#define mb() asm volatile ("": : :"memory")
-#define rmb() mb()
-#define wmb() asm volatile ("": : :"memory")
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define set_mb(var, value) do { xchg(&var, value); } while (0)
-#else /* CONFIG_SMP */
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#endif /* CONFIG_SMP */
-
-#define set_wmb(var, value) do { var = value; wmb(); } while (0)
-
-#define read_barrier_depends() do {} while (0)
-#define smp_read_barrier_depends() do {} while (0)
-
-#endif /* _ASM_BARRIER_H */
+generic-y += barrier.h
generic-y += word-at-a-time.h auxvec.h user.h cputime.h emergency-restart.h \
segment.h topology.h vga.h device.h percpu.h hw_irq.h mutex.h \
div64.h irq_regs.h kdebug.h kvm_para.h local64.h local.h param.h \
+++ /dev/null
-#ifndef __PARISC_BARRIER_H
-#define __PARISC_BARRIER_H
-
-/*
-** This is simply the barrier() macro from linux/kernel.h but when serial.c
-** uses tqueue.h uses smp_mb() defined using barrier(), linux/kernel.h
-** hasn't yet been included yet so it fails, thus repeating the macro here.
-**
-** PA-RISC architecture allows for weakly ordered memory accesses although
-** none of the processors use it. There is a strong ordered bit that is
-** set in the O-bit of the page directory entry. Operating systems that
-** can not tolerate out of order accesses should set this bit when mapping
-** pages. The O-bit of the PSW should also be set to 1 (I don't believe any
-** of the processor implemented the PSW O-bit). The PCX-W ERS states that
-** the TLB O-bit is not implemented so the page directory does not need to
-** have the O-bit set when mapping pages (section 3.1). This section also
-** states that the PSW Y, Z, G, and O bits are not implemented.
-** So it looks like nothing needs to be done for parisc-linux (yet).
-** (thanks to chada for the above comment -ggg)
-**
-** The __asm__ op below simple prevents gcc/ld from reordering
-** instructions across the mb() "call".
-*/
-#define mb() __asm__ __volatile__("":::"memory") /* barrier() */
-#define rmb() mb()
-#define wmb() mb()
-#define smp_mb() mb()
-#define smp_rmb() mb()
-#define smp_wmb() mb()
-#define smp_read_barrier_depends() do { } while(0)
-#define read_barrier_depends() do { } while(0)
-
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-
-#endif /* __PARISC_BARRIER_H */
#define SO_BUSY_POLL 0x4027
-#define SO_MAX_PACING_RATE 0x4048
+#define SO_MAX_PACING_RATE 0x4028
#endif /* _UAPI_ASM_SOCKET_H */
bool
default y
+config PANIC_TIMEOUT
+ int
+ default 180
+
config COMPAT
bool
default y if PPC64
# define SMPWMB eieio
#endif
+#define __lwsync() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
+
#define smp_mb() mb()
-#define smp_rmb() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
+#define smp_rmb() __lwsync()
#define smp_wmb() __asm__ __volatile__ (stringify_in_c(SMPWMB) : : :"memory")
#define smp_read_barrier_depends() read_barrier_depends()
#else
+#define __lwsync() barrier()
+
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
#define data_barrier(x) \
asm volatile("twi 0,%0,0; isync" : : "r" (x) : "memory");
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ __lwsync(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ __lwsync(); \
+ ___p1; \
+})
+
#endif /* _ASM_POWERPC_BARRIER_H */
void check_for_initrd(void);
void do_init_bootmem(void);
void setup_panic(void);
+#define ARCH_PANIC_TIMEOUT 180
#endif /* !__ASSEMBLY__ */
#include <asm/synch.h>
#include <asm/ppc-opcode.h>
+#define smp_mb__after_unlock_lock() smp_mb() /* Full ordering for lock. */
+
#define arch_spin_is_locked(x) ((x)->slock != 0)
#ifdef CONFIG_PPC64
struct arch_uprobe {
union {
- u8 insn[MAX_UINSN_BYTES];
- u8 ixol[MAX_UINSN_BYTES];
- u32 ainsn;
+ u32 insn;
+ u32 ixol;
};
};
if (cpu_has_feature(CPU_FTR_UNIFIED_ID_CACHE))
ucache_bsize = icache_bsize = dcache_bsize;
- /* reboot on panic */
- panic_timeout = 180;
-
if (ppc_md.panic)
setup_panic();
dcache_bsize = ppc64_caches.dline_size;
icache_bsize = ppc64_caches.iline_size;
- /* reboot on panic */
- panic_timeout = 180;
-
if (ppc_md.panic)
setup_panic();
* emulate_step() returns 1 if the insn was successfully emulated.
* For all other cases, we need to single-step in hardware.
*/
- ret = emulate_step(regs, auprobe->ainsn);
+ ret = emulate_step(regs, auprobe->insn);
if (ret > 0)
return true;
}
PPC_DIVWU(r_A, r_A, r_X);
break;
- case BPF_S_ALU_DIV_K: /* A = reciprocal_divide(A, K); */
+ case BPF_S_ALU_DIV_K: /* A /= K */
+ if (K == 1)
+ break;
PPC_LI32(r_scratch1, K);
- /* Top 32 bits of 64bit result -> A */
- PPC_MULHWU(r_A, r_A, r_scratch1);
+ PPC_DIVWU(r_A, r_A, r_scratch1);
break;
case BPF_S_ALU_AND_X:
ctx->seen |= SEEN_XREG;
static void __init pSeries_setup_arch(void)
{
- panic_timeout = 10;
+ set_arch_panic_timeout(10, ARCH_PANIC_TIMEOUT);
/* Discover PIC type and setup ppc_md accordingly */
pseries_discover_pic();
#define set_mb(var, value) do { var = value; mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
#endif /* __ASM_BARRIER_H */
#define PSW32_USER_BITS (PSW32_MASK_DAT | PSW32_MASK_IO | PSW32_MASK_EXT | \
PSW32_DEFAULT_KEY | PSW32_MASK_BASE | \
- PSW32_MASK_MCHECK | PSW32_MASK_PSTATE | PSW32_ASC_HOME)
+ PSW32_MASK_MCHECK | PSW32_MASK_PSTATE | \
+ PSW32_ASC_PRIMARY)
#define COMPAT_USER_HZ 100
#define COMPAT_UTS_MACHINE "s390\0\0\0\0"
u32 reserved2[12];
} __packed;
+/* QUERY SAMPLING INFORMATION block */
+struct hws_qsi_info_block { /* Bit(s) */
+ unsigned int b0_13:14; /* 0-13: zeros */
+ unsigned int as:1; /* 14: basic-sampling authorization */
+ unsigned int ad:1; /* 15: diag-sampling authorization */
+ unsigned int b16_21:6; /* 16-21: zeros */
+ unsigned int es:1; /* 22: basic-sampling enable control */
+ unsigned int ed:1; /* 23: diag-sampling enable control */
+ unsigned int b24_29:6; /* 24-29: zeros */
+ unsigned int cs:1; /* 30: basic-sampling activation control */
+ unsigned int cd:1; /* 31: diag-sampling activation control */
+ unsigned int bsdes:16; /* 4-5: size of basic sampling entry */
+ unsigned int dsdes:16; /* 6-7: size of diagnostic sampling entry */
+ unsigned long min_sampl_rate; /* 8-15: minimum sampling interval */
+ unsigned long max_sampl_rate; /* 16-23: maximum sampling interval*/
+ unsigned long tear; /* 24-31: TEAR contents */
+ unsigned long dear; /* 32-39: DEAR contents */
+ unsigned int rsvrd0; /* 40-43: reserved */
+ unsigned int cpu_speed; /* 44-47: CPU speed */
+ unsigned long long rsvrd1; /* 48-55: reserved */
+ unsigned long long rsvrd2; /* 56-63: reserved */
+} __packed;
+
+/* SET SAMPLING CONTROLS request block */
+struct hws_lsctl_request_block {
+ unsigned int s:1; /* 0: maximum buffer indicator */
+ unsigned int h:1; /* 1: part. level reserved for VM use*/
+ unsigned long long b2_53:52;/* 2-53: zeros */
+ unsigned int es:1; /* 54: basic-sampling enable control */
+ unsigned int ed:1; /* 55: diag-sampling enable control */
+ unsigned int b56_61:6; /* 56-61: - zeros */
+ unsigned int cs:1; /* 62: basic-sampling activation control */
+ unsigned int cd:1; /* 63: diag-sampling activation control */
+ unsigned long interval; /* 8-15: sampling interval */
+ unsigned long tear; /* 16-23: TEAR contents */
+ unsigned long dear; /* 24-31: DEAR contents */
+ /* 32-63: */
+ unsigned long rsvrd1; /* reserved */
+ unsigned long rsvrd2; /* reserved */
+ unsigned long rsvrd3; /* reserved */
+ unsigned long rsvrd4; /* reserved */
+} __packed;
+
+struct hws_basic_entry {
+ unsigned int def:16; /* 0-15 Data Entry Format */
+ unsigned int R:4; /* 16-19 reserved */
+ unsigned int U:4; /* 20-23 Number of unique instruct. */
+ unsigned int z:2; /* zeros */
+ unsigned int T:1; /* 26 PSW DAT mode */
+ unsigned int W:1; /* 27 PSW wait state */
+ unsigned int P:1; /* 28 PSW Problem state */
+ unsigned int AS:2; /* 29-30 PSW address-space control */
+ unsigned int I:1; /* 31 entry valid or invalid */
+ unsigned int:16;
+ unsigned int prim_asn:16; /* primary ASN */
+ unsigned long long ia; /* Instruction Address */
+ unsigned long long gpp; /* Guest Program Parameter */
+ unsigned long long hpp; /* Host Program Parameter */
+} __packed;
+
+struct hws_diag_entry {
+ unsigned int def:16; /* 0-15 Data Entry Format */
+ unsigned int R:14; /* 16-19 and 20-30 reserved */
+ unsigned int I:1; /* 31 entry valid or invalid */
+ u8 data[]; /* Machine-dependent sample data */
+} __packed;
+
+struct hws_combined_entry {
+ struct hws_basic_entry basic; /* Basic-sampling data entry */
+ struct hws_diag_entry diag; /* Diagnostic-sampling data entry */
+} __packed;
+
+struct hws_trailer_entry {
+ union {
+ struct {
+ unsigned int f:1; /* 0 - Block Full Indicator */
+ unsigned int a:1; /* 1 - Alert request control */
+ unsigned int t:1; /* 2 - Timestamp format */
+ unsigned long long:61; /* 3 - 63: Reserved */
+ };
+ unsigned long long flags; /* 0 - 63: All indicators */
+ };
+ unsigned long long overflow; /* 64 - sample Overflow count */
+ unsigned char timestamp[16]; /* 16 - 31 timestamp */
+ unsigned long long reserved1; /* 32 -Reserved */
+ unsigned long long reserved2; /* */
+ unsigned long long progusage1; /* 48 - reserved for programming use */
+ unsigned long long progusage2; /* */
+} __packed;
+
/* Query counter information */
static inline int qctri(struct cpumf_ctr_info *info)
{
return cc;
}
+/* Query sampling information */
+static inline int qsi(struct hws_qsi_info_block *info)
+{
+ int cc;
+ cc = 1;
+
+ asm volatile(
+ "0: .insn s,0xb2860000,0(%1)\n"
+ "1: lhi %0,0\n"
+ "2:\n"
+ EX_TABLE(0b, 2b) EX_TABLE(1b, 2b)
+ : "=d" (cc), "+a" (info)
+ : "m" (*info)
+ : "cc", "memory");
+
+ return cc ? -EINVAL : 0;
+}
+
+/* Load sampling controls */
+static inline int lsctl(struct hws_lsctl_request_block *req)
+{
+ int cc;
+
+ cc = 1;
+ asm volatile(
+ "0: .insn s,0xb2870000,0(%1)\n"
+ "1: ipm %0\n"
+ " srl %0,28\n"
+ "2:\n"
+ EX_TABLE(0b, 2b) EX_TABLE(1b, 2b)
+ : "+d" (cc), "+a" (req)
+ : "m" (*req)
+ : "cc", "memory");
+
+ return cc ? -EINVAL : 0;
+}
+
+/* Sampling control helper functions */
+
+#include <linux/time.h>
+
+static inline unsigned long freq_to_sample_rate(struct hws_qsi_info_block *qsi,
+ unsigned long freq)
+{
+ return (USEC_PER_SEC / freq) * qsi->cpu_speed;
+}
+
+static inline unsigned long sample_rate_to_freq(struct hws_qsi_info_block *qsi,
+ unsigned long rate)
+{
+ return USEC_PER_SEC * qsi->cpu_speed / rate;
+}
+
+#define SDB_TE_ALERT_REQ_MASK 0x4000000000000000UL
+#define SDB_TE_BUFFER_FULL_MASK 0x8000000000000000UL
+
+/* Return TOD timestamp contained in an trailer entry */
+static inline unsigned long long trailer_timestamp(struct hws_trailer_entry *te)
+{
+ /* TOD in STCKE format */
+ if (te->t)
+ return *((unsigned long long *) &te->timestamp[1]);
+
+ /* TOD in STCK format */
+ return *((unsigned long long *) &te->timestamp[0]);
+}
+
+/* Return pointer to trailer entry of an sample data block */
+static inline unsigned long *trailer_entry_ptr(unsigned long v)
+{
+ void *ret;
+
+ ret = (void *) v;
+ ret += PAGE_SIZE;
+ ret -= sizeof(struct hws_trailer_entry);
+
+ return (unsigned long *) ret;
+}
+
+/* Return if the entry in the sample data block table (sdbt)
+ * is a link to the next sdbt */
+static inline int is_link_entry(unsigned long *s)
+{
+ return *s & 0x1ul ? 1 : 0;
+}
+
+/* Return pointer to the linked sdbt */
+static inline unsigned long *get_next_sdbt(unsigned long *s)
+{
+ return (unsigned long *) (*s & ~0x1ul);
+}
#endif /* _ASM_S390_CPU_MF_H */
u32 fcx : 1; /* bit 88 */
u32 : 19;
u32 alt_ssi : 1; /* bit 108 */
+ u32:1;
+ u32 narf:1; /* bit 110 */
} __packed;
extern struct css_general_char css_general_characteristics;
void zpci_event_error(void *);
void zpci_event_availability(void *);
void zpci_rescan(void);
+bool zpci_is_enabled(void);
#else /* CONFIG_PCI */
static inline void zpci_event_error(void *e) {}
static inline void zpci_event_availability(void *e) {}
/*
* Performance event support - s390 specific definitions.
*
- * Copyright IBM Corp. 2009, 2012
+ * Copyright IBM Corp. 2009, 2013
* Author(s): Martin Schwidefsky <schwidefsky@de.ibm.com>
* Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
*/
-#include <asm/cpu_mf.h>
+#ifndef _ASM_S390_PERF_EVENT_H
+#define _ASM_S390_PERF_EVENT_H
-/* CPU-measurement counter facility */
-#define PERF_CPUM_CF_MAX_CTR 256
+#ifdef CONFIG_64BIT
+
+#include <linux/perf_event.h>
+#include <linux/device.h>
+#include <asm/cpu_mf.h>
/* Per-CPU flags for PMU states */
#define PMU_F_RESERVED 0x1000
#define PMU_F_ENABLED 0x2000
+#define PMU_F_IN_USE 0x4000
+#define PMU_F_ERR_IBE 0x0100
+#define PMU_F_ERR_LSDA 0x0200
+#define PMU_F_ERR_MASK (PMU_F_ERR_IBE|PMU_F_ERR_LSDA)
+
+/* Perf defintions for PMU event attributes in sysfs */
+extern __init const struct attribute_group **cpumf_cf_event_group(void);
+extern ssize_t cpumf_events_sysfs_show(struct device *dev,
+ struct device_attribute *attr,
+ char *page);
+#define EVENT_VAR(_cat, _name) event_attr_##_cat##_##_name
+#define EVENT_PTR(_cat, _name) (&EVENT_VAR(_cat, _name).attr.attr)
+
+#define CPUMF_EVENT_ATTR(cat, name, id) \
+ PMU_EVENT_ATTR(name, EVENT_VAR(cat, name), id, cpumf_events_sysfs_show)
+#define CPUMF_EVENT_PTR(cat, name) EVENT_PTR(cat, name)
-#ifdef CONFIG_64BIT
/* Perf callbacks */
struct pt_regs;
extern unsigned long perf_misc_flags(struct pt_regs *regs);
#define perf_misc_flags(regs) perf_misc_flags(regs)
+/* Perf pt_regs extension for sample-data-entry indicators */
+struct perf_sf_sde_regs {
+ unsigned char in_guest:1; /* guest sample */
+ unsigned long reserved:63; /* reserved */
+};
+
+/* Perf PMU definitions for the counter facility */
+#define PERF_CPUM_CF_MAX_CTR 256
+
+/* Perf PMU definitions for the sampling facility */
+#define PERF_CPUM_SF_MAX_CTR 2
+#define PERF_EVENT_CPUM_SF 0xB0000UL /* Event: Basic-sampling */
+#define PERF_EVENT_CPUM_SF_DIAG 0xBD000UL /* Event: Combined-sampling */
+#define PERF_CPUM_SF_BASIC_MODE 0x0001 /* Basic-sampling flag */
+#define PERF_CPUM_SF_DIAG_MODE 0x0002 /* Diagnostic-sampling flag */
+#define PERF_CPUM_SF_MODE_MASK (PERF_CPUM_SF_BASIC_MODE| \
+ PERF_CPUM_SF_DIAG_MODE)
+#define PERF_CPUM_SF_FULL_BLOCKS 0x0004 /* Process full SDBs only */
+
+#define REG_NONE 0
+#define REG_OVERFLOW 1
+#define OVERFLOW_REG(hwc) ((hwc)->extra_reg.config)
+#define SFB_ALLOC_REG(hwc) ((hwc)->extra_reg.alloc)
+#define RAWSAMPLE_REG(hwc) ((hwc)->config)
+#define TEAR_REG(hwc) ((hwc)->last_tag)
+#define SAMPL_RATE(hwc) ((hwc)->event_base)
+#define SAMPL_FLAGS(hwc) ((hwc)->config_base)
+#define SAMPL_DIAG_MODE(hwc) (SAMPL_FLAGS(hwc) & PERF_CPUM_SF_DIAG_MODE)
+#define SDB_FULL_BLOCKS(hwc) (SAMPL_FLAGS(hwc) & PERF_CPUM_SF_FULL_BLOCKS)
+
+/* Structure for sampling data entries to be passed as perf raw sample data
+ * to user space. Note that raw sample data must be aligned and, thus, might
+ * be padded with zeros.
+ */
+struct sf_raw_sample {
+#define SF_RAW_SAMPLE_BASIC PERF_CPUM_SF_BASIC_MODE
+#define SF_RAW_SAMPLE_DIAG PERF_CPUM_SF_DIAG_MODE
+ u64 format;
+ u32 size; /* Size of sf_raw_sample */
+ u16 bsdes; /* Basic-sampling data entry size */
+ u16 dsdes; /* Diagnostic-sampling data entry size */
+ struct hws_basic_entry basic; /* Basic-sampling data entry */
+ struct hws_diag_entry diag; /* Diagnostic-sampling data entry */
+ u8 padding[]; /* Padding to next multiple of 8 */
+} __packed;
+
+/* Perf hardware reserve and release functions */
+int perf_reserve_sampling(void);
+void perf_release_sampling(void);
+
#endif /* CONFIG_64BIT */
+#endif /* _ASM_S390_PERF_EVENT_H */
#define QDIO_FLAG_CLEANUP_USING_HALT 0x02
/**
- * struct qdio_initialize - qdio initalization data
+ * struct qdio_initialize - qdio initialization data
* @cdev: associated ccw device
* @q_format: queue format
* @adapter_name: name for the adapter
struct qdio_outbuf_state *output_sbal_state_array;
};
+/**
+ * enum qdio_brinfo_entry_type - type of address entry for qdio_brinfo_desc()
+ * @l3_ipv6_addr: entry contains IPv6 address
+ * @l3_ipv4_addr: entry contains IPv4 address
+ * @l2_addr_lnid: entry contains MAC address and VLAN ID
+ */
+enum qdio_brinfo_entry_type {l3_ipv6_addr, l3_ipv4_addr, l2_addr_lnid};
+
+/**
+ * struct qdio_brinfo_entry_XXX - Address entry for qdio_brinfo_desc()
+ * @nit: Network interface token
+ * @addr: Address of one of the three types
+ *
+ * The struct is passed to the callback function by qdio_brinfo_desc()
+ */
+struct qdio_brinfo_entry_l3_ipv6 {
+ u64 nit;
+ struct { unsigned char _s6_addr[16]; } addr;
+} __packed;
+struct qdio_brinfo_entry_l3_ipv4 {
+ u64 nit;
+ struct { uint32_t _s_addr; } addr;
+} __packed;
+struct qdio_brinfo_entry_l2 {
+ u64 nit;
+ struct { u8 mac[6]; u16 lnid; } addr_lnid;
+} __packed;
+
#define QDIO_STATE_INACTIVE 0x00000002 /* after qdio_cleanup */
#define QDIO_STATE_ESTABLISHED 0x00000004 /* after qdio_establish */
#define QDIO_STATE_ACTIVE 0x00000008 /* after qdio_activate */
extern int qdio_shutdown(struct ccw_device *, int);
extern int qdio_free(struct ccw_device *);
extern int qdio_get_ssqd_desc(struct ccw_device *, struct qdio_ssqd_desc *);
+extern int qdio_pnso_brinfo(struct subchannel_id schid,
+ int cnc, u16 *response,
+ void (*cb)(void *priv, enum qdio_brinfo_entry_type type,
+ void *entry),
+ void *priv);
#endif /* __QDIO_H__ */
int sclp_chp_deconfigure(struct chp_id chpid);
int sclp_chp_read_info(struct sclp_chp_info *info);
void sclp_get_ipl_info(struct sclp_ipl_info *info);
-bool sclp_has_linemode(void);
-bool sclp_has_vt220(void);
+bool __init sclp_has_linemode(void);
+bool __init sclp_has_vt220(void);
int sclp_pci_configure(u32 fid);
int sclp_pci_deconfigure(u32 fid);
int memcpy_hsa(void *dest, unsigned long src, size_t count, int mode);
unsigned short priority_window;
unsigned int status;
} __attribute__((packed));
+
+/**
+ * struct ep11_cprb - EP11 connectivity programming request block
+ * @cprb_len: CPRB header length [0x0020]
+ * @cprb_ver_id: CPRB version id. [0x04]
+ * @pad_000: Alignment pad bytes
+ * @flags: Admin cmd [0x80] or functional cmd [0x00]
+ * @func_id: Function id / subtype [0x5434]
+ * @source_id: Source id [originator id]
+ * @target_id: Target id [usage/ctrl domain id]
+ * @ret_code: Return code
+ * @reserved1: Reserved
+ * @reserved2: Reserved
+ * @payload_len: Payload length
+ */
+struct ep11_cprb {
+ uint16_t cprb_len;
+ unsigned char cprb_ver_id;
+ unsigned char pad_000[2];
+ unsigned char flags;
+ unsigned char func_id[2];
+ uint32_t source_id;
+ uint32_t target_id;
+ uint32_t ret_code;
+ uint32_t reserved1;
+ uint32_t reserved2;
+ uint32_t payload_len;
+} __attribute__((packed));
+
+/**
+ * struct ep11_target_dev - EP11 target device list
+ * @ap_id: AP device id
+ * @dom_id: Usage domain id
+ */
+struct ep11_target_dev {
+ uint16_t ap_id;
+ uint16_t dom_id;
+};
+
+/**
+ * struct ep11_urb - EP11 user request block
+ * @targets_num: Number of target adapters
+ * @targets: Addr to target adapter list
+ * @weight: Level of request priority
+ * @req_no: Request id/number
+ * @req_len: Request length
+ * @req: Addr to request block
+ * @resp_len: Response length
+ * @resp: Addr to response block
+ */
+struct ep11_urb {
+ uint16_t targets_num;
+ uint64_t targets;
+ uint64_t weight;
+ uint64_t req_no;
+ uint64_t req_len;
+ uint64_t req;
+ uint64_t resp_len;
+ uint64_t resp;
+} __attribute__((packed));
+
#define AUTOSELECT ((unsigned int)0xFFFFFFFF)
#define ZCRYPT_IOCTL_MAGIC 'z'
* ZSECSENDCPRB
* Send an arbitrary CPRB to a crypto card.
*
+ * ZSENDEP11CPRB
+ * Send an arbitrary EP11 CPRB to an EP11 coprocessor crypto card.
+ *
* Z90STAT_STATUS_MASK
* Return an 64 element array of unsigned chars for the status of
* all devices.
#define ICARSAMODEXPO _IOC(_IOC_READ|_IOC_WRITE, ZCRYPT_IOCTL_MAGIC, 0x05, 0)
#define ICARSACRT _IOC(_IOC_READ|_IOC_WRITE, ZCRYPT_IOCTL_MAGIC, 0x06, 0)
#define ZSECSENDCPRB _IOC(_IOC_READ|_IOC_WRITE, ZCRYPT_IOCTL_MAGIC, 0x81, 0)
+#define ZSENDEP11CPRB _IOC(_IOC_READ|_IOC_WRITE, ZCRYPT_IOCTL_MAGIC, 0x04, 0)
/* New status calls */
#define Z90STAT_TOTALCOUNT _IOR(ZCRYPT_IOCTL_MAGIC, 0x40, int)
obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
ifdef CONFIG_64BIT
-obj-$(CONFIG_PERF_EVENTS) += perf_event.o perf_cpum_cf.o
+obj-$(CONFIG_PERF_EVENTS) += perf_event.o perf_cpum_cf.o perf_cpum_sf.o \
+ perf_cpum_cf_events.o
obj-y += runtime_instr.o cache.o
endif
regs->gprs[14] = (__u64 __force) ka->sa.sa_restorer | PSW32_ADDR_AMODE;
} else {
regs->gprs[14] = (__u64 __force) frame->retcode | PSW32_ADDR_AMODE;
- err |= __put_user(S390_SYSCALL_OPCODE | __NR_rt_sigreturn,
- (u16 __force __user *)(frame->retcode));
+ if (__put_user(S390_SYSCALL_OPCODE | __NR_rt_sigreturn,
+ (u16 __force __user *)(frame->retcode)))
+ goto give_sigsegv;
}
/* Set up backchain. */
.endm
.macro LPP newpp
-#if defined(CONFIG_KVM) || defined(CONFIG_KVM_MODULE)
+#if IS_ENABLED(CONFIG_KVM)
tm __LC_MACHINE_FLAGS+6,0x20 # MACHINE_FLAG_LPP
jz .+8
.insn s,0xb2800000,\newpp
.endm
.macro HANDLE_SIE_INTERCEPT scratch,reason
-#if defined(CONFIG_KVM) || defined(CONFIG_KVM_MODULE)
+#if IS_ENABLED(CONFIG_KVM)
tmhh %r8,0x0001 # interrupting from user ?
jnz .+62
lgr \scratch,%r9
.quad __critical_end - __critical_start
-#if defined(CONFIG_KVM) || defined(CONFIG_KVM_MODULE)
+#if IS_ENABLED(CONFIG_KVM)
/*
* sie64a calling convention:
* %r2 pointer to sie control block
lctlg %c1,%c1,__LC_USER_ASCE # load primary asce
# some program checks are suppressing. C code (e.g. do_protection_exception)
# will rewind the PSW by the ILC, which is 4 bytes in case of SIE. Other
-# instructions beween sie64a and sie_done should not cause program
+# instructions between sie64a and sie_done should not cause program
# interrupts. So lets use a nop (47 00 00 00) as a landing pad.
# See also HANDLE_SIE_INTERCEPT
rewind_pad:
goto out;
}
+ cpumf_pmu.attr_groups = cpumf_cf_event_group();
rc = perf_pmu_register(&cpumf_pmu, "cpum_cf", PERF_TYPE_RAW);
if (rc) {
pr_err("Registering the cpum_cf PMU failed with rc=%i\n", rc);
--- /dev/null
+/*
+ * Perf PMU sysfs events attributes for available CPU-measurement counters
+ *
+ */
+
+#include <linux/slab.h>
+#include <linux/perf_event.h>
+
+
+/* BEGIN: CPUM_CF COUNTER DEFINITIONS =================================== */
+
+CPUMF_EVENT_ATTR(cf, CPU_CYCLES, 0x0000);
+CPUMF_EVENT_ATTR(cf, INSTRUCTIONS, 0x0001);
+CPUMF_EVENT_ATTR(cf, L1I_DIR_WRITES, 0x0002);
+CPUMF_EVENT_ATTR(cf, L1I_PENALTY_CYCLES, 0x0003);
+CPUMF_EVENT_ATTR(cf, PROBLEM_STATE_CPU_CYCLES, 0x0020);
+CPUMF_EVENT_ATTR(cf, PROBLEM_STATE_INSTRUCTIONS, 0x0021);
+CPUMF_EVENT_ATTR(cf, PROBLEM_STATE_L1I_DIR_WRITES, 0x0022);
+CPUMF_EVENT_ATTR(cf, PROBLEM_STATE_L1I_PENALTY_CYCLES, 0x0023);
+CPUMF_EVENT_ATTR(cf, PROBLEM_STATE_L1D_DIR_WRITES, 0x0024);
+CPUMF_EVENT_ATTR(cf, PROBLEM_STATE_L1D_PENALTY_CYCLES, 0x0025);
+CPUMF_EVENT_ATTR(cf, L1D_DIR_WRITES, 0x0004);
+CPUMF_EVENT_ATTR(cf, L1D_PENALTY_CYCLES, 0x0005);
+CPUMF_EVENT_ATTR(cf, PRNG_FUNCTIONS, 0x0040);
+CPUMF_EVENT_ATTR(cf, PRNG_CYCLES, 0x0041);
+CPUMF_EVENT_ATTR(cf, PRNG_BLOCKED_FUNCTIONS, 0x0042);
+CPUMF_EVENT_ATTR(cf, PRNG_BLOCKED_CYCLES, 0x0043);
+CPUMF_EVENT_ATTR(cf, SHA_FUNCTIONS, 0x0044);
+CPUMF_EVENT_ATTR(cf, SHA_CYCLES, 0x0045);
+CPUMF_EVENT_ATTR(cf, SHA_BLOCKED_FUNCTIONS, 0x0046);
+CPUMF_EVENT_ATTR(cf, SHA_BLOCKED_CYCLES, 0x0047);
+CPUMF_EVENT_ATTR(cf, DEA_FUNCTIONS, 0x0048);
+CPUMF_EVENT_ATTR(cf, DEA_CYCLES, 0x0049);
+CPUMF_EVENT_ATTR(cf, DEA_BLOCKED_FUNCTIONS, 0x004a);
+CPUMF_EVENT_ATTR(cf, DEA_BLOCKED_CYCLES, 0x004b);
+CPUMF_EVENT_ATTR(cf, AES_FUNCTIONS, 0x004c);
+CPUMF_EVENT_ATTR(cf, AES_CYCLES, 0x004d);
+CPUMF_EVENT_ATTR(cf, AES_BLOCKED_FUNCTIONS, 0x004e);
+CPUMF_EVENT_ATTR(cf, AES_BLOCKED_CYCLES, 0x004f);
+CPUMF_EVENT_ATTR(cf_z10, L1I_L2_SOURCED_WRITES, 0x0080);
+CPUMF_EVENT_ATTR(cf_z10, L1D_L2_SOURCED_WRITES, 0x0081);
+CPUMF_EVENT_ATTR(cf_z10, L1I_L3_LOCAL_WRITES, 0x0082);
+CPUMF_EVENT_ATTR(cf_z10, L1D_L3_LOCAL_WRITES, 0x0083);
+CPUMF_EVENT_ATTR(cf_z10, L1I_L3_REMOTE_WRITES, 0x0084);
+CPUMF_EVENT_ATTR(cf_z10, L1D_L3_REMOTE_WRITES, 0x0085);
+CPUMF_EVENT_ATTR(cf_z10, L1D_LMEM_SOURCED_WRITES, 0x0086);
+CPUMF_EVENT_ATTR(cf_z10, L1I_LMEM_SOURCED_WRITES, 0x0087);
+CPUMF_EVENT_ATTR(cf_z10, L1D_RO_EXCL_WRITES, 0x0088);
+CPUMF_EVENT_ATTR(cf_z10, L1I_CACHELINE_INVALIDATES, 0x0089);
+CPUMF_EVENT_ATTR(cf_z10, ITLB1_WRITES, 0x008a);
+CPUMF_EVENT_ATTR(cf_z10, DTLB1_WRITES, 0x008b);
+CPUMF_EVENT_ATTR(cf_z10, TLB2_PTE_WRITES, 0x008c);
+CPUMF_EVENT_ATTR(cf_z10, TLB2_CRSTE_WRITES, 0x008d);
+CPUMF_EVENT_ATTR(cf_z10, TLB2_CRSTE_HPAGE_WRITES, 0x008e);
+CPUMF_EVENT_ATTR(cf_z10, ITLB1_MISSES, 0x0091);
+CPUMF_EVENT_ATTR(cf_z10, DTLB1_MISSES, 0x0092);
+CPUMF_EVENT_ATTR(cf_z10, L2C_STORES_SENT, 0x0093);
+CPUMF_EVENT_ATTR(cf_z196, L1D_L2_SOURCED_WRITES, 0x0080);
+CPUMF_EVENT_ATTR(cf_z196, L1I_L2_SOURCED_WRITES, 0x0081);
+CPUMF_EVENT_ATTR(cf_z196, DTLB1_MISSES, 0x0082);
+CPUMF_EVENT_ATTR(cf_z196, ITLB1_MISSES, 0x0083);
+CPUMF_EVENT_ATTR(cf_z196, L2C_STORES_SENT, 0x0085);
+CPUMF_EVENT_ATTR(cf_z196, L1D_OFFBOOK_L3_SOURCED_WRITES, 0x0086);
+CPUMF_EVENT_ATTR(cf_z196, L1D_ONBOOK_L4_SOURCED_WRITES, 0x0087);
+CPUMF_EVENT_ATTR(cf_z196, L1I_ONBOOK_L4_SOURCED_WRITES, 0x0088);
+CPUMF_EVENT_ATTR(cf_z196, L1D_RO_EXCL_WRITES, 0x0089);
+CPUMF_EVENT_ATTR(cf_z196, L1D_OFFBOOK_L4_SOURCED_WRITES, 0x008a);
+CPUMF_EVENT_ATTR(cf_z196, L1I_OFFBOOK_L4_SOURCED_WRITES, 0x008b);
+CPUMF_EVENT_ATTR(cf_z196, DTLB1_HPAGE_WRITES, 0x008c);
+CPUMF_EVENT_ATTR(cf_z196, L1D_LMEM_SOURCED_WRITES, 0x008d);
+CPUMF_EVENT_ATTR(cf_z196, L1I_LMEM_SOURCED_WRITES, 0x008e);
+CPUMF_EVENT_ATTR(cf_z196, L1I_OFFBOOK_L3_SOURCED_WRITES, 0x008f);
+CPUMF_EVENT_ATTR(cf_z196, DTLB1_WRITES, 0x0090);
+CPUMF_EVENT_ATTR(cf_z196, ITLB1_WRITES, 0x0091);
+CPUMF_EVENT_ATTR(cf_z196, TLB2_PTE_WRITES, 0x0092);
+CPUMF_EVENT_ATTR(cf_z196, TLB2_CRSTE_HPAGE_WRITES, 0x0093);
+CPUMF_EVENT_ATTR(cf_z196, TLB2_CRSTE_WRITES, 0x0094);
+CPUMF_EVENT_ATTR(cf_z196, L1D_ONCHIP_L3_SOURCED_WRITES, 0x0096);
+CPUMF_EVENT_ATTR(cf_z196, L1D_OFFCHIP_L3_SOURCED_WRITES, 0x0098);
+CPUMF_EVENT_ATTR(cf_z196, L1I_ONCHIP_L3_SOURCED_WRITES, 0x0099);
+CPUMF_EVENT_ATTR(cf_z196, L1I_OFFCHIP_L3_SOURCED_WRITES, 0x009b);
+CPUMF_EVENT_ATTR(cf_zec12, DTLB1_MISSES, 0x0080);
+CPUMF_EVENT_ATTR(cf_zec12, ITLB1_MISSES, 0x0081);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_L2I_SOURCED_WRITES, 0x0082);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_L2I_SOURCED_WRITES, 0x0083);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_L2D_SOURCED_WRITES, 0x0084);
+CPUMF_EVENT_ATTR(cf_zec12, DTLB1_WRITES, 0x0085);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_LMEM_SOURCED_WRITES, 0x0087);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_LMEM_SOURCED_WRITES, 0x0089);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_RO_EXCL_WRITES, 0x008a);
+CPUMF_EVENT_ATTR(cf_zec12, DTLB1_HPAGE_WRITES, 0x008b);
+CPUMF_EVENT_ATTR(cf_zec12, ITLB1_WRITES, 0x008c);
+CPUMF_EVENT_ATTR(cf_zec12, TLB2_PTE_WRITES, 0x008d);
+CPUMF_EVENT_ATTR(cf_zec12, TLB2_CRSTE_HPAGE_WRITES, 0x008e);
+CPUMF_EVENT_ATTR(cf_zec12, TLB2_CRSTE_WRITES, 0x008f);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_ONCHIP_L3_SOURCED_WRITES, 0x0090);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_OFFCHIP_L3_SOURCED_WRITES, 0x0091);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_OFFBOOK_L3_SOURCED_WRITES, 0x0092);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_ONBOOK_L4_SOURCED_WRITES, 0x0093);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_OFFBOOK_L4_SOURCED_WRITES, 0x0094);
+CPUMF_EVENT_ATTR(cf_zec12, TX_NC_TEND, 0x0095);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_ONCHIP_L3_SOURCED_WRITES_IV, 0x0096);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_OFFCHIP_L3_SOURCED_WRITES_IV, 0x0097);
+CPUMF_EVENT_ATTR(cf_zec12, L1D_OFFBOOK_L3_SOURCED_WRITES_IV, 0x0098);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_ONCHIP_L3_SOURCED_WRITES, 0x0099);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_OFFCHIP_L3_SOURCED_WRITES, 0x009a);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_OFFBOOK_L3_SOURCED_WRITES, 0x009b);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_ONBOOK_L4_SOURCED_WRITES, 0x009c);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_OFFBOOK_L4_SOURCED_WRITES, 0x009d);
+CPUMF_EVENT_ATTR(cf_zec12, TX_C_TEND, 0x009e);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_ONCHIP_L3_SOURCED_WRITES_IV, 0x009f);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_OFFCHIP_L3_SOURCED_WRITES_IV, 0x00a0);
+CPUMF_EVENT_ATTR(cf_zec12, L1I_OFFBOOK_L3_SOURCED_WRITES_IV, 0x00a1);
+CPUMF_EVENT_ATTR(cf_zec12, TX_NC_TABORT, 0x00b1);
+CPUMF_EVENT_ATTR(cf_zec12, TX_C_TABORT_NO_SPECIAL, 0x00b2);
+CPUMF_EVENT_ATTR(cf_zec12, TX_C_TABORT_SPECIAL, 0x00b3);
+
+static struct attribute *cpumcf_pmu_event_attr[] = {
+ CPUMF_EVENT_PTR(cf, CPU_CYCLES),
+ CPUMF_EVENT_PTR(cf, INSTRUCTIONS),
+ CPUMF_EVENT_PTR(cf, L1I_DIR_WRITES),
+ CPUMF_EVENT_PTR(cf, L1I_PENALTY_CYCLES),
+ CPUMF_EVENT_PTR(cf, PROBLEM_STATE_CPU_CYCLES),
+ CPUMF_EVENT_PTR(cf, PROBLEM_STATE_INSTRUCTIONS),
+ CPUMF_EVENT_PTR(cf, PROBLEM_STATE_L1I_DIR_WRITES),
+ CPUMF_EVENT_PTR(cf, PROBLEM_STATE_L1I_PENALTY_CYCLES),
+ CPUMF_EVENT_PTR(cf, PROBLEM_STATE_L1D_DIR_WRITES),
+ CPUMF_EVENT_PTR(cf, PROBLEM_STATE_L1D_PENALTY_CYCLES),
+ CPUMF_EVENT_PTR(cf, L1D_DIR_WRITES),
+ CPUMF_EVENT_PTR(cf, L1D_PENALTY_CYCLES),
+ CPUMF_EVENT_PTR(cf, PRNG_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, PRNG_CYCLES),
+ CPUMF_EVENT_PTR(cf, PRNG_BLOCKED_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, PRNG_BLOCKED_CYCLES),
+ CPUMF_EVENT_PTR(cf, SHA_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, SHA_CYCLES),
+ CPUMF_EVENT_PTR(cf, SHA_BLOCKED_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, SHA_BLOCKED_CYCLES),
+ CPUMF_EVENT_PTR(cf, DEA_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, DEA_CYCLES),
+ CPUMF_EVENT_PTR(cf, DEA_BLOCKED_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, DEA_BLOCKED_CYCLES),
+ CPUMF_EVENT_PTR(cf, AES_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, AES_CYCLES),
+ CPUMF_EVENT_PTR(cf, AES_BLOCKED_FUNCTIONS),
+ CPUMF_EVENT_PTR(cf, AES_BLOCKED_CYCLES),
+ NULL,
+};
+
+static struct attribute *cpumcf_z10_pmu_event_attr[] __initdata = {
+ CPUMF_EVENT_PTR(cf_z10, L1I_L2_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1D_L2_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1I_L3_LOCAL_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1D_L3_LOCAL_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1I_L3_REMOTE_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1D_L3_REMOTE_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1D_LMEM_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1I_LMEM_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1D_RO_EXCL_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, L1I_CACHELINE_INVALIDATES),
+ CPUMF_EVENT_PTR(cf_z10, ITLB1_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, DTLB1_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, TLB2_PTE_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, TLB2_CRSTE_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, TLB2_CRSTE_HPAGE_WRITES),
+ CPUMF_EVENT_PTR(cf_z10, ITLB1_MISSES),
+ CPUMF_EVENT_PTR(cf_z10, DTLB1_MISSES),
+ CPUMF_EVENT_PTR(cf_z10, L2C_STORES_SENT),
+ NULL,
+};
+
+static struct attribute *cpumcf_z196_pmu_event_attr[] __initdata = {
+ CPUMF_EVENT_PTR(cf_z196, L1D_L2_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1I_L2_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, DTLB1_MISSES),
+ CPUMF_EVENT_PTR(cf_z196, ITLB1_MISSES),
+ CPUMF_EVENT_PTR(cf_z196, L2C_STORES_SENT),
+ CPUMF_EVENT_PTR(cf_z196, L1D_OFFBOOK_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1D_ONBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1I_ONBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1D_RO_EXCL_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1D_OFFBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1I_OFFBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, DTLB1_HPAGE_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1D_LMEM_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1I_LMEM_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1I_OFFBOOK_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, DTLB1_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, ITLB1_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, TLB2_PTE_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, TLB2_CRSTE_HPAGE_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, TLB2_CRSTE_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1D_ONCHIP_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1D_OFFCHIP_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1I_ONCHIP_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_z196, L1I_OFFCHIP_L3_SOURCED_WRITES),
+ NULL,
+};
+
+static struct attribute *cpumcf_zec12_pmu_event_attr[] __initdata = {
+ CPUMF_EVENT_PTR(cf_zec12, DTLB1_MISSES),
+ CPUMF_EVENT_PTR(cf_zec12, ITLB1_MISSES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_L2I_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_L2I_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_L2D_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, DTLB1_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_LMEM_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_LMEM_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_RO_EXCL_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, DTLB1_HPAGE_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, ITLB1_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, TLB2_PTE_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, TLB2_CRSTE_HPAGE_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, TLB2_CRSTE_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_ONCHIP_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_OFFCHIP_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_OFFBOOK_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_ONBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_OFFBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, TX_NC_TEND),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_ONCHIP_L3_SOURCED_WRITES_IV),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_OFFCHIP_L3_SOURCED_WRITES_IV),
+ CPUMF_EVENT_PTR(cf_zec12, L1D_OFFBOOK_L3_SOURCED_WRITES_IV),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_ONCHIP_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_OFFCHIP_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_OFFBOOK_L3_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_ONBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_OFFBOOK_L4_SOURCED_WRITES),
+ CPUMF_EVENT_PTR(cf_zec12, TX_C_TEND),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_ONCHIP_L3_SOURCED_WRITES_IV),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_OFFCHIP_L3_SOURCED_WRITES_IV),
+ CPUMF_EVENT_PTR(cf_zec12, L1I_OFFBOOK_L3_SOURCED_WRITES_IV),
+ CPUMF_EVENT_PTR(cf_zec12, TX_NC_TABORT),
+ CPUMF_EVENT_PTR(cf_zec12, TX_C_TABORT_NO_SPECIAL),
+ CPUMF_EVENT_PTR(cf_zec12, TX_C_TABORT_SPECIAL),
+ NULL,
+};
+
+/* END: CPUM_CF COUNTER DEFINITIONS ===================================== */
+
+static struct attribute_group cpumsf_pmu_events_group = {
+ .name = "events",
+ .attrs = cpumcf_pmu_event_attr,
+};
+
+PMU_FORMAT_ATTR(event, "config:0-63");
+
+static struct attribute *cpumsf_pmu_format_attr[] = {
+ &format_attr_event.attr,
+ NULL,
+};
+
+static struct attribute_group cpumsf_pmu_format_group = {
+ .name = "format",
+ .attrs = cpumsf_pmu_format_attr,
+};
+
+static const struct attribute_group *cpumsf_pmu_attr_groups[] = {
+ &cpumsf_pmu_events_group,
+ &cpumsf_pmu_format_group,
+ NULL,
+};
+
+
+static __init struct attribute **merge_attr(struct attribute **a,
+ struct attribute **b)
+{
+ struct attribute **new;
+ int j, i;
+
+ for (j = 0; a[j]; j++)
+ ;
+ for (i = 0; b[i]; i++)
+ j++;
+ j++;
+
+ new = kmalloc(sizeof(struct attribute *) * j, GFP_KERNEL);
+ if (!new)
+ return NULL;
+ j = 0;
+ for (i = 0; a[i]; i++)
+ new[j++] = a[i];
+ for (i = 0; b[i]; i++)
+ new[j++] = b[i];
+ new[j] = NULL;
+
+ return new;
+}
+
+__init const struct attribute_group **cpumf_cf_event_group(void)
+{
+ struct attribute **combined, **model;
+ struct cpuid cpu_id;
+
+ get_cpu_id(&cpu_id);
+ switch (cpu_id.machine) {
+ case 0x2097:
+ case 0x2098:
+ model = cpumcf_z10_pmu_event_attr;
+ break;
+ case 0x2817:
+ case 0x2818:
+ model = cpumcf_z196_pmu_event_attr;
+ break;
+ case 0x2827:
+ case 0x2828:
+ model = cpumcf_zec12_pmu_event_attr;
+ break;
+ default:
+ model = NULL;
+ break;
+ };
+
+ if (!model)
+ goto out;
+
+ combined = merge_attr(cpumcf_pmu_event_attr, model);
+ if (combined)
+ cpumsf_pmu_events_group.attrs = combined;
+out:
+ return cpumsf_pmu_attr_groups;
+}
--- /dev/null
+/*
+ * Performance event support for the System z CPU-measurement Sampling Facility
+ *
+ * Copyright IBM Corp. 2013
+ * Author(s): Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License (version 2 only)
+ * as published by the Free Software Foundation.
+ */
+#define KMSG_COMPONENT "cpum_sf"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
+#include <linux/kernel.h>
+#include <linux/kernel_stat.h>
+#include <linux/perf_event.h>
+#include <linux/percpu.h>
+#include <linux/notifier.h>
+#include <linux/export.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+#include <linux/moduleparam.h>
+#include <asm/cpu_mf.h>
+#include <asm/irq.h>
+#include <asm/debug.h>
+#include <asm/timex.h>
+
+/* Minimum number of sample-data-block-tables:
+ * At least one table is required for the sampling buffer structure.
+ * A single table contains up to 511 pointers to sample-data-blocks.
+ */
+#define CPUM_SF_MIN_SDBT 1
+
+/* Number of sample-data-blocks per sample-data-block-table (SDBT):
+ * A table contains SDB pointers (8 bytes) and one table-link entry
+ * that points to the origin of the next SDBT.
+ */
+#define CPUM_SF_SDB_PER_TABLE ((PAGE_SIZE - 8) / 8)
+
+/* Maximum page offset for an SDBT table-link entry:
+ * If this page offset is reached, a table-link entry to the next SDBT
+ * must be added.
+ */
+#define CPUM_SF_SDBT_TL_OFFSET (CPUM_SF_SDB_PER_TABLE * 8)
+static inline int require_table_link(const void *sdbt)
+{
+ return ((unsigned long) sdbt & ~PAGE_MASK) == CPUM_SF_SDBT_TL_OFFSET;
+}
+
+/* Minimum and maximum sampling buffer sizes:
+ *
+ * This number represents the maximum size of the sampling buffer taking
+ * the number of sample-data-block-tables into account. Note that these
+ * numbers apply to the basic-sampling function only.
+ * The maximum number of SDBs is increased by CPUM_SF_SDB_DIAG_FACTOR if
+ * the diagnostic-sampling function is active.
+ *
+ * Sampling buffer size Buffer characteristics
+ * ---------------------------------------------------
+ * 64KB == 16 pages (4KB per page)
+ * 1 page for SDB-tables
+ * 15 pages for SDBs
+ *
+ * 32MB == 8192 pages (4KB per page)
+ * 16 pages for SDB-tables
+ * 8176 pages for SDBs
+ */
+static unsigned long __read_mostly CPUM_SF_MIN_SDB = 15;
+static unsigned long __read_mostly CPUM_SF_MAX_SDB = 8176;
+static unsigned long __read_mostly CPUM_SF_SDB_DIAG_FACTOR = 1;
+
+struct sf_buffer {
+ unsigned long *sdbt; /* Sample-data-block-table origin */
+ /* buffer characteristics (required for buffer increments) */
+ unsigned long num_sdb; /* Number of sample-data-blocks */
+ unsigned long num_sdbt; /* Number of sample-data-block-tables */
+ unsigned long *tail; /* last sample-data-block-table */
+};
+
+struct cpu_hw_sf {
+ /* CPU-measurement sampling information block */
+ struct hws_qsi_info_block qsi;
+ /* CPU-measurement sampling control block */
+ struct hws_lsctl_request_block lsctl;
+ struct sf_buffer sfb; /* Sampling buffer */
+ unsigned int flags; /* Status flags */
+ struct perf_event *event; /* Scheduled perf event */
+};
+static DEFINE_PER_CPU(struct cpu_hw_sf, cpu_hw_sf);
+
+/* Debug feature */
+static debug_info_t *sfdbg;
+
+/*
+ * sf_disable() - Switch off sampling facility
+ */
+static int sf_disable(void)
+{
+ struct hws_lsctl_request_block sreq;
+
+ memset(&sreq, 0, sizeof(sreq));
+ return lsctl(&sreq);
+}
+
+/*
+ * sf_buffer_available() - Check for an allocated sampling buffer
+ */
+static int sf_buffer_available(struct cpu_hw_sf *cpuhw)
+{
+ return !!cpuhw->sfb.sdbt;
+}
+
+/*
+ * deallocate sampling facility buffer
+ */
+static void free_sampling_buffer(struct sf_buffer *sfb)
+{
+ unsigned long *sdbt, *curr;
+
+ if (!sfb->sdbt)
+ return;
+
+ sdbt = sfb->sdbt;
+ curr = sdbt;
+
+ /* Free the SDBT after all SDBs are processed... */
+ while (1) {
+ if (!*curr || !sdbt)
+ break;
+
+ /* Process table-link entries */
+ if (is_link_entry(curr)) {
+ curr = get_next_sdbt(curr);
+ if (sdbt)
+ free_page((unsigned long) sdbt);
+
+ /* If the origin is reached, sampling buffer is freed */
+ if (curr == sfb->sdbt)
+ break;
+ else
+ sdbt = curr;
+ } else {
+ /* Process SDB pointer */
+ if (*curr) {
+ free_page(*curr);
+ curr++;
+ }
+ }
+ }
+
+ debug_sprintf_event(sfdbg, 5,
+ "free_sampling_buffer: freed sdbt=%p\n", sfb->sdbt);
+ memset(sfb, 0, sizeof(*sfb));
+}
+
+static int alloc_sample_data_block(unsigned long *sdbt, gfp_t gfp_flags)
+{
+ unsigned long sdb, *trailer;
+
+ /* Allocate and initialize sample-data-block */
+ sdb = get_zeroed_page(gfp_flags);
+ if (!sdb)
+ return -ENOMEM;
+ trailer = trailer_entry_ptr(sdb);
+ *trailer = SDB_TE_ALERT_REQ_MASK;
+
+ /* Link SDB into the sample-data-block-table */
+ *sdbt = sdb;
+
+ return 0;
+}
+
+/*
+ * realloc_sampling_buffer() - extend sampler memory
+ *
+ * Allocates new sample-data-blocks and adds them to the specified sampling
+ * buffer memory.
+ *
+ * Important: This modifies the sampling buffer and must be called when the
+ * sampling facility is disabled.
+ *
+ * Returns zero on success, non-zero otherwise.
+ */
+static int realloc_sampling_buffer(struct sf_buffer *sfb,
+ unsigned long num_sdb, gfp_t gfp_flags)
+{
+ int i, rc;
+ unsigned long *new, *tail;
+
+ if (!sfb->sdbt || !sfb->tail)
+ return -EINVAL;
+
+ if (!is_link_entry(sfb->tail))
+ return -EINVAL;
+
+ /* Append to the existing sampling buffer, overwriting the table-link
+ * register.
+ * The tail variables always points to the "tail" (last and table-link)
+ * entry in an SDB-table.
+ */
+ tail = sfb->tail;
+
+ /* Do a sanity check whether the table-link entry points to
+ * the sampling buffer origin.
+ */
+ if (sfb->sdbt != get_next_sdbt(tail)) {
+ debug_sprintf_event(sfdbg, 3, "realloc_sampling_buffer: "
+ "sampling buffer is not linked: origin=%p"
+ "tail=%p\n",
+ (void *) sfb->sdbt, (void *) tail);
+ return -EINVAL;
+ }
+
+ /* Allocate remaining SDBs */
+ rc = 0;
+ for (i = 0; i < num_sdb; i++) {
+ /* Allocate a new SDB-table if it is full. */
+ if (require_table_link(tail)) {
+ new = (unsigned long *) get_zeroed_page(gfp_flags);
+ if (!new) {
+ rc = -ENOMEM;
+ break;
+ }
+ sfb->num_sdbt++;
+ /* Link current page to tail of chain */
+ *tail = (unsigned long)(void *) new + 1;
+ tail = new;
+ }
+
+ /* Allocate a new sample-data-block.
+ * If there is not enough memory, stop the realloc process
+ * and simply use what was allocated. If this is a temporary
+ * issue, a new realloc call (if required) might succeed.
+ */
+ rc = alloc_sample_data_block(tail, gfp_flags);
+ if (rc)
+ break;
+ sfb->num_sdb++;
+ tail++;
+ }
+
+ /* Link sampling buffer to its origin */
+ *tail = (unsigned long) sfb->sdbt + 1;
+ sfb->tail = tail;
+
+ debug_sprintf_event(sfdbg, 4, "realloc_sampling_buffer: new buffer"
+ " settings: sdbt=%lu sdb=%lu\n",
+ sfb->num_sdbt, sfb->num_sdb);
+ return rc;
+}
+
+/*
+ * allocate_sampling_buffer() - allocate sampler memory
+ *
+ * Allocates and initializes a sampling buffer structure using the
+ * specified number of sample-data-blocks (SDB). For each allocation,
+ * a 4K page is used. The number of sample-data-block-tables (SDBT)
+ * are calculated from SDBs.
+ * Also set the ALERT_REQ mask in each SDBs trailer.
+ *
+ * Returns zero on success, non-zero otherwise.
+ */
+static int alloc_sampling_buffer(struct sf_buffer *sfb, unsigned long num_sdb)
+{
+ int rc;
+
+ if (sfb->sdbt)
+ return -EINVAL;
+
+ /* Allocate the sample-data-block-table origin */
+ sfb->sdbt = (unsigned long *) get_zeroed_page(GFP_KERNEL);
+ if (!sfb->sdbt)
+ return -ENOMEM;
+ sfb->num_sdb = 0;
+ sfb->num_sdbt = 1;
+
+ /* Link the table origin to point to itself to prepare for
+ * realloc_sampling_buffer() invocation.
+ */
+ sfb->tail = sfb->sdbt;
+ *sfb->tail = (unsigned long)(void *) sfb->sdbt + 1;
+
+ /* Allocate requested number of sample-data-blocks */
+ rc = realloc_sampling_buffer(sfb, num_sdb, GFP_KERNEL);
+ if (rc) {
+ free_sampling_buffer(sfb);
+ debug_sprintf_event(sfdbg, 4, "alloc_sampling_buffer: "
+ "realloc_sampling_buffer failed with rc=%i\n", rc);
+ } else
+ debug_sprintf_event(sfdbg, 4,
+ "alloc_sampling_buffer: tear=%p dear=%p\n",
+ sfb->sdbt, (void *) *sfb->sdbt);
+ return rc;
+}
+
+static void sfb_set_limits(unsigned long min, unsigned long max)
+{
+ struct hws_qsi_info_block si;
+
+ CPUM_SF_MIN_SDB = min;
+ CPUM_SF_MAX_SDB = max;
+
+ memset(&si, 0, sizeof(si));
+ if (!qsi(&si))
+ CPUM_SF_SDB_DIAG_FACTOR = DIV_ROUND_UP(si.dsdes, si.bsdes);
+}
+
+static unsigned long sfb_max_limit(struct hw_perf_event *hwc)
+{
+ return SAMPL_DIAG_MODE(hwc) ? CPUM_SF_MAX_SDB * CPUM_SF_SDB_DIAG_FACTOR
+ : CPUM_SF_MAX_SDB;
+}
+
+static unsigned long sfb_pending_allocs(struct sf_buffer *sfb,
+ struct hw_perf_event *hwc)
+{
+ if (!sfb->sdbt)
+ return SFB_ALLOC_REG(hwc);
+ if (SFB_ALLOC_REG(hwc) > sfb->num_sdb)
+ return SFB_ALLOC_REG(hwc) - sfb->num_sdb;
+ return 0;
+}
+
+static int sfb_has_pending_allocs(struct sf_buffer *sfb,
+ struct hw_perf_event *hwc)
+{
+ return sfb_pending_allocs(sfb, hwc) > 0;
+}
+
+static void sfb_account_allocs(unsigned long num, struct hw_perf_event *hwc)
+{
+ /* Limit the number of SDBs to not exceed the maximum */
+ num = min_t(unsigned long, num, sfb_max_limit(hwc) - SFB_ALLOC_REG(hwc));
+ if (num)
+ SFB_ALLOC_REG(hwc) += num;
+}
+
+static void sfb_init_allocs(unsigned long num, struct hw_perf_event *hwc)
+{
+ SFB_ALLOC_REG(hwc) = 0;
+ sfb_account_allocs(num, hwc);
+}
+
+static size_t event_sample_size(struct hw_perf_event *hwc)
+{
+ struct sf_raw_sample *sfr = (struct sf_raw_sample *) RAWSAMPLE_REG(hwc);
+ size_t sample_size;
+
+ /* The sample size depends on the sampling function: The basic-sampling
+ * function must be always enabled, diagnostic-sampling function is
+ * optional.
+ */
+ sample_size = sfr->bsdes;
+ if (SAMPL_DIAG_MODE(hwc))
+ sample_size += sfr->dsdes;
+
+ return sample_size;
+}
+
+static void deallocate_buffers(struct cpu_hw_sf *cpuhw)
+{
+ if (cpuhw->sfb.sdbt)
+ free_sampling_buffer(&cpuhw->sfb);
+}
+
+static int allocate_buffers(struct cpu_hw_sf *cpuhw, struct hw_perf_event *hwc)
+{
+ unsigned long n_sdb, freq, factor;
+ size_t sfr_size, sample_size;
+ struct sf_raw_sample *sfr;
+
+ /* Allocate raw sample buffer
+ *
+ * The raw sample buffer is used to temporarily store sampling data
+ * entries for perf raw sample processing. The buffer size mainly
+ * depends on the size of diagnostic-sampling data entries which is
+ * machine-specific. The exact size calculation includes:
+ * 1. The first 4 bytes of diagnostic-sampling data entries are
+ * already reflected in the sf_raw_sample structure. Subtract
+ * these bytes.
+ * 2. The perf raw sample data must be 8-byte aligned (u64) and
+ * perf's internal data size must be considered too. So add
+ * an additional u32 for correct alignment and subtract before
+ * allocating the buffer.
+ * 3. Store the raw sample buffer pointer in the perf event
+ * hardware structure.
+ */
+ sfr_size = ALIGN((sizeof(*sfr) - sizeof(sfr->diag) + cpuhw->qsi.dsdes) +
+ sizeof(u32), sizeof(u64));
+ sfr_size -= sizeof(u32);
+ sfr = kzalloc(sfr_size, GFP_KERNEL);
+ if (!sfr)
+ return -ENOMEM;
+ sfr->size = sfr_size;
+ sfr->bsdes = cpuhw->qsi.bsdes;
+ sfr->dsdes = cpuhw->qsi.dsdes;
+ RAWSAMPLE_REG(hwc) = (unsigned long) sfr;
+
+ /* Calculate sampling buffers using 4K pages
+ *
+ * 1. Determine the sample data size which depends on the used
+ * sampling functions, for example, basic-sampling or
+ * basic-sampling with diagnostic-sampling.
+ *
+ * 2. Use the sampling frequency as input. The sampling buffer is
+ * designed for almost one second. This can be adjusted through
+ * the "factor" variable.
+ * In any case, alloc_sampling_buffer() sets the Alert Request
+ * Control indicator to trigger a measurement-alert to harvest
+ * sample-data-blocks (sdb).
+ *
+ * 3. Compute the number of sample-data-blocks and ensure a minimum
+ * of CPUM_SF_MIN_SDB. Also ensure the upper limit does not
+ * exceed a "calculated" maximum. The symbolic maximum is
+ * designed for basic-sampling only and needs to be increased if
+ * diagnostic-sampling is active.
+ * See also the remarks for these symbolic constants.
+ *
+ * 4. Compute the number of sample-data-block-tables (SDBT) and
+ * ensure a minimum of CPUM_SF_MIN_SDBT (one table can manage up
+ * to 511 SDBs).
+ */
+ sample_size = event_sample_size(hwc);
+ freq = sample_rate_to_freq(&cpuhw->qsi, SAMPL_RATE(hwc));
+ factor = 1;
+ n_sdb = DIV_ROUND_UP(freq, factor * ((PAGE_SIZE-64) / sample_size));
+ if (n_sdb < CPUM_SF_MIN_SDB)
+ n_sdb = CPUM_SF_MIN_SDB;
+
+ /* If there is already a sampling buffer allocated, it is very likely
+ * that the sampling facility is enabled too. If the event to be
+ * initialized requires a greater sampling buffer, the allocation must
+ * be postponed. Changing the sampling buffer requires the sampling
+ * facility to be in the disabled state. So, account the number of
+ * required SDBs and let cpumsf_pmu_enable() resize the buffer just
+ * before the event is started.
+ */
+ sfb_init_allocs(n_sdb, hwc);
+ if (sf_buffer_available(cpuhw))
+ return 0;
+
+ debug_sprintf_event(sfdbg, 3,
+ "allocate_buffers: rate=%lu f=%lu sdb=%lu/%lu"
+ " sample_size=%lu cpuhw=%p\n",
+ SAMPL_RATE(hwc), freq, n_sdb, sfb_max_limit(hwc),
+ sample_size, cpuhw);
+
+ return alloc_sampling_buffer(&cpuhw->sfb,
+ sfb_pending_allocs(&cpuhw->sfb, hwc));
+}
+
+static unsigned long min_percent(unsigned int percent, unsigned long base,
+ unsigned long min)
+{
+ return min_t(unsigned long, min, DIV_ROUND_UP(percent * base, 100));
+}
+
+static unsigned long compute_sfb_extent(unsigned long ratio, unsigned long base)
+{
+ /* Use a percentage-based approach to extend the sampling facility
+ * buffer. Accept up to 5% sample data loss.
+ * Vary the extents between 1% to 5% of the current number of
+ * sample-data-blocks.
+ */
+ if (ratio <= 5)
+ return 0;
+ if (ratio <= 25)
+ return min_percent(1, base, 1);
+ if (ratio <= 50)
+ return min_percent(1, base, 1);
+ if (ratio <= 75)
+ return min_percent(2, base, 2);
+ if (ratio <= 100)
+ return min_percent(3, base, 3);
+ if (ratio <= 250)
+ return min_percent(4, base, 4);
+
+ return min_percent(5, base, 8);
+}
+
+static void sfb_account_overflows(struct cpu_hw_sf *cpuhw,
+ struct hw_perf_event *hwc)
+{
+ unsigned long ratio, num;
+
+ if (!OVERFLOW_REG(hwc))
+ return;
+
+ /* The sample_overflow contains the average number of sample data
+ * that has been lost because sample-data-blocks were full.
+ *
+ * Calculate the total number of sample data entries that has been
+ * discarded. Then calculate the ratio of lost samples to total samples
+ * per second in percent.
+ */
+ ratio = DIV_ROUND_UP(100 * OVERFLOW_REG(hwc) * cpuhw->sfb.num_sdb,
+ sample_rate_to_freq(&cpuhw->qsi, SAMPL_RATE(hwc)));
+
+ /* Compute number of sample-data-blocks */
+ num = compute_sfb_extent(ratio, cpuhw->sfb.num_sdb);
+ if (num)
+ sfb_account_allocs(num, hwc);
+
+ debug_sprintf_event(sfdbg, 5, "sfb: overflow: overflow=%llu ratio=%lu"
+ " num=%lu\n", OVERFLOW_REG(hwc), ratio, num);
+ OVERFLOW_REG(hwc) = 0;
+}
+
+/* extend_sampling_buffer() - Extend sampling buffer
+ * @sfb: Sampling buffer structure (for local CPU)
+ * @hwc: Perf event hardware structure
+ *
+ * Use this function to extend the sampling buffer based on the overflow counter
+ * and postponed allocation extents stored in the specified Perf event hardware.
+ *
+ * Important: This function disables the sampling facility in order to safely
+ * change the sampling buffer structure. Do not call this function
+ * when the PMU is active.
+ */
+static void extend_sampling_buffer(struct sf_buffer *sfb,
+ struct hw_perf_event *hwc)
+{
+ unsigned long num, num_old;
+ int rc;
+
+ num = sfb_pending_allocs(sfb, hwc);
+ if (!num)
+ return;
+ num_old = sfb->num_sdb;
+
+ /* Disable the sampling facility to reset any states and also
+ * clear pending measurement alerts.
+ */
+ sf_disable();
+
+ /* Extend the sampling buffer.
+ * This memory allocation typically happens in an atomic context when
+ * called by perf. Because this is a reallocation, it is fine if the
+ * new SDB-request cannot be satisfied immediately.
+ */
+ rc = realloc_sampling_buffer(sfb, num, GFP_ATOMIC);
+ if (rc)
+ debug_sprintf_event(sfdbg, 5, "sfb: extend: realloc "
+ "failed with rc=%i\n", rc);
+
+ if (sfb_has_pending_allocs(sfb, hwc))
+ debug_sprintf_event(sfdbg, 5, "sfb: extend: "
+ "req=%lu alloc=%lu remaining=%lu\n",
+ num, sfb->num_sdb - num_old,
+ sfb_pending_allocs(sfb, hwc));
+}
+
+
+/* Number of perf events counting hardware events */
+static atomic_t num_events;
+/* Used to avoid races in calling reserve/release_cpumf_hardware */
+static DEFINE_MUTEX(pmc_reserve_mutex);
+
+#define PMC_INIT 0
+#define PMC_RELEASE 1
+#define PMC_FAILURE 2
+static void setup_pmc_cpu(void *flags)
+{
+ int err;
+ struct cpu_hw_sf *cpusf = &__get_cpu_var(cpu_hw_sf);
+
+ err = 0;
+ switch (*((int *) flags)) {
+ case PMC_INIT:
+ memset(cpusf, 0, sizeof(*cpusf));
+ err = qsi(&cpusf->qsi);
+ if (err)
+ break;
+ cpusf->flags |= PMU_F_RESERVED;
+ err = sf_disable();
+ if (err)
+ pr_err("Switching off the sampling facility failed "
+ "with rc=%i\n", err);
+ debug_sprintf_event(sfdbg, 5,
+ "setup_pmc_cpu: initialized: cpuhw=%p\n", cpusf);
+ break;
+ case PMC_RELEASE:
+ cpusf->flags &= ~PMU_F_RESERVED;
+ err = sf_disable();
+ if (err) {
+ pr_err("Switching off the sampling facility failed "
+ "with rc=%i\n", err);
+ } else
+ deallocate_buffers(cpusf);
+ debug_sprintf_event(sfdbg, 5,
+ "setup_pmc_cpu: released: cpuhw=%p\n", cpusf);
+ break;
+ }
+ if (err)
+ *((int *) flags) |= PMC_FAILURE;
+}
+
+static void release_pmc_hardware(void)
+{
+ int flags = PMC_RELEASE;
+
+ irq_subclass_unregister(IRQ_SUBCLASS_MEASUREMENT_ALERT);
+ on_each_cpu(setup_pmc_cpu, &flags, 1);
+ perf_release_sampling();
+}
+
+static int reserve_pmc_hardware(void)
+{
+ int flags = PMC_INIT;
+ int err;
+
+ err = perf_reserve_sampling();
+ if (err)
+ return err;
+ on_each_cpu(setup_pmc_cpu, &flags, 1);
+ if (flags & PMC_FAILURE) {
+ release_pmc_hardware();
+ return -ENODEV;
+ }
+ irq_subclass_register(IRQ_SUBCLASS_MEASUREMENT_ALERT);
+
+ return 0;
+}
+
+static void hw_perf_event_destroy(struct perf_event *event)
+{
+ /* Free raw sample buffer */
+ if (RAWSAMPLE_REG(&event->hw))
+ kfree((void *) RAWSAMPLE_REG(&event->hw));
+
+ /* Release PMC if this is the last perf event */
+ if (!atomic_add_unless(&num_events, -1, 1)) {
+ mutex_lock(&pmc_reserve_mutex);
+ if (atomic_dec_return(&num_events) == 0)
+ release_pmc_hardware();
+ mutex_unlock(&pmc_reserve_mutex);
+ }
+}
+
+static void hw_init_period(struct hw_perf_event *hwc, u64 period)
+{
+ hwc->sample_period = period;
+ hwc->last_period = hwc->sample_period;
+ local64_set(&hwc->period_left, hwc->sample_period);
+}
+
+static void hw_reset_registers(struct hw_perf_event *hwc,
+ unsigned long *sdbt_origin)
+{
+ struct sf_raw_sample *sfr;
+
+ /* (Re)set to first sample-data-block-table */
+ TEAR_REG(hwc) = (unsigned long) sdbt_origin;
+
+ /* (Re)set raw sampling buffer register */
+ sfr = (struct sf_raw_sample *) RAWSAMPLE_REG(hwc);
+ memset(&sfr->basic, 0, sizeof(sfr->basic));
+ memset(&sfr->diag, 0, sfr->dsdes);
+}
+
+static unsigned long hw_limit_rate(const struct hws_qsi_info_block *si,
+ unsigned long rate)
+{
+ return clamp_t(unsigned long, rate,
+ si->min_sampl_rate, si->max_sampl_rate);
+}
+
+static int __hw_perf_event_init(struct perf_event *event)
+{
+ struct cpu_hw_sf *cpuhw;
+ struct hws_qsi_info_block si;
+ struct perf_event_attr *attr = &event->attr;
+ struct hw_perf_event *hwc = &event->hw;
+ unsigned long rate;
+ int cpu, err;
+
+ /* Reserve CPU-measurement sampling facility */
+ err = 0;
+ if (!atomic_inc_not_zero(&num_events)) {
+ mutex_lock(&pmc_reserve_mutex);
+ if (atomic_read(&num_events) == 0 && reserve_pmc_hardware())
+ err = -EBUSY;
+ else
+ atomic_inc(&num_events);
+ mutex_unlock(&pmc_reserve_mutex);
+ }
+ event->destroy = hw_perf_event_destroy;
+
+ if (err)
+ goto out;
+
+ /* Access per-CPU sampling information (query sampling info) */
+ /*
+ * The event->cpu value can be -1 to count on every CPU, for example,
+ * when attaching to a task. If this is specified, use the query
+ * sampling info from the current CPU, otherwise use event->cpu to
+ * retrieve the per-CPU information.
+ * Later, cpuhw indicates whether to allocate sampling buffers for a
+ * particular CPU (cpuhw!=NULL) or each online CPU (cpuw==NULL).
+ */
+ memset(&si, 0, sizeof(si));
+ cpuhw = NULL;
+ if (event->cpu == -1)
+ qsi(&si);
+ else {
+ /* Event is pinned to a particular CPU, retrieve the per-CPU
+ * sampling structure for accessing the CPU-specific QSI.
+ */
+ cpuhw = &per_cpu(cpu_hw_sf, event->cpu);
+ si = cpuhw->qsi;
+ }
+
+ /* Check sampling facility authorization and, if not authorized,
+ * fall back to other PMUs. It is safe to check any CPU because
+ * the authorization is identical for all configured CPUs.
+ */
+ if (!si.as) {
+ err = -ENOENT;
+ goto out;
+ }
+
+ /* Always enable basic sampling */
+ SAMPL_FLAGS(hwc) = PERF_CPUM_SF_BASIC_MODE;
+
+ /* Check if diagnostic sampling is requested. Deny if the required
+ * sampling authorization is missing.
+ */
+ if (attr->config == PERF_EVENT_CPUM_SF_DIAG) {
+ if (!si.ad) {
+ err = -EPERM;
+ goto out;
+ }
+ SAMPL_FLAGS(hwc) |= PERF_CPUM_SF_DIAG_MODE;
+ }
+
+ /* Check and set other sampling flags */
+ if (attr->config1 & PERF_CPUM_SF_FULL_BLOCKS)
+ SAMPL_FLAGS(hwc) |= PERF_CPUM_SF_FULL_BLOCKS;
+
+ /* The sampling information (si) contains information about the
+ * min/max sampling intervals and the CPU speed. So calculate the
+ * correct sampling interval and avoid the whole period adjust
+ * feedback loop.
+ */
+ rate = 0;
+ if (attr->freq) {
+ rate = freq_to_sample_rate(&si, attr->sample_freq);
+ rate = hw_limit_rate(&si, rate);
+ attr->freq = 0;
+ attr->sample_period = rate;
+ } else {
+ /* The min/max sampling rates specifies the valid range
+ * of sample periods. If the specified sample period is
+ * out of range, limit the period to the range boundary.
+ */
+ rate = hw_limit_rate(&si, hwc->sample_period);
+
+ /* The perf core maintains a maximum sample rate that is
+ * configurable through the sysctl interface. Ensure the
+ * sampling rate does not exceed this value. This also helps
+ * to avoid throttling when pushing samples with
+ * perf_event_overflow().
+ */
+ if (sample_rate_to_freq(&si, rate) >
+ sysctl_perf_event_sample_rate) {
+ err = -EINVAL;
+ debug_sprintf_event(sfdbg, 1, "Sampling rate exceeds maximum perf sample rate\n");
+ goto out;
+ }
+ }
+ SAMPL_RATE(hwc) = rate;
+ hw_init_period(hwc, SAMPL_RATE(hwc));
+
+ /* Initialize sample data overflow accounting */
+ hwc->extra_reg.reg = REG_OVERFLOW;
+ OVERFLOW_REG(hwc) = 0;
+
+ /* Allocate the per-CPU sampling buffer using the CPU information
+ * from the event. If the event is not pinned to a particular
+ * CPU (event->cpu == -1; or cpuhw == NULL), allocate sampling
+ * buffers for each online CPU.
+ */
+ if (cpuhw)
+ /* Event is pinned to a particular CPU */
+ err = allocate_buffers(cpuhw, hwc);
+ else {
+ /* Event is not pinned, allocate sampling buffer on
+ * each online CPU
+ */
+ for_each_online_cpu(cpu) {
+ cpuhw = &per_cpu(cpu_hw_sf, cpu);
+ err = allocate_buffers(cpuhw, hwc);
+ if (err)
+ break;
+ }
+ }
+out:
+ return err;
+}
+
+static int cpumsf_pmu_event_init(struct perf_event *event)
+{
+ int err;
+
+ /* No support for taken branch sampling */
+ if (has_branch_stack(event))
+ return -EOPNOTSUPP;
+
+ switch (event->attr.type) {
+ case PERF_TYPE_RAW:
+ if ((event->attr.config != PERF_EVENT_CPUM_SF) &&
+ (event->attr.config != PERF_EVENT_CPUM_SF_DIAG))
+ return -ENOENT;
+ break;
+ case PERF_TYPE_HARDWARE:
+ /* Support sampling of CPU cycles in addition to the
+ * counter facility. However, the counter facility
+ * is more precise and, hence, restrict this PMU to
+ * sampling events only.
+ */
+ if (event->attr.config != PERF_COUNT_HW_CPU_CYCLES)
+ return -ENOENT;
+ if (!is_sampling_event(event))
+ return -ENOENT;
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ /* Check online status of the CPU to which the event is pinned */
+ if (event->cpu >= nr_cpumask_bits ||
+ (event->cpu >= 0 && !cpu_online(event->cpu)))
+ return -ENODEV;
+
+ /* Force reset of idle/hv excludes regardless of what the
+ * user requested.
+ */
+ if (event->attr.exclude_hv)
+ event->attr.exclude_hv = 0;
+ if (event->attr.exclude_idle)
+ event->attr.exclude_idle = 0;
+
+ err = __hw_perf_event_init(event);
+ if (unlikely(err))
+ if (event->destroy)
+ event->destroy(event);
+ return err;
+}
+
+static void cpumsf_pmu_enable(struct pmu *pmu)
+{
+ struct cpu_hw_sf *cpuhw = &__get_cpu_var(cpu_hw_sf);
+ struct hw_perf_event *hwc;
+ int err;
+
+ if (cpuhw->flags & PMU_F_ENABLED)
+ return;
+
+ if (cpuhw->flags & PMU_F_ERR_MASK)
+ return;
+
+ /* Check whether to extent the sampling buffer.
+ *
+ * Two conditions trigger an increase of the sampling buffer for a
+ * perf event:
+ * 1. Postponed buffer allocations from the event initialization.
+ * 2. Sampling overflows that contribute to pending allocations.
+ *
+ * Note that the extend_sampling_buffer() function disables the sampling
+ * facility, but it can be fully re-enabled using sampling controls that
+ * have been saved in cpumsf_pmu_disable().
+ */
+ if (cpuhw->event) {
+ hwc = &cpuhw->event->hw;
+ /* Account number of overflow-designated buffer extents */
+ sfb_account_overflows(cpuhw, hwc);
+ if (sfb_has_pending_allocs(&cpuhw->sfb, hwc))
+ extend_sampling_buffer(&cpuhw->sfb, hwc);
+ }
+
+ /* (Re)enable the PMU and sampling facility */
+ cpuhw->flags |= PMU_F_ENABLED;
+ barrier();
+
+ err = lsctl(&cpuhw->lsctl);
+ if (err) {
+ cpuhw->flags &= ~PMU_F_ENABLED;
+ pr_err("Loading sampling controls failed: op=%i err=%i\n",
+ 1, err);
+ return;
+ }
+
+ debug_sprintf_event(sfdbg, 6, "pmu_enable: es=%i cs=%i ed=%i cd=%i "
+ "tear=%p dear=%p\n", cpuhw->lsctl.es, cpuhw->lsctl.cs,
+ cpuhw->lsctl.ed, cpuhw->lsctl.cd,
+ (void *) cpuhw->lsctl.tear, (void *) cpuhw->lsctl.dear);
+}
+
+static void cpumsf_pmu_disable(struct pmu *pmu)
+{
+ struct cpu_hw_sf *cpuhw = &__get_cpu_var(cpu_hw_sf);
+ struct hws_lsctl_request_block inactive;
+ struct hws_qsi_info_block si;
+ int err;
+
+ if (!(cpuhw->flags & PMU_F_ENABLED))
+ return;
+
+ if (cpuhw->flags & PMU_F_ERR_MASK)
+ return;
+
+ /* Switch off sampling activation control */
+ inactive = cpuhw->lsctl;
+ inactive.cs = 0;
+ inactive.cd = 0;
+
+ err = lsctl(&inactive);
+ if (err) {
+ pr_err("Loading sampling controls failed: op=%i err=%i\n",
+ 2, err);
+ return;
+ }
+
+ /* Save state of TEAR and DEAR register contents */
+ if (!qsi(&si)) {
+ /* TEAR/DEAR values are valid only if the sampling facility is
+ * enabled. Note that cpumsf_pmu_disable() might be called even
+ * for a disabled sampling facility because cpumsf_pmu_enable()
+ * controls the enable/disable state.
+ */
+ if (si.es) {
+ cpuhw->lsctl.tear = si.tear;
+ cpuhw->lsctl.dear = si.dear;
+ }
+ } else
+ debug_sprintf_event(sfdbg, 3, "cpumsf_pmu_disable: "
+ "qsi() failed with err=%i\n", err);
+
+ cpuhw->flags &= ~PMU_F_ENABLED;
+}
+
+/* perf_exclude_event() - Filter event
+ * @event: The perf event
+ * @regs: pt_regs structure
+ * @sde_regs: Sample-data-entry (sde) regs structure
+ *
+ * Filter perf events according to their exclude specification.
+ *
+ * Return non-zero if the event shall be excluded.
+ */
+static int perf_exclude_event(struct perf_event *event, struct pt_regs *regs,
+ struct perf_sf_sde_regs *sde_regs)
+{
+ if (event->attr.exclude_user && user_mode(regs))
+ return 1;
+ if (event->attr.exclude_kernel && !user_mode(regs))
+ return 1;
+ if (event->attr.exclude_guest && sde_regs->in_guest)
+ return 1;
+ if (event->attr.exclude_host && !sde_regs->in_guest)
+ return 1;
+ return 0;
+}
+
+/* perf_push_sample() - Push samples to perf
+ * @event: The perf event
+ * @sample: Hardware sample data
+ *
+ * Use the hardware sample data to create perf event sample. The sample
+ * is the pushed to the event subsystem and the function checks for
+ * possible event overflows. If an event overflow occurs, the PMU is
+ * stopped.
+ *
+ * Return non-zero if an event overflow occurred.
+ */
+static int perf_push_sample(struct perf_event *event, struct sf_raw_sample *sfr)
+{
+ int overflow;
+ struct pt_regs regs;
+ struct perf_sf_sde_regs *sde_regs;
+ struct perf_sample_data data;
+ struct perf_raw_record raw;
+
+ /* Setup perf sample */
+ perf_sample_data_init(&data, 0, event->hw.last_period);
+ raw.size = sfr->size;
+ raw.data = sfr;
+ data.raw = &raw;
+
+ /* Setup pt_regs to look like an CPU-measurement external interrupt
+ * using the Program Request Alert code. The regs.int_parm_long
+ * field which is unused contains additional sample-data-entry related
+ * indicators.
+ */
+ memset(®s, 0, sizeof(regs));
+ regs.int_code = 0x1407;
+ regs.int_parm = CPU_MF_INT_SF_PRA;
+ sde_regs = (struct perf_sf_sde_regs *) ®s.int_parm_long;
+
+ regs.psw.addr = sfr->basic.ia;
+ if (sfr->basic.T)
+ regs.psw.mask |= PSW_MASK_DAT;
+ if (sfr->basic.W)
+ regs.psw.mask |= PSW_MASK_WAIT;
+ if (sfr->basic.P)
+ regs.psw.mask |= PSW_MASK_PSTATE;
+ switch (sfr->basic.AS) {
+ case 0x0:
+ regs.psw.mask |= PSW_ASC_PRIMARY;
+ break;
+ case 0x1:
+ regs.psw.mask |= PSW_ASC_ACCREG;
+ break;
+ case 0x2:
+ regs.psw.mask |= PSW_ASC_SECONDARY;
+ break;
+ case 0x3:
+ regs.psw.mask |= PSW_ASC_HOME;
+ break;
+ }
+
+ /* The host-program-parameter (hpp) contains the sie control
+ * block that is set by sie64a() in entry64.S. Check if hpp
+ * refers to a valid control block and set sde_regs flags
+ * accordingly. This would allow to use hpp values for other
+ * purposes too.
+ * For now, simply use a non-zero value as guest indicator.
+ */
+ if (sfr->basic.hpp)
+ sde_regs->in_guest = 1;
+
+ overflow = 0;
+ if (perf_exclude_event(event, ®s, sde_regs))
+ goto out;
+ if (perf_event_overflow(event, &data, ®s)) {
+ overflow = 1;
+ event->pmu->stop(event, 0);
+ }
+ perf_event_update_userpage(event);
+out:
+ return overflow;
+}
+
+static void perf_event_count_update(struct perf_event *event, u64 count)
+{
+ local64_add(count, &event->count);
+}
+
+static int sample_format_is_valid(struct hws_combined_entry *sample,
+ unsigned int flags)
+{
+ if (likely(flags & PERF_CPUM_SF_BASIC_MODE))
+ /* Only basic-sampling data entries with data-entry-format
+ * version of 0x0001 can be processed.
+ */
+ if (sample->basic.def != 0x0001)
+ return 0;
+ if (flags & PERF_CPUM_SF_DIAG_MODE)
+ /* The data-entry-format number of diagnostic-sampling data
+ * entries can vary. Because diagnostic data is just passed
+ * through, do only a sanity check on the DEF.
+ */
+ if (sample->diag.def < 0x8001)
+ return 0;
+ return 1;
+}
+
+static int sample_is_consistent(struct hws_combined_entry *sample,
+ unsigned long flags)
+{
+ /* This check applies only to basic-sampling data entries of potentially
+ * combined-sampling data entries. Invalid entries cannot be processed
+ * by the PMU and, thus, do not deliver an associated
+ * diagnostic-sampling data entry.
+ */
+ if (unlikely(!(flags & PERF_CPUM_SF_BASIC_MODE)))
+ return 0;
+ /*
+ * Samples are skipped, if they are invalid or for which the
+ * instruction address is not predictable, i.e., the wait-state bit is
+ * set.
+ */
+ if (sample->basic.I || sample->basic.W)
+ return 0;
+ return 1;
+}
+
+static void reset_sample_slot(struct hws_combined_entry *sample,
+ unsigned long flags)
+{
+ if (likely(flags & PERF_CPUM_SF_BASIC_MODE))
+ sample->basic.def = 0;
+ if (flags & PERF_CPUM_SF_DIAG_MODE)
+ sample->diag.def = 0;
+}
+
+static void sfr_store_sample(struct sf_raw_sample *sfr,
+ struct hws_combined_entry *sample)
+{
+ if (likely(sfr->format & PERF_CPUM_SF_BASIC_MODE))
+ sfr->basic = sample->basic;
+ if (sfr->format & PERF_CPUM_SF_DIAG_MODE)
+ memcpy(&sfr->diag, &sample->diag, sfr->dsdes);
+}
+
+static void debug_sample_entry(struct hws_combined_entry *sample,
+ struct hws_trailer_entry *te,
+ unsigned long flags)
+{
+ debug_sprintf_event(sfdbg, 4, "hw_collect_samples: Found unknown "
+ "sampling data entry: te->f=%i basic.def=%04x (%p)"
+ " diag.def=%04x (%p)\n", te->f,
+ sample->basic.def, &sample->basic,
+ (flags & PERF_CPUM_SF_DIAG_MODE)
+ ? sample->diag.def : 0xFFFF,
+ (flags & PERF_CPUM_SF_DIAG_MODE)
+ ? &sample->diag : NULL);
+}
+
+/* hw_collect_samples() - Walk through a sample-data-block and collect samples
+ * @event: The perf event
+ * @sdbt: Sample-data-block table
+ * @overflow: Event overflow counter
+ *
+ * Walks through a sample-data-block and collects sampling data entries that are
+ * then pushed to the perf event subsystem. Depending on the sampling function,
+ * there can be either basic-sampling or combined-sampling data entries. A
+ * combined-sampling data entry consists of a basic- and a diagnostic-sampling
+ * data entry. The sampling function is determined by the flags in the perf
+ * event hardware structure. The function always works with a combined-sampling
+ * data entry but ignores the the diagnostic portion if it is not available.
+ *
+ * Note that the implementation focuses on basic-sampling data entries and, if
+ * such an entry is not valid, the entire combined-sampling data entry is
+ * ignored.
+ *
+ * The overflow variables counts the number of samples that has been discarded
+ * due to a perf event overflow.
+ */
+static void hw_collect_samples(struct perf_event *event, unsigned long *sdbt,
+ unsigned long long *overflow)
+{
+ unsigned long flags = SAMPL_FLAGS(&event->hw);
+ struct hws_combined_entry *sample;
+ struct hws_trailer_entry *te;
+ struct sf_raw_sample *sfr;
+ size_t sample_size;
+
+ /* Prepare and initialize raw sample data */
+ sfr = (struct sf_raw_sample *) RAWSAMPLE_REG(&event->hw);
+ sfr->format = flags & PERF_CPUM_SF_MODE_MASK;
+
+ sample_size = event_sample_size(&event->hw);
+ te = (struct hws_trailer_entry *) trailer_entry_ptr(*sdbt);
+ sample = (struct hws_combined_entry *) *sdbt;
+ while ((unsigned long *) sample < (unsigned long *) te) {
+ /* Check for an empty sample */
+ if (!sample->basic.def)
+ break;
+
+ /* Update perf event period */
+ perf_event_count_update(event, SAMPL_RATE(&event->hw));
+
+ /* Check sampling data entry */
+ if (sample_format_is_valid(sample, flags)) {
+ /* If an event overflow occurred, the PMU is stopped to
+ * throttle event delivery. Remaining sample data is
+ * discarded.
+ */
+ if (!*overflow) {
+ if (sample_is_consistent(sample, flags)) {
+ /* Deliver sample data to perf */
+ sfr_store_sample(sfr, sample);
+ *overflow = perf_push_sample(event, sfr);
+ }
+ } else
+ /* Count discarded samples */
+ *overflow += 1;
+ } else {
+ debug_sample_entry(sample, te, flags);
+ /* Sample slot is not yet written or other record.
+ *
+ * This condition can occur if the buffer was reused
+ * from a combined basic- and diagnostic-sampling.
+ * If only basic-sampling is then active, entries are
+ * written into the larger diagnostic entries.
+ * This is typically the case for sample-data-blocks
+ * that are not full. Stop processing if the first
+ * invalid format was detected.
+ */
+ if (!te->f)
+ break;
+ }
+
+ /* Reset sample slot and advance to next sample */
+ reset_sample_slot(sample, flags);
+ sample += sample_size;
+ }
+}
+
+/* hw_perf_event_update() - Process sampling buffer
+ * @event: The perf event
+ * @flush_all: Flag to also flush partially filled sample-data-blocks
+ *
+ * Processes the sampling buffer and create perf event samples.
+ * The sampling buffer position are retrieved and saved in the TEAR_REG
+ * register of the specified perf event.
+ *
+ * Only full sample-data-blocks are processed. Specify the flash_all flag
+ * to also walk through partially filled sample-data-blocks. It is ignored
+ * if PERF_CPUM_SF_FULL_BLOCKS is set. The PERF_CPUM_SF_FULL_BLOCKS flag
+ * enforces the processing of full sample-data-blocks only (trailer entries
+ * with the block-full-indicator bit set).
+ */
+static void hw_perf_event_update(struct perf_event *event, int flush_all)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct hws_trailer_entry *te;
+ unsigned long *sdbt;
+ unsigned long long event_overflow, sampl_overflow, num_sdb, te_flags;
+ int done;
+
+ if (flush_all && SDB_FULL_BLOCKS(hwc))
+ flush_all = 0;
+
+ sdbt = (unsigned long *) TEAR_REG(hwc);
+ done = event_overflow = sampl_overflow = num_sdb = 0;
+ while (!done) {
+ /* Get the trailer entry of the sample-data-block */
+ te = (struct hws_trailer_entry *) trailer_entry_ptr(*sdbt);
+
+ /* Leave loop if no more work to do (block full indicator) */
+ if (!te->f) {
+ done = 1;
+ if (!flush_all)
+ break;
+ }
+
+ /* Check the sample overflow count */
+ if (te->overflow)
+ /* Account sample overflows and, if a particular limit
+ * is reached, extend the sampling buffer.
+ * For details, see sfb_account_overflows().
+ */
+ sampl_overflow += te->overflow;
+
+ /* Timestamps are valid for full sample-data-blocks only */
+ debug_sprintf_event(sfdbg, 6, "hw_perf_event_update: sdbt=%p "
+ "overflow=%llu timestamp=0x%llx\n",
+ sdbt, te->overflow,
+ (te->f) ? trailer_timestamp(te) : 0ULL);
+
+ /* Collect all samples from a single sample-data-block and
+ * flag if an (perf) event overflow happened. If so, the PMU
+ * is stopped and remaining samples will be discarded.
+ */
+ hw_collect_samples(event, sdbt, &event_overflow);
+ num_sdb++;
+
+ /* Reset trailer (using compare-double-and-swap) */
+ do {
+ te_flags = te->flags & ~SDB_TE_BUFFER_FULL_MASK;
+ te_flags |= SDB_TE_ALERT_REQ_MASK;
+ } while (!cmpxchg_double(&te->flags, &te->overflow,
+ te->flags, te->overflow,
+ te_flags, 0ULL));
+
+ /* Advance to next sample-data-block */
+ sdbt++;
+ if (is_link_entry(sdbt))
+ sdbt = get_next_sdbt(sdbt);
+
+ /* Update event hardware registers */
+ TEAR_REG(hwc) = (unsigned long) sdbt;
+
+ /* Stop processing sample-data if all samples of the current
+ * sample-data-block were flushed even if it was not full.
+ */
+ if (flush_all && done)
+ break;
+
+ /* If an event overflow happened, discard samples by
+ * processing any remaining sample-data-blocks.
+ */
+ if (event_overflow)
+ flush_all = 1;
+ }
+
+ /* Account sample overflows in the event hardware structure */
+ if (sampl_overflow)
+ OVERFLOW_REG(hwc) = DIV_ROUND_UP(OVERFLOW_REG(hwc) +
+ sampl_overflow, 1 + num_sdb);
+ if (sampl_overflow || event_overflow)
+ debug_sprintf_event(sfdbg, 4, "hw_perf_event_update: "
+ "overflow stats: sample=%llu event=%llu\n",
+ sampl_overflow, event_overflow);
+}
+
+static void cpumsf_pmu_read(struct perf_event *event)
+{
+ /* Nothing to do ... updates are interrupt-driven */
+}
+
+/* Activate sampling control.
+ * Next call of pmu_enable() starts sampling.
+ */
+static void cpumsf_pmu_start(struct perf_event *event, int flags)
+{
+ struct cpu_hw_sf *cpuhw = &__get_cpu_var(cpu_hw_sf);
+
+ if (WARN_ON_ONCE(!(event->hw.state & PERF_HES_STOPPED)))
+ return;
+
+ if (flags & PERF_EF_RELOAD)
+ WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE));
+
+ perf_pmu_disable(event->pmu);
+ event->hw.state = 0;
+ cpuhw->lsctl.cs = 1;
+ if (SAMPL_DIAG_MODE(&event->hw))
+ cpuhw->lsctl.cd = 1;
+ perf_pmu_enable(event->pmu);
+}
+
+/* Deactivate sampling control.
+ * Next call of pmu_enable() stops sampling.
+ */
+static void cpumsf_pmu_stop(struct perf_event *event, int flags)
+{
+ struct cpu_hw_sf *cpuhw = &__get_cpu_var(cpu_hw_sf);
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ perf_pmu_disable(event->pmu);
+ cpuhw->lsctl.cs = 0;
+ cpuhw->lsctl.cd = 0;
+ event->hw.state |= PERF_HES_STOPPED;
+
+ if ((flags & PERF_EF_UPDATE) && !(event->hw.state & PERF_HES_UPTODATE)) {
+ hw_perf_event_update(event, 1);
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+ perf_pmu_enable(event->pmu);
+}
+
+static int cpumsf_pmu_add(struct perf_event *event, int flags)
+{
+ struct cpu_hw_sf *cpuhw = &__get_cpu_var(cpu_hw_sf);
+ int err;
+
+ if (cpuhw->flags & PMU_F_IN_USE)
+ return -EAGAIN;
+
+ if (!cpuhw->sfb.sdbt)
+ return -EINVAL;
+
+ err = 0;
+ perf_pmu_disable(event->pmu);
+
+ event->hw.state = PERF_HES_UPTODATE | PERF_HES_STOPPED;
+
+ /* Set up sampling controls. Always program the sampling register
+ * using the SDB-table start. Reset TEAR_REG event hardware register
+ * that is used by hw_perf_event_update() to store the sampling buffer
+ * position after samples have been flushed.
+ */
+ cpuhw->lsctl.s = 0;
+ cpuhw->lsctl.h = 1;
+ cpuhw->lsctl.tear = (unsigned long) cpuhw->sfb.sdbt;
+ cpuhw->lsctl.dear = *(unsigned long *) cpuhw->sfb.sdbt;
+ cpuhw->lsctl.interval = SAMPL_RATE(&event->hw);
+ hw_reset_registers(&event->hw, cpuhw->sfb.sdbt);
+
+ /* Ensure sampling functions are in the disabled state. If disabled,
+ * switch on sampling enable control. */
+ if (WARN_ON_ONCE(cpuhw->lsctl.es == 1 || cpuhw->lsctl.ed == 1)) {
+ err = -EAGAIN;
+ goto out;
+ }
+ cpuhw->lsctl.es = 1;
+ if (SAMPL_DIAG_MODE(&event->hw))
+ cpuhw->lsctl.ed = 1;
+
+ /* Set in_use flag and store event */
+ event->hw.idx = 0; /* only one sampling event per CPU supported */
+ cpuhw->event = event;
+ cpuhw->flags |= PMU_F_IN_USE;
+
+ if (flags & PERF_EF_START)
+ cpumsf_pmu_start(event, PERF_EF_RELOAD);
+out:
+ perf_event_update_userpage(event);
+ perf_pmu_enable(event->pmu);
+ return err;
+}
+
+static void cpumsf_pmu_del(struct perf_event *event, int flags)
+{
+ struct cpu_hw_sf *cpuhw = &__get_cpu_var(cpu_hw_sf);
+
+ perf_pmu_disable(event->pmu);
+ cpumsf_pmu_stop(event, PERF_EF_UPDATE);
+
+ cpuhw->lsctl.es = 0;
+ cpuhw->lsctl.ed = 0;
+ cpuhw->flags &= ~PMU_F_IN_USE;
+ cpuhw->event = NULL;
+
+ perf_event_update_userpage(event);
+ perf_pmu_enable(event->pmu);
+}
+
+static int cpumsf_pmu_event_idx(struct perf_event *event)
+{
+ return event->hw.idx;
+}
+
+CPUMF_EVENT_ATTR(SF, SF_CYCLES_BASIC, PERF_EVENT_CPUM_SF);
+CPUMF_EVENT_ATTR(SF, SF_CYCLES_BASIC_DIAG, PERF_EVENT_CPUM_SF_DIAG);
+
+static struct attribute *cpumsf_pmu_events_attr[] = {
+ CPUMF_EVENT_PTR(SF, SF_CYCLES_BASIC),
+ CPUMF_EVENT_PTR(SF, SF_CYCLES_BASIC_DIAG),
+ NULL,
+};
+
+PMU_FORMAT_ATTR(event, "config:0-63");
+
+static struct attribute *cpumsf_pmu_format_attr[] = {
+ &format_attr_event.attr,
+ NULL,
+};
+
+static struct attribute_group cpumsf_pmu_events_group = {
+ .name = "events",
+ .attrs = cpumsf_pmu_events_attr,
+};
+static struct attribute_group cpumsf_pmu_format_group = {
+ .name = "format",
+ .attrs = cpumsf_pmu_format_attr,
+};
+static const struct attribute_group *cpumsf_pmu_attr_groups[] = {
+ &cpumsf_pmu_events_group,
+ &cpumsf_pmu_format_group,
+ NULL,
+};
+
+static struct pmu cpumf_sampling = {
+ .pmu_enable = cpumsf_pmu_enable,
+ .pmu_disable = cpumsf_pmu_disable,
+
+ .event_init = cpumsf_pmu_event_init,
+ .add = cpumsf_pmu_add,
+ .del = cpumsf_pmu_del,
+
+ .start = cpumsf_pmu_start,
+ .stop = cpumsf_pmu_stop,
+ .read = cpumsf_pmu_read,
+
+ .event_idx = cpumsf_pmu_event_idx,
+ .attr_groups = cpumsf_pmu_attr_groups,
+};
+
+static void cpumf_measurement_alert(struct ext_code ext_code,
+ unsigned int alert, unsigned long unused)
+{
+ struct cpu_hw_sf *cpuhw;
+
+ if (!(alert & CPU_MF_INT_SF_MASK))
+ return;
+ inc_irq_stat(IRQEXT_CMS);
+ cpuhw = &__get_cpu_var(cpu_hw_sf);
+
+ /* Measurement alerts are shared and might happen when the PMU
+ * is not reserved. Ignore these alerts in this case. */
+ if (!(cpuhw->flags & PMU_F_RESERVED))
+ return;
+
+ /* The processing below must take care of multiple alert events that
+ * might be indicated concurrently. */
+
+ /* Program alert request */
+ if (alert & CPU_MF_INT_SF_PRA) {
+ if (cpuhw->flags & PMU_F_IN_USE)
+ hw_perf_event_update(cpuhw->event, 0);
+ else
+ WARN_ON_ONCE(!(cpuhw->flags & PMU_F_IN_USE));
+ }
+
+ /* Report measurement alerts only for non-PRA codes */
+ if (alert != CPU_MF_INT_SF_PRA)
+ debug_sprintf_event(sfdbg, 6, "measurement alert: 0x%x\n", alert);
+
+ /* Sampling authorization change request */
+ if (alert & CPU_MF_INT_SF_SACA)
+ qsi(&cpuhw->qsi);
+
+ /* Loss of sample data due to high-priority machine activities */
+ if (alert & CPU_MF_INT_SF_LSDA) {
+ pr_err("Sample data was lost\n");
+ cpuhw->flags |= PMU_F_ERR_LSDA;
+ sf_disable();
+ }
+
+ /* Invalid sampling buffer entry */
+ if (alert & (CPU_MF_INT_SF_IAE|CPU_MF_INT_SF_ISE)) {
+ pr_err("A sampling buffer entry is incorrect (alert=0x%x)\n",
+ alert);
+ cpuhw->flags |= PMU_F_ERR_IBE;
+ sf_disable();
+ }
+}
+
+static int cpumf_pmu_notifier(struct notifier_block *self,
+ unsigned long action, void *hcpu)
+{
+ unsigned int cpu = (long) hcpu;
+ int flags;
+
+ /* Ignore the notification if no events are scheduled on the PMU.
+ * This might be racy...
+ */
+ if (!atomic_read(&num_events))
+ return NOTIFY_OK;
+
+ switch (action & ~CPU_TASKS_FROZEN) {
+ case CPU_ONLINE:
+ case CPU_ONLINE_FROZEN:
+ flags = PMC_INIT;
+ smp_call_function_single(cpu, setup_pmc_cpu, &flags, 1);
+ break;
+ case CPU_DOWN_PREPARE:
+ flags = PMC_RELEASE;
+ smp_call_function_single(cpu, setup_pmc_cpu, &flags, 1);
+ break;
+ default:
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static int param_get_sfb_size(char *buffer, const struct kernel_param *kp)
+{
+ if (!cpum_sf_avail())
+ return -ENODEV;
+ return sprintf(buffer, "%lu,%lu", CPUM_SF_MIN_SDB, CPUM_SF_MAX_SDB);
+}
+
+static int param_set_sfb_size(const char *val, const struct kernel_param *kp)
+{
+ int rc;
+ unsigned long min, max;
+
+ if (!cpum_sf_avail())
+ return -ENODEV;
+ if (!val || !strlen(val))
+ return -EINVAL;
+
+ /* Valid parameter values: "min,max" or "max" */
+ min = CPUM_SF_MIN_SDB;
+ max = CPUM_SF_MAX_SDB;
+ if (strchr(val, ','))
+ rc = (sscanf(val, "%lu,%lu", &min, &max) == 2) ? 0 : -EINVAL;
+ else
+ rc = kstrtoul(val, 10, &max);
+
+ if (min < 2 || min >= max || max > get_num_physpages())
+ rc = -EINVAL;
+ if (rc)
+ return rc;
+
+ sfb_set_limits(min, max);
+ pr_info("The sampling buffer limits have changed to: "
+ "min=%lu max=%lu (diag=x%lu)\n",
+ CPUM_SF_MIN_SDB, CPUM_SF_MAX_SDB, CPUM_SF_SDB_DIAG_FACTOR);
+ return 0;
+}
+
+#define param_check_sfb_size(name, p) __param_check(name, p, void)
+static struct kernel_param_ops param_ops_sfb_size = {
+ .set = param_set_sfb_size,
+ .get = param_get_sfb_size,
+};
+
+#define RS_INIT_FAILURE_QSI 0x0001
+#define RS_INIT_FAILURE_BSDES 0x0002
+#define RS_INIT_FAILURE_ALRT 0x0003
+#define RS_INIT_FAILURE_PERF 0x0004
+static void __init pr_cpumsf_err(unsigned int reason)
+{
+ pr_err("Sampling facility support for perf is not available: "
+ "reason=%04x\n", reason);
+}
+
+static int __init init_cpum_sampling_pmu(void)
+{
+ struct hws_qsi_info_block si;
+ int err;
+
+ if (!cpum_sf_avail())
+ return -ENODEV;
+
+ memset(&si, 0, sizeof(si));
+ if (qsi(&si)) {
+ pr_cpumsf_err(RS_INIT_FAILURE_QSI);
+ return -ENODEV;
+ }
+
+ if (si.bsdes != sizeof(struct hws_basic_entry)) {
+ pr_cpumsf_err(RS_INIT_FAILURE_BSDES);
+ return -EINVAL;
+ }
+
+ if (si.ad)
+ sfb_set_limits(CPUM_SF_MIN_SDB, CPUM_SF_MAX_SDB);
+
+ sfdbg = debug_register(KMSG_COMPONENT, 2, 1, 80);
+ if (!sfdbg)
+ pr_err("Registering for s390dbf failed\n");
+ debug_register_view(sfdbg, &debug_sprintf_view);
+
+ err = register_external_interrupt(0x1407, cpumf_measurement_alert);
+ if (err) {
+ pr_cpumsf_err(RS_INIT_FAILURE_ALRT);
+ goto out;
+ }
+
+ err = perf_pmu_register(&cpumf_sampling, "cpum_sf", PERF_TYPE_RAW);
+ if (err) {
+ pr_cpumsf_err(RS_INIT_FAILURE_PERF);
+ unregister_external_interrupt(0x1407, cpumf_measurement_alert);
+ goto out;
+ }
+ perf_cpu_notifier(cpumf_pmu_notifier);
+out:
+ return err;
+}
+arch_initcall(init_cpum_sampling_pmu);
+core_param(cpum_sfb_size, CPUM_SF_MAX_SDB, sfb_size, 0640);
/*
* Performance event support for s390x
*
- * Copyright IBM Corp. 2012
+ * Copyright IBM Corp. 2012, 2013
* Author(s): Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
*
* This program is free software; you can redistribute it and/or modify
#include <linux/kvm_host.h>
#include <linux/percpu.h>
#include <linux/export.h>
+#include <linux/seq_file.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
#include <asm/irq.h>
#include <asm/cpu_mf.h>
#include <asm/lowcore.h>
#include <asm/processor.h>
+#include <asm/sysinfo.h>
const char *perf_pmu_name(void)
{
if (cpum_cf_avail() || cpum_sf_avail())
- return "CPU-measurement facilities (CPUMF)";
+ return "CPU-Measurement Facilities (CPU-MF)";
return "pmu";
}
EXPORT_SYMBOL(perf_pmu_name);
if (cpum_cf_avail())
num += PERF_CPUM_CF_MAX_CTR;
+ if (cpum_sf_avail())
+ num += PERF_CPUM_SF_MAX_CTR;
return num;
}
{
if (user_mode(regs))
return false;
-#if defined(CONFIG_KVM) || defined(CONFIG_KVM_MODULE)
+#if IS_ENABLED(CONFIG_KVM)
return instruction_pointer(regs) == (unsigned long) &sie_exit;
#else
return false;
: PERF_RECORD_MISC_GUEST_KERNEL;
}
+static unsigned long perf_misc_flags_sf(struct pt_regs *regs)
+{
+ struct perf_sf_sde_regs *sde_regs;
+ unsigned long flags;
+
+ sde_regs = (struct perf_sf_sde_regs *) ®s->int_parm_long;
+ if (sde_regs->in_guest)
+ flags = user_mode(regs) ? PERF_RECORD_MISC_GUEST_USER
+ : PERF_RECORD_MISC_GUEST_KERNEL;
+ else
+ flags = user_mode(regs) ? PERF_RECORD_MISC_USER
+ : PERF_RECORD_MISC_KERNEL;
+ return flags;
+}
+
unsigned long perf_misc_flags(struct pt_regs *regs)
{
+ /* Check if the cpum_sf PMU has created the pt_regs structure.
+ * In this case, perf misc flags can be easily extracted. Otherwise,
+ * do regular checks on the pt_regs content.
+ */
+ if (regs->int_code == 0x1407 && regs->int_parm == CPU_MF_INT_SF_PRA)
+ if (!regs->gprs[15])
+ return perf_misc_flags_sf(regs);
+
if (is_in_guest(regs))
return perf_misc_guest_flags(regs);
: PERF_RECORD_MISC_KERNEL;
}
-void perf_event_print_debug(void)
+void print_debug_cf(void)
{
struct cpumf_ctr_info cf_info;
- unsigned long flags;
- int cpu;
-
- if (!cpum_cf_avail())
- return;
-
- local_irq_save(flags);
+ int cpu = smp_processor_id();
- cpu = smp_processor_id();
memset(&cf_info, 0, sizeof(cf_info));
if (!qctri(&cf_info))
pr_info("CPU[%i] CPUM_CF: ver=%u.%u A=%04x E=%04x C=%04x\n",
cpu, cf_info.cfvn, cf_info.csvn,
cf_info.auth_ctl, cf_info.enable_ctl, cf_info.act_ctl);
+}
+
+static void print_debug_sf(void)
+{
+ struct hws_qsi_info_block si;
+ int cpu = smp_processor_id();
+ memset(&si, 0, sizeof(si));
+ if (qsi(&si))
+ return;
+
+ pr_info("CPU[%i] CPUM_SF: basic=%i diag=%i min=%lu max=%lu cpu_speed=%u\n",
+ cpu, si.as, si.ad, si.min_sampl_rate, si.max_sampl_rate,
+ si.cpu_speed);
+
+ if (si.as)
+ pr_info("CPU[%i] CPUM_SF: Basic-sampling: a=%i e=%i c=%i"
+ " bsdes=%i tear=%016lx dear=%016lx\n", cpu,
+ si.as, si.es, si.cs, si.bsdes, si.tear, si.dear);
+ if (si.ad)
+ pr_info("CPU[%i] CPUM_SF: Diagnostic-sampling: a=%i e=%i c=%i"
+ " dsdes=%i tear=%016lx dear=%016lx\n", cpu,
+ si.ad, si.ed, si.cd, si.dsdes, si.tear, si.dear);
+}
+
+void perf_event_print_debug(void)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+ if (cpum_cf_avail())
+ print_debug_cf();
+ if (cpum_sf_avail())
+ print_debug_sf();
local_irq_restore(flags);
}
+/* Service level infrastructure */
+static void sl_print_counter(struct seq_file *m)
+{
+ struct cpumf_ctr_info ci;
+
+ memset(&ci, 0, sizeof(ci));
+ if (qctri(&ci))
+ return;
+
+ seq_printf(m, "CPU-MF: Counter facility: version=%u.%u "
+ "authorization=%04x\n", ci.cfvn, ci.csvn, ci.auth_ctl);
+}
+
+static void sl_print_sampling(struct seq_file *m)
+{
+ struct hws_qsi_info_block si;
+
+ memset(&si, 0, sizeof(si));
+ if (qsi(&si))
+ return;
+
+ if (!si.as && !si.ad)
+ return;
+
+ seq_printf(m, "CPU-MF: Sampling facility: min_rate=%lu max_rate=%lu"
+ " cpu_speed=%u\n", si.min_sampl_rate, si.max_sampl_rate,
+ si.cpu_speed);
+ if (si.as)
+ seq_printf(m, "CPU-MF: Sampling facility: mode=basic"
+ " sample_size=%u\n", si.bsdes);
+ if (si.ad)
+ seq_printf(m, "CPU-MF: Sampling facility: mode=diagnostic"
+ " sample_size=%u\n", si.dsdes);
+}
+
+static void service_level_perf_print(struct seq_file *m,
+ struct service_level *sl)
+{
+ if (cpum_cf_avail())
+ sl_print_counter(m);
+ if (cpum_sf_avail())
+ sl_print_sampling(m);
+}
+
+static struct service_level service_level_perf = {
+ .seq_print = service_level_perf_print,
+};
+
+static int __init service_level_perf_register(void)
+{
+ return register_service_level(&service_level_perf);
+}
+arch_initcall(service_level_perf_register);
+
/* See also arch/s390/kernel/traps.c */
static unsigned long __store_trace(struct perf_callchain_entry *entry,
unsigned long sp,
__store_trace(entry, head, S390_lowcore.thread_info,
S390_lowcore.thread_info + THREAD_SIZE);
}
+
+/* Perf defintions for PMU event attributes in sysfs */
+ssize_t cpumf_events_sysfs_show(struct device *dev,
+ struct device_attribute *attr, char *page)
+{
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr);
+ return sprintf(page, "event=0x%04llx,name=%s\n",
+ pmu_attr->id, attr->attr.name);
+}
+
+/* Reserve/release functions for sharing perf hardware */
+static DEFINE_SPINLOCK(perf_hw_owner_lock);
+static void *perf_sampling_owner;
+
+int perf_reserve_sampling(void)
+{
+ int err;
+
+ err = 0;
+ spin_lock(&perf_hw_owner_lock);
+ if (perf_sampling_owner) {
+ pr_warn("The sampling facility is already reserved by %p\n",
+ perf_sampling_owner);
+ err = -EBUSY;
+ } else
+ perf_sampling_owner = __builtin_return_address(0);
+ spin_unlock(&perf_hw_owner_lock);
+ return err;
+}
+EXPORT_SYMBOL(perf_reserve_sampling);
+
+void perf_release_sampling(void)
+{
+ spin_lock(&perf_hw_owner_lock);
+ WARN_ON(!perf_sampling_owner);
+ perf_sampling_owner = NULL;
+ spin_unlock(&perf_hw_owner_lock);
+}
+EXPORT_SYMBOL(perf_release_sampling);
unsigned long arch_randomize_brk(struct mm_struct *mm)
{
- unsigned long ret = PAGE_ALIGN(mm->brk + brk_rnd());
+ unsigned long ret;
- if (ret < mm->brk)
- return mm->brk;
- return ret;
+ ret = PAGE_ALIGN(mm->brk + brk_rnd());
+ return (ret > mm->brk) ? ret : mm->brk;
}
unsigned long randomize_et_dyn(unsigned long base)
{
- unsigned long ret = PAGE_ALIGN(base + brk_rnd());
+ unsigned long ret;
if (!(current->flags & PF_RANDOMIZE))
return base;
- if (ret < base)
- return base;
- return ret;
+ ret = PAGE_ALIGN(base + brk_rnd());
+ return (ret > base) ? ret : base;
}
#ifdef CONFIG_64BIT
/* Take care of the enable/disable of transactional execution. */
if (MACHINE_HAS_TE) {
- unsigned long cr[3], cr_new[3];
+ unsigned long cr, cr_new;
- __ctl_store(cr, 0, 2);
- cr_new[1] = cr[1];
+ __ctl_store(cr, 0, 0);
/* Set or clear transaction execution TXC bit 8. */
+ cr_new = cr | (1UL << 55);
if (task->thread.per_flags & PER_FLAG_NO_TE)
- cr_new[0] = cr[0] & ~(1UL << 55);
- else
- cr_new[0] = cr[0] | (1UL << 55);
+ cr_new &= ~(1UL << 55);
+ if (cr_new != cr)
+ __ctl_load(cr, 0, 0);
/* Set or clear transaction execution TDC bits 62 and 63. */
- cr_new[2] = cr[2] & ~3UL;
+ __ctl_store(cr, 2, 2);
+ cr_new = cr & ~3UL;
if (task->thread.per_flags & PER_FLAG_TE_ABORT_RAND) {
if (task->thread.per_flags & PER_FLAG_TE_ABORT_RAND_TEND)
- cr_new[2] |= 1UL;
+ cr_new |= 1UL;
else
- cr_new[2] |= 2UL;
+ cr_new |= 2UL;
}
- if (memcmp(&cr_new, &cr, sizeof(cr)))
- __ctl_load(cr_new, 0, 2);
+ if (cr_new != cr)
+ __ctl_load(cr_new, 2, 2);
}
#endif
/* Copy user specified PER registers */
void user_enable_single_step(struct task_struct *task)
{
set_tsk_thread_flag(task, TIF_SINGLE_STEP);
- if (task == current)
- update_cr_regs(task);
}
void user_disable_single_step(struct task_struct *task)
{
clear_tsk_thread_flag(task, TIF_SINGLE_STEP);
- if (task == current)
- update_cr_regs(task);
}
/*
#ifdef CONFIG_FUNCTION_TRACER
EXPORT_SYMBOL(_mcount);
#endif
-#if defined(CONFIG_KVM) || defined(CONFIG_KVM_MODULE)
+#if IS_ENABLED(CONFIG_KVM)
EXPORT_SYMBOL(sie64a);
EXPORT_SYMBOL(sie_exit);
#endif
/*
* Set up PSW restart to call ipl.c:do_restart(). Copy the relevant
- * restart data to the absolute zero lowcore. This is necesary if
+ * restart data to the absolute zero lowcore. This is necessary if
* PSW restart is done on an offline CPU that has lowcore zero.
*/
lc->restart_stack = (unsigned long) restart_stack;
};
struct pcpu {
- struct cpu cpu;
+ struct cpu *cpu;
struct _lowcore *lowcore; /* lowcore page(s) for the cpu */
unsigned long async_stack; /* async stack for the cpu */
unsigned long panic_stack; /* panic stack for the cpu */
{
int order;
- set_bit(ec_bit, &pcpu->ec_mask);
- order = pcpu_running(pcpu) ?
- SIGP_EXTERNAL_CALL : SIGP_EMERGENCY_SIGNAL;
+ if (test_and_set_bit(ec_bit, &pcpu->ec_mask))
+ return;
+ order = pcpu_running(pcpu) ? SIGP_EXTERNAL_CALL : SIGP_EMERGENCY_SIGNAL;
pcpu_sigp_retry(pcpu, order, 0);
}
void *hcpu)
{
unsigned int cpu = (unsigned int)(long)hcpu;
- struct cpu *c = &pcpu_devices[cpu].cpu;
+ struct cpu *c = pcpu_devices[cpu].cpu;
struct device *s = &c->dev;
int err = 0;
static int smp_add_present_cpu(int cpu)
{
- struct cpu *c = &pcpu_devices[cpu].cpu;
- struct device *s = &c->dev;
+ struct device *s;
+ struct cpu *c;
int rc;
+ c = kzalloc(sizeof(*c), GFP_KERNEL);
+ if (!c)
+ return -ENOMEM;
+ pcpu_devices[cpu].cpu = c;
+ s = &c->dev;
c->hotpluggable = 1;
rc = register_cpu(c, cpu);
if (rc)
return -EOPNOTSUPP;
} else {
/*
- * Set condition code 3 to stop the guest from issueing channel
+ * Set condition code 3 to stop the guest from issuing channel
* I/O instructions.
*/
kvm_s390_set_psw_cc(vcpu, 3);
/*
* Returns kernel address for user virtual address. If the returned address is
- * >= -4095 (IS_ERR_VALUE(x) returns true), a fault has occured and the address
- * contains the (negative) exception code.
+ * >= -4095 (IS_ERR_VALUE(x) returns true), a fault has occurred and the
+ * address contains the (negative) exception code.
*/
#ifdef CONFIG_64BIT
* @addr: address in the guest address space
* @len: length of the memory area to unmap
*
- * Returns 0 if the unmap succeded, -EINVAL if not.
+ * Returns 0 if the unmap succeeded, -EINVAL if not.
*/
int gmap_unmap_segment(struct gmap *gmap, unsigned long to, unsigned long len)
{
* @from: source address in the parent address space
* @to: target address in the guest address space
*
- * Returns 0 if the mmap succeded, -EINVAL or -ENOMEM if not.
+ * Returns 0 if the mmap succeeded, -EINVAL or -ENOMEM if not.
*/
int gmap_map_segment(struct gmap *gmap, unsigned long from,
unsigned long to, unsigned long len)
EMIT4_PCREL(0xa7840000, (jit->ret0_ip - jit->prg));
/* lhi %r4,0 */
EMIT4(0xa7480000);
- /* dr %r4,%r12 */
- EMIT2(0x1d4c);
+ /* dlr %r4,%r12 */
+ EMIT4(0xb997004c);
break;
- case BPF_S_ALU_DIV_K: /* A = reciprocal_divide(A, K) */
- /* m %r4,<d(K)>(%r13) */
- EMIT4_DISP(0x5c40d000, EMIT_CONST(K));
- /* lr %r5,%r4 */
- EMIT2(0x1854);
+ case BPF_S_ALU_DIV_K: /* A /= K */
+ if (K == 1)
+ break;
+ /* lhi %r4,0 */
+ EMIT4(0xa7480000);
+ /* dl %r4,<d(K)>(%r13) */
+ EMIT6_DISP(0xe340d000, 0x0097, EMIT_CONST(K));
break;
case BPF_S_ALU_MOD_X: /* A %= X */
jit->seen |= SEEN_XREG | SEEN_RET0;
EMIT4_PCREL(0xa7840000, (jit->ret0_ip - jit->prg));
/* lhi %r4,0 */
EMIT4(0xa7480000);
- /* dr %r4,%r12 */
- EMIT2(0x1d4c);
+ /* dlr %r4,%r12 */
+ EMIT4(0xb997004c);
/* lr %r5,%r4 */
EMIT2(0x1854);
break;
case BPF_S_ALU_MOD_K: /* A %= K */
+ if (K == 1) {
+ /* lhi %r5,0 */
+ EMIT4(0xa7580000);
+ break;
+ }
/* lhi %r4,0 */
EMIT4(0xa7480000);
- /* d %r4,<d(K)>(%r13) */
- EMIT4_DISP(0x5d40d000, EMIT_CONST(K));
+ /* dl %r4,<d(K)>(%r13) */
+ EMIT6_DISP(0xe340d000, 0x0097, EMIT_CONST(K));
/* lr %r5,%r4 */
EMIT2(0x1854);
break;
#define MAX_NUM_SDB 511
#define MIN_NUM_SDB 1
-#define ALERT_REQ_MASK 0x4000000000000000ul
-#define BUFFER_FULL_MASK 0x8000000000000000ul
-
DECLARE_PER_CPU(struct hws_cpu_buffer, sampler_cpu_buffer);
struct hws_execute_parms {
static unsigned char hws_flush_all;
static unsigned int hws_oom;
+static unsigned int hws_alert;
static struct workqueue_struct *hws_wq;
static unsigned int hws_state;
static unsigned long min_sampler_rate;
static unsigned long max_sampler_rate;
-static int ssctl(void *buffer)
-{
- int cc;
-
- /* set in order to detect a program check */
- cc = 1;
-
- asm volatile(
- "0: .insn s,0xB2870000,0(%1)\n"
- "1: ipm %0\n"
- " srl %0,28\n"
- "2:\n"
- EX_TABLE(0b, 2b) EX_TABLE(1b, 2b)
- : "+d" (cc), "+a" (buffer)
- : "m" (*((struct hws_ssctl_request_block *)buffer))
- : "cc", "memory");
-
- return cc ? -EINVAL : 0 ;
-}
-
-static int qsi(void *buffer)
-{
- int cc;
- cc = 1;
-
- asm volatile(
- "0: .insn s,0xB2860000,0(%1)\n"
- "1: lhi %0,0\n"
- "2:\n"
- EX_TABLE(0b, 2b) EX_TABLE(1b, 2b)
- : "=d" (cc), "+a" (buffer)
- : "m" (*((struct hws_qsi_info_block *)buffer))
- : "cc", "memory");
-
- return cc ? -EINVAL : 0;
-}
-
static void execute_qsi(void *parms)
{
struct hws_execute_parms *ep = parms;
{
struct hws_execute_parms *ep = parms;
- ep->rc = ssctl(ep->buffer);
+ ep->rc = lsctl(ep->buffer);
}
static int smp_ctl_ssctl_stop(int cpu)
return ep.rc;
}
-static inline unsigned long *trailer_entry_ptr(unsigned long v)
-{
- void *ret;
-
- ret = (void *)v;
- ret += PAGE_SIZE;
- ret -= sizeof(struct hws_trailer_entry);
-
- return (unsigned long *) ret;
-}
-
static void hws_ext_handler(struct ext_code ext_code,
unsigned int param32, unsigned long param64)
{
if (!(param32 & CPU_MF_INT_SF_MASK))
return;
+ if (!hws_alert)
+ return;
+
inc_irq_stat(IRQEXT_CMS);
atomic_xchg(&cb->ext_params, atomic_read(&cb->ext_params) | param32);
}
}
-static int is_link_entry(unsigned long *s)
-{
- return *s & 0x1ul ? 1 : 0;
-}
-
-static unsigned long *get_next_sdbt(unsigned long *s)
-{
- return (unsigned long *) (*s & ~0x1ul);
-}
-
static int prepare_cpu_buffers(void)
{
int cpu;
}
*sdbt = sdb;
trailer = trailer_entry_ptr(*sdbt);
- *trailer = ALERT_REQ_MASK;
+ *trailer = SDB_TE_ALERT_REQ_MASK;
sdbt++;
mutex_unlock(&hws_sem_oom);
}
trailer = trailer_entry_ptr(*sdbt);
/* leave loop if no more work to do */
- if (!(*trailer & BUFFER_FULL_MASK)) {
+ if (!(*trailer & SDB_TE_BUFFER_FULL_MASK)) {
done = 1;
if (!hws_flush_all)
continue;
static void add_samples_to_oprofile(unsigned int cpu, unsigned long *sdbt,
unsigned long *dear)
{
- struct hws_data_entry *sample_data_ptr;
+ struct hws_basic_entry *sample_data_ptr;
unsigned long *trailer;
trailer = trailer_entry_ptr(*sdbt);
trailer = dear;
}
- sample_data_ptr = (struct hws_data_entry *)(*sdbt);
+ sample_data_ptr = (struct hws_basic_entry *)(*sdbt);
while ((unsigned long *)sample_data_ptr < trailer) {
struct pt_regs *regs = NULL;
goto deallocate_exit;
irq_subclass_unregister(IRQ_SUBCLASS_MEASUREMENT_ALERT);
+ hws_alert = 0;
deallocate_sdbt();
hws_state = HWS_DEALLOCATED;
if (hws_state == HWS_STOPPED) {
irq_subclass_unregister(IRQ_SUBCLASS_MEASUREMENT_ALERT);
+ hws_alert = 0;
deallocate_sdbt();
}
if (hws_wq) {
hws_oom = 1;
hws_flush_all = 0;
/* now let them in, 1407 CPUMF external interrupts */
+ hws_alert = 1;
irq_subclass_register(IRQ_SUBCLASS_MEASUREMENT_ALERT);
return 0;
#define HWSAMPLER_H_
#include <linux/workqueue.h>
-
-struct hws_qsi_info_block /* QUERY SAMPLING information block */
-{ /* Bit(s) */
- unsigned int b0_13:14; /* 0-13: zeros */
- unsigned int as:1; /* 14: sampling authorisation control*/
- unsigned int b15_21:7; /* 15-21: zeros */
- unsigned int es:1; /* 22: sampling enable control */
- unsigned int b23_29:7; /* 23-29: zeros */
- unsigned int cs:1; /* 30: sampling activation control */
- unsigned int:1; /* 31: reserved */
- unsigned int bsdes:16; /* 4-5: size of sampling entry */
- unsigned int:16; /* 6-7: reserved */
- unsigned long min_sampl_rate; /* 8-15: minimum sampling interval */
- unsigned long max_sampl_rate; /* 16-23: maximum sampling interval*/
- unsigned long tear; /* 24-31: TEAR contents */
- unsigned long dear; /* 32-39: DEAR contents */
- unsigned int rsvrd0; /* 40-43: reserved */
- unsigned int cpu_speed; /* 44-47: CPU speed */
- unsigned long long rsvrd1; /* 48-55: reserved */
- unsigned long long rsvrd2; /* 56-63: reserved */
-};
+#include <asm/cpu_mf.h>
struct hws_ssctl_request_block /* SET SAMPLING CONTROLS req block */
{ /* bytes 0 - 7 Bit(s) */
unsigned int stop_mode:1;
};
-struct hws_data_entry {
- unsigned int def:16; /* 0-15 Data Entry Format */
- unsigned int R:4; /* 16-19 reserved */
- unsigned int U:4; /* 20-23 Number of unique instruct. */
- unsigned int z:2; /* zeros */
- unsigned int T:1; /* 26 PSW DAT mode */
- unsigned int W:1; /* 27 PSW wait state */
- unsigned int P:1; /* 28 PSW Problem state */
- unsigned int AS:2; /* 29-30 PSW address-space control */
- unsigned int I:1; /* 31 entry valid or invalid */
- unsigned int:16;
- unsigned int prim_asn:16; /* primary ASN */
- unsigned long long ia; /* Instruction Address */
- unsigned long long gpp; /* Guest Program Parameter */
- unsigned long long hpp; /* Host Program Parameter */
-};
-
-struct hws_trailer_entry {
- unsigned int f:1; /* 0 - Block Full Indicator */
- unsigned int a:1; /* 1 - Alert request control */
- unsigned long:62; /* 2 - 63: Reserved */
- unsigned long overflow; /* 64 - sample Overflow count */
- unsigned long timestamp; /* 16 - time-stamp */
- unsigned long timestamp1; /* */
- unsigned long reserved1; /* 32 -Reserved */
- unsigned long reserved2; /* */
- unsigned long progusage1; /* 48 - reserved for programming use */
- unsigned long progusage2; /* */
-};
-
int hwsampler_setup(void);
int hwsampler_shutdown(void);
int hwsampler_allocate(unsigned long sdbt, unsigned long sdb);
*/
#include <linux/oprofile.h>
+#include <linux/perf_event.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/fs.h>
MODULE_PARM_DESC(cpu_type, "Force legacy basic mode sampling"
"(report cpu_type \"timer\"");
+static int __oprofile_hwsampler_start(void)
+{
+ int retval;
+
+ retval = hwsampler_allocate(oprofile_sdbt_blocks, oprofile_sdb_blocks);
+ if (retval)
+ return retval;
+
+ retval = hwsampler_start_all(oprofile_hw_interval);
+ if (retval)
+ hwsampler_deallocate();
+
+ return retval;
+}
+
static int oprofile_hwsampler_start(void)
{
int retval;
if (!hwsampler_running)
return timer_ops.start();
- retval = hwsampler_allocate(oprofile_sdbt_blocks, oprofile_sdb_blocks);
+ retval = perf_reserve_sampling();
if (retval)
return retval;
- retval = hwsampler_start_all(oprofile_hw_interval);
+ retval = __oprofile_hwsampler_start();
if (retval)
- hwsampler_deallocate();
+ perf_release_sampling();
return retval;
}
hwsampler_stop_all();
hwsampler_deallocate();
+ perf_release_sampling();
return;
}
kmem_cache_destroy(zdev_fmb_cache);
}
-static unsigned int s390_pci_probe;
+static unsigned int s390_pci_probe = 1;
+static unsigned int s390_pci_initialized;
char * __init pcibios_setup(char *str)
{
- if (!strcmp(str, "on")) {
- s390_pci_probe = 1;
+ if (!strcmp(str, "off")) {
+ s390_pci_probe = 0;
return NULL;
}
return str;
}
+bool zpci_is_enabled(void)
+{
+ return s390_pci_initialized;
+}
+
static int __init pci_base_init(void)
{
int rc;
if (rc)
goto out_find;
+ s390_pci_initialized = 1;
return 0;
out_find:
void zpci_rescan(void)
{
- clp_rescan_pci_devices_simple();
+ if (zpci_is_enabled())
+ clp_rescan_pci_devices_simple();
}
flags |= ZPCI_TABLE_PROTECTED;
if (!dma_update_trans(zdev, pa, dma_addr, size, flags)) {
- atomic64_add(nr_pages, (atomic64_t *) &zdev->fmb->mapped_pages);
+ atomic64_add(nr_pages, &zdev->fmb->mapped_pages);
return dma_addr + (offset & ~PAGE_MASK);
}
zpci_err_hex(&dma_addr, sizeof(dma_addr));
}
- atomic64_add(npages, (atomic64_t *) &zdev->fmb->unmapped_pages);
+ atomic64_add(npages, &zdev->fmb->unmapped_pages);
iommu_page_index = (dma_addr - zdev->start_dma) >> PAGE_SHIFT;
dma_free_iommu(zdev, iommu_page_index, npages);
}
if (!page)
return NULL;
- atomic64_add(size / PAGE_SIZE, (atomic64_t *) &zdev->fmb->allocated_pages);
pa = page_to_phys(page);
memset((void *) pa, 0, size);
return NULL;
}
+ atomic64_add(size / PAGE_SIZE, &zdev->fmb->allocated_pages);
if (dma_handle)
*dma_handle = map;
return (void *) pa;
void *pa, dma_addr_t dma_handle,
struct dma_attrs *attrs)
{
- s390_dma_unmap_pages(dev, dma_handle, PAGE_ALIGN(size),
- DMA_BIDIRECTIONAL, NULL);
+ struct zpci_dev *zdev = get_zdev(to_pci_dev(dev));
+
+ size = PAGE_ALIGN(size);
+ atomic64_sub(size / PAGE_SIZE, &zdev->fmb->allocated_pages);
+ s390_dma_unmap_pages(dev, dma_handle, size, DMA_BIDIRECTIONAL, NULL);
free_pages((unsigned long) pa, get_order(size));
}
u16 pec; /* PCI event code */
} __packed;
-void zpci_event_error(void *data)
+static void __zpci_event_error(struct zpci_ccdf_err *ccdf)
{
- struct zpci_ccdf_err *ccdf = data;
struct zpci_dev *zdev = get_zdev_by_fid(ccdf->fid);
zpci_err("error CCDF:\n");
pci_name(zdev->pdev), ccdf->pec, ccdf->fid);
}
-void zpci_event_availability(void *data)
+void zpci_event_error(void *data)
+{
+ if (zpci_is_enabled())
+ __zpci_event_error(data);
+}
+
+static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf)
{
- struct zpci_ccdf_avail *ccdf = data;
struct zpci_dev *zdev = get_zdev_by_fid(ccdf->fid);
struct pci_dev *pdev = zdev ? zdev->pdev : NULL;
int ret;
break;
case 0x0304: /* Configured -> Standby */
- if (pdev)
+ if (pdev) {
+ /* Give the driver a hint that the function is
+ * already unusable. */
+ pdev->error_state = pci_channel_io_perm_failure;
pci_stop_and_remove_bus_device(pdev);
+ }
zdev->fh = ccdf->fh;
zpci_disable_device(zdev);
clp_rescan_pci_devices();
break;
case 0x0308: /* Standby -> Reserved */
+ if (!zdev)
+ break;
pci_stop_root_bus(zdev->bus);
pci_remove_root_bus(zdev->bus);
break;
break;
}
}
+
+void zpci_event_availability(void *data)
+{
+ if (zpci_is_enabled())
+ __zpci_event_availability(data);
+}
header-y +=
+generic-y += barrier.h
generic-y += clkdev.h
generic-y += trace_clock.h
generic-y += xor.h
+++ /dev/null
-#ifndef _ASM_SCORE_BARRIER_H
-#define _ASM_SCORE_BARRIER_H
-
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-
-#define read_barrier_depends() do {} while (0)
-#define smp_read_barrier_depends() do {} while (0)
-
-#define set_mb(var, value) do {var = value; wmb(); } while (0)
-
-#endif /* _ASM_SCORE_BARRIER_H */
select PERF_EVENTS
select ARCH_HIBERNATION_POSSIBLE if MMU
select SPARSE_IRQ
+ select HAVE_CC_STACKPROTECTOR
config SUPERH64
def_bool ARCH = "sh64"
If unsure, say N.
-config CC_STACKPROTECTOR
- bool "Enable -fstack-protector buffer overflow detection (EXPERIMENTAL)"
- depends on SUPERH32
- help
- This option turns on the -fstack-protector GCC feature. This
- feature puts, at the beginning of functions, a canary value on
- the stack just before the return address, and validates
- the value just before actually returning. Stack based buffer
- overflows (that need to overwrite this return address) now also
- overwrite the canary, which gets detected and the attack is then
- neutralized via a kernel panic.
-
- This feature requires gcc version 4.2 or above.
-
config SMP
bool "Symmetric multi-processing support"
depends on SYS_SUPPORTS_SMP
KBUILD_CFLAGS += -fasynchronous-unwind-tables
endif
-ifeq ($(CONFIG_CC_STACKPROTECTOR),y)
- KBUILD_CFLAGS += -fstack-protector
-endif
-
libs-$(CONFIG_SUPERH32) := arch/sh/lib/ $(libs-y)
libs-$(CONFIG_SUPERH64) := arch/sh/lib64/ $(libs-y)
#if defined(CONFIG_CPU_SH4A) || defined(CONFIG_CPU_SH5)
#define mb() __asm__ __volatile__ ("synco": : :"memory")
#define rmb() mb()
-#define wmb() __asm__ __volatile__ ("synco": : :"memory")
+#define wmb() mb()
#define ctrl_barrier() __icbi(PAGE_OFFSET)
-#define read_barrier_depends() do { } while(0)
#else
-#define mb() __asm__ __volatile__ ("": : :"memory")
-#define rmb() mb()
-#define wmb() __asm__ __volatile__ ("": : :"memory")
#define ctrl_barrier() __asm__ __volatile__ ("nop;nop;nop;nop;nop;nop;nop;nop")
-#define read_barrier_depends() do { } while(0)
-#endif
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while(0)
#endif
#define set_mb(var, value) do { (void)xchg(&var, value); } while (0)
+#include <asm-generic/barrier.h>
+
#endif /* __ASM_SH_BARRIER_H */
#ifndef __SPARC_BARRIER_H
#define __SPARC_BARRIER_H
-/* XXX Change this if we ever use a PSO mode kernel. */
-#define mb() __asm__ __volatile__ ("" : : : "memory")
-#define rmb() mb()
-#define wmb() mb()
-#define read_barrier_depends() do { } while(0)
-#define set_mb(__var, __value) do { __var = __value; mb(); } while(0)
-#define smp_mb() __asm__ __volatile__("":::"memory")
-#define smp_rmb() __asm__ __volatile__("":::"memory")
-#define smp_wmb() __asm__ __volatile__("":::"memory")
-#define smp_read_barrier_depends() do { } while(0)
+#include <asm/processor.h> /* for nop() */
+#include <asm-generic/barrier.h>
#endif /* !(__SPARC_BARRIER_H) */
#define smp_read_barrier_depends() do { } while(0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
#endif /* !(__SPARC64_BARRIER_H) */
case BPF_S_ALU_MUL_K: /* A *= K */
emit_alu_K(MUL, K);
break;
- case BPF_S_ALU_DIV_K: /* A /= K */
- emit_alu_K(MUL, K);
- emit_read_y(r_A);
+ case BPF_S_ALU_DIV_K: /* A /= K with K != 0*/
+ if (K == 1)
+ break;
+ emit_write_y(G0);
+#ifdef CONFIG_SPARC32
+ /* The Sparc v8 architecture requires
+ * three instructions between a %y
+ * register write and the first use.
+ */
+ emit_nop();
+ emit_nop();
+ emit_nop();
+#endif
+ emit_alu_K(DIV, K);
break;
case BPF_S_ALU_DIV_X: /* A /= X; */
emit_cmpi(r_X, 0);
#include <arch/spr_def.h>
#include <asm/timex.h>
-/*
- * read_barrier_depends - Flush all pending reads that subsequents reads
- * depend on.
- *
- * No data-dependent reads from memory-like regions are ever reordered
- * over this barrier. All reads preceding this primitive are guaranteed
- * to access memory (but not necessarily other CPUs' caches) before any
- * reads following this primitive that depend on the data return by
- * any of the preceding reads. This primitive is much lighter weight than
- * rmb() on most CPUs, and is never heavier weight than is
- * rmb().
- *
- * These ordering constraints are respected by both the local CPU
- * and the compiler.
- *
- * Ordering is not guaranteed by anything other than these primitives,
- * not even by data dependencies. See the documentation for
- * memory_barrier() for examples and URLs to more information.
- *
- * For example, the following code would force ordering (the initial
- * value of "a" is zero, "b" is one, and "p" is "&a"):
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * b = 2;
- * memory_barrier();
- * p = &b; q = p;
- * read_barrier_depends();
- * d = *q;
- * </programlisting>
- *
- * because the read of "*q" depends on the read of "p" and these
- * two reads are separated by a read_barrier_depends(). However,
- * the following code, with the same initial values for "a" and "b":
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * a = 2;
- * memory_barrier();
- * b = 3; y = b;
- * read_barrier_depends();
- * x = a;
- * </programlisting>
- *
- * does not enforce ordering, since there is no data dependency between
- * the read of "a" and the read of "b". Therefore, on some CPUs, such
- * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
- * in cases like this where there are no data dependencies.
- */
-#define read_barrier_depends() do { } while (0)
-
#define __sync() __insn_mf()
#include <hv/syscall_public.h>
#define mb() fast_mb()
#define iob() fast_iob()
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while (0)
-#endif
-
-#define set_mb(var, value) \
- do { var = value; mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_TILE_BARRIER_H */
#define dsb() __asm__ __volatile__ ("" : : : "memory")
#define dmb() __asm__ __volatile__ ("" : : : "memory")
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define read_barrier_depends() do { } while (0)
-#define smp_read_barrier_depends() do { } while (0)
-
-#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* __UNICORE_BARRIER_H__ */
select RTC_LIB
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_IRQ_EXIT_ON_IRQ_STACK if X86_64
+ select HAVE_CC_STACKPROTECTOR
config INSTRUCTION_DECODER
def_bool y
This option compiles in support for the CE4100 SOC for settop
boxes and media devices.
-config X86_WANT_INTEL_MID
+config X86_INTEL_MID
bool "Intel MID platform support"
depends on X86_32
depends on X86_EXTENDED_PLATFORM
- ---help---
- Select to build a kernel capable of supporting Intel MID platform
- systems which do not have the PCI legacy interfaces (Moorestown,
- Medfield). If you are building for a PC class system say N here.
-
-if X86_WANT_INTEL_MID
-
-config X86_INTEL_MID
- bool
-
-config X86_MDFLD
- bool "Medfield MID platform"
depends on PCI
depends on PCI_GOANY
depends on X86_IO_APIC
- select X86_INTEL_MID
select SFI
+ select I2C
select DW_APB_TIMER
select APB_TIMER
- select I2C
- select SPI
select INTEL_SCU_IPC
- select X86_PLATFORM_DEVICES
select MFD_INTEL_MSIC
---help---
- Medfield is Intel's Low Power Intel Architecture (LPIA) based Moblin
- Internet Device(MID) platform.
- Unlike standard x86 PCs, Medfield does not have many legacy devices
- nor standard legacy replacement devices/features. e.g. Medfield does
- not contain i8259, i8254, HPET, legacy BIOS, most of the io ports.
+ Select to build a kernel capable of supporting Intel MID (Mobile
+ Internet Device) platform systems which do not have the PCI legacy
+ interfaces. If you are building for a PC class system say N here.
-endif
+ Intel MID platforms are based on an Intel processor and chipset which
+ consume less power than most of the x86 derivatives.
config X86_INTEL_LPSS
bool "Intel Low Power Subsystem Support"
def_bool y
depends on MICROCODE
-config MICROCODE_INTEL_LIB
- def_bool y
- depends on MICROCODE_INTEL
-
config MICROCODE_INTEL_EARLY
def_bool n
If unsure, say Y. Only embedded should say N here.
-config CC_STACKPROTECTOR
- bool "Enable -fstack-protector buffer overflow detection"
- ---help---
- This option turns on the -fstack-protector GCC feature. This
- feature puts, at the beginning of functions, a canary value on
- the stack just before the return address, and validates
- the value just before actually returning. Stack based buffer
- overflows (that need to overwrite this return address) now also
- overwrite the canary, which gets detected and the attack is then
- neutralized via a kernel panic.
-
- This feature requires gcc version 4.2 or above, or a distribution
- gcc with the feature backported. Older versions are automatically
- detected and for those versions, this configuration option is
- ignored. (and a warning is printed during bootup)
-
source kernel/Kconfig.hz
config KEXEC
Note: If CONFIG_RELOCATABLE=y, then the kernel runs from the address
it has been loaded at and the compile time physical address
- (CONFIG_PHYSICAL_START) is ignored.
+ (CONFIG_PHYSICAL_START) is used as the minimum location.
-# Relocation on x86-32 needs some additional build support
+config RANDOMIZE_BASE
+ bool "Randomize the address of the kernel image"
+ depends on RELOCATABLE
+ depends on !HIBERNATION
+ default n
+ ---help---
+ Randomizes the physical and virtual address at which the
+ kernel image is decompressed, as a security feature that
+ deters exploit attempts relying on knowledge of the location
+ of kernel internals.
+
+ Entropy is generated using the RDRAND instruction if it is
+ supported. If RDTSC is supported, it is used as well. If
+ neither RDRAND nor RDTSC are supported, then randomness is
+ read from the i8254 timer.
+
+ The kernel will be offset by up to RANDOMIZE_BASE_MAX_OFFSET,
+ and aligned according to PHYSICAL_ALIGN. Since the kernel is
+ built using 2GiB addressing, and PHYSICAL_ALGIN must be at a
+ minimum of 2MiB, only 10 bits of entropy is theoretically
+ possible. At best, due to page table layouts, 64-bit can use
+ 9 bits of entropy and 32-bit uses 8 bits.
+
+ If unsure, say N.
+
+config RANDOMIZE_BASE_MAX_OFFSET
+ hex "Maximum kASLR offset allowed" if EXPERT
+ depends on RANDOMIZE_BASE
+ range 0x0 0x20000000 if X86_32
+ default "0x20000000" if X86_32
+ range 0x0 0x40000000 if X86_64
+ default "0x40000000" if X86_64
+ ---help---
+ The lesser of RANDOMIZE_BASE_MAX_OFFSET and available physical
+ memory is used to determine the maximal offset in bytes that will
+ be applied to the kernel when kernel Address Space Layout
+ Randomization (kASLR) is active. This must be a multiple of
+ PHYSICAL_ALIGN.
+
+ On 32-bit this is limited to 512MiB by page table layouts. The
+ default is 512MiB.
+
+ On 64-bit this is limited by how the kernel fixmap page table is
+ positioned, so this cannot be larger than 1GiB currently. Without
+ RANDOMIZE_BASE, there is a 512MiB to 1.5GiB split between kernel
+ and modules. When RANDOMIZE_BASE_MAX_OFFSET is above 512MiB, the
+ modules area will shrink to compensate, up to the current maximum
+ 1GiB to 1GiB split. The default is 1GiB.
+
+ If unsure, leave at the default value.
+
+# Relocation on x86 needs some additional build support
config X86_NEED_RELOCS
def_bool y
- depends on X86_32 && RELOCATABLE
+ depends on RANDOMIZE_BASE || (X86_32 && RELOCATABLE)
config PHYSICAL_ALIGN
hex "Alignment value to which kernel should be aligned"
- default "0x1000000"
+ default "0x200000"
range 0x2000 0x1000000 if X86_32
range 0x200000 0x1000000 if X86_64
---help---
bool
depends on STA2X11
+config IOSF_MBI
+ bool
+ depends on PCI
+ ---help---
+ To be selected by modules requiring access to the Intel OnChip System
+ Fabric (IOSF) Sideband MailBox Interface (MBI). For MBI platforms
+ enumerable by PCI.
+
source "net/Kconfig"
source "drivers/Kconfig"
KBUILD_CFLAGS += -maccumulate-outgoing-args
endif
+# Make sure compiler does not have buggy stack-protector support.
ifdef CONFIG_CC_STACKPROTECTOR
cc_has_sp := $(srctree)/scripts/gcc-x86_$(BITS)-has-stack-protector.sh
- ifeq ($(shell $(CONFIG_SHELL) $(cc_has_sp) $(CC) $(KBUILD_CPPFLAGS) $(biarch)),y)
- stackp-y := -fstack-protector
- KBUILD_CFLAGS += $(stackp-y)
- else
- $(warning stack protector enabled but no compiler support)
+ ifneq ($(shell $(CONFIG_SHELL) $(cc_has_sp) $(CC) $(KBUILD_CPPFLAGS) $(biarch)),y)
+ $(warning stack-protector enabled but compiler support broken)
endif
endif
targets += fdimage fdimage144 fdimage288 image.iso mtools.conf
subdir- := compressed
-setup-y += a20.o bioscall.o cmdline.o copy.o cpu.o cpucheck.o
+setup-y += a20.o bioscall.o cmdline.o copy.o cpu.o cpuflags.o cpucheck.o
setup-y += early_serial_console.o edd.o header.o main.o mca.o memory.o
setup-y += pm.o pmjump.o printf.o regs.o string.o tty.o video.o
setup-y += video-mode.o version.o
/* -----------------------------------------------------------------------
*
- * Copyright 2009 Intel Corporation; author H. Peter Anvin
+ * Copyright 2009-2014 Intel Corporation; author H. Peter Anvin
*
* This file is part of the Linux kernel, and is made available under
* the terms of the GNU General Public License version 2 or (at your
* touching registers they shouldn't be.
*/
- .code16gcc
- .text
+ .code16
+ .section ".inittext","ax"
.globl intcall
.type intcall, @function
intcall:
#include <asm/boot.h>
#include <asm/setup.h>
#include "bitops.h"
-#include <asm/cpufeature.h>
-#include <asm/processor-flags.h>
#include "ctype.h"
+#include "cpuflags.h"
/* Useful macros */
#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
return __cmdline_find_option_bool(cmd_line_ptr, option);
}
-
/* cpu.c, cpucheck.c */
-struct cpu_features {
- int level; /* Family, or 64 for x86-64 */
- int model;
- u32 flags[NCAPINTS];
-};
-extern struct cpu_features cpu;
int check_cpu(int *cpu_level_ptr, int *req_level_ptr, u32 **err_flags_ptr);
int validate_cpu(void);
VMLINUX_OBJS = $(obj)/vmlinux.lds $(obj)/head_$(BITS).o $(obj)/misc.o \
$(obj)/string.o $(obj)/cmdline.o $(obj)/early_serial_console.o \
- $(obj)/piggy.o
+ $(obj)/piggy.o $(obj)/cpuflags.o $(obj)/aslr.o
$(obj)/eboot.o: KBUILD_CFLAGS += -fshort-wchar -mno-red-zone
--- /dev/null
+#include "misc.h"
+
+#ifdef CONFIG_RANDOMIZE_BASE
+#include <asm/msr.h>
+#include <asm/archrandom.h>
+#include <asm/e820.h>
+
+#include <generated/compile.h>
+#include <linux/module.h>
+#include <linux/uts.h>
+#include <linux/utsname.h>
+#include <generated/utsrelease.h>
+
+/* Simplified build-specific string for starting entropy. */
+static const char build_str[] = UTS_RELEASE " (" LINUX_COMPILE_BY "@"
+ LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION;
+
+#define I8254_PORT_CONTROL 0x43
+#define I8254_PORT_COUNTER0 0x40
+#define I8254_CMD_READBACK 0xC0
+#define I8254_SELECT_COUNTER0 0x02
+#define I8254_STATUS_NOTREADY 0x40
+static inline u16 i8254(void)
+{
+ u16 status, timer;
+
+ do {
+ outb(I8254_PORT_CONTROL,
+ I8254_CMD_READBACK | I8254_SELECT_COUNTER0);
+ status = inb(I8254_PORT_COUNTER0);
+ timer = inb(I8254_PORT_COUNTER0);
+ timer |= inb(I8254_PORT_COUNTER0) << 8;
+ } while (status & I8254_STATUS_NOTREADY);
+
+ return timer;
+}
+
+static unsigned long rotate_xor(unsigned long hash, const void *area,
+ size_t size)
+{
+ size_t i;
+ unsigned long *ptr = (unsigned long *)area;
+
+ for (i = 0; i < size / sizeof(hash); i++) {
+ /* Rotate by odd number of bits and XOR. */
+ hash = (hash << ((sizeof(hash) * 8) - 7)) | (hash >> 7);
+ hash ^= ptr[i];
+ }
+
+ return hash;
+}
+
+/* Attempt to create a simple but unpredictable starting entropy. */
+static unsigned long get_random_boot(void)
+{
+ unsigned long hash = 0;
+
+ hash = rotate_xor(hash, build_str, sizeof(build_str));
+ hash = rotate_xor(hash, real_mode, sizeof(*real_mode));
+
+ return hash;
+}
+
+static unsigned long get_random_long(void)
+{
+#ifdef CONFIG_X86_64
+ const unsigned long mix_const = 0x5d6008cbf3848dd3UL;
+#else
+ const unsigned long mix_const = 0x3f39e593UL;
+#endif
+ unsigned long raw, random = get_random_boot();
+ bool use_i8254 = true;
+
+ debug_putstr("KASLR using");
+
+ if (has_cpuflag(X86_FEATURE_RDRAND)) {
+ debug_putstr(" RDRAND");
+ if (rdrand_long(&raw)) {
+ random ^= raw;
+ use_i8254 = false;
+ }
+ }
+
+ if (has_cpuflag(X86_FEATURE_TSC)) {
+ debug_putstr(" RDTSC");
+ rdtscll(raw);
+
+ random ^= raw;
+ use_i8254 = false;
+ }
+
+ if (use_i8254) {
+ debug_putstr(" i8254");
+ random ^= i8254();
+ }
+
+ /* Circular multiply for better bit diffusion */
+ asm("mul %3"
+ : "=a" (random), "=d" (raw)
+ : "a" (random), "rm" (mix_const));
+ random += raw;
+
+ debug_putstr("...\n");
+
+ return random;
+}
+
+struct mem_vector {
+ unsigned long start;
+ unsigned long size;
+};
+
+#define MEM_AVOID_MAX 5
+struct mem_vector mem_avoid[MEM_AVOID_MAX];
+
+static bool mem_contains(struct mem_vector *region, struct mem_vector *item)
+{
+ /* Item at least partially before region. */
+ if (item->start < region->start)
+ return false;
+ /* Item at least partially after region. */
+ if (item->start + item->size > region->start + region->size)
+ return false;
+ return true;
+}
+
+static bool mem_overlaps(struct mem_vector *one, struct mem_vector *two)
+{
+ /* Item one is entirely before item two. */
+ if (one->start + one->size <= two->start)
+ return false;
+ /* Item one is entirely after item two. */
+ if (one->start >= two->start + two->size)
+ return false;
+ return true;
+}
+
+static void mem_avoid_init(unsigned long input, unsigned long input_size,
+ unsigned long output, unsigned long output_size)
+{
+ u64 initrd_start, initrd_size;
+ u64 cmd_line, cmd_line_size;
+ unsigned long unsafe, unsafe_len;
+ char *ptr;
+
+ /*
+ * Avoid the region that is unsafe to overlap during
+ * decompression (see calculations at top of misc.c).
+ */
+ unsafe_len = (output_size >> 12) + 32768 + 18;
+ unsafe = (unsigned long)input + input_size - unsafe_len;
+ mem_avoid[0].start = unsafe;
+ mem_avoid[0].size = unsafe_len;
+
+ /* Avoid initrd. */
+ initrd_start = (u64)real_mode->ext_ramdisk_image << 32;
+ initrd_start |= real_mode->hdr.ramdisk_image;
+ initrd_size = (u64)real_mode->ext_ramdisk_size << 32;
+ initrd_size |= real_mode->hdr.ramdisk_size;
+ mem_avoid[1].start = initrd_start;
+ mem_avoid[1].size = initrd_size;
+
+ /* Avoid kernel command line. */
+ cmd_line = (u64)real_mode->ext_cmd_line_ptr << 32;
+ cmd_line |= real_mode->hdr.cmd_line_ptr;
+ /* Calculate size of cmd_line. */
+ ptr = (char *)(unsigned long)cmd_line;
+ for (cmd_line_size = 0; ptr[cmd_line_size++]; )
+ ;
+ mem_avoid[2].start = cmd_line;
+ mem_avoid[2].size = cmd_line_size;
+
+ /* Avoid heap memory. */
+ mem_avoid[3].start = (unsigned long)free_mem_ptr;
+ mem_avoid[3].size = BOOT_HEAP_SIZE;
+
+ /* Avoid stack memory. */
+ mem_avoid[4].start = (unsigned long)free_mem_end_ptr;
+ mem_avoid[4].size = BOOT_STACK_SIZE;
+}
+
+/* Does this memory vector overlap a known avoided area? */
+bool mem_avoid_overlap(struct mem_vector *img)
+{
+ int i;
+
+ for (i = 0; i < MEM_AVOID_MAX; i++) {
+ if (mem_overlaps(img, &mem_avoid[i]))
+ return true;
+ }
+
+ return false;
+}
+
+unsigned long slots[CONFIG_RANDOMIZE_BASE_MAX_OFFSET / CONFIG_PHYSICAL_ALIGN];
+unsigned long slot_max = 0;
+
+static void slots_append(unsigned long addr)
+{
+ /* Overflowing the slots list should be impossible. */
+ if (slot_max >= CONFIG_RANDOMIZE_BASE_MAX_OFFSET /
+ CONFIG_PHYSICAL_ALIGN)
+ return;
+
+ slots[slot_max++] = addr;
+}
+
+static unsigned long slots_fetch_random(void)
+{
+ /* Handle case of no slots stored. */
+ if (slot_max == 0)
+ return 0;
+
+ return slots[get_random_long() % slot_max];
+}
+
+static void process_e820_entry(struct e820entry *entry,
+ unsigned long minimum,
+ unsigned long image_size)
+{
+ struct mem_vector region, img;
+
+ /* Skip non-RAM entries. */
+ if (entry->type != E820_RAM)
+ return;
+
+ /* Ignore entries entirely above our maximum. */
+ if (entry->addr >= CONFIG_RANDOMIZE_BASE_MAX_OFFSET)
+ return;
+
+ /* Ignore entries entirely below our minimum. */
+ if (entry->addr + entry->size < minimum)
+ return;
+
+ region.start = entry->addr;
+ region.size = entry->size;
+
+ /* Potentially raise address to minimum location. */
+ if (region.start < minimum)
+ region.start = minimum;
+
+ /* Potentially raise address to meet alignment requirements. */
+ region.start = ALIGN(region.start, CONFIG_PHYSICAL_ALIGN);
+
+ /* Did we raise the address above the bounds of this e820 region? */
+ if (region.start > entry->addr + entry->size)
+ return;
+
+ /* Reduce size by any delta from the original address. */
+ region.size -= region.start - entry->addr;
+
+ /* Reduce maximum size to fit end of image within maximum limit. */
+ if (region.start + region.size > CONFIG_RANDOMIZE_BASE_MAX_OFFSET)
+ region.size = CONFIG_RANDOMIZE_BASE_MAX_OFFSET - region.start;
+
+ /* Walk each aligned slot and check for avoided areas. */
+ for (img.start = region.start, img.size = image_size ;
+ mem_contains(®ion, &img) ;
+ img.start += CONFIG_PHYSICAL_ALIGN) {
+ if (mem_avoid_overlap(&img))
+ continue;
+ slots_append(img.start);
+ }
+}
+
+static unsigned long find_random_addr(unsigned long minimum,
+ unsigned long size)
+{
+ int i;
+ unsigned long addr;
+
+ /* Make sure minimum is aligned. */
+ minimum = ALIGN(minimum, CONFIG_PHYSICAL_ALIGN);
+
+ /* Verify potential e820 positions, appending to slots list. */
+ for (i = 0; i < real_mode->e820_entries; i++) {
+ process_e820_entry(&real_mode->e820_map[i], minimum, size);
+ }
+
+ return slots_fetch_random();
+}
+
+unsigned char *choose_kernel_location(unsigned char *input,
+ unsigned long input_size,
+ unsigned char *output,
+ unsigned long output_size)
+{
+ unsigned long choice = (unsigned long)output;
+ unsigned long random;
+
+ if (cmdline_find_option_bool("nokaslr")) {
+ debug_putstr("KASLR disabled...\n");
+ goto out;
+ }
+
+ /* Record the various known unsafe memory ranges. */
+ mem_avoid_init((unsigned long)input, input_size,
+ (unsigned long)output, output_size);
+
+ /* Walk e820 and find a random address. */
+ random = find_random_addr(choice, output_size);
+ if (!random) {
+ debug_putstr("KASLR could not find suitable E820 region...\n");
+ goto out;
+ }
+
+ /* Always enforce the minimum. */
+ if (random < choice)
+ goto out;
+
+ choice = random;
+out:
+ return (unsigned char *)choice;
+}
+
+#endif /* CONFIG_RANDOMIZE_BASE */
#include "misc.h"
-#ifdef CONFIG_EARLY_PRINTK
+#if CONFIG_EARLY_PRINTK || CONFIG_RANDOMIZE_BASE
static unsigned long fs;
static inline void set_fs(unsigned long seg)
--- /dev/null
+#ifdef CONFIG_RANDOMIZE_BASE
+
+#include "../cpuflags.c"
+
+bool has_cpuflag(int flag)
+{
+ get_cpuflags();
+
+ return test_bit(flag, cpu.flags);
+}
+
+#endif
addl %eax, %ebx
notl %eax
andl %eax, %ebx
-#else
- movl $LOAD_PHYSICAL_ADDR, %ebx
+ cmpl $LOAD_PHYSICAL_ADDR, %ebx
+ jge 1f
#endif
+ movl $LOAD_PHYSICAL_ADDR, %ebx
+1:
/* Target address to relocate to for decompression */
addl $z_extract_offset, %ebx
leal boot_heap(%ebx), %eax
pushl %eax /* heap area */
pushl %esi /* real mode pointer */
- call decompress_kernel
+ call decompress_kernel /* returns kernel location in %eax */
addl $24, %esp
/*
* Jump to the decompressed kernel.
*/
xorl %ebx, %ebx
- jmp *%ebp
+ jmp *%eax
/*
* Stack and heap for uncompression
addl %eax, %ebx
notl %eax
andl %eax, %ebx
-#else
- movl $LOAD_PHYSICAL_ADDR, %ebx
+ cmpl $LOAD_PHYSICAL_ADDR, %ebx
+ jge 1f
#endif
+ movl $LOAD_PHYSICAL_ADDR, %ebx
+1:
/* Target address to relocate to for decompression */
addl $z_extract_offset, %ebx
addq %rax, %rbp
notq %rax
andq %rax, %rbp
-#else
- movq $LOAD_PHYSICAL_ADDR, %rbp
+ cmpq $LOAD_PHYSICAL_ADDR, %rbp
+ jge 1f
#endif
+ movq $LOAD_PHYSICAL_ADDR, %rbp
+1:
/* Target address to relocate to for decompression */
leaq z_extract_offset(%rbp), %rbx
movl $z_input_len, %ecx /* input_len */
movq %rbp, %r8 /* output target address */
movq $z_output_len, %r9 /* decompressed length */
- call decompress_kernel
+ call decompress_kernel /* returns kernel location in %rax */
popq %rsi
/*
* Jump to the decompressed kernel.
*/
- jmp *%rbp
+ jmp *%rax
.code32
no_longmode:
void *memset(void *s, int c, size_t n);
void *memcpy(void *dest, const void *src, size_t n);
-#ifdef CONFIG_X86_64
-#define memptr long
-#else
-#define memptr unsigned
-#endif
-
-static memptr free_mem_ptr;
-static memptr free_mem_end_ptr;
+memptr free_mem_ptr;
+memptr free_mem_end_ptr;
static char *vidmem;
static int vidport;
free(phdrs);
}
-asmlinkage void decompress_kernel(void *rmode, memptr heap,
+asmlinkage void *decompress_kernel(void *rmode, memptr heap,
unsigned char *input_data,
unsigned long input_len,
unsigned char *output,
free_mem_ptr = heap; /* Heap */
free_mem_end_ptr = heap + BOOT_HEAP_SIZE;
+ output = choose_kernel_location(input_data, input_len,
+ output, output_len);
+
+ /* Validate memory location choices. */
if ((unsigned long)output & (MIN_KERNEL_ALIGN - 1))
error("Destination address inappropriately aligned");
#ifdef CONFIG_X86_64
parse_elf(output);
handle_relocations(output, output_len);
debug_putstr("done.\nBooting the kernel.\n");
- return;
+ return output;
}
#define BOOT_BOOT_H
#include "../ctype.h"
+#ifdef CONFIG_X86_64
+#define memptr long
+#else
+#define memptr unsigned
+#endif
+
/* misc.c */
+extern memptr free_mem_ptr;
+extern memptr free_mem_end_ptr;
extern struct boot_params *real_mode; /* Pointer to real-mode data */
void __putstr(const char *s);
#define error_putstr(__x) __putstr(__x)
#endif
-#ifdef CONFIG_EARLY_PRINTK
-
+#if CONFIG_EARLY_PRINTK || CONFIG_RANDOMIZE_BASE
/* cmdline.c */
int cmdline_find_option(const char *option, char *buffer, int bufsize);
int cmdline_find_option_bool(const char *option);
+#endif
-/* early_serial_console.c */
-extern int early_serial_base;
-void console_init(void);
+#if CONFIG_RANDOMIZE_BASE
+/* aslr.c */
+unsigned char *choose_kernel_location(unsigned char *input,
+ unsigned long input_size,
+ unsigned char *output,
+ unsigned long output_size);
+/* cpuflags.c */
+bool has_cpuflag(int flag);
#else
+static inline
+unsigned char *choose_kernel_location(unsigned char *input,
+ unsigned long input_size,
+ unsigned char *output,
+ unsigned long output_size)
+{
+ return output;
+}
+#endif
+#ifdef CONFIG_EARLY_PRINTK
/* early_serial_console.c */
+extern int early_serial_base;
+void console_init(void);
+#else
static const int early_serial_base;
static inline void console_init(void)
{ }
-
#endif
#endif
* Memory copy routines
*/
- .code16gcc
+ .code16
.text
GLOBAL(memcpy)
rep; movsb
popw %di
popw %si
- ret
+ retl
ENDPROC(memcpy)
GLOBAL(memset)
andw $3, %cx
rep; stosb
popw %di
- ret
+ retl
ENDPROC(memset)
GLOBAL(copy_from_fs)
pushw %ds
pushw %fs
popw %ds
- call memcpy
+ calll memcpy
popw %ds
- ret
+ retl
ENDPROC(copy_from_fs)
GLOBAL(copy_to_fs)
pushw %es
pushw %fs
popw %es
- call memcpy
+ calll memcpy
popw %es
- ret
+ retl
ENDPROC(copy_to_fs)
#if 0 /* Not currently used, but can be enabled as needed */
pushw %ds
pushw %gs
popw %ds
- call memcpy
+ calll memcpy
popw %ds
- ret
+ retl
ENDPROC(copy_from_gs)
GLOBAL(copy_to_gs)
pushw %es
pushw %gs
popw %es
- call memcpy
+ calll memcpy
popw %es
- ret
+ retl
ENDPROC(copy_to_gs)
#endif
#include <asm/required-features.h>
#include <asm/msr-index.h>
-struct cpu_features cpu;
-static u32 cpu_vendor[3];
static u32 err_flags[NCAPINTS];
static const int req_level = CONFIG_X86_MINIMUM_CPU_FAMILY;
cpu_vendor[2] == A32('M', 'x', '8', '6');
}
-static int has_fpu(void)
-{
- u16 fcw = -1, fsw = -1;
- u32 cr0;
-
- asm("movl %%cr0,%0" : "=r" (cr0));
- if (cr0 & (X86_CR0_EM|X86_CR0_TS)) {
- cr0 &= ~(X86_CR0_EM|X86_CR0_TS);
- asm volatile("movl %0,%%cr0" : : "r" (cr0));
- }
-
- asm volatile("fninit ; fnstsw %0 ; fnstcw %1"
- : "+m" (fsw), "+m" (fcw));
-
- return fsw == 0 && (fcw & 0x103f) == 0x003f;
-}
-
-static int has_eflag(u32 mask)
-{
- u32 f0, f1;
-
- asm("pushfl ; "
- "pushfl ; "
- "popl %0 ; "
- "movl %0,%1 ; "
- "xorl %2,%1 ; "
- "pushl %1 ; "
- "popfl ; "
- "pushfl ; "
- "popl %1 ; "
- "popfl"
- : "=&r" (f0), "=&r" (f1)
- : "ri" (mask));
-
- return !!((f0^f1) & mask);
-}
-
-static void get_flags(void)
-{
- u32 max_intel_level, max_amd_level;
- u32 tfms;
-
- if (has_fpu())
- set_bit(X86_FEATURE_FPU, cpu.flags);
-
- if (has_eflag(X86_EFLAGS_ID)) {
- asm("cpuid"
- : "=a" (max_intel_level),
- "=b" (cpu_vendor[0]),
- "=d" (cpu_vendor[1]),
- "=c" (cpu_vendor[2])
- : "a" (0));
-
- if (max_intel_level >= 0x00000001 &&
- max_intel_level <= 0x0000ffff) {
- asm("cpuid"
- : "=a" (tfms),
- "=c" (cpu.flags[4]),
- "=d" (cpu.flags[0])
- : "a" (0x00000001)
- : "ebx");
- cpu.level = (tfms >> 8) & 15;
- cpu.model = (tfms >> 4) & 15;
- if (cpu.level >= 6)
- cpu.model += ((tfms >> 16) & 0xf) << 4;
- }
-
- asm("cpuid"
- : "=a" (max_amd_level)
- : "a" (0x80000000)
- : "ebx", "ecx", "edx");
-
- if (max_amd_level >= 0x80000001 &&
- max_amd_level <= 0x8000ffff) {
- u32 eax = 0x80000001;
- asm("cpuid"
- : "+a" (eax),
- "=c" (cpu.flags[6]),
- "=d" (cpu.flags[1])
- : : "ebx");
- }
- }
-}
-
/* Returns a bitmask of which words we have error bits in */
-static int check_flags(void)
+static int check_cpuflags(void)
{
u32 err;
int i;
if (has_eflag(X86_EFLAGS_AC))
cpu.level = 4;
- get_flags();
- err = check_flags();
+ get_cpuflags();
+ err = check_cpuflags();
if (test_bit(X86_FEATURE_LM, cpu.flags))
cpu.level = 64;
eax &= ~(1 << 15);
asm("wrmsr" : : "a" (eax), "d" (edx), "c" (ecx));
- get_flags(); /* Make sure it really did something */
- err = check_flags();
+ get_cpuflags(); /* Make sure it really did something */
+ err = check_cpuflags();
} else if (err == 0x01 &&
!(err_flags[0] & ~(1 << X86_FEATURE_CX8)) &&
is_centaur() && cpu.model >= 6) {
asm("wrmsr" : : "a" (eax), "d" (edx), "c" (ecx));
set_bit(X86_FEATURE_CX8, cpu.flags);
- err = check_flags();
+ err = check_cpuflags();
} else if (err == 0x01 && is_transmeta()) {
/* Transmeta might have masked feature bits in word 0 */
: : "ecx", "ebx");
asm("wrmsr" : : "a" (eax), "d" (edx), "c" (ecx));
- err = check_flags();
+ err = check_cpuflags();
}
if (err_flags_ptr)
--- /dev/null
+#include <linux/types.h>
+#include "bitops.h"
+
+#include <asm/processor-flags.h>
+#include <asm/required-features.h>
+#include <asm/msr-index.h>
+#include "cpuflags.h"
+
+struct cpu_features cpu;
+u32 cpu_vendor[3];
+
+static bool loaded_flags;
+
+static int has_fpu(void)
+{
+ u16 fcw = -1, fsw = -1;
+ unsigned long cr0;
+
+ asm volatile("mov %%cr0,%0" : "=r" (cr0));
+ if (cr0 & (X86_CR0_EM|X86_CR0_TS)) {
+ cr0 &= ~(X86_CR0_EM|X86_CR0_TS);
+ asm volatile("mov %0,%%cr0" : : "r" (cr0));
+ }
+
+ asm volatile("fninit ; fnstsw %0 ; fnstcw %1"
+ : "+m" (fsw), "+m" (fcw));
+
+ return fsw == 0 && (fcw & 0x103f) == 0x003f;
+}
+
+int has_eflag(unsigned long mask)
+{
+ unsigned long f0, f1;
+
+ asm volatile("pushf \n\t"
+ "pushf \n\t"
+ "pop %0 \n\t"
+ "mov %0,%1 \n\t"
+ "xor %2,%1 \n\t"
+ "push %1 \n\t"
+ "popf \n\t"
+ "pushf \n\t"
+ "pop %1 \n\t"
+ "popf"
+ : "=&r" (f0), "=&r" (f1)
+ : "ri" (mask));
+
+ return !!((f0^f1) & mask);
+}
+
+/* Handle x86_32 PIC using ebx. */
+#if defined(__i386__) && defined(__PIC__)
+# define EBX_REG "=r"
+#else
+# define EBX_REG "=b"
+#endif
+
+static inline void cpuid(u32 id, u32 *a, u32 *b, u32 *c, u32 *d)
+{
+ asm volatile(".ifnc %%ebx,%3 ; movl %%ebx,%3 ; .endif \n\t"
+ "cpuid \n\t"
+ ".ifnc %%ebx,%3 ; xchgl %%ebx,%3 ; .endif \n\t"
+ : "=a" (*a), "=c" (*c), "=d" (*d), EBX_REG (*b)
+ : "a" (id)
+ );
+}
+
+void get_cpuflags(void)
+{
+ u32 max_intel_level, max_amd_level;
+ u32 tfms;
+ u32 ignored;
+
+ if (loaded_flags)
+ return;
+ loaded_flags = true;
+
+ if (has_fpu())
+ set_bit(X86_FEATURE_FPU, cpu.flags);
+
+ if (has_eflag(X86_EFLAGS_ID)) {
+ cpuid(0x0, &max_intel_level, &cpu_vendor[0], &cpu_vendor[2],
+ &cpu_vendor[1]);
+
+ if (max_intel_level >= 0x00000001 &&
+ max_intel_level <= 0x0000ffff) {
+ cpuid(0x1, &tfms, &ignored, &cpu.flags[4],
+ &cpu.flags[0]);
+ cpu.level = (tfms >> 8) & 15;
+ cpu.model = (tfms >> 4) & 15;
+ if (cpu.level >= 6)
+ cpu.model += ((tfms >> 16) & 0xf) << 4;
+ }
+
+ cpuid(0x80000000, &max_amd_level, &ignored, &ignored,
+ &ignored);
+
+ if (max_amd_level >= 0x80000001 &&
+ max_amd_level <= 0x8000ffff) {
+ cpuid(0x80000001, &ignored, &ignored, &cpu.flags[6],
+ &cpu.flags[1]);
+ }
+ }
+}
--- /dev/null
+#ifndef BOOT_CPUFLAGS_H
+#define BOOT_CPUFLAGS_H
+
+#include <asm/cpufeature.h>
+#include <asm/processor-flags.h>
+
+struct cpu_features {
+ int level; /* Family, or 64 for x86-64 */
+ int model;
+ u32 flags[NCAPINTS];
+};
+
+extern struct cpu_features cpu;
+extern u32 cpu_vendor[3];
+
+int has_eflag(unsigned long mask);
+void get_cpuflags(void);
+
+#endif
#else
# define XLF23 0
#endif
- .word XLF0 | XLF1 | XLF23
+
+#if defined(CONFIG_X86_64) && defined(CONFIG_EFI) && defined(CONFIG_KEXEC)
+# define XLF4 XLF_EFI_KEXEC
+#else
+# define XLF4 0
+#endif
+
+ .word XLF0 | XLF1 | XLF23 | XLF4
cmdline_size: .long COMMAND_LINE_SIZE-1 #length of the command line,
#added with boot protocol
#ifdef CONFIG_ARCH_RANDOM
+/* Instead of arch_get_random_long() when alternatives haven't run. */
+static inline int rdrand_long(unsigned long *v)
+{
+ int ok;
+ asm volatile("1: " RDRAND_LONG "\n\t"
+ "jc 2f\n\t"
+ "decl %0\n\t"
+ "jnz 1b\n\t"
+ "2:"
+ : "=r" (ok), "=a" (*v)
+ : "0" (RDRAND_RETRY_LOOPS));
+ return ok;
+}
+
#define GET_RANDOM(name, type, rdrand, nop) \
static inline int name(type *v) \
{ \
#endif /* CONFIG_X86_64 */
+#else
+
+static inline int rdrand_long(unsigned long *v)
+{
+ return 0;
+}
+
#endif /* CONFIG_ARCH_RANDOM */
extern void x86_init_rdrand(struct cpuinfo_x86 *c);
#endif
#define smp_read_barrier_depends() read_barrier_depends()
#define set_mb(var, value) do { (void)xchg(&var, value); } while (0)
-#else
+#else /* !SMP */
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
#define smp_read_barrier_depends() do { } while (0)
#define set_mb(var, value) do { var = value; barrier(); } while (0)
+#endif /* SMP */
+
+#if defined(CONFIG_X86_OOSTORE) || defined(CONFIG_X86_PPRO_FENCE)
+
+/*
+ * For either of these options x86 doesn't have a strong TSO memory
+ * model and we should fall back to full barriers.
+ */
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
+#else /* regular x86 TSO memory ordering */
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
#endif
/*
#define X86_FEATURE_ERMS (9*32+ 9) /* Enhanced REP MOVSB/STOSB */
#define X86_FEATURE_INVPCID (9*32+10) /* Invalidate Processor Context ID */
#define X86_FEATURE_RTM (9*32+11) /* Restricted Transactional Memory */
+#define X86_FEATURE_MPX (9*32+14) /* Memory Protection Extension */
#define X86_FEATURE_RDSEED (9*32+18) /* The RDSEED instruction */
#define X86_FEATURE_ADX (9*32+19) /* The ADCX and ADOX instructions */
#define X86_FEATURE_SMAP (9*32+20) /* Supervisor Mode Access Prevention */
#ifndef _ASM_X86_EFI_H
#define _ASM_X86_EFI_H
+/*
+ * We map the EFI regions needed for runtime services non-contiguously,
+ * with preserved alignment on virtual addresses starting from -4G down
+ * for a total max space of 64G. This way, we provide for stable runtime
+ * services addresses across kernels so that a kexec'd kernel can still
+ * use them.
+ *
+ * This is the main reason why we're doing stable VA mappings for RT
+ * services.
+ *
+ * This flag is used in conjuction with a chicken bit called
+ * "efi=old_map" which can be used as a fallback to the old runtime
+ * services mapping method in case there's some b0rkage with a
+ * particular EFI implementation (haha, it is hard to hold up the
+ * sarcasm here...).
+ */
+#define EFI_OLD_MEMMAP EFI_ARCH_1
+
#ifdef CONFIG_X86_32
#define EFI_LOADER_SIGNATURE "EL32"
efi_call6((f), (u64)(a1), (u64)(a2), (u64)(a3), \
(u64)(a4), (u64)(a5), (u64)(a6))
+#define _efi_call_virtX(x, f, ...) \
+({ \
+ efi_status_t __s; \
+ \
+ efi_sync_low_kernel_mappings(); \
+ preempt_disable(); \
+ __s = efi_call##x((void *)efi.systab->runtime->f, __VA_ARGS__); \
+ preempt_enable(); \
+ __s; \
+})
+
#define efi_call_virt0(f) \
- efi_call0((efi.systab->runtime->f))
-#define efi_call_virt1(f, a1) \
- efi_call1((efi.systab->runtime->f), (u64)(a1))
-#define efi_call_virt2(f, a1, a2) \
- efi_call2((efi.systab->runtime->f), (u64)(a1), (u64)(a2))
-#define efi_call_virt3(f, a1, a2, a3) \
- efi_call3((efi.systab->runtime->f), (u64)(a1), (u64)(a2), \
- (u64)(a3))
-#define efi_call_virt4(f, a1, a2, a3, a4) \
- efi_call4((efi.systab->runtime->f), (u64)(a1), (u64)(a2), \
- (u64)(a3), (u64)(a4))
-#define efi_call_virt5(f, a1, a2, a3, a4, a5) \
- efi_call5((efi.systab->runtime->f), (u64)(a1), (u64)(a2), \
- (u64)(a3), (u64)(a4), (u64)(a5))
-#define efi_call_virt6(f, a1, a2, a3, a4, a5, a6) \
- efi_call6((efi.systab->runtime->f), (u64)(a1), (u64)(a2), \
- (u64)(a3), (u64)(a4), (u64)(a5), (u64)(a6))
+ _efi_call_virtX(0, f)
+#define efi_call_virt1(f, a1) \
+ _efi_call_virtX(1, f, (u64)(a1))
+#define efi_call_virt2(f, a1, a2) \
+ _efi_call_virtX(2, f, (u64)(a1), (u64)(a2))
+#define efi_call_virt3(f, a1, a2, a3) \
+ _efi_call_virtX(3, f, (u64)(a1), (u64)(a2), (u64)(a3))
+#define efi_call_virt4(f, a1, a2, a3, a4) \
+ _efi_call_virtX(4, f, (u64)(a1), (u64)(a2), (u64)(a3), (u64)(a4))
+#define efi_call_virt5(f, a1, a2, a3, a4, a5) \
+ _efi_call_virtX(5, f, (u64)(a1), (u64)(a2), (u64)(a3), (u64)(a4), (u64)(a5))
+#define efi_call_virt6(f, a1, a2, a3, a4, a5, a6) \
+ _efi_call_virtX(6, f, (u64)(a1), (u64)(a2), (u64)(a3), (u64)(a4), (u64)(a5), (u64)(a6))
extern void __iomem *efi_ioremap(unsigned long addr, unsigned long size,
u32 type, u64 attribute);
extern int add_efi_memmap;
extern unsigned long x86_efi_facility;
+extern struct efi_scratch efi_scratch;
extern void efi_set_executable(efi_memory_desc_t *md, bool executable);
extern int efi_memblock_x86_reserve_range(void);
extern void efi_call_phys_prelog(void);
extern void efi_call_phys_epilog(void);
extern void efi_unmap_memmap(void);
extern void efi_memory_uc(u64 addr, unsigned long size);
+extern void __init efi_map_region(efi_memory_desc_t *md);
+extern void __init efi_map_region_fixed(efi_memory_desc_t *md);
+extern void efi_sync_low_kernel_mappings(void);
+extern void efi_setup_page_tables(void);
+extern void __init old_map_region(efi_memory_desc_t *md);
+
+struct efi_setup_data {
+ u64 fw_vendor;
+ u64 runtime;
+ u64 tables;
+ u64 smbios;
+ u64 reserved[8];
+};
+
+extern u64 efi_setup;
#ifdef CONFIG_EFI
}
extern struct console early_efi_console;
-
+extern void parse_efi_setup(u64 phys_addr, u32 data_len);
#else
/*
* IF EFI is not configured, have the EFI calls return -ENOSYS.
#define efi_call4(_f, _a1, _a2, _a3, _a4) (-ENOSYS)
#define efi_call5(_f, _a1, _a2, _a3, _a4, _a5) (-ENOSYS)
#define efi_call6(_f, _a1, _a2, _a3, _a4, _a5, _a6) (-ENOSYS)
+static inline void parse_efi_setup(u64 phys_addr, u32 data_len) {}
#endif /* CONFIG_EFI */
#endif /* _ASM_X86_EFI_H */
static inline int futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr,
u32 oldval, u32 newval)
{
- int ret = 0;
-
- if (!access_ok(VERIFY_WRITE, uaddr, sizeof(u32)))
- return -EFAULT;
-
- asm volatile("\t" ASM_STAC "\n"
- "1:\t" LOCK_PREFIX "cmpxchgl %4, %2\n"
- "2:\t" ASM_CLAC "\n"
- "\t.section .fixup, \"ax\"\n"
- "3:\tmov %3, %0\n"
- "\tjmp 2b\n"
- "\t.previous\n"
- _ASM_EXTABLE(1b, 3b)
- : "+r" (ret), "=a" (oldval), "+m" (*uaddr)
- : "i" (-EFAULT), "r" (newval), "1" (oldval)
- : "memory"
- );
-
- *uval = oldval;
- return ret;
+ return user_atomic_cmpxchg_inatomic(uval, uaddr, oldval, newval);
}
#endif
#define trace_interrupt interrupt
#endif
+#define VECTOR_UNDEFINED -1
+#define VECTOR_RETRIGGERED -2
+
typedef int vector_irq_t[NR_VECTORS];
DECLARE_PER_CPU(vector_irq_t, vector_irq);
extern void setup_vector_irq(int cpu);
enum intel_mid_cpu_type {
/* 1 was Moorestown */
INTEL_MID_CPU_CHIP_PENWELL = 2,
+ INTEL_MID_CPU_CHIP_CLOVERVIEW,
+ INTEL_MID_CPU_CHIP_TANGIER,
};
extern enum intel_mid_cpu_type __intel_mid_cpu_chip;
+/**
+ * struct intel_mid_ops - Interface between intel-mid & sub archs
+ * @arch_setup: arch_setup function to re-initialize platform
+ * structures (x86_init, x86_platform_init)
+ *
+ * This structure can be extended if any new interface is required
+ * between intel-mid & its sub arch files.
+ */
+struct intel_mid_ops {
+ void (*arch_setup)(void);
+};
+
+/* Helper API's for INTEL_MID_OPS_INIT */
+#define DECLARE_INTEL_MID_OPS_INIT(cpuname, cpuid) \
+ [cpuid] = get_##cpuname##_ops
+
+/* Maximum number of CPU ops */
+#define MAX_CPU_OPS(a) (sizeof(a)/sizeof(void *))
+
+/*
+ * For every new cpu addition, a weak get_<cpuname>_ops() function needs be
+ * declared in arch/x86/platform/intel_mid/intel_mid_weak_decls.h.
+ */
+#define INTEL_MID_OPS_INIT {\
+ DECLARE_INTEL_MID_OPS_INIT(penwell, INTEL_MID_CPU_CHIP_PENWELL), \
+ DECLARE_INTEL_MID_OPS_INIT(cloverview, INTEL_MID_CPU_CHIP_CLOVERVIEW), \
+ DECLARE_INTEL_MID_OPS_INIT(tangier, INTEL_MID_CPU_CHIP_TANGIER) \
+};
+
#ifdef CONFIG_X86_INTEL_MID
static inline enum intel_mid_cpu_type intel_mid_identify_cpu(void)
* Penwell uses spread spectrum clock, so the freq number is not exactly
* the same as reported by MSR based on SDM.
*/
-#define PENWELL_FSB_FREQ_83SKU 83200
-#define PENWELL_FSB_FREQ_100SKU 99840
+#define FSB_FREQ_83SKU 83200
+#define FSB_FREQ_100SKU 99840
+#define FSB_FREQ_133SKU 133000
+
+#define FSB_FREQ_167SKU 167000
+#define FSB_FREQ_200SKU 200000
+#define FSB_FREQ_267SKU 267000
+#define FSB_FREQ_333SKU 333000
+#define FSB_FREQ_400SKU 400000
+
+/* Bus Select SoC Fuse value */
+#define BSEL_SOC_FUSE_MASK 0x7
+#define BSEL_SOC_FUSE_001 0x1 /* FSB 133MHz */
+#define BSEL_SOC_FUSE_101 0x5 /* FSB 100MHz */
+#define BSEL_SOC_FUSE_111 0x7 /* FSB 83MHz */
#define SFI_MTMR_MAX_NUM 8
#define SFI_MRTC_MAX 8
--- /dev/null
+/*
+ * iosf_mbi.h: Intel OnChip System Fabric MailBox access support
+ */
+
+#ifndef IOSF_MBI_SYMS_H
+#define IOSF_MBI_SYMS_H
+
+#define MBI_MCR_OFFSET 0xD0
+#define MBI_MDR_OFFSET 0xD4
+#define MBI_MCRX_OFFSET 0xD8
+
+#define MBI_RD_MASK 0xFEFFFFFF
+#define MBI_WR_MASK 0X01000000
+
+#define MBI_MASK_HI 0xFFFFFF00
+#define MBI_MASK_LO 0x000000FF
+#define MBI_ENABLE 0xF0
+
+/* Baytrail available units */
+#define BT_MBI_UNIT_AUNIT 0x00
+#define BT_MBI_UNIT_SMC 0x01
+#define BT_MBI_UNIT_CPU 0x02
+#define BT_MBI_UNIT_BUNIT 0x03
+#define BT_MBI_UNIT_PMC 0x04
+#define BT_MBI_UNIT_GFX 0x06
+#define BT_MBI_UNIT_SMI 0x0C
+#define BT_MBI_UNIT_USB 0x43
+#define BT_MBI_UNIT_SATA 0xA3
+#define BT_MBI_UNIT_PCIE 0xA6
+
+/* Baytrail read/write opcodes */
+#define BT_MBI_AUNIT_READ 0x10
+#define BT_MBI_AUNIT_WRITE 0x11
+#define BT_MBI_SMC_READ 0x10
+#define BT_MBI_SMC_WRITE 0x11
+#define BT_MBI_CPU_READ 0x10
+#define BT_MBI_CPU_WRITE 0x11
+#define BT_MBI_BUNIT_READ 0x10
+#define BT_MBI_BUNIT_WRITE 0x11
+#define BT_MBI_PMC_READ 0x06
+#define BT_MBI_PMC_WRITE 0x07
+#define BT_MBI_GFX_READ 0x00
+#define BT_MBI_GFX_WRITE 0x01
+#define BT_MBI_SMIO_READ 0x06
+#define BT_MBI_SMIO_WRITE 0x07
+#define BT_MBI_USB_READ 0x06
+#define BT_MBI_USB_WRITE 0x07
+#define BT_MBI_SATA_READ 0x00
+#define BT_MBI_SATA_WRITE 0x01
+#define BT_MBI_PCIE_READ 0x00
+#define BT_MBI_PCIE_WRITE 0x01
+
+/**
+ * iosf_mbi_read() - MailBox Interface read command
+ * @port: port indicating subunit being accessed
+ * @opcode: port specific read or write opcode
+ * @offset: register address offset
+ * @mdr: register data to be read
+ *
+ * Locking is handled by spinlock - cannot sleep.
+ * Return: Nonzero on error
+ */
+int iosf_mbi_read(u8 port, u8 opcode, u32 offset, u32 *mdr);
+
+/**
+ * iosf_mbi_write() - MailBox unmasked write command
+ * @port: port indicating subunit being accessed
+ * @opcode: port specific read or write opcode
+ * @offset: register address offset
+ * @mdr: register data to be written
+ *
+ * Locking is handled by spinlock - cannot sleep.
+ * Return: Nonzero on error
+ */
+int iosf_mbi_write(u8 port, u8 opcode, u32 offset, u32 mdr);
+
+/**
+ * iosf_mbi_modify() - MailBox masked write command
+ * @port: port indicating subunit being accessed
+ * @opcode: port specific read or write opcode
+ * @offset: register address offset
+ * @mdr: register data being modified
+ * @mask: mask indicating bits in mdr to be modified
+ *
+ * Locking is handled by spinlock - cannot sleep.
+ * Return: Nonzero on error
+ */
+int iosf_mbi_modify(u8 port, u8 opcode, u32 offset, u32 mdr, u32 mask);
+
+#endif /* IOSF_MBI_SYMS_H */
extern void mce_unregister_decode_chain(struct notifier_block *nb);
#include <linux/percpu.h>
-#include <linux/init.h>
#include <linux/atomic.h>
extern int mce_p5_enabled;
#ifndef _ASM_X86_MICROCODE_H
#define _ASM_X86_MICROCODE_H
+#define native_rdmsr(msr, val1, val2) \
+do { \
+ u64 __val = native_read_msr((msr)); \
+ (void)((val1) = (u32)__val); \
+ (void)((val2) = (u32)(__val >> 32)); \
+} while (0)
+
+#define native_wrmsr(msr, low, high) \
+ native_write_msr(msr, low, high)
+
+#define native_wrmsrl(msr, val) \
+ native_write_msr((msr), \
+ (u32)((u64)(val)), \
+ (u32)((u64)(val) >> 32))
+
struct cpu_signature {
unsigned int sig;
unsigned int pf;
extern int apply_microcode_amd(int cpu);
extern enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size);
+#define PATCH_MAX_SIZE PAGE_SIZE
+extern u8 amd_ucode_patch[PATCH_MAX_SIZE];
+
#ifdef CONFIG_MICROCODE_AMD_EARLY
-#ifdef CONFIG_X86_32
-#define MPB_MAX_SIZE PAGE_SIZE
-extern u8 amd_bsp_mpb[MPB_MAX_SIZE];
-#endif
extern void __init load_ucode_amd_bsp(void);
extern void load_ucode_amd_ap(void);
extern int __init save_microcode_in_initrd_amd(void);
#ifndef _ASM_X86_MPSPEC_H
#define _ASM_X86_MPSPEC_H
-#include <linux/init.h>
#include <asm/mpspec_def.h>
#include <asm/x86_init.h>
#ifndef _ASM_X86_MWAIT_H
#define _ASM_X86_MWAIT_H
+#include <linux/sched.h>
+
#define MWAIT_SUBSTATE_MASK 0xf
#define MWAIT_CSTATE_MASK 0xf
#define MWAIT_SUBSTATE_SIZE 4
#define MWAIT_ECX_INTERRUPT_BREAK 0x1
+static inline void __monitor(const void *eax, unsigned long ecx,
+ unsigned long edx)
+{
+ /* "monitor %eax, %ecx, %edx;" */
+ asm volatile(".byte 0x0f, 0x01, 0xc8;"
+ :: "a" (eax), "c" (ecx), "d"(edx));
+}
+
+static inline void __mwait(unsigned long eax, unsigned long ecx)
+{
+ /* "mwait %eax, %ecx;" */
+ asm volatile(".byte 0x0f, 0x01, 0xc9;"
+ :: "a" (eax), "c" (ecx));
+}
+
+/*
+ * This uses new MONITOR/MWAIT instructions on P4 processors with PNI,
+ * which can obviate IPI to trigger checking of need_resched.
+ * We execute MONITOR against need_resched and enter optimized wait state
+ * through MWAIT. Whenever someone changes need_resched, we would be woken
+ * up from MWAIT (without an IPI).
+ *
+ * New with Core Duo processors, MWAIT can take some hints based on CPU
+ * capability.
+ */
+static inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx)
+{
+ if (!current_set_polling_and_test()) {
+ if (static_cpu_has(X86_FEATURE_CLFLUSH_MONITOR)) {
+ mb();
+ clflush((void *)¤t_thread_info()->flags);
+ mb();
+ }
+
+ __monitor((void *)¤t_thread_info()->flags, 0, 0);
+ if (!need_resched())
+ __mwait(eax, ecx);
+ }
+ current_clr_polling();
+}
+
#endif /* _ASM_X86_MWAIT_H */
#include <asm-generic/getorder.h>
#define __HAVE_ARCH_GATE_AREA 1
+#define HAVE_ARCH_HUGETLB_UNMAPPED_AREA
#endif /* __KERNEL__ */
#endif /* _ASM_X86_PAGE_H */
#ifndef __ASSEMBLY__
-#ifdef CONFIG_HUGETLB_PAGE
-#define HAVE_ARCH_HUGETLB_UNMAPPED_AREA
-#endif
-
#define __phys_addr_nodebug(x) ((x) - PAGE_OFFSET)
#ifdef CONFIG_DEBUG_VIRTUAL
extern unsigned long __phys_addr(unsigned long);
#define __VIRTUAL_MASK_SHIFT 47
/*
- * Kernel image size is limited to 512 MB (see level2_kernel_pgt in
- * arch/x86/kernel/head_64.S), and it is mapped here:
+ * Kernel image size is limited to 1GiB due to the fixmap living in the
+ * next 1GiB (see level2_kernel_pgt in arch/x86/kernel/head_64.S). Use
+ * 512MiB by default, leaving 1.5GiB for modules once the page tables
+ * are fully set up. If kernel ASLR is configured, it can extend the
+ * kernel page table mapping, reducing the size of the modules area.
*/
-#define KERNEL_IMAGE_SIZE (512 * 1024 * 1024)
+#define KERNEL_IMAGE_SIZE_DEFAULT (512 * 1024 * 1024)
+#if defined(CONFIG_RANDOMIZE_BASE) && \
+ CONFIG_RANDOMIZE_BASE_MAX_OFFSET > KERNEL_IMAGE_SIZE_DEFAULT
+#define KERNEL_IMAGE_SIZE CONFIG_RANDOMIZE_BASE_MAX_OFFSET
+#else
+#define KERNEL_IMAGE_SIZE KERNEL_IMAGE_SIZE_DEFAULT
+#endif
#endif /* _ASM_X86_PAGE_64_DEFS_H */
#define native_pmdp_get_and_clear(xp) native_local_pmdp_get_and_clear(xp)
#endif
+/* Bit manipulation helper on pte/pgoff entry */
+static inline unsigned long pte_bitop(unsigned long value, unsigned int rightshift,
+ unsigned long mask, unsigned int leftshift)
+{
+ return ((value >> rightshift) & mask) << leftshift;
+}
+
#ifdef CONFIG_MEM_SOFT_DIRTY
/*
#define PTE_FILE_BITS2 (PTE_FILE_SHIFT3 - PTE_FILE_SHIFT2 - 1)
#define PTE_FILE_BITS3 (PTE_FILE_SHIFT4 - PTE_FILE_SHIFT3 - 1)
-#define pte_to_pgoff(pte) \
- ((((pte).pte_low >> (PTE_FILE_SHIFT1)) \
- & ((1U << PTE_FILE_BITS1) - 1))) \
- + ((((pte).pte_low >> (PTE_FILE_SHIFT2)) \
- & ((1U << PTE_FILE_BITS2) - 1)) \
- << (PTE_FILE_BITS1)) \
- + ((((pte).pte_low >> (PTE_FILE_SHIFT3)) \
- & ((1U << PTE_FILE_BITS3) - 1)) \
- << (PTE_FILE_BITS1 + PTE_FILE_BITS2)) \
- + ((((pte).pte_low >> (PTE_FILE_SHIFT4))) \
- << (PTE_FILE_BITS1 + PTE_FILE_BITS2 + PTE_FILE_BITS3))
-
-#define pgoff_to_pte(off) \
- ((pte_t) { .pte_low = \
- ((((off)) & ((1U << PTE_FILE_BITS1) - 1)) << PTE_FILE_SHIFT1) \
- + ((((off) >> PTE_FILE_BITS1) \
- & ((1U << PTE_FILE_BITS2) - 1)) \
- << PTE_FILE_SHIFT2) \
- + ((((off) >> (PTE_FILE_BITS1 + PTE_FILE_BITS2)) \
- & ((1U << PTE_FILE_BITS3) - 1)) \
- << PTE_FILE_SHIFT3) \
- + ((((off) >> \
- (PTE_FILE_BITS1 + PTE_FILE_BITS2 + PTE_FILE_BITS3))) \
- << PTE_FILE_SHIFT4) \
- + _PAGE_FILE })
+#define PTE_FILE_MASK1 ((1U << PTE_FILE_BITS1) - 1)
+#define PTE_FILE_MASK2 ((1U << PTE_FILE_BITS2) - 1)
+#define PTE_FILE_MASK3 ((1U << PTE_FILE_BITS3) - 1)
+
+#define PTE_FILE_LSHIFT2 (PTE_FILE_BITS1)
+#define PTE_FILE_LSHIFT3 (PTE_FILE_BITS1 + PTE_FILE_BITS2)
+#define PTE_FILE_LSHIFT4 (PTE_FILE_BITS1 + PTE_FILE_BITS2 + PTE_FILE_BITS3)
+
+static __always_inline pgoff_t pte_to_pgoff(pte_t pte)
+{
+ return (pgoff_t)
+ (pte_bitop(pte.pte_low, PTE_FILE_SHIFT1, PTE_FILE_MASK1, 0) +
+ pte_bitop(pte.pte_low, PTE_FILE_SHIFT2, PTE_FILE_MASK2, PTE_FILE_LSHIFT2) +
+ pte_bitop(pte.pte_low, PTE_FILE_SHIFT3, PTE_FILE_MASK3, PTE_FILE_LSHIFT3) +
+ pte_bitop(pte.pte_low, PTE_FILE_SHIFT4, -1UL, PTE_FILE_LSHIFT4));
+}
+
+static __always_inline pte_t pgoff_to_pte(pgoff_t off)
+{
+ return (pte_t){
+ .pte_low =
+ pte_bitop(off, 0, PTE_FILE_MASK1, PTE_FILE_SHIFT1) +
+ pte_bitop(off, PTE_FILE_LSHIFT2, PTE_FILE_MASK2, PTE_FILE_SHIFT2) +
+ pte_bitop(off, PTE_FILE_LSHIFT3, PTE_FILE_MASK3, PTE_FILE_SHIFT3) +
+ pte_bitop(off, PTE_FILE_LSHIFT4, -1UL, PTE_FILE_SHIFT4) +
+ _PAGE_FILE,
+ };
+}
#else /* CONFIG_MEM_SOFT_DIRTY */
#define PTE_FILE_BITS1 (PTE_FILE_SHIFT2 - PTE_FILE_SHIFT1 - 1)
#define PTE_FILE_BITS2 (PTE_FILE_SHIFT3 - PTE_FILE_SHIFT2 - 1)
-#define pte_to_pgoff(pte) \
- ((((pte).pte_low >> PTE_FILE_SHIFT1) \
- & ((1U << PTE_FILE_BITS1) - 1)) \
- + ((((pte).pte_low >> PTE_FILE_SHIFT2) \
- & ((1U << PTE_FILE_BITS2) - 1)) << PTE_FILE_BITS1) \
- + (((pte).pte_low >> PTE_FILE_SHIFT3) \
- << (PTE_FILE_BITS1 + PTE_FILE_BITS2)))
-
-#define pgoff_to_pte(off) \
- ((pte_t) { .pte_low = \
- (((off) & ((1U << PTE_FILE_BITS1) - 1)) << PTE_FILE_SHIFT1) \
- + ((((off) >> PTE_FILE_BITS1) & ((1U << PTE_FILE_BITS2) - 1)) \
- << PTE_FILE_SHIFT2) \
- + (((off) >> (PTE_FILE_BITS1 + PTE_FILE_BITS2)) \
- << PTE_FILE_SHIFT3) \
- + _PAGE_FILE })
+#define PTE_FILE_MASK1 ((1U << PTE_FILE_BITS1) - 1)
+#define PTE_FILE_MASK2 ((1U << PTE_FILE_BITS2) - 1)
+
+#define PTE_FILE_LSHIFT2 (PTE_FILE_BITS1)
+#define PTE_FILE_LSHIFT3 (PTE_FILE_BITS1 + PTE_FILE_BITS2)
+
+static __always_inline pgoff_t pte_to_pgoff(pte_t pte)
+{
+ return (pgoff_t)
+ (pte_bitop(pte.pte_low, PTE_FILE_SHIFT1, PTE_FILE_MASK1, 0) +
+ pte_bitop(pte.pte_low, PTE_FILE_SHIFT2, PTE_FILE_MASK2, PTE_FILE_LSHIFT2) +
+ pte_bitop(pte.pte_low, PTE_FILE_SHIFT3, -1UL, PTE_FILE_LSHIFT3));
+}
+
+static __always_inline pte_t pgoff_to_pte(pgoff_t off)
+{
+ return (pte_t){
+ .pte_low =
+ pte_bitop(off, 0, PTE_FILE_MASK1, PTE_FILE_SHIFT1) +
+ pte_bitop(off, PTE_FILE_LSHIFT2, PTE_FILE_MASK2, PTE_FILE_SHIFT2) +
+ pte_bitop(off, PTE_FILE_LSHIFT3, -1UL, PTE_FILE_SHIFT3) +
+ _PAGE_FILE,
+ };
+}
#endif /* CONFIG_MEM_SOFT_DIRTY */
#define VMALLOC_START _AC(0xffffc90000000000, UL)
#define VMALLOC_END _AC(0xffffe8ffffffffff, UL)
#define VMEMMAP_START _AC(0xffffea0000000000, UL)
-#define MODULES_VADDR _AC(0xffffffffa0000000, UL)
+#define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE)
#define MODULES_END _AC(0xffffffffff000000, UL)
#define MODULES_LEN (MODULES_END - MODULES_VADDR)
*/
extern pte_t *lookup_address(unsigned long address, unsigned int *level);
extern phys_addr_t slow_virt_to_phys(void *__address);
-
+extern int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
+ unsigned numpages, unsigned long page_flags);
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_PGTABLE_DEFS_H */
#include <linux/cache.h>
#include <linux/threads.h>
#include <linux/math64.h>
-#include <linux/init.h>
#include <linux/err.h>
#include <linux/irqflags.h>
extern u16 __read_mostly tlb_lld_4k[NR_INFO];
extern u16 __read_mostly tlb_lld_2m[NR_INFO];
extern u16 __read_mostly tlb_lld_4m[NR_INFO];
+extern u16 __read_mostly tlb_lld_1g[NR_INFO];
extern s8 __read_mostly tlb_flushall_shift;
/*
u32 ymmh_space[64];
};
+/* We don't support LWP yet: */
+struct lwp_struct {
+ u8 reserved[128];
+};
+
+struct bndregs_struct {
+ u64 bndregs[8];
+} __packed;
+
+struct bndcsr_struct {
+ u64 cfg_reg_u;
+ u64 status_reg;
+} __packed;
+
struct xsave_hdr_struct {
u64 xstate_bv;
u64 reserved1[2];
struct i387_fxsave_struct i387;
struct xsave_hdr_struct xsave_hdr;
struct ymmh_struct ymmh;
+ struct lwp_struct lwp;
+ struct bndregs_struct bndregs;
+ struct bndcsr_struct bndcsr;
/* new processor state extensions will go here */
} __attribute__ ((packed, aligned (64)));
#endif
}
-static inline void __monitor(const void *eax, unsigned long ecx,
- unsigned long edx)
-{
- /* "monitor %eax, %ecx, %edx;" */
- asm volatile(".byte 0x0f, 0x01, 0xc8;"
- :: "a" (eax), "c" (ecx), "d"(edx));
-}
-
-static inline void __mwait(unsigned long eax, unsigned long ecx)
-{
- /* "mwait %eax, %ecx;" */
- asm volatile(".byte 0x0f, 0x01, 0xc9;"
- :: "a" (eax), "c" (ecx));
-}
-
-static inline void __sti_mwait(unsigned long eax, unsigned long ecx)
-{
- trace_hardirqs_on();
- /* "mwait %eax, %ecx;" */
- asm volatile("sti; .byte 0x0f, 0x01, 0xc9;"
- :: "a" (eax), "c" (ecx));
-}
-
extern void select_idle_routine(const struct cpuinfo_x86 *c);
extern void init_amd_e400_c1e_mask(void);
#endif /* !__i386__ */
-#include <linux/init.h>
#ifdef CONFIG_PARAVIRT
#include <asm/paravirt_types.h>
#endif
#include <uapi/asm/setup.h>
-
#define COMMAND_LINE_SIZE 2048
#include <linux/linkage.h>
#include <asm/bootparam.h>
#include <asm/x86_init.h>
+extern u64 relocated_ramdisk;
+
/* Interrupt control for vSMPowered x86_64 systems */
#ifdef CONFIG_X86_64
void vsmp_init(void);
#define _ASM_X86_SMP_H
#ifndef __ASSEMBLY__
#include <linux/cpumask.h>
-#include <linux/init.h>
#include <asm/percpu.h>
/*
#ifndef _ASM_X86_TIMER_H
#define _ASM_X86_TIMER_H
-#include <linux/init.h>
#include <linux/pm.h>
#include <linux/percpu.h>
#include <linux/interrupt.h>
+#include <linux/math64.h>
#define TICK_SIZE (tick_nsec / 1000)
extern int no_timer_check;
-/* Accelerators for sched_clock()
- * convert from cycles(64bits) => nanoseconds (64bits)
- * basic equation:
- * ns = cycles / (freq / ns_per_sec)
- * ns = cycles * (ns_per_sec / freq)
- * ns = cycles * (10^9 / (cpu_khz * 10^3))
- * ns = cycles * (10^6 / cpu_khz)
+/*
+ * We use the full linear equation: f(x) = a + b*x, in order to allow
+ * a continuous function in the face of dynamic freq changes.
*
- * Then we use scaling math (suggested by george@mvista.com) to get:
- * ns = cycles * (10^6 * SC / cpu_khz) / SC
- * ns = cycles * cyc2ns_scale / SC
+ * Continuity means that when our frequency changes our slope (b); we want to
+ * ensure that: f(t) == f'(t), which gives: a + b*t == a' + b'*t.
*
- * And since SC is a constant power of two, we can convert the div
- * into a shift.
+ * Without an offset (a) the above would not be possible.
*
- * We can use khz divisor instead of mhz to keep a better precision, since
- * cyc2ns_scale is limited to 10^6 * 2^10, which fits in 32 bits.
- * (mathieu.desnoyers@polymtl.ca)
- *
- * -johnstul@us.ibm.com "math is hard, lets go shopping!"
- *
- * In:
- *
- * ns = cycles * cyc2ns_scale / SC
- *
- * Although we may still have enough bits to store the value of ns,
- * in some cases, we may not have enough bits to store cycles * cyc2ns_scale,
- * leading to an incorrect result.
- *
- * To avoid this, we can decompose 'cycles' into quotient and remainder
- * of division by SC. Then,
- *
- * ns = (quot * SC + rem) * cyc2ns_scale / SC
- * = quot * cyc2ns_scale + (rem * cyc2ns_scale) / SC
- *
- * - sqazi@google.com
+ * See the comment near cycles_2_ns() for details on how we compute (b).
*/
-
-DECLARE_PER_CPU(unsigned long, cyc2ns);
-DECLARE_PER_CPU(unsigned long long, cyc2ns_offset);
-
-#define CYC2NS_SCALE_FACTOR 10 /* 2^10, carefully chosen */
-
-static inline unsigned long long __cycles_2_ns(unsigned long long cyc)
-{
- int cpu = smp_processor_id();
- unsigned long long ns = per_cpu(cyc2ns_offset, cpu);
- ns += mult_frac(cyc, per_cpu(cyc2ns, cpu),
- (1UL << CYC2NS_SCALE_FACTOR));
- return ns;
-}
-
-static inline unsigned long long cycles_2_ns(unsigned long long cyc)
-{
- unsigned long long ns;
- unsigned long flags;
-
- local_irq_save(flags);
- ns = __cycles_2_ns(cyc);
- local_irq_restore(flags);
-
- return ns;
-}
+struct cyc2ns_data {
+ u32 cyc2ns_mul;
+ u32 cyc2ns_shift;
+ u64 cyc2ns_offset;
+ u32 __count;
+ /* u32 hole */
+}; /* 24 bytes -- do not grow */
+
+extern struct cyc2ns_data *cyc2ns_read_begin(void);
+extern void cyc2ns_read_end(struct cyc2ns_data *);
#endif /* _ASM_X86_TIMER_H */
extern void tsc_save_sched_clock_state(void);
extern void tsc_restore_sched_clock_state(void);
+/* MSR based TSC calibration for Intel Atom SoC platforms */
+int try_msr_calibrate_tsc(unsigned long *fast_calibrate);
+
#endif /* _ASM_X86_TSC_H */
/*
* Test whether a block of memory is a valid user space address.
* Returns 0 if the range is valid, nonzero otherwise.
- *
- * This is equivalent to the following test:
- * (u33)addr + (u33)size > (u33)current->addr_limit.seg (u65 for x86_64)
- *
- * This needs 33-bit (65-bit for x86_64) arithmetic. We have a carry...
*/
+static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, unsigned long limit)
+{
+ /*
+ * If we have used "sizeof()" for the size,
+ * we know it won't overflow the limit (but
+ * it might overflow the 'addr', so it's
+ * important to subtract the size from the
+ * limit, not add it to the address).
+ */
+ if (__builtin_constant_p(size))
+ return addr > limit - size;
+
+ /* Arbitrary sizes? Be careful about overflow */
+ addr += size;
+ if (addr < size)
+ return true;
+ return addr > limit;
+}
#define __range_not_ok(addr, size, limit) \
({ \
- unsigned long flag, roksum; \
__chk_user_ptr(addr); \
- asm("add %3,%1 ; sbb %0,%0 ; cmp %1,%4 ; sbb $0,%0" \
- : "=&r" (flag), "=r" (roksum) \
- : "1" (addr), "g" ((long)(size)), \
- "rm" (limit)); \
- flag; \
+ __chk_range_not_ok((unsigned long __force)(addr), size, limit); \
})
/**
* this function, memory access functions may still return -EFAULT.
*/
#define access_ok(type, addr, size) \
- (likely(__range_not_ok(addr, size, user_addr_max()) == 0))
+ likely(!__range_not_ok(addr, size, user_addr_max()))
/*
* The exception table consists of pairs of addresses relative to the
unsigned long __must_check clear_user(void __user *mem, unsigned long len);
unsigned long __must_check __clear_user(void __user *mem, unsigned long len);
+extern void __cmpxchg_wrong_size(void)
+ __compiletime_error("Bad argument size for cmpxchg");
+
+#define __user_atomic_cmpxchg_inatomic(uval, ptr, old, new, size) \
+({ \
+ int __ret = 0; \
+ __typeof__(ptr) __uval = (uval); \
+ __typeof__(*(ptr)) __old = (old); \
+ __typeof__(*(ptr)) __new = (new); \
+ switch (size) { \
+ case 1: \
+ { \
+ asm volatile("\t" ASM_STAC "\n" \
+ "1:\t" LOCK_PREFIX "cmpxchgb %4, %2\n" \
+ "2:\t" ASM_CLAC "\n" \
+ "\t.section .fixup, \"ax\"\n" \
+ "3:\tmov %3, %0\n" \
+ "\tjmp 2b\n" \
+ "\t.previous\n" \
+ _ASM_EXTABLE(1b, 3b) \
+ : "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
+ : "i" (-EFAULT), "q" (__new), "1" (__old) \
+ : "memory" \
+ ); \
+ break; \
+ } \
+ case 2: \
+ { \
+ asm volatile("\t" ASM_STAC "\n" \
+ "1:\t" LOCK_PREFIX "cmpxchgw %4, %2\n" \
+ "2:\t" ASM_CLAC "\n" \
+ "\t.section .fixup, \"ax\"\n" \
+ "3:\tmov %3, %0\n" \
+ "\tjmp 2b\n" \
+ "\t.previous\n" \
+ _ASM_EXTABLE(1b, 3b) \
+ : "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
+ : "i" (-EFAULT), "r" (__new), "1" (__old) \
+ : "memory" \
+ ); \
+ break; \
+ } \
+ case 4: \
+ { \
+ asm volatile("\t" ASM_STAC "\n" \
+ "1:\t" LOCK_PREFIX "cmpxchgl %4, %2\n" \
+ "2:\t" ASM_CLAC "\n" \
+ "\t.section .fixup, \"ax\"\n" \
+ "3:\tmov %3, %0\n" \
+ "\tjmp 2b\n" \
+ "\t.previous\n" \
+ _ASM_EXTABLE(1b, 3b) \
+ : "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
+ : "i" (-EFAULT), "r" (__new), "1" (__old) \
+ : "memory" \
+ ); \
+ break; \
+ } \
+ case 8: \
+ { \
+ if (!IS_ENABLED(CONFIG_X86_64)) \
+ __cmpxchg_wrong_size(); \
+ \
+ asm volatile("\t" ASM_STAC "\n" \
+ "1:\t" LOCK_PREFIX "cmpxchgq %4, %2\n" \
+ "2:\t" ASM_CLAC "\n" \
+ "\t.section .fixup, \"ax\"\n" \
+ "3:\tmov %3, %0\n" \
+ "\tjmp 2b\n" \
+ "\t.previous\n" \
+ _ASM_EXTABLE(1b, 3b) \
+ : "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
+ : "i" (-EFAULT), "r" (__new), "1" (__old) \
+ : "memory" \
+ ); \
+ break; \
+ } \
+ default: \
+ __cmpxchg_wrong_size(); \
+ } \
+ *__uval = __old; \
+ __ret; \
+})
+
+#define user_atomic_cmpxchg_inatomic(uval, ptr, old, new) \
+({ \
+ access_ok(VERIFY_WRITE, (ptr), sizeof(*(ptr))) ? \
+ __user_atomic_cmpxchg_inatomic((uval), (ptr), \
+ (old), (new), sizeof(*(ptr))) : \
+ -EFAULT; \
+})
+
/*
* movsl can be slow when source and dest are not both 8-byte aligned
*/
static __must_check __always_inline int
__copy_from_user_inatomic(void *dst, const void __user *src, unsigned size)
{
- return __copy_from_user_nocheck(dst, (__force const void *)src, size);
+ return __copy_from_user_nocheck(dst, src, size);
}
static __must_check __always_inline int
__copy_to_user_inatomic(void __user *dst, const void *src, unsigned size)
{
- return __copy_to_user_nocheck((__force void *)dst, src, size);
+ return __copy_to_user_nocheck(dst, src, size);
}
extern long __copy_user_nocache(void *dst, const void __user *src,
#define XSTATE_FP 0x1
#define XSTATE_SSE 0x2
#define XSTATE_YMM 0x4
+#define XSTATE_BNDREGS 0x8
+#define XSTATE_BNDCSR 0x10
#define XSTATE_FPSSE (XSTATE_FP | XSTATE_SSE)
#define XSAVE_YMM_SIZE 256
#define XSAVE_YMM_OFFSET (XSAVE_HDR_SIZE + XSAVE_HDR_OFFSET)
-/*
- * These are the features that the OS can handle currently.
- */
-#define XCNTXT_MASK (XSTATE_FP | XSTATE_SSE | XSTATE_YMM)
+/* Supported features which support lazy state saving */
+#define XSTATE_LAZY (XSTATE_FP | XSTATE_SSE | XSTATE_YMM)
+
+/* Supported features which require eager state saving */
+#define XSTATE_EAGER (XSTATE_BNDREGS | XSTATE_BNDCSR)
+
+/* All currently supported features */
+#define XCNTXT_MASK (XSTATE_LAZY | XSTATE_EAGER)
#ifdef CONFIG_X86_64
#define REX_PREFIX "0x48, "
#define SETUP_E820_EXT 1
#define SETUP_DTB 2
#define SETUP_PCI 3
+#define SETUP_EFI 4
/* ram_size flags */
#define RAMDISK_IMAGE_START_MASK 0x07FF
#define XLF_CAN_BE_LOADED_ABOVE_4G (1<<1)
#define XLF_EFI_HANDOVER_32 (1<<2)
#define XLF_EFI_HANDOVER_64 (1<<3)
+#define XLF_EFI_KEXEC (1<<4)
#ifndef __ASSEMBLY__
#ifndef _ASM_X86_STAT_H
#define _ASM_X86_STAT_H
+#include <asm/posix_types.h>
+
#define STAT_HAVE_NSEC 1
#ifdef __i386__
#else /* __i386__ */
struct stat {
- unsigned long st_dev;
- unsigned long st_ino;
- unsigned long st_nlink;
-
- unsigned int st_mode;
- unsigned int st_uid;
- unsigned int st_gid;
- unsigned int __pad0;
- unsigned long st_rdev;
- long st_size;
- long st_blksize;
- long st_blocks; /* Number 512-byte blocks allocated. */
-
- unsigned long st_atime;
- unsigned long st_atime_nsec;
- unsigned long st_mtime;
- unsigned long st_mtime_nsec;
- unsigned long st_ctime;
- unsigned long st_ctime_nsec;
- long __unused[3];
+ __kernel_ulong_t st_dev;
+ __kernel_ulong_t st_ino;
+ __kernel_ulong_t st_nlink;
+
+ unsigned int st_mode;
+ unsigned int st_uid;
+ unsigned int st_gid;
+ unsigned int __pad0;
+ __kernel_ulong_t st_rdev;
+ __kernel_long_t st_size;
+ __kernel_long_t st_blksize;
+ __kernel_long_t st_blocks; /* Number 512-byte blocks allocated. */
+
+ __kernel_ulong_t st_atime;
+ __kernel_ulong_t st_atime_nsec;
+ __kernel_ulong_t st_mtime;
+ __kernel_ulong_t st_mtime_nsec;
+ __kernel_ulong_t st_ctime;
+ __kernel_ulong_t st_ctime_nsec;
+ __kernel_long_t __unused[3];
};
/* We don't need to memset the whole thing just to initialize the padding */
obj-y += syscall_$(BITS).o
obj-$(CONFIG_X86_64) += vsyscall_64.o
obj-$(CONFIG_X86_64) += vsyscall_emu_64.o
+obj-$(CONFIG_SYSFS) += ksysfs.o
obj-y += bootflag.o e820.o
obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
obj-y += alternative.o i8253.o pci-nommu.o hw_breakpoint.o
-obj-y += tsc.o io_delay.o rtc.o
+obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
obj-y += pci-iommu_table.o
obj-y += resource.o
obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
-obj-$(CONFIG_MICROCODE_EARLY) += microcode_core_early.o
-obj-$(CONFIG_MICROCODE_INTEL_EARLY) += microcode_intel_early.o
-obj-$(CONFIG_MICROCODE_INTEL_LIB) += microcode_intel_lib.o
-microcode-y := microcode_core.o
-microcode-$(CONFIG_MICROCODE_INTEL) += microcode_intel.o
-microcode-$(CONFIG_MICROCODE_AMD) += microcode_amd.o
-obj-$(CONFIG_MICROCODE_AMD_EARLY) += microcode_amd_early.o
-obj-$(CONFIG_MICROCODE) += microcode.o
-
obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_PERF_EVENTS) += perf_regs.o
obj-$(CONFIG_TRACING) += tracepoint.o
+obj-$(CONFIG_IOSF_MBI) += iosf_mbi.o
###
# 64 bit specific files
}
EXPORT_SYMBOL_GPL(acpi_processor_ffh_cstate_probe);
-/*
- * This uses new MONITOR/MWAIT instructions on P4 processors with PNI,
- * which can obviate IPI to trigger checking of need_resched.
- * We execute MONITOR against need_resched and enter optimized wait state
- * through MWAIT. Whenever someone changes need_resched, we would be woken
- * up from MWAIT (without an IPI).
- *
- * New with Core Duo processors, MWAIT can take some hints based on CPU
- * capability.
- */
-void mwait_idle_with_hints(unsigned long ax, unsigned long cx)
-{
- if (!need_resched()) {
- if (this_cpu_has(X86_FEATURE_CLFLUSH_MONITOR))
- clflush((void *)¤t_thread_info()->flags);
-
- __monitor((void *)¤t_thread_info()->flags, 0, 0);
- smp_mb();
- if (!need_resched())
- __mwait(ax, cx);
- }
-}
-
void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx)
{
unsigned int cpu = smp_processor_id();
*/
physid_mask_t phys_cpu_present_map;
+/*
+ * Processor to be disabled specified by kernel parameter
+ * disable_cpu_apicid=<int>, mostly used for the kdump 2nd kernel to
+ * avoid undefined behaviour caused by sending INIT from AP to BSP.
+ */
+static unsigned int disabled_cpu_apicid __read_mostly = BAD_APICID;
+
/*
* Map cpu index to physical APIC ID
*/
*/
static inline void __smp_error_interrupt(struct pt_regs *regs)
{
- u32 v0, v1;
+ u32 v;
u32 i = 0;
static const char * const error_interrupt_reason[] = {
"Send CS error", /* APIC Error Bit 0 */
};
/* First tickle the hardware, only then report what went on. -- REW */
- v0 = apic_read(APIC_ESR);
apic_write(APIC_ESR, 0);
- v1 = apic_read(APIC_ESR);
+ v = apic_read(APIC_ESR);
ack_APIC_irq();
atomic_inc(&irq_err_count);
- apic_printk(APIC_DEBUG, KERN_DEBUG "APIC error on CPU%d: %02x(%02x)",
- smp_processor_id(), v0 , v1);
+ apic_printk(APIC_DEBUG, KERN_DEBUG "APIC error on CPU%d: %02x",
+ smp_processor_id(), v);
- v1 = v1 & 0xff;
- while (v1) {
- if (v1 & 0x1)
+ v &= 0xff;
+ while (v) {
+ if (v & 0x1)
apic_printk(APIC_DEBUG, KERN_CONT " : %s", error_interrupt_reason[i]);
i++;
- v1 >>= 1;
+ v >>= 1;
}
apic_printk(APIC_DEBUG, KERN_CONT "\n");
bool boot_cpu_detected = physid_isset(boot_cpu_physical_apicid,
phys_cpu_present_map);
+ /*
+ * boot_cpu_physical_apicid is designed to have the apicid
+ * returned by read_apic_id(), i.e, the apicid of the
+ * currently booting-up processor. However, on some platforms,
+ * it is temporarily modified by the apicid reported as BSP
+ * through MP table. Concretely:
+ *
+ * - arch/x86/kernel/mpparse.c: MP_processor_info()
+ * - arch/x86/mm/amdtopology.c: amd_numa_init()
+ * - arch/x86/platform/visws/visws_quirks.c: MP_processor_info()
+ *
+ * This function is executed with the modified
+ * boot_cpu_physical_apicid. So, disabled_cpu_apicid kernel
+ * parameter doesn't work to disable APs on kdump 2nd kernel.
+ *
+ * Since fixing handling of boot_cpu_physical_apicid requires
+ * another discussion and tests on each platform, we leave it
+ * for now and here we use read_apic_id() directly in this
+ * function, generic_processor_info().
+ */
+ if (disabled_cpu_apicid != BAD_APICID &&
+ disabled_cpu_apicid != read_apic_id() &&
+ disabled_cpu_apicid == apicid) {
+ int thiscpu = num_processors + disabled_cpus;
+
+ pr_warning("APIC: Disabling requested cpu."
+ " Processor %d/0x%x ignored.\n",
+ thiscpu, apicid);
+
+ disabled_cpus++;
+ return -ENODEV;
+ }
+
/*
* If boot cpu has not been detected yet, then only allow upto
* nr_cpu_ids - 1 processors and keep one slot free for boot cpu
* that is using request_resource
*/
late_initcall(lapic_insert_resource);
+
+static int __init apic_set_disabled_cpu_apicid(char *arg)
+{
+ if (!arg || !get_option(&arg, &disabled_cpu_apicid))
+ return -EINVAL;
+
+ return 0;
+}
+early_param("disable_cpu_apicid", apic_set_disabled_cpu_apicid);
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/ctype.h>
-#include <linux/init.h>
#include <linux/hardirq.h>
#include <linux/module.h>
#include <asm/smp.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/ctype.h>
-#include <linux/init.h>
#include <linux/errno.h>
#include <asm/fixmap.h>
#include <asm/mpspec.h>
if (test_bit(vector, used_vectors))
goto next;
- for_each_cpu_and(new_cpu, tmp_mask, cpu_online_mask)
- if (per_cpu(vector_irq, new_cpu)[vector] != -1)
+ for_each_cpu_and(new_cpu, tmp_mask, cpu_online_mask) {
+ if (per_cpu(vector_irq, new_cpu)[vector] > VECTOR_UNDEFINED)
goto next;
+ }
/* Found one! */
current_vector = vector;
current_offset = offset;
vector = cfg->vector;
for_each_cpu_and(cpu, cfg->domain, cpu_online_mask)
- per_cpu(vector_irq, cpu)[vector] = -1;
+ per_cpu(vector_irq, cpu)[vector] = VECTOR_UNDEFINED;
cfg->vector = 0;
cpumask_clear(cfg->domain);
if (likely(!cfg->move_in_progress))
return;
for_each_cpu_and(cpu, cfg->old_domain, cpu_online_mask) {
- for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS;
- vector++) {
+ for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS; vector++) {
if (per_cpu(vector_irq, cpu)[vector] != irq)
continue;
- per_cpu(vector_irq, cpu)[vector] = -1;
+ per_cpu(vector_irq, cpu)[vector] = VECTOR_UNDEFINED;
break;
}
}
/* Mark the free vectors */
for (vector = 0; vector < NR_VECTORS; ++vector) {
irq = per_cpu(vector_irq, cpu)[vector];
- if (irq < 0)
+ if (irq <= VECTOR_UNDEFINED)
continue;
cfg = irq_cfg(irq);
if (!cpumask_test_cpu(cpu, cfg->domain))
- per_cpu(vector_irq, cpu)[vector] = -1;
+ per_cpu(vector_irq, cpu)[vector] = VECTOR_UNDEFINED;
}
raw_spin_unlock(&vector_lock);
}
me = smp_processor_id();
for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS; vector++) {
- unsigned int irq;
+ int irq;
unsigned int irr;
struct irq_desc *desc;
struct irq_cfg *cfg;
irq = __this_cpu_read(vector_irq[vector]);
- if (irq == -1)
+ if (irq <= VECTOR_UNDEFINED)
continue;
desc = irq_to_desc(irq);
#include <linux/cpumask.h>
#include <linux/interrupt.h>
-#include <linux/init.h>
#include <linux/mm.h>
#include <linux/delay.h>
#define pr_fmt(fmt) "summit: %s: " fmt, __func__
#include <linux/mm.h>
-#include <linux/init.h>
#include <asm/io.h>
#include <asm/bios_ebda.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/ctype.h>
-#include <linux/init.h>
#include <linux/dmar.h>
#include <linux/cpu.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/ctype.h>
-#include <linux/init.h>
#include <linux/dmar.h>
#include <asm/smp.h>
endif
obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_p6.o perf_event_knc.o perf_event_p4.o
obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o
-obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_uncore.o
+obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_uncore.o perf_event_intel_rapl.o
endif
obj-$(CONFIG_X86_MCE) += mcheck/
obj-$(CONFIG_MTRR) += mtrr/
+obj-$(CONFIG_MICROCODE) += microcode/
obj-$(CONFIG_X86_LOCAL_APIC) += perfctr-watchdog.o perf_event_amd_ibs.o
#include <linux/export.h>
-#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/elf.h>
#include <linux/mm.h>
set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC);
set_cpu_cap(c, X86_FEATURE_NONSTOP_TSC);
if (!check_tsc_unstable())
- sched_clock_stable = 1;
+ set_sched_clock_stable();
}
#ifdef CONFIG_X86_64
}
/* Handle DTLB 2M and 4M sizes, fall back to L1 if L2 is disabled */
- if (!((eax >> 16) & mask)) {
- u32 a, b, c, d;
-
- cpuid(0x80000005, &a, &b, &c, &d);
- tlb_lld_2m[ENTRIES] = (a >> 16) & 0xff;
- } else {
+ if (!((eax >> 16) & mask))
+ tlb_lld_2m[ENTRIES] = (cpuid_eax(0x80000005) >> 16) & 0xff;
+ else
tlb_lld_2m[ENTRIES] = (eax >> 16) & mask;
- }
/* a 4M entry uses two 2M entries */
tlb_lld_4m[ENTRIES] = tlb_lld_2m[ENTRIES] >> 1;
#include <linux/bitops.h>
#include <linux/kernel.h>
-#include <linux/init.h>
#include <asm/processor.h>
#include <asm/e820.h>
u16 __read_mostly tlb_lld_4k[NR_INFO];
u16 __read_mostly tlb_lld_2m[NR_INFO];
u16 __read_mostly tlb_lld_4m[NR_INFO];
+u16 __read_mostly tlb_lld_1g[NR_INFO];
/*
* tlb_flushall_shift shows the balance point in replacing cr3 write
if (this_cpu->c_detect_tlb)
this_cpu->c_detect_tlb(c);
- printk(KERN_INFO "Last level iTLB entries: 4KB %d, 2MB %d, 4MB %d\n" \
- "Last level dTLB entries: 4KB %d, 2MB %d, 4MB %d\n" \
+ printk(KERN_INFO "Last level iTLB entries: 4KB %d, 2MB %d, 4MB %d\n"
+ "Last level dTLB entries: 4KB %d, 2MB %d, 4MB %d, 1GB %d\n"
"tlb_flushall_shift: %d\n",
tlb_lli_4k[ENTRIES], tlb_lli_2m[ENTRIES],
tlb_lli_4m[ENTRIES], tlb_lld_4k[ENTRIES],
tlb_lld_2m[ENTRIES], tlb_lld_4m[ENTRIES],
- tlb_flushall_shift);
+ tlb_lld_1g[ENTRIES], tlb_flushall_shift);
}
void detect_ht(struct cpuinfo_x86 *c)
-#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/delay.h>
#include <linux/pci.h>
-#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/string.h>
set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC);
set_cpu_cap(c, X86_FEATURE_NONSTOP_TSC);
if (!check_tsc_unstable())
- sched_clock_stable = 1;
+ set_sched_clock_stable();
}
/* Penwell and Cloverview have the TSC which doesn't sleep on S3 */
#define TLB_DATA0_2M_4M 0x23
#define STLB_4K 0x41
+#define STLB_4K_2M 0x42
static const struct _tlb_table intel_tlb_table[] = {
{ 0x01, TLB_INST_4K, 32, " TLB_INST 4 KByte pages, 4-way set associative" },
{ 0x5b, TLB_DATA_4K_4M, 64, " TLB_DATA 4 KByte and 4 MByte pages" },
{ 0x5c, TLB_DATA_4K_4M, 128, " TLB_DATA 4 KByte and 4 MByte pages" },
{ 0x5d, TLB_DATA_4K_4M, 256, " TLB_DATA 4 KByte and 4 MByte pages" },
+ { 0x61, TLB_INST_4K, 48, " TLB_INST 4 KByte pages, full associative" },
+ { 0x63, TLB_DATA_1G, 4, " TLB_DATA 1 GByte pages, 4-way set associative" },
+ { 0x76, TLB_INST_2M_4M, 8, " TLB_INST 2-MByte or 4-MByte pages, fully associative" },
{ 0xb0, TLB_INST_4K, 128, " TLB_INST 4 KByte pages, 4-way set associative" },
{ 0xb1, TLB_INST_2M_4M, 4, " TLB_INST 2M pages, 4-way, 8 entries or 4M pages, 4-way entries" },
{ 0xb2, TLB_INST_4K, 64, " TLB_INST 4KByte pages, 4-way set associative" },
{ 0xb3, TLB_DATA_4K, 128, " TLB_DATA 4 KByte pages, 4-way set associative" },
{ 0xb4, TLB_DATA_4K, 256, " TLB_DATA 4 KByte pages, 4-way associative" },
+ { 0xb5, TLB_INST_4K, 64, " TLB_INST 4 KByte pages, 8-way set ssociative" },
+ { 0xb6, TLB_INST_4K, 128, " TLB_INST 4 KByte pages, 8-way set ssociative" },
{ 0xba, TLB_DATA_4K, 64, " TLB_DATA 4 KByte pages, 4-way associative" },
{ 0xc0, TLB_DATA_4K_4M, 8, " TLB_DATA 4 KByte and 4 MByte pages, 4-way associative" },
+ { 0xc1, STLB_4K_2M, 1024, " STLB 4 KByte and 2 MByte pages, 8-way associative" },
+ { 0xc2, TLB_DATA_2M_4M, 16, " DTLB 2 MByte/4MByte pages, 4-way associative" },
{ 0xca, STLB_4K, 512, " STLB 4 KByte pages, 4-way associative" },
{ 0x00, 0, 0 }
};
if (tlb_lld_4k[ENTRIES] < intel_tlb_table[k].entries)
tlb_lld_4k[ENTRIES] = intel_tlb_table[k].entries;
break;
+ case STLB_4K_2M:
+ if (tlb_lli_4k[ENTRIES] < intel_tlb_table[k].entries)
+ tlb_lli_4k[ENTRIES] = intel_tlb_table[k].entries;
+ if (tlb_lld_4k[ENTRIES] < intel_tlb_table[k].entries)
+ tlb_lld_4k[ENTRIES] = intel_tlb_table[k].entries;
+ if (tlb_lli_2m[ENTRIES] < intel_tlb_table[k].entries)
+ tlb_lli_2m[ENTRIES] = intel_tlb_table[k].entries;
+ if (tlb_lld_2m[ENTRIES] < intel_tlb_table[k].entries)
+ tlb_lld_2m[ENTRIES] = intel_tlb_table[k].entries;
+ if (tlb_lli_4m[ENTRIES] < intel_tlb_table[k].entries)
+ tlb_lli_4m[ENTRIES] = intel_tlb_table[k].entries;
+ if (tlb_lld_4m[ENTRIES] < intel_tlb_table[k].entries)
+ tlb_lld_4m[ENTRIES] = intel_tlb_table[k].entries;
+ break;
case TLB_INST_ALL:
if (tlb_lli_4k[ENTRIES] < intel_tlb_table[k].entries)
tlb_lli_4k[ENTRIES] = intel_tlb_table[k].entries;
if (tlb_lld_4m[ENTRIES] < intel_tlb_table[k].entries)
tlb_lld_4m[ENTRIES] = intel_tlb_table[k].entries;
break;
+ case TLB_DATA_1G:
+ if (tlb_lld_1g[ENTRIES] < intel_tlb_table[k].entries)
+ tlb_lld_1g[ENTRIES] = intel_tlb_table[k].entries;
+ break;
}
}
#include <linux/acpi.h>
#include <linux/cper.h>
#include <acpi/apei.h>
+#include <acpi/ghes.h>
#include <asm/mce.h>
#include "mce-internal.h"
-void apei_mce_report_mem_error(int corrected, struct cper_sec_mem_err *mem_err)
+void apei_mce_report_mem_error(int severity, struct cper_sec_mem_err *mem_err)
{
struct mce m;
- /* Only corrected MC is reported */
- if (!corrected || !(mem_err->validation_bits & CPER_MEM_VALID_PA))
+ if (!(mem_err->validation_bits & CPER_MEM_VALID_PA))
return;
mce_setup(&m);
m.bank = 1;
- /* Fake a memory read corrected error with unknown channel */
+ /* Fake a memory read error with unknown channel */
m.status = MCI_STATUS_VAL | MCI_STATUS_EN | MCI_STATUS_ADDRV | 0x9f;
+
+ if (severity >= GHES_SEV_RECOVERABLE)
+ m.status |= MCI_STATUS_UC;
+ if (severity >= GHES_SEV_PANIC)
+ m.status |= MCI_STATUS_PCC;
+
m.addr = mem_err->physical_addr;
mce_log(&m);
mce_notify_irq();
static void mce_start_timer(unsigned int cpu, struct timer_list *t)
{
- unsigned long iv = mce_adjust_timer(check_interval * HZ);
-
- __this_cpu_write(mce_next_interval, iv);
+ unsigned long iv = check_interval * HZ;
if (mca_cfg.ignore_ce || !iv)
return;
+ per_cpu(mce_next_interval, cpu) = iv;
+
t->expires = round_jiffies(jiffies + iv);
- add_timer_on(t, smp_processor_id());
+ add_timer_on(t, cpu);
}
static void __mcheck_cpu_init_timer(void)
dev->release = &mce_device_release;
err = device_register(dev);
- if (err)
+ if (err) {
+ put_device(dev);
return err;
+ }
for (i = 0; mce_device_attrs[i]; i++) {
err = device_create_file(dev, mce_device_attrs[i]);
*/
#include <linux/gfp.h>
-#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/percpu.h>
#include <linux/sched.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/types.h>
-#include <linux/init.h>
#include <linux/smp.h>
#include <asm/processor.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/types.h>
-#include <linux/init.h>
#include <asm/processor.h>
#include <asm/mce.h>
--- /dev/null
+microcode-y := core.o
+obj-$(CONFIG_MICROCODE) += microcode.o
+microcode-$(CONFIG_MICROCODE_INTEL) += intel.o intel_lib.o
+microcode-$(CONFIG_MICROCODE_AMD) += amd.o
+obj-$(CONFIG_MICROCODE_EARLY) += core_early.o
+obj-$(CONFIG_MICROCODE_INTEL_EARLY) += intel_early.o
+obj-$(CONFIG_MICROCODE_AMD_EARLY) += amd_early.o
--- /dev/null
+/*
+ * AMD CPU Microcode Update Driver for Linux
+ * Copyright (C) 2008-2011 Advanced Micro Devices Inc.
+ *
+ * Author: Peter Oruba <peter.oruba@amd.com>
+ *
+ * Based on work by:
+ * Tigran Aivazian <tigran@aivazian.fsnet.co.uk>
+ *
+ * Maintainers:
+ * Andreas Herrmann <herrmann.der.user@googlemail.com>
+ * Borislav Petkov <bp@alien8.de>
+ *
+ * This driver allows to upgrade microcode on F10h AMD
+ * CPUs and later.
+ *
+ * Licensed under the terms of the GNU General Public
+ * License version 2. See file COPYING for details.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/firmware.h>
+#include <linux/pci_ids.h>
+#include <linux/uaccess.h>
+#include <linux/vmalloc.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+
+#include <asm/microcode.h>
+#include <asm/processor.h>
+#include <asm/msr.h>
+#include <asm/microcode_amd.h>
+
+MODULE_DESCRIPTION("AMD Microcode Update Driver");
+MODULE_AUTHOR("Peter Oruba");
+MODULE_LICENSE("GPL v2");
+
+static struct equiv_cpu_entry *equiv_cpu_table;
+
+struct ucode_patch {
+ struct list_head plist;
+ void *data;
+ u32 patch_id;
+ u16 equiv_cpu;
+};
+
+static LIST_HEAD(pcache);
+
+static u16 __find_equiv_id(unsigned int cpu)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ return find_equiv_id(equiv_cpu_table, uci->cpu_sig.sig);
+}
+
+static u32 find_cpu_family_by_equiv_cpu(u16 equiv_cpu)
+{
+ int i = 0;
+
+ BUG_ON(!equiv_cpu_table);
+
+ while (equiv_cpu_table[i].equiv_cpu != 0) {
+ if (equiv_cpu == equiv_cpu_table[i].equiv_cpu)
+ return equiv_cpu_table[i].installed_cpu;
+ i++;
+ }
+ return 0;
+}
+
+/*
+ * a small, trivial cache of per-family ucode patches
+ */
+static struct ucode_patch *cache_find_patch(u16 equiv_cpu)
+{
+ struct ucode_patch *p;
+
+ list_for_each_entry(p, &pcache, plist)
+ if (p->equiv_cpu == equiv_cpu)
+ return p;
+ return NULL;
+}
+
+static void update_cache(struct ucode_patch *new_patch)
+{
+ struct ucode_patch *p;
+
+ list_for_each_entry(p, &pcache, plist) {
+ if (p->equiv_cpu == new_patch->equiv_cpu) {
+ if (p->patch_id >= new_patch->patch_id)
+ /* we already have the latest patch */
+ return;
+
+ list_replace(&p->plist, &new_patch->plist);
+ kfree(p->data);
+ kfree(p);
+ return;
+ }
+ }
+ /* no patch found, add it */
+ list_add_tail(&new_patch->plist, &pcache);
+}
+
+static void free_cache(void)
+{
+ struct ucode_patch *p, *tmp;
+
+ list_for_each_entry_safe(p, tmp, &pcache, plist) {
+ __list_del(p->plist.prev, p->plist.next);
+ kfree(p->data);
+ kfree(p);
+ }
+}
+
+static struct ucode_patch *find_patch(unsigned int cpu)
+{
+ u16 equiv_id;
+
+ equiv_id = __find_equiv_id(cpu);
+ if (!equiv_id)
+ return NULL;
+
+ return cache_find_patch(equiv_id);
+}
+
+static int collect_cpu_info_amd(int cpu, struct cpu_signature *csig)
+{
+ struct cpuinfo_x86 *c = &cpu_data(cpu);
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ struct ucode_patch *p;
+
+ csig->sig = cpuid_eax(0x00000001);
+ csig->rev = c->microcode;
+
+ /*
+ * a patch could have been loaded early, set uci->mc so that
+ * mc_bp_resume() can call apply_microcode()
+ */
+ p = find_patch(cpu);
+ if (p && (p->patch_id == csig->rev))
+ uci->mc = p->data;
+
+ pr_info("CPU%d: patch_level=0x%08x\n", cpu, csig->rev);
+
+ return 0;
+}
+
+static unsigned int verify_patch_size(u8 family, u32 patch_size,
+ unsigned int size)
+{
+ u32 max_size;
+
+#define F1XH_MPB_MAX_SIZE 2048
+#define F14H_MPB_MAX_SIZE 1824
+#define F15H_MPB_MAX_SIZE 4096
+#define F16H_MPB_MAX_SIZE 3458
+
+ switch (family) {
+ case 0x14:
+ max_size = F14H_MPB_MAX_SIZE;
+ break;
+ case 0x15:
+ max_size = F15H_MPB_MAX_SIZE;
+ break;
+ case 0x16:
+ max_size = F16H_MPB_MAX_SIZE;
+ break;
+ default:
+ max_size = F1XH_MPB_MAX_SIZE;
+ break;
+ }
+
+ if (patch_size > min_t(u32, size, max_size)) {
+ pr_err("patch size mismatch\n");
+ return 0;
+ }
+
+ return patch_size;
+}
+
+int __apply_microcode_amd(struct microcode_amd *mc_amd)
+{
+ u32 rev, dummy;
+
+ native_wrmsrl(MSR_AMD64_PATCH_LOADER, (u64)(long)&mc_amd->hdr.data_code);
+
+ /* verify patch application was successful */
+ native_rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy);
+ if (rev != mc_amd->hdr.patch_id)
+ return -1;
+
+ return 0;
+}
+
+int apply_microcode_amd(int cpu)
+{
+ struct cpuinfo_x86 *c = &cpu_data(cpu);
+ struct microcode_amd *mc_amd;
+ struct ucode_cpu_info *uci;
+ struct ucode_patch *p;
+ u32 rev, dummy;
+
+ BUG_ON(raw_smp_processor_id() != cpu);
+
+ uci = ucode_cpu_info + cpu;
+
+ p = find_patch(cpu);
+ if (!p)
+ return 0;
+
+ mc_amd = p->data;
+ uci->mc = p->data;
+
+ rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy);
+
+ /* need to apply patch? */
+ if (rev >= mc_amd->hdr.patch_id) {
+ c->microcode = rev;
+ uci->cpu_sig.rev = rev;
+ return 0;
+ }
+
+ if (__apply_microcode_amd(mc_amd)) {
+ pr_err("CPU%d: update failed for patch_level=0x%08x\n",
+ cpu, mc_amd->hdr.patch_id);
+ return -1;
+ }
+ pr_info("CPU%d: new patch_level=0x%08x\n", cpu,
+ mc_amd->hdr.patch_id);
+
+ uci->cpu_sig.rev = mc_amd->hdr.patch_id;
+ c->microcode = mc_amd->hdr.patch_id;
+
+ return 0;
+}
+
+static int install_equiv_cpu_table(const u8 *buf)
+{
+ unsigned int *ibuf = (unsigned int *)buf;
+ unsigned int type = ibuf[1];
+ unsigned int size = ibuf[2];
+
+ if (type != UCODE_EQUIV_CPU_TABLE_TYPE || !size) {
+ pr_err("empty section/"
+ "invalid type field in container file section header\n");
+ return -EINVAL;
+ }
+
+ equiv_cpu_table = vmalloc(size);
+ if (!equiv_cpu_table) {
+ pr_err("failed to allocate equivalent CPU table\n");
+ return -ENOMEM;
+ }
+
+ memcpy(equiv_cpu_table, buf + CONTAINER_HDR_SZ, size);
+
+ /* add header length */
+ return size + CONTAINER_HDR_SZ;
+}
+
+static void free_equiv_cpu_table(void)
+{
+ vfree(equiv_cpu_table);
+ equiv_cpu_table = NULL;
+}
+
+static void cleanup(void)
+{
+ free_equiv_cpu_table();
+ free_cache();
+}
+
+/*
+ * We return the current size even if some of the checks failed so that
+ * we can skip over the next patch. If we return a negative value, we
+ * signal a grave error like a memory allocation has failed and the
+ * driver cannot continue functioning normally. In such cases, we tear
+ * down everything we've used up so far and exit.
+ */
+static int verify_and_add_patch(u8 family, u8 *fw, unsigned int leftover)
+{
+ struct microcode_header_amd *mc_hdr;
+ struct ucode_patch *patch;
+ unsigned int patch_size, crnt_size, ret;
+ u32 proc_fam;
+ u16 proc_id;
+
+ patch_size = *(u32 *)(fw + 4);
+ crnt_size = patch_size + SECTION_HDR_SIZE;
+ mc_hdr = (struct microcode_header_amd *)(fw + SECTION_HDR_SIZE);
+ proc_id = mc_hdr->processor_rev_id;
+
+ proc_fam = find_cpu_family_by_equiv_cpu(proc_id);
+ if (!proc_fam) {
+ pr_err("No patch family for equiv ID: 0x%04x\n", proc_id);
+ return crnt_size;
+ }
+
+ /* check if patch is for the current family */
+ proc_fam = ((proc_fam >> 8) & 0xf) + ((proc_fam >> 20) & 0xff);
+ if (proc_fam != family)
+ return crnt_size;
+
+ if (mc_hdr->nb_dev_id || mc_hdr->sb_dev_id) {
+ pr_err("Patch-ID 0x%08x: chipset-specific code unsupported.\n",
+ mc_hdr->patch_id);
+ return crnt_size;
+ }
+
+ ret = verify_patch_size(family, patch_size, leftover);
+ if (!ret) {
+ pr_err("Patch-ID 0x%08x: size mismatch.\n", mc_hdr->patch_id);
+ return crnt_size;
+ }
+
+ patch = kzalloc(sizeof(*patch), GFP_KERNEL);
+ if (!patch) {
+ pr_err("Patch allocation failure.\n");
+ return -EINVAL;
+ }
+
+ patch->data = kzalloc(patch_size, GFP_KERNEL);
+ if (!patch->data) {
+ pr_err("Patch data allocation failure.\n");
+ kfree(patch);
+ return -EINVAL;
+ }
+
+ /* All looks ok, copy patch... */
+ memcpy(patch->data, fw + SECTION_HDR_SIZE, patch_size);
+ INIT_LIST_HEAD(&patch->plist);
+ patch->patch_id = mc_hdr->patch_id;
+ patch->equiv_cpu = proc_id;
+
+ pr_debug("%s: Added patch_id: 0x%08x, proc_id: 0x%04x\n",
+ __func__, patch->patch_id, proc_id);
+
+ /* ... and add to cache. */
+ update_cache(patch);
+
+ return crnt_size;
+}
+
+static enum ucode_state __load_microcode_amd(u8 family, const u8 *data,
+ size_t size)
+{
+ enum ucode_state ret = UCODE_ERROR;
+ unsigned int leftover;
+ u8 *fw = (u8 *)data;
+ int crnt_size = 0;
+ int offset;
+
+ offset = install_equiv_cpu_table(data);
+ if (offset < 0) {
+ pr_err("failed to create equivalent cpu table\n");
+ return ret;
+ }
+ fw += offset;
+ leftover = size - offset;
+
+ if (*(u32 *)fw != UCODE_UCODE_TYPE) {
+ pr_err("invalid type field in container file section header\n");
+ free_equiv_cpu_table();
+ return ret;
+ }
+
+ while (leftover) {
+ crnt_size = verify_and_add_patch(family, fw, leftover);
+ if (crnt_size < 0)
+ return ret;
+
+ fw += crnt_size;
+ leftover -= crnt_size;
+ }
+
+ return UCODE_OK;
+}
+
+enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size)
+{
+ enum ucode_state ret;
+
+ /* free old equiv table */
+ free_equiv_cpu_table();
+
+ ret = __load_microcode_amd(family, data, size);
+
+ if (ret != UCODE_OK)
+ cleanup();
+
+#if defined(CONFIG_MICROCODE_AMD_EARLY) && defined(CONFIG_X86_32)
+ /* save BSP's matching patch for early load */
+ if (cpu_data(smp_processor_id()).cpu_index == boot_cpu_data.cpu_index) {
+ struct ucode_patch *p = find_patch(smp_processor_id());
+ if (p) {
+ memset(amd_ucode_patch, 0, PATCH_MAX_SIZE);
+ memcpy(amd_ucode_patch, p->data, min_t(u32, ksize(p->data),
+ PATCH_MAX_SIZE));
+ }
+ }
+#endif
+ return ret;
+}
+
+/*
+ * AMD microcode firmware naming convention, up to family 15h they are in
+ * the legacy file:
+ *
+ * amd-ucode/microcode_amd.bin
+ *
+ * This legacy file is always smaller than 2K in size.
+ *
+ * Beginning with family 15h, they are in family-specific firmware files:
+ *
+ * amd-ucode/microcode_amd_fam15h.bin
+ * amd-ucode/microcode_amd_fam16h.bin
+ * ...
+ *
+ * These might be larger than 2K.
+ */
+static enum ucode_state request_microcode_amd(int cpu, struct device *device,
+ bool refresh_fw)
+{
+ char fw_name[36] = "amd-ucode/microcode_amd.bin";
+ struct cpuinfo_x86 *c = &cpu_data(cpu);
+ enum ucode_state ret = UCODE_NFOUND;
+ const struct firmware *fw;
+
+ /* reload ucode container only on the boot cpu */
+ if (!refresh_fw || c->cpu_index != boot_cpu_data.cpu_index)
+ return UCODE_OK;
+
+ if (c->x86 >= 0x15)
+ snprintf(fw_name, sizeof(fw_name), "amd-ucode/microcode_amd_fam%.2xh.bin", c->x86);
+
+ if (request_firmware(&fw, (const char *)fw_name, device)) {
+ pr_debug("failed to load file %s\n", fw_name);
+ goto out;
+ }
+
+ ret = UCODE_ERROR;
+ if (*(u32 *)fw->data != UCODE_MAGIC) {
+ pr_err("invalid magic value (0x%08x)\n", *(u32 *)fw->data);
+ goto fw_release;
+ }
+
+ ret = load_microcode_amd(c->x86, fw->data, fw->size);
+
+ fw_release:
+ release_firmware(fw);
+
+ out:
+ return ret;
+}
+
+static enum ucode_state
+request_microcode_user(int cpu, const void __user *buf, size_t size)
+{
+ return UCODE_ERROR;
+}
+
+static void microcode_fini_cpu_amd(int cpu)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+
+ uci->mc = NULL;
+}
+
+static struct microcode_ops microcode_amd_ops = {
+ .request_microcode_user = request_microcode_user,
+ .request_microcode_fw = request_microcode_amd,
+ .collect_cpu_info = collect_cpu_info_amd,
+ .apply_microcode = apply_microcode_amd,
+ .microcode_fini_cpu = microcode_fini_cpu_amd,
+};
+
+struct microcode_ops * __init init_amd_microcode(void)
+{
+ struct cpuinfo_x86 *c = &cpu_data(0);
+
+ if (c->x86_vendor != X86_VENDOR_AMD || c->x86 < 0x10) {
+ pr_warning("AMD CPU family 0x%x not supported\n", c->x86);
+ return NULL;
+ }
+
+ return µcode_amd_ops;
+}
+
+void __exit exit_amd_microcode(void)
+{
+ cleanup();
+}
--- /dev/null
+/*
+ * Copyright (C) 2013 Advanced Micro Devices, Inc.
+ *
+ * Author: Jacob Shin <jacob.shin@amd.com>
+ * Fixes: Borislav Petkov <bp@suse.de>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/earlycpio.h>
+#include <linux/initrd.h>
+
+#include <asm/cpu.h>
+#include <asm/setup.h>
+#include <asm/microcode_amd.h>
+
+/*
+ * This points to the current valid container of microcode patches which we will
+ * save from the initrd before jettisoning its contents.
+ */
+static u8 *container;
+static size_t container_size;
+
+static u32 ucode_new_rev;
+u8 amd_ucode_patch[PATCH_MAX_SIZE];
+static u16 this_equiv_id;
+
+struct cpio_data ucode_cpio;
+
+/*
+ * Microcode patch container file is prepended to the initrd in cpio format.
+ * See Documentation/x86/early-microcode.txt
+ */
+static __initdata char ucode_path[] = "kernel/x86/microcode/AuthenticAMD.bin";
+
+static struct cpio_data __init find_ucode_in_initrd(void)
+{
+ long offset = 0;
+ char *path;
+ void *start;
+ size_t size;
+
+#ifdef CONFIG_X86_32
+ struct boot_params *p;
+
+ /*
+ * On 32-bit, early load occurs before paging is turned on so we need
+ * to use physical addresses.
+ */
+ p = (struct boot_params *)__pa_nodebug(&boot_params);
+ path = (char *)__pa_nodebug(ucode_path);
+ start = (void *)p->hdr.ramdisk_image;
+ size = p->hdr.ramdisk_size;
+#else
+ path = ucode_path;
+ start = (void *)(boot_params.hdr.ramdisk_image + PAGE_OFFSET);
+ size = boot_params.hdr.ramdisk_size;
+#endif
+
+ return find_cpio_data(path, start, size, &offset);
+}
+
+static size_t compute_container_size(u8 *data, u32 total_size)
+{
+ size_t size = 0;
+ u32 *header = (u32 *)data;
+
+ if (header[0] != UCODE_MAGIC ||
+ header[1] != UCODE_EQUIV_CPU_TABLE_TYPE || /* type */
+ header[2] == 0) /* size */
+ return size;
+
+ size = header[2] + CONTAINER_HDR_SZ;
+ total_size -= size;
+ data += size;
+
+ while (total_size) {
+ u16 patch_size;
+
+ header = (u32 *)data;
+
+ if (header[0] != UCODE_UCODE_TYPE)
+ break;
+
+ /*
+ * Sanity-check patch size.
+ */
+ patch_size = header[1];
+ if (patch_size > PATCH_MAX_SIZE)
+ break;
+
+ size += patch_size + SECTION_HDR_SIZE;
+ data += patch_size + SECTION_HDR_SIZE;
+ total_size -= patch_size + SECTION_HDR_SIZE;
+ }
+
+ return size;
+}
+
+/*
+ * Early load occurs before we can vmalloc(). So we look for the microcode
+ * patch container file in initrd, traverse equivalent cpu table, look for a
+ * matching microcode patch, and update, all in initrd memory in place.
+ * When vmalloc() is available for use later -- on 64-bit during first AP load,
+ * and on 32-bit during save_microcode_in_initrd_amd() -- we can call
+ * load_microcode_amd() to save equivalent cpu table and microcode patches in
+ * kernel heap memory.
+ */
+static void apply_ucode_in_initrd(void *ucode, size_t size)
+{
+ struct equiv_cpu_entry *eq;
+ size_t *cont_sz;
+ u32 *header;
+ u8 *data, **cont;
+ u16 eq_id = 0;
+ int offset, left;
+ u32 rev, eax, ebx, ecx, edx;
+ u32 *new_rev;
+
+#ifdef CONFIG_X86_32
+ new_rev = (u32 *)__pa_nodebug(&ucode_new_rev);
+ cont_sz = (size_t *)__pa_nodebug(&container_size);
+ cont = (u8 **)__pa_nodebug(&container);
+#else
+ new_rev = &ucode_new_rev;
+ cont_sz = &container_size;
+ cont = &container;
+#endif
+
+ data = ucode;
+ left = size;
+ header = (u32 *)data;
+
+ /* find equiv cpu table */
+ if (header[0] != UCODE_MAGIC ||
+ header[1] != UCODE_EQUIV_CPU_TABLE_TYPE || /* type */
+ header[2] == 0) /* size */
+ return;
+
+ eax = 0x00000001;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+
+ while (left > 0) {
+ eq = (struct equiv_cpu_entry *)(data + CONTAINER_HDR_SZ);
+
+ *cont = data;
+
+ /* Advance past the container header */
+ offset = header[2] + CONTAINER_HDR_SZ;
+ data += offset;
+ left -= offset;
+
+ eq_id = find_equiv_id(eq, eax);
+ if (eq_id) {
+ this_equiv_id = eq_id;
+ *cont_sz = compute_container_size(*cont, left + offset);
+
+ /*
+ * truncate how much we need to iterate over in the
+ * ucode update loop below
+ */
+ left = *cont_sz - offset;
+ break;
+ }
+
+ /*
+ * support multiple container files appended together. if this
+ * one does not have a matching equivalent cpu entry, we fast
+ * forward to the next container file.
+ */
+ while (left > 0) {
+ header = (u32 *)data;
+ if (header[0] == UCODE_MAGIC &&
+ header[1] == UCODE_EQUIV_CPU_TABLE_TYPE)
+ break;
+
+ offset = header[1] + SECTION_HDR_SIZE;
+ data += offset;
+ left -= offset;
+ }
+
+ /* mark where the next microcode container file starts */
+ offset = data - (u8 *)ucode;
+ ucode = data;
+ }
+
+ if (!eq_id) {
+ *cont = NULL;
+ *cont_sz = 0;
+ return;
+ }
+
+ /* find ucode and update if needed */
+
+ native_rdmsr(MSR_AMD64_PATCH_LEVEL, rev, eax);
+
+ while (left > 0) {
+ struct microcode_amd *mc;
+
+ header = (u32 *)data;
+ if (header[0] != UCODE_UCODE_TYPE || /* type */
+ header[1] == 0) /* size */
+ break;
+
+ mc = (struct microcode_amd *)(data + SECTION_HDR_SIZE);
+
+ if (eq_id == mc->hdr.processor_rev_id && rev < mc->hdr.patch_id) {
+
+ if (!__apply_microcode_amd(mc)) {
+ rev = mc->hdr.patch_id;
+ *new_rev = rev;
+
+ /* save ucode patch */
+ memcpy(amd_ucode_patch, mc,
+ min_t(u32, header[1], PATCH_MAX_SIZE));
+ }
+ }
+
+ offset = header[1] + SECTION_HDR_SIZE;
+ data += offset;
+ left -= offset;
+ }
+}
+
+void __init load_ucode_amd_bsp(void)
+{
+ struct cpio_data cp;
+ void **data;
+ size_t *size;
+
+#ifdef CONFIG_X86_32
+ data = (void **)__pa_nodebug(&ucode_cpio.data);
+ size = (size_t *)__pa_nodebug(&ucode_cpio.size);
+#else
+ data = &ucode_cpio.data;
+ size = &ucode_cpio.size;
+#endif
+
+ cp = find_ucode_in_initrd();
+ if (!cp.data)
+ return;
+
+ *data = cp.data;
+ *size = cp.size;
+
+ apply_ucode_in_initrd(cp.data, cp.size);
+}
+
+#ifdef CONFIG_X86_32
+/*
+ * On 32-bit, since AP's early load occurs before paging is turned on, we
+ * cannot traverse cpu_equiv_table and pcache in kernel heap memory. So during
+ * cold boot, AP will apply_ucode_in_initrd() just like the BSP. During
+ * save_microcode_in_initrd_amd() BSP's patch is copied to amd_ucode_patch,
+ * which is used upon resume from suspend.
+ */
+void load_ucode_amd_ap(void)
+{
+ struct microcode_amd *mc;
+ size_t *usize;
+ void **ucode;
+
+ mc = (struct microcode_amd *)__pa(amd_ucode_patch);
+ if (mc->hdr.patch_id && mc->hdr.processor_rev_id) {
+ __apply_microcode_amd(mc);
+ return;
+ }
+
+ ucode = (void *)__pa_nodebug(&container);
+ usize = (size_t *)__pa_nodebug(&container_size);
+
+ if (!*ucode || !*usize)
+ return;
+
+ apply_ucode_in_initrd(*ucode, *usize);
+}
+
+static void __init collect_cpu_sig_on_bsp(void *arg)
+{
+ unsigned int cpu = smp_processor_id();
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+
+ uci->cpu_sig.sig = cpuid_eax(0x00000001);
+}
+#else
+void load_ucode_amd_ap(void)
+{
+ unsigned int cpu = smp_processor_id();
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ struct equiv_cpu_entry *eq;
+ struct microcode_amd *mc;
+ u32 rev, eax;
+ u16 eq_id;
+
+ /* Exit if called on the BSP. */
+ if (!cpu)
+ return;
+
+ if (!container)
+ return;
+
+ rdmsr(MSR_AMD64_PATCH_LEVEL, rev, eax);
+
+ uci->cpu_sig.rev = rev;
+ uci->cpu_sig.sig = eax;
+
+ eax = cpuid_eax(0x00000001);
+ eq = (struct equiv_cpu_entry *)(container + CONTAINER_HDR_SZ);
+
+ eq_id = find_equiv_id(eq, eax);
+ if (!eq_id)
+ return;
+
+ if (eq_id == this_equiv_id) {
+ mc = (struct microcode_amd *)amd_ucode_patch;
+
+ if (mc && rev < mc->hdr.patch_id) {
+ if (!__apply_microcode_amd(mc))
+ ucode_new_rev = mc->hdr.patch_id;
+ }
+
+ } else {
+ if (!ucode_cpio.data)
+ return;
+
+ /*
+ * AP has a different equivalence ID than BSP, looks like
+ * mixed-steppings silicon so go through the ucode blob anew.
+ */
+ apply_ucode_in_initrd(ucode_cpio.data, ucode_cpio.size);
+ }
+}
+#endif
+
+int __init save_microcode_in_initrd_amd(void)
+{
+ enum ucode_state ret;
+ u32 eax;
+
+#ifdef CONFIG_X86_32
+ unsigned int bsp = boot_cpu_data.cpu_index;
+ struct ucode_cpu_info *uci = ucode_cpu_info + bsp;
+
+ if (!uci->cpu_sig.sig)
+ smp_call_function_single(bsp, collect_cpu_sig_on_bsp, NULL, 1);
+
+ /*
+ * Take into account the fact that the ramdisk might get relocated
+ * and therefore we need to recompute the container's position in
+ * virtual memory space.
+ */
+ container = (u8 *)(__va((u32)relocated_ramdisk) +
+ ((u32)container - boot_params.hdr.ramdisk_image));
+#endif
+ if (ucode_new_rev)
+ pr_info("microcode: updated early to new patch_level=0x%08x\n",
+ ucode_new_rev);
+
+ if (!container)
+ return -EINVAL;
+
+ eax = cpuid_eax(0x00000001);
+ eax = ((eax >> 8) & 0xf) + ((eax >> 20) & 0xff);
+
+ ret = load_microcode_amd(eax, container, container_size);
+ if (ret != UCODE_OK)
+ return -EINVAL;
+
+ /*
+ * This will be freed any msec now, stash patches for the current
+ * family and switch to patch cache for cpu hotplug, etc later.
+ */
+ container = NULL;
+ container_size = 0;
+
+ return 0;
+}
--- /dev/null
+/*
+ * Intel CPU Microcode Update Driver for Linux
+ *
+ * Copyright (C) 2000-2006 Tigran Aivazian <tigran@aivazian.fsnet.co.uk>
+ * 2006 Shaohua Li <shaohua.li@intel.com>
+ *
+ * This driver allows to upgrade microcode on Intel processors
+ * belonging to IA-32 family - PentiumPro, Pentium II,
+ * Pentium III, Xeon, Pentium 4, etc.
+ *
+ * Reference: Section 8.11 of Volume 3a, IA-32 Intel? Architecture
+ * Software Developer's Manual
+ * Order Number 253668 or free download from:
+ *
+ * http://developer.intel.com/Assets/PDF/manual/253668.pdf
+ *
+ * For more information, go to http://www.urbanmyth.org/microcode
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ * 1.0 16 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Initial release.
+ * 1.01 18 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Added read() support + cleanups.
+ * 1.02 21 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Added 'device trimming' support. open(O_WRONLY) zeroes
+ * and frees the saved copy of applied microcode.
+ * 1.03 29 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Made to use devfs (/dev/cpu/microcode) + cleanups.
+ * 1.04 06 Jun 2000, Simon Trimmer <simon@veritas.com>
+ * Added misc device support (now uses both devfs and misc).
+ * Added MICROCODE_IOCFREE ioctl to clear memory.
+ * 1.05 09 Jun 2000, Simon Trimmer <simon@veritas.com>
+ * Messages for error cases (non Intel & no suitable microcode).
+ * 1.06 03 Aug 2000, Tigran Aivazian <tigran@veritas.com>
+ * Removed ->release(). Removed exclusive open and status bitmap.
+ * Added microcode_rwsem to serialize read()/write()/ioctl().
+ * Removed global kernel lock usage.
+ * 1.07 07 Sep 2000, Tigran Aivazian <tigran@veritas.com>
+ * Write 0 to 0x8B msr and then cpuid before reading revision,
+ * so that it works even if there were no update done by the
+ * BIOS. Otherwise, reading from 0x8B gives junk (which happened
+ * to be 0 on my machine which is why it worked even when I
+ * disabled update by the BIOS)
+ * Thanks to Eric W. Biederman <ebiederman@lnxi.com> for the fix.
+ * 1.08 11 Dec 2000, Richard Schaal <richard.schaal@intel.com> and
+ * Tigran Aivazian <tigran@veritas.com>
+ * Intel Pentium 4 processor support and bugfixes.
+ * 1.09 30 Oct 2001, Tigran Aivazian <tigran@veritas.com>
+ * Bugfix for HT (Hyper-Threading) enabled processors
+ * whereby processor resources are shared by all logical processors
+ * in a single CPU package.
+ * 1.10 28 Feb 2002 Asit K Mallick <asit.k.mallick@intel.com> and
+ * Tigran Aivazian <tigran@veritas.com>,
+ * Serialize updates as required on HT processors due to
+ * speculative nature of implementation.
+ * 1.11 22 Mar 2002 Tigran Aivazian <tigran@veritas.com>
+ * Fix the panic when writing zero-length microcode chunk.
+ * 1.12 29 Sep 2003 Nitin Kamble <nitin.a.kamble@intel.com>,
+ * Jun Nakajima <jun.nakajima@intel.com>
+ * Support for the microcode updates in the new format.
+ * 1.13 10 Oct 2003 Tigran Aivazian <tigran@veritas.com>
+ * Removed ->read() method and obsoleted MICROCODE_IOCFREE ioctl
+ * because we no longer hold a copy of applied microcode
+ * in kernel memory.
+ * 1.14 25 Jun 2004 Tigran Aivazian <tigran@veritas.com>
+ * Fix sigmatch() macro to handle old CPUs with pf == 0.
+ * Thanks to Stuart Swales for pointing out this bug.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/platform_device.h>
+#include <linux/miscdevice.h>
+#include <linux/capability.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/cpu.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/syscore_ops.h>
+
+#include <asm/microcode.h>
+#include <asm/processor.h>
+#include <asm/cpu_device_id.h>
+#include <asm/perf_event.h>
+
+MODULE_DESCRIPTION("Microcode Update Driver");
+MODULE_AUTHOR("Tigran Aivazian <tigran@aivazian.fsnet.co.uk>");
+MODULE_LICENSE("GPL");
+
+#define MICROCODE_VERSION "2.00"
+
+static struct microcode_ops *microcode_ops;
+
+/*
+ * Synchronization.
+ *
+ * All non cpu-hotplug-callback call sites use:
+ *
+ * - microcode_mutex to synchronize with each other;
+ * - get/put_online_cpus() to synchronize with
+ * the cpu-hotplug-callback call sites.
+ *
+ * We guarantee that only a single cpu is being
+ * updated at any particular moment of time.
+ */
+static DEFINE_MUTEX(microcode_mutex);
+
+struct ucode_cpu_info ucode_cpu_info[NR_CPUS];
+EXPORT_SYMBOL_GPL(ucode_cpu_info);
+
+/*
+ * Operations that are run on a target cpu:
+ */
+
+struct cpu_info_ctx {
+ struct cpu_signature *cpu_sig;
+ int err;
+};
+
+static void collect_cpu_info_local(void *arg)
+{
+ struct cpu_info_ctx *ctx = arg;
+
+ ctx->err = microcode_ops->collect_cpu_info(smp_processor_id(),
+ ctx->cpu_sig);
+}
+
+static int collect_cpu_info_on_target(int cpu, struct cpu_signature *cpu_sig)
+{
+ struct cpu_info_ctx ctx = { .cpu_sig = cpu_sig, .err = 0 };
+ int ret;
+
+ ret = smp_call_function_single(cpu, collect_cpu_info_local, &ctx, 1);
+ if (!ret)
+ ret = ctx.err;
+
+ return ret;
+}
+
+static int collect_cpu_info(int cpu)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ int ret;
+
+ memset(uci, 0, sizeof(*uci));
+
+ ret = collect_cpu_info_on_target(cpu, &uci->cpu_sig);
+ if (!ret)
+ uci->valid = 1;
+
+ return ret;
+}
+
+struct apply_microcode_ctx {
+ int err;
+};
+
+static void apply_microcode_local(void *arg)
+{
+ struct apply_microcode_ctx *ctx = arg;
+
+ ctx->err = microcode_ops->apply_microcode(smp_processor_id());
+}
+
+static int apply_microcode_on_target(int cpu)
+{
+ struct apply_microcode_ctx ctx = { .err = 0 };
+ int ret;
+
+ ret = smp_call_function_single(cpu, apply_microcode_local, &ctx, 1);
+ if (!ret)
+ ret = ctx.err;
+
+ return ret;
+}
+
+#ifdef CONFIG_MICROCODE_OLD_INTERFACE
+static int do_microcode_update(const void __user *buf, size_t size)
+{
+ int error = 0;
+ int cpu;
+
+ for_each_online_cpu(cpu) {
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ enum ucode_state ustate;
+
+ if (!uci->valid)
+ continue;
+
+ ustate = microcode_ops->request_microcode_user(cpu, buf, size);
+ if (ustate == UCODE_ERROR) {
+ error = -1;
+ break;
+ } else if (ustate == UCODE_OK)
+ apply_microcode_on_target(cpu);
+ }
+
+ return error;
+}
+
+static int microcode_open(struct inode *inode, struct file *file)
+{
+ return capable(CAP_SYS_RAWIO) ? nonseekable_open(inode, file) : -EPERM;
+}
+
+static ssize_t microcode_write(struct file *file, const char __user *buf,
+ size_t len, loff_t *ppos)
+{
+ ssize_t ret = -EINVAL;
+
+ if ((len >> PAGE_SHIFT) > totalram_pages) {
+ pr_err("too much data (max %ld pages)\n", totalram_pages);
+ return ret;
+ }
+
+ get_online_cpus();
+ mutex_lock(µcode_mutex);
+
+ if (do_microcode_update(buf, len) == 0)
+ ret = (ssize_t)len;
+
+ if (ret > 0)
+ perf_check_microcode();
+
+ mutex_unlock(µcode_mutex);
+ put_online_cpus();
+
+ return ret;
+}
+
+static const struct file_operations microcode_fops = {
+ .owner = THIS_MODULE,
+ .write = microcode_write,
+ .open = microcode_open,
+ .llseek = no_llseek,
+};
+
+static struct miscdevice microcode_dev = {
+ .minor = MICROCODE_MINOR,
+ .name = "microcode",
+ .nodename = "cpu/microcode",
+ .fops = µcode_fops,
+};
+
+static int __init microcode_dev_init(void)
+{
+ int error;
+
+ error = misc_register(µcode_dev);
+ if (error) {
+ pr_err("can't misc_register on minor=%d\n", MICROCODE_MINOR);
+ return error;
+ }
+
+ return 0;
+}
+
+static void __exit microcode_dev_exit(void)
+{
+ misc_deregister(µcode_dev);
+}
+
+MODULE_ALIAS_MISCDEV(MICROCODE_MINOR);
+MODULE_ALIAS("devname:cpu/microcode");
+#else
+#define microcode_dev_init() 0
+#define microcode_dev_exit() do { } while (0)
+#endif
+
+/* fake device for request_firmware */
+static struct platform_device *microcode_pdev;
+
+static int reload_for_cpu(int cpu)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ enum ucode_state ustate;
+ int err = 0;
+
+ if (!uci->valid)
+ return err;
+
+ ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev, true);
+ if (ustate == UCODE_OK)
+ apply_microcode_on_target(cpu);
+ else
+ if (ustate == UCODE_ERROR)
+ err = -EINVAL;
+ return err;
+}
+
+static ssize_t reload_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ int cpu;
+ ssize_t ret = 0, tmp_ret;
+
+ ret = kstrtoul(buf, 0, &val);
+ if (ret)
+ return ret;
+
+ if (val != 1)
+ return size;
+
+ get_online_cpus();
+ mutex_lock(µcode_mutex);
+ for_each_online_cpu(cpu) {
+ tmp_ret = reload_for_cpu(cpu);
+ if (tmp_ret != 0)
+ pr_warn("Error reloading microcode on CPU %d\n", cpu);
+
+ /* save retval of the first encountered reload error */
+ if (!ret)
+ ret = tmp_ret;
+ }
+ if (!ret)
+ perf_check_microcode();
+ mutex_unlock(µcode_mutex);
+ put_online_cpus();
+
+ if (!ret)
+ ret = size;
+
+ return ret;
+}
+
+static ssize_t version_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + dev->id;
+
+ return sprintf(buf, "0x%x\n", uci->cpu_sig.rev);
+}
+
+static ssize_t pf_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + dev->id;
+
+ return sprintf(buf, "0x%x\n", uci->cpu_sig.pf);
+}
+
+static DEVICE_ATTR(reload, 0200, NULL, reload_store);
+static DEVICE_ATTR(version, 0400, version_show, NULL);
+static DEVICE_ATTR(processor_flags, 0400, pf_show, NULL);
+
+static struct attribute *mc_default_attrs[] = {
+ &dev_attr_version.attr,
+ &dev_attr_processor_flags.attr,
+ NULL
+};
+
+static struct attribute_group mc_attr_group = {
+ .attrs = mc_default_attrs,
+ .name = "microcode",
+};
+
+static void microcode_fini_cpu(int cpu)
+{
+ microcode_ops->microcode_fini_cpu(cpu);
+}
+
+static enum ucode_state microcode_resume_cpu(int cpu)
+{
+ pr_debug("CPU%d updated upon resume\n", cpu);
+
+ if (apply_microcode_on_target(cpu))
+ return UCODE_ERROR;
+
+ return UCODE_OK;
+}
+
+static enum ucode_state microcode_init_cpu(int cpu, bool refresh_fw)
+{
+ enum ucode_state ustate;
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+
+ if (uci && uci->valid)
+ return UCODE_OK;
+
+ if (collect_cpu_info(cpu))
+ return UCODE_ERROR;
+
+ /* --dimm. Trigger a delayed update? */
+ if (system_state != SYSTEM_RUNNING)
+ return UCODE_NFOUND;
+
+ ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev,
+ refresh_fw);
+
+ if (ustate == UCODE_OK) {
+ pr_debug("CPU%d updated upon init\n", cpu);
+ apply_microcode_on_target(cpu);
+ }
+
+ return ustate;
+}
+
+static enum ucode_state microcode_update_cpu(int cpu)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+
+ if (uci->valid)
+ return microcode_resume_cpu(cpu);
+
+ return microcode_init_cpu(cpu, false);
+}
+
+static int mc_device_add(struct device *dev, struct subsys_interface *sif)
+{
+ int err, cpu = dev->id;
+
+ if (!cpu_online(cpu))
+ return 0;
+
+ pr_debug("CPU%d added\n", cpu);
+
+ err = sysfs_create_group(&dev->kobj, &mc_attr_group);
+ if (err)
+ return err;
+
+ if (microcode_init_cpu(cpu, true) == UCODE_ERROR)
+ return -EINVAL;
+
+ return err;
+}
+
+static int mc_device_remove(struct device *dev, struct subsys_interface *sif)
+{
+ int cpu = dev->id;
+
+ if (!cpu_online(cpu))
+ return 0;
+
+ pr_debug("CPU%d removed\n", cpu);
+ microcode_fini_cpu(cpu);
+ sysfs_remove_group(&dev->kobj, &mc_attr_group);
+ return 0;
+}
+
+static struct subsys_interface mc_cpu_interface = {
+ .name = "microcode",
+ .subsys = &cpu_subsys,
+ .add_dev = mc_device_add,
+ .remove_dev = mc_device_remove,
+};
+
+/**
+ * mc_bp_resume - Update boot CPU microcode during resume.
+ */
+static void mc_bp_resume(void)
+{
+ int cpu = smp_processor_id();
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+
+ if (uci->valid && uci->mc)
+ microcode_ops->apply_microcode(cpu);
+}
+
+static struct syscore_ops mc_syscore_ops = {
+ .resume = mc_bp_resume,
+};
+
+static int
+mc_cpu_callback(struct notifier_block *nb, unsigned long action, void *hcpu)
+{
+ unsigned int cpu = (unsigned long)hcpu;
+ struct device *dev;
+
+ dev = get_cpu_device(cpu);
+
+ switch (action & ~CPU_TASKS_FROZEN) {
+ case CPU_ONLINE:
+ microcode_update_cpu(cpu);
+ pr_debug("CPU%d added\n", cpu);
+ /*
+ * "break" is missing on purpose here because we want to fall
+ * through in order to create the sysfs group.
+ */
+
+ case CPU_DOWN_FAILED:
+ if (sysfs_create_group(&dev->kobj, &mc_attr_group))
+ pr_err("Failed to create group for CPU%d\n", cpu);
+ break;
+
+ case CPU_DOWN_PREPARE:
+ /* Suspend is in progress, only remove the interface */
+ sysfs_remove_group(&dev->kobj, &mc_attr_group);
+ pr_debug("CPU%d removed\n", cpu);
+ break;
+
+ /*
+ * case CPU_DEAD:
+ *
+ * When a CPU goes offline, don't free up or invalidate the copy of
+ * the microcode in kernel memory, so that we can reuse it when the
+ * CPU comes back online without unnecessarily requesting the userspace
+ * for it again.
+ */
+ }
+
+ /* The CPU refused to come up during a system resume */
+ if (action == CPU_UP_CANCELED_FROZEN)
+ microcode_fini_cpu(cpu);
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block __refdata mc_cpu_notifier = {
+ .notifier_call = mc_cpu_callback,
+};
+
+#ifdef MODULE
+/* Autoload on Intel and AMD systems */
+static const struct x86_cpu_id __initconst microcode_id[] = {
+#ifdef CONFIG_MICROCODE_INTEL
+ { X86_VENDOR_INTEL, X86_FAMILY_ANY, X86_MODEL_ANY, },
+#endif
+#ifdef CONFIG_MICROCODE_AMD
+ { X86_VENDOR_AMD, X86_FAMILY_ANY, X86_MODEL_ANY, },
+#endif
+ {}
+};
+MODULE_DEVICE_TABLE(x86cpu, microcode_id);
+#endif
+
+static struct attribute *cpu_root_microcode_attrs[] = {
+ &dev_attr_reload.attr,
+ NULL
+};
+
+static struct attribute_group cpu_root_microcode_group = {
+ .name = "microcode",
+ .attrs = cpu_root_microcode_attrs,
+};
+
+static int __init microcode_init(void)
+{
+ struct cpuinfo_x86 *c = &cpu_data(0);
+ int error;
+
+ if (c->x86_vendor == X86_VENDOR_INTEL)
+ microcode_ops = init_intel_microcode();
+ else if (c->x86_vendor == X86_VENDOR_AMD)
+ microcode_ops = init_amd_microcode();
+ else
+ pr_err("no support for this CPU vendor\n");
+
+ if (!microcode_ops)
+ return -ENODEV;
+
+ microcode_pdev = platform_device_register_simple("microcode", -1,
+ NULL, 0);
+ if (IS_ERR(microcode_pdev))
+ return PTR_ERR(microcode_pdev);
+
+ get_online_cpus();
+ mutex_lock(µcode_mutex);
+
+ error = subsys_interface_register(&mc_cpu_interface);
+ if (!error)
+ perf_check_microcode();
+ mutex_unlock(µcode_mutex);
+ put_online_cpus();
+
+ if (error)
+ goto out_pdev;
+
+ error = sysfs_create_group(&cpu_subsys.dev_root->kobj,
+ &cpu_root_microcode_group);
+
+ if (error) {
+ pr_err("Error creating microcode group!\n");
+ goto out_driver;
+ }
+
+ error = microcode_dev_init();
+ if (error)
+ goto out_ucode_group;
+
+ register_syscore_ops(&mc_syscore_ops);
+ register_hotcpu_notifier(&mc_cpu_notifier);
+
+ pr_info("Microcode Update Driver: v" MICROCODE_VERSION
+ " <tigran@aivazian.fsnet.co.uk>, Peter Oruba\n");
+
+ return 0;
+
+ out_ucode_group:
+ sysfs_remove_group(&cpu_subsys.dev_root->kobj,
+ &cpu_root_microcode_group);
+
+ out_driver:
+ get_online_cpus();
+ mutex_lock(µcode_mutex);
+
+ subsys_interface_unregister(&mc_cpu_interface);
+
+ mutex_unlock(µcode_mutex);
+ put_online_cpus();
+
+ out_pdev:
+ platform_device_unregister(microcode_pdev);
+ return error;
+
+}
+module_init(microcode_init);
+
+static void __exit microcode_exit(void)
+{
+ struct cpuinfo_x86 *c = &cpu_data(0);
+
+ microcode_dev_exit();
+
+ unregister_hotcpu_notifier(&mc_cpu_notifier);
+ unregister_syscore_ops(&mc_syscore_ops);
+
+ sysfs_remove_group(&cpu_subsys.dev_root->kobj,
+ &cpu_root_microcode_group);
+
+ get_online_cpus();
+ mutex_lock(µcode_mutex);
+
+ subsys_interface_unregister(&mc_cpu_interface);
+
+ mutex_unlock(µcode_mutex);
+ put_online_cpus();
+
+ platform_device_unregister(microcode_pdev);
+
+ microcode_ops = NULL;
+
+ if (c->x86_vendor == X86_VENDOR_AMD)
+ exit_amd_microcode();
+
+ pr_info("Microcode Update Driver: v" MICROCODE_VERSION " removed.\n");
+}
+module_exit(microcode_exit);
--- /dev/null
+/*
+ * X86 CPU microcode early update for Linux
+ *
+ * Copyright (C) 2012 Fenghua Yu <fenghua.yu@intel.com>
+ * H Peter Anvin" <hpa@zytor.com>
+ *
+ * This driver allows to early upgrade microcode on Intel processors
+ * belonging to IA-32 family - PentiumPro, Pentium II,
+ * Pentium III, Xeon, Pentium 4, etc.
+ *
+ * Reference: Section 9.11 of Volume 3, IA-32 Intel Architecture
+ * Software Developer's Manual.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#include <linux/module.h>
+#include <asm/microcode_intel.h>
+#include <asm/microcode_amd.h>
+#include <asm/processor.h>
+
+#define QCHAR(a, b, c, d) ((a) + ((b) << 8) + ((c) << 16) + ((d) << 24))
+#define CPUID_INTEL1 QCHAR('G', 'e', 'n', 'u')
+#define CPUID_INTEL2 QCHAR('i', 'n', 'e', 'I')
+#define CPUID_INTEL3 QCHAR('n', 't', 'e', 'l')
+#define CPUID_AMD1 QCHAR('A', 'u', 't', 'h')
+#define CPUID_AMD2 QCHAR('e', 'n', 't', 'i')
+#define CPUID_AMD3 QCHAR('c', 'A', 'M', 'D')
+
+#define CPUID_IS(a, b, c, ebx, ecx, edx) \
+ (!((ebx ^ (a))|(edx ^ (b))|(ecx ^ (c))))
+
+/*
+ * In early loading microcode phase on BSP, boot_cpu_data is not set up yet.
+ * x86_vendor() gets vendor id for BSP.
+ *
+ * In 32 bit AP case, accessing boot_cpu_data needs linear address. To simplify
+ * coding, we still use x86_vendor() to get vendor id for AP.
+ *
+ * x86_vendor() gets vendor information directly through cpuid.
+ */
+static int x86_vendor(void)
+{
+ u32 eax = 0x00000000;
+ u32 ebx, ecx = 0, edx;
+
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+
+ if (CPUID_IS(CPUID_INTEL1, CPUID_INTEL2, CPUID_INTEL3, ebx, ecx, edx))
+ return X86_VENDOR_INTEL;
+
+ if (CPUID_IS(CPUID_AMD1, CPUID_AMD2, CPUID_AMD3, ebx, ecx, edx))
+ return X86_VENDOR_AMD;
+
+ return X86_VENDOR_UNKNOWN;
+}
+
+static int x86_family(void)
+{
+ u32 eax = 0x00000001;
+ u32 ebx, ecx = 0, edx;
+ int x86;
+
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+
+ x86 = (eax >> 8) & 0xf;
+ if (x86 == 15)
+ x86 += (eax >> 20) & 0xff;
+
+ return x86;
+}
+
+void __init load_ucode_bsp(void)
+{
+ int vendor, x86;
+
+ if (!have_cpuid_p())
+ return;
+
+ vendor = x86_vendor();
+ x86 = x86_family();
+
+ switch (vendor) {
+ case X86_VENDOR_INTEL:
+ if (x86 >= 6)
+ load_ucode_intel_bsp();
+ break;
+ case X86_VENDOR_AMD:
+ if (x86 >= 0x10)
+ load_ucode_amd_bsp();
+ break;
+ default:
+ break;
+ }
+}
+
+void load_ucode_ap(void)
+{
+ int vendor, x86;
+
+ if (!have_cpuid_p())
+ return;
+
+ vendor = x86_vendor();
+ x86 = x86_family();
+
+ switch (vendor) {
+ case X86_VENDOR_INTEL:
+ if (x86 >= 6)
+ load_ucode_intel_ap();
+ break;
+ case X86_VENDOR_AMD:
+ if (x86 >= 0x10)
+ load_ucode_amd_ap();
+ break;
+ default:
+ break;
+ }
+}
+
+int __init save_microcode_in_initrd(void)
+{
+ struct cpuinfo_x86 *c = &boot_cpu_data;
+
+ switch (c->x86_vendor) {
+ case X86_VENDOR_INTEL:
+ if (c->x86 >= 6)
+ save_microcode_in_initrd_intel();
+ break;
+ case X86_VENDOR_AMD:
+ if (c->x86 >= 0x10)
+ save_microcode_in_initrd_amd();
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
--- /dev/null
+/*
+ * Intel CPU Microcode Update Driver for Linux
+ *
+ * Copyright (C) 2000-2006 Tigran Aivazian <tigran@aivazian.fsnet.co.uk>
+ * 2006 Shaohua Li <shaohua.li@intel.com>
+ *
+ * This driver allows to upgrade microcode on Intel processors
+ * belonging to IA-32 family - PentiumPro, Pentium II,
+ * Pentium III, Xeon, Pentium 4, etc.
+ *
+ * Reference: Section 8.11 of Volume 3a, IA-32 Intel? Architecture
+ * Software Developer's Manual
+ * Order Number 253668 or free download from:
+ *
+ * http://developer.intel.com/Assets/PDF/manual/253668.pdf
+ *
+ * For more information, go to http://www.urbanmyth.org/microcode
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ * 1.0 16 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Initial release.
+ * 1.01 18 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Added read() support + cleanups.
+ * 1.02 21 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Added 'device trimming' support. open(O_WRONLY) zeroes
+ * and frees the saved copy of applied microcode.
+ * 1.03 29 Feb 2000, Tigran Aivazian <tigran@sco.com>
+ * Made to use devfs (/dev/cpu/microcode) + cleanups.
+ * 1.04 06 Jun 2000, Simon Trimmer <simon@veritas.com>
+ * Added misc device support (now uses both devfs and misc).
+ * Added MICROCODE_IOCFREE ioctl to clear memory.
+ * 1.05 09 Jun 2000, Simon Trimmer <simon@veritas.com>
+ * Messages for error cases (non Intel & no suitable microcode).
+ * 1.06 03 Aug 2000, Tigran Aivazian <tigran@veritas.com>
+ * Removed ->release(). Removed exclusive open and status bitmap.
+ * Added microcode_rwsem to serialize read()/write()/ioctl().
+ * Removed global kernel lock usage.
+ * 1.07 07 Sep 2000, Tigran Aivazian <tigran@veritas.com>
+ * Write 0 to 0x8B msr and then cpuid before reading revision,
+ * so that it works even if there were no update done by the
+ * BIOS. Otherwise, reading from 0x8B gives junk (which happened
+ * to be 0 on my machine which is why it worked even when I
+ * disabled update by the BIOS)
+ * Thanks to Eric W. Biederman <ebiederman@lnxi.com> for the fix.
+ * 1.08 11 Dec 2000, Richard Schaal <richard.schaal@intel.com> and
+ * Tigran Aivazian <tigran@veritas.com>
+ * Intel Pentium 4 processor support and bugfixes.
+ * 1.09 30 Oct 2001, Tigran Aivazian <tigran@veritas.com>
+ * Bugfix for HT (Hyper-Threading) enabled processors
+ * whereby processor resources are shared by all logical processors
+ * in a single CPU package.
+ * 1.10 28 Feb 2002 Asit K Mallick <asit.k.mallick@intel.com> and
+ * Tigran Aivazian <tigran@veritas.com>,
+ * Serialize updates as required on HT processors due to
+ * speculative nature of implementation.
+ * 1.11 22 Mar 2002 Tigran Aivazian <tigran@veritas.com>
+ * Fix the panic when writing zero-length microcode chunk.
+ * 1.12 29 Sep 2003 Nitin Kamble <nitin.a.kamble@intel.com>,
+ * Jun Nakajima <jun.nakajima@intel.com>
+ * Support for the microcode updates in the new format.
+ * 1.13 10 Oct 2003 Tigran Aivazian <tigran@veritas.com>
+ * Removed ->read() method and obsoleted MICROCODE_IOCFREE ioctl
+ * because we no longer hold a copy of applied microcode
+ * in kernel memory.
+ * 1.14 25 Jun 2004 Tigran Aivazian <tigran@veritas.com>
+ * Fix sigmatch() macro to handle old CPUs with pf == 0.
+ * Thanks to Stuart Swales for pointing out this bug.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/firmware.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/vmalloc.h>
+
+#include <asm/microcode_intel.h>
+#include <asm/processor.h>
+#include <asm/msr.h>
+
+MODULE_DESCRIPTION("Microcode Update Driver");
+MODULE_AUTHOR("Tigran Aivazian <tigran@aivazian.fsnet.co.uk>");
+MODULE_LICENSE("GPL");
+
+static int collect_cpu_info(int cpu_num, struct cpu_signature *csig)
+{
+ struct cpuinfo_x86 *c = &cpu_data(cpu_num);
+ unsigned int val[2];
+
+ memset(csig, 0, sizeof(*csig));
+
+ csig->sig = cpuid_eax(0x00000001);
+
+ if ((c->x86_model >= 5) || (c->x86 > 6)) {
+ /* get processor flags from MSR 0x17 */
+ rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
+ csig->pf = 1 << ((val[1] >> 18) & 7);
+ }
+
+ csig->rev = c->microcode;
+ pr_info("CPU%d sig=0x%x, pf=0x%x, revision=0x%x\n",
+ cpu_num, csig->sig, csig->pf, csig->rev);
+
+ return 0;
+}
+
+/*
+ * return 0 - no update found
+ * return 1 - found update
+ */
+static int get_matching_mc(struct microcode_intel *mc_intel, int cpu)
+{
+ struct cpu_signature cpu_sig;
+ unsigned int csig, cpf, crev;
+
+ collect_cpu_info(cpu, &cpu_sig);
+
+ csig = cpu_sig.sig;
+ cpf = cpu_sig.pf;
+ crev = cpu_sig.rev;
+
+ return get_matching_microcode(csig, cpf, mc_intel, crev);
+}
+
+int apply_microcode(int cpu)
+{
+ struct microcode_intel *mc_intel;
+ struct ucode_cpu_info *uci;
+ unsigned int val[2];
+ int cpu_num = raw_smp_processor_id();
+ struct cpuinfo_x86 *c = &cpu_data(cpu_num);
+
+ uci = ucode_cpu_info + cpu;
+ mc_intel = uci->mc;
+
+ /* We should bind the task to the CPU */
+ BUG_ON(cpu_num != cpu);
+
+ if (mc_intel == NULL)
+ return 0;
+
+ /*
+ * Microcode on this CPU could be updated earlier. Only apply the
+ * microcode patch in mc_intel when it is newer than the one on this
+ * CPU.
+ */
+ if (get_matching_mc(mc_intel, cpu) == 0)
+ return 0;
+
+ /* write microcode via MSR 0x79 */
+ wrmsr(MSR_IA32_UCODE_WRITE,
+ (unsigned long) mc_intel->bits,
+ (unsigned long) mc_intel->bits >> 16 >> 16);
+ wrmsr(MSR_IA32_UCODE_REV, 0, 0);
+
+ /* As documented in the SDM: Do a CPUID 1 here */
+ sync_core();
+
+ /* get the current revision from MSR 0x8B */
+ rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
+
+ if (val[1] != mc_intel->hdr.rev) {
+ pr_err("CPU%d update to revision 0x%x failed\n",
+ cpu_num, mc_intel->hdr.rev);
+ return -1;
+ }
+ pr_info("CPU%d updated to revision 0x%x, date = %04x-%02x-%02x\n",
+ cpu_num, val[1],
+ mc_intel->hdr.date & 0xffff,
+ mc_intel->hdr.date >> 24,
+ (mc_intel->hdr.date >> 16) & 0xff);
+
+ uci->cpu_sig.rev = val[1];
+ c->microcode = val[1];
+
+ return 0;
+}
+
+static enum ucode_state generic_load_microcode(int cpu, void *data, size_t size,
+ int (*get_ucode_data)(void *, const void *, size_t))
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ u8 *ucode_ptr = data, *new_mc = NULL, *mc = NULL;
+ int new_rev = uci->cpu_sig.rev;
+ unsigned int leftover = size;
+ enum ucode_state state = UCODE_OK;
+ unsigned int curr_mc_size = 0;
+ unsigned int csig, cpf;
+
+ while (leftover) {
+ struct microcode_header_intel mc_header;
+ unsigned int mc_size;
+
+ if (get_ucode_data(&mc_header, ucode_ptr, sizeof(mc_header)))
+ break;
+
+ mc_size = get_totalsize(&mc_header);
+ if (!mc_size || mc_size > leftover) {
+ pr_err("error! Bad data in microcode data file\n");
+ break;
+ }
+
+ /* For performance reasons, reuse mc area when possible */
+ if (!mc || mc_size > curr_mc_size) {
+ vfree(mc);
+ mc = vmalloc(mc_size);
+ if (!mc)
+ break;
+ curr_mc_size = mc_size;
+ }
+
+ if (get_ucode_data(mc, ucode_ptr, mc_size) ||
+ microcode_sanity_check(mc, 1) < 0) {
+ break;
+ }
+
+ csig = uci->cpu_sig.sig;
+ cpf = uci->cpu_sig.pf;
+ if (get_matching_microcode(csig, cpf, mc, new_rev)) {
+ vfree(new_mc);
+ new_rev = mc_header.rev;
+ new_mc = mc;
+ mc = NULL; /* trigger new vmalloc */
+ }
+
+ ucode_ptr += mc_size;
+ leftover -= mc_size;
+ }
+
+ vfree(mc);
+
+ if (leftover) {
+ vfree(new_mc);
+ state = UCODE_ERROR;
+ goto out;
+ }
+
+ if (!new_mc) {
+ state = UCODE_NFOUND;
+ goto out;
+ }
+
+ vfree(uci->mc);
+ uci->mc = (struct microcode_intel *)new_mc;
+
+ /*
+ * If early loading microcode is supported, save this mc into
+ * permanent memory. So it will be loaded early when a CPU is hot added
+ * or resumes.
+ */
+ save_mc_for_early(new_mc);
+
+ pr_debug("CPU%d found a matching microcode update with version 0x%x (current=0x%x)\n",
+ cpu, new_rev, uci->cpu_sig.rev);
+out:
+ return state;
+}
+
+static int get_ucode_fw(void *to, const void *from, size_t n)
+{
+ memcpy(to, from, n);
+ return 0;
+}
+
+static enum ucode_state request_microcode_fw(int cpu, struct device *device,
+ bool refresh_fw)
+{
+ char name[30];
+ struct cpuinfo_x86 *c = &cpu_data(cpu);
+ const struct firmware *firmware;
+ enum ucode_state ret;
+
+ sprintf(name, "intel-ucode/%02x-%02x-%02x",
+ c->x86, c->x86_model, c->x86_mask);
+
+ if (request_firmware(&firmware, name, device)) {
+ pr_debug("data file %s load failed\n", name);
+ return UCODE_NFOUND;
+ }
+
+ ret = generic_load_microcode(cpu, (void *)firmware->data,
+ firmware->size, &get_ucode_fw);
+
+ release_firmware(firmware);
+
+ return ret;
+}
+
+static int get_ucode_user(void *to, const void *from, size_t n)
+{
+ return copy_from_user(to, from, n);
+}
+
+static enum ucode_state
+request_microcode_user(int cpu, const void __user *buf, size_t size)
+{
+ return generic_load_microcode(cpu, (void *)buf, size, &get_ucode_user);
+}
+
+static void microcode_fini_cpu(int cpu)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+
+ vfree(uci->mc);
+ uci->mc = NULL;
+}
+
+static struct microcode_ops microcode_intel_ops = {
+ .request_microcode_user = request_microcode_user,
+ .request_microcode_fw = request_microcode_fw,
+ .collect_cpu_info = collect_cpu_info,
+ .apply_microcode = apply_microcode,
+ .microcode_fini_cpu = microcode_fini_cpu,
+};
+
+struct microcode_ops * __init init_intel_microcode(void)
+{
+ struct cpuinfo_x86 *c = &cpu_data(0);
+
+ if (c->x86_vendor != X86_VENDOR_INTEL || c->x86 < 6 ||
+ cpu_has(c, X86_FEATURE_IA64)) {
+ pr_err("Intel CPU family 0x%x not supported\n", c->x86);
+ return NULL;
+ }
+
+ return µcode_intel_ops;
+}
+
--- /dev/null
+/*
+ * Intel CPU microcode early update for Linux
+ *
+ * Copyright (C) 2012 Fenghua Yu <fenghua.yu@intel.com>
+ * H Peter Anvin" <hpa@zytor.com>
+ *
+ * This allows to early upgrade microcode on Intel processors
+ * belonging to IA-32 family - PentiumPro, Pentium II,
+ * Pentium III, Xeon, Pentium 4, etc.
+ *
+ * Reference: Section 9.11 of Volume 3, IA-32 Intel Architecture
+ * Software Developer's Manual.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#include <linux/module.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <linux/earlycpio.h>
+#include <linux/initrd.h>
+#include <linux/cpu.h>
+#include <asm/msr.h>
+#include <asm/microcode_intel.h>
+#include <asm/processor.h>
+#include <asm/tlbflush.h>
+#include <asm/setup.h>
+
+unsigned long mc_saved_in_initrd[MAX_UCODE_COUNT];
+struct mc_saved_data {
+ unsigned int mc_saved_count;
+ struct microcode_intel **mc_saved;
+} mc_saved_data;
+
+static enum ucode_state
+generic_load_microcode_early(struct microcode_intel **mc_saved_p,
+ unsigned int mc_saved_count,
+ struct ucode_cpu_info *uci)
+{
+ struct microcode_intel *ucode_ptr, *new_mc = NULL;
+ int new_rev = uci->cpu_sig.rev;
+ enum ucode_state state = UCODE_OK;
+ unsigned int mc_size;
+ struct microcode_header_intel *mc_header;
+ unsigned int csig = uci->cpu_sig.sig;
+ unsigned int cpf = uci->cpu_sig.pf;
+ int i;
+
+ for (i = 0; i < mc_saved_count; i++) {
+ ucode_ptr = mc_saved_p[i];
+
+ mc_header = (struct microcode_header_intel *)ucode_ptr;
+ mc_size = get_totalsize(mc_header);
+ if (get_matching_microcode(csig, cpf, ucode_ptr, new_rev)) {
+ new_rev = mc_header->rev;
+ new_mc = ucode_ptr;
+ }
+ }
+
+ if (!new_mc) {
+ state = UCODE_NFOUND;
+ goto out;
+ }
+
+ uci->mc = (struct microcode_intel *)new_mc;
+out:
+ return state;
+}
+
+static void
+microcode_pointer(struct microcode_intel **mc_saved,
+ unsigned long *mc_saved_in_initrd,
+ unsigned long initrd_start, int mc_saved_count)
+{
+ int i;
+
+ for (i = 0; i < mc_saved_count; i++)
+ mc_saved[i] = (struct microcode_intel *)
+ (mc_saved_in_initrd[i] + initrd_start);
+}
+
+#ifdef CONFIG_X86_32
+static void
+microcode_phys(struct microcode_intel **mc_saved_tmp,
+ struct mc_saved_data *mc_saved_data)
+{
+ int i;
+ struct microcode_intel ***mc_saved;
+
+ mc_saved = (struct microcode_intel ***)
+ __pa_nodebug(&mc_saved_data->mc_saved);
+ for (i = 0; i < mc_saved_data->mc_saved_count; i++) {
+ struct microcode_intel *p;
+
+ p = *(struct microcode_intel **)
+ __pa_nodebug(mc_saved_data->mc_saved + i);
+ mc_saved_tmp[i] = (struct microcode_intel *)__pa_nodebug(p);
+ }
+}
+#endif
+
+static enum ucode_state
+load_microcode(struct mc_saved_data *mc_saved_data,
+ unsigned long *mc_saved_in_initrd,
+ unsigned long initrd_start,
+ struct ucode_cpu_info *uci)
+{
+ struct microcode_intel *mc_saved_tmp[MAX_UCODE_COUNT];
+ unsigned int count = mc_saved_data->mc_saved_count;
+
+ if (!mc_saved_data->mc_saved) {
+ microcode_pointer(mc_saved_tmp, mc_saved_in_initrd,
+ initrd_start, count);
+
+ return generic_load_microcode_early(mc_saved_tmp, count, uci);
+ } else {
+#ifdef CONFIG_X86_32
+ microcode_phys(mc_saved_tmp, mc_saved_data);
+ return generic_load_microcode_early(mc_saved_tmp, count, uci);
+#else
+ return generic_load_microcode_early(mc_saved_data->mc_saved,
+ count, uci);
+#endif
+ }
+}
+
+static u8 get_x86_family(unsigned long sig)
+{
+ u8 x86;
+
+ x86 = (sig >> 8) & 0xf;
+
+ if (x86 == 0xf)
+ x86 += (sig >> 20) & 0xff;
+
+ return x86;
+}
+
+static u8 get_x86_model(unsigned long sig)
+{
+ u8 x86, x86_model;
+
+ x86 = get_x86_family(sig);
+ x86_model = (sig >> 4) & 0xf;
+
+ if (x86 == 0x6 || x86 == 0xf)
+ x86_model += ((sig >> 16) & 0xf) << 4;
+
+ return x86_model;
+}
+
+/*
+ * Given CPU signature and a microcode patch, this function finds if the
+ * microcode patch has matching family and model with the CPU.
+ */
+static enum ucode_state
+matching_model_microcode(struct microcode_header_intel *mc_header,
+ unsigned long sig)
+{
+ u8 x86, x86_model;
+ u8 x86_ucode, x86_model_ucode;
+ struct extended_sigtable *ext_header;
+ unsigned long total_size = get_totalsize(mc_header);
+ unsigned long data_size = get_datasize(mc_header);
+ int ext_sigcount, i;
+ struct extended_signature *ext_sig;
+
+ x86 = get_x86_family(sig);
+ x86_model = get_x86_model(sig);
+
+ x86_ucode = get_x86_family(mc_header->sig);
+ x86_model_ucode = get_x86_model(mc_header->sig);
+
+ if (x86 == x86_ucode && x86_model == x86_model_ucode)
+ return UCODE_OK;
+
+ /* Look for ext. headers: */
+ if (total_size <= data_size + MC_HEADER_SIZE)
+ return UCODE_NFOUND;
+
+ ext_header = (struct extended_sigtable *)
+ mc_header + data_size + MC_HEADER_SIZE;
+ ext_sigcount = ext_header->count;
+ ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
+
+ for (i = 0; i < ext_sigcount; i++) {
+ x86_ucode = get_x86_family(ext_sig->sig);
+ x86_model_ucode = get_x86_model(ext_sig->sig);
+
+ if (x86 == x86_ucode && x86_model == x86_model_ucode)
+ return UCODE_OK;
+
+ ext_sig++;
+ }
+
+ return UCODE_NFOUND;
+}
+
+static int
+save_microcode(struct mc_saved_data *mc_saved_data,
+ struct microcode_intel **mc_saved_src,
+ unsigned int mc_saved_count)
+{
+ int i, j;
+ struct microcode_intel **mc_saved_p;
+ int ret;
+
+ if (!mc_saved_count)
+ return -EINVAL;
+
+ /*
+ * Copy new microcode data.
+ */
+ mc_saved_p = kmalloc(mc_saved_count*sizeof(struct microcode_intel *),
+ GFP_KERNEL);
+ if (!mc_saved_p)
+ return -ENOMEM;
+
+ for (i = 0; i < mc_saved_count; i++) {
+ struct microcode_intel *mc = mc_saved_src[i];
+ struct microcode_header_intel *mc_header = &mc->hdr;
+ unsigned long mc_size = get_totalsize(mc_header);
+ mc_saved_p[i] = kmalloc(mc_size, GFP_KERNEL);
+ if (!mc_saved_p[i]) {
+ ret = -ENOMEM;
+ goto err;
+ }
+ if (!mc_saved_src[i]) {
+ ret = -EINVAL;
+ goto err;
+ }
+ memcpy(mc_saved_p[i], mc, mc_size);
+ }
+
+ /*
+ * Point to newly saved microcode.
+ */
+ mc_saved_data->mc_saved = mc_saved_p;
+ mc_saved_data->mc_saved_count = mc_saved_count;
+
+ return 0;
+
+err:
+ for (j = 0; j <= i; j++)
+ kfree(mc_saved_p[j]);
+ kfree(mc_saved_p);
+
+ return ret;
+}
+
+/*
+ * A microcode patch in ucode_ptr is saved into mc_saved
+ * - if it has matching signature and newer revision compared to an existing
+ * patch mc_saved.
+ * - or if it is a newly discovered microcode patch.
+ *
+ * The microcode patch should have matching model with CPU.
+ */
+static void _save_mc(struct microcode_intel **mc_saved, u8 *ucode_ptr,
+ unsigned int *mc_saved_count_p)
+{
+ int i;
+ int found = 0;
+ unsigned int mc_saved_count = *mc_saved_count_p;
+ struct microcode_header_intel *mc_header;
+
+ mc_header = (struct microcode_header_intel *)ucode_ptr;
+ for (i = 0; i < mc_saved_count; i++) {
+ unsigned int sig, pf;
+ unsigned int new_rev;
+ struct microcode_header_intel *mc_saved_header =
+ (struct microcode_header_intel *)mc_saved[i];
+ sig = mc_saved_header->sig;
+ pf = mc_saved_header->pf;
+ new_rev = mc_header->rev;
+
+ if (get_matching_sig(sig, pf, ucode_ptr, new_rev)) {
+ found = 1;
+ if (update_match_revision(mc_header, new_rev)) {
+ /*
+ * Found an older ucode saved before.
+ * Replace the older one with this newer
+ * one.
+ */
+ mc_saved[i] =
+ (struct microcode_intel *)ucode_ptr;
+ break;
+ }
+ }
+ }
+ if (i >= mc_saved_count && !found)
+ /*
+ * This ucode is first time discovered in ucode file.
+ * Save it to memory.
+ */
+ mc_saved[mc_saved_count++] =
+ (struct microcode_intel *)ucode_ptr;
+
+ *mc_saved_count_p = mc_saved_count;
+}
+
+/*
+ * Get microcode matching with BSP's model. Only CPUs with the same model as
+ * BSP can stay in the platform.
+ */
+static enum ucode_state __init
+get_matching_model_microcode(int cpu, unsigned long start,
+ void *data, size_t size,
+ struct mc_saved_data *mc_saved_data,
+ unsigned long *mc_saved_in_initrd,
+ struct ucode_cpu_info *uci)
+{
+ u8 *ucode_ptr = data;
+ unsigned int leftover = size;
+ enum ucode_state state = UCODE_OK;
+ unsigned int mc_size;
+ struct microcode_header_intel *mc_header;
+ struct microcode_intel *mc_saved_tmp[MAX_UCODE_COUNT];
+ unsigned int mc_saved_count = mc_saved_data->mc_saved_count;
+ int i;
+
+ while (leftover) {
+ mc_header = (struct microcode_header_intel *)ucode_ptr;
+
+ mc_size = get_totalsize(mc_header);
+ if (!mc_size || mc_size > leftover ||
+ microcode_sanity_check(ucode_ptr, 0) < 0)
+ break;
+
+ leftover -= mc_size;
+
+ /*
+ * Since APs with same family and model as the BSP may boot in
+ * the platform, we need to find and save microcode patches
+ * with the same family and model as the BSP.
+ */
+ if (matching_model_microcode(mc_header, uci->cpu_sig.sig) !=
+ UCODE_OK) {
+ ucode_ptr += mc_size;
+ continue;
+ }
+
+ _save_mc(mc_saved_tmp, ucode_ptr, &mc_saved_count);
+
+ ucode_ptr += mc_size;
+ }
+
+ if (leftover) {
+ state = UCODE_ERROR;
+ goto out;
+ }
+
+ if (mc_saved_count == 0) {
+ state = UCODE_NFOUND;
+ goto out;
+ }
+
+ for (i = 0; i < mc_saved_count; i++)
+ mc_saved_in_initrd[i] = (unsigned long)mc_saved_tmp[i] - start;
+
+ mc_saved_data->mc_saved_count = mc_saved_count;
+out:
+ return state;
+}
+
+static int collect_cpu_info_early(struct ucode_cpu_info *uci)
+{
+ unsigned int val[2];
+ u8 x86, x86_model;
+ struct cpu_signature csig;
+ unsigned int eax, ebx, ecx, edx;
+
+ csig.sig = 0;
+ csig.pf = 0;
+ csig.rev = 0;
+
+ memset(uci, 0, sizeof(*uci));
+
+ eax = 0x00000001;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ csig.sig = eax;
+
+ x86 = get_x86_family(csig.sig);
+ x86_model = get_x86_model(csig.sig);
+
+ if ((x86_model >= 5) || (x86 > 6)) {
+ /* get processor flags from MSR 0x17 */
+ native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
+ csig.pf = 1 << ((val[1] >> 18) & 7);
+ }
+ native_wrmsr(MSR_IA32_UCODE_REV, 0, 0);
+
+ /* As documented in the SDM: Do a CPUID 1 here */
+ sync_core();
+
+ /* get the current revision from MSR 0x8B */
+ native_rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
+
+ csig.rev = val[1];
+
+ uci->cpu_sig = csig;
+ uci->valid = 1;
+
+ return 0;
+}
+
+#ifdef DEBUG
+static void __ref show_saved_mc(void)
+{
+ int i, j;
+ unsigned int sig, pf, rev, total_size, data_size, date;
+ struct ucode_cpu_info uci;
+
+ if (mc_saved_data.mc_saved_count == 0) {
+ pr_debug("no micorcode data saved.\n");
+ return;
+ }
+ pr_debug("Total microcode saved: %d\n", mc_saved_data.mc_saved_count);
+
+ collect_cpu_info_early(&uci);
+
+ sig = uci.cpu_sig.sig;
+ pf = uci.cpu_sig.pf;
+ rev = uci.cpu_sig.rev;
+ pr_debug("CPU%d: sig=0x%x, pf=0x%x, rev=0x%x\n",
+ smp_processor_id(), sig, pf, rev);
+
+ for (i = 0; i < mc_saved_data.mc_saved_count; i++) {
+ struct microcode_header_intel *mc_saved_header;
+ struct extended_sigtable *ext_header;
+ int ext_sigcount;
+ struct extended_signature *ext_sig;
+
+ mc_saved_header = (struct microcode_header_intel *)
+ mc_saved_data.mc_saved[i];
+ sig = mc_saved_header->sig;
+ pf = mc_saved_header->pf;
+ rev = mc_saved_header->rev;
+ total_size = get_totalsize(mc_saved_header);
+ data_size = get_datasize(mc_saved_header);
+ date = mc_saved_header->date;
+
+ pr_debug("mc_saved[%d]: sig=0x%x, pf=0x%x, rev=0x%x, toal size=0x%x, date = %04x-%02x-%02x\n",
+ i, sig, pf, rev, total_size,
+ date & 0xffff,
+ date >> 24,
+ (date >> 16) & 0xff);
+
+ /* Look for ext. headers: */
+ if (total_size <= data_size + MC_HEADER_SIZE)
+ continue;
+
+ ext_header = (struct extended_sigtable *)
+ mc_saved_header + data_size + MC_HEADER_SIZE;
+ ext_sigcount = ext_header->count;
+ ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
+
+ for (j = 0; j < ext_sigcount; j++) {
+ sig = ext_sig->sig;
+ pf = ext_sig->pf;
+
+ pr_debug("\tExtended[%d]: sig=0x%x, pf=0x%x\n",
+ j, sig, pf);
+
+ ext_sig++;
+ }
+
+ }
+}
+#else
+static inline void show_saved_mc(void)
+{
+}
+#endif
+
+#if defined(CONFIG_MICROCODE_INTEL_EARLY) && defined(CONFIG_HOTPLUG_CPU)
+static DEFINE_MUTEX(x86_cpu_microcode_mutex);
+/*
+ * Save this mc into mc_saved_data. So it will be loaded early when a CPU is
+ * hot added or resumes.
+ *
+ * Please make sure this mc should be a valid microcode patch before calling
+ * this function.
+ */
+int save_mc_for_early(u8 *mc)
+{
+ struct microcode_intel *mc_saved_tmp[MAX_UCODE_COUNT];
+ unsigned int mc_saved_count_init;
+ unsigned int mc_saved_count;
+ struct microcode_intel **mc_saved;
+ int ret = 0;
+ int i;
+
+ /*
+ * Hold hotplug lock so mc_saved_data is not accessed by a CPU in
+ * hotplug.
+ */
+ mutex_lock(&x86_cpu_microcode_mutex);
+
+ mc_saved_count_init = mc_saved_data.mc_saved_count;
+ mc_saved_count = mc_saved_data.mc_saved_count;
+ mc_saved = mc_saved_data.mc_saved;
+
+ if (mc_saved && mc_saved_count)
+ memcpy(mc_saved_tmp, mc_saved,
+ mc_saved_count * sizeof(struct mirocode_intel *));
+ /*
+ * Save the microcode patch mc in mc_save_tmp structure if it's a newer
+ * version.
+ */
+
+ _save_mc(mc_saved_tmp, mc, &mc_saved_count);
+
+ /*
+ * Save the mc_save_tmp in global mc_saved_data.
+ */
+ ret = save_microcode(&mc_saved_data, mc_saved_tmp, mc_saved_count);
+ if (ret) {
+ pr_err("Cannot save microcode patch.\n");
+ goto out;
+ }
+
+ show_saved_mc();
+
+ /*
+ * Free old saved microcod data.
+ */
+ if (mc_saved) {
+ for (i = 0; i < mc_saved_count_init; i++)
+ kfree(mc_saved[i]);
+ kfree(mc_saved);
+ }
+
+out:
+ mutex_unlock(&x86_cpu_microcode_mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(save_mc_for_early);
+#endif
+
+static __initdata char ucode_name[] = "kernel/x86/microcode/GenuineIntel.bin";
+static __init enum ucode_state
+scan_microcode(unsigned long start, unsigned long end,
+ struct mc_saved_data *mc_saved_data,
+ unsigned long *mc_saved_in_initrd,
+ struct ucode_cpu_info *uci)
+{
+ unsigned int size = end - start + 1;
+ struct cpio_data cd;
+ long offset = 0;
+#ifdef CONFIG_X86_32
+ char *p = (char *)__pa_nodebug(ucode_name);
+#else
+ char *p = ucode_name;
+#endif
+
+ cd.data = NULL;
+ cd.size = 0;
+
+ cd = find_cpio_data(p, (void *)start, size, &offset);
+ if (!cd.data)
+ return UCODE_ERROR;
+
+
+ return get_matching_model_microcode(0, start, cd.data, cd.size,
+ mc_saved_data, mc_saved_in_initrd,
+ uci);
+}
+
+/*
+ * Print ucode update info.
+ */
+static void
+print_ucode_info(struct ucode_cpu_info *uci, unsigned int date)
+{
+ int cpu = smp_processor_id();
+
+ pr_info("CPU%d microcode updated early to revision 0x%x, date = %04x-%02x-%02x\n",
+ cpu,
+ uci->cpu_sig.rev,
+ date & 0xffff,
+ date >> 24,
+ (date >> 16) & 0xff);
+}
+
+#ifdef CONFIG_X86_32
+
+static int delay_ucode_info;
+static int current_mc_date;
+
+/*
+ * Print early updated ucode info after printk works. This is delayed info dump.
+ */
+void show_ucode_info_early(void)
+{
+ struct ucode_cpu_info uci;
+
+ if (delay_ucode_info) {
+ collect_cpu_info_early(&uci);
+ print_ucode_info(&uci, current_mc_date);
+ delay_ucode_info = 0;
+ }
+}
+
+/*
+ * At this point, we can not call printk() yet. Keep microcode patch number in
+ * mc_saved_data.mc_saved and delay printing microcode info in
+ * show_ucode_info_early() until printk() works.
+ */
+static void print_ucode(struct ucode_cpu_info *uci)
+{
+ struct microcode_intel *mc_intel;
+ int *delay_ucode_info_p;
+ int *current_mc_date_p;
+
+ mc_intel = uci->mc;
+ if (mc_intel == NULL)
+ return;
+
+ delay_ucode_info_p = (int *)__pa_nodebug(&delay_ucode_info);
+ current_mc_date_p = (int *)__pa_nodebug(¤t_mc_date);
+
+ *delay_ucode_info_p = 1;
+ *current_mc_date_p = mc_intel->hdr.date;
+}
+#else
+
+/*
+ * Flush global tlb. We only do this in x86_64 where paging has been enabled
+ * already and PGE should be enabled as well.
+ */
+static inline void flush_tlb_early(void)
+{
+ __native_flush_tlb_global_irq_disabled();
+}
+
+static inline void print_ucode(struct ucode_cpu_info *uci)
+{
+ struct microcode_intel *mc_intel;
+
+ mc_intel = uci->mc;
+ if (mc_intel == NULL)
+ return;
+
+ print_ucode_info(uci, mc_intel->hdr.date);
+}
+#endif
+
+static int apply_microcode_early(struct mc_saved_data *mc_saved_data,
+ struct ucode_cpu_info *uci)
+{
+ struct microcode_intel *mc_intel;
+ unsigned int val[2];
+
+ mc_intel = uci->mc;
+ if (mc_intel == NULL)
+ return 0;
+
+ /* write microcode via MSR 0x79 */
+ native_wrmsr(MSR_IA32_UCODE_WRITE,
+ (unsigned long) mc_intel->bits,
+ (unsigned long) mc_intel->bits >> 16 >> 16);
+ native_wrmsr(MSR_IA32_UCODE_REV, 0, 0);
+
+ /* As documented in the SDM: Do a CPUID 1 here */
+ sync_core();
+
+ /* get the current revision from MSR 0x8B */
+ native_rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
+ if (val[1] != mc_intel->hdr.rev)
+ return -1;
+
+#ifdef CONFIG_X86_64
+ /* Flush global tlb. This is precaution. */
+ flush_tlb_early();
+#endif
+ uci->cpu_sig.rev = val[1];
+
+ print_ucode(uci);
+
+ return 0;
+}
+
+/*
+ * This function converts microcode patch offsets previously stored in
+ * mc_saved_in_initrd to pointers and stores the pointers in mc_saved_data.
+ */
+int __init save_microcode_in_initrd_intel(void)
+{
+ unsigned int count = mc_saved_data.mc_saved_count;
+ struct microcode_intel *mc_saved[MAX_UCODE_COUNT];
+ int ret = 0;
+
+ if (count == 0)
+ return ret;
+
+ microcode_pointer(mc_saved, mc_saved_in_initrd, initrd_start, count);
+ ret = save_microcode(&mc_saved_data, mc_saved, count);
+ if (ret)
+ pr_err("Cannot save microcode patches from initrd.\n");
+
+ show_saved_mc();
+
+ return ret;
+}
+
+static void __init
+_load_ucode_intel_bsp(struct mc_saved_data *mc_saved_data,
+ unsigned long *mc_saved_in_initrd,
+ unsigned long initrd_start_early,
+ unsigned long initrd_end_early,
+ struct ucode_cpu_info *uci)
+{
+ collect_cpu_info_early(uci);
+ scan_microcode(initrd_start_early, initrd_end_early, mc_saved_data,
+ mc_saved_in_initrd, uci);
+ load_microcode(mc_saved_data, mc_saved_in_initrd,
+ initrd_start_early, uci);
+ apply_microcode_early(mc_saved_data, uci);
+}
+
+void __init
+load_ucode_intel_bsp(void)
+{
+ u64 ramdisk_image, ramdisk_size;
+ unsigned long initrd_start_early, initrd_end_early;
+ struct ucode_cpu_info uci;
+#ifdef CONFIG_X86_32
+ struct boot_params *boot_params_p;
+
+ boot_params_p = (struct boot_params *)__pa_nodebug(&boot_params);
+ ramdisk_image = boot_params_p->hdr.ramdisk_image;
+ ramdisk_size = boot_params_p->hdr.ramdisk_size;
+ initrd_start_early = ramdisk_image;
+ initrd_end_early = initrd_start_early + ramdisk_size;
+
+ _load_ucode_intel_bsp(
+ (struct mc_saved_data *)__pa_nodebug(&mc_saved_data),
+ (unsigned long *)__pa_nodebug(&mc_saved_in_initrd),
+ initrd_start_early, initrd_end_early, &uci);
+#else
+ ramdisk_image = boot_params.hdr.ramdisk_image;
+ ramdisk_size = boot_params.hdr.ramdisk_size;
+ initrd_start_early = ramdisk_image + PAGE_OFFSET;
+ initrd_end_early = initrd_start_early + ramdisk_size;
+
+ _load_ucode_intel_bsp(&mc_saved_data, mc_saved_in_initrd,
+ initrd_start_early, initrd_end_early, &uci);
+#endif
+}
+
+void load_ucode_intel_ap(void)
+{
+ struct mc_saved_data *mc_saved_data_p;
+ struct ucode_cpu_info uci;
+ unsigned long *mc_saved_in_initrd_p;
+ unsigned long initrd_start_addr;
+#ifdef CONFIG_X86_32
+ unsigned long *initrd_start_p;
+
+ mc_saved_in_initrd_p =
+ (unsigned long *)__pa_nodebug(mc_saved_in_initrd);
+ mc_saved_data_p = (struct mc_saved_data *)__pa_nodebug(&mc_saved_data);
+ initrd_start_p = (unsigned long *)__pa_nodebug(&initrd_start);
+ initrd_start_addr = (unsigned long)__pa_nodebug(*initrd_start_p);
+#else
+ mc_saved_data_p = &mc_saved_data;
+ mc_saved_in_initrd_p = mc_saved_in_initrd;
+ initrd_start_addr = initrd_start;
+#endif
+
+ /*
+ * If there is no valid ucode previously saved in memory, no need to
+ * update ucode on this AP.
+ */
+ if (mc_saved_data_p->mc_saved_count == 0)
+ return;
+
+ collect_cpu_info_early(&uci);
+ load_microcode(mc_saved_data_p, mc_saved_in_initrd_p,
+ initrd_start_addr, &uci);
+ apply_microcode_early(mc_saved_data_p, &uci);
+}
--- /dev/null
+/*
+ * Intel CPU Microcode Update Driver for Linux
+ *
+ * Copyright (C) 2012 Fenghua Yu <fenghua.yu@intel.com>
+ * H Peter Anvin" <hpa@zytor.com>
+ *
+ * This driver allows to upgrade microcode on Intel processors
+ * belonging to IA-32 family - PentiumPro, Pentium II,
+ * Pentium III, Xeon, Pentium 4, etc.
+ *
+ * Reference: Section 8.11 of Volume 3a, IA-32 Intel? Architecture
+ * Software Developer's Manual
+ * Order Number 253668 or free download from:
+ *
+ * http://developer.intel.com/Assets/PDF/manual/253668.pdf
+ *
+ * For more information, go to http://www.urbanmyth.org/microcode
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ */
+#include <linux/firmware.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+#include <asm/microcode_intel.h>
+#include <asm/processor.h>
+#include <asm/msr.h>
+
+static inline int
+update_match_cpu(unsigned int csig, unsigned int cpf,
+ unsigned int sig, unsigned int pf)
+{
+ return (!sigmatch(sig, csig, pf, cpf)) ? 0 : 1;
+}
+
+int
+update_match_revision(struct microcode_header_intel *mc_header, int rev)
+{
+ return (mc_header->rev <= rev) ? 0 : 1;
+}
+
+int microcode_sanity_check(void *mc, int print_err)
+{
+ unsigned long total_size, data_size, ext_table_size;
+ struct microcode_header_intel *mc_header = mc;
+ struct extended_sigtable *ext_header = NULL;
+ int sum, orig_sum, ext_sigcount = 0, i;
+ struct extended_signature *ext_sig;
+
+ total_size = get_totalsize(mc_header);
+ data_size = get_datasize(mc_header);
+
+ if (data_size + MC_HEADER_SIZE > total_size) {
+ if (print_err)
+ pr_err("error! Bad data size in microcode data file\n");
+ return -EINVAL;
+ }
+
+ if (mc_header->ldrver != 1 || mc_header->hdrver != 1) {
+ if (print_err)
+ pr_err("error! Unknown microcode update format\n");
+ return -EINVAL;
+ }
+ ext_table_size = total_size - (MC_HEADER_SIZE + data_size);
+ if (ext_table_size) {
+ if ((ext_table_size < EXT_HEADER_SIZE)
+ || ((ext_table_size - EXT_HEADER_SIZE) % EXT_SIGNATURE_SIZE)) {
+ if (print_err)
+ pr_err("error! Small exttable size in microcode data file\n");
+ return -EINVAL;
+ }
+ ext_header = mc + MC_HEADER_SIZE + data_size;
+ if (ext_table_size != exttable_size(ext_header)) {
+ if (print_err)
+ pr_err("error! Bad exttable size in microcode data file\n");
+ return -EFAULT;
+ }
+ ext_sigcount = ext_header->count;
+ }
+
+ /* check extended table checksum */
+ if (ext_table_size) {
+ int ext_table_sum = 0;
+ int *ext_tablep = (int *)ext_header;
+
+ i = ext_table_size / DWSIZE;
+ while (i--)
+ ext_table_sum += ext_tablep[i];
+ if (ext_table_sum) {
+ if (print_err)
+ pr_warn("aborting, bad extended signature table checksum\n");
+ return -EINVAL;
+ }
+ }
+
+ /* calculate the checksum */
+ orig_sum = 0;
+ i = (MC_HEADER_SIZE + data_size) / DWSIZE;
+ while (i--)
+ orig_sum += ((int *)mc)[i];
+ if (orig_sum) {
+ if (print_err)
+ pr_err("aborting, bad checksum\n");
+ return -EINVAL;
+ }
+ if (!ext_table_size)
+ return 0;
+ /* check extended signature checksum */
+ for (i = 0; i < ext_sigcount; i++) {
+ ext_sig = (void *)ext_header + EXT_HEADER_SIZE +
+ EXT_SIGNATURE_SIZE * i;
+ sum = orig_sum
+ - (mc_header->sig + mc_header->pf + mc_header->cksum)
+ + (ext_sig->sig + ext_sig->pf + ext_sig->cksum);
+ if (sum) {
+ if (print_err)
+ pr_err("aborting, bad checksum\n");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+EXPORT_SYMBOL_GPL(microcode_sanity_check);
+
+/*
+ * return 0 - no update found
+ * return 1 - found update
+ */
+int get_matching_sig(unsigned int csig, int cpf, void *mc, int rev)
+{
+ struct microcode_header_intel *mc_header = mc;
+ struct extended_sigtable *ext_header;
+ unsigned long total_size = get_totalsize(mc_header);
+ int ext_sigcount, i;
+ struct extended_signature *ext_sig;
+
+ if (update_match_cpu(csig, cpf, mc_header->sig, mc_header->pf))
+ return 1;
+
+ /* Look for ext. headers: */
+ if (total_size <= get_datasize(mc_header) + MC_HEADER_SIZE)
+ return 0;
+
+ ext_header = mc + get_datasize(mc_header) + MC_HEADER_SIZE;
+ ext_sigcount = ext_header->count;
+ ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
+
+ for (i = 0; i < ext_sigcount; i++) {
+ if (update_match_cpu(csig, cpf, ext_sig->sig, ext_sig->pf))
+ return 1;
+ ext_sig++;
+ }
+ return 0;
+}
+
+/*
+ * return 0 - no update found
+ * return 1 - found update
+ */
+int get_matching_microcode(unsigned int csig, int cpf, void *mc, int rev)
+{
+ struct microcode_header_intel *mc_header = mc;
+
+ if (!update_match_revision(mc_header, rev))
+ return 0;
+
+ return get_matching_sig(csig, cpf, mc, rev);
+}
+EXPORT_SYMBOL_GPL(get_matching_microcode);
void arch_perf_update_userpage(struct perf_event_mmap_page *userpg, u64 now)
{
+ struct cyc2ns_data *data;
+
userpg->cap_user_time = 0;
userpg->cap_user_time_zero = 0;
userpg->cap_user_rdpmc = x86_pmu.attr_rdpmc;
userpg->pmc_width = x86_pmu.cntval_bits;
- if (!sched_clock_stable)
+ if (!sched_clock_stable())
return;
+ data = cyc2ns_read_begin();
+
userpg->cap_user_time = 1;
- userpg->time_mult = this_cpu_read(cyc2ns);
- userpg->time_shift = CYC2NS_SCALE_FACTOR;
- userpg->time_offset = this_cpu_read(cyc2ns_offset) - now;
+ userpg->time_mult = data->cyc2ns_mul;
+ userpg->time_shift = data->cyc2ns_shift;
+ userpg->time_offset = data->cyc2ns_offset - now;
userpg->cap_user_time_zero = 1;
- userpg->time_zero = this_cpu_read(cyc2ns_offset);
+ userpg->time_zero = data->cyc2ns_offset;
+
+ cyc2ns_read_end(data);
}
/*
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/ptrace.h>
+#include <linux/syscore_ops.h>
#include <asm/apic.h>
return ret;
}
+static void ibs_eilvt_setup(void)
+{
+ /*
+ * Force LVT offset assignment for family 10h: The offsets are
+ * not assigned by the BIOS for this family, so the OS is
+ * responsible for doing it. If the OS assignment fails, fall
+ * back to BIOS settings and try to setup this.
+ */
+ if (boot_cpu_data.x86 == 0x10)
+ force_ibs_eilvt_setup();
+}
+
static inline int get_ibs_lvt_offset(void)
{
u64 val;
setup_APIC_eilvt(offset, 0, APIC_EILVT_MSG_FIX, 1);
}
+#ifdef CONFIG_PM
+
+static int perf_ibs_suspend(void)
+{
+ clear_APIC_ibs(NULL);
+ return 0;
+}
+
+static void perf_ibs_resume(void)
+{
+ ibs_eilvt_setup();
+ setup_APIC_ibs(NULL);
+}
+
+static struct syscore_ops perf_ibs_syscore_ops = {
+ .resume = perf_ibs_resume,
+ .suspend = perf_ibs_suspend,
+};
+
+static void perf_ibs_pm_init(void)
+{
+ register_syscore_ops(&perf_ibs_syscore_ops);
+}
+
+#else
+
+static inline void perf_ibs_pm_init(void) { }
+
+#endif
+
static int
perf_ibs_cpu_notifier(struct notifier_block *self, unsigned long action, void *hcpu)
{
if (!caps)
return -ENODEV; /* ibs not supported by the cpu */
- /*
- * Force LVT offset assignment for family 10h: The offsets are
- * not assigned by the BIOS for this family, so the OS is
- * responsible for doing it. If the OS assignment fails, fall
- * back to BIOS settings and try to setup this.
- */
- if (boot_cpu_data.x86 == 0x10)
- force_ibs_eilvt_setup();
+ ibs_eilvt_setup();
if (!ibs_eilvt_valid())
goto out;
+ perf_ibs_pm_init();
get_online_cpus();
ibs_caps = caps;
/* make ibs_caps visible to other cpus: */
--- /dev/null
+/*
+ * perf_event_intel_rapl.c: support Intel RAPL energy consumption counters
+ * Copyright (C) 2013 Google, Inc., Stephane Eranian
+ *
+ * Intel RAPL interface is specified in the IA-32 Manual Vol3b
+ * section 14.7.1 (September 2013)
+ *
+ * RAPL provides more controls than just reporting energy consumption
+ * however here we only expose the 3 energy consumption free running
+ * counters (pp0, pkg, dram).
+ *
+ * Each of those counters increments in a power unit defined by the
+ * RAPL_POWER_UNIT MSR. On SandyBridge, this unit is 1/(2^16) Joules
+ * but it can vary.
+ *
+ * Counter to rapl events mappings:
+ *
+ * pp0 counter: consumption of all physical cores (power plane 0)
+ * event: rapl_energy_cores
+ * perf code: 0x1
+ *
+ * pkg counter: consumption of the whole processor package
+ * event: rapl_energy_pkg
+ * perf code: 0x2
+ *
+ * dram counter: consumption of the dram domain (servers only)
+ * event: rapl_energy_dram
+ * perf code: 0x3
+ *
+ * dram counter: consumption of the builtin-gpu domain (client only)
+ * event: rapl_energy_gpu
+ * perf code: 0x4
+ *
+ * We manage those counters as free running (read-only). They may be
+ * use simultaneously by other tools, such as turbostat.
+ *
+ * The events only support system-wide mode counting. There is no
+ * sampling support because it does not make sense and is not
+ * supported by the RAPL hardware.
+ *
+ * Because we want to avoid floating-point operations in the kernel,
+ * the events are all reported in fixed point arithmetic (32.32).
+ * Tools must adjust the counts to convert them to Watts using
+ * the duration of the measurement. Tools may use a function such as
+ * ldexp(raw_count, -32);
+ */
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/perf_event.h>
+#include <asm/cpu_device_id.h>
+#include "perf_event.h"
+
+/*
+ * RAPL energy status counters
+ */
+#define RAPL_IDX_PP0_NRG_STAT 0 /* all cores */
+#define INTEL_RAPL_PP0 0x1 /* pseudo-encoding */
+#define RAPL_IDX_PKG_NRG_STAT 1 /* entire package */
+#define INTEL_RAPL_PKG 0x2 /* pseudo-encoding */
+#define RAPL_IDX_RAM_NRG_STAT 2 /* DRAM */
+#define INTEL_RAPL_RAM 0x3 /* pseudo-encoding */
+#define RAPL_IDX_PP1_NRG_STAT 3 /* DRAM */
+#define INTEL_RAPL_PP1 0x4 /* pseudo-encoding */
+
+/* Clients have PP0, PKG */
+#define RAPL_IDX_CLN (1<<RAPL_IDX_PP0_NRG_STAT|\
+ 1<<RAPL_IDX_PKG_NRG_STAT|\
+ 1<<RAPL_IDX_PP1_NRG_STAT)
+
+/* Servers have PP0, PKG, RAM */
+#define RAPL_IDX_SRV (1<<RAPL_IDX_PP0_NRG_STAT|\
+ 1<<RAPL_IDX_PKG_NRG_STAT|\
+ 1<<RAPL_IDX_RAM_NRG_STAT)
+
+/*
+ * event code: LSB 8 bits, passed in attr->config
+ * any other bit is reserved
+ */
+#define RAPL_EVENT_MASK 0xFFULL
+
+#define DEFINE_RAPL_FORMAT_ATTR(_var, _name, _format) \
+static ssize_t __rapl_##_var##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, \
+ char *page) \
+{ \
+ BUILD_BUG_ON(sizeof(_format) >= PAGE_SIZE); \
+ return sprintf(page, _format "\n"); \
+} \
+static struct kobj_attribute format_attr_##_var = \
+ __ATTR(_name, 0444, __rapl_##_var##_show, NULL)
+
+#define RAPL_EVENT_DESC(_name, _config) \
+{ \
+ .attr = __ATTR(_name, 0444, rapl_event_show, NULL), \
+ .config = _config, \
+}
+
+#define RAPL_CNTR_WIDTH 32 /* 32-bit rapl counters */
+
+struct rapl_pmu {
+ spinlock_t lock;
+ int hw_unit; /* 1/2^hw_unit Joule */
+ int n_active; /* number of active events */
+ struct list_head active_list;
+ struct pmu *pmu; /* pointer to rapl_pmu_class */
+ ktime_t timer_interval; /* in ktime_t unit */
+ struct hrtimer hrtimer;
+};
+
+static struct pmu rapl_pmu_class;
+static cpumask_t rapl_cpu_mask;
+static int rapl_cntr_mask;
+
+static DEFINE_PER_CPU(struct rapl_pmu *, rapl_pmu);
+static DEFINE_PER_CPU(struct rapl_pmu *, rapl_pmu_to_free);
+
+static inline u64 rapl_read_counter(struct perf_event *event)
+{
+ u64 raw;
+ rdmsrl(event->hw.event_base, raw);
+ return raw;
+}
+
+static inline u64 rapl_scale(u64 v)
+{
+ /*
+ * scale delta to smallest unit (1/2^32)
+ * users must then scale back: count * 1/(1e9*2^32) to get Joules
+ * or use ldexp(count, -32).
+ * Watts = Joules/Time delta
+ */
+ return v << (32 - __get_cpu_var(rapl_pmu)->hw_unit);
+}
+
+static u64 rapl_event_update(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ u64 prev_raw_count, new_raw_count;
+ s64 delta, sdelta;
+ int shift = RAPL_CNTR_WIDTH;
+
+again:
+ prev_raw_count = local64_read(&hwc->prev_count);
+ rdmsrl(event->hw.event_base, new_raw_count);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev_raw_count,
+ new_raw_count) != prev_raw_count) {
+ cpu_relax();
+ goto again;
+ }
+
+ /*
+ * Now we have the new raw value and have updated the prev
+ * timestamp already. We can now calculate the elapsed delta
+ * (event-)time and add that to the generic event.
+ *
+ * Careful, not all hw sign-extends above the physical width
+ * of the count.
+ */
+ delta = (new_raw_count << shift) - (prev_raw_count << shift);
+ delta >>= shift;
+
+ sdelta = rapl_scale(delta);
+
+ local64_add(sdelta, &event->count);
+
+ return new_raw_count;
+}
+
+static void rapl_start_hrtimer(struct rapl_pmu *pmu)
+{
+ __hrtimer_start_range_ns(&pmu->hrtimer,
+ pmu->timer_interval, 0,
+ HRTIMER_MODE_REL_PINNED, 0);
+}
+
+static void rapl_stop_hrtimer(struct rapl_pmu *pmu)
+{
+ hrtimer_cancel(&pmu->hrtimer);
+}
+
+static enum hrtimer_restart rapl_hrtimer_handle(struct hrtimer *hrtimer)
+{
+ struct rapl_pmu *pmu = __get_cpu_var(rapl_pmu);
+ struct perf_event *event;
+ unsigned long flags;
+
+ if (!pmu->n_active)
+ return HRTIMER_NORESTART;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+
+ list_for_each_entry(event, &pmu->active_list, active_entry) {
+ rapl_event_update(event);
+ }
+
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ hrtimer_forward_now(hrtimer, pmu->timer_interval);
+
+ return HRTIMER_RESTART;
+}
+
+static void rapl_hrtimer_init(struct rapl_pmu *pmu)
+{
+ struct hrtimer *hr = &pmu->hrtimer;
+
+ hrtimer_init(hr, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ hr->function = rapl_hrtimer_handle;
+}
+
+static void __rapl_pmu_event_start(struct rapl_pmu *pmu,
+ struct perf_event *event)
+{
+ if (WARN_ON_ONCE(!(event->hw.state & PERF_HES_STOPPED)))
+ return;
+
+ event->hw.state = 0;
+
+ list_add_tail(&event->active_entry, &pmu->active_list);
+
+ local64_set(&event->hw.prev_count, rapl_read_counter(event));
+
+ pmu->n_active++;
+ if (pmu->n_active == 1)
+ rapl_start_hrtimer(pmu);
+}
+
+static void rapl_pmu_event_start(struct perf_event *event, int mode)
+{
+ struct rapl_pmu *pmu = __get_cpu_var(rapl_pmu);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ __rapl_pmu_event_start(pmu, event);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+static void rapl_pmu_event_stop(struct perf_event *event, int mode)
+{
+ struct rapl_pmu *pmu = __get_cpu_var(rapl_pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+
+ /* mark event as deactivated and stopped */
+ if (!(hwc->state & PERF_HES_STOPPED)) {
+ WARN_ON_ONCE(pmu->n_active <= 0);
+ pmu->n_active--;
+ if (pmu->n_active == 0)
+ rapl_stop_hrtimer(pmu);
+
+ list_del(&event->active_entry);
+
+ WARN_ON_ONCE(hwc->state & PERF_HES_STOPPED);
+ hwc->state |= PERF_HES_STOPPED;
+ }
+
+ /* check if update of sw counter is necessary */
+ if ((mode & PERF_EF_UPDATE) && !(hwc->state & PERF_HES_UPTODATE)) {
+ /*
+ * Drain the remaining delta count out of a event
+ * that we are disabling:
+ */
+ rapl_event_update(event);
+ hwc->state |= PERF_HES_UPTODATE;
+ }
+
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+static int rapl_pmu_event_add(struct perf_event *event, int mode)
+{
+ struct rapl_pmu *pmu = __get_cpu_var(rapl_pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+
+ hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED;
+
+ if (mode & PERF_EF_START)
+ __rapl_pmu_event_start(pmu, event);
+
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ return 0;
+}
+
+static void rapl_pmu_event_del(struct perf_event *event, int flags)
+{
+ rapl_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int rapl_pmu_event_init(struct perf_event *event)
+{
+ u64 cfg = event->attr.config & RAPL_EVENT_MASK;
+ int bit, msr, ret = 0;
+
+ /* only look at RAPL events */
+ if (event->attr.type != rapl_pmu_class.type)
+ return -ENOENT;
+
+ /* check only supported bits are set */
+ if (event->attr.config & ~RAPL_EVENT_MASK)
+ return -EINVAL;
+
+ /*
+ * check event is known (determines counter)
+ */
+ switch (cfg) {
+ case INTEL_RAPL_PP0:
+ bit = RAPL_IDX_PP0_NRG_STAT;
+ msr = MSR_PP0_ENERGY_STATUS;
+ break;
+ case INTEL_RAPL_PKG:
+ bit = RAPL_IDX_PKG_NRG_STAT;
+ msr = MSR_PKG_ENERGY_STATUS;
+ break;
+ case INTEL_RAPL_RAM:
+ bit = RAPL_IDX_RAM_NRG_STAT;
+ msr = MSR_DRAM_ENERGY_STATUS;
+ break;
+ case INTEL_RAPL_PP1:
+ bit = RAPL_IDX_PP1_NRG_STAT;
+ msr = MSR_PP1_ENERGY_STATUS;
+ break;
+ default:
+ return -EINVAL;
+ }
+ /* check event supported */
+ if (!(rapl_cntr_mask & (1 << bit)))
+ return -EINVAL;
+
+ /* unsupported modes and filters */
+ if (event->attr.exclude_user ||
+ event->attr.exclude_kernel ||
+ event->attr.exclude_hv ||
+ event->attr.exclude_idle ||
+ event->attr.exclude_host ||
+ event->attr.exclude_guest ||
+ event->attr.sample_period) /* no sampling */
+ return -EINVAL;
+
+ /* must be done before validate_group */
+ event->hw.event_base = msr;
+ event->hw.config = cfg;
+ event->hw.idx = bit;
+
+ return ret;
+}
+
+static void rapl_pmu_event_read(struct perf_event *event)
+{
+ rapl_event_update(event);
+}
+
+static ssize_t rapl_get_attr_cpumask(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int n = cpulist_scnprintf(buf, PAGE_SIZE - 2, &rapl_cpu_mask);
+
+ buf[n++] = '\n';
+ buf[n] = '\0';
+ return n;
+}
+
+static DEVICE_ATTR(cpumask, S_IRUGO, rapl_get_attr_cpumask, NULL);
+
+static struct attribute *rapl_pmu_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static struct attribute_group rapl_pmu_attr_group = {
+ .attrs = rapl_pmu_attrs,
+};
+
+EVENT_ATTR_STR(energy-cores, rapl_cores, "event=0x01");
+EVENT_ATTR_STR(energy-pkg , rapl_pkg, "event=0x02");
+EVENT_ATTR_STR(energy-ram , rapl_ram, "event=0x03");
+EVENT_ATTR_STR(energy-gpu , rapl_gpu, "event=0x04");
+
+EVENT_ATTR_STR(energy-cores.unit, rapl_cores_unit, "Joules");
+EVENT_ATTR_STR(energy-pkg.unit , rapl_pkg_unit, "Joules");
+EVENT_ATTR_STR(energy-ram.unit , rapl_ram_unit, "Joules");
+EVENT_ATTR_STR(energy-gpu.unit , rapl_gpu_unit, "Joules");
+
+/*
+ * we compute in 0.23 nJ increments regardless of MSR
+ */
+EVENT_ATTR_STR(energy-cores.scale, rapl_cores_scale, "2.3283064365386962890625e-10");
+EVENT_ATTR_STR(energy-pkg.scale, rapl_pkg_scale, "2.3283064365386962890625e-10");
+EVENT_ATTR_STR(energy-ram.scale, rapl_ram_scale, "2.3283064365386962890625e-10");
+EVENT_ATTR_STR(energy-gpu.scale, rapl_gpu_scale, "2.3283064365386962890625e-10");
+
+static struct attribute *rapl_events_srv_attr[] = {
+ EVENT_PTR(rapl_cores),
+ EVENT_PTR(rapl_pkg),
+ EVENT_PTR(rapl_ram),
+
+ EVENT_PTR(rapl_cores_unit),
+ EVENT_PTR(rapl_pkg_unit),
+ EVENT_PTR(rapl_ram_unit),
+
+ EVENT_PTR(rapl_cores_scale),
+ EVENT_PTR(rapl_pkg_scale),
+ EVENT_PTR(rapl_ram_scale),
+ NULL,
+};
+
+static struct attribute *rapl_events_cln_attr[] = {
+ EVENT_PTR(rapl_cores),
+ EVENT_PTR(rapl_pkg),
+ EVENT_PTR(rapl_gpu),
+
+ EVENT_PTR(rapl_cores_unit),
+ EVENT_PTR(rapl_pkg_unit),
+ EVENT_PTR(rapl_gpu_unit),
+
+ EVENT_PTR(rapl_cores_scale),
+ EVENT_PTR(rapl_pkg_scale),
+ EVENT_PTR(rapl_gpu_scale),
+ NULL,
+};
+
+static struct attribute_group rapl_pmu_events_group = {
+ .name = "events",
+ .attrs = NULL, /* patched at runtime */
+};
+
+DEFINE_RAPL_FORMAT_ATTR(event, event, "config:0-7");
+static struct attribute *rapl_formats_attr[] = {
+ &format_attr_event.attr,
+ NULL,
+};
+
+static struct attribute_group rapl_pmu_format_group = {
+ .name = "format",
+ .attrs = rapl_formats_attr,
+};
+
+const struct attribute_group *rapl_attr_groups[] = {
+ &rapl_pmu_attr_group,
+ &rapl_pmu_format_group,
+ &rapl_pmu_events_group,
+ NULL,
+};
+
+static struct pmu rapl_pmu_class = {
+ .attr_groups = rapl_attr_groups,
+ .task_ctx_nr = perf_invalid_context, /* system-wide only */
+ .event_init = rapl_pmu_event_init,
+ .add = rapl_pmu_event_add, /* must have */
+ .del = rapl_pmu_event_del, /* must have */
+ .start = rapl_pmu_event_start,
+ .stop = rapl_pmu_event_stop,
+ .read = rapl_pmu_event_read,
+};
+
+static void rapl_cpu_exit(int cpu)
+{
+ struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
+ int i, phys_id = topology_physical_package_id(cpu);
+ int target = -1;
+
+ /* find a new cpu on same package */
+ for_each_online_cpu(i) {
+ if (i == cpu)
+ continue;
+ if (phys_id == topology_physical_package_id(i)) {
+ target = i;
+ break;
+ }
+ }
+ /*
+ * clear cpu from cpumask
+ * if was set in cpumask and still some cpu on package,
+ * then move to new cpu
+ */
+ if (cpumask_test_and_clear_cpu(cpu, &rapl_cpu_mask) && target >= 0)
+ cpumask_set_cpu(target, &rapl_cpu_mask);
+
+ WARN_ON(cpumask_empty(&rapl_cpu_mask));
+ /*
+ * migrate events and context to new cpu
+ */
+ if (target >= 0)
+ perf_pmu_migrate_context(pmu->pmu, cpu, target);
+
+ /* cancel overflow polling timer for CPU */
+ rapl_stop_hrtimer(pmu);
+}
+
+static void rapl_cpu_init(int cpu)
+{
+ int i, phys_id = topology_physical_package_id(cpu);
+
+ /* check if phys_is is already covered */
+ for_each_cpu(i, &rapl_cpu_mask) {
+ if (phys_id == topology_physical_package_id(i))
+ return;
+ }
+ /* was not found, so add it */
+ cpumask_set_cpu(cpu, &rapl_cpu_mask);
+}
+
+static int rapl_cpu_prepare(int cpu)
+{
+ struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
+ int phys_id = topology_physical_package_id(cpu);
+ u64 ms;
+
+ if (pmu)
+ return 0;
+
+ if (phys_id < 0)
+ return -1;
+
+ pmu = kzalloc_node(sizeof(*pmu), GFP_KERNEL, cpu_to_node(cpu));
+ if (!pmu)
+ return -1;
+
+ spin_lock_init(&pmu->lock);
+
+ INIT_LIST_HEAD(&pmu->active_list);
+
+ /*
+ * grab power unit as: 1/2^unit Joules
+ *
+ * we cache in local PMU instance
+ */
+ rdmsrl(MSR_RAPL_POWER_UNIT, pmu->hw_unit);
+ pmu->hw_unit = (pmu->hw_unit >> 8) & 0x1FULL;
+ pmu->pmu = &rapl_pmu_class;
+
+ /*
+ * use reference of 200W for scaling the timeout
+ * to avoid missing counter overflows.
+ * 200W = 200 Joules/sec
+ * divide interval by 2 to avoid lockstep (2 * 100)
+ * if hw unit is 32, then we use 2 ms 1/200/2
+ */
+ if (pmu->hw_unit < 32)
+ ms = (1000 / (2 * 100)) * (1ULL << (32 - pmu->hw_unit - 1));
+ else
+ ms = 2;
+
+ pmu->timer_interval = ms_to_ktime(ms);
+
+ rapl_hrtimer_init(pmu);
+
+ /* set RAPL pmu for this cpu for now */
+ per_cpu(rapl_pmu, cpu) = pmu;
+ per_cpu(rapl_pmu_to_free, cpu) = NULL;
+
+ return 0;
+}
+
+static void rapl_cpu_kfree(int cpu)
+{
+ struct rapl_pmu *pmu = per_cpu(rapl_pmu_to_free, cpu);
+
+ kfree(pmu);
+
+ per_cpu(rapl_pmu_to_free, cpu) = NULL;
+}
+
+static int rapl_cpu_dying(int cpu)
+{
+ struct rapl_pmu *pmu = per_cpu(rapl_pmu, cpu);
+
+ if (!pmu)
+ return 0;
+
+ per_cpu(rapl_pmu, cpu) = NULL;
+
+ per_cpu(rapl_pmu_to_free, cpu) = pmu;
+
+ return 0;
+}
+
+static int rapl_cpu_notifier(struct notifier_block *self,
+ unsigned long action, void *hcpu)
+{
+ unsigned int cpu = (long)hcpu;
+
+ switch (action & ~CPU_TASKS_FROZEN) {
+ case CPU_UP_PREPARE:
+ rapl_cpu_prepare(cpu);
+ break;
+ case CPU_STARTING:
+ rapl_cpu_init(cpu);
+ break;
+ case CPU_UP_CANCELED:
+ case CPU_DYING:
+ rapl_cpu_dying(cpu);
+ break;
+ case CPU_ONLINE:
+ case CPU_DEAD:
+ rapl_cpu_kfree(cpu);
+ break;
+ case CPU_DOWN_PREPARE:
+ rapl_cpu_exit(cpu);
+ break;
+ default:
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static const struct x86_cpu_id rapl_cpu_match[] = {
+ [0] = { .vendor = X86_VENDOR_INTEL, .family = 6 },
+ [1] = {},
+};
+
+static int __init rapl_pmu_init(void)
+{
+ struct rapl_pmu *pmu;
+ int cpu, ret;
+
+ /*
+ * check for Intel processor family 6
+ */
+ if (!x86_match_cpu(rapl_cpu_match))
+ return 0;
+
+ /* check supported CPU */
+ switch (boot_cpu_data.x86_model) {
+ case 42: /* Sandy Bridge */
+ case 58: /* Ivy Bridge */
+ case 60: /* Haswell */
+ case 69: /* Haswell-Celeron */
+ rapl_cntr_mask = RAPL_IDX_CLN;
+ rapl_pmu_events_group.attrs = rapl_events_cln_attr;
+ break;
+ case 45: /* Sandy Bridge-EP */
+ case 62: /* IvyTown */
+ rapl_cntr_mask = RAPL_IDX_SRV;
+ rapl_pmu_events_group.attrs = rapl_events_srv_attr;
+ break;
+
+ default:
+ /* unsupported */
+ return 0;
+ }
+ get_online_cpus();
+
+ for_each_online_cpu(cpu) {
+ rapl_cpu_prepare(cpu);
+ rapl_cpu_init(cpu);
+ }
+
+ perf_cpu_notifier(rapl_cpu_notifier);
+
+ ret = perf_pmu_register(&rapl_pmu_class, "power", -1);
+ if (WARN_ON(ret)) {
+ pr_info("RAPL PMU detected, registration failed (%d), RAPL PMU disabled\n", ret);
+ put_online_cpus();
+ return -1;
+ }
+
+ pmu = __get_cpu_var(rapl_pmu);
+
+ pr_info("RAPL PMU detected, hw unit 2^-%d Joules,"
+ " API unit is 2^-32 Joules,"
+ " %d fixed counters"
+ " %llu ms ovfl timer\n",
+ pmu->hw_unit,
+ hweight32(rapl_cntr_mask),
+ ktime_to_ms(pmu->timer_interval));
+
+ put_online_cpus();
+
+ return 0;
+}
+device_initcall(rapl_pmu_init);
}
__setup("nordrand", x86_rdrand_setup);
-/* We can't use arch_get_random_long() here since alternatives haven't run */
-static inline int rdrand_long(unsigned long *v)
-{
- int ok;
- asm volatile("1: " RDRAND_LONG "\n\t"
- "jc 2f\n\t"
- "decl %0\n\t"
- "jnz 1b\n\t"
- "2:"
- : "=r" (ok), "=a" (*v)
- : "0" (RDRAND_RETRY_LOOPS));
- return ok;
-}
-
/*
* Force a reseed cycle; we are architecturally guaranteed a reseed
* after no more than 512 128-bit chunks of random data. This also
#include <linux/kernel.h>
#include <linux/mm.h>
-#include <linux/init.h>
#include <asm/processor.h>
#include <asm/msr.h>
#include "cpu.h"
#include <linux/kernel.h>
-#include <linux/init.h>
#include <asm/processor.h>
#include "cpu.h"
*
*/
-#include <linux/init.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/smp.h>
#include <linux/mm.h>
#include <linux/sched.h>
-#include <linux/init.h>
#include <linux/init_task.h>
#include <linux/fs.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/sched.h>
-#include <linux/init.h>
#include <linux/smp.h>
#include <asm/hw_breakpoint.h>
--- /dev/null
+/*
+ * IOSF-SB MailBox Interface Driver
+ * Copyright (c) 2013, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ *
+ * The IOSF-SB is a fabric bus available on Atom based SOC's that uses a
+ * mailbox interface (MBI) to communicate with mutiple devices. This
+ * driver implements access to this interface for those platforms that can
+ * enumerate the device using PCI.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/spinlock.h>
+#include <linux/pci.h>
+
+#include <asm/iosf_mbi.h>
+
+static DEFINE_SPINLOCK(iosf_mbi_lock);
+
+static inline u32 iosf_mbi_form_mcr(u8 op, u8 port, u8 offset)
+{
+ return (op << 24) | (port << 16) | (offset << 8) | MBI_ENABLE;
+}
+
+static struct pci_dev *mbi_pdev; /* one mbi device */
+
+static int iosf_mbi_pci_read_mdr(u32 mcrx, u32 mcr, u32 *mdr)
+{
+ int result;
+
+ if (!mbi_pdev)
+ return -ENODEV;
+
+ if (mcrx) {
+ result = pci_write_config_dword(mbi_pdev, MBI_MCRX_OFFSET,
+ mcrx);
+ if (result < 0)
+ goto fail_read;
+ }
+
+ result = pci_write_config_dword(mbi_pdev, MBI_MCR_OFFSET, mcr);
+ if (result < 0)
+ goto fail_read;
+
+ result = pci_read_config_dword(mbi_pdev, MBI_MDR_OFFSET, mdr);
+ if (result < 0)
+ goto fail_read;
+
+ return 0;
+
+fail_read:
+ dev_err(&mbi_pdev->dev, "PCI config access failed with %d\n", result);
+ return result;
+}
+
+static int iosf_mbi_pci_write_mdr(u32 mcrx, u32 mcr, u32 mdr)
+{
+ int result;
+
+ if (!mbi_pdev)
+ return -ENODEV;
+
+ result = pci_write_config_dword(mbi_pdev, MBI_MDR_OFFSET, mdr);
+ if (result < 0)
+ goto fail_write;
+
+ if (mcrx) {
+ result = pci_write_config_dword(mbi_pdev, MBI_MCRX_OFFSET,
+ mcrx);
+ if (result < 0)
+ goto fail_write;
+ }
+
+ result = pci_write_config_dword(mbi_pdev, MBI_MCR_OFFSET, mcr);
+ if (result < 0)
+ goto fail_write;
+
+ return 0;
+
+fail_write:
+ dev_err(&mbi_pdev->dev, "PCI config access failed with %d\n", result);
+ return result;
+}
+
+int iosf_mbi_read(u8 port, u8 opcode, u32 offset, u32 *mdr)
+{
+ u32 mcr, mcrx;
+ unsigned long flags;
+ int ret;
+
+ /*Access to the GFX unit is handled by GPU code */
+ if (port == BT_MBI_UNIT_GFX) {
+ WARN_ON(1);
+ return -EPERM;
+ }
+
+ mcr = iosf_mbi_form_mcr(opcode, port, offset & MBI_MASK_LO);
+ mcrx = offset & MBI_MASK_HI;
+
+ spin_lock_irqsave(&iosf_mbi_lock, flags);
+ ret = iosf_mbi_pci_read_mdr(mcrx, mcr, mdr);
+ spin_unlock_irqrestore(&iosf_mbi_lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL(iosf_mbi_read);
+
+int iosf_mbi_write(u8 port, u8 opcode, u32 offset, u32 mdr)
+{
+ u32 mcr, mcrx;
+ unsigned long flags;
+ int ret;
+
+ /*Access to the GFX unit is handled by GPU code */
+ if (port == BT_MBI_UNIT_GFX) {
+ WARN_ON(1);
+ return -EPERM;
+ }
+
+ mcr = iosf_mbi_form_mcr(opcode, port, offset & MBI_MASK_LO);
+ mcrx = offset & MBI_MASK_HI;
+
+ spin_lock_irqsave(&iosf_mbi_lock, flags);
+ ret = iosf_mbi_pci_write_mdr(mcrx, mcr, mdr);
+ spin_unlock_irqrestore(&iosf_mbi_lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL(iosf_mbi_write);
+
+int iosf_mbi_modify(u8 port, u8 opcode, u32 offset, u32 mdr, u32 mask)
+{
+ u32 mcr, mcrx;
+ u32 value;
+ unsigned long flags;
+ int ret;
+
+ /*Access to the GFX unit is handled by GPU code */
+ if (port == BT_MBI_UNIT_GFX) {
+ WARN_ON(1);
+ return -EPERM;
+ }
+
+ mcr = iosf_mbi_form_mcr(opcode, port, offset & MBI_MASK_LO);
+ mcrx = offset & MBI_MASK_HI;
+
+ spin_lock_irqsave(&iosf_mbi_lock, flags);
+
+ /* Read current mdr value */
+ ret = iosf_mbi_pci_read_mdr(mcrx, mcr & MBI_RD_MASK, &value);
+ if (ret < 0) {
+ spin_unlock_irqrestore(&iosf_mbi_lock, flags);
+ return ret;
+ }
+
+ /* Apply mask */
+ value &= ~mask;
+ mdr &= mask;
+ value |= mdr;
+
+ /* Write back */
+ ret = iosf_mbi_pci_write_mdr(mcrx, mcr | MBI_WR_MASK, value);
+
+ spin_unlock_irqrestore(&iosf_mbi_lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL(iosf_mbi_modify);
+
+static int iosf_mbi_probe(struct pci_dev *pdev,
+ const struct pci_device_id *unused)
+{
+ int ret;
+
+ ret = pci_enable_device(pdev);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "error: could not enable device\n");
+ return ret;
+ }
+
+ mbi_pdev = pci_dev_get(pdev);
+ return 0;
+}
+
+static DEFINE_PCI_DEVICE_TABLE(iosf_mbi_pci_ids) = {
+ { PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x0F00) },
+ { 0, },
+};
+MODULE_DEVICE_TABLE(pci, iosf_mbi_pci_ids);
+
+static struct pci_driver iosf_mbi_pci_driver = {
+ .name = "iosf_mbi_pci",
+ .probe = iosf_mbi_probe,
+ .id_table = iosf_mbi_pci_ids,
+};
+
+static int __init iosf_mbi_init(void)
+{
+ return pci_register_driver(&iosf_mbi_pci_driver);
+}
+
+static void __exit iosf_mbi_exit(void)
+{
+ pci_unregister_driver(&iosf_mbi_pci_driver);
+ if (mbi_pdev) {
+ pci_dev_put(mbi_pdev);
+ mbi_pdev = NULL;
+ }
+}
+
+module_init(iosf_mbi_init);
+module_exit(iosf_mbi_exit);
+
+MODULE_AUTHOR("David E. Box <david.e.box@linux.intel.com>");
+MODULE_DESCRIPTION("IOSF Mailbox Interface accessor");
+MODULE_LICENSE("GPL v2");
if (!handle_irq(irq, regs)) {
ack_APIC_irq();
- if (printk_ratelimit())
- pr_emerg("%s: %d.%d No irq handler for vector (irq %d)\n",
- __func__, smp_processor_id(), vector, irq);
+ if (irq != VECTOR_RETRIGGERED) {
+ pr_emerg_ratelimited("%s: %d.%d No irq handler for vector (irq %d)\n",
+ __func__, smp_processor_id(),
+ vector, irq);
+ } else {
+ __this_cpu_write(vector_irq[vector], VECTOR_UNDEFINED);
+ }
}
irq_exit();
for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS; vector++) {
unsigned int irr;
- if (__this_cpu_read(vector_irq[vector]) < 0)
+ if (__this_cpu_read(vector_irq[vector]) <= VECTOR_UNDEFINED)
continue;
irr = apic_read(APIC_IRR + (vector / 32 * 0x10));
data = irq_desc_get_irq_data(desc);
chip = irq_data_get_irq_chip(data);
raw_spin_lock(&desc->lock);
- if (chip->irq_retrigger)
+ if (chip->irq_retrigger) {
chip->irq_retrigger(data);
+ __this_cpu_write(vector_irq[vector], VECTOR_RETRIGGERED);
+ }
raw_spin_unlock(&desc->lock);
}
- __this_cpu_write(vector_irq[vector], -1);
+ if (__this_cpu_read(vector_irq[vector]) != VECTOR_RETRIGGERED)
+ __this_cpu_write(vector_irq[vector], VECTOR_UNDEFINED);
}
}
#endif
};
DEFINE_PER_CPU(vector_irq_t, vector_irq) = {
- [0 ... NR_VECTORS - 1] = -1,
+ [0 ... NR_VECTORS - 1] = VECTOR_UNDEFINED,
};
int vector_used_by_percpu_irq(unsigned int vector)
int cpu;
for_each_online_cpu(cpu) {
- if (per_cpu(vector_irq, cpu)[vector] != -1)
+ if (per_cpu(vector_irq, cpu)[vector] > VECTOR_UNDEFINED)
return 1;
}
#include <linux/sched.h>
#include <linux/delay.h>
#include <linux/kgdb.h>
-#include <linux/init.h>
#include <linux/smp.h>
#include <linux/nmi.h>
#include <linux/hw_breakpoint.h>
--- /dev/null
+/*
+ * Architecture specific sysfs attributes in /sys/kernel
+ *
+ * Copyright (C) 2007, Intel Corp.
+ * Huang Ying <ying.huang@intel.com>
+ * Copyright (C) 2013, 2013 Red Hat, Inc.
+ * Dave Young <dyoung@redhat.com>
+ *
+ * This file is released under the GPLv2
+ */
+
+#include <linux/kobject.h>
+#include <linux/string.h>
+#include <linux/sysfs.h>
+#include <linux/init.h>
+#include <linux/stat.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+
+#include <asm/io.h>
+#include <asm/setup.h>
+
+static ssize_t version_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return sprintf(buf, "0x%04x\n", boot_params.hdr.version);
+}
+
+static struct kobj_attribute boot_params_version_attr = __ATTR_RO(version);
+
+static ssize_t boot_params_data_read(struct file *fp, struct kobject *kobj,
+ struct bin_attribute *bin_attr,
+ char *buf, loff_t off, size_t count)
+{
+ memcpy(buf, (void *)&boot_params + off, count);
+ return count;
+}
+
+static struct bin_attribute boot_params_data_attr = {
+ .attr = {
+ .name = "data",
+ .mode = S_IRUGO,
+ },
+ .read = boot_params_data_read,
+ .size = sizeof(boot_params),
+};
+
+static struct attribute *boot_params_version_attrs[] = {
+ &boot_params_version_attr.attr,
+ NULL,
+};
+
+static struct bin_attribute *boot_params_data_attrs[] = {
+ &boot_params_data_attr,
+ NULL,
+};
+
+static struct attribute_group boot_params_attr_group = {
+ .attrs = boot_params_version_attrs,
+ .bin_attrs = boot_params_data_attrs,
+};
+
+static int kobj_to_setup_data_nr(struct kobject *kobj, int *nr)
+{
+ const char *name;
+
+ name = kobject_name(kobj);
+ return kstrtoint(name, 10, nr);
+}
+
+static int get_setup_data_paddr(int nr, u64 *paddr)
+{
+ int i = 0;
+ struct setup_data *data;
+ u64 pa_data = boot_params.hdr.setup_data;
+
+ while (pa_data) {
+ if (nr == i) {
+ *paddr = pa_data;
+ return 0;
+ }
+ data = ioremap_cache(pa_data, sizeof(*data));
+ if (!data)
+ return -ENOMEM;
+
+ pa_data = data->next;
+ iounmap(data);
+ i++;
+ }
+ return -EINVAL;
+}
+
+static int __init get_setup_data_size(int nr, size_t *size)
+{
+ int i = 0;
+ struct setup_data *data;
+ u64 pa_data = boot_params.hdr.setup_data;
+
+ while (pa_data) {
+ data = ioremap_cache(pa_data, sizeof(*data));
+ if (!data)
+ return -ENOMEM;
+ if (nr == i) {
+ *size = data->len;
+ iounmap(data);
+ return 0;
+ }
+
+ pa_data = data->next;
+ iounmap(data);
+ i++;
+ }
+ return -EINVAL;
+}
+
+static ssize_t type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int nr, ret;
+ u64 paddr;
+ struct setup_data *data;
+
+ ret = kobj_to_setup_data_nr(kobj, &nr);
+ if (ret)
+ return ret;
+
+ ret = get_setup_data_paddr(nr, &paddr);
+ if (ret)
+ return ret;
+ data = ioremap_cache(paddr, sizeof(*data));
+ if (!data)
+ return -ENOMEM;
+
+ ret = sprintf(buf, "0x%x\n", data->type);
+ iounmap(data);
+ return ret;
+}
+
+static ssize_t setup_data_data_read(struct file *fp,
+ struct kobject *kobj,
+ struct bin_attribute *bin_attr,
+ char *buf,
+ loff_t off, size_t count)
+{
+ int nr, ret = 0;
+ u64 paddr;
+ struct setup_data *data;
+ void *p;
+
+ ret = kobj_to_setup_data_nr(kobj, &nr);
+ if (ret)
+ return ret;
+
+ ret = get_setup_data_paddr(nr, &paddr);
+ if (ret)
+ return ret;
+ data = ioremap_cache(paddr, sizeof(*data));
+ if (!data)
+ return -ENOMEM;
+
+ if (off > data->len) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (count > data->len - off)
+ count = data->len - off;
+
+ if (!count)
+ goto out;
+
+ ret = count;
+ p = ioremap_cache(paddr + sizeof(*data), data->len);
+ if (!p) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ memcpy(buf, p + off, count);
+ iounmap(p);
+out:
+ iounmap(data);
+ return ret;
+}
+
+static struct kobj_attribute type_attr = __ATTR_RO(type);
+
+static struct bin_attribute data_attr = {
+ .attr = {
+ .name = "data",
+ .mode = S_IRUGO,
+ },
+ .read = setup_data_data_read,
+};
+
+static struct attribute *setup_data_type_attrs[] = {
+ &type_attr.attr,
+ NULL,
+};
+
+static struct bin_attribute *setup_data_data_attrs[] = {
+ &data_attr,
+ NULL,
+};
+
+static struct attribute_group setup_data_attr_group = {
+ .attrs = setup_data_type_attrs,
+ .bin_attrs = setup_data_data_attrs,
+};
+
+static int __init create_setup_data_node(struct kobject *parent,
+ struct kobject **kobjp, int nr)
+{
+ int ret = 0;
+ size_t size;
+ struct kobject *kobj;
+ char name[16]; /* should be enough for setup_data nodes numbers */
+ snprintf(name, 16, "%d", nr);
+
+ kobj = kobject_create_and_add(name, parent);
+ if (!kobj)
+ return -ENOMEM;
+
+ ret = get_setup_data_size(nr, &size);
+ if (ret)
+ goto out_kobj;
+
+ data_attr.size = size;
+ ret = sysfs_create_group(kobj, &setup_data_attr_group);
+ if (ret)
+ goto out_kobj;
+ *kobjp = kobj;
+
+ return 0;
+out_kobj:
+ kobject_put(kobj);
+ return ret;
+}
+
+static void __init cleanup_setup_data_node(struct kobject *kobj)
+{
+ sysfs_remove_group(kobj, &setup_data_attr_group);
+ kobject_put(kobj);
+}
+
+static int __init get_setup_data_total_num(u64 pa_data, int *nr)
+{
+ int ret = 0;
+ struct setup_data *data;
+
+ *nr = 0;
+ while (pa_data) {
+ *nr += 1;
+ data = ioremap_cache(pa_data, sizeof(*data));
+ if (!data) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ pa_data = data->next;
+ iounmap(data);
+ }
+
+out:
+ return ret;
+}
+
+static int __init create_setup_data_nodes(struct kobject *parent)
+{
+ struct kobject *setup_data_kobj, **kobjp;
+ u64 pa_data;
+ int i, j, nr, ret = 0;
+
+ pa_data = boot_params.hdr.setup_data;
+ if (!pa_data)
+ return 0;
+
+ setup_data_kobj = kobject_create_and_add("setup_data", parent);
+ if (!setup_data_kobj) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ret = get_setup_data_total_num(pa_data, &nr);
+ if (ret)
+ goto out_setup_data_kobj;
+
+ kobjp = kmalloc(sizeof(*kobjp) * nr, GFP_KERNEL);
+ if (!kobjp) {
+ ret = -ENOMEM;
+ goto out_setup_data_kobj;
+ }
+
+ for (i = 0; i < nr; i++) {
+ ret = create_setup_data_node(setup_data_kobj, kobjp + i, i);
+ if (ret)
+ goto out_clean_nodes;
+ }
+
+ kfree(kobjp);
+ return 0;
+
+out_clean_nodes:
+ for (j = i - 1; j > 0; j--)
+ cleanup_setup_data_node(*(kobjp + j));
+ kfree(kobjp);
+out_setup_data_kobj:
+ kobject_put(setup_data_kobj);
+out:
+ return ret;
+}
+
+static int __init boot_params_ksysfs_init(void)
+{
+ int ret;
+ struct kobject *boot_params_kobj;
+
+ boot_params_kobj = kobject_create_and_add("boot_params",
+ kernel_kobj);
+ if (!boot_params_kobj) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ret = sysfs_create_group(boot_params_kobj, &boot_params_attr_group);
+ if (ret)
+ goto out_boot_params_kobj;
+
+ ret = create_setup_data_nodes(boot_params_kobj);
+ if (ret)
+ goto out_create_group;
+
+ return 0;
+out_create_group:
+ sysfs_remove_group(boot_params_kobj, &boot_params_attr_group);
+out_boot_params_kobj:
+ kobject_put(boot_params_kobj);
+out:
+ return ret;
+}
+
+arch_initcall(boot_params_ksysfs_init);
#include <linux/mm.h>
#include <linux/kexec.h>
#include <linux/delay.h>
-#include <linux/init.h>
#include <linux/numa.h>
#include <linux/ftrace.h>
#include <linux/suspend.h>
+++ /dev/null
-/*
- * AMD CPU Microcode Update Driver for Linux
- * Copyright (C) 2008-2011 Advanced Micro Devices Inc.
- *
- * Author: Peter Oruba <peter.oruba@amd.com>
- *
- * Based on work by:
- * Tigran Aivazian <tigran@aivazian.fsnet.co.uk>
- *
- * Maintainers:
- * Andreas Herrmann <herrmann.der.user@googlemail.com>
- * Borislav Petkov <bp@alien8.de>
- *
- * This driver allows to upgrade microcode on F10h AMD
- * CPUs and later.
- *
- * Licensed under the terms of the GNU General Public
- * License version 2. See file COPYING for details.
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/firmware.h>
-#include <linux/pci_ids.h>
-#include <linux/uaccess.h>
-#include <linux/vmalloc.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/pci.h>
-
-#include <asm/microcode.h>
-#include <asm/processor.h>
-#include <asm/msr.h>
-#include <asm/microcode_amd.h>
-
-MODULE_DESCRIPTION("AMD Microcode Update Driver");
-MODULE_AUTHOR("Peter Oruba");
-MODULE_LICENSE("GPL v2");
-
-static struct equiv_cpu_entry *equiv_cpu_table;
-
-struct ucode_patch {
- struct list_head plist;
- void *data;
- u32 patch_id;
- u16 equiv_cpu;
-};
-
-static LIST_HEAD(pcache);
-
-static u16 __find_equiv_id(unsigned int cpu)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- return find_equiv_id(equiv_cpu_table, uci->cpu_sig.sig);
-}
-
-static u32 find_cpu_family_by_equiv_cpu(u16 equiv_cpu)
-{
- int i = 0;
-
- BUG_ON(!equiv_cpu_table);
-
- while (equiv_cpu_table[i].equiv_cpu != 0) {
- if (equiv_cpu == equiv_cpu_table[i].equiv_cpu)
- return equiv_cpu_table[i].installed_cpu;
- i++;
- }
- return 0;
-}
-
-/*
- * a small, trivial cache of per-family ucode patches
- */
-static struct ucode_patch *cache_find_patch(u16 equiv_cpu)
-{
- struct ucode_patch *p;
-
- list_for_each_entry(p, &pcache, plist)
- if (p->equiv_cpu == equiv_cpu)
- return p;
- return NULL;
-}
-
-static void update_cache(struct ucode_patch *new_patch)
-{
- struct ucode_patch *p;
-
- list_for_each_entry(p, &pcache, plist) {
- if (p->equiv_cpu == new_patch->equiv_cpu) {
- if (p->patch_id >= new_patch->patch_id)
- /* we already have the latest patch */
- return;
-
- list_replace(&p->plist, &new_patch->plist);
- kfree(p->data);
- kfree(p);
- return;
- }
- }
- /* no patch found, add it */
- list_add_tail(&new_patch->plist, &pcache);
-}
-
-static void free_cache(void)
-{
- struct ucode_patch *p, *tmp;
-
- list_for_each_entry_safe(p, tmp, &pcache, plist) {
- __list_del(p->plist.prev, p->plist.next);
- kfree(p->data);
- kfree(p);
- }
-}
-
-static struct ucode_patch *find_patch(unsigned int cpu)
-{
- u16 equiv_id;
-
- equiv_id = __find_equiv_id(cpu);
- if (!equiv_id)
- return NULL;
-
- return cache_find_patch(equiv_id);
-}
-
-static int collect_cpu_info_amd(int cpu, struct cpu_signature *csig)
-{
- struct cpuinfo_x86 *c = &cpu_data(cpu);
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- struct ucode_patch *p;
-
- csig->sig = cpuid_eax(0x00000001);
- csig->rev = c->microcode;
-
- /*
- * a patch could have been loaded early, set uci->mc so that
- * mc_bp_resume() can call apply_microcode()
- */
- p = find_patch(cpu);
- if (p && (p->patch_id == csig->rev))
- uci->mc = p->data;
-
- pr_info("CPU%d: patch_level=0x%08x\n", cpu, csig->rev);
-
- return 0;
-}
-
-static unsigned int verify_patch_size(u8 family, u32 patch_size,
- unsigned int size)
-{
- u32 max_size;
-
-#define F1XH_MPB_MAX_SIZE 2048
-#define F14H_MPB_MAX_SIZE 1824
-#define F15H_MPB_MAX_SIZE 4096
-#define F16H_MPB_MAX_SIZE 3458
-
- switch (family) {
- case 0x14:
- max_size = F14H_MPB_MAX_SIZE;
- break;
- case 0x15:
- max_size = F15H_MPB_MAX_SIZE;
- break;
- case 0x16:
- max_size = F16H_MPB_MAX_SIZE;
- break;
- default:
- max_size = F1XH_MPB_MAX_SIZE;
- break;
- }
-
- if (patch_size > min_t(u32, size, max_size)) {
- pr_err("patch size mismatch\n");
- return 0;
- }
-
- return patch_size;
-}
-
-int __apply_microcode_amd(struct microcode_amd *mc_amd)
-{
- u32 rev, dummy;
-
- wrmsrl(MSR_AMD64_PATCH_LOADER, (u64)(long)&mc_amd->hdr.data_code);
-
- /* verify patch application was successful */
- rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy);
- if (rev != mc_amd->hdr.patch_id)
- return -1;
-
- return 0;
-}
-
-int apply_microcode_amd(int cpu)
-{
- struct cpuinfo_x86 *c = &cpu_data(cpu);
- struct microcode_amd *mc_amd;
- struct ucode_cpu_info *uci;
- struct ucode_patch *p;
- u32 rev, dummy;
-
- BUG_ON(raw_smp_processor_id() != cpu);
-
- uci = ucode_cpu_info + cpu;
-
- p = find_patch(cpu);
- if (!p)
- return 0;
-
- mc_amd = p->data;
- uci->mc = p->data;
-
- rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy);
-
- /* need to apply patch? */
- if (rev >= mc_amd->hdr.patch_id) {
- c->microcode = rev;
- uci->cpu_sig.rev = rev;
- return 0;
- }
-
- if (__apply_microcode_amd(mc_amd)) {
- pr_err("CPU%d: update failed for patch_level=0x%08x\n",
- cpu, mc_amd->hdr.patch_id);
- return -1;
- }
- pr_info("CPU%d: new patch_level=0x%08x\n", cpu,
- mc_amd->hdr.patch_id);
-
- uci->cpu_sig.rev = mc_amd->hdr.patch_id;
- c->microcode = mc_amd->hdr.patch_id;
-
- return 0;
-}
-
-static int install_equiv_cpu_table(const u8 *buf)
-{
- unsigned int *ibuf = (unsigned int *)buf;
- unsigned int type = ibuf[1];
- unsigned int size = ibuf[2];
-
- if (type != UCODE_EQUIV_CPU_TABLE_TYPE || !size) {
- pr_err("empty section/"
- "invalid type field in container file section header\n");
- return -EINVAL;
- }
-
- equiv_cpu_table = vmalloc(size);
- if (!equiv_cpu_table) {
- pr_err("failed to allocate equivalent CPU table\n");
- return -ENOMEM;
- }
-
- memcpy(equiv_cpu_table, buf + CONTAINER_HDR_SZ, size);
-
- /* add header length */
- return size + CONTAINER_HDR_SZ;
-}
-
-static void free_equiv_cpu_table(void)
-{
- vfree(equiv_cpu_table);
- equiv_cpu_table = NULL;
-}
-
-static void cleanup(void)
-{
- free_equiv_cpu_table();
- free_cache();
-}
-
-/*
- * We return the current size even if some of the checks failed so that
- * we can skip over the next patch. If we return a negative value, we
- * signal a grave error like a memory allocation has failed and the
- * driver cannot continue functioning normally. In such cases, we tear
- * down everything we've used up so far and exit.
- */
-static int verify_and_add_patch(u8 family, u8 *fw, unsigned int leftover)
-{
- struct microcode_header_amd *mc_hdr;
- struct ucode_patch *patch;
- unsigned int patch_size, crnt_size, ret;
- u32 proc_fam;
- u16 proc_id;
-
- patch_size = *(u32 *)(fw + 4);
- crnt_size = patch_size + SECTION_HDR_SIZE;
- mc_hdr = (struct microcode_header_amd *)(fw + SECTION_HDR_SIZE);
- proc_id = mc_hdr->processor_rev_id;
-
- proc_fam = find_cpu_family_by_equiv_cpu(proc_id);
- if (!proc_fam) {
- pr_err("No patch family for equiv ID: 0x%04x\n", proc_id);
- return crnt_size;
- }
-
- /* check if patch is for the current family */
- proc_fam = ((proc_fam >> 8) & 0xf) + ((proc_fam >> 20) & 0xff);
- if (proc_fam != family)
- return crnt_size;
-
- if (mc_hdr->nb_dev_id || mc_hdr->sb_dev_id) {
- pr_err("Patch-ID 0x%08x: chipset-specific code unsupported.\n",
- mc_hdr->patch_id);
- return crnt_size;
- }
-
- ret = verify_patch_size(family, patch_size, leftover);
- if (!ret) {
- pr_err("Patch-ID 0x%08x: size mismatch.\n", mc_hdr->patch_id);
- return crnt_size;
- }
-
- patch = kzalloc(sizeof(*patch), GFP_KERNEL);
- if (!patch) {
- pr_err("Patch allocation failure.\n");
- return -EINVAL;
- }
-
- patch->data = kzalloc(patch_size, GFP_KERNEL);
- if (!patch->data) {
- pr_err("Patch data allocation failure.\n");
- kfree(patch);
- return -EINVAL;
- }
-
- /* All looks ok, copy patch... */
- memcpy(patch->data, fw + SECTION_HDR_SIZE, patch_size);
- INIT_LIST_HEAD(&patch->plist);
- patch->patch_id = mc_hdr->patch_id;
- patch->equiv_cpu = proc_id;
-
- /* ... and add to cache. */
- update_cache(patch);
-
- return crnt_size;
-}
-
-static enum ucode_state __load_microcode_amd(u8 family, const u8 *data,
- size_t size)
-{
- enum ucode_state ret = UCODE_ERROR;
- unsigned int leftover;
- u8 *fw = (u8 *)data;
- int crnt_size = 0;
- int offset;
-
- offset = install_equiv_cpu_table(data);
- if (offset < 0) {
- pr_err("failed to create equivalent cpu table\n");
- return ret;
- }
- fw += offset;
- leftover = size - offset;
-
- if (*(u32 *)fw != UCODE_UCODE_TYPE) {
- pr_err("invalid type field in container file section header\n");
- free_equiv_cpu_table();
- return ret;
- }
-
- while (leftover) {
- crnt_size = verify_and_add_patch(family, fw, leftover);
- if (crnt_size < 0)
- return ret;
-
- fw += crnt_size;
- leftover -= crnt_size;
- }
-
- return UCODE_OK;
-}
-
-enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size)
-{
- enum ucode_state ret;
-
- /* free old equiv table */
- free_equiv_cpu_table();
-
- ret = __load_microcode_amd(family, data, size);
-
- if (ret != UCODE_OK)
- cleanup();
-
-#if defined(CONFIG_MICROCODE_AMD_EARLY) && defined(CONFIG_X86_32)
- /* save BSP's matching patch for early load */
- if (cpu_data(smp_processor_id()).cpu_index == boot_cpu_data.cpu_index) {
- struct ucode_patch *p = find_patch(smp_processor_id());
- if (p) {
- memset(amd_bsp_mpb, 0, MPB_MAX_SIZE);
- memcpy(amd_bsp_mpb, p->data, min_t(u32, ksize(p->data),
- MPB_MAX_SIZE));
- }
- }
-#endif
- return ret;
-}
-
-/*
- * AMD microcode firmware naming convention, up to family 15h they are in
- * the legacy file:
- *
- * amd-ucode/microcode_amd.bin
- *
- * This legacy file is always smaller than 2K in size.
- *
- * Beginning with family 15h, they are in family-specific firmware files:
- *
- * amd-ucode/microcode_amd_fam15h.bin
- * amd-ucode/microcode_amd_fam16h.bin
- * ...
- *
- * These might be larger than 2K.
- */
-static enum ucode_state request_microcode_amd(int cpu, struct device *device,
- bool refresh_fw)
-{
- char fw_name[36] = "amd-ucode/microcode_amd.bin";
- struct cpuinfo_x86 *c = &cpu_data(cpu);
- enum ucode_state ret = UCODE_NFOUND;
- const struct firmware *fw;
-
- /* reload ucode container only on the boot cpu */
- if (!refresh_fw || c->cpu_index != boot_cpu_data.cpu_index)
- return UCODE_OK;
-
- if (c->x86 >= 0x15)
- snprintf(fw_name, sizeof(fw_name), "amd-ucode/microcode_amd_fam%.2xh.bin", c->x86);
-
- if (request_firmware(&fw, (const char *)fw_name, device)) {
- pr_debug("failed to load file %s\n", fw_name);
- goto out;
- }
-
- ret = UCODE_ERROR;
- if (*(u32 *)fw->data != UCODE_MAGIC) {
- pr_err("invalid magic value (0x%08x)\n", *(u32 *)fw->data);
- goto fw_release;
- }
-
- ret = load_microcode_amd(c->x86, fw->data, fw->size);
-
- fw_release:
- release_firmware(fw);
-
- out:
- return ret;
-}
-
-static enum ucode_state
-request_microcode_user(int cpu, const void __user *buf, size_t size)
-{
- return UCODE_ERROR;
-}
-
-static void microcode_fini_cpu_amd(int cpu)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-
- uci->mc = NULL;
-}
-
-static struct microcode_ops microcode_amd_ops = {
- .request_microcode_user = request_microcode_user,
- .request_microcode_fw = request_microcode_amd,
- .collect_cpu_info = collect_cpu_info_amd,
- .apply_microcode = apply_microcode_amd,
- .microcode_fini_cpu = microcode_fini_cpu_amd,
-};
-
-struct microcode_ops * __init init_amd_microcode(void)
-{
- struct cpuinfo_x86 *c = &cpu_data(0);
-
- if (c->x86_vendor != X86_VENDOR_AMD || c->x86 < 0x10) {
- pr_warning("AMD CPU family 0x%x not supported\n", c->x86);
- return NULL;
- }
-
- return µcode_amd_ops;
-}
-
-void __exit exit_amd_microcode(void)
-{
- cleanup();
-}
+++ /dev/null
-/*
- * Copyright (C) 2013 Advanced Micro Devices, Inc.
- *
- * Author: Jacob Shin <jacob.shin@amd.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#include <linux/earlycpio.h>
-#include <linux/initrd.h>
-
-#include <asm/cpu.h>
-#include <asm/setup.h>
-#include <asm/microcode_amd.h>
-
-static bool ucode_loaded;
-static u32 ucode_new_rev;
-static unsigned long ucode_offset;
-static size_t ucode_size;
-
-/*
- * Microcode patch container file is prepended to the initrd in cpio format.
- * See Documentation/x86/early-microcode.txt
- */
-static __initdata char ucode_path[] = "kernel/x86/microcode/AuthenticAMD.bin";
-
-static struct cpio_data __init find_ucode_in_initrd(void)
-{
- long offset = 0;
- char *path;
- void *start;
- size_t size;
- unsigned long *uoffset;
- size_t *usize;
- struct cpio_data cd;
-
-#ifdef CONFIG_X86_32
- struct boot_params *p;
-
- /*
- * On 32-bit, early load occurs before paging is turned on so we need
- * to use physical addresses.
- */
- p = (struct boot_params *)__pa_nodebug(&boot_params);
- path = (char *)__pa_nodebug(ucode_path);
- start = (void *)p->hdr.ramdisk_image;
- size = p->hdr.ramdisk_size;
- uoffset = (unsigned long *)__pa_nodebug(&ucode_offset);
- usize = (size_t *)__pa_nodebug(&ucode_size);
-#else
- path = ucode_path;
- start = (void *)(boot_params.hdr.ramdisk_image + PAGE_OFFSET);
- size = boot_params.hdr.ramdisk_size;
- uoffset = &ucode_offset;
- usize = &ucode_size;
-#endif
-
- cd = find_cpio_data(path, start, size, &offset);
- if (!cd.data)
- return cd;
-
- if (*(u32 *)cd.data != UCODE_MAGIC) {
- cd.data = NULL;
- cd.size = 0;
- return cd;
- }
-
- *uoffset = (u8 *)cd.data - (u8 *)start;
- *usize = cd.size;
-
- return cd;
-}
-
-/*
- * Early load occurs before we can vmalloc(). So we look for the microcode
- * patch container file in initrd, traverse equivalent cpu table, look for a
- * matching microcode patch, and update, all in initrd memory in place.
- * When vmalloc() is available for use later -- on 64-bit during first AP load,
- * and on 32-bit during save_microcode_in_initrd_amd() -- we can call
- * load_microcode_amd() to save equivalent cpu table and microcode patches in
- * kernel heap memory.
- */
-static void apply_ucode_in_initrd(void *ucode, size_t size)
-{
- struct equiv_cpu_entry *eq;
- u32 *header;
- u8 *data;
- u16 eq_id = 0;
- int offset, left;
- u32 rev, eax;
- u32 *new_rev;
- unsigned long *uoffset;
- size_t *usize;
-
-#ifdef CONFIG_X86_32
- new_rev = (u32 *)__pa_nodebug(&ucode_new_rev);
- uoffset = (unsigned long *)__pa_nodebug(&ucode_offset);
- usize = (size_t *)__pa_nodebug(&ucode_size);
-#else
- new_rev = &ucode_new_rev;
- uoffset = &ucode_offset;
- usize = &ucode_size;
-#endif
-
- data = ucode;
- left = size;
- header = (u32 *)data;
-
- /* find equiv cpu table */
-
- if (header[1] != UCODE_EQUIV_CPU_TABLE_TYPE || /* type */
- header[2] == 0) /* size */
- return;
-
- eax = cpuid_eax(0x00000001);
-
- while (left > 0) {
- eq = (struct equiv_cpu_entry *)(data + CONTAINER_HDR_SZ);
-
- offset = header[2] + CONTAINER_HDR_SZ;
- data += offset;
- left -= offset;
-
- eq_id = find_equiv_id(eq, eax);
- if (eq_id)
- break;
-
- /*
- * support multiple container files appended together. if this
- * one does not have a matching equivalent cpu entry, we fast
- * forward to the next container file.
- */
- while (left > 0) {
- header = (u32 *)data;
- if (header[0] == UCODE_MAGIC &&
- header[1] == UCODE_EQUIV_CPU_TABLE_TYPE)
- break;
-
- offset = header[1] + SECTION_HDR_SIZE;
- data += offset;
- left -= offset;
- }
-
- /* mark where the next microcode container file starts */
- offset = data - (u8 *)ucode;
- *uoffset += offset;
- *usize -= offset;
- ucode = data;
- }
-
- if (!eq_id) {
- *usize = 0;
- return;
- }
-
- /* find ucode and update if needed */
-
- rdmsr(MSR_AMD64_PATCH_LEVEL, rev, eax);
-
- while (left > 0) {
- struct microcode_amd *mc;
-
- header = (u32 *)data;
- if (header[0] != UCODE_UCODE_TYPE || /* type */
- header[1] == 0) /* size */
- break;
-
- mc = (struct microcode_amd *)(data + SECTION_HDR_SIZE);
- if (eq_id == mc->hdr.processor_rev_id && rev < mc->hdr.patch_id)
- if (__apply_microcode_amd(mc) == 0) {
- rev = mc->hdr.patch_id;
- *new_rev = rev;
- }
-
- offset = header[1] + SECTION_HDR_SIZE;
- data += offset;
- left -= offset;
- }
-
- /* mark where this microcode container file ends */
- offset = *usize - (data - (u8 *)ucode);
- *usize -= offset;
-
- if (!(*new_rev))
- *usize = 0;
-}
-
-void __init load_ucode_amd_bsp(void)
-{
- struct cpio_data cd = find_ucode_in_initrd();
- if (!cd.data)
- return;
-
- apply_ucode_in_initrd(cd.data, cd.size);
-}
-
-#ifdef CONFIG_X86_32
-u8 amd_bsp_mpb[MPB_MAX_SIZE];
-
-/*
- * On 32-bit, since AP's early load occurs before paging is turned on, we
- * cannot traverse cpu_equiv_table and pcache in kernel heap memory. So during
- * cold boot, AP will apply_ucode_in_initrd() just like the BSP. During
- * save_microcode_in_initrd_amd() BSP's patch is copied to amd_bsp_mpb, which
- * is used upon resume from suspend.
- */
-void load_ucode_amd_ap(void)
-{
- struct microcode_amd *mc;
- unsigned long *initrd;
- unsigned long *uoffset;
- size_t *usize;
- void *ucode;
-
- mc = (struct microcode_amd *)__pa(amd_bsp_mpb);
- if (mc->hdr.patch_id && mc->hdr.processor_rev_id) {
- __apply_microcode_amd(mc);
- return;
- }
-
- initrd = (unsigned long *)__pa(&initrd_start);
- uoffset = (unsigned long *)__pa(&ucode_offset);
- usize = (size_t *)__pa(&ucode_size);
-
- if (!*usize || !*initrd)
- return;
-
- ucode = (void *)((unsigned long)__pa(*initrd) + *uoffset);
- apply_ucode_in_initrd(ucode, *usize);
-}
-
-static void __init collect_cpu_sig_on_bsp(void *arg)
-{
- unsigned int cpu = smp_processor_id();
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- uci->cpu_sig.sig = cpuid_eax(0x00000001);
-}
-#else
-void load_ucode_amd_ap(void)
-{
- unsigned int cpu = smp_processor_id();
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- u32 rev, eax;
-
- rdmsr(MSR_AMD64_PATCH_LEVEL, rev, eax);
- eax = cpuid_eax(0x00000001);
-
- uci->cpu_sig.rev = rev;
- uci->cpu_sig.sig = eax;
-
- if (cpu && !ucode_loaded) {
- void *ucode;
-
- if (!ucode_size || !initrd_start)
- return;
-
- ucode = (void *)(initrd_start + ucode_offset);
- eax = ((eax >> 8) & 0xf) + ((eax >> 20) & 0xff);
- if (load_microcode_amd(eax, ucode, ucode_size) != UCODE_OK)
- return;
-
- ucode_loaded = true;
- }
-
- apply_microcode_amd(cpu);
-}
-#endif
-
-int __init save_microcode_in_initrd_amd(void)
-{
- enum ucode_state ret;
- void *ucode;
- u32 eax;
-
-#ifdef CONFIG_X86_32
- unsigned int bsp = boot_cpu_data.cpu_index;
- struct ucode_cpu_info *uci = ucode_cpu_info + bsp;
-
- if (!uci->cpu_sig.sig)
- smp_call_function_single(bsp, collect_cpu_sig_on_bsp, NULL, 1);
-#endif
- if (ucode_new_rev)
- pr_info("microcode: updated early to new patch_level=0x%08x\n",
- ucode_new_rev);
-
- if (ucode_loaded || !ucode_size || !initrd_start)
- return 0;
-
- ucode = (void *)(initrd_start + ucode_offset);
- eax = cpuid_eax(0x00000001);
- eax = ((eax >> 8) & 0xf) + ((eax >> 20) & 0xff);
-
- ret = load_microcode_amd(eax, ucode, ucode_size);
- if (ret != UCODE_OK)
- return -EINVAL;
-
- ucode_loaded = true;
- return 0;
-}
+++ /dev/null
-/*
- * Intel CPU Microcode Update Driver for Linux
- *
- * Copyright (C) 2000-2006 Tigran Aivazian <tigran@aivazian.fsnet.co.uk>
- * 2006 Shaohua Li <shaohua.li@intel.com>
- *
- * This driver allows to upgrade microcode on Intel processors
- * belonging to IA-32 family - PentiumPro, Pentium II,
- * Pentium III, Xeon, Pentium 4, etc.
- *
- * Reference: Section 8.11 of Volume 3a, IA-32 Intel? Architecture
- * Software Developer's Manual
- * Order Number 253668 or free download from:
- *
- * http://developer.intel.com/Assets/PDF/manual/253668.pdf
- *
- * For more information, go to http://www.urbanmyth.org/microcode
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
- * 1.0 16 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Initial release.
- * 1.01 18 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Added read() support + cleanups.
- * 1.02 21 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Added 'device trimming' support. open(O_WRONLY) zeroes
- * and frees the saved copy of applied microcode.
- * 1.03 29 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Made to use devfs (/dev/cpu/microcode) + cleanups.
- * 1.04 06 Jun 2000, Simon Trimmer <simon@veritas.com>
- * Added misc device support (now uses both devfs and misc).
- * Added MICROCODE_IOCFREE ioctl to clear memory.
- * 1.05 09 Jun 2000, Simon Trimmer <simon@veritas.com>
- * Messages for error cases (non Intel & no suitable microcode).
- * 1.06 03 Aug 2000, Tigran Aivazian <tigran@veritas.com>
- * Removed ->release(). Removed exclusive open and status bitmap.
- * Added microcode_rwsem to serialize read()/write()/ioctl().
- * Removed global kernel lock usage.
- * 1.07 07 Sep 2000, Tigran Aivazian <tigran@veritas.com>
- * Write 0 to 0x8B msr and then cpuid before reading revision,
- * so that it works even if there were no update done by the
- * BIOS. Otherwise, reading from 0x8B gives junk (which happened
- * to be 0 on my machine which is why it worked even when I
- * disabled update by the BIOS)
- * Thanks to Eric W. Biederman <ebiederman@lnxi.com> for the fix.
- * 1.08 11 Dec 2000, Richard Schaal <richard.schaal@intel.com> and
- * Tigran Aivazian <tigran@veritas.com>
- * Intel Pentium 4 processor support and bugfixes.
- * 1.09 30 Oct 2001, Tigran Aivazian <tigran@veritas.com>
- * Bugfix for HT (Hyper-Threading) enabled processors
- * whereby processor resources are shared by all logical processors
- * in a single CPU package.
- * 1.10 28 Feb 2002 Asit K Mallick <asit.k.mallick@intel.com> and
- * Tigran Aivazian <tigran@veritas.com>,
- * Serialize updates as required on HT processors due to
- * speculative nature of implementation.
- * 1.11 22 Mar 2002 Tigran Aivazian <tigran@veritas.com>
- * Fix the panic when writing zero-length microcode chunk.
- * 1.12 29 Sep 2003 Nitin Kamble <nitin.a.kamble@intel.com>,
- * Jun Nakajima <jun.nakajima@intel.com>
- * Support for the microcode updates in the new format.
- * 1.13 10 Oct 2003 Tigran Aivazian <tigran@veritas.com>
- * Removed ->read() method and obsoleted MICROCODE_IOCFREE ioctl
- * because we no longer hold a copy of applied microcode
- * in kernel memory.
- * 1.14 25 Jun 2004 Tigran Aivazian <tigran@veritas.com>
- * Fix sigmatch() macro to handle old CPUs with pf == 0.
- * Thanks to Stuart Swales for pointing out this bug.
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/platform_device.h>
-#include <linux/miscdevice.h>
-#include <linux/capability.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/mutex.h>
-#include <linux/cpu.h>
-#include <linux/fs.h>
-#include <linux/mm.h>
-#include <linux/syscore_ops.h>
-
-#include <asm/microcode.h>
-#include <asm/processor.h>
-#include <asm/cpu_device_id.h>
-#include <asm/perf_event.h>
-
-MODULE_DESCRIPTION("Microcode Update Driver");
-MODULE_AUTHOR("Tigran Aivazian <tigran@aivazian.fsnet.co.uk>");
-MODULE_LICENSE("GPL");
-
-#define MICROCODE_VERSION "2.00"
-
-static struct microcode_ops *microcode_ops;
-
-/*
- * Synchronization.
- *
- * All non cpu-hotplug-callback call sites use:
- *
- * - microcode_mutex to synchronize with each other;
- * - get/put_online_cpus() to synchronize with
- * the cpu-hotplug-callback call sites.
- *
- * We guarantee that only a single cpu is being
- * updated at any particular moment of time.
- */
-static DEFINE_MUTEX(microcode_mutex);
-
-struct ucode_cpu_info ucode_cpu_info[NR_CPUS];
-EXPORT_SYMBOL_GPL(ucode_cpu_info);
-
-/*
- * Operations that are run on a target cpu:
- */
-
-struct cpu_info_ctx {
- struct cpu_signature *cpu_sig;
- int err;
-};
-
-static void collect_cpu_info_local(void *arg)
-{
- struct cpu_info_ctx *ctx = arg;
-
- ctx->err = microcode_ops->collect_cpu_info(smp_processor_id(),
- ctx->cpu_sig);
-}
-
-static int collect_cpu_info_on_target(int cpu, struct cpu_signature *cpu_sig)
-{
- struct cpu_info_ctx ctx = { .cpu_sig = cpu_sig, .err = 0 };
- int ret;
-
- ret = smp_call_function_single(cpu, collect_cpu_info_local, &ctx, 1);
- if (!ret)
- ret = ctx.err;
-
- return ret;
-}
-
-static int collect_cpu_info(int cpu)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- int ret;
-
- memset(uci, 0, sizeof(*uci));
-
- ret = collect_cpu_info_on_target(cpu, &uci->cpu_sig);
- if (!ret)
- uci->valid = 1;
-
- return ret;
-}
-
-struct apply_microcode_ctx {
- int err;
-};
-
-static void apply_microcode_local(void *arg)
-{
- struct apply_microcode_ctx *ctx = arg;
-
- ctx->err = microcode_ops->apply_microcode(smp_processor_id());
-}
-
-static int apply_microcode_on_target(int cpu)
-{
- struct apply_microcode_ctx ctx = { .err = 0 };
- int ret;
-
- ret = smp_call_function_single(cpu, apply_microcode_local, &ctx, 1);
- if (!ret)
- ret = ctx.err;
-
- return ret;
-}
-
-#ifdef CONFIG_MICROCODE_OLD_INTERFACE
-static int do_microcode_update(const void __user *buf, size_t size)
-{
- int error = 0;
- int cpu;
-
- for_each_online_cpu(cpu) {
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- enum ucode_state ustate;
-
- if (!uci->valid)
- continue;
-
- ustate = microcode_ops->request_microcode_user(cpu, buf, size);
- if (ustate == UCODE_ERROR) {
- error = -1;
- break;
- } else if (ustate == UCODE_OK)
- apply_microcode_on_target(cpu);
- }
-
- return error;
-}
-
-static int microcode_open(struct inode *inode, struct file *file)
-{
- return capable(CAP_SYS_RAWIO) ? nonseekable_open(inode, file) : -EPERM;
-}
-
-static ssize_t microcode_write(struct file *file, const char __user *buf,
- size_t len, loff_t *ppos)
-{
- ssize_t ret = -EINVAL;
-
- if ((len >> PAGE_SHIFT) > totalram_pages) {
- pr_err("too much data (max %ld pages)\n", totalram_pages);
- return ret;
- }
-
- get_online_cpus();
- mutex_lock(µcode_mutex);
-
- if (do_microcode_update(buf, len) == 0)
- ret = (ssize_t)len;
-
- if (ret > 0)
- perf_check_microcode();
-
- mutex_unlock(µcode_mutex);
- put_online_cpus();
-
- return ret;
-}
-
-static const struct file_operations microcode_fops = {
- .owner = THIS_MODULE,
- .write = microcode_write,
- .open = microcode_open,
- .llseek = no_llseek,
-};
-
-static struct miscdevice microcode_dev = {
- .minor = MICROCODE_MINOR,
- .name = "microcode",
- .nodename = "cpu/microcode",
- .fops = µcode_fops,
-};
-
-static int __init microcode_dev_init(void)
-{
- int error;
-
- error = misc_register(µcode_dev);
- if (error) {
- pr_err("can't misc_register on minor=%d\n", MICROCODE_MINOR);
- return error;
- }
-
- return 0;
-}
-
-static void __exit microcode_dev_exit(void)
-{
- misc_deregister(µcode_dev);
-}
-
-MODULE_ALIAS_MISCDEV(MICROCODE_MINOR);
-MODULE_ALIAS("devname:cpu/microcode");
-#else
-#define microcode_dev_init() 0
-#define microcode_dev_exit() do { } while (0)
-#endif
-
-/* fake device for request_firmware */
-static struct platform_device *microcode_pdev;
-
-static int reload_for_cpu(int cpu)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- enum ucode_state ustate;
- int err = 0;
-
- if (!uci->valid)
- return err;
-
- ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev, true);
- if (ustate == UCODE_OK)
- apply_microcode_on_target(cpu);
- else
- if (ustate == UCODE_ERROR)
- err = -EINVAL;
- return err;
-}
-
-static ssize_t reload_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- int cpu;
- ssize_t ret = 0, tmp_ret;
-
- ret = kstrtoul(buf, 0, &val);
- if (ret)
- return ret;
-
- if (val != 1)
- return size;
-
- get_online_cpus();
- mutex_lock(µcode_mutex);
- for_each_online_cpu(cpu) {
- tmp_ret = reload_for_cpu(cpu);
- if (tmp_ret != 0)
- pr_warn("Error reloading microcode on CPU %d\n", cpu);
-
- /* save retval of the first encountered reload error */
- if (!ret)
- ret = tmp_ret;
- }
- if (!ret)
- perf_check_microcode();
- mutex_unlock(µcode_mutex);
- put_online_cpus();
-
- if (!ret)
- ret = size;
-
- return ret;
-}
-
-static ssize_t version_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + dev->id;
-
- return sprintf(buf, "0x%x\n", uci->cpu_sig.rev);
-}
-
-static ssize_t pf_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + dev->id;
-
- return sprintf(buf, "0x%x\n", uci->cpu_sig.pf);
-}
-
-static DEVICE_ATTR(reload, 0200, NULL, reload_store);
-static DEVICE_ATTR(version, 0400, version_show, NULL);
-static DEVICE_ATTR(processor_flags, 0400, pf_show, NULL);
-
-static struct attribute *mc_default_attrs[] = {
- &dev_attr_version.attr,
- &dev_attr_processor_flags.attr,
- NULL
-};
-
-static struct attribute_group mc_attr_group = {
- .attrs = mc_default_attrs,
- .name = "microcode",
-};
-
-static void microcode_fini_cpu(int cpu)
-{
- microcode_ops->microcode_fini_cpu(cpu);
-}
-
-static enum ucode_state microcode_resume_cpu(int cpu)
-{
- pr_debug("CPU%d updated upon resume\n", cpu);
-
- if (apply_microcode_on_target(cpu))
- return UCODE_ERROR;
-
- return UCODE_OK;
-}
-
-static enum ucode_state microcode_init_cpu(int cpu, bool refresh_fw)
-{
- enum ucode_state ustate;
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-
- if (uci && uci->valid)
- return UCODE_OK;
-
- if (collect_cpu_info(cpu))
- return UCODE_ERROR;
-
- /* --dimm. Trigger a delayed update? */
- if (system_state != SYSTEM_RUNNING)
- return UCODE_NFOUND;
-
- ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev,
- refresh_fw);
-
- if (ustate == UCODE_OK) {
- pr_debug("CPU%d updated upon init\n", cpu);
- apply_microcode_on_target(cpu);
- }
-
- return ustate;
-}
-
-static enum ucode_state microcode_update_cpu(int cpu)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-
- if (uci->valid)
- return microcode_resume_cpu(cpu);
-
- return microcode_init_cpu(cpu, false);
-}
-
-static int mc_device_add(struct device *dev, struct subsys_interface *sif)
-{
- int err, cpu = dev->id;
-
- if (!cpu_online(cpu))
- return 0;
-
- pr_debug("CPU%d added\n", cpu);
-
- err = sysfs_create_group(&dev->kobj, &mc_attr_group);
- if (err)
- return err;
-
- if (microcode_init_cpu(cpu, true) == UCODE_ERROR)
- return -EINVAL;
-
- return err;
-}
-
-static int mc_device_remove(struct device *dev, struct subsys_interface *sif)
-{
- int cpu = dev->id;
-
- if (!cpu_online(cpu))
- return 0;
-
- pr_debug("CPU%d removed\n", cpu);
- microcode_fini_cpu(cpu);
- sysfs_remove_group(&dev->kobj, &mc_attr_group);
- return 0;
-}
-
-static struct subsys_interface mc_cpu_interface = {
- .name = "microcode",
- .subsys = &cpu_subsys,
- .add_dev = mc_device_add,
- .remove_dev = mc_device_remove,
-};
-
-/**
- * mc_bp_resume - Update boot CPU microcode during resume.
- */
-static void mc_bp_resume(void)
-{
- int cpu = smp_processor_id();
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-
- if (uci->valid && uci->mc)
- microcode_ops->apply_microcode(cpu);
-}
-
-static struct syscore_ops mc_syscore_ops = {
- .resume = mc_bp_resume,
-};
-
-static int
-mc_cpu_callback(struct notifier_block *nb, unsigned long action, void *hcpu)
-{
- unsigned int cpu = (unsigned long)hcpu;
- struct device *dev;
-
- dev = get_cpu_device(cpu);
-
- switch (action & ~CPU_TASKS_FROZEN) {
- case CPU_ONLINE:
- microcode_update_cpu(cpu);
- pr_debug("CPU%d added\n", cpu);
- /*
- * "break" is missing on purpose here because we want to fall
- * through in order to create the sysfs group.
- */
-
- case CPU_DOWN_FAILED:
- if (sysfs_create_group(&dev->kobj, &mc_attr_group))
- pr_err("Failed to create group for CPU%d\n", cpu);
- break;
-
- case CPU_DOWN_PREPARE:
- /* Suspend is in progress, only remove the interface */
- sysfs_remove_group(&dev->kobj, &mc_attr_group);
- pr_debug("CPU%d removed\n", cpu);
- break;
-
- /*
- * case CPU_DEAD:
- *
- * When a CPU goes offline, don't free up or invalidate the copy of
- * the microcode in kernel memory, so that we can reuse it when the
- * CPU comes back online without unnecessarily requesting the userspace
- * for it again.
- */
- }
-
- /* The CPU refused to come up during a system resume */
- if (action == CPU_UP_CANCELED_FROZEN)
- microcode_fini_cpu(cpu);
-
- return NOTIFY_OK;
-}
-
-static struct notifier_block __refdata mc_cpu_notifier = {
- .notifier_call = mc_cpu_callback,
-};
-
-#ifdef MODULE
-/* Autoload on Intel and AMD systems */
-static const struct x86_cpu_id __initconst microcode_id[] = {
-#ifdef CONFIG_MICROCODE_INTEL
- { X86_VENDOR_INTEL, X86_FAMILY_ANY, X86_MODEL_ANY, },
-#endif
-#ifdef CONFIG_MICROCODE_AMD
- { X86_VENDOR_AMD, X86_FAMILY_ANY, X86_MODEL_ANY, },
-#endif
- {}
-};
-MODULE_DEVICE_TABLE(x86cpu, microcode_id);
-#endif
-
-static struct attribute *cpu_root_microcode_attrs[] = {
- &dev_attr_reload.attr,
- NULL
-};
-
-static struct attribute_group cpu_root_microcode_group = {
- .name = "microcode",
- .attrs = cpu_root_microcode_attrs,
-};
-
-static int __init microcode_init(void)
-{
- struct cpuinfo_x86 *c = &cpu_data(0);
- int error;
-
- if (c->x86_vendor == X86_VENDOR_INTEL)
- microcode_ops = init_intel_microcode();
- else if (c->x86_vendor == X86_VENDOR_AMD)
- microcode_ops = init_amd_microcode();
- else
- pr_err("no support for this CPU vendor\n");
-
- if (!microcode_ops)
- return -ENODEV;
-
- microcode_pdev = platform_device_register_simple("microcode", -1,
- NULL, 0);
- if (IS_ERR(microcode_pdev))
- return PTR_ERR(microcode_pdev);
-
- get_online_cpus();
- mutex_lock(µcode_mutex);
-
- error = subsys_interface_register(&mc_cpu_interface);
- if (!error)
- perf_check_microcode();
- mutex_unlock(µcode_mutex);
- put_online_cpus();
-
- if (error)
- goto out_pdev;
-
- error = sysfs_create_group(&cpu_subsys.dev_root->kobj,
- &cpu_root_microcode_group);
-
- if (error) {
- pr_err("Error creating microcode group!\n");
- goto out_driver;
- }
-
- error = microcode_dev_init();
- if (error)
- goto out_ucode_group;
-
- register_syscore_ops(&mc_syscore_ops);
- register_hotcpu_notifier(&mc_cpu_notifier);
-
- pr_info("Microcode Update Driver: v" MICROCODE_VERSION
- " <tigran@aivazian.fsnet.co.uk>, Peter Oruba\n");
-
- return 0;
-
- out_ucode_group:
- sysfs_remove_group(&cpu_subsys.dev_root->kobj,
- &cpu_root_microcode_group);
-
- out_driver:
- get_online_cpus();
- mutex_lock(µcode_mutex);
-
- subsys_interface_unregister(&mc_cpu_interface);
-
- mutex_unlock(µcode_mutex);
- put_online_cpus();
-
- out_pdev:
- platform_device_unregister(microcode_pdev);
- return error;
-
-}
-module_init(microcode_init);
-
-static void __exit microcode_exit(void)
-{
- struct cpuinfo_x86 *c = &cpu_data(0);
-
- microcode_dev_exit();
-
- unregister_hotcpu_notifier(&mc_cpu_notifier);
- unregister_syscore_ops(&mc_syscore_ops);
-
- sysfs_remove_group(&cpu_subsys.dev_root->kobj,
- &cpu_root_microcode_group);
-
- get_online_cpus();
- mutex_lock(µcode_mutex);
-
- subsys_interface_unregister(&mc_cpu_interface);
-
- mutex_unlock(µcode_mutex);
- put_online_cpus();
-
- platform_device_unregister(microcode_pdev);
-
- microcode_ops = NULL;
-
- if (c->x86_vendor == X86_VENDOR_AMD)
- exit_amd_microcode();
-
- pr_info("Microcode Update Driver: v" MICROCODE_VERSION " removed.\n");
-}
-module_exit(microcode_exit);
+++ /dev/null
-/*
- * X86 CPU microcode early update for Linux
- *
- * Copyright (C) 2012 Fenghua Yu <fenghua.yu@intel.com>
- * H Peter Anvin" <hpa@zytor.com>
- *
- * This driver allows to early upgrade microcode on Intel processors
- * belonging to IA-32 family - PentiumPro, Pentium II,
- * Pentium III, Xeon, Pentium 4, etc.
- *
- * Reference: Section 9.11 of Volume 3, IA-32 Intel Architecture
- * Software Developer's Manual.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- */
-#include <linux/module.h>
-#include <asm/microcode_intel.h>
-#include <asm/microcode_amd.h>
-#include <asm/processor.h>
-
-#define QCHAR(a, b, c, d) ((a) + ((b) << 8) + ((c) << 16) + ((d) << 24))
-#define CPUID_INTEL1 QCHAR('G', 'e', 'n', 'u')
-#define CPUID_INTEL2 QCHAR('i', 'n', 'e', 'I')
-#define CPUID_INTEL3 QCHAR('n', 't', 'e', 'l')
-#define CPUID_AMD1 QCHAR('A', 'u', 't', 'h')
-#define CPUID_AMD2 QCHAR('e', 'n', 't', 'i')
-#define CPUID_AMD3 QCHAR('c', 'A', 'M', 'D')
-
-#define CPUID_IS(a, b, c, ebx, ecx, edx) \
- (!((ebx ^ (a))|(edx ^ (b))|(ecx ^ (c))))
-
-/*
- * In early loading microcode phase on BSP, boot_cpu_data is not set up yet.
- * x86_vendor() gets vendor id for BSP.
- *
- * In 32 bit AP case, accessing boot_cpu_data needs linear address. To simplify
- * coding, we still use x86_vendor() to get vendor id for AP.
- *
- * x86_vendor() gets vendor information directly through cpuid.
- */
-static int x86_vendor(void)
-{
- u32 eax = 0x00000000;
- u32 ebx, ecx = 0, edx;
-
- native_cpuid(&eax, &ebx, &ecx, &edx);
-
- if (CPUID_IS(CPUID_INTEL1, CPUID_INTEL2, CPUID_INTEL3, ebx, ecx, edx))
- return X86_VENDOR_INTEL;
-
- if (CPUID_IS(CPUID_AMD1, CPUID_AMD2, CPUID_AMD3, ebx, ecx, edx))
- return X86_VENDOR_AMD;
-
- return X86_VENDOR_UNKNOWN;
-}
-
-static int x86_family(void)
-{
- u32 eax = 0x00000001;
- u32 ebx, ecx = 0, edx;
- int x86;
-
- native_cpuid(&eax, &ebx, &ecx, &edx);
-
- x86 = (eax >> 8) & 0xf;
- if (x86 == 15)
- x86 += (eax >> 20) & 0xff;
-
- return x86;
-}
-
-void __init load_ucode_bsp(void)
-{
- int vendor, x86;
-
- if (!have_cpuid_p())
- return;
-
- vendor = x86_vendor();
- x86 = x86_family();
-
- switch (vendor) {
- case X86_VENDOR_INTEL:
- if (x86 >= 6)
- load_ucode_intel_bsp();
- break;
- case X86_VENDOR_AMD:
- if (x86 >= 0x10)
- load_ucode_amd_bsp();
- break;
- default:
- break;
- }
-}
-
-void load_ucode_ap(void)
-{
- int vendor, x86;
-
- if (!have_cpuid_p())
- return;
-
- vendor = x86_vendor();
- x86 = x86_family();
-
- switch (vendor) {
- case X86_VENDOR_INTEL:
- if (x86 >= 6)
- load_ucode_intel_ap();
- break;
- case X86_VENDOR_AMD:
- if (x86 >= 0x10)
- load_ucode_amd_ap();
- break;
- default:
- break;
- }
-}
-
-int __init save_microcode_in_initrd(void)
-{
- struct cpuinfo_x86 *c = &boot_cpu_data;
-
- switch (c->x86_vendor) {
- case X86_VENDOR_INTEL:
- if (c->x86 >= 6)
- save_microcode_in_initrd_intel();
- break;
- case X86_VENDOR_AMD:
- if (c->x86 >= 0x10)
- save_microcode_in_initrd_amd();
- break;
- default:
- break;
- }
-
- return 0;
-}
+++ /dev/null
-/*
- * Intel CPU Microcode Update Driver for Linux
- *
- * Copyright (C) 2000-2006 Tigran Aivazian <tigran@aivazian.fsnet.co.uk>
- * 2006 Shaohua Li <shaohua.li@intel.com>
- *
- * This driver allows to upgrade microcode on Intel processors
- * belonging to IA-32 family - PentiumPro, Pentium II,
- * Pentium III, Xeon, Pentium 4, etc.
- *
- * Reference: Section 8.11 of Volume 3a, IA-32 Intel? Architecture
- * Software Developer's Manual
- * Order Number 253668 or free download from:
- *
- * http://developer.intel.com/Assets/PDF/manual/253668.pdf
- *
- * For more information, go to http://www.urbanmyth.org/microcode
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
- * 1.0 16 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Initial release.
- * 1.01 18 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Added read() support + cleanups.
- * 1.02 21 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Added 'device trimming' support. open(O_WRONLY) zeroes
- * and frees the saved copy of applied microcode.
- * 1.03 29 Feb 2000, Tigran Aivazian <tigran@sco.com>
- * Made to use devfs (/dev/cpu/microcode) + cleanups.
- * 1.04 06 Jun 2000, Simon Trimmer <simon@veritas.com>
- * Added misc device support (now uses both devfs and misc).
- * Added MICROCODE_IOCFREE ioctl to clear memory.
- * 1.05 09 Jun 2000, Simon Trimmer <simon@veritas.com>
- * Messages for error cases (non Intel & no suitable microcode).
- * 1.06 03 Aug 2000, Tigran Aivazian <tigran@veritas.com>
- * Removed ->release(). Removed exclusive open and status bitmap.
- * Added microcode_rwsem to serialize read()/write()/ioctl().
- * Removed global kernel lock usage.
- * 1.07 07 Sep 2000, Tigran Aivazian <tigran@veritas.com>
- * Write 0 to 0x8B msr and then cpuid before reading revision,
- * so that it works even if there were no update done by the
- * BIOS. Otherwise, reading from 0x8B gives junk (which happened
- * to be 0 on my machine which is why it worked even when I
- * disabled update by the BIOS)
- * Thanks to Eric W. Biederman <ebiederman@lnxi.com> for the fix.
- * 1.08 11 Dec 2000, Richard Schaal <richard.schaal@intel.com> and
- * Tigran Aivazian <tigran@veritas.com>
- * Intel Pentium 4 processor support and bugfixes.
- * 1.09 30 Oct 2001, Tigran Aivazian <tigran@veritas.com>
- * Bugfix for HT (Hyper-Threading) enabled processors
- * whereby processor resources are shared by all logical processors
- * in a single CPU package.
- * 1.10 28 Feb 2002 Asit K Mallick <asit.k.mallick@intel.com> and
- * Tigran Aivazian <tigran@veritas.com>,
- * Serialize updates as required on HT processors due to
- * speculative nature of implementation.
- * 1.11 22 Mar 2002 Tigran Aivazian <tigran@veritas.com>
- * Fix the panic when writing zero-length microcode chunk.
- * 1.12 29 Sep 2003 Nitin Kamble <nitin.a.kamble@intel.com>,
- * Jun Nakajima <jun.nakajima@intel.com>
- * Support for the microcode updates in the new format.
- * 1.13 10 Oct 2003 Tigran Aivazian <tigran@veritas.com>
- * Removed ->read() method and obsoleted MICROCODE_IOCFREE ioctl
- * because we no longer hold a copy of applied microcode
- * in kernel memory.
- * 1.14 25 Jun 2004 Tigran Aivazian <tigran@veritas.com>
- * Fix sigmatch() macro to handle old CPUs with pf == 0.
- * Thanks to Stuart Swales for pointing out this bug.
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/firmware.h>
-#include <linux/uaccess.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/vmalloc.h>
-
-#include <asm/microcode_intel.h>
-#include <asm/processor.h>
-#include <asm/msr.h>
-
-MODULE_DESCRIPTION("Microcode Update Driver");
-MODULE_AUTHOR("Tigran Aivazian <tigran@aivazian.fsnet.co.uk>");
-MODULE_LICENSE("GPL");
-
-static int collect_cpu_info(int cpu_num, struct cpu_signature *csig)
-{
- struct cpuinfo_x86 *c = &cpu_data(cpu_num);
- unsigned int val[2];
-
- memset(csig, 0, sizeof(*csig));
-
- csig->sig = cpuid_eax(0x00000001);
-
- if ((c->x86_model >= 5) || (c->x86 > 6)) {
- /* get processor flags from MSR 0x17 */
- rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
- csig->pf = 1 << ((val[1] >> 18) & 7);
- }
-
- csig->rev = c->microcode;
- pr_info("CPU%d sig=0x%x, pf=0x%x, revision=0x%x\n",
- cpu_num, csig->sig, csig->pf, csig->rev);
-
- return 0;
-}
-
-/*
- * return 0 - no update found
- * return 1 - found update
- */
-static int get_matching_mc(struct microcode_intel *mc_intel, int cpu)
-{
- struct cpu_signature cpu_sig;
- unsigned int csig, cpf, crev;
-
- collect_cpu_info(cpu, &cpu_sig);
-
- csig = cpu_sig.sig;
- cpf = cpu_sig.pf;
- crev = cpu_sig.rev;
-
- return get_matching_microcode(csig, cpf, mc_intel, crev);
-}
-
-int apply_microcode(int cpu)
-{
- struct microcode_intel *mc_intel;
- struct ucode_cpu_info *uci;
- unsigned int val[2];
- int cpu_num = raw_smp_processor_id();
- struct cpuinfo_x86 *c = &cpu_data(cpu_num);
-
- uci = ucode_cpu_info + cpu;
- mc_intel = uci->mc;
-
- /* We should bind the task to the CPU */
- BUG_ON(cpu_num != cpu);
-
- if (mc_intel == NULL)
- return 0;
-
- /*
- * Microcode on this CPU could be updated earlier. Only apply the
- * microcode patch in mc_intel when it is newer than the one on this
- * CPU.
- */
- if (get_matching_mc(mc_intel, cpu) == 0)
- return 0;
-
- /* write microcode via MSR 0x79 */
- wrmsr(MSR_IA32_UCODE_WRITE,
- (unsigned long) mc_intel->bits,
- (unsigned long) mc_intel->bits >> 16 >> 16);
- wrmsr(MSR_IA32_UCODE_REV, 0, 0);
-
- /* As documented in the SDM: Do a CPUID 1 here */
- sync_core();
-
- /* get the current revision from MSR 0x8B */
- rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
-
- if (val[1] != mc_intel->hdr.rev) {
- pr_err("CPU%d update to revision 0x%x failed\n",
- cpu_num, mc_intel->hdr.rev);
- return -1;
- }
- pr_info("CPU%d updated to revision 0x%x, date = %04x-%02x-%02x\n",
- cpu_num, val[1],
- mc_intel->hdr.date & 0xffff,
- mc_intel->hdr.date >> 24,
- (mc_intel->hdr.date >> 16) & 0xff);
-
- uci->cpu_sig.rev = val[1];
- c->microcode = val[1];
-
- return 0;
-}
-
-static enum ucode_state generic_load_microcode(int cpu, void *data, size_t size,
- int (*get_ucode_data)(void *, const void *, size_t))
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- u8 *ucode_ptr = data, *new_mc = NULL, *mc = NULL;
- int new_rev = uci->cpu_sig.rev;
- unsigned int leftover = size;
- enum ucode_state state = UCODE_OK;
- unsigned int curr_mc_size = 0;
- unsigned int csig, cpf;
-
- while (leftover) {
- struct microcode_header_intel mc_header;
- unsigned int mc_size;
-
- if (get_ucode_data(&mc_header, ucode_ptr, sizeof(mc_header)))
- break;
-
- mc_size = get_totalsize(&mc_header);
- if (!mc_size || mc_size > leftover) {
- pr_err("error! Bad data in microcode data file\n");
- break;
- }
-
- /* For performance reasons, reuse mc area when possible */
- if (!mc || mc_size > curr_mc_size) {
- vfree(mc);
- mc = vmalloc(mc_size);
- if (!mc)
- break;
- curr_mc_size = mc_size;
- }
-
- if (get_ucode_data(mc, ucode_ptr, mc_size) ||
- microcode_sanity_check(mc, 1) < 0) {
- break;
- }
-
- csig = uci->cpu_sig.sig;
- cpf = uci->cpu_sig.pf;
- if (get_matching_microcode(csig, cpf, mc, new_rev)) {
- vfree(new_mc);
- new_rev = mc_header.rev;
- new_mc = mc;
- mc = NULL; /* trigger new vmalloc */
- }
-
- ucode_ptr += mc_size;
- leftover -= mc_size;
- }
-
- vfree(mc);
-
- if (leftover) {
- vfree(new_mc);
- state = UCODE_ERROR;
- goto out;
- }
-
- if (!new_mc) {
- state = UCODE_NFOUND;
- goto out;
- }
-
- vfree(uci->mc);
- uci->mc = (struct microcode_intel *)new_mc;
-
- /*
- * If early loading microcode is supported, save this mc into
- * permanent memory. So it will be loaded early when a CPU is hot added
- * or resumes.
- */
- save_mc_for_early(new_mc);
-
- pr_debug("CPU%d found a matching microcode update with version 0x%x (current=0x%x)\n",
- cpu, new_rev, uci->cpu_sig.rev);
-out:
- return state;
-}
-
-static int get_ucode_fw(void *to, const void *from, size_t n)
-{
- memcpy(to, from, n);
- return 0;
-}
-
-static enum ucode_state request_microcode_fw(int cpu, struct device *device,
- bool refresh_fw)
-{
- char name[30];
- struct cpuinfo_x86 *c = &cpu_data(cpu);
- const struct firmware *firmware;
- enum ucode_state ret;
-
- sprintf(name, "intel-ucode/%02x-%02x-%02x",
- c->x86, c->x86_model, c->x86_mask);
-
- if (request_firmware(&firmware, name, device)) {
- pr_debug("data file %s load failed\n", name);
- return UCODE_NFOUND;
- }
-
- ret = generic_load_microcode(cpu, (void *)firmware->data,
- firmware->size, &get_ucode_fw);
-
- release_firmware(firmware);
-
- return ret;
-}
-
-static int get_ucode_user(void *to, const void *from, size_t n)
-{
- return copy_from_user(to, from, n);
-}
-
-static enum ucode_state
-request_microcode_user(int cpu, const void __user *buf, size_t size)
-{
- return generic_load_microcode(cpu, (void *)buf, size, &get_ucode_user);
-}
-
-static void microcode_fini_cpu(int cpu)
-{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-
- vfree(uci->mc);
- uci->mc = NULL;
-}
-
-static struct microcode_ops microcode_intel_ops = {
- .request_microcode_user = request_microcode_user,
- .request_microcode_fw = request_microcode_fw,
- .collect_cpu_info = collect_cpu_info,
- .apply_microcode = apply_microcode,
- .microcode_fini_cpu = microcode_fini_cpu,
-};
-
-struct microcode_ops * __init init_intel_microcode(void)
-{
- struct cpuinfo_x86 *c = &cpu_data(0);
-
- if (c->x86_vendor != X86_VENDOR_INTEL || c->x86 < 6 ||
- cpu_has(c, X86_FEATURE_IA64)) {
- pr_err("Intel CPU family 0x%x not supported\n", c->x86);
- return NULL;
- }
-
- return µcode_intel_ops;
-}
-
+++ /dev/null
-/*
- * Intel CPU microcode early update for Linux
- *
- * Copyright (C) 2012 Fenghua Yu <fenghua.yu@intel.com>
- * H Peter Anvin" <hpa@zytor.com>
- *
- * This allows to early upgrade microcode on Intel processors
- * belonging to IA-32 family - PentiumPro, Pentium II,
- * Pentium III, Xeon, Pentium 4, etc.
- *
- * Reference: Section 9.11 of Volume 3, IA-32 Intel Architecture
- * Software Developer's Manual.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- */
-#include <linux/module.h>
-#include <linux/mm.h>
-#include <linux/slab.h>
-#include <linux/earlycpio.h>
-#include <linux/initrd.h>
-#include <linux/cpu.h>
-#include <asm/msr.h>
-#include <asm/microcode_intel.h>
-#include <asm/processor.h>
-#include <asm/tlbflush.h>
-#include <asm/setup.h>
-
-unsigned long mc_saved_in_initrd[MAX_UCODE_COUNT];
-struct mc_saved_data {
- unsigned int mc_saved_count;
- struct microcode_intel **mc_saved;
-} mc_saved_data;
-
-static enum ucode_state
-generic_load_microcode_early(struct microcode_intel **mc_saved_p,
- unsigned int mc_saved_count,
- struct ucode_cpu_info *uci)
-{
- struct microcode_intel *ucode_ptr, *new_mc = NULL;
- int new_rev = uci->cpu_sig.rev;
- enum ucode_state state = UCODE_OK;
- unsigned int mc_size;
- struct microcode_header_intel *mc_header;
- unsigned int csig = uci->cpu_sig.sig;
- unsigned int cpf = uci->cpu_sig.pf;
- int i;
-
- for (i = 0; i < mc_saved_count; i++) {
- ucode_ptr = mc_saved_p[i];
-
- mc_header = (struct microcode_header_intel *)ucode_ptr;
- mc_size = get_totalsize(mc_header);
- if (get_matching_microcode(csig, cpf, ucode_ptr, new_rev)) {
- new_rev = mc_header->rev;
- new_mc = ucode_ptr;
- }
- }
-
- if (!new_mc) {
- state = UCODE_NFOUND;
- goto out;
- }
-
- uci->mc = (struct microcode_intel *)new_mc;
-out:
- return state;
-}
-
-static void
-microcode_pointer(struct microcode_intel **mc_saved,
- unsigned long *mc_saved_in_initrd,
- unsigned long initrd_start, int mc_saved_count)
-{
- int i;
-
- for (i = 0; i < mc_saved_count; i++)
- mc_saved[i] = (struct microcode_intel *)
- (mc_saved_in_initrd[i] + initrd_start);
-}
-
-#ifdef CONFIG_X86_32
-static void
-microcode_phys(struct microcode_intel **mc_saved_tmp,
- struct mc_saved_data *mc_saved_data)
-{
- int i;
- struct microcode_intel ***mc_saved;
-
- mc_saved = (struct microcode_intel ***)
- __pa_nodebug(&mc_saved_data->mc_saved);
- for (i = 0; i < mc_saved_data->mc_saved_count; i++) {
- struct microcode_intel *p;
-
- p = *(struct microcode_intel **)
- __pa_nodebug(mc_saved_data->mc_saved + i);
- mc_saved_tmp[i] = (struct microcode_intel *)__pa_nodebug(p);
- }
-}
-#endif
-
-static enum ucode_state
-load_microcode(struct mc_saved_data *mc_saved_data,
- unsigned long *mc_saved_in_initrd,
- unsigned long initrd_start,
- struct ucode_cpu_info *uci)
-{
- struct microcode_intel *mc_saved_tmp[MAX_UCODE_COUNT];
- unsigned int count = mc_saved_data->mc_saved_count;
-
- if (!mc_saved_data->mc_saved) {
- microcode_pointer(mc_saved_tmp, mc_saved_in_initrd,
- initrd_start, count);
-
- return generic_load_microcode_early(mc_saved_tmp, count, uci);
- } else {
-#ifdef CONFIG_X86_32
- microcode_phys(mc_saved_tmp, mc_saved_data);
- return generic_load_microcode_early(mc_saved_tmp, count, uci);
-#else
- return generic_load_microcode_early(mc_saved_data->mc_saved,
- count, uci);
-#endif
- }
-}
-
-static u8 get_x86_family(unsigned long sig)
-{
- u8 x86;
-
- x86 = (sig >> 8) & 0xf;
-
- if (x86 == 0xf)
- x86 += (sig >> 20) & 0xff;
-
- return x86;
-}
-
-static u8 get_x86_model(unsigned long sig)
-{
- u8 x86, x86_model;
-
- x86 = get_x86_family(sig);
- x86_model = (sig >> 4) & 0xf;
-
- if (x86 == 0x6 || x86 == 0xf)
- x86_model += ((sig >> 16) & 0xf) << 4;
-
- return x86_model;
-}
-
-/*
- * Given CPU signature and a microcode patch, this function finds if the
- * microcode patch has matching family and model with the CPU.
- */
-static enum ucode_state
-matching_model_microcode(struct microcode_header_intel *mc_header,
- unsigned long sig)
-{
- u8 x86, x86_model;
- u8 x86_ucode, x86_model_ucode;
- struct extended_sigtable *ext_header;
- unsigned long total_size = get_totalsize(mc_header);
- unsigned long data_size = get_datasize(mc_header);
- int ext_sigcount, i;
- struct extended_signature *ext_sig;
-
- x86 = get_x86_family(sig);
- x86_model = get_x86_model(sig);
-
- x86_ucode = get_x86_family(mc_header->sig);
- x86_model_ucode = get_x86_model(mc_header->sig);
-
- if (x86 == x86_ucode && x86_model == x86_model_ucode)
- return UCODE_OK;
-
- /* Look for ext. headers: */
- if (total_size <= data_size + MC_HEADER_SIZE)
- return UCODE_NFOUND;
-
- ext_header = (struct extended_sigtable *)
- mc_header + data_size + MC_HEADER_SIZE;
- ext_sigcount = ext_header->count;
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
-
- for (i = 0; i < ext_sigcount; i++) {
- x86_ucode = get_x86_family(ext_sig->sig);
- x86_model_ucode = get_x86_model(ext_sig->sig);
-
- if (x86 == x86_ucode && x86_model == x86_model_ucode)
- return UCODE_OK;
-
- ext_sig++;
- }
-
- return UCODE_NFOUND;
-}
-
-static int
-save_microcode(struct mc_saved_data *mc_saved_data,
- struct microcode_intel **mc_saved_src,
- unsigned int mc_saved_count)
-{
- int i, j;
- struct microcode_intel **mc_saved_p;
- int ret;
-
- if (!mc_saved_count)
- return -EINVAL;
-
- /*
- * Copy new microcode data.
- */
- mc_saved_p = kmalloc(mc_saved_count*sizeof(struct microcode_intel *),
- GFP_KERNEL);
- if (!mc_saved_p)
- return -ENOMEM;
-
- for (i = 0; i < mc_saved_count; i++) {
- struct microcode_intel *mc = mc_saved_src[i];
- struct microcode_header_intel *mc_header = &mc->hdr;
- unsigned long mc_size = get_totalsize(mc_header);
- mc_saved_p[i] = kmalloc(mc_size, GFP_KERNEL);
- if (!mc_saved_p[i]) {
- ret = -ENOMEM;
- goto err;
- }
- if (!mc_saved_src[i]) {
- ret = -EINVAL;
- goto err;
- }
- memcpy(mc_saved_p[i], mc, mc_size);
- }
-
- /*
- * Point to newly saved microcode.
- */
- mc_saved_data->mc_saved = mc_saved_p;
- mc_saved_data->mc_saved_count = mc_saved_count;
-
- return 0;
-
-err:
- for (j = 0; j <= i; j++)
- kfree(mc_saved_p[j]);
- kfree(mc_saved_p);
-
- return ret;
-}
-
-/*
- * A microcode patch in ucode_ptr is saved into mc_saved
- * - if it has matching signature and newer revision compared to an existing
- * patch mc_saved.
- * - or if it is a newly discovered microcode patch.
- *
- * The microcode patch should have matching model with CPU.
- */
-static void _save_mc(struct microcode_intel **mc_saved, u8 *ucode_ptr,
- unsigned int *mc_saved_count_p)
-{
- int i;
- int found = 0;
- unsigned int mc_saved_count = *mc_saved_count_p;
- struct microcode_header_intel *mc_header;
-
- mc_header = (struct microcode_header_intel *)ucode_ptr;
- for (i = 0; i < mc_saved_count; i++) {
- unsigned int sig, pf;
- unsigned int new_rev;
- struct microcode_header_intel *mc_saved_header =
- (struct microcode_header_intel *)mc_saved[i];
- sig = mc_saved_header->sig;
- pf = mc_saved_header->pf;
- new_rev = mc_header->rev;
-
- if (get_matching_sig(sig, pf, ucode_ptr, new_rev)) {
- found = 1;
- if (update_match_revision(mc_header, new_rev)) {
- /*
- * Found an older ucode saved before.
- * Replace the older one with this newer
- * one.
- */
- mc_saved[i] =
- (struct microcode_intel *)ucode_ptr;
- break;
- }
- }
- }
- if (i >= mc_saved_count && !found)
- /*
- * This ucode is first time discovered in ucode file.
- * Save it to memory.
- */
- mc_saved[mc_saved_count++] =
- (struct microcode_intel *)ucode_ptr;
-
- *mc_saved_count_p = mc_saved_count;
-}
-
-/*
- * Get microcode matching with BSP's model. Only CPUs with the same model as
- * BSP can stay in the platform.
- */
-static enum ucode_state __init
-get_matching_model_microcode(int cpu, unsigned long start,
- void *data, size_t size,
- struct mc_saved_data *mc_saved_data,
- unsigned long *mc_saved_in_initrd,
- struct ucode_cpu_info *uci)
-{
- u8 *ucode_ptr = data;
- unsigned int leftover = size;
- enum ucode_state state = UCODE_OK;
- unsigned int mc_size;
- struct microcode_header_intel *mc_header;
- struct microcode_intel *mc_saved_tmp[MAX_UCODE_COUNT];
- unsigned int mc_saved_count = mc_saved_data->mc_saved_count;
- int i;
-
- while (leftover) {
- mc_header = (struct microcode_header_intel *)ucode_ptr;
-
- mc_size = get_totalsize(mc_header);
- if (!mc_size || mc_size > leftover ||
- microcode_sanity_check(ucode_ptr, 0) < 0)
- break;
-
- leftover -= mc_size;
-
- /*
- * Since APs with same family and model as the BSP may boot in
- * the platform, we need to find and save microcode patches
- * with the same family and model as the BSP.
- */
- if (matching_model_microcode(mc_header, uci->cpu_sig.sig) !=
- UCODE_OK) {
- ucode_ptr += mc_size;
- continue;
- }
-
- _save_mc(mc_saved_tmp, ucode_ptr, &mc_saved_count);
-
- ucode_ptr += mc_size;
- }
-
- if (leftover) {
- state = UCODE_ERROR;
- goto out;
- }
-
- if (mc_saved_count == 0) {
- state = UCODE_NFOUND;
- goto out;
- }
-
- for (i = 0; i < mc_saved_count; i++)
- mc_saved_in_initrd[i] = (unsigned long)mc_saved_tmp[i] - start;
-
- mc_saved_data->mc_saved_count = mc_saved_count;
-out:
- return state;
-}
-
-#define native_rdmsr(msr, val1, val2) \
-do { \
- u64 __val = native_read_msr((msr)); \
- (void)((val1) = (u32)__val); \
- (void)((val2) = (u32)(__val >> 32)); \
-} while (0)
-
-#define native_wrmsr(msr, low, high) \
- native_write_msr(msr, low, high);
-
-static int collect_cpu_info_early(struct ucode_cpu_info *uci)
-{
- unsigned int val[2];
- u8 x86, x86_model;
- struct cpu_signature csig;
- unsigned int eax, ebx, ecx, edx;
-
- csig.sig = 0;
- csig.pf = 0;
- csig.rev = 0;
-
- memset(uci, 0, sizeof(*uci));
-
- eax = 0x00000001;
- ecx = 0;
- native_cpuid(&eax, &ebx, &ecx, &edx);
- csig.sig = eax;
-
- x86 = get_x86_family(csig.sig);
- x86_model = get_x86_model(csig.sig);
-
- if ((x86_model >= 5) || (x86 > 6)) {
- /* get processor flags from MSR 0x17 */
- native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
- csig.pf = 1 << ((val[1] >> 18) & 7);
- }
- native_wrmsr(MSR_IA32_UCODE_REV, 0, 0);
-
- /* As documented in the SDM: Do a CPUID 1 here */
- sync_core();
-
- /* get the current revision from MSR 0x8B */
- native_rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
-
- csig.rev = val[1];
-
- uci->cpu_sig = csig;
- uci->valid = 1;
-
- return 0;
-}
-
-#ifdef DEBUG
-static void __ref show_saved_mc(void)
-{
- int i, j;
- unsigned int sig, pf, rev, total_size, data_size, date;
- struct ucode_cpu_info uci;
-
- if (mc_saved_data.mc_saved_count == 0) {
- pr_debug("no micorcode data saved.\n");
- return;
- }
- pr_debug("Total microcode saved: %d\n", mc_saved_data.mc_saved_count);
-
- collect_cpu_info_early(&uci);
-
- sig = uci.cpu_sig.sig;
- pf = uci.cpu_sig.pf;
- rev = uci.cpu_sig.rev;
- pr_debug("CPU%d: sig=0x%x, pf=0x%x, rev=0x%x\n",
- smp_processor_id(), sig, pf, rev);
-
- for (i = 0; i < mc_saved_data.mc_saved_count; i++) {
- struct microcode_header_intel *mc_saved_header;
- struct extended_sigtable *ext_header;
- int ext_sigcount;
- struct extended_signature *ext_sig;
-
- mc_saved_header = (struct microcode_header_intel *)
- mc_saved_data.mc_saved[i];
- sig = mc_saved_header->sig;
- pf = mc_saved_header->pf;
- rev = mc_saved_header->rev;
- total_size = get_totalsize(mc_saved_header);
- data_size = get_datasize(mc_saved_header);
- date = mc_saved_header->date;
-
- pr_debug("mc_saved[%d]: sig=0x%x, pf=0x%x, rev=0x%x, toal size=0x%x, date = %04x-%02x-%02x\n",
- i, sig, pf, rev, total_size,
- date & 0xffff,
- date >> 24,
- (date >> 16) & 0xff);
-
- /* Look for ext. headers: */
- if (total_size <= data_size + MC_HEADER_SIZE)
- continue;
-
- ext_header = (struct extended_sigtable *)
- mc_saved_header + data_size + MC_HEADER_SIZE;
- ext_sigcount = ext_header->count;
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
-
- for (j = 0; j < ext_sigcount; j++) {
- sig = ext_sig->sig;
- pf = ext_sig->pf;
-
- pr_debug("\tExtended[%d]: sig=0x%x, pf=0x%x\n",
- j, sig, pf);
-
- ext_sig++;
- }
-
- }
-}
-#else
-static inline void show_saved_mc(void)
-{
-}
-#endif
-
-#if defined(CONFIG_MICROCODE_INTEL_EARLY) && defined(CONFIG_HOTPLUG_CPU)
-static DEFINE_MUTEX(x86_cpu_microcode_mutex);
-/*
- * Save this mc into mc_saved_data. So it will be loaded early when a CPU is
- * hot added or resumes.
- *
- * Please make sure this mc should be a valid microcode patch before calling
- * this function.
- */
-int save_mc_for_early(u8 *mc)
-{
- struct microcode_intel *mc_saved_tmp[MAX_UCODE_COUNT];
- unsigned int mc_saved_count_init;
- unsigned int mc_saved_count;
- struct microcode_intel **mc_saved;
- int ret = 0;
- int i;
-
- /*
- * Hold hotplug lock so mc_saved_data is not accessed by a CPU in
- * hotplug.
- */
- mutex_lock(&x86_cpu_microcode_mutex);
-
- mc_saved_count_init = mc_saved_data.mc_saved_count;
- mc_saved_count = mc_saved_data.mc_saved_count;
- mc_saved = mc_saved_data.mc_saved;
-
- if (mc_saved && mc_saved_count)
- memcpy(mc_saved_tmp, mc_saved,
- mc_saved_count * sizeof(struct mirocode_intel *));
- /*
- * Save the microcode patch mc in mc_save_tmp structure if it's a newer
- * version.
- */
-
- _save_mc(mc_saved_tmp, mc, &mc_saved_count);
-
- /*
- * Save the mc_save_tmp in global mc_saved_data.
- */
- ret = save_microcode(&mc_saved_data, mc_saved_tmp, mc_saved_count);
- if (ret) {
- pr_err("Cannot save microcode patch.\n");
- goto out;
- }
-
- show_saved_mc();
-
- /*
- * Free old saved microcod data.
- */
- if (mc_saved) {
- for (i = 0; i < mc_saved_count_init; i++)
- kfree(mc_saved[i]);
- kfree(mc_saved);
- }
-
-out:
- mutex_unlock(&x86_cpu_microcode_mutex);
-
- return ret;
-}
-EXPORT_SYMBOL_GPL(save_mc_for_early);
-#endif
-
-static __initdata char ucode_name[] = "kernel/x86/microcode/GenuineIntel.bin";
-static __init enum ucode_state
-scan_microcode(unsigned long start, unsigned long end,
- struct mc_saved_data *mc_saved_data,
- unsigned long *mc_saved_in_initrd,
- struct ucode_cpu_info *uci)
-{
- unsigned int size = end - start + 1;
- struct cpio_data cd;
- long offset = 0;
-#ifdef CONFIG_X86_32
- char *p = (char *)__pa_nodebug(ucode_name);
-#else
- char *p = ucode_name;
-#endif
-
- cd.data = NULL;
- cd.size = 0;
-
- cd = find_cpio_data(p, (void *)start, size, &offset);
- if (!cd.data)
- return UCODE_ERROR;
-
-
- return get_matching_model_microcode(0, start, cd.data, cd.size,
- mc_saved_data, mc_saved_in_initrd,
- uci);
-}
-
-/*
- * Print ucode update info.
- */
-static void
-print_ucode_info(struct ucode_cpu_info *uci, unsigned int date)
-{
- int cpu = smp_processor_id();
-
- pr_info("CPU%d microcode updated early to revision 0x%x, date = %04x-%02x-%02x\n",
- cpu,
- uci->cpu_sig.rev,
- date & 0xffff,
- date >> 24,
- (date >> 16) & 0xff);
-}
-
-#ifdef CONFIG_X86_32
-
-static int delay_ucode_info;
-static int current_mc_date;
-
-/*
- * Print early updated ucode info after printk works. This is delayed info dump.
- */
-void show_ucode_info_early(void)
-{
- struct ucode_cpu_info uci;
-
- if (delay_ucode_info) {
- collect_cpu_info_early(&uci);
- print_ucode_info(&uci, current_mc_date);
- delay_ucode_info = 0;
- }
-}
-
-/*
- * At this point, we can not call printk() yet. Keep microcode patch number in
- * mc_saved_data.mc_saved and delay printing microcode info in
- * show_ucode_info_early() until printk() works.
- */
-static void print_ucode(struct ucode_cpu_info *uci)
-{
- struct microcode_intel *mc_intel;
- int *delay_ucode_info_p;
- int *current_mc_date_p;
-
- mc_intel = uci->mc;
- if (mc_intel == NULL)
- return;
-
- delay_ucode_info_p = (int *)__pa_nodebug(&delay_ucode_info);
- current_mc_date_p = (int *)__pa_nodebug(¤t_mc_date);
-
- *delay_ucode_info_p = 1;
- *current_mc_date_p = mc_intel->hdr.date;
-}
-#else
-
-/*
- * Flush global tlb. We only do this in x86_64 where paging has been enabled
- * already and PGE should be enabled as well.
- */
-static inline void flush_tlb_early(void)
-{
- __native_flush_tlb_global_irq_disabled();
-}
-
-static inline void print_ucode(struct ucode_cpu_info *uci)
-{
- struct microcode_intel *mc_intel;
-
- mc_intel = uci->mc;
- if (mc_intel == NULL)
- return;
-
- print_ucode_info(uci, mc_intel->hdr.date);
-}
-#endif
-
-static int apply_microcode_early(struct mc_saved_data *mc_saved_data,
- struct ucode_cpu_info *uci)
-{
- struct microcode_intel *mc_intel;
- unsigned int val[2];
-
- mc_intel = uci->mc;
- if (mc_intel == NULL)
- return 0;
-
- /* write microcode via MSR 0x79 */
- native_wrmsr(MSR_IA32_UCODE_WRITE,
- (unsigned long) mc_intel->bits,
- (unsigned long) mc_intel->bits >> 16 >> 16);
- native_wrmsr(MSR_IA32_UCODE_REV, 0, 0);
-
- /* As documented in the SDM: Do a CPUID 1 here */
- sync_core();
-
- /* get the current revision from MSR 0x8B */
- native_rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
- if (val[1] != mc_intel->hdr.rev)
- return -1;
-
-#ifdef CONFIG_X86_64
- /* Flush global tlb. This is precaution. */
- flush_tlb_early();
-#endif
- uci->cpu_sig.rev = val[1];
-
- print_ucode(uci);
-
- return 0;
-}
-
-/*
- * This function converts microcode patch offsets previously stored in
- * mc_saved_in_initrd to pointers and stores the pointers in mc_saved_data.
- */
-int __init save_microcode_in_initrd_intel(void)
-{
- unsigned int count = mc_saved_data.mc_saved_count;
- struct microcode_intel *mc_saved[MAX_UCODE_COUNT];
- int ret = 0;
-
- if (count == 0)
- return ret;
-
- microcode_pointer(mc_saved, mc_saved_in_initrd, initrd_start, count);
- ret = save_microcode(&mc_saved_data, mc_saved, count);
- if (ret)
- pr_err("Cannot save microcode patches from initrd.\n");
-
- show_saved_mc();
-
- return ret;
-}
-
-static void __init
-_load_ucode_intel_bsp(struct mc_saved_data *mc_saved_data,
- unsigned long *mc_saved_in_initrd,
- unsigned long initrd_start_early,
- unsigned long initrd_end_early,
- struct ucode_cpu_info *uci)
-{
- collect_cpu_info_early(uci);
- scan_microcode(initrd_start_early, initrd_end_early, mc_saved_data,
- mc_saved_in_initrd, uci);
- load_microcode(mc_saved_data, mc_saved_in_initrd,
- initrd_start_early, uci);
- apply_microcode_early(mc_saved_data, uci);
-}
-
-void __init
-load_ucode_intel_bsp(void)
-{
- u64 ramdisk_image, ramdisk_size;
- unsigned long initrd_start_early, initrd_end_early;
- struct ucode_cpu_info uci;
-#ifdef CONFIG_X86_32
- struct boot_params *boot_params_p;
-
- boot_params_p = (struct boot_params *)__pa_nodebug(&boot_params);
- ramdisk_image = boot_params_p->hdr.ramdisk_image;
- ramdisk_size = boot_params_p->hdr.ramdisk_size;
- initrd_start_early = ramdisk_image;
- initrd_end_early = initrd_start_early + ramdisk_size;
-
- _load_ucode_intel_bsp(
- (struct mc_saved_data *)__pa_nodebug(&mc_saved_data),
- (unsigned long *)__pa_nodebug(&mc_saved_in_initrd),
- initrd_start_early, initrd_end_early, &uci);
-#else
- ramdisk_image = boot_params.hdr.ramdisk_image;
- ramdisk_size = boot_params.hdr.ramdisk_size;
- initrd_start_early = ramdisk_image + PAGE_OFFSET;
- initrd_end_early = initrd_start_early + ramdisk_size;
-
- _load_ucode_intel_bsp(&mc_saved_data, mc_saved_in_initrd,
- initrd_start_early, initrd_end_early, &uci);
-#endif
-}
-
-void load_ucode_intel_ap(void)
-{
- struct mc_saved_data *mc_saved_data_p;
- struct ucode_cpu_info uci;
- unsigned long *mc_saved_in_initrd_p;
- unsigned long initrd_start_addr;
-#ifdef CONFIG_X86_32
- unsigned long *initrd_start_p;
-
- mc_saved_in_initrd_p =
- (unsigned long *)__pa_nodebug(mc_saved_in_initrd);
- mc_saved_data_p = (struct mc_saved_data *)__pa_nodebug(&mc_saved_data);
- initrd_start_p = (unsigned long *)__pa_nodebug(&initrd_start);
- initrd_start_addr = (unsigned long)__pa_nodebug(*initrd_start_p);
-#else
- mc_saved_data_p = &mc_saved_data;
- mc_saved_in_initrd_p = mc_saved_in_initrd;
- initrd_start_addr = initrd_start;
-#endif
-
- /*
- * If there is no valid ucode previously saved in memory, no need to
- * update ucode on this AP.
- */
- if (mc_saved_data_p->mc_saved_count == 0)
- return;
-
- collect_cpu_info_early(&uci);
- load_microcode(mc_saved_data_p, mc_saved_in_initrd_p,
- initrd_start_addr, &uci);
- apply_microcode_early(mc_saved_data_p, &uci);
-}
+++ /dev/null
-/*
- * Intel CPU Microcode Update Driver for Linux
- *
- * Copyright (C) 2012 Fenghua Yu <fenghua.yu@intel.com>
- * H Peter Anvin" <hpa@zytor.com>
- *
- * This driver allows to upgrade microcode on Intel processors
- * belonging to IA-32 family - PentiumPro, Pentium II,
- * Pentium III, Xeon, Pentium 4, etc.
- *
- * Reference: Section 8.11 of Volume 3a, IA-32 Intel? Architecture
- * Software Developer's Manual
- * Order Number 253668 or free download from:
- *
- * http://developer.intel.com/Assets/PDF/manual/253668.pdf
- *
- * For more information, go to http://www.urbanmyth.org/microcode
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
- */
-#include <linux/firmware.h>
-#include <linux/uaccess.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-
-#include <asm/microcode_intel.h>
-#include <asm/processor.h>
-#include <asm/msr.h>
-
-static inline int
-update_match_cpu(unsigned int csig, unsigned int cpf,
- unsigned int sig, unsigned int pf)
-{
- return (!sigmatch(sig, csig, pf, cpf)) ? 0 : 1;
-}
-
-int
-update_match_revision(struct microcode_header_intel *mc_header, int rev)
-{
- return (mc_header->rev <= rev) ? 0 : 1;
-}
-
-int microcode_sanity_check(void *mc, int print_err)
-{
- unsigned long total_size, data_size, ext_table_size;
- struct microcode_header_intel *mc_header = mc;
- struct extended_sigtable *ext_header = NULL;
- int sum, orig_sum, ext_sigcount = 0, i;
- struct extended_signature *ext_sig;
-
- total_size = get_totalsize(mc_header);
- data_size = get_datasize(mc_header);
-
- if (data_size + MC_HEADER_SIZE > total_size) {
- if (print_err)
- pr_err("error! Bad data size in microcode data file\n");
- return -EINVAL;
- }
-
- if (mc_header->ldrver != 1 || mc_header->hdrver != 1) {
- if (print_err)
- pr_err("error! Unknown microcode update format\n");
- return -EINVAL;
- }
- ext_table_size = total_size - (MC_HEADER_SIZE + data_size);
- if (ext_table_size) {
- if ((ext_table_size < EXT_HEADER_SIZE)
- || ((ext_table_size - EXT_HEADER_SIZE) % EXT_SIGNATURE_SIZE)) {
- if (print_err)
- pr_err("error! Small exttable size in microcode data file\n");
- return -EINVAL;
- }
- ext_header = mc + MC_HEADER_SIZE + data_size;
- if (ext_table_size != exttable_size(ext_header)) {
- if (print_err)
- pr_err("error! Bad exttable size in microcode data file\n");
- return -EFAULT;
- }
- ext_sigcount = ext_header->count;
- }
-
- /* check extended table checksum */
- if (ext_table_size) {
- int ext_table_sum = 0;
- int *ext_tablep = (int *)ext_header;
-
- i = ext_table_size / DWSIZE;
- while (i--)
- ext_table_sum += ext_tablep[i];
- if (ext_table_sum) {
- if (print_err)
- pr_warn("aborting, bad extended signature table checksum\n");
- return -EINVAL;
- }
- }
-
- /* calculate the checksum */
- orig_sum = 0;
- i = (MC_HEADER_SIZE + data_size) / DWSIZE;
- while (i--)
- orig_sum += ((int *)mc)[i];
- if (orig_sum) {
- if (print_err)
- pr_err("aborting, bad checksum\n");
- return -EINVAL;
- }
- if (!ext_table_size)
- return 0;
- /* check extended signature checksum */
- for (i = 0; i < ext_sigcount; i++) {
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE +
- EXT_SIGNATURE_SIZE * i;
- sum = orig_sum
- - (mc_header->sig + mc_header->pf + mc_header->cksum)
- + (ext_sig->sig + ext_sig->pf + ext_sig->cksum);
- if (sum) {
- if (print_err)
- pr_err("aborting, bad checksum\n");
- return -EINVAL;
- }
- }
- return 0;
-}
-EXPORT_SYMBOL_GPL(microcode_sanity_check);
-
-/*
- * return 0 - no update found
- * return 1 - found update
- */
-int get_matching_sig(unsigned int csig, int cpf, void *mc, int rev)
-{
- struct microcode_header_intel *mc_header = mc;
- struct extended_sigtable *ext_header;
- unsigned long total_size = get_totalsize(mc_header);
- int ext_sigcount, i;
- struct extended_signature *ext_sig;
-
- if (update_match_cpu(csig, cpf, mc_header->sig, mc_header->pf))
- return 1;
-
- /* Look for ext. headers: */
- if (total_size <= get_datasize(mc_header) + MC_HEADER_SIZE)
- return 0;
-
- ext_header = mc + get_datasize(mc_header) + MC_HEADER_SIZE;
- ext_sigcount = ext_header->count;
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
-
- for (i = 0; i < ext_sigcount; i++) {
- if (update_match_cpu(csig, cpf, ext_sig->sig, ext_sig->pf))
- return 1;
- ext_sig++;
- }
- return 0;
-}
-
-/*
- * return 0 - no update found
- * return 1 - found update
- */
-int get_matching_microcode(unsigned int csig, int cpf, void *mc, int rev)
-{
- struct microcode_header_intel *mc_header = mc;
-
- if (!update_match_revision(mc_header, rev))
- return 0;
-
- return get_matching_sig(csig, cpf, mc, rev);
-}
-EXPORT_SYMBOL_GPL(get_matching_microcode);
#include <linux/dma-mapping.h>
#include <linux/scatterlist.h>
#include <linux/string.h>
-#include <linux/init.h>
#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/reboot.h>
-#include <linux/init.h>
#include <linux/mc146818rtc.h>
#include <linux/module.h>
#include <linux/kallsyms.h>
_brk_start = 0;
}
+u64 relocated_ramdisk;
+
#ifdef CONFIG_BLK_DEV_INITRD
static u64 __init get_ramdisk_image(void)
u64 ramdisk_image = get_ramdisk_image();
u64 ramdisk_size = get_ramdisk_size();
u64 area_size = PAGE_ALIGN(ramdisk_size);
- u64 ramdisk_here;
unsigned long slop, clen, mapaddr;
char *p, *q;
/* We need to move the initrd down into directly mapped mem */
- ramdisk_here = memblock_find_in_range(0, PFN_PHYS(max_pfn_mapped),
- area_size, PAGE_SIZE);
+ relocated_ramdisk = memblock_find_in_range(0, PFN_PHYS(max_pfn_mapped),
+ area_size, PAGE_SIZE);
- if (!ramdisk_here)
+ if (!relocated_ramdisk)
panic("Cannot find place for new RAMDISK of size %lld\n",
- ramdisk_size);
+ ramdisk_size);
/* Note: this includes all the mem currently occupied by
the initrd, we rely on that fact to keep the data intact. */
- memblock_reserve(ramdisk_here, area_size);
- initrd_start = ramdisk_here + PAGE_OFFSET;
+ memblock_reserve(relocated_ramdisk, area_size);
+ initrd_start = relocated_ramdisk + PAGE_OFFSET;
initrd_end = initrd_start + ramdisk_size;
printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n",
- ramdisk_here, ramdisk_here + ramdisk_size - 1);
+ relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1);
q = (char *)initrd_start;
printk(KERN_INFO "Move RAMDISK from [mem %#010llx-%#010llx] to"
" [mem %#010llx-%#010llx]\n",
ramdisk_image, ramdisk_image + ramdisk_size - 1,
- ramdisk_here, ramdisk_here + ramdisk_size - 1);
+ relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1);
}
static void __init early_reserve_initrd(void)
case SETUP_DTB:
add_dtb(pa_data);
break;
+ case SETUP_EFI:
+ parse_efi_setup(pa_data, data_len);
+ break;
default:
break;
}
memblock_reserve(0, ALIGN(reserve_low, PAGE_SIZE));
}
+/*
+ * Dump out kernel offset information on panic.
+ */
+static int
+dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
+{
+ pr_emerg("Kernel Offset: 0x%lx from 0x%lx "
+ "(relocation range: 0x%lx-0x%lx)\n",
+ (unsigned long)&_text - __START_KERNEL, __START_KERNEL,
+ __START_KERNEL_map, MODULES_VADDR-1);
+
+ return 0;
+}
+
/*
* Determine if we were loaded by an EFI loader. If so, then we have also been
* passed the efi memmap, systab, etc., so we should use these data structures
iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
setup_memory_map();
parse_setup_data();
- /* update the e820_saved too */
- e820_reserve_setup_data();
copy_edd();
early_dump_pci_devices();
#endif
+ /* update the e820_saved too */
+ e820_reserve_setup_data();
finish_e820_parsing();
if (efi_enabled(EFI_BOOT))
}
#endif /* CONFIG_X86_32 */
+
+static struct notifier_block kernel_offset_notifier = {
+ .notifier_call = dump_kernel_offset
+};
+
+static int __init register_kernel_offset_dumper(void)
+{
+ atomic_notifier_chain_register(&panic_notifier_list,
+ &kernel_offset_notifier);
+ return 0;
+}
+__initcall(register_kernel_offset_dumper);
* The WBINVD is insufficient due to the spurious-wakeup
* case where we return around the loop.
*/
+ mb();
clflush(mwait_ptr);
+ mb();
__monitor(mwait_ptr, 0, 0);
mb();
__mwait(eax, 0);
exception_exit(prev_state); \
}
-DO_ERROR_INFO(X86_TRAP_DE, SIGFPE, "divide error", divide_error, FPE_INTDIV,
- regs->ip)
-DO_ERROR(X86_TRAP_OF, SIGSEGV, "overflow", overflow)
-DO_ERROR(X86_TRAP_BR, SIGSEGV, "bounds", bounds)
-DO_ERROR_INFO(X86_TRAP_UD, SIGILL, "invalid opcode", invalid_op, ILL_ILLOPN,
- regs->ip)
-DO_ERROR(X86_TRAP_OLD_MF, SIGFPE, "coprocessor segment overrun",
- coprocessor_segment_overrun)
-DO_ERROR(X86_TRAP_TS, SIGSEGV, "invalid TSS", invalid_TSS)
-DO_ERROR(X86_TRAP_NP, SIGBUS, "segment not present", segment_not_present)
+DO_ERROR_INFO(X86_TRAP_DE, SIGFPE, "divide error", divide_error, FPE_INTDIV, regs->ip )
+DO_ERROR (X86_TRAP_OF, SIGSEGV, "overflow", overflow )
+DO_ERROR (X86_TRAP_BR, SIGSEGV, "bounds", bounds )
+DO_ERROR_INFO(X86_TRAP_UD, SIGILL, "invalid opcode", invalid_op, ILL_ILLOPN, regs->ip )
+DO_ERROR (X86_TRAP_OLD_MF, SIGFPE, "coprocessor segment overrun", coprocessor_segment_overrun )
+DO_ERROR (X86_TRAP_TS, SIGSEGV, "invalid TSS", invalid_TSS )
+DO_ERROR (X86_TRAP_NP, SIGBUS, "segment not present", segment_not_present )
#ifdef CONFIG_X86_32
-DO_ERROR(X86_TRAP_SS, SIGBUS, "stack segment", stack_segment)
+DO_ERROR (X86_TRAP_SS, SIGBUS, "stack segment", stack_segment )
#endif
-DO_ERROR_INFO(X86_TRAP_AC, SIGBUS, "alignment check", alignment_check,
- BUS_ADRALN, 0)
+DO_ERROR_INFO(X86_TRAP_AC, SIGBUS, "alignment check", alignment_check, BUS_ADRALN, 0 )
#ifdef CONFIG_X86_64
/* Runs on IST stack */
#include <linux/clocksource.h>
#include <linux/percpu.h>
#include <linux/timex.h>
+#include <linux/static_key.h>
#include <asm/hpet.h>
#include <asm/timer.h>
erroneous rdtsc usage on !cpu_has_tsc processors */
static int __read_mostly tsc_disabled = -1;
+static struct static_key __use_tsc = STATIC_KEY_INIT;
+
int tsc_clocksource_reliable;
+
+/*
+ * Use a ring-buffer like data structure, where a writer advances the head by
+ * writing a new data entry and a reader advances the tail when it observes a
+ * new entry.
+ *
+ * Writers are made to wait on readers until there's space to write a new
+ * entry.
+ *
+ * This means that we can always use an {offset, mul} pair to compute a ns
+ * value that is 'roughly' in the right direction, even if we're writing a new
+ * {offset, mul} pair during the clock read.
+ *
+ * The down-side is that we can no longer guarantee strict monotonicity anymore
+ * (assuming the TSC was that to begin with), because while we compute the
+ * intersection point of the two clock slopes and make sure the time is
+ * continuous at the point of switching; we can no longer guarantee a reader is
+ * strictly before or after the switch point.
+ *
+ * It does mean a reader no longer needs to disable IRQs in order to avoid
+ * CPU-Freq updates messing with his times, and similarly an NMI reader will
+ * no longer run the risk of hitting half-written state.
+ */
+
+struct cyc2ns {
+ struct cyc2ns_data data[2]; /* 0 + 2*24 = 48 */
+ struct cyc2ns_data *head; /* 48 + 8 = 56 */
+ struct cyc2ns_data *tail; /* 56 + 8 = 64 */
+}; /* exactly fits one cacheline */
+
+static DEFINE_PER_CPU_ALIGNED(struct cyc2ns, cyc2ns);
+
+struct cyc2ns_data *cyc2ns_read_begin(void)
+{
+ struct cyc2ns_data *head;
+
+ preempt_disable();
+
+ head = this_cpu_read(cyc2ns.head);
+ /*
+ * Ensure we observe the entry when we observe the pointer to it.
+ * matches the wmb from cyc2ns_write_end().
+ */
+ smp_read_barrier_depends();
+ head->__count++;
+ barrier();
+
+ return head;
+}
+
+void cyc2ns_read_end(struct cyc2ns_data *head)
+{
+ barrier();
+ /*
+ * If we're the outer most nested read; update the tail pointer
+ * when we're done. This notifies possible pending writers
+ * that we've observed the head pointer and that the other
+ * entry is now free.
+ */
+ if (!--head->__count) {
+ /*
+ * x86-TSO does not reorder writes with older reads;
+ * therefore once this write becomes visible to another
+ * cpu, we must be finished reading the cyc2ns_data.
+ *
+ * matches with cyc2ns_write_begin().
+ */
+ this_cpu_write(cyc2ns.tail, head);
+ }
+ preempt_enable();
+}
+
+/*
+ * Begin writing a new @data entry for @cpu.
+ *
+ * Assumes some sort of write side lock; currently 'provided' by the assumption
+ * that cpufreq will call its notifiers sequentially.
+ */
+static struct cyc2ns_data *cyc2ns_write_begin(int cpu)
+{
+ struct cyc2ns *c2n = &per_cpu(cyc2ns, cpu);
+ struct cyc2ns_data *data = c2n->data;
+
+ if (data == c2n->head)
+ data++;
+
+ /* XXX send an IPI to @cpu in order to guarantee a read? */
+
+ /*
+ * When we observe the tail write from cyc2ns_read_end(),
+ * the cpu must be done with that entry and its safe
+ * to start writing to it.
+ */
+ while (c2n->tail == data)
+ cpu_relax();
+
+ return data;
+}
+
+static void cyc2ns_write_end(int cpu, struct cyc2ns_data *data)
+{
+ struct cyc2ns *c2n = &per_cpu(cyc2ns, cpu);
+
+ /*
+ * Ensure the @data writes are visible before we publish the
+ * entry. Matches the data-depencency in cyc2ns_read_begin().
+ */
+ smp_wmb();
+
+ ACCESS_ONCE(c2n->head) = data;
+}
+
+/*
+ * Accelerators for sched_clock()
+ * convert from cycles(64bits) => nanoseconds (64bits)
+ * basic equation:
+ * ns = cycles / (freq / ns_per_sec)
+ * ns = cycles * (ns_per_sec / freq)
+ * ns = cycles * (10^9 / (cpu_khz * 10^3))
+ * ns = cycles * (10^6 / cpu_khz)
+ *
+ * Then we use scaling math (suggested by george@mvista.com) to get:
+ * ns = cycles * (10^6 * SC / cpu_khz) / SC
+ * ns = cycles * cyc2ns_scale / SC
+ *
+ * And since SC is a constant power of two, we can convert the div
+ * into a shift.
+ *
+ * We can use khz divisor instead of mhz to keep a better precision, since
+ * cyc2ns_scale is limited to 10^6 * 2^10, which fits in 32 bits.
+ * (mathieu.desnoyers@polymtl.ca)
+ *
+ * -johnstul@us.ibm.com "math is hard, lets go shopping!"
+ */
+
+#define CYC2NS_SCALE_FACTOR 10 /* 2^10, carefully chosen */
+
+static void cyc2ns_data_init(struct cyc2ns_data *data)
+{
+ data->cyc2ns_mul = 1U << CYC2NS_SCALE_FACTOR;
+ data->cyc2ns_shift = CYC2NS_SCALE_FACTOR;
+ data->cyc2ns_offset = 0;
+ data->__count = 0;
+}
+
+static void cyc2ns_init(int cpu)
+{
+ struct cyc2ns *c2n = &per_cpu(cyc2ns, cpu);
+
+ cyc2ns_data_init(&c2n->data[0]);
+ cyc2ns_data_init(&c2n->data[1]);
+
+ c2n->head = c2n->data;
+ c2n->tail = c2n->data;
+}
+
+static inline unsigned long long cycles_2_ns(unsigned long long cyc)
+{
+ struct cyc2ns_data *data, *tail;
+ unsigned long long ns;
+
+ /*
+ * See cyc2ns_read_*() for details; replicated in order to avoid
+ * an extra few instructions that came with the abstraction.
+ * Notable, it allows us to only do the __count and tail update
+ * dance when its actually needed.
+ */
+
+ preempt_disable();
+ data = this_cpu_read(cyc2ns.head);
+ tail = this_cpu_read(cyc2ns.tail);
+
+ if (likely(data == tail)) {
+ ns = data->cyc2ns_offset;
+ ns += mul_u64_u32_shr(cyc, data->cyc2ns_mul, CYC2NS_SCALE_FACTOR);
+ } else {
+ data->__count++;
+
+ barrier();
+
+ ns = data->cyc2ns_offset;
+ ns += mul_u64_u32_shr(cyc, data->cyc2ns_mul, CYC2NS_SCALE_FACTOR);
+
+ barrier();
+
+ if (!--data->__count)
+ this_cpu_write(cyc2ns.tail, data);
+ }
+ preempt_enable();
+
+ return ns;
+}
+
+/* XXX surely we already have this someplace in the kernel?! */
+#define DIV_ROUND(n, d) (((n) + ((d) / 2)) / (d))
+
+static void set_cyc2ns_scale(unsigned long cpu_khz, int cpu)
+{
+ unsigned long long tsc_now, ns_now;
+ struct cyc2ns_data *data;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ sched_clock_idle_sleep_event();
+
+ if (!cpu_khz)
+ goto done;
+
+ data = cyc2ns_write_begin(cpu);
+
+ rdtscll(tsc_now);
+ ns_now = cycles_2_ns(tsc_now);
+
+ /*
+ * Compute a new multiplier as per the above comment and ensure our
+ * time function is continuous; see the comment near struct
+ * cyc2ns_data.
+ */
+ data->cyc2ns_mul = DIV_ROUND(NSEC_PER_MSEC << CYC2NS_SCALE_FACTOR, cpu_khz);
+ data->cyc2ns_shift = CYC2NS_SCALE_FACTOR;
+ data->cyc2ns_offset = ns_now -
+ mul_u64_u32_shr(tsc_now, data->cyc2ns_mul, CYC2NS_SCALE_FACTOR);
+
+ cyc2ns_write_end(cpu, data);
+
+done:
+ sched_clock_idle_wakeup_event(0);
+ local_irq_restore(flags);
+}
/*
* Scheduler clock - returns current time in nanosec units.
*/
u64 native_sched_clock(void)
{
- u64 this_offset;
+ u64 tsc_now;
/*
* Fall back to jiffies if there's no TSC available:
* very important for it to be as fast as the platform
* can achieve it. )
*/
- if (unlikely(tsc_disabled)) {
+ if (!static_key_false(&__use_tsc)) {
/* No locking but a rare wrong value is not a big deal: */
return (jiffies_64 - INITIAL_JIFFIES) * (1000000000 / HZ);
}
/* read the Time Stamp Counter: */
- rdtscll(this_offset);
+ rdtscll(tsc_now);
/* return the value in ns */
- return __cycles_2_ns(this_offset);
+ return cycles_2_ns(tsc_now);
}
/* We need to define a real function for sched_clock, to override the
unsigned long flags, latch, ms, fast_calibrate;
int hpet = is_hpet_enabled(), i, loopmin;
+ /* Calibrate TSC using MSR for Intel Atom SoCs */
+ local_irq_save(flags);
+ i = try_msr_calibrate_tsc(&fast_calibrate);
+ local_irq_restore(flags);
+ if (i >= 0) {
+ if (i == 0)
+ pr_warn("Fast TSC calibration using MSR failed\n");
+ return fast_calibrate;
+ }
+
local_irq_save(flags);
fast_calibrate = quick_pit_calibrate();
local_irq_restore(flags);
EXPORT_SYMBOL(recalibrate_cpu_khz);
-/* Accelerators for sched_clock()
- * convert from cycles(64bits) => nanoseconds (64bits)
- * basic equation:
- * ns = cycles / (freq / ns_per_sec)
- * ns = cycles * (ns_per_sec / freq)
- * ns = cycles * (10^9 / (cpu_khz * 10^3))
- * ns = cycles * (10^6 / cpu_khz)
- *
- * Then we use scaling math (suggested by george@mvista.com) to get:
- * ns = cycles * (10^6 * SC / cpu_khz) / SC
- * ns = cycles * cyc2ns_scale / SC
- *
- * And since SC is a constant power of two, we can convert the div
- * into a shift.
- *
- * We can use khz divisor instead of mhz to keep a better precision, since
- * cyc2ns_scale is limited to 10^6 * 2^10, which fits in 32 bits.
- * (mathieu.desnoyers@polymtl.ca)
- *
- * -johnstul@us.ibm.com "math is hard, lets go shopping!"
- */
-
-DEFINE_PER_CPU(unsigned long, cyc2ns);
-DEFINE_PER_CPU(unsigned long long, cyc2ns_offset);
-
-static void set_cyc2ns_scale(unsigned long cpu_khz, int cpu)
-{
- unsigned long long tsc_now, ns_now, *offset;
- unsigned long flags, *scale;
-
- local_irq_save(flags);
- sched_clock_idle_sleep_event();
-
- scale = &per_cpu(cyc2ns, cpu);
- offset = &per_cpu(cyc2ns_offset, cpu);
-
- rdtscll(tsc_now);
- ns_now = __cycles_2_ns(tsc_now);
-
- if (cpu_khz) {
- *scale = ((NSEC_PER_MSEC << CYC2NS_SCALE_FACTOR) +
- cpu_khz / 2) / cpu_khz;
- *offset = ns_now - mult_frac(tsc_now, *scale,
- (1UL << CYC2NS_SCALE_FACTOR));
- }
-
- sched_clock_idle_wakeup_event(0);
- local_irq_restore(flags);
-}
-
static unsigned long long cyc2ns_suspend;
void tsc_save_sched_clock_state(void)
{
- if (!sched_clock_stable)
+ if (!sched_clock_stable())
return;
cyc2ns_suspend = sched_clock();
unsigned long flags;
int cpu;
- if (!sched_clock_stable)
+ if (!sched_clock_stable())
return;
local_irq_save(flags);
- __this_cpu_write(cyc2ns_offset, 0);
+ /*
+ * We're comming out of suspend, there's no concurrency yet; don't
+ * bother being nice about the RCU stuff, just write to both
+ * data fields.
+ */
+
+ this_cpu_write(cyc2ns.data[0].cyc2ns_offset, 0);
+ this_cpu_write(cyc2ns.data[1].cyc2ns_offset, 0);
+
offset = cyc2ns_suspend - sched_clock();
- for_each_possible_cpu(cpu)
- per_cpu(cyc2ns_offset, cpu) = offset;
+ for_each_possible_cpu(cpu) {
+ per_cpu(cyc2ns.data[0].cyc2ns_offset, cpu) = offset;
+ per_cpu(cyc2ns.data[1].cyc2ns_offset, cpu) = offset;
+ }
local_irq_restore(flags);
}
{
if (!tsc_unstable) {
tsc_unstable = 1;
- sched_clock_stable = 0;
+ clear_sched_clock_stable();
disable_sched_clock_irqtime();
pr_info("Marking TSC unstable due to %s\n", reason);
/* Change only the rating, when not registered */
* speed as the bootup CPU. (cpufreq notifiers will fix this
* up if their speed diverges)
*/
- for_each_possible_cpu(cpu)
+ for_each_possible_cpu(cpu) {
+ cyc2ns_init(cpu);
set_cyc2ns_scale(cpu_khz, cpu);
+ }
if (tsc_disabled > 0)
return;
/* now allow native_sched_clock() to use rdtsc */
+
tsc_disabled = 0;
+ static_key_slow_inc(&__use_tsc);
if (!no_sched_irq_time)
enable_sched_clock_irqtime();
--- /dev/null
+/*
+ * tsc_msr.c - MSR based TSC calibration on Intel Atom SoC platforms.
+ *
+ * TSC in Intel Atom SoC runs at a constant rate which can be figured
+ * by this formula:
+ * <maximum core-clock to bus-clock ratio> * <maximum resolved frequency>
+ * See Intel 64 and IA-32 System Programming Guid section 16.12 and 30.11.5
+ * for details.
+ * Especially some Intel Atom SoCs don't have PIT(i8254) or HPET, so MSR
+ * based calibration is the only option.
+ *
+ *
+ * Copyright (C) 2013 Intel Corporation
+ * Author: Bin Gao <bin.gao@intel.com>
+ *
+ * This file is released under the GPLv2.
+ */
+
+#include <linux/kernel.h>
+#include <asm/processor.h>
+#include <asm/setup.h>
+#include <asm/apic.h>
+#include <asm/param.h>
+
+/* CPU reference clock frequency: in KHz */
+#define FREQ_83 83200
+#define FREQ_100 99840
+#define FREQ_133 133200
+#define FREQ_166 166400
+
+#define MAX_NUM_FREQS 8
+
+/*
+ * According to Intel 64 and IA-32 System Programming Guide,
+ * if MSR_PERF_STAT[31] is set, the maximum resolved bus ratio can be
+ * read in MSR_PLATFORM_ID[12:8], otherwise in MSR_PERF_STAT[44:40].
+ * Unfortunately some Intel Atom SoCs aren't quite compliant to this,
+ * so we need manually differentiate SoC families. This is what the
+ * field msr_plat does.
+ */
+struct freq_desc {
+ u8 x86_family; /* CPU family */
+ u8 x86_model; /* model */
+ u8 msr_plat; /* 1: use MSR_PLATFORM_INFO, 0: MSR_IA32_PERF_STATUS */
+ u32 freqs[MAX_NUM_FREQS];
+};
+
+static struct freq_desc freq_desc_tables[] = {
+ /* PNW */
+ { 6, 0x27, 0, { 0, 0, 0, 0, 0, FREQ_100, 0, FREQ_83 } },
+ /* CLV+ */
+ { 6, 0x35, 0, { 0, FREQ_133, 0, 0, 0, FREQ_100, 0, FREQ_83 } },
+ /* TNG */
+ { 6, 0x4a, 1, { 0, FREQ_100, FREQ_133, 0, 0, 0, 0, 0 } },
+ /* VLV2 */
+ { 6, 0x37, 1, { 0, FREQ_100, FREQ_133, FREQ_166, 0, 0, 0, 0 } },
+ /* ANN */
+ { 6, 0x5a, 1, { FREQ_83, FREQ_100, FREQ_133, FREQ_100, 0, 0, 0, 0 } },
+};
+
+static int match_cpu(u8 family, u8 model)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(freq_desc_tables); i++) {
+ if ((family == freq_desc_tables[i].x86_family) &&
+ (model == freq_desc_tables[i].x86_model))
+ return i;
+ }
+
+ return -1;
+}
+
+/* Map CPU reference clock freq ID(0-7) to CPU reference clock freq(KHz) */
+#define id_to_freq(cpu_index, freq_id) \
+ (freq_desc_tables[cpu_index].freqs[freq_id])
+
+/*
+ * Do MSR calibration only for known/supported CPUs.
+ * Return values:
+ * -1: CPU is unknown/unsupported for MSR based calibration
+ * 0: CPU is known/supported, but calibration failed
+ * 1: CPU is known/supported, and calibration succeeded
+ */
+int try_msr_calibrate_tsc(unsigned long *fast_calibrate)
+{
+ int cpu_index;
+ u32 lo, hi, ratio, freq_id, freq;
+
+ cpu_index = match_cpu(boot_cpu_data.x86, boot_cpu_data.x86_model);
+ if (cpu_index < 0)
+ return -1;
+
+ *fast_calibrate = 0;
+
+ if (freq_desc_tables[cpu_index].msr_plat) {
+ rdmsr(MSR_PLATFORM_INFO, lo, hi);
+ ratio = (lo >> 8) & 0x1f;
+ } else {
+ rdmsr(MSR_IA32_PERF_STATUS, lo, hi);
+ ratio = (hi >> 8) & 0x1f;
+ }
+ pr_info("Maximum core-clock to bus-clock ratio: 0x%x\n", ratio);
+
+ if (!ratio)
+ return 0;
+
+ /* Get FSB FREQ ID */
+ rdmsr(MSR_FSB_FREQ, lo, hi);
+ freq_id = lo & 0x7;
+ freq = id_to_freq(cpu_index, freq_id);
+ pr_info("Resolved frequency ID: %u, frequency: %u KHz\n",
+ freq_id, freq);
+ if (!freq)
+ return 0;
+
+ /* TSC frequency = maximum resolved freq * maximum resolved bus ratio */
+ *fast_calibrate = freq * ratio;
+ pr_info("TSC runs at %lu KHz\n", *fast_calibrate);
+
+#ifdef CONFIG_X86_LOCAL_APIC
+ lapic_timer_frequency = (freq * 1000) / HZ;
+ pr_info("lapic_timer_frequency = %d\n", lapic_timer_frequency);
+#endif
+
+ return 1;
+}
*/
#include <linux/spinlock.h>
#include <linux/kernel.h>
-#include <linux/init.h>
#include <linux/smp.h>
#include <linux/nmi.h>
#include <asm/tsc.h>
if (cpu_has_xsaveopt && eagerfpu != DISABLE)
eagerfpu = ENABLE;
+ if (pcntxt_mask & XSTATE_EAGER) {
+ if (eagerfpu == DISABLE) {
+ pr_err("eagerfpu not present, disabling some xstate features: 0x%llx\n",
+ pcntxt_mask & XSTATE_EAGER);
+ pcntxt_mask &= ~XSTATE_EAGER;
+ } else {
+ eagerfpu = ENABLE;
+ }
+ }
+
pr_info("enabled xstate_bv 0x%llx, cntxt size 0x%x\n",
pcntxt_mask, xstate_size);
}
vcpu->arch.apic_base = value;
/* update jump label if enable bit changes */
- if ((vcpu->arch.apic_base ^ value) & MSR_IA32_APICBASE_ENABLE) {
+ if ((old_value ^ value) & MSR_IA32_APICBASE_ENABLE) {
if (value & MSR_IA32_APICBASE_ENABLE)
static_key_slow_dec_deferred(&apic_hw_disabled);
else
30: shll $6,%ecx
addl %ecx,%edx
jmp 60f
-40: lea (%rdx,%rcx,8),%rdx
+40: leal (%rdx,%rcx,8),%edx
jmp 60f
50: movl %ecx,%edx
60: jmp copy_user_handle_tail /* ecx is zerorest also */
ENTRY(copy_user_generic_string)
CFI_STARTPROC
ASM_STAC
- andl %edx,%edx
- jz 4f
cmpl $8,%edx
jb 2f /* less than 8 bytes, go to byte copy loop */
ALIGN_DESTINATION
2: movl %edx,%ecx
3: rep
movsb
-4: xorl %eax,%eax
+ xorl %eax,%eax
ASM_CLAC
ret
.section .fixup,"ax"
-11: lea (%rdx,%rcx,8),%rcx
+11: leal (%rdx,%rcx,8),%ecx
12: movl %ecx,%edx /* ecx is zerorest also */
jmp copy_user_handle_tail
.previous
ENTRY(copy_user_enhanced_fast_string)
CFI_STARTPROC
ASM_STAC
- andl %edx,%edx
- jz 2f
movl %edx,%ecx
1: rep
movsb
-2: xorl %eax,%eax
+ xorl %eax,%eax
ASM_CLAC
ret
#include <linux/timex.h>
#include <linux/preempt.h>
#include <linux/delay.h>
-#include <linux/init.h>
#include <asm/processor.h>
#include <asm/delay.h>
17: vmovhps Mq,Vq (v1) | vmovhpd Mq,Vq (66),(v1)
18: Grp16 (1A)
19:
-1a:
-1b:
+1a: BNDCL Ev,Gv | BNDCU Ev,Gv | BNDMOV Gv,Ev | BNDLDX Gv,Ev,Gv
+1b: BNDCN Ev,Gv | BNDMOV Ev,Gv | BNDMK Gv,Ev | BNDSTX Ev,GV,Gv
1c:
1d:
1e:
/* Are we prepared to handle this kernel fault? */
if (fixup_exception(regs)) {
+ /*
+ * Any interrupt that takes a fault gets the fixup. This makes
+ * the below recursive fault logic only apply to a faults from
+ * task context.
+ */
+ if (in_interrupt())
+ return;
+
+ /*
+ * Per the above we're !in_interrupt(), aka. task context.
+ *
+ * In this case we need to make sure we're not recursively
+ * faulting through the emulate_vsyscall() logic.
+ */
if (current_thread_info()->sig_on_uaccess_error && signal) {
tsk->thread.trap_nr = X86_TRAP_PF;
tsk->thread.error_code = error_code | PF_USER;
/* XXX: hwpoison faults will set the wrong code. */
force_sig_info_fault(signal, si_code, address, tsk, 0);
}
+
+ /*
+ * Barring that, we can do the fixup and be happy.
+ */
return;
}
}
#endif
-/* x86_64 also uses this file */
-
-#ifdef HAVE_ARCH_HUGETLB_UNMAPPED_AREA
+#ifdef CONFIG_HUGETLB_PAGE
static unsigned long hugetlb_get_unmapped_area_bottomup(struct file *file,
unsigned long addr, unsigned long len,
unsigned long pgoff, unsigned long flags)
info.flags = 0;
info.length = len;
- info.low_limit = TASK_UNMAPPED_BASE;
+ info.low_limit = current->mm->mmap_legacy_base;
info.high_limit = TASK_SIZE;
info.align_mask = PAGE_MASK & ~huge_page_mask(h);
info.align_offset = 0;
return hugetlb_get_unmapped_area_topdown(file, addr, len,
pgoff, flags);
}
-
-#endif /*HAVE_ARCH_HUGETLB_UNMAPPED_AREA*/
+#endif /* CONFIG_HUGETLB_PAGE */
#ifdef CONFIG_X86_64
static __init int setup_hugepagesz(char *opt)
BUILD_BUG_ON(VMALLOC_START >= VMALLOC_END);
#undef high_memory
#undef __FIXADDR_TOP
+#ifdef CONFIG_RANDOMIZE_BASE
+ BUILD_BUG_ON(CONFIG_RANDOMIZE_BASE_MAX_OFFSET > KERNEL_IMAGE_SIZE);
+#endif
#ifdef CONFIG_HIGHMEM
BUG_ON(PKMAP_BASE + LAST_PKMAP*PAGE_SIZE > FIXADDR_START);
#include <linux/rculist.h>
#include <linux/spinlock.h>
#include <linux/hash.h>
-#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/uaccess.h>
*/
nd_pa = memblock_alloc_nid(nd_size, SMP_CACHE_BYTES, nid);
if (!nd_pa) {
- pr_err("Cannot find %zu bytes in node %d\n",
- nd_size, nid);
- return;
+ nd_pa = __memblock_alloc_base(nd_size, SMP_CACHE_BYTES,
+ MEMBLOCK_ALLOC_ACCESSIBLE);
+ if (!nd_pa) {
+ pr_err("Cannot find %zu bytes in node %d\n",
+ nd_size, nid);
+ return;
+ }
}
nd = __va(nd_pa);
#include <linux/kthread.h>
#include <linux/random.h>
#include <linux/kernel.h>
-#include <linux/init.h>
#include <linux/mm.h>
#include <asm/cacheflush.h>
*/
struct cpa_data {
unsigned long *vaddr;
+ pgd_t *pgd;
pgprot_t mask_set;
pgprot_t mask_clr;
int numpages;
return prot;
}
-/*
- * Lookup the page table entry for a virtual address. Return a pointer
- * to the entry and the level of the mapping.
- *
- * Note: We return pud and pmd either when the entry is marked large
- * or when the present bit is not set. Otherwise we would return a
- * pointer to a nonexisting mapping.
- */
-pte_t *lookup_address(unsigned long address, unsigned int *level)
+static pte_t *__lookup_address_in_pgd(pgd_t *pgd, unsigned long address,
+ unsigned int *level)
{
- pgd_t *pgd = pgd_offset_k(address);
pud_t *pud;
pmd_t *pmd;
return pte_offset_kernel(pmd, address);
}
+
+/*
+ * Lookup the page table entry for a virtual address. Return a pointer
+ * to the entry and the level of the mapping.
+ *
+ * Note: We return pud and pmd either when the entry is marked large
+ * or when the present bit is not set. Otherwise we would return a
+ * pointer to a nonexisting mapping.
+ */
+pte_t *lookup_address(unsigned long address, unsigned int *level)
+{
+ return __lookup_address_in_pgd(pgd_offset_k(address), address, level);
+}
EXPORT_SYMBOL_GPL(lookup_address);
+static pte_t *_lookup_address_cpa(struct cpa_data *cpa, unsigned long address,
+ unsigned int *level)
+{
+ if (cpa->pgd)
+ return __lookup_address_in_pgd(cpa->pgd + pgd_index(address),
+ address, level);
+
+ return lookup_address(address, level);
+}
+
/*
* This is necessary because __pa() does not work on some
* kinds of memory, like vmalloc() or the alloc_remap()
* Check for races, another CPU might have split this page
* up already:
*/
- tmp = lookup_address(address, &level);
+ tmp = _lookup_address_cpa(cpa, address, &level);
if (tmp != kpte)
goto out_unlock;
}
static int
-__split_large_page(pte_t *kpte, unsigned long address, struct page *base)
+__split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
+ struct page *base)
{
pte_t *pbase = (pte_t *)page_address(base);
unsigned long pfn, pfninc = 1;
* Check for races, another CPU might have split this page
* up for us already:
*/
- tmp = lookup_address(address, &level);
+ tmp = _lookup_address_cpa(cpa, address, &level);
if (tmp != kpte) {
spin_unlock(&pgd_lock);
return 1;
return 0;
}
-static int split_large_page(pte_t *kpte, unsigned long address)
+static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
+ unsigned long address)
{
struct page *base;
if (!base)
return -ENOMEM;
- if (__split_large_page(kpte, address, base))
+ if (__split_large_page(cpa, kpte, address, base))
__free_page(base);
return 0;
}
+static bool try_to_free_pte_page(pte_t *pte)
+{
+ int i;
+
+ for (i = 0; i < PTRS_PER_PTE; i++)
+ if (!pte_none(pte[i]))
+ return false;
+
+ free_page((unsigned long)pte);
+ return true;
+}
+
+static bool try_to_free_pmd_page(pmd_t *pmd)
+{
+ int i;
+
+ for (i = 0; i < PTRS_PER_PMD; i++)
+ if (!pmd_none(pmd[i]))
+ return false;
+
+ free_page((unsigned long)pmd);
+ return true;
+}
+
+static bool unmap_pte_range(pmd_t *pmd, unsigned long start, unsigned long end)
+{
+ pte_t *pte = pte_offset_kernel(pmd, start);
+
+ while (start < end) {
+ set_pte(pte, __pte(0));
+
+ start += PAGE_SIZE;
+ pte++;
+ }
+
+ if (try_to_free_pte_page((pte_t *)pmd_page_vaddr(*pmd))) {
+ pmd_clear(pmd);
+ return true;
+ }
+ return false;
+}
+
+static void __unmap_pmd_range(pud_t *pud, pmd_t *pmd,
+ unsigned long start, unsigned long end)
+{
+ if (unmap_pte_range(pmd, start, end))
+ if (try_to_free_pmd_page((pmd_t *)pud_page_vaddr(*pud)))
+ pud_clear(pud);
+}
+
+static void unmap_pmd_range(pud_t *pud, unsigned long start, unsigned long end)
+{
+ pmd_t *pmd = pmd_offset(pud, start);
+
+ /*
+ * Not on a 2MB page boundary?
+ */
+ if (start & (PMD_SIZE - 1)) {
+ unsigned long next_page = (start + PMD_SIZE) & PMD_MASK;
+ unsigned long pre_end = min_t(unsigned long, end, next_page);
+
+ __unmap_pmd_range(pud, pmd, start, pre_end);
+
+ start = pre_end;
+ pmd++;
+ }
+
+ /*
+ * Try to unmap in 2M chunks.
+ */
+ while (end - start >= PMD_SIZE) {
+ if (pmd_large(*pmd))
+ pmd_clear(pmd);
+ else
+ __unmap_pmd_range(pud, pmd, start, start + PMD_SIZE);
+
+ start += PMD_SIZE;
+ pmd++;
+ }
+
+ /*
+ * 4K leftovers?
+ */
+ if (start < end)
+ return __unmap_pmd_range(pud, pmd, start, end);
+
+ /*
+ * Try again to free the PMD page if haven't succeeded above.
+ */
+ if (!pud_none(*pud))
+ if (try_to_free_pmd_page((pmd_t *)pud_page_vaddr(*pud)))
+ pud_clear(pud);
+}
+
+static void unmap_pud_range(pgd_t *pgd, unsigned long start, unsigned long end)
+{
+ pud_t *pud = pud_offset(pgd, start);
+
+ /*
+ * Not on a GB page boundary?
+ */
+ if (start & (PUD_SIZE - 1)) {
+ unsigned long next_page = (start + PUD_SIZE) & PUD_MASK;
+ unsigned long pre_end = min_t(unsigned long, end, next_page);
+
+ unmap_pmd_range(pud, start, pre_end);
+
+ start = pre_end;
+ pud++;
+ }
+
+ /*
+ * Try to unmap in 1G chunks?
+ */
+ while (end - start >= PUD_SIZE) {
+
+ if (pud_large(*pud))
+ pud_clear(pud);
+ else
+ unmap_pmd_range(pud, start, start + PUD_SIZE);
+
+ start += PUD_SIZE;
+ pud++;
+ }
+
+ /*
+ * 2M leftovers?
+ */
+ if (start < end)
+ unmap_pmd_range(pud, start, end);
+
+ /*
+ * No need to try to free the PUD page because we'll free it in
+ * populate_pgd's error path
+ */
+}
+
+static int alloc_pte_page(pmd_t *pmd)
+{
+ pte_t *pte = (pte_t *)get_zeroed_page(GFP_KERNEL | __GFP_NOTRACK);
+ if (!pte)
+ return -1;
+
+ set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
+ return 0;
+}
+
+static int alloc_pmd_page(pud_t *pud)
+{
+ pmd_t *pmd = (pmd_t *)get_zeroed_page(GFP_KERNEL | __GFP_NOTRACK);
+ if (!pmd)
+ return -1;
+
+ set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
+ return 0;
+}
+
+static void populate_pte(struct cpa_data *cpa,
+ unsigned long start, unsigned long end,
+ unsigned num_pages, pmd_t *pmd, pgprot_t pgprot)
+{
+ pte_t *pte;
+
+ pte = pte_offset_kernel(pmd, start);
+
+ while (num_pages-- && start < end) {
+
+ /* deal with the NX bit */
+ if (!(pgprot_val(pgprot) & _PAGE_NX))
+ cpa->pfn &= ~_PAGE_NX;
+
+ set_pte(pte, pfn_pte(cpa->pfn >> PAGE_SHIFT, pgprot));
+
+ start += PAGE_SIZE;
+ cpa->pfn += PAGE_SIZE;
+ pte++;
+ }
+}
+
+static int populate_pmd(struct cpa_data *cpa,
+ unsigned long start, unsigned long end,
+ unsigned num_pages, pud_t *pud, pgprot_t pgprot)
+{
+ unsigned int cur_pages = 0;
+ pmd_t *pmd;
+
+ /*
+ * Not on a 2M boundary?
+ */
+ if (start & (PMD_SIZE - 1)) {
+ unsigned long pre_end = start + (num_pages << PAGE_SHIFT);
+ unsigned long next_page = (start + PMD_SIZE) & PMD_MASK;
+
+ pre_end = min_t(unsigned long, pre_end, next_page);
+ cur_pages = (pre_end - start) >> PAGE_SHIFT;
+ cur_pages = min_t(unsigned int, num_pages, cur_pages);
+
+ /*
+ * Need a PTE page?
+ */
+ pmd = pmd_offset(pud, start);
+ if (pmd_none(*pmd))
+ if (alloc_pte_page(pmd))
+ return -1;
+
+ populate_pte(cpa, start, pre_end, cur_pages, pmd, pgprot);
+
+ start = pre_end;
+ }
+
+ /*
+ * We mapped them all?
+ */
+ if (num_pages == cur_pages)
+ return cur_pages;
+
+ while (end - start >= PMD_SIZE) {
+
+ /*
+ * We cannot use a 1G page so allocate a PMD page if needed.
+ */
+ if (pud_none(*pud))
+ if (alloc_pmd_page(pud))
+ return -1;
+
+ pmd = pmd_offset(pud, start);
+
+ set_pmd(pmd, __pmd(cpa->pfn | _PAGE_PSE | massage_pgprot(pgprot)));
+
+ start += PMD_SIZE;
+ cpa->pfn += PMD_SIZE;
+ cur_pages += PMD_SIZE >> PAGE_SHIFT;
+ }
+
+ /*
+ * Map trailing 4K pages.
+ */
+ if (start < end) {
+ pmd = pmd_offset(pud, start);
+ if (pmd_none(*pmd))
+ if (alloc_pte_page(pmd))
+ return -1;
+
+ populate_pte(cpa, start, end, num_pages - cur_pages,
+ pmd, pgprot);
+ }
+ return num_pages;
+}
+
+static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,
+ pgprot_t pgprot)
+{
+ pud_t *pud;
+ unsigned long end;
+ int cur_pages = 0;
+
+ end = start + (cpa->numpages << PAGE_SHIFT);
+
+ /*
+ * Not on a Gb page boundary? => map everything up to it with
+ * smaller pages.
+ */
+ if (start & (PUD_SIZE - 1)) {
+ unsigned long pre_end;
+ unsigned long next_page = (start + PUD_SIZE) & PUD_MASK;
+
+ pre_end = min_t(unsigned long, end, next_page);
+ cur_pages = (pre_end - start) >> PAGE_SHIFT;
+ cur_pages = min_t(int, (int)cpa->numpages, cur_pages);
+
+ pud = pud_offset(pgd, start);
+
+ /*
+ * Need a PMD page?
+ */
+ if (pud_none(*pud))
+ if (alloc_pmd_page(pud))
+ return -1;
+
+ cur_pages = populate_pmd(cpa, start, pre_end, cur_pages,
+ pud, pgprot);
+ if (cur_pages < 0)
+ return cur_pages;
+
+ start = pre_end;
+ }
+
+ /* We mapped them all? */
+ if (cpa->numpages == cur_pages)
+ return cur_pages;
+
+ pud = pud_offset(pgd, start);
+
+ /*
+ * Map everything starting from the Gb boundary, possibly with 1G pages
+ */
+ while (end - start >= PUD_SIZE) {
+ set_pud(pud, __pud(cpa->pfn | _PAGE_PSE | massage_pgprot(pgprot)));
+
+ start += PUD_SIZE;
+ cpa->pfn += PUD_SIZE;
+ cur_pages += PUD_SIZE >> PAGE_SHIFT;
+ pud++;
+ }
+
+ /* Map trailing leftover */
+ if (start < end) {
+ int tmp;
+
+ pud = pud_offset(pgd, start);
+ if (pud_none(*pud))
+ if (alloc_pmd_page(pud))
+ return -1;
+
+ tmp = populate_pmd(cpa, start, end, cpa->numpages - cur_pages,
+ pud, pgprot);
+ if (tmp < 0)
+ return cur_pages;
+
+ cur_pages += tmp;
+ }
+ return cur_pages;
+}
+
+/*
+ * Restrictions for kernel page table do not necessarily apply when mapping in
+ * an alternate PGD.
+ */
+static int populate_pgd(struct cpa_data *cpa, unsigned long addr)
+{
+ pgprot_t pgprot = __pgprot(_KERNPG_TABLE);
+ bool allocd_pgd = false;
+ pgd_t *pgd_entry;
+ pud_t *pud = NULL; /* shut up gcc */
+ int ret;
+
+ pgd_entry = cpa->pgd + pgd_index(addr);
+
+ /*
+ * Allocate a PUD page and hand it down for mapping.
+ */
+ if (pgd_none(*pgd_entry)) {
+ pud = (pud_t *)get_zeroed_page(GFP_KERNEL | __GFP_NOTRACK);
+ if (!pud)
+ return -1;
+
+ set_pgd(pgd_entry, __pgd(__pa(pud) | _KERNPG_TABLE));
+ allocd_pgd = true;
+ }
+
+ pgprot_val(pgprot) &= ~pgprot_val(cpa->mask_clr);
+ pgprot_val(pgprot) |= pgprot_val(cpa->mask_set);
+
+ ret = populate_pud(cpa, addr, pgd_entry, pgprot);
+ if (ret < 0) {
+ unmap_pud_range(pgd_entry, addr,
+ addr + (cpa->numpages << PAGE_SHIFT));
+
+ if (allocd_pgd) {
+ /*
+ * If I allocated this PUD page, I can just as well
+ * free it in this error path.
+ */
+ pgd_clear(pgd_entry);
+ free_page((unsigned long)pud);
+ }
+ return ret;
+ }
+ cpa->numpages = ret;
+ return 0;
+}
+
static int __cpa_process_fault(struct cpa_data *cpa, unsigned long vaddr,
int primary)
{
+ if (cpa->pgd)
+ return populate_pgd(cpa, vaddr);
+
/*
* Ignore all non primary paths.
*/
else
address = *cpa->vaddr;
repeat:
- kpte = lookup_address(address, &level);
+ kpte = _lookup_address_cpa(cpa, address, &level);
if (!kpte)
return __cpa_process_fault(cpa, address, primary);
/*
* We have to split the large page:
*/
- err = split_large_page(kpte, address);
+ err = split_large_page(cpa, kpte, address);
if (!err) {
/*
* Do a global flush tlb after splitting the large page
int ret, cache, checkalias;
unsigned long baddr = 0;
+ memset(&cpa, 0, sizeof(cpa));
+
/*
* Check, if we are requested to change a not supported
* feature:
{
unsigned long tempaddr = (unsigned long) page_address(page);
struct cpa_data cpa = { .vaddr = &tempaddr,
+ .pgd = NULL,
.numpages = numpages,
.mask_set = __pgprot(_PAGE_PRESENT | _PAGE_RW),
.mask_clr = __pgprot(0),
{
unsigned long tempaddr = (unsigned long) page_address(page);
struct cpa_data cpa = { .vaddr = &tempaddr,
+ .pgd = NULL,
.numpages = numpages,
.mask_set = __pgprot(0),
.mask_clr = __pgprot(_PAGE_PRESENT | _PAGE_RW),
#endif /* CONFIG_DEBUG_PAGEALLOC */
+int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
+ unsigned numpages, unsigned long page_flags)
+{
+ int retval = -EINVAL;
+
+ struct cpa_data cpa = {
+ .vaddr = &address,
+ .pfn = pfn,
+ .pgd = pgd,
+ .numpages = numpages,
+ .mask_set = __pgprot(0),
+ .mask_clr = __pgprot(0),
+ .flags = 0,
+ };
+
+ if (!(__supported_pte_mask & _PAGE_NX))
+ goto out;
+
+ if (!(page_flags & _PAGE_NX))
+ cpa.mask_clr = __pgprot(_PAGE_NX);
+
+ cpa.mask_set = __pgprot(_PAGE_PRESENT | page_flags);
+
+ retval = __change_page_attr_set_clr(&cpa, 0);
+ __flush_tlb_all();
+
+out:
+ return retval;
+}
+
/*
* The testcases use internal knowledge of the implementation that shouldn't
* be exposed to the rest of the kernel. Include these directly here.
EMIT2(0x89, 0xd0); /* mov %edx,%eax */
break;
case BPF_S_ALU_MOD_K: /* A %= K; */
+ if (K == 1) {
+ CLEAR_A();
+ break;
+ }
EMIT2(0x31, 0xd2); /* xor %edx,%edx */
EMIT1(0xb9);EMIT(K, 4); /* mov imm32,%ecx */
EMIT2(0xf7, 0xf1); /* div %ecx */
EMIT2(0x89, 0xd0); /* mov %edx,%eax */
break;
- case BPF_S_ALU_DIV_K: /* A = reciprocal_divide(A, K); */
- EMIT3(0x48, 0x69, 0xc0); /* imul imm32,%rax,%rax */
- EMIT(K, 4);
- EMIT4(0x48, 0xc1, 0xe8, 0x20); /* shr $0x20,%rax */
+ case BPF_S_ALU_DIV_K: /* A /= K */
+ if (K == 1)
+ break;
+ EMIT2(0x31, 0xd2); /* xor %edx,%edx */
+ EMIT1(0xb9);EMIT(K, 4); /* mov imm32,%ecx */
+ EMIT2(0xf7, 0xf1); /* div %ecx */
break;
case BPF_S_ALU_AND_X:
seen |= SEEN_XREG;
#include <linux/delay.h>
#include <linux/dmi.h>
#include <linux/pci.h>
-#include <linux/init.h>
#include <linux/vgaarb.h>
#include <asm/pci_x86.h>
#include <asm/pci_x86.h>
#include <asm/hw_irq.h>
#include <asm/io_apic.h>
+#include <asm/intel-mid.h>
#define PCIE_CAP_OFFSET 0x100
irq_attr.ioapic = mp_find_ioapic(dev->irq);
irq_attr.ioapic_pin = dev->irq;
irq_attr.trigger = 1; /* level */
- irq_attr.polarity = 1; /* active low */
+ if (intel_mid_identify_cpu() == INTEL_MID_CPU_CHIP_TANGIER)
+ irq_attr.polarity = 0; /* active high */
+ else
+ irq_attr.polarity = 1; /* active low */
io_apic_set_pci_routing(&dev->dev, dev->irq, &irq_attr);
return 0;
* Bibo Mao <bibo.mao@intel.com>
* Chandramouli Narayanan <mouli@linux.intel.com>
* Huang Ying <ying.huang@intel.com>
+ * Copyright (C) 2013 SuSE Labs
+ * Borislav Petkov <bp@suse.de> - runtime services VA mapping
*
* Copied from efi_32.c to eliminate the duplicated code between EFI
* 32/64 support code. --ying 2007-10-26
#include <asm/x86_init.h>
#include <asm/rtc.h>
-#define EFI_DEBUG 1
+#define EFI_DEBUG
#define EFI_MIN_RESERVE 5120
{NULL_GUID, NULL, NULL},
};
+u64 efi_setup; /* efi setup_data physical address */
+
/*
* Returns 1 if 'facility' is enabled, 0 otherwise.
*/
}
early_param("efi_no_storage_paranoia", setup_storage_paranoia);
-
static efi_status_t virt_efi_get_time(efi_time_t *tm, efi_time_cap_t *tc)
{
unsigned long flags;
return 0;
}
-#if EFI_DEBUG
static void __init print_efi_memmap(void)
{
+#ifdef EFI_DEBUG
efi_memory_desc_t *md;
void *p;
int i;
md->phys_addr + (md->num_pages << EFI_PAGE_SHIFT),
(md->num_pages >> (20 - EFI_PAGE_SHIFT)));
}
-}
#endif /* EFI_DEBUG */
+}
void __init efi_reserve_boot_services(void)
{
* - Not within any part of the kernel
* - Not the bios reserved area
*/
- if ((start+size >= __pa_symbol(_text)
+ if ((start + size > __pa_symbol(_text)
&& start <= __pa_symbol(_end)) ||
!e820_all_mapped(start, start+size, E820_RAM) ||
memblock_is_region_reserved(start, size)) {
{
if (efi_enabled(EFI_64BIT)) {
efi_system_table_64_t *systab64;
+ struct efi_setup_data *data = NULL;
u64 tmp = 0;
+ if (efi_setup) {
+ data = early_memremap(efi_setup, sizeof(*data));
+ if (!data)
+ return -ENOMEM;
+ }
systab64 = early_ioremap((unsigned long)phys,
sizeof(*systab64));
if (systab64 == NULL) {
pr_err("Couldn't map the system table!\n");
+ if (data)
+ early_iounmap(data, sizeof(*data));
return -ENOMEM;
}
efi_systab.hdr = systab64->hdr;
- efi_systab.fw_vendor = systab64->fw_vendor;
- tmp |= systab64->fw_vendor;
+ efi_systab.fw_vendor = data ? (unsigned long)data->fw_vendor :
+ systab64->fw_vendor;
+ tmp |= data ? data->fw_vendor : systab64->fw_vendor;
efi_systab.fw_revision = systab64->fw_revision;
efi_systab.con_in_handle = systab64->con_in_handle;
tmp |= systab64->con_in_handle;
tmp |= systab64->stderr_handle;
efi_systab.stderr = systab64->stderr;
tmp |= systab64->stderr;
- efi_systab.runtime = (void *)(unsigned long)systab64->runtime;
- tmp |= systab64->runtime;
+ efi_systab.runtime = data ?
+ (void *)(unsigned long)data->runtime :
+ (void *)(unsigned long)systab64->runtime;
+ tmp |= data ? data->runtime : systab64->runtime;
efi_systab.boottime = (void *)(unsigned long)systab64->boottime;
tmp |= systab64->boottime;
efi_systab.nr_tables = systab64->nr_tables;
- efi_systab.tables = systab64->tables;
- tmp |= systab64->tables;
+ efi_systab.tables = data ? (unsigned long)data->tables :
+ systab64->tables;
+ tmp |= data ? data->tables : systab64->tables;
early_iounmap(systab64, sizeof(*systab64));
+ if (data)
+ early_iounmap(data, sizeof(*data));
#ifdef CONFIG_X86_32
if (tmp >> 32) {
pr_err("EFI data located above 4GB, disabling EFI.\n");
return 0;
}
+/*
+ * A number of config table entries get remapped to virtual addresses
+ * after entering EFI virtual mode. However, the kexec kernel requires
+ * their physical addresses therefore we pass them via setup_data and
+ * correct those entries to their respective physical addresses here.
+ *
+ * Currently only handles smbios which is necessary for some firmware
+ * implementation.
+ */
+static int __init efi_reuse_config(u64 tables, int nr_tables)
+{
+ int i, sz, ret = 0;
+ void *p, *tablep;
+ struct efi_setup_data *data;
+
+ if (!efi_setup)
+ return 0;
+
+ if (!efi_enabled(EFI_64BIT))
+ return 0;
+
+ data = early_memremap(efi_setup, sizeof(*data));
+ if (!data) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ if (!data->smbios)
+ goto out_memremap;
+
+ sz = sizeof(efi_config_table_64_t);
+
+ p = tablep = early_memremap(tables, nr_tables * sz);
+ if (!p) {
+ pr_err("Could not map Configuration table!\n");
+ ret = -ENOMEM;
+ goto out_memremap;
+ }
+
+ for (i = 0; i < efi.systab->nr_tables; i++) {
+ efi_guid_t guid;
+
+ guid = ((efi_config_table_64_t *)p)->guid;
+
+ if (!efi_guidcmp(guid, SMBIOS_TABLE_GUID))
+ ((efi_config_table_64_t *)p)->table = data->smbios;
+ p += sz;
+ }
+ early_iounmap(tablep, nr_tables * sz);
+
+out_memremap:
+ early_iounmap(data, sizeof(*data));
+out:
+ return ret;
+}
+
void __init efi_init(void)
{
efi_char16_t *c16;
set_bit(EFI_SYSTEM_TABLES, &x86_efi_facility);
+ efi.config_table = (unsigned long)efi.systab->tables;
+ efi.fw_vendor = (unsigned long)efi.systab->fw_vendor;
+ efi.runtime = (unsigned long)efi.systab->runtime;
+
/*
* Show what we know for posterity
*/
efi.systab->hdr.revision >> 16,
efi.systab->hdr.revision & 0xffff, vendor);
+ if (efi_reuse_config(efi.systab->tables, efi.systab->nr_tables))
+ return;
+
if (efi_config_init(arch_tables))
return;
return;
set_bit(EFI_RUNTIME_SERVICES, &x86_efi_facility);
}
-
if (efi_memmap_init())
return;
set_bit(EFI_MEMMAP, &x86_efi_facility);
-#if EFI_DEBUG
print_efi_memmap();
-#endif
}
void __init efi_late_init(void)
set_memory_uc(addr, npages);
}
-/*
- * This function will switch the EFI runtime services to virtual mode.
- * Essentially, look through the EFI memmap and map every region that
- * has the runtime attribute bit set in its memory descriptor and update
- * that memory descriptor with the virtual address obtained from ioremap().
- * This enables the runtime services to be called without having to
- * thunk back into physical mode for every invocation.
- */
-void __init efi_enter_virtual_mode(void)
+void __init old_map_region(efi_memory_desc_t *md)
{
- efi_memory_desc_t *md, *prev_md = NULL;
- efi_status_t status;
+ u64 start_pfn, end_pfn, end;
unsigned long size;
- u64 end, systab, start_pfn, end_pfn;
- void *p, *va, *new_memmap = NULL;
- int count = 0;
+ void *va;
- efi.systab = NULL;
+ start_pfn = PFN_DOWN(md->phys_addr);
+ size = md->num_pages << PAGE_SHIFT;
+ end = md->phys_addr + size;
+ end_pfn = PFN_UP(end);
- /*
- * We don't do virtual mode, since we don't do runtime services, on
- * non-native EFI
- */
+ if (pfn_range_is_mapped(start_pfn, end_pfn)) {
+ va = __va(md->phys_addr);
- if (!efi_is_native()) {
- efi_unmap_memmap();
- return;
- }
+ if (!(md->attribute & EFI_MEMORY_WB))
+ efi_memory_uc((u64)(unsigned long)va, size);
+ } else
+ va = efi_ioremap(md->phys_addr, size,
+ md->type, md->attribute);
+
+ md->virt_addr = (u64) (unsigned long) va;
+ if (!va)
+ pr_err("ioremap of 0x%llX failed!\n",
+ (unsigned long long)md->phys_addr);
+}
+
+/* Merge contiguous regions of the same type and attribute */
+static void __init efi_merge_regions(void)
+{
+ void *p;
+ efi_memory_desc_t *md, *prev_md = NULL;
- /* Merge contiguous regions of the same type and attribute */
for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
u64 prev_size;
md = p;
}
prev_md = md;
}
+}
+
+static void __init get_systab_virt_addr(efi_memory_desc_t *md)
+{
+ unsigned long size;
+ u64 end, systab;
+
+ size = md->num_pages << EFI_PAGE_SHIFT;
+ end = md->phys_addr + size;
+ systab = (u64)(unsigned long)efi_phys.systab;
+ if (md->phys_addr <= systab && systab < end) {
+ systab += md->virt_addr - md->phys_addr;
+ efi.systab = (efi_system_table_t *)(unsigned long)systab;
+ }
+}
+
+static int __init save_runtime_map(void)
+{
+ efi_memory_desc_t *md;
+ void *tmp, *p, *q = NULL;
+ int count = 0;
+
+ for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
+ md = p;
+
+ if (!(md->attribute & EFI_MEMORY_RUNTIME) ||
+ (md->type == EFI_BOOT_SERVICES_CODE) ||
+ (md->type == EFI_BOOT_SERVICES_DATA))
+ continue;
+ tmp = krealloc(q, (count + 1) * memmap.desc_size, GFP_KERNEL);
+ if (!tmp)
+ goto out;
+ q = tmp;
+
+ memcpy(q + count * memmap.desc_size, md, memmap.desc_size);
+ count++;
+ }
+
+ efi_runtime_map_setup(q, count, memmap.desc_size);
+
+ return 0;
+out:
+ kfree(q);
+ return -ENOMEM;
+}
+
+/*
+ * Map efi regions which were passed via setup_data. The virt_addr is a fixed
+ * addr which was used in first kernel of a kexec boot.
+ */
+static void __init efi_map_regions_fixed(void)
+{
+ void *p;
+ efi_memory_desc_t *md;
+
+ for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
+ md = p;
+ efi_map_region_fixed(md); /* FIXME: add error handling */
+ get_systab_virt_addr(md);
+ }
+
+}
+
+/*
+ * Map efi memory ranges for runtime serivce and update new_memmap with virtual
+ * addresses.
+ */
+static void * __init efi_map_regions(int *count)
+{
+ efi_memory_desc_t *md;
+ void *p, *tmp, *new_memmap = NULL;
for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
md = p;
continue;
}
- size = md->num_pages << EFI_PAGE_SHIFT;
- end = md->phys_addr + size;
+ efi_map_region(md);
+ get_systab_virt_addr(md);
- start_pfn = PFN_DOWN(md->phys_addr);
- end_pfn = PFN_UP(end);
- if (pfn_range_is_mapped(start_pfn, end_pfn)) {
- va = __va(md->phys_addr);
+ tmp = krealloc(new_memmap, (*count + 1) * memmap.desc_size,
+ GFP_KERNEL);
+ if (!tmp)
+ goto out;
+ new_memmap = tmp;
+ memcpy(new_memmap + (*count * memmap.desc_size), md,
+ memmap.desc_size);
+ (*count)++;
+ }
- if (!(md->attribute & EFI_MEMORY_WB))
- efi_memory_uc((u64)(unsigned long)va, size);
- } else
- va = efi_ioremap(md->phys_addr, size,
- md->type, md->attribute);
+ return new_memmap;
+out:
+ kfree(new_memmap);
+ return NULL;
+}
+
+/*
+ * This function will switch the EFI runtime services to virtual mode.
+ * Essentially, we look through the EFI memmap and map every region that
+ * has the runtime attribute bit set in its memory descriptor into the
+ * ->trampoline_pgd page table using a top-down VA allocation scheme.
+ *
+ * The old method which used to update that memory descriptor with the
+ * virtual address obtained from ioremap() is still supported when the
+ * kernel is booted with efi=old_map on its command line. Same old
+ * method enabled the runtime services to be called without having to
+ * thunk back into physical mode for every invocation.
+ *
+ * The new method does a pagetable switch in a preemption-safe manner
+ * so that we're in a different address space when calling a runtime
+ * function. For function arguments passing we do copy the PGDs of the
+ * kernel page table into ->trampoline_pgd prior to each call.
+ *
+ * Specially for kexec boot, efi runtime maps in previous kernel should
+ * be passed in via setup_data. In that case runtime ranges will be mapped
+ * to the same virtual addresses as the first kernel.
+ */
+void __init efi_enter_virtual_mode(void)
+{
+ efi_status_t status;
+ void *new_memmap = NULL;
+ int err, count = 0;
- md->virt_addr = (u64) (unsigned long) va;
+ efi.systab = NULL;
- if (!va) {
- pr_err("ioremap of 0x%llX failed!\n",
- (unsigned long long)md->phys_addr);
- continue;
- }
+ /*
+ * We don't do virtual mode, since we don't do runtime services, on
+ * non-native EFI
+ */
+ if (!efi_is_native()) {
+ efi_unmap_memmap();
+ return;
+ }
- systab = (u64) (unsigned long) efi_phys.systab;
- if (md->phys_addr <= systab && systab < end) {
- systab += md->virt_addr - md->phys_addr;
- efi.systab = (efi_system_table_t *) (unsigned long) systab;
+ if (efi_setup) {
+ efi_map_regions_fixed();
+ } else {
+ efi_merge_regions();
+ new_memmap = efi_map_regions(&count);
+ if (!new_memmap) {
+ pr_err("Error reallocating memory, EFI runtime non-functional!\n");
+ return;
}
- new_memmap = krealloc(new_memmap,
- (count + 1) * memmap.desc_size,
- GFP_KERNEL);
- memcpy(new_memmap + (count * memmap.desc_size), md,
- memmap.desc_size);
- count++;
}
+ err = save_runtime_map();
+ if (err)
+ pr_err("Error saving runtime map, efi runtime on kexec non-functional!!\n");
+
BUG_ON(!efi.systab);
- status = phys_efi_set_virtual_address_map(
- memmap.desc_size * count,
- memmap.desc_size,
- memmap.desc_version,
- (efi_memory_desc_t *)__pa(new_memmap));
+ efi_setup_page_tables();
+ efi_sync_low_kernel_mappings();
- if (status != EFI_SUCCESS) {
- pr_alert("Unable to switch EFI into virtual mode "
- "(status=%lx)!\n", status);
- panic("EFI call to SetVirtualAddressMap() failed!");
+ if (!efi_setup) {
+ status = phys_efi_set_virtual_address_map(
+ memmap.desc_size * count,
+ memmap.desc_size,
+ memmap.desc_version,
+ (efi_memory_desc_t *)__pa(new_memmap));
+
+ if (status != EFI_SUCCESS) {
+ pr_alert("Unable to switch EFI into virtual mode (status=%lx)!\n",
+ status);
+ panic("EFI call to SetVirtualAddressMap() failed!");
+ }
}
/*
efi.query_variable_info = virt_efi_query_variable_info;
efi.update_capsule = virt_efi_update_capsule;
efi.query_capsule_caps = virt_efi_query_capsule_caps;
- if (__supported_pte_mask & _PAGE_NX)
+
+ if (efi_enabled(EFI_OLD_MEMMAP) && (__supported_pte_mask & _PAGE_NX))
runtime_code_page_mkexec();
kfree(new_memmap);
return EFI_SUCCESS;
}
EXPORT_SYMBOL_GPL(efi_query_variable_store);
+
+static int __init parse_efi_cmdline(char *str)
+{
+ if (*str == '=')
+ str++;
+
+ if (!strncmp(str, "old_map", 7))
+ set_bit(EFI_OLD_MEMMAP, &x86_efi_facility);
+
+ return 0;
+}
+early_param("efi", parse_efi_cmdline);
* claim EFI runtime service handler exclusively and to duplicate a memory in
* low memory space say 0 - 3G.
*/
-
static unsigned long efi_rt_eflags;
+void efi_sync_low_kernel_mappings(void) {}
+void efi_setup_page_tables(void) {}
+
+void __init efi_map_region(efi_memory_desc_t *md)
+{
+ old_map_region(md);
+}
+
+void __init efi_map_region_fixed(efi_memory_desc_t *md) {}
+void __init parse_efi_setup(u64 phys_addr, u32 data_len) {}
+
void efi_call_phys_prelog(void)
{
struct desc_ptr gdt_descr;
#include <asm/efi.h>
#include <asm/cacheflush.h>
#include <asm/fixmap.h>
+#include <asm/realmode.h>
static pgd_t *save_pgd __initdata;
static unsigned long efi_flags __initdata;
+/*
+ * We allocate runtime services regions bottom-up, starting from -4G, i.e.
+ * 0xffff_ffff_0000_0000 and limit EFI VA mapping space to 64G.
+ */
+static u64 efi_va = -4 * (1UL << 30);
+#define EFI_VA_END (-68 * (1UL << 30))
+
+/*
+ * Scratch space used for switching the pagetable in the EFI stub
+ */
+struct efi_scratch {
+ u64 r15;
+ u64 prev_cr3;
+ pgd_t *efi_pgt;
+ bool use_pgd;
+};
+
static void __init early_code_mapping_set_exec(int executable)
{
efi_memory_desc_t *md;
int pgd;
int n_pgds;
+ if (!efi_enabled(EFI_OLD_MEMMAP))
+ return;
+
early_code_mapping_set_exec(1);
local_irq_save(efi_flags);
*/
int pgd;
int n_pgds = DIV_ROUND_UP((max_pfn << PAGE_SHIFT) , PGDIR_SIZE);
+
+ if (!efi_enabled(EFI_OLD_MEMMAP))
+ return;
+
for (pgd = 0; pgd < n_pgds; pgd++)
set_pgd(pgd_offset_k(pgd * PGDIR_SIZE), save_pgd[pgd]);
kfree(save_pgd);
early_code_mapping_set_exec(0);
}
+/*
+ * Add low kernel mappings for passing arguments to EFI functions.
+ */
+void efi_sync_low_kernel_mappings(void)
+{
+ unsigned num_pgds;
+ pgd_t *pgd = (pgd_t *)__va(real_mode_header->trampoline_pgd);
+
+ if (efi_enabled(EFI_OLD_MEMMAP))
+ return;
+
+ num_pgds = pgd_index(MODULES_END - 1) - pgd_index(PAGE_OFFSET);
+
+ memcpy(pgd + pgd_index(PAGE_OFFSET),
+ init_mm.pgd + pgd_index(PAGE_OFFSET),
+ sizeof(pgd_t) * num_pgds);
+}
+
+void efi_setup_page_tables(void)
+{
+ efi_scratch.efi_pgt = (pgd_t *)(unsigned long)real_mode_header->trampoline_pgd;
+
+ if (!efi_enabled(EFI_OLD_MEMMAP))
+ efi_scratch.use_pgd = true;
+}
+
+static void __init __map_region(efi_memory_desc_t *md, u64 va)
+{
+ pgd_t *pgd = (pgd_t *)__va(real_mode_header->trampoline_pgd);
+ unsigned long pf = 0;
+
+ if (!(md->attribute & EFI_MEMORY_WB))
+ pf |= _PAGE_PCD;
+
+ if (kernel_map_pages_in_pgd(pgd, md->phys_addr, va, md->num_pages, pf))
+ pr_warn("Error mapping PA 0x%llx -> VA 0x%llx!\n",
+ md->phys_addr, va);
+}
+
+void __init efi_map_region(efi_memory_desc_t *md)
+{
+ unsigned long size = md->num_pages << PAGE_SHIFT;
+ u64 pa = md->phys_addr;
+
+ if (efi_enabled(EFI_OLD_MEMMAP))
+ return old_map_region(md);
+
+ /*
+ * Make sure the 1:1 mappings are present as a catch-all for b0rked
+ * firmware which doesn't update all internal pointers after switching
+ * to virtual mode and would otherwise crap on us.
+ */
+ __map_region(md, md->phys_addr);
+
+ efi_va -= size;
+
+ /* Is PA 2M-aligned? */
+ if (!(pa & (PMD_SIZE - 1))) {
+ efi_va &= PMD_MASK;
+ } else {
+ u64 pa_offset = pa & (PMD_SIZE - 1);
+ u64 prev_va = efi_va;
+
+ /* get us the same offset within this 2M page */
+ efi_va = (efi_va & PMD_MASK) + pa_offset;
+
+ if (efi_va > prev_va)
+ efi_va -= PMD_SIZE;
+ }
+
+ if (efi_va < EFI_VA_END) {
+ pr_warn(FW_WARN "VA address range overflow!\n");
+ return;
+ }
+
+ /* Do the VA map */
+ __map_region(md, efi_va);
+ md->virt_addr = efi_va;
+}
+
+/*
+ * kexec kernel will use efi_map_region_fixed to map efi runtime memory ranges.
+ * md->virt_addr is the original virtual address which had been mapped in kexec
+ * 1st kernel.
+ */
+void __init efi_map_region_fixed(efi_memory_desc_t *md)
+{
+ __map_region(md, md->virt_addr);
+}
+
void __iomem *__init efi_ioremap(unsigned long phys_addr, unsigned long size,
u32 type, u64 attribute)
{
return (void __iomem *)__va(phys_addr);
}
+
+void __init parse_efi_setup(u64 phys_addr, u32 data_len)
+{
+ efi_setup = phys_addr + sizeof(struct setup_data);
+}
mov %rsi, %cr0; \
mov (%rsp), %rsp
+ /* stolen from gcc */
+ .macro FLUSH_TLB_ALL
+ movq %r15, efi_scratch(%rip)
+ movq %r14, efi_scratch+8(%rip)
+ movq %cr4, %r15
+ movq %r15, %r14
+ andb $0x7f, %r14b
+ movq %r14, %cr4
+ movq %r15, %cr4
+ movq efi_scratch+8(%rip), %r14
+ movq efi_scratch(%rip), %r15
+ .endm
+
+ .macro SWITCH_PGT
+ cmpb $0, efi_scratch+24(%rip)
+ je 1f
+ movq %r15, efi_scratch(%rip) # r15
+ # save previous CR3
+ movq %cr3, %r15
+ movq %r15, efi_scratch+8(%rip) # prev_cr3
+ movq efi_scratch+16(%rip), %r15 # EFI pgt
+ movq %r15, %cr3
+ 1:
+ .endm
+
+ .macro RESTORE_PGT
+ cmpb $0, efi_scratch+24(%rip)
+ je 2f
+ movq efi_scratch+8(%rip), %r15
+ movq %r15, %cr3
+ movq efi_scratch(%rip), %r15
+ FLUSH_TLB_ALL
+ 2:
+ .endm
+
ENTRY(efi_call0)
SAVE_XMM
subq $32, %rsp
+ SWITCH_PGT
call *%rdi
+ RESTORE_PGT
addq $32, %rsp
RESTORE_XMM
ret
SAVE_XMM
subq $32, %rsp
mov %rsi, %rcx
+ SWITCH_PGT
call *%rdi
+ RESTORE_PGT
addq $32, %rsp
RESTORE_XMM
ret
SAVE_XMM
subq $32, %rsp
mov %rsi, %rcx
+ SWITCH_PGT
call *%rdi
+ RESTORE_PGT
addq $32, %rsp
RESTORE_XMM
ret
subq $32, %rsp
mov %rcx, %r8
mov %rsi, %rcx
+ SWITCH_PGT
call *%rdi
+ RESTORE_PGT
addq $32, %rsp
RESTORE_XMM
ret
mov %r8, %r9
mov %rcx, %r8
mov %rsi, %rcx
+ SWITCH_PGT
call *%rdi
+ RESTORE_PGT
addq $32, %rsp
RESTORE_XMM
ret
mov %r8, %r9
mov %rcx, %r8
mov %rsi, %rcx
+ SWITCH_PGT
call *%rdi
+ RESTORE_PGT
addq $48, %rsp
RESTORE_XMM
ret
mov %r8, %r9
mov %rcx, %r8
mov %rsi, %rcx
+ SWITCH_PGT
call *%rdi
+ RESTORE_PGT
addq $48, %rsp
RESTORE_XMM
ret
ENDPROC(efi_call6)
+
+ .data
+ENTRY(efi_scratch)
+ .fill 3,8,0
+ .byte 0
-obj-$(CONFIG_X86_INTEL_MID) += intel-mid.o
-obj-$(CONFIG_X86_INTEL_MID) += intel_mid_vrtc.o
+obj-$(CONFIG_X86_INTEL_MID) += intel-mid.o intel_mid_vrtc.o mfld.o mrfl.o
obj-$(CONFIG_EARLY_PRINTK_INTEL_MID) += early_printk_intel_mid.o
+
# SFI specific code
ifdef CONFIG_X86_INTEL_MID
obj-$(CONFIG_SFI) += sfi.o device_libs/
int intr = get_gpio_by_name("thermal_int");
int intr2nd = get_gpio_by_name("thermal_alert");
- if (intr == -1 || intr2nd == -1)
+ if (intr < 0)
+ return NULL;
+ if (intr2nd < 0)
return NULL;
i2c_info->irq = intr + INTEL_MID_IRQ_OFFSET;
gb[i].gpio = get_gpio_by_name(gb[i].desc);
pr_debug("info[%2d]: name = %s, gpio = %d\n", i, gb[i].desc,
gb[i].gpio);
- if (gb[i].gpio == -1)
+ if (gb[i].gpio < 0)
continue;
if (i != good)
int intr = get_gpio_by_name("accel_int");
int intr2nd = get_gpio_by_name("accel_2");
- if (intr == -1 || intr2nd == -1)
+ if (intr < 0)
+ return NULL;
+ if (intr2nd < 0)
return NULL;
i2c_info->irq = intr + INTEL_MID_IRQ_OFFSET;
gpio_base = get_gpio_by_name(base_pin_name);
intr = get_gpio_by_name(intr_pin_name);
- if (gpio_base == -1)
+ if (gpio_base < 0)
return NULL;
max7315->gpio_base = gpio_base;
if (intr != -1) {
struct i2c_board_info *i2c_info = info;
int intr = get_gpio_by_name("mpu3050_int");
- if (intr == -1)
+ if (intr < 0)
return NULL;
i2c_info->irq = intr + INTEL_MID_IRQ_OFFSET;
static struct intel_pmic_gpio_platform_data pmic_gpio_pdata;
int gpio_base = get_gpio_by_name("pmic_gpio_base");
- if (gpio_base == -1)
+ if (gpio_base < 0)
gpio_base = 64;
pmic_gpio_pdata.gpio_base = gpio_base;
pmic_gpio_pdata.irq_base = gpio_base + INTEL_MID_IRQ_OFFSET;
gpio_base = get_gpio_by_name(base_pin_name);
intr = get_gpio_by_name(intr_pin_name);
- if (gpio_base == -1)
+ if (gpio_base < 0)
return NULL;
tca6416.gpio_base = gpio_base;
- if (intr != -1) {
+ if (intr >= 0) {
i2c_info->irq = intr + INTEL_MID_IRQ_OFFSET;
tca6416.irq_base = gpio_base + INTEL_MID_IRQ_OFFSET;
} else {
#include <linux/console.h>
#include <linux/kernel.h>
#include <linux/delay.h>
-#include <linux/init.h>
#include <linux/io.h>
#include <asm/fixmap.h>
#include <asm/apb_timer.h>
#include <asm/reboot.h>
+#include "intel_mid_weak_decls.h"
+
/*
* the clockevent devices on Moorestown/Medfield can be APBT or LAPIC clock,
* cmdline option x86_intel_mid_timer can be used to override the configuration
enum intel_mid_timer_options intel_mid_timer_options;
+/* intel_mid_ops to store sub arch ops */
+struct intel_mid_ops *intel_mid_ops;
+/* getter function for sub arch ops*/
+static void *(*get_intel_mid_ops[])(void) = INTEL_MID_OPS_INIT;
enum intel_mid_cpu_type __intel_mid_cpu_chip;
EXPORT_SYMBOL_GPL(__intel_mid_cpu_chip);
static void intel_mid_power_off(void)
{
-}
+};
static void intel_mid_reboot(void)
{
static unsigned long __init intel_mid_calibrate_tsc(void)
{
- unsigned long fast_calibrate;
- u32 lo, hi, ratio, fsb;
-
- rdmsr(MSR_IA32_PERF_STATUS, lo, hi);
- pr_debug("IA32 perf status is 0x%x, 0x%0x\n", lo, hi);
- ratio = (hi >> 8) & 0x1f;
- pr_debug("ratio is %d\n", ratio);
- if (!ratio) {
- pr_err("read a zero ratio, should be incorrect!\n");
- pr_err("force tsc ratio to 16 ...\n");
- ratio = 16;
- }
- rdmsr(MSR_FSB_FREQ, lo, hi);
- if ((lo & 0x7) == 0x7)
- fsb = PENWELL_FSB_FREQ_83SKU;
- else
- fsb = PENWELL_FSB_FREQ_100SKU;
- fast_calibrate = ratio * fsb;
- pr_debug("read penwell tsc %lu khz\n", fast_calibrate);
- lapic_timer_frequency = fsb * 1000 / HZ;
- /* mark tsc clocksource as reliable */
- set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
-
- if (fast_calibrate)
- return fast_calibrate;
-
return 0;
}
static void intel_mid_arch_setup(void)
{
- if (boot_cpu_data.x86 == 6 && boot_cpu_data.x86_model == 0x27)
- __intel_mid_cpu_chip = INTEL_MID_CPU_CHIP_PENWELL;
- else {
+ if (boot_cpu_data.x86 != 6) {
pr_err("Unknown Intel MID CPU (%d:%d), default to Penwell\n",
boot_cpu_data.x86, boot_cpu_data.x86_model);
__intel_mid_cpu_chip = INTEL_MID_CPU_CHIP_PENWELL;
+ goto out;
}
+
+ switch (boot_cpu_data.x86_model) {
+ case 0x35:
+ __intel_mid_cpu_chip = INTEL_MID_CPU_CHIP_CLOVERVIEW;
+ break;
+ case 0x3C:
+ case 0x4A:
+ __intel_mid_cpu_chip = INTEL_MID_CPU_CHIP_TANGIER;
+ break;
+ case 0x27:
+ default:
+ __intel_mid_cpu_chip = INTEL_MID_CPU_CHIP_PENWELL;
+ break;
+ }
+
+ if (__intel_mid_cpu_chip < MAX_CPU_OPS(get_intel_mid_ops))
+ intel_mid_ops = get_intel_mid_ops[__intel_mid_cpu_chip]();
+ else {
+ intel_mid_ops = get_intel_mid_ops[INTEL_MID_CPU_CHIP_PENWELL]();
+ pr_info("ARCH: Uknown SoC, assuming PENWELL!\n");
+ }
+
+out:
+ if (intel_mid_ops->arch_setup)
+ intel_mid_ops->arch_setup();
}
/* MID systems don't have i8042 controller */
--- /dev/null
+/*
+ * intel_mid_weak_decls.h: Weak declarations of intel-mid.c
+ *
+ * (C) Copyright 2013 Intel Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+
+
+/* __attribute__((weak)) makes these declarations overridable */
+/* For every CPU addition a new get_<cpuname>_ops interface needs
+ * to be added.
+ */
+extern void * __cpuinit get_penwell_ops(void) __attribute__((weak));
+extern void * __cpuinit get_cloverview_ops(void) __attribute__((weak));
+extern void * __init get_tangier_ops(void) __attribute__((weak));
--- /dev/null
+/*
+ * mfld.c: Intel Medfield platform setup code
+ *
+ * (C) Copyright 2013 Intel Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+
+#include <linux/init.h>
+
+#include <asm/apic.h>
+#include <asm/intel-mid.h>
+#include <asm/intel_mid_vrtc.h>
+
+#include "intel_mid_weak_decls.h"
+
+static void penwell_arch_setup(void);
+/* penwell arch ops */
+static struct intel_mid_ops penwell_ops = {
+ .arch_setup = penwell_arch_setup,
+};
+
+static void mfld_power_off(void)
+{
+}
+
+static unsigned long __init mfld_calibrate_tsc(void)
+{
+ unsigned long fast_calibrate;
+ u32 lo, hi, ratio, fsb;
+
+ rdmsr(MSR_IA32_PERF_STATUS, lo, hi);
+ pr_debug("IA32 perf status is 0x%x, 0x%0x\n", lo, hi);
+ ratio = (hi >> 8) & 0x1f;
+ pr_debug("ratio is %d\n", ratio);
+ if (!ratio) {
+ pr_err("read a zero ratio, should be incorrect!\n");
+ pr_err("force tsc ratio to 16 ...\n");
+ ratio = 16;
+ }
+ rdmsr(MSR_FSB_FREQ, lo, hi);
+ if ((lo & 0x7) == 0x7)
+ fsb = FSB_FREQ_83SKU;
+ else
+ fsb = FSB_FREQ_100SKU;
+ fast_calibrate = ratio * fsb;
+ pr_debug("read penwell tsc %lu khz\n", fast_calibrate);
+ lapic_timer_frequency = fsb * 1000 / HZ;
+ /* mark tsc clocksource as reliable */
+ set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+
+ if (fast_calibrate)
+ return fast_calibrate;
+
+ return 0;
+}
+
+static void __init penwell_arch_setup()
+{
+ x86_platform.calibrate_tsc = mfld_calibrate_tsc;
+ pm_power_off = mfld_power_off;
+}
+
+void * __cpuinit get_penwell_ops()
+{
+ return &penwell_ops;
+}
+
+void * __cpuinit get_cloverview_ops()
+{
+ return &penwell_ops;
+}
--- /dev/null
+/*
+ * mrfl.c: Intel Merrifield platform specific setup code
+ *
+ * (C) Copyright 2013 Intel Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+
+#include <linux/init.h>
+
+#include <asm/apic.h>
+#include <asm/intel-mid.h>
+
+#include "intel_mid_weak_decls.h"
+
+static unsigned long __init tangier_calibrate_tsc(void)
+{
+ unsigned long fast_calibrate;
+ u32 lo, hi, ratio, fsb, bus_freq;
+
+ /* *********************** */
+ /* Compute TSC:Ratio * FSB */
+ /* *********************** */
+
+ /* Compute Ratio */
+ rdmsr(MSR_PLATFORM_INFO, lo, hi);
+ pr_debug("IA32 PLATFORM_INFO is 0x%x : %x\n", hi, lo);
+
+ ratio = (lo >> 8) & 0xFF;
+ pr_debug("ratio is %d\n", ratio);
+ if (!ratio) {
+ pr_err("Read a zero ratio, force tsc ratio to 4 ...\n");
+ ratio = 4;
+ }
+
+ /* Compute FSB */
+ rdmsr(MSR_FSB_FREQ, lo, hi);
+ pr_debug("Actual FSB frequency detected by SOC 0x%x : %x\n",
+ hi, lo);
+
+ bus_freq = lo & 0x7;
+ pr_debug("bus_freq = 0x%x\n", bus_freq);
+
+ if (bus_freq == 0)
+ fsb = FSB_FREQ_100SKU;
+ else if (bus_freq == 1)
+ fsb = FSB_FREQ_100SKU;
+ else if (bus_freq == 2)
+ fsb = FSB_FREQ_133SKU;
+ else if (bus_freq == 3)
+ fsb = FSB_FREQ_167SKU;
+ else if (bus_freq == 4)
+ fsb = FSB_FREQ_83SKU;
+ else if (bus_freq == 5)
+ fsb = FSB_FREQ_400SKU;
+ else if (bus_freq == 6)
+ fsb = FSB_FREQ_267SKU;
+ else if (bus_freq == 7)
+ fsb = FSB_FREQ_333SKU;
+ else {
+ BUG();
+ pr_err("Invalid bus_freq! Setting to minimal value!\n");
+ fsb = FSB_FREQ_100SKU;
+ }
+
+ /* TSC = FSB Freq * Resolved HFM Ratio */
+ fast_calibrate = ratio * fsb;
+ pr_debug("calculate tangier tsc %lu KHz\n", fast_calibrate);
+
+ /* ************************************ */
+ /* Calculate Local APIC Timer Frequency */
+ /* ************************************ */
+ lapic_timer_frequency = (fsb * 1000) / HZ;
+
+ pr_debug("Setting lapic_timer_frequency = %d\n",
+ lapic_timer_frequency);
+
+ /* mark tsc clocksource as reliable */
+ set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+
+ if (fast_calibrate)
+ return fast_calibrate;
+
+ return 0;
+}
+
+static void __init tangier_arch_setup(void)
+{
+ x86_platform.calibrate_tsc = tangier_calibrate_tsc;
+}
+
+/* tangier arch ops */
+static struct intel_mid_ops tangier_ops = {
+ .arch_setup = tangier_arch_setup,
+};
+
+void * __cpuinit get_tangier_ops()
+{
+ return &tangier_ops;
+}
if (!strncmp(name, pentry->pin_name, SFI_NAME_LEN))
return pentry->pin_no;
}
- return -1;
+ return -EINVAL;
}
void __init intel_scu_device_register(struct platform_device *pdev)
sdev->modalias);
return;
}
- memcpy(new_dev, sdev, sizeof(*sdev));
+ *new_dev = *sdev;
spi_devs[spi_next_dev++] = new_dev;
}
idev->type);
return;
}
- memcpy(new_dev, idev, sizeof(*idev));
+ *new_dev = *idev;
i2c_bus[i2c_next_dev] = bus;
i2c_devs[i2c_next_dev++] = new_dev;
pr_debug("IPC bus, name = %16.16s, irq = 0x%2x\n",
pentry->name, pentry->irq);
pdata = intel_mid_sfi_get_pdata(dev, pentry);
+ if (IS_ERR(pdata))
+ return;
pdev = platform_device_alloc(pentry->name, 0);
if (pdev == NULL) {
spi_info.chip_select);
pdata = intel_mid_sfi_get_pdata(dev, &spi_info);
+ if (IS_ERR(pdata))
+ return;
spi_info.platform_data = pdata;
if (dev->delay)
i2c_info.addr);
pdata = intel_mid_sfi_get_pdata(dev, &i2c_info);
i2c_info.platform_data = pdata;
+ if (IS_ERR(pdata))
+ return;
if (dev->delay)
intel_scu_i2c_device_register(pentry->host_num, &i2c_info);
* so we have to enable them one by one here
*/
ioapic = mp_find_ioapic(irq);
- irq_attr.ioapic = ioapic;
- irq_attr.ioapic_pin = irq;
- irq_attr.trigger = 1;
- irq_attr.polarity = 1;
- io_apic_set_pci_routing(NULL, irq, &irq_attr);
- } else
+ if (ioapic >= 0) {
+ irq_attr.ioapic = ioapic;
+ irq_attr.ioapic_pin = irq;
+ irq_attr.trigger = 1;
+ if (intel_mid_identify_cpu() ==
+ INTEL_MID_CPU_CHIP_TANGIER) {
+ if (!strncmp(pentry->name,
+ "r69001-ts-i2c", 13))
+ /* active low */
+ irq_attr.polarity = 1;
+ else if (!strncmp(pentry->name,
+ "synaptics_3202", 14))
+ /* active low */
+ irq_attr.polarity = 1;
+ else if (irq == 41)
+ /* fast_int_1 */
+ irq_attr.polarity = 1;
+ else
+ /* active high */
+ irq_attr.polarity = 0;
+ } else {
+ /* PNW and CLV go with active low */
+ irq_attr.polarity = 1;
+ }
+ io_apic_set_pci_routing(NULL, irq, &irq_attr);
+ }
+ } else {
irq = 0; /* No irq */
+ }
dev = get_device_id(pentry->type, pentry->name);
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/delay.h>
-#include <linux/init.h>
#include <linux/pm.h>
#include <asm/io.h>
return;
}
-static inline unsigned long cycles_2_us(unsigned long long cyc)
+/*
+ * Not to be confused with cycles_2_ns() from tsc.c; this gives a relative
+ * number, not an absolute. It converts a duration in cycles to a duration in
+ * ns.
+ */
+static inline unsigned long long cycles_2_ns(unsigned long long cyc)
{
+ struct cyc2ns_data *data = cyc2ns_read_begin();
unsigned long long ns;
- unsigned long us;
- int cpu = smp_processor_id();
- ns = (cyc * per_cpu(cyc2ns, cpu)) >> CYC2NS_SCALE_FACTOR;
- us = ns / 1000;
- return us;
+ ns = mul_u64_u32_shr(cyc, data->cyc2ns_mul, data->cyc2ns_shift);
+
+ cyc2ns_read_end(data);
+ return ns;
+}
+
+/*
+ * The reverse of the above; converts a duration in ns to a duration in cycles.
+ */
+static inline unsigned long long ns_2_cycles(unsigned long long ns)
+{
+ struct cyc2ns_data *data = cyc2ns_read_begin();
+ unsigned long long cyc;
+
+ cyc = (ns << data->cyc2ns_shift) / data->cyc2ns_mul;
+
+ cyc2ns_read_end(data);
+ return cyc;
+}
+
+static inline unsigned long cycles_2_us(unsigned long long cyc)
+{
+ return cycles_2_ns(cyc) / NSEC_PER_USEC;
+}
+
+static inline cycles_t sec_2_cycles(unsigned long sec)
+{
+ return ns_2_cycles(sec * NSEC_PER_SEC);
+}
+
+static inline unsigned long long usec_2_cycles(unsigned long usec)
+{
+ return ns_2_cycles(usec * NSEC_PER_USEC);
}
/*
bcp, try);
}
-static inline cycles_t sec_2_cycles(unsigned long sec)
-{
- unsigned long ns;
- cycles_t cyc;
-
- ns = sec * 1000000000;
- cyc = (ns << CYC2NS_SCALE_FACTOR)/(per_cpu(cyc2ns, smp_processor_id()));
- return cyc;
-}
-
/*
* Our retries are blocked by all destination sw ack resources being
* in use, and a timeout is pending. In that case hardware immediately
{
}
-static inline unsigned long long usec_2_cycles(unsigned long microsec)
-{
- unsigned long ns;
- unsigned long long cyc;
-
- ns = microsec * 1000;
- cyc = (ns << CYC2NS_SCALE_FACTOR)/(per_cpu(cyc2ns, smp_processor_id()));
- return cyc;
-}
-
/*
* Display the statistics thru /proc/sgi_uv/ptc_statistics
* 'data' points to the cpu number
void __init setup_real_mode(void)
{
u16 real_mode_seg;
- u32 *rel;
+ const u32 *rel;
u32 count;
- u32 *ptr;
- u16 *seg;
- int i;
unsigned char *base;
+ unsigned long phys_base;
struct trampoline_header *trampoline_header;
size_t size = PAGE_ALIGN(real_mode_blob_end - real_mode_blob);
#ifdef CONFIG_X86_64
memcpy(base, real_mode_blob, size);
- real_mode_seg = __pa(base) >> 4;
+ phys_base = __pa(base);
+ real_mode_seg = phys_base >> 4;
+
rel = (u32 *) real_mode_relocs;
/* 16-bit segment relocations. */
- count = rel[0];
- rel = &rel[1];
- for (i = 0; i < count; i++) {
- seg = (u16 *) (base + rel[i]);
+ count = *rel++;
+ while (count--) {
+ u16 *seg = (u16 *) (base + *rel++);
*seg = real_mode_seg;
}
/* 32-bit linear relocations. */
- count = rel[i];
- rel = &rel[i + 1];
- for (i = 0; i < count; i++) {
- ptr = (u32 *) (base + rel[i]);
- *ptr += __pa(base);
+ count = *rel++;
+ while (count--) {
+ u32 *ptr = (u32 *) (base + *rel++);
+ *ptr += phys_base;
}
/* Must be perfomed *after* relocation. */
#include <linux/linkage.h>
-#include <linux/init.h>
#include <asm/segment.h>
#include <asm/page_types.h>
#include <asm/processor-flags.h>
*/
#include <linux/linkage.h>
-#include <linux/init.h>
#include <asm/segment.h>
#include <asm/page_types.h>
#include "realmode.h"
*/
#include <linux/linkage.h>
-#include <linux/init.h>
#include <asm/pgtable_types.h>
#include <asm/page_types.h>
#include <asm/msr.h>
348 i386 process_vm_writev sys_process_vm_writev compat_sys_process_vm_writev
349 i386 kcmp sys_kcmp
350 i386 finit_module sys_finit_module
+351 i386 sched_setattr sys_sched_setattr
+352 i386 sched_getattr sys_sched_getattr
311 64 process_vm_writev sys_process_vm_writev
312 common kcmp sys_kcmp
313 common finit_module sys_finit_module
+314 common sched_setattr sys_sched_setattr
+315 common sched_getattr sys_sched_getattr
#
# x32-specific system call numbers start at 512 to avoid cache impact
/*
* Check to see if a symbol lies in the .data..percpu section.
- * For some as yet not understood reason the "__init_begin"
- * symbol which immediately preceeds the .data..percpu section
- * also shows up as it it were part of it so we do an explict
- * check for that symbol name and ignore it.
+ *
+ * The linker incorrectly associates some symbols with the
+ * .data..percpu section so we also need to check the symbol
+ * name to make sure that we classify the symbol correctly.
+ *
+ * The GNU linker incorrectly associates:
+ * __init_begin
+ * __per_cpu_load
+ *
+ * The "gold" linker incorrectly associates:
+ * init_per_cpu__irq_stack_union
+ * init_per_cpu__gdt_page
*/
static int is_percpu_sym(ElfW(Sym) *sym, const char *symname)
{
return (sym->st_shndx == per_cpu_shndx) &&
- strcmp(symname, "__init_begin");
+ strcmp(symname, "__init_begin") &&
+ strcmp(symname, "__per_cpu_load") &&
+ strncmp(symname, "init_per_cpu_", 13);
}
ts->tv_nsec = 0;
do {
- seq = read_seqcount_begin_no_lockdep(>od->seq);
+ seq = raw_read_seqcount_begin(>od->seq);
mode = gtod->clock.vclock_mode;
ts->tv_sec = gtod->wall_time_sec;
ns = gtod->wall_time_snsec;
ts->tv_nsec = 0;
do {
- seq = read_seqcount_begin_no_lockdep(>od->seq);
+ seq = raw_read_seqcount_begin(>od->seq);
mode = gtod->clock.vclock_mode;
ts->tv_sec = gtod->monotonic_time_sec;
ns = gtod->monotonic_time_snsec;
{
unsigned long seq;
do {
- seq = read_seqcount_begin_no_lockdep(>od->seq);
+ seq = raw_read_seqcount_begin(>od->seq);
ts->tv_sec = gtod->wall_time_coarse.tv_sec;
ts->tv_nsec = gtod->wall_time_coarse.tv_nsec;
} while (unlikely(read_seqcount_retry(>od->seq, seq)));
{
unsigned long seq;
do {
- seq = read_seqcount_begin_no_lockdep(>od->seq);
+ seq = raw_read_seqcount_begin(>od->seq);
ts->tv_sec = gtod->monotonic_time_coarse.tv_sec;
ts->tv_nsec = gtod->monotonic_time_coarse.tv_nsec;
} while (unlikely(read_seqcount_retry(>od->seq, seq)));
#include <asm/page_types.h>
#include <linux/linkage.h>
-#include <linux/init.h>
__PAGE_ALIGNED_DATA
#include <asm/page_types.h>
#include <linux/linkage.h>
-#include <linux/init.h>
__PAGE_ALIGNED_DATA
#ifndef _XTENSA_SYSTEM_H
#define _XTENSA_SYSTEM_H
-#define smp_read_barrier_depends() do { } while(0)
-#define read_barrier_depends() do { } while(0)
-
#define mb() ({ __asm__ __volatile__("memw" : : : "memory"); })
#define rmb() barrier()
#define wmb() mb()
#ifdef CONFIG_SMP
#error smp_* not defined
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
#endif
-#define set_mb(var, value) do { var = value; mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* _XTENSA_SYSTEM_H */
#include <acpi/acpi_bus.h>
#include <linux/cper.h>
#include <linux/ratelimit.h>
+#include <linux/edac.h>
#include <asm/cpu.h>
#include <asm/mce.h>
u8 rev1[12];
};
+static int old_edac_report_status;
+
static u8 extlog_dsm_uuid[] = "663E35AF-CC10-41A4-88EA-5470AF055295";
/* L1 table related physical address */
rc = print_extlog_rcd(NULL, (struct acpi_generic_status *)elog_buf, cpu);
- return NOTIFY_DONE;
+ return NOTIFY_STOP;
}
static int extlog_get_dsm(acpi_handle handle, int rev, int func, u64 *ret)
u64 cap;
int rc;
- rc = -ENODEV;
+ if (get_edac_report_status() == EDAC_REPORTING_FORCE) {
+ pr_warn("Not loading eMCA, error reporting force-enabled through EDAC.\n");
+ return -EPERM;
+ }
+ rc = -ENODEV;
rdmsrl(MSR_IA32_MCG_CAP, cap);
if (!(cap & MCG_ELOG_P))
return rc;
if (elog_buf == NULL)
goto err_release_elog;
+ /*
+ * eMCA event report method has higher priority than EDAC method,
+ * unless EDAC event report method is mandatory.
+ */
+ old_edac_report_status = get_edac_report_status();
+ set_edac_report_status(EDAC_REPORTING_DISABLED);
mce_register_decode_chain(&extlog_mce_dec);
/* enable OS to be involved to take over management from BIOS */
((struct extlog_l1_head *)extlog_l1_addr)->flags |= FLAG_OS_OPTIN;
static void __exit extlog_exit(void)
{
+ set_edac_report_status(old_edac_report_status);
mce_unregister_decode_chain(&extlog_mce_dec);
((struct extlog_l1_head *)extlog_l1_addr)->flags &= ~FLAG_OS_OPTIN;
if (extlog_l1_addr)
{ "80860F14", (unsigned long)&byt_sdio_dev_desc },
{ "80860F41", (unsigned long)&byt_i2c_dev_desc },
{ "INT33B2", },
- { "INT33FC", },
{ "INT3430", (unsigned long)&lpt_dev_desc },
{ "INT3431", (unsigned long)&lpt_dev_desc },
CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
stop_critical_timings();
- __monitor((void *)¤t_thread_info()->flags, 0, 0);
- smp_mb();
- if (!need_resched())
- __mwait(power_saving_mwait_eax, 1);
+ mwait_idle_with_hints(power_saving_mwait_eax, 1);
start_critical_timings();
if (lapic_marked_unstable)
#include <linux/rculist.h>
#include <linux/interrupt.h>
#include <linux/debugfs.h>
+#include <asm/unaligned.h>
#include "apei-internal.h"
bit_offset = reg->bit_offset;
access_size_code = reg->access_width;
space_id = reg->space_id;
- /* Handle possible alignment issues */
- memcpy(paddr, ®->address, sizeof(*paddr));
+ *paddr = get_unaligned(®->address);
if (!*paddr) {
pr_warning(FW_BUG APEI_PFX
"Invalid physical address in GAR [0x%llx/%u/%u/%u/%u]\n",
#include <linux/delay.h>
#include <linux/mm.h>
#include <acpi/acpi.h>
+#include <asm/unaligned.h>
#include "apei-internal.h"
static void *einj_get_parameter_address(void)
{
int i;
- u64 paddrv4 = 0, paddrv5 = 0;
+ u64 pa_v4 = 0, pa_v5 = 0;
struct acpi_whea_header *entry;
entry = EINJ_TAB_ENTRY(einj_tab);
entry->instruction == ACPI_EINJ_WRITE_REGISTER &&
entry->register_region.space_id ==
ACPI_ADR_SPACE_SYSTEM_MEMORY)
- memcpy(&paddrv4, &entry->register_region.address,
- sizeof(paddrv4));
+ pa_v4 = get_unaligned(&entry->register_region.address);
if (entry->action == ACPI_EINJ_SET_ERROR_TYPE_WITH_ADDRESS &&
entry->instruction == ACPI_EINJ_WRITE_REGISTER &&
entry->register_region.space_id ==
ACPI_ADR_SPACE_SYSTEM_MEMORY)
- memcpy(&paddrv5, &entry->register_region.address,
- sizeof(paddrv5));
+ pa_v5 = get_unaligned(&entry->register_region.address);
entry++;
}
- if (paddrv5) {
+ if (pa_v5) {
struct set_error_type_with_address *v5param;
- v5param = acpi_os_map_memory(paddrv5, sizeof(*v5param));
+ v5param = acpi_os_map_memory(pa_v5, sizeof(*v5param));
if (v5param) {
acpi5 = 1;
- check_vendor_extension(paddrv5, v5param);
+ check_vendor_extension(pa_v5, v5param);
return v5param;
}
}
- if (param_extension && paddrv4) {
+ if (param_extension && pa_v4) {
struct einj_parameter *v4param;
- v4param = acpi_os_map_memory(paddrv4, sizeof(*v4param));
+ v4param = acpi_os_map_memory(pa_v4, sizeof(*v4param));
if (!v4param)
return NULL;
if (v4param->reserved1 || v4param->reserved2) {
return rc;
}
-static int __einj_error_inject(u32 type, u64 param1, u64 param2)
+static int __einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2,
+ u64 param3, u64 param4)
{
struct apei_exec_context ctx;
u64 val, trigger_paddr, timeout = FIRMWARE_TIMEOUT;
break;
}
v5param->flags = vendor_flags;
+ } else if (flags) {
+ v5param->flags = flags;
+ v5param->memory_address = param1;
+ v5param->memory_address_range = param2;
+ v5param->apicid = param3;
+ v5param->pcie_sbdf = param4;
} else {
switch (type) {
case ACPI_EINJ_PROCESSOR_CORRECTABLE:
}
/* Inject the specified hardware error */
-static int einj_error_inject(u32 type, u64 param1, u64 param2)
+static int einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2,
+ u64 param3, u64 param4)
{
int rc;
unsigned long pfn;
+ /* If user manually set "flags", make sure it is legal */
+ if (flags && (flags &
+ ~(SETWA_FLAGS_APICID|SETWA_FLAGS_MEM|SETWA_FLAGS_PCIE_SBDF)))
+ return -EINVAL;
+
/*
* We need extra sanity checks for memory errors.
* Other types leap directly to injection.
if (type & ACPI5_VENDOR_BIT) {
if (vendor_flags != SETWA_FLAGS_MEM)
goto inject;
- } else if (!(type & MEM_ERROR_MASK))
+ } else if (!(type & MEM_ERROR_MASK) && !(flags & SETWA_FLAGS_MEM))
goto inject;
/*
inject:
mutex_lock(&einj_mutex);
- rc = __einj_error_inject(type, param1, param2);
+ rc = __einj_error_inject(type, flags, param1, param2, param3, param4);
mutex_unlock(&einj_mutex);
return rc;
}
static u32 error_type;
+static u32 error_flags;
static u64 error_param1;
static u64 error_param2;
+static u64 error_param3;
+static u64 error_param4;
static struct dentry *einj_debug_dir;
static int available_error_type_show(struct seq_file *m, void *v)
if (!error_type)
return -EINVAL;
- return einj_error_inject(error_type, error_param1, error_param2);
+ return einj_error_inject(error_type, error_flags, error_param1, error_param2,
+ error_param3, error_param4);
}
DEFINE_SIMPLE_ATTRIBUTE(error_inject_fops, NULL,
rc = -ENOMEM;
einj_param = einj_get_parameter_address();
if ((param_extension || acpi5) && einj_param) {
+ fentry = debugfs_create_x32("flags", S_IRUSR | S_IWUSR,
+ einj_debug_dir, &error_flags);
+ if (!fentry)
+ goto err_unmap;
fentry = debugfs_create_x64("param1", S_IRUSR | S_IWUSR,
einj_debug_dir, &error_param1);
if (!fentry)
einj_debug_dir, &error_param2);
if (!fentry)
goto err_unmap;
+ fentry = debugfs_create_x64("param3", S_IRUSR | S_IWUSR,
+ einj_debug_dir, &error_param3);
+ if (!fentry)
+ goto err_unmap;
+ fentry = debugfs_create_x64("param4", S_IRUSR | S_IWUSR,
+ einj_debug_dir, &error_param4);
+ if (!fentry)
+ goto err_unmap;
fentry = debugfs_create_x32("notrigger", S_IRUSR | S_IWUSR,
einj_debug_dir, ¬rigger);
if (entries[i] == APEI_ERST_INVALID_RECORD_ID)
continue;
if (wpos != i)
- memcpy(&entries[wpos], &entries[i], sizeof(entries[i]));
+ entries[wpos] = entries[i];
wpos++;
}
erst_record_id_cache.len = wpos;
{
#ifdef CONFIG_ACPI_APEI_MEMORY_FAILURE
unsigned long pfn;
+ int flags = -1;
int sec_sev = ghes_severity(gdata->error_severity);
struct cper_sec_mem_err *mem_err;
mem_err = (struct cper_sec_mem_err *)(gdata + 1);
- if (sec_sev == GHES_SEV_CORRECTED &&
- (gdata->flags & CPER_SEC_ERROR_THRESHOLD_EXCEEDED) &&
- (mem_err->validation_bits & CPER_MEM_VALID_PA)) {
- pfn = mem_err->physical_addr >> PAGE_SHIFT;
- if (pfn_valid(pfn))
- memory_failure_queue(pfn, 0, MF_SOFT_OFFLINE);
- else if (printk_ratelimit())
- pr_warn(FW_WARN GHES_PFX
- "Invalid address in generic error data: %#llx\n",
- mem_err->physical_addr);
- }
- if (sev == GHES_SEV_RECOVERABLE &&
- sec_sev == GHES_SEV_RECOVERABLE &&
- mem_err->validation_bits & CPER_MEM_VALID_PA) {
- pfn = mem_err->physical_addr >> PAGE_SHIFT;
- memory_failure_queue(pfn, 0, 0);
+ if (!(mem_err->validation_bits & CPER_MEM_VALID_PA))
+ return;
+
+ pfn = mem_err->physical_addr >> PAGE_SHIFT;
+ if (!pfn_valid(pfn)) {
+ pr_warn_ratelimited(FW_WARN GHES_PFX
+ "Invalid address in generic error data: %#llx\n",
+ mem_err->physical_addr);
+ return;
}
+
+ /* iff following two events can be handled properly by now */
+ if (sec_sev == GHES_SEV_CORRECTED &&
+ (gdata->flags & CPER_SEC_ERROR_THRESHOLD_EXCEEDED))
+ flags = MF_SOFT_OFFLINE;
+ if (sev == GHES_SEV_RECOVERABLE && sec_sev == GHES_SEV_RECOVERABLE)
+ flags = 0;
+
+ if (flags != -1)
+ memory_failure_queue(pfn, 0, flags);
#endif
}
ghes_edac_report_mem_error(ghes, sev, mem_err);
#ifdef CONFIG_X86_MCE
- apei_mce_report_mem_error(sev == GHES_SEV_CORRECTED,
- mem_err);
+ apei_mce_report_mem_error(sev, mem_err);
#endif
ghes_handle_memory_failure(gdata, sev);
}
if (unlikely(!pr))
return -EINVAL;
- if (cx->entry_method == ACPI_CSTATE_FFH) {
- if (current_set_polling_and_test())
- return -EINVAL;
- }
-
lapic_timer_state_broadcast(pr, cx, 1);
acpi_idle_do_entry(cx);
if (unlikely(!pr))
return -EINVAL;
- if (cx->entry_method == ACPI_CSTATE_FFH) {
- if (current_set_polling_and_test())
- return -EINVAL;
- }
-
/*
* Must be done before busmaster disable as we might need to
* access HPET !
}
}
- if (cx->entry_method == ACPI_CSTATE_FFH) {
- if (current_set_polling_and_test())
- return -EINVAL;
- }
-
acpi_unlazy_tlb(smp_processor_id());
/* Tell the scheduler that we are going deep-idle: */
#include <linux/zorro.h>
-extern int m68k_realnum_memory;
-extern struct mem_info m68k_memory[NUM_MEMINFO];
-
#define Z2MINOR_COMBINED (0)
#define Z2MINOR_Z2ONLY (1)
#define Z2MINOR_CHIPONLY (2)
if ( test_bit( i, zorro_unused_z2ram ) )
{
z2_count++;
- z2ram_map[ z2ram_size++ ] =
- ZTWO_VADDR( Z2RAM_START ) + ( i << Z2RAM_CHUNKSHIFT );
+ z2ram_map[z2ram_size++] = (unsigned long)ZTWO_VADDR(Z2RAM_START) +
+ (i << Z2RAM_CHUNKSHIFT);
clear_bit( i, zorro_unused_z2ram );
}
}
select CLKSRC_MMIO
bool
+config SUN5I_HSTIMER
+ select CLKSRC_MMIO
+ bool
+
config VT8500_TIMER
bool
obj-$(CONFIG_ARCH_MXS) += mxs_timer.o
obj-$(CONFIG_ARCH_PRIMA2) += timer-prima2.o
obj-$(CONFIG_SUN4I_TIMER) += sun4i_timer.o
+obj-$(CONFIG_SUN5I_HSTIMER) += timer-sun5i.o
obj-$(CONFIG_ARCH_TEGRA) += tegra20_timer.o
obj-$(CONFIG_VT8500_TIMER) += vt8500_timer.o
obj-$(CONFIG_ARCH_NSPIRE) += zevio-timer.o
};
#ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK
-static u32 notrace gt_sched_clock_read(void)
+static u64 notrace gt_sched_clock_read(void)
{
return gt_counter_read();
}
writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
#ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK
- setup_sched_clock(gt_sched_clock_read, 32, gt_clk_rate);
+ sched_clock_register(gt_sched_clock_read, 64, gt_clk_rate);
#endif
clocksource_register_hz(>_clocksource, gt_clk_rate);
}
return;
}
-static const struct of_device_id bcm_timer_ids[] __initconst = {
- {.compatible = "brcm,kona-timer"},
- {.compatible = "bcm,kona-timer"}, /* deprecated name */
- {},
-};
-
static void __init kona_timers_init(struct device_node *node)
{
u32 freq;
* struct ttc_timer - This definition defines local timer structure
*
* @base_addr: Base address of timer
+ * @freq: Timer input clock frequency
* @clk: Associated clock source
* @clk_rate_change_nb Notifier block for clock rate changes
*/
struct ttc_timer {
void __iomem *base_addr;
+ unsigned long freq;
struct clk *clk;
struct notifier_block clk_rate_change_nb;
};
TTC_COUNT_VAL_OFFSET);
}
-static u32 notrace ttc_sched_clock_read(void)
+static u64 notrace ttc_sched_clock_read(void)
{
return __raw_readl(ttc_sched_clock_val_reg);
}
switch (mode) {
case CLOCK_EVT_MODE_PERIODIC:
- ttc_set_interval(timer,
- DIV_ROUND_CLOSEST(clk_get_rate(ttce->ttc.clk),
- PRESCALE * HZ));
+ ttc_set_interval(timer, DIV_ROUND_CLOSEST(ttce->ttc.freq,
+ PRESCALE * HZ));
break;
case CLOCK_EVT_MODE_ONESHOT:
case CLOCK_EVT_MODE_UNUSED:
return;
}
+ ttccs->ttc.freq = clk_get_rate(ttccs->ttc.clk);
+
ttccs->ttc.clk_rate_change_nb.notifier_call =
ttc_rate_change_clocksource_cb;
ttccs->ttc.clk_rate_change_nb.next = NULL;
__raw_writel(CNT_CNTRL_RESET,
ttccs->ttc.base_addr + TTC_CNT_CNTRL_OFFSET);
- err = clocksource_register_hz(&ttccs->cs,
- clk_get_rate(ttccs->ttc.clk) / PRESCALE);
+ err = clocksource_register_hz(&ttccs->cs, ttccs->ttc.freq / PRESCALE);
if (WARN_ON(err)) {
kfree(ttccs);
return;
}
ttc_sched_clock_val_reg = base + TTC_COUNT_VAL_OFFSET;
- setup_sched_clock(ttc_sched_clock_read, 16,
- clk_get_rate(ttccs->ttc.clk) / PRESCALE);
+ sched_clock_register(ttc_sched_clock_read, 16, ttccs->ttc.freq / PRESCALE);
}
static int ttc_rate_change_clockevent_cb(struct notifier_block *nb,
ndata->new_rate / PRESCALE);
local_irq_restore(flags);
+ /* update cached frequency */
+ ttc->freq = ndata->new_rate;
+
/* fall through */
}
case PRE_RATE_CHANGE:
if (clk_notifier_register(ttcce->ttc.clk,
&ttcce->ttc.clk_rate_change_nb))
pr_warn("Unable to register clock notifier.\n");
+ ttcce->ttc.freq = clk_get_rate(ttcce->ttc.clk);
ttcce->ttc.base_addr = base;
ttcce->ce.name = "ttc_clockevent";
__raw_writel(0x1, ttcce->ttc.base_addr + TTC_IER_OFFSET);
err = request_irq(irq, ttc_clock_event_interrupt,
- IRQF_DISABLED | IRQF_TIMER,
- ttcce->ce.name, ttcce);
+ IRQF_TIMER, ttcce->ce.name, ttcce);
if (WARN_ON(err)) {
kfree(ttcce);
return;
}
clockevents_config_and_register(&ttcce->ce,
- clk_get_rate(ttcce->ttc.clk) / PRESCALE, 1, 0xfffe);
+ ttcce->ttc.freq / PRESCALE, 1, 0xfffe);
}
/**
struct device_node *np;
const struct of_device_id *match;
clocksource_of_init_fn init_func;
+ unsigned clocksources = 0;
for_each_matching_node_and_match(np, __clksrc_of_table, &match) {
if (!of_device_is_available(np))
init_func = match->data;
init_func(np);
+ clocksources++;
}
+ if (!clocksources)
+ pr_crit("%s: no matching clocksources found\n", __func__);
}
static struct irqaction mfgptirq = {
.handler = mfgpt_tick,
- .flags = IRQF_DISABLED | IRQF_NOBALANCING | IRQF_TIMER | IRQF_SHARED,
+ .flags = IRQF_NOBALANCING | IRQF_TIMER | IRQF_SHARED,
.name = DRV_NAME,
};
dw_ced->irqaction.dev_id = &dw_ced->ced;
dw_ced->irqaction.irq = irq;
dw_ced->irqaction.flags = IRQF_TIMER | IRQF_IRQPOLL |
- IRQF_NOBALANCING |
- IRQF_DISABLED;
+ IRQF_NOBALANCING;
dw_ced->eoi = apbt_eoi;
err = setup_irq(irq, &dw_ced->irqaction);
static struct irqaction nmdk_timer_irq = {
.name = "Nomadik Timer Tick",
- .flags = IRQF_DISABLED | IRQF_TIMER,
+ .flags = IRQF_TIMER,
.handler = nmdk_timer_interrupt,
.dev_id = &nmdk_clkevt,
};
static struct irqaction samsung_clock_event_irq = {
.name = "samsung_time_irq",
- .flags = IRQF_DISABLED | IRQF_TIMER | IRQF_IRQPOLL,
+ .flags = IRQF_TIMER | IRQF_IRQPOLL,
.handler = samsung_clock_event_isr,
.dev_id = &time_event_device,
};
static void sh_cmt_clock_event_suspend(struct clock_event_device *ced)
{
- pm_genpd_syscore_poweroff(&ced_to_sh_cmt(ced)->pdev->dev);
+ struct sh_cmt_priv *p = ced_to_sh_cmt(ced);
+
+ pm_genpd_syscore_poweroff(&p->pdev->dev);
+ clk_unprepare(p->clk);
}
static void sh_cmt_clock_event_resume(struct clock_event_device *ced)
{
- pm_genpd_syscore_poweron(&ced_to_sh_cmt(ced)->pdev->dev);
+ struct sh_cmt_priv *p = ced_to_sh_cmt(ced);
+
+ clk_prepare(p->clk);
+ pm_genpd_syscore_poweron(&p->pdev->dev);
}
static void sh_cmt_register_clockevent(struct sh_cmt_priv *p,
p->irqaction.name = dev_name(&p->pdev->dev);
p->irqaction.handler = sh_cmt_interrupt;
p->irqaction.dev_id = p;
- p->irqaction.flags = IRQF_DISABLED | IRQF_TIMER | \
- IRQF_IRQPOLL | IRQF_NOBALANCING;
+ p->irqaction.flags = IRQF_TIMER | IRQF_IRQPOLL | IRQF_NOBALANCING;
/* get hold of clock */
p->clk = clk_get(&p->pdev->dev, "cmt_fck");
goto err2;
}
+ ret = clk_prepare(p->clk);
+ if (ret < 0)
+ goto err3;
+
if (res2 && (resource_size(res2) == 4)) {
/* assume both CMSTR and CMCSR to be 32-bit */
p->read_control = sh_cmt_read32;
cfg->clocksource_rating);
if (ret) {
dev_err(&p->pdev->dev, "registration failed\n");
- goto err3;
+ goto err4;
}
p->cs_enabled = false;
ret = setup_irq(irq, &p->irqaction);
if (ret) {
dev_err(&p->pdev->dev, "failed to request irq %d\n", irq);
- goto err3;
+ goto err4;
}
platform_set_drvdata(pdev, p);
return 0;
+err4:
+ clk_unprepare(p->clk);
err3:
clk_put(p->clk);
err2:
p->irqaction.handler = sh_mtu2_interrupt;
p->irqaction.dev_id = p;
p->irqaction.irq = irq;
- p->irqaction.flags = IRQF_DISABLED | IRQF_TIMER | \
- IRQF_IRQPOLL | IRQF_NOBALANCING;
+ p->irqaction.flags = IRQF_TIMER | IRQF_IRQPOLL | IRQF_NOBALANCING;
/* get hold of clock */
p->clk = clk_get(&p->pdev->dev, "mtu2_fck");
ret = sh_mtu2_setup(p, pdev);
if (ret) {
kfree(p);
- platform_set_drvdata(pdev, NULL);
pm_runtime_idle(&pdev->dev);
return ret;
}
p->irqaction.handler = sh_tmu_interrupt;
p->irqaction.dev_id = p;
p->irqaction.irq = irq;
- p->irqaction.flags = IRQF_DISABLED | IRQF_TIMER | \
- IRQF_IRQPOLL | IRQF_NOBALANCING;
+ p->irqaction.flags = IRQF_TIMER | IRQF_IRQPOLL | IRQF_NOBALANCING;
/* get hold of clock */
p->clk = clk_get(&p->pdev->dev, "tmu_fck");
ret = sh_tmu_setup(p, pdev);
if (ret) {
kfree(p);
- platform_set_drvdata(pdev, NULL);
pm_runtime_idle(&pdev->dev);
return ret;
}
static struct clock_event_device sun4i_clockevent = {
.name = "sun4i_tick",
- .rating = 300,
+ .rating = 350,
.features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT,
.set_mode = sun4i_clkevt_mode,
.set_next_event = sun4i_clkevt_next_event,
.dev_id = &sun4i_clockevent,
};
-static u32 sun4i_timer_sched_read(void)
+static u64 notrace sun4i_timer_sched_read(void)
{
return ~readl(timer_base + TIMER_CNTVAL_REG(1));
}
TIMER_CTL_CLK_SRC(TIMER_CTL_CLK_SRC_OSC24M),
timer_base + TIMER_CTL_REG(1));
- setup_sched_clock(sun4i_timer_sched_read, 32, rate);
+ sched_clock_register(sun4i_timer_sched_read, 32, rate);
clocksource_mmio_init(timer_base + TIMER_CNTVAL_REG(1), node->name,
- rate, 300, 32, clocksource_mmio_readl_down);
+ rate, 350, 32, clocksource_mmio_readl_down);
ticks_per_jiffy = DIV_ROUND_UP(rate, HZ);
val = readl(timer_base + TIMER_IRQ_EN_REG);
writel(val | TIMER_IRQ_EN(0), timer_base + TIMER_IRQ_EN_REG);
- sun4i_clockevent.cpumask = cpumask_of(0);
+ sun4i_clockevent.cpumask = cpu_possible_mask;
+ sun4i_clockevent.irq = irq;
clockevents_config_and_register(&sun4i_clockevent, rate,
TIMER_SYNC_TICKS, 0xffffffff);
static struct irqaction tegra_timer_irq = {
.name = "timer0",
- .flags = IRQF_DISABLED | IRQF_TIMER | IRQF_TRIGGER_HIGH,
+ .flags = IRQF_TIMER | IRQF_TRIGGER_HIGH,
.handler = tegra_timer_interrupt,
.dev_id = &tegra_clockevent,
};
static void __iomem *timer_base, *local_base;
static unsigned int timer_clk;
static bool timer25Mhz = true;
+static u32 enable_mask;
/*
* Number of timer ticks per jiffy.
/*
* Enable the timer.
*/
- local_timer_ctrl_clrset(TIMER0_RELOAD_EN,
- TIMER0_EN | TIMER0_DIV(TIMER_DIVIDER_SHIFT));
+ local_timer_ctrl_clrset(TIMER0_RELOAD_EN, enable_mask);
return 0;
}
/*
* Enable timer.
*/
- local_timer_ctrl_clrset(0, TIMER0_RELOAD_EN |
- TIMER0_EN |
- TIMER0_DIV(TIMER_DIVIDER_SHIFT));
+ local_timer_ctrl_clrset(0, TIMER0_RELOAD_EN | enable_mask);
} else {
/*
* Disable timer.
WARN_ON(!timer_base);
local_base = of_iomap(np, 1);
- if (timer25Mhz)
+ if (timer25Mhz) {
set = TIMER0_25MHZ;
- else
+ enable_mask = TIMER0_EN;
+ } else {
clr = TIMER0_25MHZ;
+ enable_mask = TIMER0_EN | TIMER0_DIV(TIMER_DIVIDER_SHIFT);
+ }
timer_ctrl_clrset(clr, set);
local_timer_ctrl_clrset(clr, set);
writel(0xffffffff, timer_base + TIMER0_VAL_OFF);
writel(0xffffffff, timer_base + TIMER0_RELOAD_OFF);
- timer_ctrl_clrset(0, TIMER0_EN | TIMER0_RELOAD_EN |
- TIMER0_DIV(TIMER_DIVIDER_SHIFT));
+ timer_ctrl_clrset(0, TIMER0_RELOAD_EN | enable_mask);
/*
* Set scale and timer for sched_clock.
/*
* Free-running clocksource handling.
*/
-static u32 notrace orion_read_sched_clock(void)
+static u64 notrace orion_read_sched_clock(void)
{
return ~readl(timer_base + TIMER0_VAL);
}
clocksource_mmio_init(timer_base + TIMER0_VAL, "orion_clocksource",
clk_get_rate(clk), 300, 32,
clocksource_mmio_readl_down);
- setup_sched_clock(orion_read_sched_clock, 32, clk_get_rate(clk));
+ sched_clock_register(orion_read_sched_clock, 32, clk_get_rate(clk));
/* setup timer1 as clockevent timer */
if (setup_irq(irq, &orion_clkevt_irq))
--- /dev/null
+/*
+ * Allwinner SoCs hstimer driver.
+ *
+ * Copyright (C) 2013 Maxime Ripard
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/clk.h>
+#include <linux/clockchips.h>
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/irqreturn.h>
+#include <linux/sched_clock.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+
+#define TIMER_IRQ_EN_REG 0x00
+#define TIMER_IRQ_EN(val) BIT(val)
+#define TIMER_IRQ_ST_REG 0x04
+#define TIMER_CTL_REG(val) (0x20 * (val) + 0x10)
+#define TIMER_CTL_ENABLE BIT(0)
+#define TIMER_CTL_RELOAD BIT(1)
+#define TIMER_CTL_CLK_PRES(val) (((val) & 0x7) << 4)
+#define TIMER_CTL_ONESHOT BIT(7)
+#define TIMER_INTVAL_LO_REG(val) (0x20 * (val) + 0x14)
+#define TIMER_INTVAL_HI_REG(val) (0x20 * (val) + 0x18)
+#define TIMER_CNTVAL_LO_REG(val) (0x20 * (val) + 0x1c)
+#define TIMER_CNTVAL_HI_REG(val) (0x20 * (val) + 0x20)
+
+#define TIMER_SYNC_TICKS 3
+
+static void __iomem *timer_base;
+static u32 ticks_per_jiffy;
+
+/*
+ * When we disable a timer, we need to wait at least for 2 cycles of
+ * the timer source clock. We will use for that the clocksource timer
+ * that is already setup and runs at the same frequency than the other
+ * timers, and we never will be disabled.
+ */
+static void sun5i_clkevt_sync(void)
+{
+ u32 old = readl(timer_base + TIMER_CNTVAL_LO_REG(1));
+
+ while ((old - readl(timer_base + TIMER_CNTVAL_LO_REG(1))) < TIMER_SYNC_TICKS)
+ cpu_relax();
+}
+
+static void sun5i_clkevt_time_stop(u8 timer)
+{
+ u32 val = readl(timer_base + TIMER_CTL_REG(timer));
+ writel(val & ~TIMER_CTL_ENABLE, timer_base + TIMER_CTL_REG(timer));
+
+ sun5i_clkevt_sync();
+}
+
+static void sun5i_clkevt_time_setup(u8 timer, u32 delay)
+{
+ writel(delay, timer_base + TIMER_INTVAL_LO_REG(timer));
+}
+
+static void sun5i_clkevt_time_start(u8 timer, bool periodic)
+{
+ u32 val = readl(timer_base + TIMER_CTL_REG(timer));
+
+ if (periodic)
+ val &= ~TIMER_CTL_ONESHOT;
+ else
+ val |= TIMER_CTL_ONESHOT;
+
+ writel(val | TIMER_CTL_ENABLE | TIMER_CTL_RELOAD,
+ timer_base + TIMER_CTL_REG(timer));
+}
+
+static void sun5i_clkevt_mode(enum clock_event_mode mode,
+ struct clock_event_device *clk)
+{
+ switch (mode) {
+ case CLOCK_EVT_MODE_PERIODIC:
+ sun5i_clkevt_time_stop(0);
+ sun5i_clkevt_time_setup(0, ticks_per_jiffy);
+ sun5i_clkevt_time_start(0, true);
+ break;
+ case CLOCK_EVT_MODE_ONESHOT:
+ sun5i_clkevt_time_stop(0);
+ sun5i_clkevt_time_start(0, false);
+ break;
+ case CLOCK_EVT_MODE_UNUSED:
+ case CLOCK_EVT_MODE_SHUTDOWN:
+ default:
+ sun5i_clkevt_time_stop(0);
+ break;
+ }
+}
+
+static int sun5i_clkevt_next_event(unsigned long evt,
+ struct clock_event_device *unused)
+{
+ sun5i_clkevt_time_stop(0);
+ sun5i_clkevt_time_setup(0, evt - TIMER_SYNC_TICKS);
+ sun5i_clkevt_time_start(0, false);
+
+ return 0;
+}
+
+static struct clock_event_device sun5i_clockevent = {
+ .name = "sun5i_tick",
+ .rating = 340,
+ .features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT,
+ .set_mode = sun5i_clkevt_mode,
+ .set_next_event = sun5i_clkevt_next_event,
+};
+
+
+static irqreturn_t sun5i_timer_interrupt(int irq, void *dev_id)
+{
+ struct clock_event_device *evt = (struct clock_event_device *)dev_id;
+
+ writel(0x1, timer_base + TIMER_IRQ_ST_REG);
+ evt->event_handler(evt);
+
+ return IRQ_HANDLED;
+}
+
+static struct irqaction sun5i_timer_irq = {
+ .name = "sun5i_timer0",
+ .flags = IRQF_TIMER | IRQF_IRQPOLL,
+ .handler = sun5i_timer_interrupt,
+ .dev_id = &sun5i_clockevent,
+};
+
+static u64 sun5i_timer_sched_read(void)
+{
+ return ~readl(timer_base + TIMER_CNTVAL_LO_REG(1));
+}
+
+static void __init sun5i_timer_init(struct device_node *node)
+{
+ unsigned long rate;
+ struct clk *clk;
+ int ret, irq;
+ u32 val;
+
+ timer_base = of_iomap(node, 0);
+ if (!timer_base)
+ panic("Can't map registers");
+
+ irq = irq_of_parse_and_map(node, 0);
+ if (irq <= 0)
+ panic("Can't parse IRQ");
+
+ clk = of_clk_get(node, 0);
+ if (IS_ERR(clk))
+ panic("Can't get timer clock");
+ clk_prepare_enable(clk);
+ rate = clk_get_rate(clk);
+
+ writel(~0, timer_base + TIMER_INTVAL_LO_REG(1));
+ writel(TIMER_CTL_ENABLE | TIMER_CTL_RELOAD,
+ timer_base + TIMER_CTL_REG(1));
+
+ sched_clock_register(sun5i_timer_sched_read, 32, rate);
+ clocksource_mmio_init(timer_base + TIMER_CNTVAL_LO_REG(1), node->name,
+ rate, 340, 32, clocksource_mmio_readl_down);
+
+ ticks_per_jiffy = DIV_ROUND_UP(rate, HZ);
+
+ ret = setup_irq(irq, &sun5i_timer_irq);
+ if (ret)
+ pr_warn("failed to setup irq %d\n", irq);
+
+ /* Enable timer0 interrupt */
+ val = readl(timer_base + TIMER_IRQ_EN_REG);
+ writel(val | TIMER_IRQ_EN(0), timer_base + TIMER_IRQ_EN_REG);
+
+ sun5i_clockevent.cpumask = cpu_possible_mask;
+ sun5i_clockevent.irq = irq;
+
+ clockevents_config_and_register(&sun5i_clockevent, rate,
+ TIMER_SYNC_TICKS, 0xffffffff);
+}
+CLOCKSOURCE_OF_DECLARE(sun5i_a13, "allwinner,sun5i-a13-hstimer",
+ sun5i_timer_init);
+CLOCKSOURCE_OF_DECLARE(sun7i_a20, "allwinner,sun7i-a20-hstimer",
+ sun5i_timer_init);
static struct irqaction irq = {
.name = "vt8500_timer",
- .flags = IRQF_DISABLED | IRQF_TIMER | IRQF_IRQPOLL,
+ .flags = IRQF_TIMER | IRQF_IRQPOLL,
.handler = vt8500_timer_interrupt,
.dev_id = &clockevent,
};
#include "amd64_edac.h"
#include <asm/amd_nb.h>
-static struct edac_pci_ctl_info *amd64_ctl_pci;
+static struct edac_pci_ctl_info *pci_ctl;
static int report_gart_errors;
module_param(report_gart_errors, int, 0644);
* scan the scrub rate mapping table for a close or matching bandwidth value to
* issue. If requested is too big, then use last maximum value found.
*/
-static int __amd64_set_scrub_rate(struct pci_dev *ctl, u32 new_bw, u32 min_rate)
+static int __set_scrub_rate(struct pci_dev *ctl, u32 new_bw, u32 min_rate)
{
u32 scrubval;
int i;
return 0;
}
-static int amd64_set_scrub_rate(struct mem_ctl_info *mci, u32 bw)
+static int set_scrub_rate(struct mem_ctl_info *mci, u32 bw)
{
struct amd64_pvt *pvt = mci->pvt_info;
u32 min_scrubrate = 0x5;
if (pvt->fam == 0x15 && pvt->model < 0x10)
f15h_select_dct(pvt, 0);
- return __amd64_set_scrub_rate(pvt->F3, bw, min_scrubrate);
+ return __set_scrub_rate(pvt->F3, bw, min_scrubrate);
}
-static int amd64_get_scrub_rate(struct mem_ctl_info *mci)
+static int get_scrub_rate(struct mem_ctl_info *mci)
{
struct amd64_pvt *pvt = mci->pvt_info;
u32 scrubval = 0;
* returns true if the SysAddr given by sys_addr matches the
* DRAM base/limit associated with node_id
*/
-static bool amd64_base_limit_match(struct amd64_pvt *pvt, u64 sys_addr,
- u8 nid)
+static bool base_limit_match(struct amd64_pvt *pvt, u64 sys_addr, u8 nid)
{
u64 addr;
if (intlv_en == 0) {
for (node_id = 0; node_id < DRAM_RANGES; node_id++) {
- if (amd64_base_limit_match(pvt, sys_addr, node_id))
+ if (base_limit_match(pvt, sys_addr, node_id))
goto found;
}
goto err_no_match;
}
/* sanity test for sys_addr */
- if (unlikely(!amd64_base_limit_match(pvt, sys_addr, node_id))) {
+ if (unlikely(!base_limit_match(pvt, sys_addr, node_id))) {
amd64_warn("%s: sys_addr 0x%llx falls outside base/limit address"
"range for node %d with node interleaving enabled.\n",
__func__, sys_addr, node_id);
* Determine if the DIMMs have ECC enabled. ECC is enabled ONLY if all the DIMMs
* are ECC capable.
*/
-static unsigned long amd64_determine_edac_cap(struct amd64_pvt *pvt)
+static unsigned long determine_edac_cap(struct amd64_pvt *pvt)
{
u8 bit;
unsigned long edac_cap = EDAC_FLAG_NONE;
return edac_cap;
}
-static void amd64_debug_display_dimm_sizes(struct amd64_pvt *, u8);
+static void debug_display_dimm_sizes(struct amd64_pvt *, u8);
-static void amd64_dump_dramcfg_low(struct amd64_pvt *pvt, u32 dclr, int chan)
+static void debug_dump_dramcfg_low(struct amd64_pvt *pvt, u32 dclr, int chan)
{
edac_dbg(1, "F2x%d90 (DRAM Cfg Low): 0x%08x\n", chan, dclr);
(pvt->nbcap & NBCAP_SECDED) ? "yes" : "no",
(pvt->nbcap & NBCAP_CHIPKILL) ? "yes" : "no");
- amd64_dump_dramcfg_low(pvt, pvt->dclr0, 0);
+ debug_dump_dramcfg_low(pvt, pvt->dclr0, 0);
edac_dbg(1, "F3xB0 (Online Spare): 0x%08x\n", pvt->online_spare);
edac_dbg(1, " DramHoleValid: %s\n", dhar_valid(pvt) ? "yes" : "no");
- amd64_debug_display_dimm_sizes(pvt, 0);
+ debug_display_dimm_sizes(pvt, 0);
/* everything below this point is Fam10h and above */
if (pvt->fam == 0xf)
return;
- amd64_debug_display_dimm_sizes(pvt, 1);
+ debug_display_dimm_sizes(pvt, 1);
amd64_info("using %s syndromes.\n", ((pvt->ecc_sym_sz == 8) ? "x8" : "x4"));
/* Only if NOT ganged does dclr1 have valid info */
if (!dct_ganging_enabled(pvt))
- amd64_dump_dramcfg_low(pvt, pvt->dclr1, 1);
+ debug_dump_dramcfg_low(pvt, pvt->dclr1, 1);
}
/*
}
}
-static enum mem_type amd64_determine_memory_type(struct amd64_pvt *pvt, int cs)
+static enum mem_type determine_memory_type(struct amd64_pvt *pvt, int cs)
{
enum mem_type type;
num_dcts_intlv, dct_sel);
/* Verify we stay within the MAX number of channels allowed */
- if (channel > 4 || channel < 0)
+ if (channel > 3)
return -EINVAL;
leg_mmio_hole = (u8) (dct_cont_base_reg >> 1 & BIT(0));
* debug routine to display the memory sizes of all logical DIMMs and its
* CSROWs
*/
-static void amd64_debug_display_dimm_sizes(struct amd64_pvt *pvt, u8 ctrl)
+static void debug_display_dimm_sizes(struct amd64_pvt *pvt, u8 ctrl)
{
int dimm, size0, size1;
u32 *dcsb = ctrl ? pvt->csels[1].csbases : pvt->csels[0].csbases;
}
}
-static struct amd64_family_type amd64_family_types[] = {
+static struct amd64_family_type family_types[] = {
[K8_CPUS] = {
.ctl_name = "K8",
.f1_id = PCI_DEVICE_ID_AMD_K8_NB_ADDRMAP,
string, "");
}
-static inline void __amd64_decode_bus_error(struct mem_ctl_info *mci,
- struct mce *m)
+static inline void decode_bus_error(int node_id, struct mce *m)
{
+ struct mem_ctl_info *mci = mcis[node_id];
struct amd64_pvt *pvt = mci->pvt_info;
u8 ecc_type = (m->status >> 45) & 0x3;
u8 xec = XEC(m->status, 0x1f);
__log_bus_error(mci, &err, ecc_type);
}
-void amd64_decode_bus_error(int node_id, struct mce *m)
-{
- __amd64_decode_bus_error(mcis[node_id], m);
-}
-
/*
* Use pvt->F2 which contains the F2 CPU PCI device to get the related
* F1 (AddrMap) and F3 (Misc) devices. Return negative value on error.
* encompasses
*
*/
-static u32 amd64_csrow_nr_pages(struct amd64_pvt *pvt, u8 dct, int csrow_nr)
+static u32 get_csrow_nr_pages(struct amd64_pvt *pvt, u8 dct, int csrow_nr)
{
u32 cs_mode, nr_pages;
u32 dbam = dct ? pvt->dbam1 : pvt->dbam0;
pvt->mc_node_id, i);
if (row_dct0) {
- nr_pages = amd64_csrow_nr_pages(pvt, 0, i);
+ nr_pages = get_csrow_nr_pages(pvt, 0, i);
csrow->channels[0]->dimm->nr_pages = nr_pages;
}
/* K8 has only one DCT */
if (pvt->fam != 0xf && row_dct1) {
- int row_dct1_pages = amd64_csrow_nr_pages(pvt, 1, i);
+ int row_dct1_pages = get_csrow_nr_pages(pvt, 1, i);
csrow->channels[1]->dimm->nr_pages = row_dct1_pages;
nr_pages += row_dct1_pages;
}
- mtype = amd64_determine_memory_type(pvt, i);
+ mtype = determine_memory_type(pvt, i);
edac_dbg(1, "Total csrow%d pages: %u\n", i, nr_pages);
}
/* check MCG_CTL on all the cpus on this node */
-static bool amd64_nb_mce_bank_enabled_on_node(u16 nid)
+static bool nb_mce_bank_enabled_on_node(u16 nid)
{
cpumask_var_t mask;
int cpu, nbe;
ecc_en = !!(value & NBCFG_ECC_ENABLE);
amd64_info("DRAM ECC %s.\n", (ecc_en ? "enabled" : "disabled"));
- nb_mce_en = amd64_nb_mce_bank_enabled_on_node(nid);
+ nb_mce_en = nb_mce_bank_enabled_on_node(nid);
if (!nb_mce_en)
amd64_notice("NB MCE bank disabled, set MSR "
"0x%08x[4] on node %d to enable.\n",
if (pvt->nbcap & NBCAP_CHIPKILL)
mci->edac_ctl_cap |= EDAC_FLAG_S4ECD4ED;
- mci->edac_cap = amd64_determine_edac_cap(pvt);
+ mci->edac_cap = determine_edac_cap(pvt);
mci->mod_name = EDAC_MOD_STR;
mci->mod_ver = EDAC_AMD64_VERSION;
mci->ctl_name = fam->ctl_name;
mci->ctl_page_to_phys = NULL;
/* memory scrubber interface */
- mci->set_sdram_scrub_rate = amd64_set_scrub_rate;
- mci->get_sdram_scrub_rate = amd64_get_scrub_rate;
+ mci->set_sdram_scrub_rate = set_scrub_rate;
+ mci->get_sdram_scrub_rate = get_scrub_rate;
}
/*
* returns a pointer to the family descriptor on success, NULL otherwise.
*/
-static struct amd64_family_type *amd64_per_family_init(struct amd64_pvt *pvt)
+static struct amd64_family_type *per_family_init(struct amd64_pvt *pvt)
{
struct amd64_family_type *fam_type = NULL;
switch (pvt->fam) {
case 0xf:
- fam_type = &amd64_family_types[K8_CPUS];
- pvt->ops = &amd64_family_types[K8_CPUS].ops;
+ fam_type = &family_types[K8_CPUS];
+ pvt->ops = &family_types[K8_CPUS].ops;
break;
case 0x10:
- fam_type = &amd64_family_types[F10_CPUS];
- pvt->ops = &amd64_family_types[F10_CPUS].ops;
+ fam_type = &family_types[F10_CPUS];
+ pvt->ops = &family_types[F10_CPUS].ops;
break;
case 0x15:
if (pvt->model == 0x30) {
- fam_type = &amd64_family_types[F15_M30H_CPUS];
- pvt->ops = &amd64_family_types[F15_M30H_CPUS].ops;
+ fam_type = &family_types[F15_M30H_CPUS];
+ pvt->ops = &family_types[F15_M30H_CPUS].ops;
break;
}
- fam_type = &amd64_family_types[F15_CPUS];
- pvt->ops = &amd64_family_types[F15_CPUS].ops;
+ fam_type = &family_types[F15_CPUS];
+ pvt->ops = &family_types[F15_CPUS].ops;
break;
case 0x16:
- fam_type = &amd64_family_types[F16_CPUS];
- pvt->ops = &amd64_family_types[F16_CPUS].ops;
+ fam_type = &family_types[F16_CPUS];
+ pvt->ops = &family_types[F16_CPUS].ops;
break;
default:
return fam_type;
}
-static int amd64_init_one_instance(struct pci_dev *F2)
+static int init_one_instance(struct pci_dev *F2)
{
struct amd64_pvt *pvt = NULL;
struct amd64_family_type *fam_type = NULL;
pvt->F2 = F2;
ret = -EINVAL;
- fam_type = amd64_per_family_init(pvt);
+ fam_type = per_family_init(pvt);
if (!fam_type)
goto err_free;
if (report_gart_errors)
amd_report_gart_errors(true);
- amd_register_ecc_decoder(amd64_decode_bus_error);
+ amd_register_ecc_decoder(decode_bus_error);
mcis[nid] = mci;
return ret;
}
-static int amd64_probe_one_instance(struct pci_dev *pdev,
- const struct pci_device_id *mc_type)
+static int probe_one_instance(struct pci_dev *pdev,
+ const struct pci_device_id *mc_type)
{
u16 nid = amd_get_node_id(pdev);
struct pci_dev *F3 = node_to_amd_nb(nid)->misc;
goto err_enable;
}
- ret = amd64_init_one_instance(pdev);
+ ret = init_one_instance(pdev);
if (ret < 0) {
amd64_err("Error probing instance: %d\n", nid);
restore_ecc_error_reporting(s, nid, F3);
return ret;
}
-static void amd64_remove_one_instance(struct pci_dev *pdev)
+static void remove_one_instance(struct pci_dev *pdev)
{
struct mem_ctl_info *mci;
struct amd64_pvt *pvt;
/* unregister from EDAC MCE */
amd_report_gart_errors(false);
- amd_unregister_ecc_decoder(amd64_decode_bus_error);
+ amd_unregister_ecc_decoder(decode_bus_error);
kfree(ecc_stngs[nid]);
ecc_stngs[nid] = NULL;
* PCI core identifies what devices are on a system during boot, and then
* inquiry this table to see if this driver is for a given device found.
*/
-static DEFINE_PCI_DEVICE_TABLE(amd64_pci_table) = {
+static const struct pci_device_id amd64_pci_table[] = {
{
.vendor = PCI_VENDOR_ID_AMD,
.device = PCI_DEVICE_ID_AMD_K8_NB_MEMCTL,
static struct pci_driver amd64_pci_driver = {
.name = EDAC_MOD_STR,
- .probe = amd64_probe_one_instance,
- .remove = amd64_remove_one_instance,
+ .probe = probe_one_instance,
+ .remove = remove_one_instance,
.id_table = amd64_pci_table,
};
struct mem_ctl_info *mci;
struct amd64_pvt *pvt;
- if (amd64_ctl_pci)
+ if (pci_ctl)
return;
mci = mcis[0];
- if (mci) {
-
- pvt = mci->pvt_info;
- amd64_ctl_pci =
- edac_pci_create_generic_ctl(&pvt->F2->dev, EDAC_MOD_STR);
-
- if (!amd64_ctl_pci) {
- pr_warning("%s(): Unable to create PCI control\n",
- __func__);
+ if (!mci)
+ return;
- pr_warning("%s(): PCI error report via EDAC not set\n",
- __func__);
- }
+ pvt = mci->pvt_info;
+ pci_ctl = edac_pci_create_generic_ctl(&pvt->F2->dev, EDAC_MOD_STR);
+ if (!pci_ctl) {
+ pr_warn("%s(): Unable to create PCI control\n", __func__);
+ pr_warn("%s(): PCI error report via EDAC not set\n", __func__);
}
}
static void __exit amd64_edac_exit(void)
{
- if (amd64_ctl_pci)
- edac_pci_release_generic_ctl(amd64_ctl_pci);
+ if (pci_ctl)
+ edac_pci_release_generic_ctl(pci_ctl);
pci_unregister_driver(&amd64_pci_driver);
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(amd76x_pci_tbl) = {
+static const struct pci_device_id amd76x_pci_tbl[] = {
{
PCI_VEND_DEV(AMD, FE_GATE_700C), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
AMD762},
pvt->bridge_ck = pci_get_device(PCI_VENDOR_ID_INTEL,
pvt->dev_info->err_dev, pvt->bridge_ck);
- if (pvt->bridge_ck == NULL)
+ if (pvt->bridge_ck == NULL) {
pvt->bridge_ck = pci_scan_single_device(pdev->bus,
PCI_DEVFN(0, 1));
+ pci_dev_get(pvt->bridge_ck);
+ }
if (pvt->bridge_ck == NULL) {
e752x_printk(KERN_ERR, "error reporting device not found:"
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(e752x_pci_tbl) = {
+static const struct pci_device_id e752x_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, 7520_0), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
E7520},
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(e7xxx_pci_tbl) = {
+static const struct pci_device_id e7xxx_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, 7205_0), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
E7205},
{
int status;
+ if (!edac_dev->edac_check)
+ return;
+
status = cancel_delayed_work(&edac_dev->work);
if (status == 0) {
/* workq instance might be running, wait for it */
debugfs_remove(edac_debugfs);
}
-int edac_create_debug_nodes(struct mem_ctl_info *mci)
+static int edac_create_debug_nodes(struct mem_ctl_info *mci)
{
struct dentry *d, *parent;
char name[80];
static atomic_t edac_subsys_valid = ATOMIC_INIT(0);
+int edac_report_status = EDAC_REPORTING_ENABLED;
+EXPORT_SYMBOL_GPL(edac_report_status);
+
+static int __init edac_report_setup(char *str)
+{
+ if (!str)
+ return -EINVAL;
+
+ if (!strncmp(str, "on", 2))
+ set_edac_report_status(EDAC_REPORTING_ENABLED);
+ else if (!strncmp(str, "off", 3))
+ set_edac_report_status(EDAC_REPORTING_DISABLED);
+ else if (!strncmp(str, "force", 5))
+ set_edac_report_status(EDAC_REPORTING_FORCE);
+
+ return 0;
+}
+__setup("edac_report=", edac_report_setup);
+
/*
* called to determine if there is an EDAC driver interested in
* knowing an event (such as NMI) occurred
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(i3000_pci_tbl) = {
+static const struct pci_device_id i3000_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, 3000_HB), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
I3000},
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(i3200_pci_tbl) = {
+static const struct pci_device_id i3200_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, 3200_HB), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
I3200},
*
* The "E500P" device is the first device supported.
*/
-static DEFINE_PCI_DEVICE_TABLE(i5000_pci_tbl) = {
+static const struct pci_device_id i5000_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_I5000_DEV16),
.driver_data = I5000P},
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(i5100_pci_tbl) = {
+static const struct pci_device_id i5100_pci_tbl[] = {
/* Device 16, Function 0, Channel 0 Memory Map, Error Flag/Mask, ... */
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_5100_16) },
{ 0, }
*
* The "E500P" device is the first device supported.
*/
-static DEFINE_PCI_DEVICE_TABLE(i5400_pci_tbl) = {
+static const struct pci_device_id i5400_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_5400_ERR)},
{0,} /* 0 terminated list. */
};
*
* Has only 8086:360c PCI ID
*/
-static DEFINE_PCI_DEVICE_TABLE(i7300_pci_tbl) = {
+static const struct pci_device_id i7300_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_I7300_MCH_ERR)},
{0,} /* 0 terminated list. */
};
/*
* pci_device_id table for which devices we are looking for
*/
-static DEFINE_PCI_DEVICE_TABLE(i7core_pci_tbl) = {
+static const struct pci_device_id i7core_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_X58_HUB_MGMT)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_LYNNFIELD_QPI_LINK0)},
{0,} /* 0 terminated list. */
EXPORT_SYMBOL_GPL(i82443bxgx_edacmc_remove_one);
-static DEFINE_PCI_DEVICE_TABLE(i82443bxgx_pci_tbl) = {
+static const struct pci_device_id i82443bxgx_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82443BX_0)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82443BX_2)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82443GX_0)},
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(i82860_pci_tbl) = {
+static const struct pci_device_id i82860_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, 82860_0), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
I82860},
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(i82875p_pci_tbl) = {
+static const struct pci_device_id i82875p_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, 82875_0), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
I82875P},
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(i82975x_pci_tbl) = {
+static const struct pci_device_id i82975x_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, 82975_0), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
I82975X
/*
* Freescale MPC85xx Memory Controller kenel module
*
+ * Parts Copyrighted (c) 2013 by Freescale Semiconductor, Inc.
+ *
* Author: Dave Jiang <djiang@mvista.com>
*
* 2006-2007 (c) MontaVista Software, Inc. This file is licensed under
edac_pci_handle_npe(pci, pci->ctl_name);
}
+static void mpc85xx_pcie_check(struct edac_pci_ctl_info *pci)
+{
+ struct mpc85xx_pci_pdata *pdata = pci->pvt_info;
+ u32 err_detect;
+
+ err_detect = in_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_DR);
+
+ pr_err("PCIe error(s) detected\n");
+ pr_err("PCIe ERR_DR register: 0x%08x\n", err_detect);
+ pr_err("PCIe ERR_CAP_STAT register: 0x%08x\n",
+ in_be32(pdata->pci_vbase + MPC85XX_PCI_GAS_TIMR));
+ pr_err("PCIe ERR_CAP_R0 register: 0x%08x\n",
+ in_be32(pdata->pci_vbase + MPC85XX_PCIE_ERR_CAP_R0));
+ pr_err("PCIe ERR_CAP_R1 register: 0x%08x\n",
+ in_be32(pdata->pci_vbase + MPC85XX_PCIE_ERR_CAP_R1));
+ pr_err("PCIe ERR_CAP_R2 register: 0x%08x\n",
+ in_be32(pdata->pci_vbase + MPC85XX_PCIE_ERR_CAP_R2));
+ pr_err("PCIe ERR_CAP_R3 register: 0x%08x\n",
+ in_be32(pdata->pci_vbase + MPC85XX_PCIE_ERR_CAP_R3));
+
+ /* clear error bits */
+ out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_DR, err_detect);
+}
+
+static int mpc85xx_pcie_find_capability(struct device_node *np)
+{
+ struct pci_controller *hose;
+
+ if (!np)
+ return -EINVAL;
+
+ hose = pci_find_hose_for_OF_device(np);
+
+ return early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP);
+}
+
static irqreturn_t mpc85xx_pci_isr(int irq, void *dev_id)
{
struct edac_pci_ctl_info *pci = dev_id;
if (!err_detect)
return IRQ_NONE;
- mpc85xx_pci_check(pci);
+ if (pdata->is_pcie)
+ mpc85xx_pcie_check(pci);
+ else
+ mpc85xx_pci_check(pci);
return IRQ_HANDLED;
}
pdata = pci->pvt_info;
pdata->name = "mpc85xx_pci_err";
pdata->irq = NO_IRQ;
+
+ if (mpc85xx_pcie_find_capability(op->dev.of_node) > 0)
+ pdata->is_pcie = true;
+
dev_set_drvdata(&op->dev, pci);
pci->dev = &op->dev;
pci->mod_name = EDAC_MOD_STR;
pci->ctl_name = pdata->name;
pci->dev_name = dev_name(&op->dev);
- if (edac_op_state == EDAC_OPSTATE_POLL)
- pci->edac_check = mpc85xx_pci_check;
+ if (edac_op_state == EDAC_OPSTATE_POLL) {
+ if (pdata->is_pcie)
+ pci->edac_check = mpc85xx_pcie_check;
+ else
+ pci->edac_check = mpc85xx_pci_check;
+ }
pdata->edac_idx = edac_pci_idx++;
goto err;
}
- orig_pci_err_cap_dr =
- in_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_CAP_DR);
+ if (pdata->is_pcie) {
+ orig_pci_err_cap_dr =
+ in_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_ADDR);
+ out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_ADDR, ~0);
+ orig_pci_err_en =
+ in_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_EN);
+ out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_EN, 0);
+ } else {
+ orig_pci_err_cap_dr =
+ in_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_CAP_DR);
- /* PCI master abort is expected during config cycles */
- out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_CAP_DR, 0x40);
+ /* PCI master abort is expected during config cycles */
+ out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_CAP_DR, 0x40);
- orig_pci_err_en = in_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_EN);
+ orig_pci_err_en =
+ in_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_EN);
- /* disable master abort reporting */
- out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_EN, ~0x40);
+ /* disable master abort reporting */
+ out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_EN, ~0x40);
+ }
/* clear error bits */
out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_DR, ~0);
if (edac_op_state == EDAC_OPSTATE_INT) {
pdata->irq = irq_of_parse_and_map(op->dev.of_node, 0);
res = devm_request_irq(&op->dev, pdata->irq,
- mpc85xx_pci_isr, IRQF_DISABLED,
+ mpc85xx_pci_isr,
+ IRQF_DISABLED | IRQF_SHARED,
"[EDAC] PCI err", pci);
if (res < 0) {
printk(KERN_ERR
pdata->irq);
}
+ if (pdata->is_pcie) {
+ /*
+ * Enable all PCIe error interrupt & error detect except invalid
+ * PEX_CONFIG_ADDR/PEX_CONFIG_DATA access interrupt generation
+ * enable bit and invalid PEX_CONFIG_ADDR/PEX_CONFIG_DATA access
+ * detection enable bit. Because PCIe bus code to initialize and
+ * configure these PCIe devices on booting will use some invalid
+ * PEX_CONFIG_ADDR/PEX_CONFIG_DATA, edac driver prints the much
+ * notice information. So disable this detect to fix ugly print.
+ */
+ out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_EN, ~0
+ & ~PEX_ERR_ICCAIE_EN_BIT);
+ out_be32(pdata->pci_vbase + MPC85XX_PCI_ERR_ADDR, 0
+ | PEX_ERR_ICCAD_DISR_BIT);
+ }
+
devres_remove_group(&op->dev, mpc85xx_pci_err_probe);
edac_dbg(3, "success\n");
printk(KERN_INFO EDAC_MOD_STR " PCI err registered\n");
#define MPC85XX_PCI_ERR_DR 0x0000
#define MPC85XX_PCI_ERR_CAP_DR 0x0004
#define MPC85XX_PCI_ERR_EN 0x0008
+#define PEX_ERR_ICCAIE_EN_BIT 0x00020000
#define MPC85XX_PCI_ERR_ATTRIB 0x000c
#define MPC85XX_PCI_ERR_ADDR 0x0010
+#define PEX_ERR_ICCAD_DISR_BIT 0x00020000
#define MPC85XX_PCI_ERR_EXT_ADDR 0x0014
#define MPC85XX_PCI_ERR_DL 0x0018
#define MPC85XX_PCI_ERR_DH 0x001c
#define MPC85XX_PCI_GAS_TIMR 0x0020
#define MPC85XX_PCI_PCIX_TIMR 0x0024
+#define MPC85XX_PCIE_ERR_CAP_R0 0x0028
+#define MPC85XX_PCIE_ERR_CAP_R1 0x002c
+#define MPC85XX_PCIE_ERR_CAP_R2 0x0030
+#define MPC85XX_PCIE_ERR_CAP_R3 0x0034
struct mpc85xx_mc_pdata {
char *name;
struct mpc85xx_pci_pdata {
char *name;
+ bool is_pcie;
int edac_idx;
void __iomem *pci_vbase;
int irq;
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(r82600_pci_tbl) = {
+static const struct pci_device_id r82600_pci_tbl[] = {
{
PCI_DEVICE(PCI_VENDOR_ID_RADISYS, R82600_BRIDGE_ID)
},
/*
* pci_device_id table for which devices we are looking for
*/
-static DEFINE_PCI_DEVICE_TABLE(sbridge_pci_tbl) = {
+static const struct pci_device_id sbridge_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_TA)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_IBRIDGE_IMC_HA0_TA)},
{0,} /* 0 terminated list. */
}
}
-struct mem_ctl_info *get_mci_for_node_id(u8 node_id)
+static struct mem_ctl_info *get_mci_for_node_id(u8 node_id)
{
struct sbridge_dev *sbridge_dev;
struct mem_ctl_info *mci;
struct sbridge_pvt *pvt;
+ if (get_edac_report_status() == EDAC_REPORTING_DISABLED)
+ return NOTIFY_DONE;
+
mci = get_mci_for_node_id(mce->socketid);
if (!mci)
return NOTIFY_BAD;
opstate_init();
pci_rc = pci_register_driver(&sbridge_driver);
-
if (pci_rc >= 0) {
mce_register_decode_chain(&sbridge_mce_dec);
+ if (get_edac_report_status() == EDAC_REPORTING_DISABLED)
+ sbridge_printk(KERN_WARNING, "Loading driver, error reporting disabled.\n");
return 0;
}
edac_mc_free(mci);
}
-static DEFINE_PCI_DEVICE_TABLE(x38_pci_tbl) = {
+static const struct pci_device_id x38_pci_tbl[] = {
{
PCI_VEND_DEV(INTEL, X38_HB), PCI_ANY_ID, PCI_ANY_ID, 0, 0,
X38},
backend for pstore by default. This setting can be overridden
using the efivars module's pstore_disable parameter.
+config EFI_RUNTIME_MAP
+ bool "Export efi runtime maps to sysfs"
+ depends on X86 && EFI && KEXEC
+ default y
+ help
+ Export efi runtime memory maps to /sys/firmware/efi/runtime-map.
+ That memory map is used for example by kexec to set up efi virtual
+ mapping the 2nd kernel, but can also be used for debugging purposes.
+
+ See also Documentation/ABI/testing/sysfs-firmware-efi-runtime-map.
+
endmenu
config UEFI_CPER
obj-$(CONFIG_EFI_VARS) += efivars.o
obj-$(CONFIG_EFI_VARS_PSTORE) += efi-pstore.o
obj-$(CONFIG_UEFI_CPER) += cper.o
+obj-$(CONFIG_EFI_RUNTIME_MAP) += runtime-map.o
.hcdp = EFI_INVALID_TABLE_ADDR,
.uga = EFI_INVALID_TABLE_ADDR,
.uv_systab = EFI_INVALID_TABLE_ADDR,
+ .fw_vendor = EFI_INVALID_TABLE_ADDR,
+ .runtime = EFI_INVALID_TABLE_ADDR,
+ .config_table = EFI_INVALID_TABLE_ADDR,
};
EXPORT_SYMBOL(efi);
static struct kobj_attribute efi_attr_systab =
__ATTR(systab, 0400, systab_show, NULL);
+#define EFI_FIELD(var) efi.var
+
+#define EFI_ATTR_SHOW(name) \
+static ssize_t name##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buf) \
+{ \
+ return sprintf(buf, "0x%lx\n", EFI_FIELD(name)); \
+}
+
+EFI_ATTR_SHOW(fw_vendor);
+EFI_ATTR_SHOW(runtime);
+EFI_ATTR_SHOW(config_table);
+
+static struct kobj_attribute efi_attr_fw_vendor = __ATTR_RO(fw_vendor);
+static struct kobj_attribute efi_attr_runtime = __ATTR_RO(runtime);
+static struct kobj_attribute efi_attr_config_table = __ATTR_RO(config_table);
+
static struct attribute *efi_subsys_attrs[] = {
&efi_attr_systab.attr,
- NULL, /* maybe more in the future? */
+ &efi_attr_fw_vendor.attr,
+ &efi_attr_runtime.attr,
+ &efi_attr_config_table.attr,
+ NULL,
};
+static umode_t efi_attr_is_visible(struct kobject *kobj,
+ struct attribute *attr, int n)
+{
+ umode_t mode = attr->mode;
+
+ if (attr == &efi_attr_fw_vendor.attr)
+ return (efi.fw_vendor == EFI_INVALID_TABLE_ADDR) ? 0 : mode;
+ else if (attr == &efi_attr_runtime.attr)
+ return (efi.runtime == EFI_INVALID_TABLE_ADDR) ? 0 : mode;
+ else if (attr == &efi_attr_config_table.attr)
+ return (efi.config_table == EFI_INVALID_TABLE_ADDR) ? 0 : mode;
+
+ return mode;
+}
+
static struct attribute_group efi_subsys_attr_group = {
.attrs = efi_subsys_attrs,
+ .is_visible = efi_attr_is_visible,
};
static struct efivars generic_efivars;
goto err_unregister;
}
+ error = efi_runtime_map_init(efi_kobj);
+ if (error)
+ goto err_remove_group;
+
/* and the standard mountpoint for efivarfs */
efivars_kobj = kobject_create_and_add("efivars", efi_kobj);
if (!efivars_kobj) {
--- /dev/null
+/*
+ * linux/drivers/efi/runtime-map.c
+ * Copyright (C) 2013 Red Hat, Inc., Dave Young <dyoung@redhat.com>
+ *
+ * This file is released under the GPLv2.
+ */
+
+#include <linux/string.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/efi.h>
+#include <linux/slab.h>
+
+#include <asm/setup.h>
+
+static void *efi_runtime_map;
+static int nr_efi_runtime_map;
+static u32 efi_memdesc_size;
+
+struct efi_runtime_map_entry {
+ efi_memory_desc_t md;
+ struct kobject kobj; /* kobject for each entry */
+};
+
+static struct efi_runtime_map_entry **map_entries;
+
+struct map_attribute {
+ struct attribute attr;
+ ssize_t (*show)(struct efi_runtime_map_entry *entry, char *buf);
+};
+
+static inline struct map_attribute *to_map_attr(struct attribute *attr)
+{
+ return container_of(attr, struct map_attribute, attr);
+}
+
+static ssize_t type_show(struct efi_runtime_map_entry *entry, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "0x%x\n", entry->md.type);
+}
+
+#define EFI_RUNTIME_FIELD(var) entry->md.var
+
+#define EFI_RUNTIME_U64_ATTR_SHOW(name) \
+static ssize_t name##_show(struct efi_runtime_map_entry *entry, char *buf) \
+{ \
+ return snprintf(buf, PAGE_SIZE, "0x%llx\n", EFI_RUNTIME_FIELD(name)); \
+}
+
+EFI_RUNTIME_U64_ATTR_SHOW(phys_addr);
+EFI_RUNTIME_U64_ATTR_SHOW(virt_addr);
+EFI_RUNTIME_U64_ATTR_SHOW(num_pages);
+EFI_RUNTIME_U64_ATTR_SHOW(attribute);
+
+static inline struct efi_runtime_map_entry *to_map_entry(struct kobject *kobj)
+{
+ return container_of(kobj, struct efi_runtime_map_entry, kobj);
+}
+
+static ssize_t map_attr_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ struct efi_runtime_map_entry *entry = to_map_entry(kobj);
+ struct map_attribute *map_attr = to_map_attr(attr);
+
+ return map_attr->show(entry, buf);
+}
+
+static struct map_attribute map_type_attr = __ATTR_RO(type);
+static struct map_attribute map_phys_addr_attr = __ATTR_RO(phys_addr);
+static struct map_attribute map_virt_addr_attr = __ATTR_RO(virt_addr);
+static struct map_attribute map_num_pages_attr = __ATTR_RO(num_pages);
+static struct map_attribute map_attribute_attr = __ATTR_RO(attribute);
+
+/*
+ * These are default attributes that are added for every memmap entry.
+ */
+static struct attribute *def_attrs[] = {
+ &map_type_attr.attr,
+ &map_phys_addr_attr.attr,
+ &map_virt_addr_attr.attr,
+ &map_num_pages_attr.attr,
+ &map_attribute_attr.attr,
+ NULL
+};
+
+static const struct sysfs_ops map_attr_ops = {
+ .show = map_attr_show,
+};
+
+static void map_release(struct kobject *kobj)
+{
+ struct efi_runtime_map_entry *entry;
+
+ entry = to_map_entry(kobj);
+ kfree(entry);
+}
+
+static struct kobj_type __refdata map_ktype = {
+ .sysfs_ops = &map_attr_ops,
+ .default_attrs = def_attrs,
+ .release = map_release,
+};
+
+static struct kset *map_kset;
+
+static struct efi_runtime_map_entry *
+add_sysfs_runtime_map_entry(struct kobject *kobj, int nr)
+{
+ int ret;
+ struct efi_runtime_map_entry *entry;
+
+ if (!map_kset) {
+ map_kset = kset_create_and_add("runtime-map", NULL, kobj);
+ if (!map_kset)
+ return ERR_PTR(-ENOMEM);
+ }
+
+ entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry) {
+ kset_unregister(map_kset);
+ return entry;
+ }
+
+ memcpy(&entry->md, efi_runtime_map + nr * efi_memdesc_size,
+ sizeof(efi_memory_desc_t));
+
+ kobject_init(&entry->kobj, &map_ktype);
+ entry->kobj.kset = map_kset;
+ ret = kobject_add(&entry->kobj, NULL, "%d", nr);
+ if (ret) {
+ kobject_put(&entry->kobj);
+ kset_unregister(map_kset);
+ return ERR_PTR(ret);
+ }
+
+ return entry;
+}
+
+void efi_runtime_map_setup(void *map, int nr_entries, u32 desc_size)
+{
+ efi_runtime_map = map;
+ nr_efi_runtime_map = nr_entries;
+ efi_memdesc_size = desc_size;
+}
+
+int __init efi_runtime_map_init(struct kobject *efi_kobj)
+{
+ int i, j, ret = 0;
+ struct efi_runtime_map_entry *entry;
+
+ if (!efi_runtime_map)
+ return 0;
+
+ map_entries = kzalloc(nr_efi_runtime_map * sizeof(entry), GFP_KERNEL);
+ if (!map_entries) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ for (i = 0; i < nr_efi_runtime_map; i++) {
+ entry = add_sysfs_runtime_map_entry(efi_kobj, i);
+ if (IS_ERR(entry)) {
+ ret = PTR_ERR(entry);
+ goto out_add_entry;
+ }
+ *(map_entries + i) = entry;
+ }
+
+ return 0;
+out_add_entry:
+ for (j = i - 1; j > 0; j--) {
+ entry = *(map_entries + j);
+ kobject_put(&entry->kobj);
+ }
+ if (map_kset)
+ kset_unregister(map_kset);
+out:
+ return ret;
+}
/* if equal delete the probed mode */
mode->status = pmode->status;
/* Merge type bits together */
- mode->type = pmode->type;
+ mode->type |= pmode->type;
list_del(&pmode->head);
drm_mode_destroy(connector->dev, pmode);
break;
#undef GEN8_IRQ_INIT_NDX
POSTING_READ(GEN8_PCU_IIR);
+
+ ibx_irq_preinstall(dev);
}
static void ibx_hpd_irq_setup(struct drm_device *dev)
enum pipe pipe;
struct intel_crtc *intel_crtc;
+ dev_priv->ddi_plls.spll_refcount = 0;
+ dev_priv->ddi_plls.wrpll1_refcount = 0;
+ dev_priv->ddi_plls.wrpll2_refcount = 0;
+
for_each_pipe(pipe) {
intel_crtc =
to_intel_crtc(dev_priv->pipe_to_crtc_mapping[pipe]);
- if (!intel_crtc->active)
+ if (!intel_crtc->active) {
+ intel_crtc->ddi_pll_sel = PORT_CLK_SEL_NONE;
continue;
+ }
intel_crtc->ddi_pll_sel = intel_ddi_get_crtc_pll(dev_priv,
pipe);
intel_setup_overlay(dev);
- drm_modeset_lock_all(dev);
+ mutex_lock(&dev->mode_config.mutex);
drm_mode_config_reset(dev);
intel_modeset_setup_hw_state(dev, false);
- drm_modeset_unlock_all(dev);
+ mutex_unlock(&dev->mode_config.mutex);
}
void intel_modeset_cleanup(struct drm_device *dev)
int (*identify)(struct nouveau_i2c *, int index,
const char *what, struct nouveau_i2c_board_info *,
bool (*match)(struct nouveau_i2c_port *,
- struct i2c_board_info *));
+ struct i2c_board_info *, void *), void *);
struct list_head ports;
};
static inline struct nouveau_instmem *
nouveau_instmem(void *obj)
{
+ /* nv04/nv40 impls need to create objects in their constructor,
+ * which is before the subdev pointer is valid
+ */
+ if (nv_iclass(obj, NV_SUBDEV_CLASS) &&
+ nv_subidx(obj) == NVDEV_SUBDEV_INSTMEM)
+ return obj;
+
return (void *)nv_device(obj)->subdev[NVDEV_SUBDEV_INSTMEM];
}
nouveau_i2c_identify(struct nouveau_i2c *i2c, int index, const char *what,
struct nouveau_i2c_board_info *info,
bool (*match)(struct nouveau_i2c_port *,
- struct i2c_board_info *))
+ struct i2c_board_info *, void *), void *data)
{
struct nouveau_i2c_port *port = nouveau_i2c_find(i2c, index);
int i;
}
if (nv_probe_i2c(port, info[i].dev.addr) &&
- (!match || match(port, &info[i].dev))) {
+ (!match || match(port, &info[i].dev, data))) {
nv_info(i2c, "detected %s: %s\n", what,
info[i].dev.type);
return i;
static int
mxm_dcb_sanitise_entry(struct nouveau_bios *bios, void *data, int idx, u16 pdcb)
{
- struct nouveau_mxm *mxm = nouveau_mxm(bios);
+ struct nouveau_mxm *mxm = data;
struct context ctx = { .outp = (u32 *)(bios->data + pdcb) };
u8 type, i2cidx, link, ver, len;
u8 *conn;
return;
}
- dcb_outp_foreach(bios, NULL, mxm_dcb_sanitise_entry);
+ dcb_outp_foreach(bios, mxm, mxm_dcb_sanitise_entry);
mxms_foreach(mxm, 0x01, mxm_show_unmatched, NULL);
}
static bool
probe_monitoring_device(struct nouveau_i2c_port *i2c,
- struct i2c_board_info *info)
+ struct i2c_board_info *info, void *data)
{
- struct nouveau_therm_priv *priv = (void *)nouveau_therm(i2c);
+ struct nouveau_therm_priv *priv = data;
struct nvbios_therm_sensor *sensor = &priv->bios_sensor;
struct i2c_client *client;
};
i2c->identify(i2c, NV_I2C_DEFAULT(0), "monitoring device",
- board, probe_monitoring_device);
+ board, probe_monitoring_device, therm);
if (priv->ic)
return;
}
};
i2c->identify(i2c, NV_I2C_DEFAULT(0), "monitoring device",
- board, probe_monitoring_device);
+ board, probe_monitoring_device, therm);
if (priv->ic)
return;
}
device. Let's try our static list.
*/
i2c->identify(i2c, NV_I2C_DEFAULT(0), "monitoring device",
- nv_board_infos, probe_monitoring_device);
+ nv_board_infos, probe_monitoring_device, therm);
}
get_tmds_slave(encoder))
return;
- type = i2c->identify(i2c, 2, "TMDS transmitter", info, NULL);
+ type = i2c->identify(i2c, 2, "TMDS transmitter", info, NULL, NULL);
if (type < 0)
return;
struct nouveau_i2c *i2c = nouveau_i2c(drm->device);
return i2c->identify(i2c, i2c_index, "TV encoder",
- nv04_tv_encoder_info, NULL);
+ nv04_tv_encoder_info, NULL, NULL);
}
#include <linux/cpu.h>
#include <linux/smp.h>
#include <linux/moduleparam.h>
+#include <linux/pci.h>
#include <asm/msr.h>
#include <asm/processor.h>
#include <asm/cpu_device_id.h>
#define BASE_SYSFS_ATTR_NO 2 /* Sysfs Base attr no for coretemp */
#define NUM_REAL_CORES 32 /* Number of Real cores per cpu */
-#define CORETEMP_NAME_LENGTH 17 /* String Length of attrs */
+#define CORETEMP_NAME_LENGTH 19 /* String Length of attrs */
#define MAX_CORE_ATTRS 4 /* Maximum no of basic attrs */
#define TOTAL_ATTRS (MAX_CORE_ATTRS + 1)
#define MAX_CORE_DATA (NUM_REAL_CORES + BASE_SYSFS_ATTR_NO)
/* Check whether the time interval has elapsed */
if (!tdata->valid || time_after(jiffies, tdata->last_updated + HZ)) {
rdmsr_on_cpu(tdata->cpu, tdata->status_reg, &eax, &edx);
- tdata->valid = 0;
- /* Check whether the data is valid */
- if (eax & 0x80000000) {
- tdata->temp = tdata->tjmax -
- ((eax >> 16) & 0x7f) * 1000;
- tdata->valid = 1;
- }
+ /*
+ * Ignore the valid bit. In all observed cases the register
+ * value is either low or zero if the valid bit is 0.
+ * Return it instead of reporting an error which doesn't
+ * really help at all.
+ */
+ tdata->temp = tdata->tjmax - ((eax >> 16) & 0x7f) * 1000;
+ tdata->valid = 1;
tdata->last_updated = jiffies;
}
mutex_unlock(&tdata->update_lock);
- return tdata->valid ? sprintf(buf, "%d\n", tdata->temp) : -EAGAIN;
+ return sprintf(buf, "%d\n", tdata->temp);
}
+struct tjmax_pci {
+ unsigned int device;
+ int tjmax;
+};
+
+static const struct tjmax_pci tjmax_pci_table[] = {
+ { 0x0708, 110000 }, /* CE41x0 (Sodaville ) */
+ { 0x0c72, 102000 }, /* Atom S1240 (Centerton) */
+ { 0x0c73, 95000 }, /* Atom S1220 (Centerton) */
+ { 0x0c75, 95000 }, /* Atom S1260 (Centerton) */
+};
+
struct tjmax {
char const *id;
int tjmax;
static const struct tjmax tjmax_table[] = {
{ "CPU 230", 100000 }, /* Model 0x1c, stepping 2 */
{ "CPU 330", 125000 }, /* Model 0x1c, stepping 2 */
- { "CPU CE4110", 110000 }, /* Model 0x1c, stepping 10 Sodaville */
- { "CPU CE4150", 110000 }, /* Model 0x1c, stepping 10 */
- { "CPU CE4170", 110000 }, /* Model 0x1c, stepping 10 */
};
struct tjmax_model {
* is undetectable by software
*/
{ 0x27, ANY, 90000 }, /* Atom Medfield (Z2460) */
- { 0x35, ANY, 90000 }, /* Atom Clover Trail/Cloverview (Z2760) */
- { 0x36, ANY, 100000 }, /* Atom Cedar Trail/Cedarview (N2xxx, D2xxx) */
+ { 0x35, ANY, 90000 }, /* Atom Clover Trail/Cloverview (Z27x0) */
+ { 0x36, ANY, 100000 }, /* Atom Cedar Trail/Cedarview (N2xxx, D2xxx)
+ * Also matches S12x0 (stepping 9), covered by
+ * PCI table
+ */
};
static int adjust_tjmax(struct cpuinfo_x86 *c, u32 id, struct device *dev)
int err;
u32 eax, edx;
int i;
+ struct pci_dev *host_bridge = pci_get_bus_and_slot(0, PCI_DEVFN(0, 0));
+
+ /*
+ * Explicit tjmax table entries override heuristics.
+ * First try PCI host bridge IDs, followed by model ID strings
+ * and model/stepping information.
+ */
+ if (host_bridge && host_bridge->vendor == PCI_VENDOR_ID_INTEL) {
+ for (i = 0; i < ARRAY_SIZE(tjmax_pci_table); i++) {
+ if (host_bridge->device == tjmax_pci_table[i].device)
+ return tjmax_pci_table[i].tjmax;
+ }
+ }
- /* explicit tjmax table entries override heuristics */
for (i = 0; i < ARRAY_SIZE(tjmax_table); i++) {
if (strstr(c->x86_model_id, tjmax_table[i].id))
return tjmax_table[i].tjmax;
if (cpu_has_tjmax(c))
dev_warn(dev, "Unable to read TjMax from CPU %u\n", id);
} else {
- val = (eax >> 16) & 0xff;
+ val = (eax >> 16) & 0x7f;
/*
* If the TjMax is not plausible, an assumption
* will be used
*/
- if (val) {
+ if (val >= 85) {
dev_dbg(dev, "TjMax is %d degrees C\n", val);
return val * 1000;
}
/* Conversion function for VDDOUT and VBAT */
static inline int volt_reg_to_mv(int value)
{
- return DIV_ROUND_CLOSEST(value * 1000, 512) + 2500;
+ return DIV_ROUND_CLOSEST(value * 2000, 1023) + 2500;
}
/* Conversion function for ADC channels 4, 5 and 6 */
/* Conversion function for VBBAT */
static inline int vbbat_reg_to_mv(int value)
{
- return DIV_ROUND_CLOSEST(value * 2500, 512);
+ return DIV_ROUND_CLOSEST(value * 5000, 1023);
}
static inline int da9052_enable_vddout_channel(struct da9052 *da9052)
sysfs_remove_group(&dev->kobj, &fam15h_power_attr_group);
}
-static DEFINE_PCI_DEVICE_TABLE(fam15h_power_id_table) = {
+static const struct pci_device_id fam15h_power_id_table[] = {
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_15H_NB_F4) },
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_16H_NB_F4) },
{}
&sensor_dev_attr_temp1_crit_hyst.dev_attr);
}
-static DEFINE_PCI_DEVICE_TABLE(k10temp_id_table) = {
+static const struct pci_device_id k10temp_id_table[] = {
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_10H_NB_MISC) },
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_11H_NB_MISC) },
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_CNB17H_F3) },
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_15H_NB_F3) },
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_15H_M10H_F3) },
+ { PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_15H_M30H_NB_F3) },
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_16H_NB_F3) },
{}
};
static SENSOR_DEVICE_ATTR_2(temp4_input, S_IRUGO, show_temp, NULL, 1, 1);
static DEVICE_ATTR(name, S_IRUGO, show_name, NULL);
-static DEFINE_PCI_DEVICE_TABLE(k8temp_ids) = {
+static const struct pci_device_id k8temp_ids[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_K8_NB_MISC) },
{ 0 },
};
return PTR_ERR_OR_ZERO(hwmon_dev);
}
+static void nct6791_enable_io_mapping(int sioaddr)
+{
+ int val;
+
+ val = superio_inb(sioaddr, NCT6791_REG_HM_IO_SPACE_LOCK_ENABLE);
+ if (val & 0x10) {
+ pr_info("Enabling hardware monitor logical device mappings.\n");
+ superio_outb(sioaddr, NCT6791_REG_HM_IO_SPACE_LOCK_ENABLE,
+ val & ~0x10);
+ }
+}
+
#ifdef CONFIG_PM
static int nct6775_suspend(struct device *dev)
{
static int nct6775_resume(struct device *dev)
{
struct nct6775_data *data = dev_get_drvdata(dev);
- int i, j;
+ int i, j, err = 0;
mutex_lock(&data->update_lock);
data->bank = 0xff; /* Force initial bank selection */
+ if (data->kind == nct6791) {
+ err = superio_enter(data->sioreg);
+ if (err)
+ goto abort;
+
+ nct6791_enable_io_mapping(data->sioreg);
+ superio_exit(data->sioreg);
+ }
+
/* Restore limits */
for (i = 0; i < data->in_num; i++) {
if (!(data->have_in & (1 << i)))
nct6775_write_value(data, NCT6775_REG_FANDIV2, data->fandiv2);
}
+abort:
/* Force re-reading all values */
data->valid = false;
mutex_unlock(&data->update_lock);
- return 0;
+ return err;
}
static const struct dev_pm_ops nct6775_dev_pm_ops = {
pr_warn("Forcibly enabling Super-I/O. Sensor is probably unusable.\n");
superio_outb(sioaddr, SIO_REG_ENABLE, val | 0x01);
}
- if (sio_data->kind == nct6791) {
- val = superio_inb(sioaddr, NCT6791_REG_HM_IO_SPACE_LOCK_ENABLE);
- if (val & 0x10) {
- pr_info("Enabling hardware monitor logical device mappings.\n");
- superio_outb(sioaddr,
- NCT6791_REG_HM_IO_SPACE_LOCK_ENABLE,
- val & ~0x10);
- }
- }
+
+ if (sio_data->kind == nct6791)
+ nct6791_enable_io_mapping(sioaddr);
superio_exit(sioaddr);
pr_info("Found %s or compatible chip at %#x:%#x\n",
return data;
}
-static DEFINE_PCI_DEVICE_TABLE(sis5595_pci_ids) = {
+static const struct pci_device_id sis5595_pci_ids[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_SI, PCI_DEVICE_ID_SI_503) },
{ 0, }
};
return data;
}
-static DEFINE_PCI_DEVICE_TABLE(via686a_pci_ids) = {
+static const struct pci_device_id via686a_pci_ids[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_VIA, PCI_DEVICE_ID_VIA_82C686_4) },
{ }
};
.remove = vt8231_remove,
};
-static DEFINE_PCI_DEVICE_TABLE(vt8231_pci_ids) = {
+static const struct pci_device_id vt8231_pci_ids[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_VIA, PCI_DEVICE_ID_VIA_8231_4) },
{ 0, }
};
continue;
}
}
- buddha_board = ZTWO_VADDR(board);
+ buddha_board = (unsigned long)ZTWO_VADDR(board);
/* write to BUDDHA_IRQ_MR to enable the board IRQ */
/* X-Surf doesn't have this. IRQs are always on */
if (!(lapic_timer_reliable_states & (1 << (cstate))))
clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
- if (!current_set_polling_and_test()) {
-
- if (this_cpu_has(X86_FEATURE_CLFLUSH_MONITOR))
- clflush((void *)¤t_thread_info()->flags);
-
- __monitor((void *)¤t_thread_info()->flags, 0, 0);
- smp_mb();
- if (!need_resched())
- __mwait(eax, ecx);
- }
+ mwait_idle_with_hints(eax, ecx);
if (!(lapic_timer_reliable_states & (1 << (cstate))))
clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
config ADB_MACIISI
bool "Include Mac IIsi ADB driver"
- depends on ADB && MAC
+ depends on ADB && MAC && BROKEN
help
Say Y here if want your kernel to support Macintosh systems that use
the Mac IIsi style ADB. This includes the IIsi, IIvi, IIvx, Classic
rdev->raid_disk = -1;
clear_bit(Faulty, &rdev->flags);
clear_bit(In_sync, &rdev->flags);
+ clear_bit(Bitmap_sync, &rdev->flags);
clear_bit(WriteMostly, &rdev->flags);
if (mddev->raid_disks == 0) {
*/
if (ev1 < mddev->bitmap->events_cleared)
return 0;
+ if (ev1 < mddev->events)
+ set_bit(Bitmap_sync, &rdev->flags);
} else {
if (ev1 < mddev->events)
/* just a hot-add of a new device, leave raid_disk at -1 */
rdev->raid_disk = -1;
clear_bit(Faulty, &rdev->flags);
clear_bit(In_sync, &rdev->flags);
+ clear_bit(Bitmap_sync, &rdev->flags);
clear_bit(WriteMostly, &rdev->flags);
if (mddev->raid_disks == 0) {
*/
if (ev1 < mddev->bitmap->events_cleared)
return 0;
+ if (ev1 < mddev->events)
+ set_bit(Bitmap_sync, &rdev->flags);
} else {
if (ev1 < mddev->events)
/* just a hot-add of a new device, leave raid_disk at -1 */
else
rdev->saved_raid_disk = -1;
clear_bit(In_sync, &rdev->flags);
+ clear_bit(Bitmap_sync, &rdev->flags);
err = rdev->mddev->pers->
hot_add_disk(rdev->mddev, rdev);
if (err) {
info->raid_disk < mddev->raid_disks) {
rdev->raid_disk = info->raid_disk;
set_bit(In_sync, &rdev->flags);
+ clear_bit(Bitmap_sync, &rdev->flags);
} else
rdev->raid_disk = -1;
} else
if (test_bit(Faulty, &rdev->flags))
continue;
if (mddev->ro &&
- rdev->saved_raid_disk < 0)
+ ! (rdev->saved_raid_disk >= 0 &&
+ !test_bit(Bitmap_sync, &rdev->flags)))
continue;
rdev->recovery_offset = 0;
* As we only add devices that are already in-sync,
* we can activate the spares immediately.
*/
- clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
remove_and_add_spares(mddev, NULL);
- mddev->pers->spare_active(mddev);
+ /* There is no thread, but we need to call
+ * ->spare_active and clear saved_raid_disk
+ */
+ md_reap_sync_thread(mddev);
+ clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
goto unlock;
}
enum flag_bits {
Faulty, /* device is known to have a fault */
In_sync, /* device is in_sync with rest of array */
+ Bitmap_sync, /* ..actually, not quite In_sync. Need a
+ * bitmap-based recovery to get fully in sync
+ */
Unmerged, /* device is being added to array and should
* be considerred for bvec_merge_fn but not
* yet for actual IO
conf->next_window_requests++;
else
conf->current_window_requests++;
- }
- if (bio->bi_sector >= conf->start_next_window)
sector = conf->start_next_window;
+ }
}
conf->nr_pending++;
/* Could not read all from this device, so we will
* need another r10_bio.
*/
- sectors_handled = (r10_bio->sectors + max_sectors
+ sectors_handled = (r10_bio->sector + max_sectors
- bio->bi_sector);
r10_bio->sectors = max_sectors;
spin_lock_irq(&conf->device_lock);
bio->bi_phys_segments = 2;
else
bio->bi_phys_segments++;
- spin_unlock(&conf->device_lock);
+ spin_unlock_irq(&conf->device_lock);
/* Cannot call generic_make_request directly
* as that will be queued in __generic_make_request
* and subsequent mempool_alloc might block
if (j == conf->copies) {
/* Cannot recover, so abort the recovery or
* record a bad block */
- put_buf(r10_bio);
- if (rb2)
- atomic_dec(&rb2->remaining);
- r10_bio = rb2;
if (any_working) {
/* problem is that there are bad blocks
* on other device(s)
mirror->recovery_disabled
= mddev->recovery_disabled;
}
+ put_buf(r10_bio);
+ if (rb2)
+ atomic_dec(&rb2->remaining);
+ r10_bio = rb2;
break;
}
}
} else {
if (!test_bit(STRIPE_HANDLE, &sh->state))
atomic_inc(&conf->active_stripes);
- BUG_ON(list_empty(&sh->lru));
+ BUG_ON(list_empty(&sh->lru) &&
+ !test_bit(STRIPE_EXPANDING, &sh->state));
list_del_init(&sh->lru);
if (sh->group) {
sh->group->stripes_cnt--;
*/
set_bit(R5_Insync, &dev->flags);
- if (rdev && test_bit(R5_WriteError, &dev->flags)) {
+ if (test_bit(R5_WriteError, &dev->flags)) {
/* This flag does not apply to '.replacement'
* only to .rdev, so make sure to check that*/
struct md_rdev *rdev2 = rcu_dereference(
} else
clear_bit(R5_WriteError, &dev->flags);
}
- if (rdev && test_bit(R5_MadeGood, &dev->flags)) {
+ if (test_bit(R5_MadeGood, &dev->flags)) {
/* This flag does not apply to '.replacement'
* only to .rdev, so make sure to check that*/
struct md_rdev *rdev2 = rcu_dereference(
}
if (all) {
- rcu_assign_pointer(bond->curr_active_slave, NULL);
+ RCU_INIT_POINTER(bond->curr_active_slave, NULL);
} else if (oldcurrent == slave) {
/*
* Note that we hold RTNL over this sequence, so there
static int hydra_init(struct zorro_dev *z)
{
struct net_device *dev;
- unsigned long board = ZTWO_VADDR(z->resource.start);
+ unsigned long board = (unsigned long)ZTWO_VADDR(z->resource.start);
unsigned long ioaddr = board+HYDRA_NIC_BASE;
const char name[] = "NE2000";
int start_page, stop_page;
};
static int zorro8390_init(struct net_device *dev, unsigned long board,
- const char *name, unsigned long ioaddr)
+ const char *name, void __iomem *ioaddr)
{
int i;
int err;
start_page = NESM_START_PG;
stop_page = NESM_STOP_PG;
- dev->base_addr = ioaddr;
+ dev->base_addr = (unsigned long)ioaddr;
dev->irq = IRQ_AMIGA_PORTS;
/* Install the Interrupt handler */
#include <linux/zorro.h>
#include <linux/bitops.h>
+#include <asm/byteorder.h>
#include <asm/irq.h>
#include <asm/amigaints.h>
#include <asm/amigahw.h>
unsigned long base_addr = board + A2065_LANCE;
unsigned long mem_start = board + A2065_RAM;
struct resource *r1, *r2;
+ u32 serial;
int err;
r1 = request_mem_region(base_addr, sizeof(struct lance_regs),
r1->name = dev->name;
r2->name = dev->name;
+ serial = be32_to_cpu(z->rom.er_SerialNumber);
dev->dev_addr[0] = 0x00;
if (z->id != ZORRO_PROD_AMERISTAR_A2065) { /* Commodore */
dev->dev_addr[1] = 0x80;
dev->dev_addr[1] = 0x00;
dev->dev_addr[2] = 0x9f;
}
- dev->dev_addr[3] = (z->rom.er_SerialNumber >> 16) & 0xff;
- dev->dev_addr[4] = (z->rom.er_SerialNumber >> 8) & 0xff;
- dev->dev_addr[5] = z->rom.er_SerialNumber & 0xff;
- dev->base_addr = ZTWO_VADDR(base_addr);
- dev->mem_start = ZTWO_VADDR(mem_start);
+ dev->dev_addr[3] = (serial >> 16) & 0xff;
+ dev->dev_addr[4] = (serial >> 8) & 0xff;
+ dev->dev_addr[5] = serial & 0xff;
+ dev->base_addr = (unsigned long)ZTWO_VADDR(base_addr);
+ dev->mem_start = (unsigned long)ZTWO_VADDR(mem_start);
dev->mem_end = dev->mem_start + A2065_RAM_SIZE;
priv->ll = (volatile struct lance_regs *)dev->base_addr;
#include <linux/zorro.h>
#include <linux/bitops.h>
+#include <asm/byteorder.h>
#include <asm/amigaints.h>
#include <asm/amigahw.h>
#include <asm/irq.h>
struct resource *r1, *r2;
struct net_device *dev;
struct ariadne_private *priv;
+ u32 serial;
int err;
r1 = request_mem_region(base_addr, sizeof(struct Am79C960), "Am79C960");
r1->name = dev->name;
r2->name = dev->name;
+ serial = be32_to_cpu(z->rom.er_SerialNumber);
dev->dev_addr[0] = 0x00;
dev->dev_addr[1] = 0x60;
dev->dev_addr[2] = 0x30;
- dev->dev_addr[3] = (z->rom.er_SerialNumber >> 16) & 0xff;
- dev->dev_addr[4] = (z->rom.er_SerialNumber >> 8) & 0xff;
- dev->dev_addr[5] = z->rom.er_SerialNumber & 0xff;
- dev->base_addr = ZTWO_VADDR(base_addr);
- dev->mem_start = ZTWO_VADDR(mem_start);
+ dev->dev_addr[3] = (serial >> 16) & 0xff;
+ dev->dev_addr[4] = (serial >> 8) & 0xff;
+ dev->dev_addr[5] = serial & 0xff;
+ dev->base_addr = (unsigned long)ZTWO_VADDR(base_addr);
+ dev->mem_start = (unsigned long)ZTWO_VADDR(mem_start);
dev->mem_end = dev->mem_start + ARIADNE_RAM_SIZE;
dev->netdev_ops = &ariadne_netdev_ops;
pci_set_power_state(pdev, PCI_D3hot);
}
- if (bp->regview)
- iounmap(bp->regview);
+ if (remove_netdev) {
+ if (bp->regview)
+ iounmap(bp->regview);
- /* for vf doorbells are part of the regview and were unmapped along with
- * it. FW is only loaded by PF.
- */
- if (IS_PF(bp)) {
- if (bp->doorbells)
- iounmap(bp->doorbells);
+ /* For vfs, doorbells are part of the regview and were unmapped
+ * along with it. FW is only loaded by PF.
+ */
+ if (IS_PF(bp)) {
+ if (bp->doorbells)
+ iounmap(bp->doorbells);
- bnx2x_release_firmware(bp);
- }
- bnx2x_free_mem_bp(bp);
+ bnx2x_release_firmware(bp);
+ }
+ bnx2x_free_mem_bp(bp);
- if (remove_netdev)
free_netdev(dev);
- if (atomic_read(&pdev->enable_cnt) == 1)
- pci_release_regions(pdev);
+ if (atomic_read(&pdev->enable_cnt) == 1)
+ pci_release_regions(pdev);
+ }
pci_disable_device(pdev);
}
* in the Compressed Filter Tuple.
*/
if (tp->vlan_shift >= 0 && l2t->vlan != VLAN_NONE)
- ntuple |= (F_FT_VLAN_VLD | l2t->vlan) << tp->vlan_shift;
+ ntuple |= (u64)(F_FT_VLAN_VLD | l2t->vlan) << tp->vlan_shift;
if (tp->port_shift >= 0)
ntuple |= (u64)l2t->lport << tp->port_shift;
struct be_rx_page_info *page_info = NULL, *prev_page_info = NULL;
struct be_queue_info *rxq = &rxo->q;
struct page *pagep = NULL;
+ struct device *dev = &adapter->pdev->dev;
struct be_eth_rx_d *rxd;
u64 page_dmaaddr = 0, frag_dmaaddr;
u32 posted, page_offset = 0;
rx_stats(rxo)->rx_post_fail++;
break;
}
- page_dmaaddr = dma_map_page(&adapter->pdev->dev, pagep,
- 0, adapter->big_page_size,
+ page_dmaaddr = dma_map_page(dev, pagep, 0,
+ adapter->big_page_size,
DMA_FROM_DEVICE);
+ if (dma_mapping_error(dev, page_dmaaddr)) {
+ put_page(pagep);
+ pagep = NULL;
+ rx_stats(rxo)->rx_post_fail++;
+ break;
+ }
page_info->page_offset = 0;
} else {
get_page(pagep);
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int e1000_suspend(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
return __e1000_resume(pdev);
}
-#endif /* CONFIG_PM */
+#endif /* CONFIG_PM_SLEEP */
#ifdef CONFIG_PM_RUNTIME
static int e1000_runtime_suspend(struct device *dev)
};
MODULE_DEVICE_TABLE(pci, e1000_pci_tbl);
-#ifdef CONFIG_PM
static const struct dev_pm_ops e1000_pm_ops = {
SET_SYSTEM_SLEEP_PM_OPS(e1000_suspend, e1000_resume)
SET_RUNTIME_PM_OPS(e1000_runtime_suspend, e1000_runtime_resume,
e1000_idle)
};
-#endif
/* PCI Device API Driver */
static struct pci_driver e1000_driver = {
.id_table = e1000_pci_tbl,
.probe = e1000_probe,
.remove = e1000_remove,
-#ifdef CONFIG_PM
.driver = {
.pm = &e1000_pm_ops,
},
-#endif
.shutdown = e1000_shutdown,
.err_handler = &e1000_err_handler
};
#include <linux/bitrev.h>
#include <linux/slab.h>
-#include <asm/bootinfo.h>
#include <asm/pgtable.h>
#include <asm/io.h>
#include <asm/hwtest.h>
NETIF_F_RXCSUM;
ndev->features = ndev->hw_features;
ndev->vlan_features = ndev->hw_features;
+ /* vlan gets same features (except vlan filter) */
+ ndev->vlan_features &= ~NETIF_F_HW_VLAN_CTAG_FILTER;
if (test_bit(QL_DMA64, &qdev->flags))
ndev->features |= NETIF_F_HIGHDMA;
goto out_unlock;
napi_disable(&rp->napi);
+ netif_tx_disable(dev);
spin_lock_bh(&rp->lock);
/* clear all descriptors */
USB_DEVICE(0x0a46, 0x9621), /* DM9621A USB to Fast Ethernet Adapter */
.driver_info = (unsigned long)&dm9601_info,
},
+ {
+ USB_DEVICE(0x0a46, 0x9622), /* DM9622 USB to Fast Ethernet Adapter */
+ .driver_info = (unsigned long)&dm9601_info,
+ },
+ {
+ USB_DEVICE(0x0a46, 0x0269), /* DM962OA USB to Fast Ethernet Adapter */
+ .driver_info = (unsigned long)&dm9601_info,
+ },
+ {
+ USB_DEVICE(0x0a46, 0x1269), /* DM9621A USB to Fast Ethernet Adapter */
+ .driver_info = (unsigned long)&dm9601_info,
+ },
{}, // END
};
return -ENOMEM;
urb->num_sgs = num_sgs;
- sg_init_table(urb->sg, urb->num_sgs);
+ sg_init_table(urb->sg, urb->num_sgs + 1);
sg_set_buf(&urb->sg[s++], skb->data, skb_headlen(skb));
total_len += skb_headlen(skb);
if (!request_mem_region(piabase, sizeof(struct pia), "PIA"))
continue;
- pp = (struct pia *)ZTWO_VADDR(piabase);
+ pp = ZTWO_VADDR(piabase);
pp->crb = 0;
pp->pddrb = 255; /* all data pins output */
pp->crb = PIA_DDR|32|8;
static const struct acpi_device_id byt_gpio_acpi_match[] = {
{ "INT33B2", 0 },
- { "INT33FC", 0 },
{ }
};
MODULE_DEVICE_TABLE(acpi, byt_gpio_acpi_match);
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/platform_device.h>
-#include <linux/mod_devicetable.h>
#include <linux/log2.h>
#include <linux/pm.h>
#include <linux/of.h>
#include <linux/of_platform.h>
+#include <linux/dmi.h>
/* this is for "generic access to PC-style RTC" using CMOS_READ/CMOS_WRITE */
#include <asm-generic/rtc.h>
return 0;
}
+/*
+ * Do not disable RTC alarm on shutdown - workaround for b0rked BIOSes.
+ */
+static bool alarm_disable_quirk;
+
+static int __init set_alarm_disable_quirk(const struct dmi_system_id *id)
+{
+ alarm_disable_quirk = true;
+ pr_info("rtc-cmos: BIOS has alarm-disable quirk. ");
+ pr_info("RTC alarms disabled\n");
+ return 0;
+}
+
+static const struct dmi_system_id rtc_quirks[] __initconst = {
+ /* https://bugzilla.novell.com/show_bug.cgi?id=805740 */
+ {
+ .callback = set_alarm_disable_quirk,
+ .ident = "IBM Truman",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "TOSHIBA"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "4852570"),
+ },
+ },
+ /* https://bugzilla.novell.com/show_bug.cgi?id=812592 */
+ {
+ .callback = set_alarm_disable_quirk,
+ .ident = "Gigabyte GA-990XA-UD3",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR,
+ "Gigabyte Technology Co., Ltd."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "GA-990XA-UD3"),
+ },
+ },
+ /* http://permalink.gmane.org/gmane.linux.kernel/1604474 */
+ {
+ .callback = set_alarm_disable_quirk,
+ .ident = "Toshiba Satellite L300",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "TOSHIBA"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Satellite L300"),
+ },
+ },
+ {}
+};
+
static int cmos_alarm_irq_enable(struct device *dev, unsigned int enabled)
{
struct cmos_rtc *cmos = dev_get_drvdata(dev);
if (!is_valid_irq(cmos->irq))
return -EINVAL;
+ if (alarm_disable_quirk)
+ return 0;
+
spin_lock_irqsave(&rtc_lock, flags);
if (enabled)
platform_driver_registered = true;
}
+ dmi_check_system(rtc_quirks);
+
if (retval == 0)
return 0;
if (test_bit(DASD_FLAG_SAFE_OFFLINE_RUNNING, &device->flags)) {
/*
- * safe offline allready running
+ * safe offline already running
* could only be called by normal offline so safe_offline flag
* needs to be removed to run normal offline and kill all I/O
*/
extern u8 sclp_fac84;
extern unsigned long long sclp_rzm;
extern unsigned long long sclp_rnmax;
-extern __initdata int sclp_early_read_info_sccb_valid;
/* useful inlines */
if (OLDMEM_BASE) /* No standby memory in kdump mode */
return 0;
- if (!sclp_early_read_info_sccb_valid)
- return 0;
if ((sclp_facilities & 0xe00000000000ULL) != 0xe00000000000ULL)
return 0;
rc = -ENOMEM;
u8 _reserved5[4096 - 112]; /* 112-4095 */
} __packed __aligned(PAGE_SIZE);
-static __initdata struct read_info_sccb early_read_info_sccb;
-static __initdata char sccb_early[PAGE_SIZE] __aligned(PAGE_SIZE);
+static char sccb_early[PAGE_SIZE] __aligned(PAGE_SIZE) __initdata;
+static unsigned int sclp_con_has_vt220 __initdata;
+static unsigned int sclp_con_has_linemode __initdata;
static unsigned long sclp_hsa_size;
+static struct sclp_ipl_info sclp_ipl_info;
-__initdata int sclp_early_read_info_sccb_valid;
u64 sclp_facilities;
u8 sclp_fac84;
unsigned long long sclp_rzm;
return rc;
}
-static void __init sclp_read_info_early(void)
+static int __init sclp_read_info_early(struct read_info_sccb *sccb)
{
- int rc;
- int i;
- struct read_info_sccb *sccb;
+ int rc, i;
sclp_cmdw_t commands[] = {SCLP_CMDW_READ_SCP_INFO_FORCED,
SCLP_CMDW_READ_SCP_INFO};
- sccb = &early_read_info_sccb;
for (i = 0; i < ARRAY_SIZE(commands); i++) {
do {
memset(sccb, 0, sizeof(*sccb));
if (rc)
break;
- if (sccb->header.response_code == 0x10) {
- sclp_early_read_info_sccb_valid = 1;
- break;
- }
+ if (sccb->header.response_code == 0x10)
+ return 0;
if (sccb->header.response_code != 0x1f0)
break;
}
+ return -EIO;
}
-static void __init sclp_facilities_detect(void)
+static void __init sclp_facilities_detect(struct read_info_sccb *sccb)
{
- struct read_info_sccb *sccb;
-
- sclp_read_info_early();
- if (!sclp_early_read_info_sccb_valid)
+ if (sclp_read_info_early(sccb))
return;
- sccb = &early_read_info_sccb;
sclp_facilities = sccb->facilities;
sclp_fac84 = sccb->fac84;
if (sccb->fac85 & 0x02)
sclp_rnmax = sccb->rnmax ? sccb->rnmax : sccb->rnmax2;
sclp_rzm = sccb->rnsize ? sccb->rnsize : sccb->rnsize2;
sclp_rzm <<= 20;
+
+ /* Save IPL information */
+ sclp_ipl_info.is_valid = 1;
+ if (sccb->flags & 0x2)
+ sclp_ipl_info.has_dump = 1;
+ memcpy(&sclp_ipl_info.loadparm, &sccb->loadparm, LOADPARM_LEN);
}
bool __init sclp_has_linemode(void)
{
- struct init_sccb *sccb = (void *) &sccb_early;
-
- if (sccb->header.response_code != 0x20)
- return 0;
- if (!(sccb->sclp_send_mask & (EVTYP_OPCMD_MASK | EVTYP_PMSGCMD_MASK)))
- return 0;
- if (!(sccb->sclp_receive_mask & (EVTYP_MSG_MASK | EVTYP_PMSGCMD_MASK)))
- return 0;
- return 1;
+ return !!sclp_con_has_linemode;
}
bool __init sclp_has_vt220(void)
{
- struct init_sccb *sccb = (void *) &sccb_early;
-
- if (sccb->header.response_code != 0x20)
- return 0;
- if (sccb->sclp_send_mask & EVTYP_VT220MSG_MASK)
- return 1;
- return 0;
+ return !!sclp_con_has_vt220;
}
unsigned long long sclp_get_rnmax(void)
/*
* This function will be called after sclp_facilities_detect(), which gets
- * called from early.c code. Therefore the sccb should have valid contents.
+ * called from early.c code. The sclp_facilities_detect() function retrieves
+ * and saves the IPL information.
*/
void __init sclp_get_ipl_info(struct sclp_ipl_info *info)
{
- struct read_info_sccb *sccb;
-
- if (!sclp_early_read_info_sccb_valid)
- return;
- sccb = &early_read_info_sccb;
- info->is_valid = 1;
- if (sccb->flags & 0x2)
- info->has_dump = 1;
- memcpy(&info->loadparm, &sccb->loadparm, LOADPARM_LEN);
+ *info = sclp_ipl_info;
}
static int __init sclp_cmd_early(sclp_cmdw_t cmd, void *sccb)
sccb->evbuf.dbs = 1;
}
-static int __init sclp_set_event_mask(unsigned long receive_mask,
+static int __init sclp_set_event_mask(struct init_sccb *sccb,
+ unsigned long receive_mask,
unsigned long send_mask)
{
- struct init_sccb *sccb = (void *) &sccb_early;
-
memset(sccb, 0, sizeof(*sccb));
sccb->header.length = sizeof(*sccb);
sccb->mask_length = sizeof(sccb_mask_t);
return sclp_cmd_early(SCLP_CMDW_WRITE_EVENT_MASK, sccb);
}
-static long __init sclp_hsa_size_init(void)
+static long __init sclp_hsa_size_init(struct sdias_sccb *sccb)
{
- struct sdias_sccb *sccb = (void *) &sccb_early;
-
sccb_init_eq_size(sccb);
if (sclp_cmd_early(SCLP_CMDW_WRITE_EVENT_DATA, sccb))
return -EIO;
return 0;
}
-static long __init sclp_hsa_copy_wait(void)
+static long __init sclp_hsa_copy_wait(struct sccb_header *sccb)
{
- struct sccb_header *sccb = (void *) &sccb_early;
-
memset(sccb, 0, PAGE_SIZE);
sccb->length = PAGE_SIZE;
if (sclp_cmd_early(SCLP_CMDW_READ_EVENT_DATA, sccb))
return sclp_hsa_size;
}
-static void __init sclp_hsa_size_detect(void)
+static void __init sclp_hsa_size_detect(void *sccb)
{
long size;
/* First try synchronous interface (LPAR) */
- if (sclp_set_event_mask(0, 0x40000010))
+ if (sclp_set_event_mask(sccb, 0, 0x40000010))
return;
- size = sclp_hsa_size_init();
+ size = sclp_hsa_size_init(sccb);
if (size < 0)
return;
if (size != 0)
goto out;
/* Then try asynchronous interface (z/VM) */
- if (sclp_set_event_mask(0x00000010, 0x40000010))
+ if (sclp_set_event_mask(sccb, 0x00000010, 0x40000010))
return;
- size = sclp_hsa_size_init();
+ size = sclp_hsa_size_init(sccb);
if (size < 0)
return;
- size = sclp_hsa_copy_wait();
+ size = sclp_hsa_copy_wait(sccb);
if (size < 0)
return;
out:
sclp_hsa_size = size;
}
+static unsigned int __init sclp_con_check_linemode(struct init_sccb *sccb)
+{
+ if (!(sccb->sclp_send_mask & (EVTYP_OPCMD_MASK | EVTYP_PMSGCMD_MASK)))
+ return 0;
+ if (!(sccb->sclp_receive_mask & (EVTYP_MSG_MASK | EVTYP_PMSGCMD_MASK)))
+ return 0;
+ return 1;
+}
+
+static void __init sclp_console_detect(struct init_sccb *sccb)
+{
+ if (sccb->header.response_code != 0x20)
+ return;
+
+ if (sccb->sclp_send_mask & EVTYP_VT220MSG_MASK)
+ sclp_con_has_vt220 = 1;
+
+ if (sclp_con_check_linemode(sccb))
+ sclp_con_has_linemode = 1;
+}
+
void __init sclp_early_detect(void)
{
- sclp_facilities_detect();
- sclp_hsa_size_detect();
- sclp_set_event_mask(0, 0);
+ void *sccb = &sccb_early;
+
+ sclp_facilities_detect(sccb);
+ sclp_hsa_size_detect(sccb);
+
+ /* Turn off SCLP event notifications. Also save remote masks in the
+ * sccb. These are sufficient to detect sclp console capabilities.
+ */
+ sclp_set_event_mask(sccb, 0, 0);
+ sclp_console_detect(sccb);
}
*/
static void tty3270_set_timer(struct tty3270 *tp, int expires)
{
- if (expires == 0)
- del_timer(&tp->timer);
- else
- mod_timer(&tp->timer, jiffies + expires);
+ mod_timer(&tp->timer, jiffies + expires);
}
/*
{
int pages;
- del_timer_sync(&tp->timer);
kbd_free(tp->kbd);
raw3270_request_free(tp->kreset);
raw3270_request_free(tp->read);
{
struct tty3270 *tp = container_of(view, struct tty3270, view);
+ del_timer_sync(&tp->timer);
tty3270_free_screen(tp->screen, tp->view.rows);
tty3270_free_view(tp);
}
parm = strsep(&buf, " ");
- if (strcmp("free", parm) == 0)
+ if (strcmp("free", parm) == 0) {
rc = blacklist_parse_parameters(buf, free, 0);
- else if (strcmp("add", parm) == 0)
+ css_schedule_eval_all_unreg(0);
+ } else if (strcmp("add", parm) == 0)
rc = blacklist_parse_parameters(buf, add, 0);
else if (strcmp("purge", parm) == 0)
return ccw_purge_blacklisted();
else
return -EINVAL;
- css_schedule_reprobe();
return rc;
}
const char *buf, size_t count)
{
struct ccwgroup_device *gdev = to_ccwgroupdev(dev);
- struct ccwgroup_driver *gdrv = to_ccwgroupdrv(dev->driver);
unsigned long value;
int ret;
- if (!dev->driver)
- return -EINVAL;
- if (!try_module_get(gdrv->driver.owner))
- return -EINVAL;
+ device_lock(dev);
+ if (!dev->driver) {
+ ret = -EINVAL;
+ goto out;
+ }
ret = kstrtoul(buf, 0, &value);
if (ret)
else
ret = -EINVAL;
out:
- module_put(gdrv->driver.owner);
+ device_unlock(dev);
return (ret == 0) ? count : ret;
}
case 0x0004:
return -EOPNOTSUPP;
case 0x000b:
+ case 0x0107: /* "Channel busy" for the op 0x003d */
return -EBUSY;
case 0x0100:
case 0x0102:
for_each_subchannel_staged(s390_subchannel_remove_chpid, NULL, &link);
}
-static int s390_process_res_acc_new_sch(struct subchannel_id schid, void *data)
-{
- struct schib schib;
- /*
- * We don't know the device yet, but since a path
- * may be available now to the device we'll have
- * to do recognition again.
- * Since we don't have any idea about which chpid
- * that beast may be on we'll have to do a stsch
- * on all devices, grr...
- */
- if (stsch_err(schid, &schib))
- /* We're through */
- return -ENXIO;
-
- /* Put it on the slow path. */
- css_schedule_eval(schid);
- return 0;
-}
-
static int __s390_process_res_acc(struct subchannel *sch, void *data)
{
spin_lock_irq(sch->lock);
* The more information we have (info), the less scanning
* will we have to do.
*/
- for_each_subchannel_staged(__s390_process_res_acc,
- s390_process_res_acc_new_sch, link);
+ for_each_subchannel_staged(__s390_process_res_acc, NULL, link);
+ css_schedule_reprobe();
}
static int
return 0;
}
-static int
-__s390_vary_chpid_on(struct subchannel_id schid, void *data)
-{
- struct schib schib;
-
- if (stsch_err(schid, &schib))
- /* We're through */
- return -ENXIO;
- /* Put it on the slow path. */
- css_schedule_eval(schid);
- return 0;
-}
-
/**
* chsc_chp_vary - propagate channel-path vary operation to subchannels
* @chpid: channl-path ID
/* Try to update the channel path description. */
chp_update_desc(chp);
for_each_subchannel_staged(s390_subchannel_vary_chpid_on,
- __s390_vary_chpid_on, &chpid);
+ NULL, &chpid);
+ css_schedule_reprobe();
} else
for_each_subchannel_staged(s390_subchannel_vary_chpid_off,
NULL, &chpid);
return ret;
}
EXPORT_SYMBOL_GPL(chsc_scm_info);
+
+/**
+ * chsc_pnso_brinfo() - Perform Network-Subchannel Operation, Bridge Info.
+ * @schid: id of the subchannel on which PNSO is performed
+ * @brinfo_area: request and response block for the operation
+ * @resume_token: resume token for multiblock response
+ * @cnc: Boolean change-notification control
+ *
+ * brinfo_area must be allocated by the caller with get_zeroed_page(GFP_KERNEL)
+ *
+ * Returns 0 on success.
+ */
+int chsc_pnso_brinfo(struct subchannel_id schid,
+ struct chsc_pnso_area *brinfo_area,
+ struct chsc_brinfo_resume_token resume_token,
+ int cnc)
+{
+ memset(brinfo_area, 0, sizeof(*brinfo_area));
+ brinfo_area->request.length = 0x0030;
+ brinfo_area->request.code = 0x003d; /* network-subchannel operation */
+ brinfo_area->m = schid.m;
+ brinfo_area->ssid = schid.ssid;
+ brinfo_area->sch = schid.sch_no;
+ brinfo_area->cssid = schid.cssid;
+ brinfo_area->oc = 0; /* Store-network-bridging-information list */
+ brinfo_area->resume_token = resume_token;
+ brinfo_area->n = (cnc != 0);
+ if (chsc(brinfo_area))
+ return -EIO;
+ return chsc_error_from_response(brinfo_area->response.code);
+}
+EXPORT_SYMBOL_GPL(chsc_pnso_brinfo);
u32 : 20;
u32 scssc : 1; /* bit 107 */
u32 scsscf : 1; /* bit 108 */
- u32 : 19;
+ u32:7;
+ u32 pnso:1; /* bit 116 */
+ u32:11;
}__attribute__((packed));
extern struct css_chsc_char css_chsc_characteristics;
int chsc_scm_info(struct chsc_scm_info *scm_area, u64 token);
+struct chsc_brinfo_resume_token {
+ u64 t1;
+ u64 t2;
+} __packed;
+
+struct chsc_brinfo_naihdr {
+ struct chsc_brinfo_resume_token resume_token;
+ u32:32;
+ u32 instance;
+ u32:24;
+ u8 naids;
+ u32 reserved[3];
+} __packed;
+
+struct chsc_pnso_area {
+ struct chsc_header request;
+ u8:2;
+ u8 m:1;
+ u8:5;
+ u8:2;
+ u8 ssid:2;
+ u8 fmt:4;
+ u16 sch;
+ u8:8;
+ u8 cssid;
+ u16:16;
+ u8 oc;
+ u32:24;
+ struct chsc_brinfo_resume_token resume_token;
+ u32 n:1;
+ u32:31;
+ u32 reserved[3];
+ struct chsc_header response;
+ u32:32;
+ struct chsc_brinfo_naihdr naihdr;
+ union {
+ struct qdio_brinfo_entry_l3_ipv6 l3_ipv6[0];
+ struct qdio_brinfo_entry_l3_ipv4 l3_ipv4[0];
+ struct qdio_brinfo_entry_l2 l2[0];
+ } entries;
+} __packed;
+
+int chsc_pnso_brinfo(struct subchannel_id schid,
+ struct chsc_pnso_area *brinfo_area,
+ struct chsc_brinfo_resume_token resume_token,
+ int cnc);
+
#ifdef CONFIG_SCM_BUS
int scm_update_information(void);
int scm_process_availability_information(void);
struct cb_data *cb = data;
int rc = 0;
- idset_sch_del(cb->set, sch->schid);
+ if (cb->set)
+ idset_sch_del(cb->set, sch->schid);
if (cb->fn_known_sch)
rc = cb->fn_known_sch(sch, cb->data);
return rc;
cb.fn_known_sch = fn_known;
cb.fn_unknown_sch = fn_unknown;
+ if (fn_known && !fn_unknown) {
+ /* Skip idset allocation in case of known-only loop. */
+ cb.set = NULL;
+ return bus_for_each_dev(&css_bus_type, NULL, &cb,
+ call_fn_known_sch);
+ }
+
cb.set = idset_sch_new();
if (!cb.set)
/* fall back to brute force scanning in case of oom */
default:
rc = 0;
}
+ /* Allow scheduling here since the containing loop might
+ * take a while. */
+ cond_resched();
}
return rc;
}
spin_unlock_irqrestore(&slow_subchannel_lock, flags);
}
-static DECLARE_WORK(slow_path_work, css_slow_path_func);
+static DECLARE_DELAYED_WORK(slow_path_work, css_slow_path_func);
struct workqueue_struct *cio_work_q;
void css_schedule_eval(struct subchannel_id schid)
spin_lock_irqsave(&slow_subchannel_lock, flags);
idset_sch_add(slow_subchannel_set, schid);
atomic_set(&css_eval_scheduled, 1);
- queue_work(cio_work_q, &slow_path_work);
+ queue_delayed_work(cio_work_q, &slow_path_work, 0);
spin_unlock_irqrestore(&slow_subchannel_lock, flags);
}
spin_lock_irqsave(&slow_subchannel_lock, flags);
idset_fill(slow_subchannel_set);
atomic_set(&css_eval_scheduled, 1);
- queue_work(cio_work_q, &slow_path_work);
+ queue_delayed_work(cio_work_q, &slow_path_work, 0);
spin_unlock_irqrestore(&slow_subchannel_lock, flags);
}
return 0;
}
-static void css_schedule_eval_all_unreg(void)
+void css_schedule_eval_all_unreg(unsigned long delay)
{
unsigned long flags;
struct idset *unreg_set;
spin_lock_irqsave(&slow_subchannel_lock, flags);
idset_add_set(slow_subchannel_set, unreg_set);
atomic_set(&css_eval_scheduled, 1);
- queue_work(cio_work_q, &slow_path_work);
+ queue_delayed_work(cio_work_q, &slow_path_work, delay);
spin_unlock_irqrestore(&slow_subchannel_lock, flags);
idset_free(unreg_set);
}
/* Schedule reprobing of all unregistered subchannels. */
void css_schedule_reprobe(void)
{
- css_schedule_eval_all_unreg();
+ /* Schedule with a delay to allow merging of subsequent calls. */
+ css_schedule_eval_all_unreg(1 * HZ);
}
EXPORT_SYMBOL_GPL(css_schedule_reprobe);
/* Helper functions to build lists for the slow path. */
void css_schedule_eval(struct subchannel_id schid);
void css_schedule_eval_all(void);
+void css_schedule_eval_all_unreg(unsigned long delay);
int css_complete_work(void);
int sch_is_pseudo_sch(struct subchannel *);
if (ret != 0)
return ret;
}
- cdev->online = 0;
spin_lock_irq(cdev->ccwlock);
sch = to_subchannel(cdev->dev.parent);
+ cdev->online = 0;
/* Wait until a final state or DISCONNECTED is reached */
while (!dev_fsm_final_state(cdev) &&
cdev->private->state != DEV_STATE_DISCONNECTED) {
ret = cdev->drv->set_online(cdev);
if (ret)
goto rollback;
+
+ spin_lock_irq(cdev->ccwlock);
cdev->online = 1;
+ spin_unlock_irq(cdev->ccwlock);
return 0;
rollback:
if (!dev_fsm_final_state(cdev) &&
cdev->private->state != DEV_STATE_DISCONNECTED) {
ret = -EAGAIN;
- goto out_onoff;
+ goto out;
}
/* Prevent conflict between pending work and on-/offline processing.*/
if (work_pending(&cdev->private->todo_work)) {
ret = -EAGAIN;
- goto out_onoff;
- }
-
- if (cdev->drv && !try_module_get(cdev->drv->driver.owner)) {
- ret = -EINVAL;
- goto out_onoff;
+ goto out;
}
if (!strncmp(buf, "force\n", count)) {
force = 1;
}
if (ret)
goto out;
+
+ device_lock(dev);
switch (i) {
case 0:
ret = online_store_handle_offline(cdev);
default:
ret = -EINVAL;
}
+ device_unlock(dev);
+
out:
- if (cdev->drv)
- module_put(cdev->drv->driver.owner);
-out_onoff:
atomic_set(&cdev->private->onoff, 0);
return (ret < 0) ? ret : count;
}
return 0;
}
-static int
-ccw_device_remove (struct device *dev)
+static int ccw_device_remove(struct device *dev)
{
struct ccw_device *cdev = to_ccwdev(dev);
struct ccw_driver *cdrv = cdev->drv;
if (cdrv->remove)
cdrv->remove(cdev);
+
+ spin_lock_irq(cdev->ccwlock);
if (cdev->online) {
cdev->online = 0;
- spin_lock_irq(cdev->ccwlock);
ret = ccw_device_offline(cdev);
spin_unlock_irq(cdev->ccwlock);
if (ret == 0)
cdev->private->dev_id.devno);
/* Give up reference obtained in ccw_device_set_online(). */
put_device(&cdev->dev);
+ spin_lock_irq(cdev->ccwlock);
}
ccw_device_set_timeout(cdev, 0);
cdev->drv = NULL;
cdev->private->int_class = IRQIO_CIO;
+ spin_unlock_irq(cdev->ccwlock);
return 0;
}
}
EXPORT_SYMBOL(qdio_stop_irq);
+/**
+ * qdio_pnso_brinfo() - perform network subchannel op #0 - bridge info.
+ * @schid: Subchannel ID.
+ * @cnc: Boolean Change-Notification Control
+ * @response: Response code will be stored at this address
+ * @cb: Callback function will be executed for each element
+ * of the address list
+ * @priv: Pointer passed from the caller to qdio_pnso_brinfo()
+ * @type: Type of the address entry passed to the callback
+ * @entry: Entry containg the address of the specified type
+ * @priv: Pointer to pass to the callback function.
+ *
+ * Performs "Store-network-bridging-information list" operation and calls
+ * the callback function for every entry in the list. If "change-
+ * notification-control" is set, further changes in the address list
+ * will be reported via the IPA command.
+ */
+int qdio_pnso_brinfo(struct subchannel_id schid,
+ int cnc, u16 *response,
+ void (*cb)(void *priv, enum qdio_brinfo_entry_type type,
+ void *entry),
+ void *priv)
+{
+ struct chsc_pnso_area *rr;
+ int rc;
+ u32 prev_instance = 0;
+ int isfirstblock = 1;
+ int i, size, elems;
+
+ rr = (struct chsc_pnso_area *)get_zeroed_page(GFP_KERNEL);
+ if (rr == NULL)
+ return -ENOMEM;
+ do {
+ /* on the first iteration, naihdr.resume_token will be zero */
+ rc = chsc_pnso_brinfo(schid, rr, rr->naihdr.resume_token, cnc);
+ if (rc != 0 && rc != -EBUSY)
+ goto out;
+ if (rr->response.code != 1) {
+ rc = -EIO;
+ continue;
+ } else
+ rc = 0;
+
+ if (cb == NULL)
+ continue;
+
+ size = rr->naihdr.naids;
+ elems = (rr->response.length -
+ sizeof(struct chsc_header) -
+ sizeof(struct chsc_brinfo_naihdr)) /
+ size;
+
+ if (!isfirstblock && (rr->naihdr.instance != prev_instance)) {
+ /* Inform the caller that they need to scrap */
+ /* the data that was already reported via cb */
+ rc = -EAGAIN;
+ break;
+ }
+ isfirstblock = 0;
+ prev_instance = rr->naihdr.instance;
+ for (i = 0; i < elems; i++)
+ switch (size) {
+ case sizeof(struct qdio_brinfo_entry_l3_ipv6):
+ (*cb)(priv, l3_ipv6_addr,
+ &rr->entries.l3_ipv6[i]);
+ break;
+ case sizeof(struct qdio_brinfo_entry_l3_ipv4):
+ (*cb)(priv, l3_ipv4_addr,
+ &rr->entries.l3_ipv4[i]);
+ break;
+ case sizeof(struct qdio_brinfo_entry_l2):
+ (*cb)(priv, l2_addr_lnid,
+ &rr->entries.l2[i]);
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ rc = -EIO;
+ goto out;
+ }
+ } while (rr->response.code == 0x0107 || /* channel busy */
+ (rr->response.code == 1 && /* list stored */
+ /* resume token is non-zero => list incomplete */
+ (rr->naihdr.resume_token.t1 || rr->naihdr.resume_token.t2)));
+ (*response) = rr->response.code;
+
+out:
+ free_page((unsigned long)rr);
+ return rc;
+}
+EXPORT_SYMBOL_GPL(qdio_pnso_brinfo);
+
static int __init init_QDIO(void)
{
int rc;
if (rc != -ENODEV && rc != -EBUSY)
break;
if (i < AP_MAX_RESET - 1) {
- udelay(5);
+ /* Time we are waiting until we give up (0.7sec * 90).
+ * Since the actual request (in progress) will not
+ * interrupted immediately for the reset command,
+ * we have to be patient. In worst case we have to
+ * wait 60sec + reset time (some msec).
+ */
+ schedule_timeout(AP_RESET_TIMEOUT);
status = ap_test_queue(qid, &dummy, &dummy);
}
}
static BUS_ATTR(ap_domain, 0444, ap_domain_show, NULL);
+static ssize_t ap_control_domain_mask_show(struct bus_type *bus, char *buf)
+{
+ if (ap_configuration != NULL) { /* QCI not supported */
+ if (test_facility(76)) { /* format 1 - 256 bit domain field */
+ return snprintf(buf, PAGE_SIZE,
+ "0x%08x%08x%08x%08x%08x%08x%08x%08x\n",
+ ap_configuration->adm[0], ap_configuration->adm[1],
+ ap_configuration->adm[2], ap_configuration->adm[3],
+ ap_configuration->adm[4], ap_configuration->adm[5],
+ ap_configuration->adm[6], ap_configuration->adm[7]);
+ } else { /* format 0 - 16 bit domain field */
+ return snprintf(buf, PAGE_SIZE, "%08x%08x\n",
+ ap_configuration->adm[0], ap_configuration->adm[1]);
+ }
+ } else {
+ return snprintf(buf, PAGE_SIZE, "not supported\n");
+ }
+}
+
+static BUS_ATTR(ap_control_domain_mask, 0444,
+ ap_control_domain_mask_show, NULL);
+
static ssize_t ap_config_time_show(struct bus_type *bus, char *buf)
{
return snprintf(buf, PAGE_SIZE, "%d\n", ap_config_time);
static struct bus_attribute *const ap_bus_attrs[] = {
&bus_attr_ap_domain,
+ &bus_attr_ap_control_domain_mask,
&bus_attr_config_time,
&bus_attr_poll_thread,
&bus_attr_ap_interrupts,
#define AP_DEVICES 64 /* Number of AP devices. */
#define AP_DOMAINS 16 /* Number of AP domains. */
#define AP_MAX_RESET 90 /* Maximum number of resets. */
-#define AP_RESET_TIMEOUT (HZ/2) /* Time in ticks for reset timeouts. */
+#define AP_RESET_TIMEOUT (HZ*0.7) /* Time in ticks for reset timeouts. */
#define AP_CONFIG_TIME 30 /* Time in seconds between AP bus rescans. */
#define AP_POLL_TIME 1 /* Time in ticks between receive polls. */
#define AP_FUNC_CRT4K 2
#define AP_FUNC_COPRO 3
#define AP_FUNC_ACCEL 4
+#define AP_FUNC_EP11 5
+#define AP_FUNC_APXA 6
/*
* AP reset flag states
#include "zcrypt_debug.h"
#include "zcrypt_api.h"
+#include "zcrypt_msgtype6.h"
+
/*
* Module description.
*/
spin_lock_bh(&zcrypt_device_lock);
list_for_each_entry(zdev, &zcrypt_device_list, list) {
if (!zdev->online || !zdev->ops->send_cprb ||
- (xcRB->user_defined != AUTOSELECT &&
- AP_QID_DEVICE(zdev->ap_dev->qid) != xcRB->user_defined)
- )
+ (zdev->ops->variant == MSGTYPE06_VARIANT_EP11) ||
+ (xcRB->user_defined != AUTOSELECT &&
+ AP_QID_DEVICE(zdev->ap_dev->qid) != xcRB->user_defined))
continue;
zcrypt_device_get(zdev);
get_device(&zdev->ap_dev->device);
return -ENODEV;
}
+struct ep11_target_dev_list {
+ unsigned short targets_num;
+ struct ep11_target_dev *targets;
+};
+
+static bool is_desired_ep11dev(unsigned int dev_qid,
+ struct ep11_target_dev_list dev_list)
+{
+ int n;
+
+ for (n = 0; n < dev_list.targets_num; n++, dev_list.targets++) {
+ if ((AP_QID_DEVICE(dev_qid) == dev_list.targets->ap_id) &&
+ (AP_QID_QUEUE(dev_qid) == dev_list.targets->dom_id)) {
+ return true;
+ }
+ }
+ return false;
+}
+
+static long zcrypt_send_ep11_cprb(struct ep11_urb *xcrb)
+{
+ struct zcrypt_device *zdev;
+ bool autoselect = false;
+ int rc;
+ struct ep11_target_dev_list ep11_dev_list = {
+ .targets_num = 0x00,
+ .targets = NULL,
+ };
+
+ ep11_dev_list.targets_num = (unsigned short) xcrb->targets_num;
+
+ /* empty list indicates autoselect (all available targets) */
+ if (ep11_dev_list.targets_num == 0)
+ autoselect = true;
+ else {
+ ep11_dev_list.targets = kcalloc((unsigned short)
+ xcrb->targets_num,
+ sizeof(struct ep11_target_dev),
+ GFP_KERNEL);
+ if (!ep11_dev_list.targets)
+ return -ENOMEM;
+
+ if (copy_from_user(ep11_dev_list.targets,
+ (struct ep11_target_dev *)xcrb->targets,
+ xcrb->targets_num *
+ sizeof(struct ep11_target_dev)))
+ return -EFAULT;
+ }
+
+ spin_lock_bh(&zcrypt_device_lock);
+ list_for_each_entry(zdev, &zcrypt_device_list, list) {
+ /* check if device is eligible */
+ if (!zdev->online ||
+ zdev->ops->variant != MSGTYPE06_VARIANT_EP11)
+ continue;
+
+ /* check if device is selected as valid target */
+ if (!is_desired_ep11dev(zdev->ap_dev->qid, ep11_dev_list) &&
+ !autoselect)
+ continue;
+
+ zcrypt_device_get(zdev);
+ get_device(&zdev->ap_dev->device);
+ zdev->request_count++;
+ __zcrypt_decrease_preference(zdev);
+ if (try_module_get(zdev->ap_dev->drv->driver.owner)) {
+ spin_unlock_bh(&zcrypt_device_lock);
+ rc = zdev->ops->send_ep11_cprb(zdev, xcrb);
+ spin_lock_bh(&zcrypt_device_lock);
+ module_put(zdev->ap_dev->drv->driver.owner);
+ } else {
+ rc = -EAGAIN;
+ }
+ zdev->request_count--;
+ __zcrypt_increase_preference(zdev);
+ put_device(&zdev->ap_dev->device);
+ zcrypt_device_put(zdev);
+ spin_unlock_bh(&zcrypt_device_lock);
+ return rc;
+ }
+ spin_unlock_bh(&zcrypt_device_lock);
+ return -ENODEV;
+}
+
static long zcrypt_rng(char *buffer)
{
struct zcrypt_device *zdev;
return -EFAULT;
return rc;
}
+ case ZSENDEP11CPRB: {
+ struct ep11_urb __user *uxcrb = (void __user *)arg;
+ struct ep11_urb xcrb;
+ if (copy_from_user(&xcrb, uxcrb, sizeof(xcrb)))
+ return -EFAULT;
+ do {
+ rc = zcrypt_send_ep11_cprb(&xcrb);
+ } while (rc == -EAGAIN);
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ do {
+ rc = zcrypt_send_ep11_cprb(&xcrb);
+ } while (rc == -EAGAIN);
+ if (copy_to_user(uxcrb, &xcrb, sizeof(xcrb)))
+ return -EFAULT;
+ return rc;
+ }
case Z90STAT_STATUS_MASK: {
char status[AP_DEVICES];
zcrypt_status_mask(status);
#define ZCRYPT_CEX2A 6
#define ZCRYPT_CEX3C 7
#define ZCRYPT_CEX3A 8
+#define ZCRYPT_CEX4 10
/**
* Large random numbers are pulled in 4096 byte chunks from the crypto cards
long (*rsa_modexpo_crt)(struct zcrypt_device *,
struct ica_rsa_modexpo_crt *);
long (*send_cprb)(struct zcrypt_device *, struct ica_xcRB *);
+ long (*send_ep11_cprb)(struct zcrypt_device *, struct ep11_urb *);
long (*rng)(struct zcrypt_device *, char *);
struct list_head list; /* zcrypt ops list. */
struct module *owner;
#define CEX4A_MAX_MESSAGE_SIZE MSGTYPE50_CRB3_MAX_MSG_SIZE
#define CEX4C_MAX_MESSAGE_SIZE MSGTYPE06_MAX_MSG_SIZE
-#define CEX4_CLEANUP_TIME (15*HZ)
+/* Waiting time for requests to be processed.
+ * Currently there are some types of request which are not deterministic.
+ * But the maximum time limit managed by the stomper code is set to 60sec.
+ * Hence we have to wait at least that time period.
+ */
+#define CEX4_CLEANUP_TIME (61*HZ)
static struct ap_device_id zcrypt_cex4_ids[] = {
{ AP_DEVICE(AP_DEVICE_TYPE_CEX4) },
zdev->speed_rating = CEX4C_SPEED_RATING;
zdev->ops = zcrypt_msgtype_request(MSGTYPE06_NAME,
MSGTYPE06_VARIANT_DEFAULT);
+ } else if (ap_test_bit(&ap_dev->functions, AP_FUNC_EP11)) {
+ zdev = zcrypt_device_alloc(CEX4C_MAX_MESSAGE_SIZE);
+ if (!zdev)
+ return -ENOMEM;
+ zdev->type_string = "CEX4P";
+ zdev->user_space_type = ZCRYPT_CEX4;
+ zdev->min_mod_size = CEX4C_MIN_MOD_SIZE;
+ zdev->max_mod_size = CEX4C_MAX_MOD_SIZE;
+ zdev->max_exp_bit_length = CEX4C_MAX_MOD_SIZE;
+ zdev->short_crt = 0;
+ zdev->speed_rating = CEX4C_SPEED_RATING;
+ zdev->ops = zcrypt_msgtype_request(MSGTYPE06_NAME,
+ MSGTYPE06_VARIANT_EP11);
}
break;
}
// REP88_ERROR_MESSAGE_TYPE // '20' CEX2A
/*
* To sent a message of the wrong type is a bug in the
- * device driver. Warn about it, disable the device
+ * device driver. Send error msg, disable the device
* and then repeat the request.
*/
- WARN_ON(1);
atomic_set(&zcrypt_rescan_req, 1);
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
- zdev->ap_dev->qid,
- zdev->online, ehdr->reply_code);
+ zdev->ap_dev->qid, zdev->online, ehdr->reply_code);
return -EAGAIN;
case REP82_ERROR_TRANSPORT_FAIL:
case REP82_ERROR_MACHINE_FAILURE:
/* If a card fails disable it and repeat the request. */
atomic_set(&zcrypt_rescan_req, 1);
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
- zdev->ap_dev->qid,
- zdev->online, ehdr->reply_code);
+ zdev->ap_dev->qid, zdev->online, ehdr->reply_code);
return -EAGAIN;
default:
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
- zdev->ap_dev->qid,
- zdev->online, ehdr->reply_code);
+ zdev->ap_dev->qid, zdev->online, ehdr->reply_code);
return -EAGAIN; /* repeat the request on a different device. */
}
}
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#define KMSG_COMPONENT "zcrypt"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/init.h>
if (t80h->len < sizeof(*t80h) + outputdatalength) {
/* The result is too short, the CEX2A card may not do that.. */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
+ zdev->ap_dev->qid, zdev->online, t80h->code);
+
return -EAGAIN; /* repeat the request on a different device. */
}
if (zdev->user_space_type == ZCRYPT_CEX2A)
outputdata, outputdatalength);
default: /* Unknown response type, this should NEVER EVER happen */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%dfail",
+ zdev->ap_dev->qid, zdev->online);
return -EAGAIN; /* repeat the request on a different device. */
}
}
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#define KMSG_COMPONENT "zcrypt"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
#include <linux/module.h>
#include <linux/init.h>
#include <linux/err.h>
};
#define PCIXCC_RESPONSE_TYPE_ICA 0
#define PCIXCC_RESPONSE_TYPE_XCRB 1
+#define PCIXCC_RESPONSE_TYPE_EP11 2
MODULE_AUTHOR("IBM Corporation");
MODULE_DESCRIPTION("Cryptographic Coprocessor (message type 6), " \
return 0;
}
+static int xcrb_msg_to_type6_ep11cprb_msgx(struct zcrypt_device *zdev,
+ struct ap_message *ap_msg,
+ struct ep11_urb *xcRB)
+{
+ unsigned int lfmt;
+
+ static struct type6_hdr static_type6_ep11_hdr = {
+ .type = 0x06,
+ .rqid = {0x00, 0x01},
+ .function_code = {0x00, 0x00},
+ .agent_id[0] = 0x58, /* {'X'} */
+ .agent_id[1] = 0x43, /* {'C'} */
+ .offset1 = 0x00000058,
+ };
+
+ struct {
+ struct type6_hdr hdr;
+ struct ep11_cprb cprbx;
+ unsigned char pld_tag; /* fixed value 0x30 */
+ unsigned char pld_lenfmt; /* payload length format */
+ } __packed * msg = ap_msg->message;
+
+ struct pld_hdr {
+ unsigned char func_tag; /* fixed value 0x4 */
+ unsigned char func_len; /* fixed value 0x4 */
+ unsigned int func_val; /* function ID */
+ unsigned char dom_tag; /* fixed value 0x4 */
+ unsigned char dom_len; /* fixed value 0x4 */
+ unsigned int dom_val; /* domain id */
+ } __packed * payload_hdr;
+
+ /* length checks */
+ ap_msg->length = sizeof(struct type6_hdr) + xcRB->req_len;
+ if (CEIL4(xcRB->req_len) > MSGTYPE06_MAX_MSG_SIZE -
+ (sizeof(struct type6_hdr)))
+ return -EINVAL;
+
+ if (CEIL4(xcRB->resp_len) > MSGTYPE06_MAX_MSG_SIZE -
+ (sizeof(struct type86_fmt2_msg)))
+ return -EINVAL;
+
+ /* prepare type6 header */
+ msg->hdr = static_type6_ep11_hdr;
+ msg->hdr.ToCardLen1 = xcRB->req_len;
+ msg->hdr.FromCardLen1 = xcRB->resp_len;
+
+ /* Import CPRB data from the ioctl input parameter */
+ if (copy_from_user(&(msg->cprbx.cprb_len),
+ (char *)xcRB->req, xcRB->req_len)) {
+ return -EFAULT;
+ }
+
+ /*
+ The target domain field within the cprb body/payload block will be
+ replaced by the usage domain for non-management commands only.
+ Therefore we check the first bit of the 'flags' parameter for
+ management command indication.
+ 0 - non management command
+ 1 - management command
+ */
+ if (!((msg->cprbx.flags & 0x80) == 0x80)) {
+ msg->cprbx.target_id = (unsigned int)
+ AP_QID_QUEUE(zdev->ap_dev->qid);
+
+ if ((msg->pld_lenfmt & 0x80) == 0x80) { /*ext.len.fmt 2 or 3*/
+ switch (msg->pld_lenfmt & 0x03) {
+ case 1:
+ lfmt = 2;
+ break;
+ case 2:
+ lfmt = 3;
+ break;
+ default:
+ return -EINVAL;
+ }
+ } else {
+ lfmt = 1; /* length format #1 */
+ }
+ payload_hdr = (struct pld_hdr *)((&(msg->pld_lenfmt))+lfmt);
+ payload_hdr->dom_val = (unsigned int)
+ AP_QID_QUEUE(zdev->ap_dev->qid);
+ }
+ return 0;
+}
+
/**
* Copy results from a type 86 ICA reply message back to user space.
*
char text[0];
} __packed;
+struct type86_ep11_reply {
+ struct type86_hdr hdr;
+ struct type86_fmt2_ext fmt2;
+ struct ep11_cprb cprbx;
+} __packed;
+
static int convert_type86_ica(struct zcrypt_device *zdev,
struct ap_message *reply,
char __user *outputdata,
if (service_rc == 8 && service_rs == 72)
return -EINVAL;
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
+ zdev->ap_dev->qid, zdev->online,
+ msg->hdr.reply_code);
return -EAGAIN; /* repeat the request on a different device. */
}
data = msg->text;
return 0;
}
+/**
+ * Copy results from a type 86 EP11 XCRB reply message back to user space.
+ *
+ * @zdev: crypto device pointer
+ * @reply: reply AP message.
+ * @xcRB: pointer to EP11 user request block
+ *
+ * Returns 0 on success or -EINVAL, -EFAULT, -EAGAIN in case of an error.
+ */
+static int convert_type86_ep11_xcrb(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ struct ep11_urb *xcRB)
+{
+ struct type86_fmt2_msg *msg = reply->message;
+ char *data = reply->message;
+
+ if (xcRB->resp_len < msg->fmt2.count1)
+ return -EINVAL;
+
+ /* Copy response CPRB to user */
+ if (copy_to_user((char *)xcRB->resp,
+ data + msg->fmt2.offset1, msg->fmt2.count1))
+ return -EFAULT;
+ xcRB->resp_len = msg->fmt2.count1;
+ return 0;
+}
+
static int convert_type86_rng(struct zcrypt_device *zdev,
struct ap_message *reply,
char *buffer)
* response */
default: /* Unknown response type, this should NEVER EVER happen */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%dfail",
+ zdev->ap_dev->qid, zdev->online);
return -EAGAIN; /* repeat the request on a different device. */
}
}
default: /* Unknown response type, this should NEVER EVER happen */
xcRB->status = 0x0008044DL; /* HDD_InvalidParm */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%dfail",
+ zdev->ap_dev->qid, zdev->online);
return -EAGAIN; /* repeat the request on a different device. */
}
}
+static int convert_response_ep11_xcrb(struct zcrypt_device *zdev,
+ struct ap_message *reply, struct ep11_urb *xcRB)
+{
+ struct type86_ep11_reply *msg = reply->message;
+
+ /* Response type byte is the second byte in the response. */
+ switch (((unsigned char *)reply->message)[1]) {
+ case TYPE82_RSP_CODE:
+ case TYPE87_RSP_CODE:
+ return convert_error(zdev, reply);
+ case TYPE86_RSP_CODE:
+ if (msg->hdr.reply_code)
+ return convert_error(zdev, reply);
+ if (msg->cprbx.cprb_ver_id == 0x04)
+ return convert_type86_ep11_xcrb(zdev, reply, xcRB);
+ /* Fall through, no break, incorrect cprb version is an unknown resp.*/
+ default: /* Unknown response type, this should NEVER EVER happen */
+ zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%dfail",
+ zdev->ap_dev->qid, zdev->online);
+ return -EAGAIN; /* repeat the request on a different device. */
+ }
+}
+
static int convert_response_rng(struct zcrypt_device *zdev,
struct ap_message *reply,
char *data)
* response */
default: /* Unknown response type, this should NEVER EVER happen */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%dfail",
+ zdev->ap_dev->qid, zdev->online);
return -EAGAIN; /* repeat the request on a different device. */
}
}
complete(&(resp_type->work));
}
+/**
+ * This function is called from the AP bus code after a crypto request
+ * "msg" has finished with the reply message "reply".
+ * It is called from tasklet context.
+ * @ap_dev: pointer to the AP device
+ * @msg: pointer to the AP message
+ * @reply: pointer to the AP reply message
+ */
+static void zcrypt_msgtype6_receive_ep11(struct ap_device *ap_dev,
+ struct ap_message *msg,
+ struct ap_message *reply)
+{
+ static struct error_hdr error_reply = {
+ .type = TYPE82_RSP_CODE,
+ .reply_code = REP82_ERROR_MACHINE_FAILURE,
+ };
+ struct response_type *resp_type =
+ (struct response_type *)msg->private;
+ struct type86_ep11_reply *t86r;
+ int length;
+
+ /* Copy the reply message to the request message buffer. */
+ if (IS_ERR(reply)) {
+ memcpy(msg->message, &error_reply, sizeof(error_reply));
+ goto out;
+ }
+ t86r = reply->message;
+ if (t86r->hdr.type == TYPE86_RSP_CODE &&
+ t86r->cprbx.cprb_ver_id == 0x04) {
+ switch (resp_type->type) {
+ case PCIXCC_RESPONSE_TYPE_EP11:
+ length = t86r->fmt2.offset1 + t86r->fmt2.count1;
+ length = min(MSGTYPE06_MAX_MSG_SIZE, length);
+ memcpy(msg->message, reply->message, length);
+ break;
+ default:
+ memcpy(msg->message, &error_reply, sizeof(error_reply));
+ }
+ } else {
+ memcpy(msg->message, reply->message, sizeof(error_reply));
+ }
+out:
+ complete(&(resp_type->work));
+}
+
static atomic_t zcrypt_step = ATOMIC_INIT(0);
/**
return rc;
}
+/**
+ * The request distributor calls this function if it picked the CEX4P
+ * device to handle a send_ep11_cprb request.
+ * @zdev: pointer to zcrypt_device structure that identifies the
+ * CEX4P device to the request distributor
+ * @xcRB: pointer to the ep11 user request block
+ */
+static long zcrypt_msgtype6_send_ep11_cprb(struct zcrypt_device *zdev,
+ struct ep11_urb *xcrb)
+{
+ struct ap_message ap_msg;
+ struct response_type resp_type = {
+ .type = PCIXCC_RESPONSE_TYPE_EP11,
+ };
+ int rc;
+
+ ap_init_message(&ap_msg);
+ ap_msg.message = kmalloc(MSGTYPE06_MAX_MSG_SIZE, GFP_KERNEL);
+ if (!ap_msg.message)
+ return -ENOMEM;
+ ap_msg.receive = zcrypt_msgtype6_receive_ep11;
+ ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
+ atomic_inc_return(&zcrypt_step);
+ ap_msg.private = &resp_type;
+ rc = xcrb_msg_to_type6_ep11cprb_msgx(zdev, &ap_msg, xcrb);
+ if (rc)
+ goto out_free;
+ init_completion(&resp_type.work);
+ ap_queue_message(zdev->ap_dev, &ap_msg);
+ rc = wait_for_completion_interruptible(&resp_type.work);
+ if (rc == 0)
+ rc = convert_response_ep11_xcrb(zdev, &ap_msg, xcrb);
+ else /* Signal pending. */
+ ap_cancel_message(zdev->ap_dev, &ap_msg);
+
+out_free:
+ kzfree(ap_msg.message);
+ return rc;
+}
+
/**
* The request distributor calls this function if it picked the PCIXCC/CEX2C
* device to generate random data.
.rng = zcrypt_msgtype6_rng,
};
+static struct zcrypt_ops zcrypt_msgtype6_ep11_ops = {
+ .owner = THIS_MODULE,
+ .variant = MSGTYPE06_VARIANT_EP11,
+ .rsa_modexpo = NULL,
+ .rsa_modexpo_crt = NULL,
+ .send_ep11_cprb = zcrypt_msgtype6_send_ep11_cprb,
+};
+
int __init zcrypt_msgtype6_init(void)
{
zcrypt_msgtype_register(&zcrypt_msgtype6_norng_ops);
zcrypt_msgtype_register(&zcrypt_msgtype6_ops);
+ zcrypt_msgtype_register(&zcrypt_msgtype6_ep11_ops);
return 0;
}
{
zcrypt_msgtype_unregister(&zcrypt_msgtype6_norng_ops);
zcrypt_msgtype_unregister(&zcrypt_msgtype6_ops);
+ zcrypt_msgtype_unregister(&zcrypt_msgtype6_ep11_ops);
}
module_init(zcrypt_msgtype6_init);
#define MSGTYPE06_NAME "zcrypt_msgtype6"
#define MSGTYPE06_VARIANT_DEFAULT 0
#define MSGTYPE06_VARIANT_NORNG 1
+#define MSGTYPE06_VARIANT_EP11 2
#define MSGTYPE06_MAX_MSG_SIZE (12*1024)
} __packed;
#define TYPE86_RSP_CODE 0x86
+#define TYPE87_RSP_CODE 0x87
#define TYPE86_FMT2 0x02
struct type86_fmt2_ext {
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#define KMSG_COMPONENT "zcrypt"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/init.h>
if (t84h->len < sizeof(*t84h) + outputdatalength) {
/* The result is too short, the PCICA card may not do that.. */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
+ zdev->ap_dev->qid, zdev->online, t84h->code);
return -EAGAIN; /* repeat the request on a different device. */
}
BUG_ON(t84h->len > PCICA_MAX_RESPONSE_SIZE);
outputdata, outputdatalength);
default: /* Unknown response type, this should NEVER EVER happen */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%dfail",
+ zdev->ap_dev->qid, zdev->online);
return -EAGAIN; /* repeat the request on a different device. */
}
}
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#define KMSG_COMPONENT "zcrypt"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
#include <linux/module.h>
#include <linux/init.h>
#include <linux/gfp.h>
if (service_rc == 8 && service_rs == 72)
return -EINVAL;
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
+ zdev->ap_dev->qid, zdev->online,
+ msg->hdr.reply_code);
return -EAGAIN; /* repeat the request on a different device. */
}
data = msg->text;
/* no break, incorrect cprb version is an unknown response */
default: /* Unknown response type, this should NEVER EVER happen */
zdev->online = 0;
+ pr_err("Cryptographic device %x failed and was set offline\n",
+ zdev->ap_dev->qid);
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%dfail",
+ zdev->ap_dev->qid, zdev->online);
return -EAGAIN; /* repeat the request on a different device. */
}
}
instance->irq = IRQ_AMIGA_PORTS;
instance->unique_id = z->slotaddr;
- regs = (struct a2091_scsiregs *)ZTWO_VADDR(z->resource.start);
+ regs = ZTWO_VADDR(z->resource.start);
regs->DAWR = DAWR_A2091;
wdregs.SASR = ®s->SASR;
instance->irq = IRQ_AMIGA_PORTS;
- regs = (struct a3000_scsiregs *)ZTWO_VADDR(res->start);
+ regs = ZTWO_VADDR(res->start);
regs->DAWR = DAWR_A3000;
wdregs.SASR = ®s->SASR;
scsi_addr = res->start + A4000T_SCSI_OFFSET;
/* Fill in the required pieces of hostdata */
- hostdata->base = (void __iomem *)ZTWO_VADDR(scsi_addr);
+ hostdata->base = ZTWO_VADDR(scsi_addr);
hostdata->clock = 50;
hostdata->chip710 = 1;
hostdata->dmode_extra = DMODE_FC2;
if (!request_mem_region(address, 256, "wd33c93"))
return -EBUSY;
- regs = (struct gvp11_scsiregs *)(ZTWO_VADDR(address));
+ regs = ZTWO_VADDR(address);
error = check_wd33c93(regs);
if (error)
if (ioaddr > 0x01000000)
hostdata->base = ioremap(ioaddr, zorro_resource_len(z));
else
- hostdata->base = (void __iomem *)ZTWO_VADDR(ioaddr);
+ hostdata->base = ZTWO_VADDR(ioaddr);
hostdata->clock = 50;
hostdata->chip710 = 1;
*/
local_touch_nmi();
stop_critical_timings();
- __monitor((void *)¤t_thread_info()->flags, 0, 0);
- cpu_relax(); /* allow HT sibling to run */
- __mwait(eax, ecx);
+ mwait_idle_with_hints(eax, ecx);
start_critical_timings();
atomic_inc(&idle_wakeup_counter);
}
tick_nohz_idle_exit();
- preempt_enable_no_resched();
+ preempt_enable();
}
del_timer_sync(&wakeup_timer);
clear_bit(cpunr, cpu_clamping_mask);
if (!videomemory) {
dev_warn(&pdev->dev,
"Unable to map videomem cached writethrough\n");
- info->screen_base = (char *)ZTWO_VADDR(info->fix.smem_start);
+ info->screen_base = ZTWO_VADDR(info->fix.smem_start);
} else
info->screen_base = (char *)videomemory;
info->fix.mmio_start = regbase;
cinfo->regbase = regbase > 16 * MB_ ? ioremap(regbase, 64 * 1024)
- : (caddr_t)ZTWO_VADDR(regbase);
+ : ZTWO_VADDR(regbase);
if (!cinfo->regbase) {
dev_err(info->device, "Cannot map registers\n");
error = -EIO;
info->fix.smem_start = rambase;
info->screen_size = ramsize;
info->screen_base = rambase > 16 * MB_ ? ioremap(rambase, ramsize)
- : (caddr_t)ZTWO_VADDR(rambase);
+ : ZTWO_VADDR(rambase);
if (!info->screen_base) {
dev_err(info->device, "Cannot map video RAM\n");
error = -EIO;
#include <linux/fb.h>
#include <asm/setup.h>
-#include <asm/bootinfo.h>
#include <asm/macintosh.h>
#include <asm/io.h>
#include <linux/cuda.h>
#include <asm/io.h>
#ifdef CONFIG_MAC
-#include <asm/bootinfo.h>
#include <asm/macintosh.h>
#else
#include <asm/prom.h>
# Makefile for the Zorro bus specific drivers.
#
-obj-$(CONFIG_ZORRO) += zorro.o zorro-driver.o zorro-sysfs.o names.o
+obj-$(CONFIG_ZORRO) += zorro.o zorro-driver.o zorro-sysfs.o
obj-$(CONFIG_PROC_FS) += proc.o
+obj-$(CONFIG_ZORRO_NAMES) += names.o
hostprogs-y := gen-devlist
#include <linux/zorro.h>
-#ifdef CONFIG_ZORRO_NAMES
-
struct zorro_prod_info {
__u16 prod;
unsigned short seen;
} while (--i);
/* Couldn't find either the manufacturer nor the product */
- sprintf(name, "Zorro device %08x", dev->id);
return;
match_manuf: {
}
}
}
-
-#else
-
-void __init zorro_name_device(struct zorro_dev *dev)
-{
-}
-
-#endif
#include <linux/seq_file.h>
#include <linux/init.h>
#include <linux/export.h>
+
+#include <asm/byteorder.h>
#include <asm/uaccess.h>
#include <asm/amigahw.h>
#include <asm/setup.h>
/* Construct a ConfigDev */
memset(&cd, 0, sizeof(cd));
cd.cd_Rom = z->rom;
- cd.cd_SlotAddr = z->slotaddr;
- cd.cd_SlotSize = z->slotsize;
- cd.cd_BoardAddr = (void *)zorro_resource_start(z);
- cd.cd_BoardSize = zorro_resource_len(z);
+ cd.cd_SlotAddr = cpu_to_be16(z->slotaddr);
+ cd.cd_SlotSize = cpu_to_be16(z->slotsize);
+ cd.cd_BoardAddr = cpu_to_be32(zorro_resource_start(z));
+ cd.cd_BoardSize = cpu_to_be32(zorro_resource_len(z));
if (copy_to_user(buf, (void *)&cd + pos, nbytes))
return -EFAULT;
}
struct bus_type zorro_bus_type = {
- .name = "zorro",
- .match = zorro_bus_match,
- .uevent = zorro_uevent,
- .probe = zorro_device_probe,
- .remove = zorro_device_remove,
+ .name = "zorro",
+ .dev_name = "zorro",
+ .match = zorro_bus_match,
+ .uevent = zorro_uevent,
+ .probe = zorro_device_probe,
+ .remove = zorro_device_remove,
};
EXPORT_SYMBOL(zorro_bus_type);
#include <linux/stat.h>
#include <linux/string.h>
+#include <asm/byteorder.h>
+
#include "zorro.h"
zorro_config_attr(id, id, "0x%08x\n");
zorro_config_attr(type, rom.er_Type, "0x%02x\n");
-zorro_config_attr(serial, rom.er_SerialNumber, "0x%08x\n");
zorro_config_attr(slotaddr, slotaddr, "0x%04x\n");
zorro_config_attr(slotsize, slotsize, "0x%04x\n");
+static ssize_t
+show_serial(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct zorro_dev *z;
+
+ z = to_zorro_dev(dev);
+ return sprintf(buf, "0x%08x\n", be32_to_cpu(z->rom.er_SerialNumber));
+}
+
+static DEVICE_ATTR(serial, S_IRUGO, show_serial, NULL);
+
static ssize_t zorro_show_resource(struct device *dev, struct device_attribute *attr, char *buf)
{
struct zorro_dev *z = to_zorro_dev(dev);
/* Construct a ConfigDev */
memset(&cd, 0, sizeof(cd));
cd.cd_Rom = z->rom;
- cd.cd_SlotAddr = z->slotaddr;
- cd.cd_SlotSize = z->slotsize;
- cd.cd_BoardAddr = (void *)zorro_resource_start(z);
- cd.cd_BoardSize = zorro_resource_len(z);
+ cd.cd_SlotAddr = cpu_to_be16(z->slotaddr);
+ cd.cd_SlotSize = cpu_to_be16(z->slotsize);
+ cd.cd_BoardAddr = cpu_to_be32(zorro_resource_start(z));
+ cd.cd_BoardSize = cpu_to_be32(zorro_resource_len(z));
return memory_read_from_buffer(buf, count, &off, &cd, sizeof(cd));
}
#include <linux/platform_device.h>
#include <linux/slab.h>
+#include <asm/byteorder.h>
#include <asm/setup.h>
#include <asm/amigahw.h>
*/
unsigned int zorro_num_autocon;
-struct zorro_dev zorro_autocon[ZORRO_NUM_AUTO];
+struct zorro_dev_init zorro_autocon_init[ZORRO_NUM_AUTO] __initdata;
+struct zorro_dev *zorro_autocon;
/*
struct zorro_bus {
struct device dev;
+ struct zorro_dev devices[0];
};
static int __init amiga_zorro_probe(struct platform_device *pdev)
{
struct zorro_bus *bus;
+ struct zorro_dev_init *zi;
struct zorro_dev *z;
struct resource *r;
unsigned int i;
int error;
/* Initialize the Zorro bus */
- bus = kzalloc(sizeof(*bus), GFP_KERNEL);
+ bus = kzalloc(sizeof(*bus) +
+ zorro_num_autocon * sizeof(bus->devices[0]),
+ GFP_KERNEL);
if (!bus)
return -ENOMEM;
+ zorro_autocon = bus->devices;
bus->dev.parent = &pdev->dev;
- dev_set_name(&bus->dev, "zorro");
+ dev_set_name(&bus->dev, zorro_bus_type.name);
error = device_register(&bus->dev);
if (error) {
pr_err("Zorro: Error registering zorro_bus\n");
/* First identify all devices ... */
for (i = 0; i < zorro_num_autocon; i++) {
+ zi = &zorro_autocon_init[i];
z = &zorro_autocon[i];
- z->id = (z->rom.er_Manufacturer<<16) | (z->rom.er_Product<<8);
+
+ z->rom = zi->rom;
+ z->id = (be16_to_cpu(z->rom.er_Manufacturer) << 16) |
+ (z->rom.er_Product << 8);
if (z->id == ZORRO_PROD_GVP_EPC_BASE) {
/* GVP quirk */
- unsigned long magic = zorro_resource_start(z)+0x8000;
+ unsigned long magic = zi->boardaddr + 0x8000;
z->id |= *(u16 *)ZTWO_VADDR(magic) & GVP_PRODMASK;
}
+ z->slotaddr = zi->slotaddr;
+ z->slotsize = zi->slotsize;
sprintf(z->name, "Zorro device %08x", z->id);
zorro_name_device(z);
+ z->resource.start = zi->boardaddr;
+ z->resource.end = zi->boardaddr + zi->boardsize - 1;
z->resource.name = z->name;
r = zorro_find_parent_resource(pdev, z);
error = request_resource(r, &z->resource);
dev_err(&bus->dev,
"Address space collision on device %s %pR\n",
z->name, &z->resource);
- dev_set_name(&z->dev, "%02x", i);
z->dev.parent = &bus->dev;
z->dev.bus = &zorro_bus_type;
+ z->dev.id = i;
}
/* ... then register them */
+#ifdef CONFIG_ZORRO_NAMES
extern void zorro_name_device(struct zorro_dev *z);
+#else
+static inline void zorro_name_device(struct zorro_dev *dev) { }
+#endif
+
extern int zorro_create_sysfs_dev_files(struct zorro_dev *z);
* thus don't need to be hashed. They also don't need a name until a
* user wants to identify the object in /proc/pid/fd/. The little hack
* below allows us to generate a name for these objects on demand:
+ *
+ * Some pseudo inodes are mountable. When they are mounted
+ * path->dentry == path->mnt->mnt_root. In that case don't call d_dname
+ * and instead have d_path return the mounted path.
*/
- if (path->dentry->d_op && path->dentry->d_op->d_dname)
+ if (path->dentry->d_op && path->dentry->d_op->d_dname &&
+ (!IS_ROOT(path->dentry) || path->dentry != path->mnt->mnt_root))
return path->dentry->d_op->d_dname(path->dentry, buf, buflen);
rcu_read_lock();
}
WARN_ON(inode->i_state & I_SYNC);
/*
- * Skip inode if it is clean. We don't want to mess with writeback
- * lists in this function since flusher thread may be doing for example
- * sync in parallel and if we move the inode, it could get skipped. So
- * here we make sure inode is on some writeback list and leave it there
- * unless we have completely cleaned the inode.
+ * Skip inode if it is clean and we have no outstanding writeback in
+ * WB_SYNC_ALL mode. We don't want to mess with writeback lists in this
+ * function since flusher thread may be doing for example sync in
+ * parallel and if we move the inode, it could get skipped. So here we
+ * make sure inode is on some writeback list and leave it there unless
+ * we have completely cleaned the inode.
*/
- if (!(inode->i_state & I_DIRTY))
+ if (!(inode->i_state & I_DIRTY) &&
+ (wbc->sync_mode != WB_SYNC_ALL ||
+ !mapping_tagged(inode->i_mapping, PAGECACHE_TAG_WRITEBACK)))
goto out;
inode->i_state |= I_SYNC;
spin_unlock(&inode->i_lock);
struct inode *inode = child->mnt_mountpoint->d_inode;
if (!S_ISDIR(inode->i_mode))
goto next;
- if (inode->i_nlink != 2)
+ if (inode->i_nlink > 2)
goto next;
}
visible = true;
nilfs_clear_logs(&sci->sc_segbufs);
- err = nilfs_segctor_extend_segments(sci, nilfs, nadd);
- if (unlikely(err))
- return err;
-
if (sci->sc_stage.flags & NILFS_CF_SUFREED) {
err = nilfs_sufile_cancel_freev(nilfs->ns_sufile,
sci->sc_freesegs,
sci->sc_nfreesegs,
NULL);
WARN_ON(err); /* do not happen */
+ sci->sc_stage.flags &= ~NILFS_CF_SUFREED;
}
+
+ err = nilfs_segctor_extend_segments(sci, nilfs, nadd);
+ if (unlikely(err))
+ return err;
+
nadd = min_t(int, nadd << 1, SC_MAX_SEGDELTA);
sci->sc_stage = prev_stage;
}
-/* Generic barrier definitions, based on MN10300 definitions.
+/*
+ * Generic barrier definitions, originally based on MN10300 definitions.
*
* It should be possible to use these on really simple architectures,
* but it serves more as a starting point for new ports.
#ifndef __ASSEMBLY__
-#define nop() asm volatile ("nop")
+#include <linux/compiler.h>
+
+#ifndef nop
+#define nop() asm volatile ("nop")
+#endif
/*
- * Force strict CPU ordering.
- * And yes, this is required on UP too when we're talking
- * to devices.
+ * Force strict CPU ordering. And yes, this is required on UP too when we're
+ * talking to devices.
*
- * This implementation only contains a compiler barrier.
+ * Fall back to compiler barriers if nothing better is provided.
*/
-#define mb() asm volatile ("": : :"memory")
+#ifndef mb
+#define mb() barrier()
+#endif
+
+#ifndef rmb
#define rmb() mb()
-#define wmb() asm volatile ("": : :"memory")
+#endif
+
+#ifndef wmb
+#define wmb() mb()
+#endif
+
+#ifndef read_barrier_depends
+#define read_barrier_depends() do { } while (0)
+#endif
#ifdef CONFIG_SMP
#define smp_mb() mb()
#define smp_rmb() rmb()
#define smp_wmb() wmb()
+#define smp_read_barrier_depends() read_barrier_depends()
#else
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
+#define smp_read_barrier_depends() do { } while (0)
+#endif
+
+#ifndef set_mb
+#define set_mb(var, value) do { (var) = (value); mb(); } while (0)
#endif
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#define set_wmb(var, value) do { var = value; wmb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
-#define read_barrier_depends() do {} while (0)
-#define smp_read_barrier_depends() do {} while (0)
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_GENERIC_BARRIER_H */
#ifndef _LINUX_BH_H
#define _LINUX_BH_H
-extern void local_bh_disable(void);
+#include <linux/preempt.h>
+#include <linux/preempt_mask.h>
+
+#ifdef CONFIG_TRACE_IRQFLAGS
+extern void __local_bh_disable_ip(unsigned long ip, unsigned int cnt);
+#else
+static __always_inline void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
+{
+ preempt_count_add(cnt);
+ barrier();
+}
+#endif
+
+static inline void local_bh_disable(void)
+{
+ __local_bh_disable_ip(_THIS_IP_, SOFTIRQ_DISABLE_OFFSET);
+}
+
extern void _local_bh_enable(void);
-extern void local_bh_enable(void);
-extern void local_bh_enable_ip(unsigned long ip);
+extern void __local_bh_enable_ip(unsigned long ip, unsigned int cnt);
+
+static inline void local_bh_enable_ip(unsigned long ip)
+{
+ __local_bh_enable_ip(ip, SOFTIRQ_DISABLE_OFFSET);
+}
+
+static inline void local_bh_enable(void)
+{
+ __local_bh_enable_ip(_THIS_IP_, SOFTIRQ_DISABLE_OFFSET);
+}
#endif /* _LINUX_BH_H */
# define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
#endif
+/* Is this type a native word size -- useful for atomic operations */
+#ifndef __native_word
+# define __native_word(t) (sizeof(t) == sizeof(int) || sizeof(t) == sizeof(long))
+#endif
+
/* Compile time object size, -1 for unknown */
#ifndef __compiletime_object_size
# define __compiletime_object_size(obj) -1
#define compiletime_assert(condition, msg) \
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
+#define compiletime_assert_atomic_type(t) \
+ compiletime_assert(__native_word(t), \
+ "Need native word sized stores/loads for atomicity.")
+
/*
* Prevent the compiler from merging or refetching accesses. The compiler
* is also forbidden from reordering successive instances of ACCESS_ONCE(),
static inline void user_enter(void)
{
- if (static_key_false(&context_tracking_enabled))
+ if (context_tracking_is_enabled())
context_tracking_user_enter();
}
static inline void user_exit(void)
{
- if (static_key_false(&context_tracking_enabled))
+ if (context_tracking_is_enabled())
context_tracking_user_exit();
}
{
enum ctx_state prev_ctx;
- if (!static_key_false(&context_tracking_enabled))
+ if (!context_tracking_is_enabled())
return 0;
prev_ctx = this_cpu_read(context_tracking.state);
static inline void exception_exit(enum ctx_state prev_ctx)
{
- if (static_key_false(&context_tracking_enabled)) {
+ if (context_tracking_is_enabled()) {
if (prev_ctx == IN_USER)
context_tracking_user_enter();
}
static inline void context_tracking_task_switch(struct task_struct *prev,
struct task_struct *next)
{
- if (static_key_false(&context_tracking_enabled))
+ if (context_tracking_is_enabled())
__context_tracking_task_switch(prev, next);
}
#else
extern struct static_key context_tracking_enabled;
DECLARE_PER_CPU(struct context_tracking, context_tracking);
-static inline bool context_tracking_in_user(void)
+static inline bool context_tracking_is_enabled(void)
{
- return __this_cpu_read(context_tracking.state) == IN_USER;
+ return static_key_false(&context_tracking_enabled);
}
-static inline bool context_tracking_active(void)
+static inline bool context_tracking_cpu_is_enabled(void)
{
return __this_cpu_read(context_tracking.active);
}
+
+static inline bool context_tracking_in_user(void)
+{
+ return __this_cpu_read(context_tracking.state) == IN_USER;
+}
#else
static inline bool context_tracking_in_user(void) { return false; }
static inline bool context_tracking_active(void) { return false; }
#include <linux/proc_fs.h>
#include <linux/elf.h>
+#include <asm/pgtable.h> /* for pgprot_t */
+
#define ELFCORE_ADDR_MAX (-1ULL)
#define ELFCORE_ADDR_ERR (-2ULL)
extern struct bus_type *edac_get_sysfs_subsys(void);
extern void edac_put_sysfs_subsys(void);
+enum {
+ EDAC_REPORTING_ENABLED,
+ EDAC_REPORTING_DISABLED,
+ EDAC_REPORTING_FORCE
+};
+
+extern int edac_report_status;
+#ifdef CONFIG_EDAC
+static inline int get_edac_report_status(void)
+{
+ return edac_report_status;
+}
+
+static inline void set_edac_report_status(int new)
+{
+ edac_report_status = new;
+}
+#else
+static inline int get_edac_report_status(void)
+{
+ return EDAC_REPORTING_DISABLED;
+}
+
+static inline void set_edac_report_status(int new)
+{
+}
+#endif
+
static inline void opstate_init(void)
{
switch (edac_op_state) {
unsigned long hcdp; /* HCDP table */
unsigned long uga; /* UGA table */
unsigned long uv_systab; /* UV system table */
+ unsigned long fw_vendor; /* fw_vendor */
+ unsigned long runtime; /* runtime table */
+ unsigned long config_table; /* config tables */
efi_get_time_t *get_time;
efi_set_time_t *set_time;
efi_get_wakeup_time_t *get_wakeup_time;
#define EFI_RUNTIME_SERVICES 3 /* Can we use runtime services? */
#define EFI_MEMMAP 4 /* Can we use EFI memory map? */
#define EFI_64BIT 5 /* Is the firmware 64-bit? */
+#define EFI_ARCH_1 6 /* First arch-specific bit */
#ifdef CONFIG_EFI
# ifdef CONFIG_X86
#endif /* CONFIG_EFI_VARS */
+#ifdef CONFIG_EFI_RUNTIME_MAP
+int efi_runtime_map_init(struct kobject *);
+void efi_runtime_map_setup(void *, int, u32);
+#else
+static inline int efi_runtime_map_init(struct kobject *kobj)
+{
+ return 0;
+}
+
+static inline void
+efi_runtime_map_setup(void *map, int nr_entries, u32 desc_size) {}
+#endif
+
#endif /* _LINUX_EFI_H */
#include <linux/lockdep.h>
#include <linux/ftrace_irq.h>
#include <linux/vtime.h>
+#include <asm/hardirq.h>
extern void synchronize_irq(unsigned int irq);
static inline struct i2c_adapter *
i2c_parent_is_i2c_adapter(const struct i2c_adapter *adapter)
{
-#if IS_ENABLED(I2C_MUX)
+#if IS_ENABLED(CONFIG_I2C_MUX)
struct device *parent = adapter->dev.parent;
if (parent != NULL && parent->type == &i2c_adapter_type)
#include <linux/user_namespace.h>
#include <linux/securebits.h>
#include <linux/seqlock.h>
+#include <linux/rbtree.h>
#include <net/net_namespace.h>
#include <linux/sched/rt.h>
#define INIT_TASK_COMM "swapper"
+#ifdef CONFIG_RT_MUTEXES
+# define INIT_RT_MUTEXES(tsk) \
+ .pi_waiters = RB_ROOT, \
+ .pi_waiters_leftmost = NULL,
+#else
+# define INIT_RT_MUTEXES(tsk)
+#endif
+
/*
* INIT_TASK is used to set up the first task table, touch at
* your own risk!. Base=0, limit=0x1fffff (=2MB)
INIT_TRACE_RECURSION \
INIT_TASK_RCU_PREEMPT(tsk) \
INIT_CPUSET_SEQ(tsk) \
+ INIT_RT_MUTEXES(tsk) \
INIT_VTIME(tsk) \
}
extern int panic_on_unrecovered_nmi;
extern int panic_on_io_nmi;
extern int sysctl_panic_on_stackoverflow;
+/*
+ * Only to be used by arch init code. If the user over-wrote the default
+ * CONFIG_PANIC_TIMEOUT, honor it.
+ */
+static inline void set_arch_panic_timeout(int timeout, int arch_default_timeout)
+{
+ if (panic_timeout == arch_default_timeout)
+ panic_timeout = timeout;
+}
extern const char *print_tainted(void);
enum lockdep_ok {
LOCKDEP_STILL_OK,
struct list_head migrate_entry;
struct hlist_node hlist_entry;
+ struct list_head active_entry;
int nr_siblings;
int group_flags;
struct perf_event *group_leader;
-/* linux/arch/arm/plat-s3c/include/plat/hwmon.h
- *
+/*
* Copyright 2005 Simtec Electronics
* Ben Dooks <ben@simtec.co.uk>
* http://armlinux.simtec.co.uk/
* published by the Free Software Foundation.
*/
-#ifndef __ASM_ARCH_ADC_HWMON_H
-#define __ASM_ARCH_ADC_HWMON_H __FILE__
+#ifndef __HWMON_S3C_H__
+#define __HWMON_S3C_H__
/**
* s3c_hwmon_chcfg - channel configuration
*/
extern void __init s3c_hwmon_set_platdata(struct s3c_hwmon_pdata *pd);
-#endif /* __ASM_ARCH_ADC_HWMON_H */
-
+#endif /* __HWMON_S3C_H__ */
* For further information, see the Documentation/hwmon/max197 file.
*/
+#ifndef _PDATA_MAX197_H
+#define _PDATA_MAX197_H
+
/**
* struct max197_platform_data - MAX197 connectivity info
* @convert: Function used to start a conversion with control byte ctrl.
struct max197_platform_data {
int (*convert)(u8 ctrl);
};
+
+#endif /* _PDATA_MAX197_H */
* For further information, see the Documentation/hwmon/sht15 file.
*/
+#ifndef _PDATA_SHT15_H
+#define _PDATA_SHT15_H
+
/**
* struct sht15_platform_data - sht15 connectivity info
* @gpio_data: no. of gpio to which bidirectional data line is
bool no_otp_reload;
bool low_resolution;
};
+
+#endif /* _PDATA_SHT15_H */
} while (0)
#else
-#define preempt_enable() preempt_enable_no_resched()
+#define preempt_enable() \
+do { \
+ barrier(); \
+ preempt_count_dec(); \
+} while (0)
#define preempt_check_resched() do { } while (0)
#endif
__preempt_schedule_context(); \
} while (0)
#else
-#define preempt_enable_notrace() preempt_enable_no_resched_notrace()
+#define preempt_enable_notrace() \
+do { \
+ barrier(); \
+ __preempt_count_dec(); \
+} while (0)
#endif
#else /* !CONFIG_PREEMPT_COUNT */
#endif /* CONFIG_PREEMPT_COUNT */
+#ifdef MODULE
+/*
+ * Modules have no business playing preemption tricks.
+ */
+#undef sched_preempt_enable_no_resched
+#undef preempt_enable_no_resched
+#undef preempt_enable_no_resched_notrace
+#undef preempt_check_resched
+#endif
+
+#ifdef CONFIG_PREEMPT
+#define preempt_set_need_resched() \
+do { \
+ set_preempt_need_resched(); \
+} while (0)
+#define preempt_fold_need_resched() \
+do { \
+ if (tif_need_resched()) \
+ set_preempt_need_resched(); \
+} while (0)
+#else
+#define preempt_set_need_resched() do { } while (0)
+#define preempt_fold_need_resched() do { } while (0)
+#endif
+
#ifdef CONFIG_PREEMPT_NOTIFIERS
struct preempt_notifier;
#define LINUX_PREEMPT_MASK_H
#include <linux/preempt.h>
-#include <asm/hardirq.h>
/*
* We put the hardirq and softirq counter into the preemption
# define PREEMPT_CHECK_OFFSET 0
#endif
+/*
+ * The preempt_count offset needed for things like:
+ *
+ * spin_lock_bh()
+ *
+ * Which need to disable both preemption (CONFIG_PREEMPT_COUNT) and
+ * softirqs, such that unlock sequences of:
+ *
+ * spin_unlock();
+ * local_bh_enable();
+ *
+ * Work as expected.
+ */
+#define SOFTIRQ_LOCK_OFFSET (SOFTIRQ_DISABLE_OFFSET + PREEMPT_CHECK_OFFSET)
+
/*
* Are we running in atomic context? WARNING: this macro cannot
* always detect atomic context; in particular, it cannot know about
next->prev = new;
}
#else
-extern void __list_add_rcu(struct list_head *new,
- struct list_head *prev, struct list_head *next);
+void __list_add_rcu(struct list_head *new,
+ struct list_head *prev, struct list_head *next);
#endif
/**
#endif /* #ifdef CONFIG_RCU_TORTURE_TEST */
#if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU)
-extern void rcutorture_record_test_transition(void);
-extern void rcutorture_record_progress(unsigned long vernum);
-extern void do_trace_rcu_torture_read(const char *rcutorturename,
- struct rcu_head *rhp,
- unsigned long secs,
- unsigned long c_old,
- unsigned long c);
+void rcutorture_record_test_transition(void);
+void rcutorture_record_progress(unsigned long vernum);
+void do_trace_rcu_torture_read(const char *rcutorturename,
+ struct rcu_head *rhp,
+ unsigned long secs,
+ unsigned long c_old,
+ unsigned long c);
#else
static inline void rcutorture_record_test_transition(void)
{
{
}
#ifdef CONFIG_RCU_TRACE
-extern void do_trace_rcu_torture_read(const char *rcutorturename,
- struct rcu_head *rhp,
- unsigned long secs,
- unsigned long c_old,
- unsigned long c);
+void do_trace_rcu_torture_read(const char *rcutorturename,
+ struct rcu_head *rhp,
+ unsigned long secs,
+ unsigned long c_old,
+ unsigned long c);
#else
#define do_trace_rcu_torture_read(rcutorturename, rhp, secs, c_old, c) \
do { } while (0)
* if CPU A and CPU B are the same CPU (but again only if the system has
* more than one CPU).
*/
-extern void call_rcu(struct rcu_head *head,
- void (*func)(struct rcu_head *head));
+void call_rcu(struct rcu_head *head,
+ void (*func)(struct rcu_head *head));
#else /* #ifdef CONFIG_PREEMPT_RCU */
* See the description of call_rcu() for more detailed information on
* memory ordering guarantees.
*/
-extern void call_rcu_bh(struct rcu_head *head,
- void (*func)(struct rcu_head *head));
+void call_rcu_bh(struct rcu_head *head,
+ void (*func)(struct rcu_head *head));
/**
* call_rcu_sched() - Queue an RCU for invocation after sched grace period.
* See the description of call_rcu() for more detailed information on
* memory ordering guarantees.
*/
-extern void call_rcu_sched(struct rcu_head *head,
- void (*func)(struct rcu_head *rcu));
+void call_rcu_sched(struct rcu_head *head,
+ void (*func)(struct rcu_head *rcu));
-extern void synchronize_sched(void);
+void synchronize_sched(void);
#ifdef CONFIG_PREEMPT_RCU
-extern void __rcu_read_lock(void);
-extern void __rcu_read_unlock(void);
-extern void rcu_read_unlock_special(struct task_struct *t);
+void __rcu_read_lock(void);
+void __rcu_read_unlock(void);
+void rcu_read_unlock_special(struct task_struct *t);
void synchronize_rcu(void);
/*
#endif /* #else #ifdef CONFIG_PREEMPT_RCU */
/* Internal to kernel */
-extern void rcu_init(void);
-extern void rcu_sched_qs(int cpu);
-extern void rcu_bh_qs(int cpu);
-extern void rcu_check_callbacks(int cpu, int user);
+void rcu_init(void);
+void rcu_sched_qs(int cpu);
+void rcu_bh_qs(int cpu);
+void rcu_check_callbacks(int cpu, int user);
struct notifier_block;
-extern void rcu_idle_enter(void);
-extern void rcu_idle_exit(void);
-extern void rcu_irq_enter(void);
-extern void rcu_irq_exit(void);
+void rcu_idle_enter(void);
+void rcu_idle_exit(void);
+void rcu_irq_enter(void);
+void rcu_irq_exit(void);
#ifdef CONFIG_RCU_USER_QS
-extern void rcu_user_enter(void);
-extern void rcu_user_exit(void);
+void rcu_user_enter(void);
+void rcu_user_exit(void);
#else
static inline void rcu_user_enter(void) { }
static inline void rcu_user_exit(void) { }
} while (0)
#if defined(CONFIG_DEBUG_LOCK_ALLOC) || defined(CONFIG_RCU_TRACE) || defined(CONFIG_SMP)
-extern bool __rcu_is_watching(void);
+bool __rcu_is_watching(void);
#endif /* #if defined(CONFIG_DEBUG_LOCK_ALLOC) || defined(CONFIG_RCU_TRACE) || defined(CONFIG_SMP) */
/*
* initialization.
*/
#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD
-extern void init_rcu_head_on_stack(struct rcu_head *head);
-extern void destroy_rcu_head_on_stack(struct rcu_head *head);
+void init_rcu_head_on_stack(struct rcu_head *head);
+void destroy_rcu_head_on_stack(struct rcu_head *head);
#else /* !CONFIG_DEBUG_OBJECTS_RCU_HEAD */
static inline void init_rcu_head_on_stack(struct rcu_head *head)
{
extern struct lockdep_map rcu_lock_map;
extern struct lockdep_map rcu_bh_lock_map;
extern struct lockdep_map rcu_sched_lock_map;
+extern struct lockdep_map rcu_callback_map;
extern int debug_lockdep_rcu_enabled(void);
/**
* rcu_read_lock_bh_held() is defined out of line to avoid #include-file
* hell.
*/
-extern int rcu_read_lock_bh_held(void);
+int rcu_read_lock_bh_held(void);
/**
* rcu_read_lock_sched_held() - might we be in RCU-sched read-side critical section?
#ifdef CONFIG_PROVE_RCU
-extern int rcu_my_thread_group_empty(void);
+int rcu_my_thread_group_empty(void);
/**
* rcu_lockdep_assert - emit lockdep splat if specified condition not met
smp_read_barrier_depends(); \
(_________p1); \
})
-#define __rcu_assign_pointer(p, v, space) \
+
+/**
+ * RCU_INITIALIZER() - statically initialize an RCU-protected global variable
+ * @v: The value to statically initialize with.
+ */
+#define RCU_INITIALIZER(v) (typeof(*(v)) __force __rcu *)(v)
+
+/**
+ * rcu_assign_pointer() - assign to RCU-protected pointer
+ * @p: pointer to assign to
+ * @v: value to assign (publish)
+ *
+ * Assigns the specified value to the specified RCU-protected
+ * pointer, ensuring that any concurrent RCU readers will see
+ * any prior initialization.
+ *
+ * Inserts memory barriers on architectures that require them
+ * (which is most of them), and also prevents the compiler from
+ * reordering the code that initializes the structure after the pointer
+ * assignment. More importantly, this call documents which pointers
+ * will be dereferenced by RCU read-side code.
+ *
+ * In some special cases, you may use RCU_INIT_POINTER() instead
+ * of rcu_assign_pointer(). RCU_INIT_POINTER() is a bit faster due
+ * to the fact that it does not constrain either the CPU or the compiler.
+ * That said, using RCU_INIT_POINTER() when you should have used
+ * rcu_assign_pointer() is a very bad thing that results in
+ * impossible-to-diagnose memory corruption. So please be careful.
+ * See the RCU_INIT_POINTER() comment header for details.
+ *
+ * Note that rcu_assign_pointer() evaluates each of its arguments only
+ * once, appearances notwithstanding. One of the "extra" evaluations
+ * is in typeof() and the other visible only to sparse (__CHECKER__),
+ * neither of which actually execute the argument. As with most cpp
+ * macros, this execute-arguments-only-once property is important, so
+ * please be careful when making changes to rcu_assign_pointer() and the
+ * other macros that it invokes.
+ */
+#define rcu_assign_pointer(p, v) \
do { \
smp_wmb(); \
- (p) = (typeof(*v) __force space *)(v); \
+ ACCESS_ONCE(p) = RCU_INITIALIZER(v); \
} while (0)
preempt_enable_notrace();
}
-/**
- * rcu_assign_pointer() - assign to RCU-protected pointer
- * @p: pointer to assign to
- * @v: value to assign (publish)
- *
- * Assigns the specified value to the specified RCU-protected
- * pointer, ensuring that any concurrent RCU readers will see
- * any prior initialization.
- *
- * Inserts memory barriers on architectures that require them
- * (which is most of them), and also prevents the compiler from
- * reordering the code that initializes the structure after the pointer
- * assignment. More importantly, this call documents which pointers
- * will be dereferenced by RCU read-side code.
- *
- * In some special cases, you may use RCU_INIT_POINTER() instead
- * of rcu_assign_pointer(). RCU_INIT_POINTER() is a bit faster due
- * to the fact that it does not constrain either the CPU or the compiler.
- * That said, using RCU_INIT_POINTER() when you should have used
- * rcu_assign_pointer() is a very bad thing that results in
- * impossible-to-diagnose memory corruption. So please be careful.
- * See the RCU_INIT_POINTER() comment header for details.
- */
-#define rcu_assign_pointer(p, v) \
- __rcu_assign_pointer((p), (v), __rcu)
-
/**
* RCU_INIT_POINTER() - initialize an RCU protected pointer
*
*/
#define RCU_INIT_POINTER(p, v) \
do { \
- p = (typeof(*v) __force __rcu *)(v); \
+ p = RCU_INITIALIZER(v); \
} while (0)
/**
* GCC-style initialization for an RCU-protected pointer in a structure field.
*/
#define RCU_POINTER_INITIALIZER(p, v) \
- .p = (typeof(*v) __force __rcu *)(v)
+ .p = RCU_INITIALIZER(v)
/*
* Does the specified offset indicate that the corresponding rcu_head
__kfree_rcu(&((ptr)->rcu_head), offsetof(typeof(*(ptr)), rcu_head))
#ifdef CONFIG_RCU_NOCB_CPU
-extern bool rcu_is_nocb_cpu(int cpu);
+bool rcu_is_nocb_cpu(int cpu);
#else
static inline bool rcu_is_nocb_cpu(int cpu) { return false; }
#endif /* #else #ifdef CONFIG_RCU_NOCB_CPU */
/* Only for use by adaptive-ticks code. */
#ifdef CONFIG_NO_HZ_FULL_SYSIDLE
-extern bool rcu_sys_is_idle(void);
-extern void rcu_sysidle_force_exit(void);
+bool rcu_sys_is_idle(void);
+void rcu_sysidle_force_exit(void);
#else /* #ifdef CONFIG_NO_HZ_FULL_SYSIDLE */
static inline bool rcu_sys_is_idle(void)
#ifdef CONFIG_DEBUG_LOCK_ALLOC
extern int rcu_scheduler_active __read_mostly;
-extern void rcu_scheduler_starting(void);
+void rcu_scheduler_starting(void);
#else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */
static inline void rcu_scheduler_starting(void)
{
#ifndef __LINUX_RCUTREE_H
#define __LINUX_RCUTREE_H
-extern void rcu_note_context_switch(int cpu);
-extern int rcu_needs_cpu(int cpu, unsigned long *delta_jiffies);
-extern void rcu_cpu_stall_reset(void);
+void rcu_note_context_switch(int cpu);
+int rcu_needs_cpu(int cpu, unsigned long *delta_jiffies);
+void rcu_cpu_stall_reset(void);
/*
* Note a virtualization-based context switch. This is simply a
rcu_note_context_switch(cpu);
}
-extern void synchronize_rcu_bh(void);
-extern void synchronize_sched_expedited(void);
-extern void synchronize_rcu_expedited(void);
+void synchronize_rcu_bh(void);
+void synchronize_sched_expedited(void);
+void synchronize_rcu_expedited(void);
void kfree_call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu));
synchronize_sched_expedited();
}
-extern void rcu_barrier(void);
-extern void rcu_barrier_bh(void);
-extern void rcu_barrier_sched(void);
+void rcu_barrier(void);
+void rcu_barrier_bh(void);
+void rcu_barrier_sched(void);
extern unsigned long rcutorture_testseq;
extern unsigned long rcutorture_vernum;
-extern long rcu_batches_completed(void);
-extern long rcu_batches_completed_bh(void);
-extern long rcu_batches_completed_sched(void);
+long rcu_batches_completed(void);
+long rcu_batches_completed_bh(void);
+long rcu_batches_completed_sched(void);
-extern void rcu_force_quiescent_state(void);
-extern void rcu_bh_force_quiescent_state(void);
-extern void rcu_sched_force_quiescent_state(void);
+void rcu_force_quiescent_state(void);
+void rcu_bh_force_quiescent_state(void);
+void rcu_sched_force_quiescent_state(void);
-extern void exit_rcu(void);
+void exit_rcu(void);
-extern void rcu_scheduler_starting(void);
+void rcu_scheduler_starting(void);
extern int rcu_scheduler_active __read_mostly;
-extern bool rcu_is_watching(void);
+bool rcu_is_watching(void);
#endif /* __LINUX_RCUTREE_H */
#define __LINUX_RT_MUTEX_H
#include <linux/linkage.h>
-#include <linux/plist.h>
+#include <linux/rbtree.h>
#include <linux/spinlock_types.h>
extern int max_lock_depth; /* for sysctl */
* The rt_mutex structure
*
* @wait_lock: spinlock to protect the structure
- * @wait_list: pilist head to enqueue waiters in priority order
+ * @waiters: rbtree root to enqueue waiters in priority order
+ * @waiters_leftmost: top waiter
* @owner: the mutex owner
*/
struct rt_mutex {
raw_spinlock_t wait_lock;
- struct plist_head wait_list;
+ struct rb_root waiters;
+ struct rb_node *waiters_leftmost;
struct task_struct *owner;
#ifdef CONFIG_DEBUG_RT_MUTEXES
int save_state;
#define __RT_MUTEX_INITIALIZER(mutexname) \
{ .wait_lock = __RAW_SPIN_LOCK_UNLOCKED(mutexname.wait_lock) \
- , .wait_list = PLIST_HEAD_INIT(mutexname.wait_list) \
+ , .waiters = RB_ROOT \
, .owner = NULL \
__DEBUG_RT_MUTEX_INITIALIZER(mutexname)}
extern void rt_mutex_unlock(struct rt_mutex *lock);
-#ifdef CONFIG_RT_MUTEXES
-# define INIT_RT_MUTEXES(tsk) \
- .pi_waiters = PLIST_HEAD_INIT(tsk.pi_waiters), \
- INIT_RT_MUTEX_DEBUG(tsk)
-#else
-# define INIT_RT_MUTEXES(tsk)
-#endif
-
#endif
static inline void __raw_read_lock_bh(rwlock_t *lock)
{
- local_bh_disable();
- preempt_disable();
+ __local_bh_disable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
rwlock_acquire_read(&lock->dep_map, 0, 0, _RET_IP_);
LOCK_CONTENDED(lock, do_raw_read_trylock, do_raw_read_lock);
}
static inline void __raw_write_lock_bh(rwlock_t *lock)
{
- local_bh_disable();
- preempt_disable();
+ __local_bh_disable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
rwlock_acquire(&lock->dep_map, 0, 0, _RET_IP_);
LOCK_CONTENDED(lock, do_raw_write_trylock, do_raw_write_lock);
}
{
rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_read_unlock(lock);
- preempt_enable_no_resched();
- local_bh_enable_ip((unsigned long)__builtin_return_address(0));
+ __local_bh_enable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
}
static inline void __raw_write_unlock_irqrestore(rwlock_t *lock,
{
rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_write_unlock(lock);
- preempt_enable_no_resched();
- local_bh_enable_ip((unsigned long)__builtin_return_address(0));
+ __local_bh_enable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
}
#endif /* __LINUX_RWLOCK_API_SMP_H */
#include <linux/types.h>
#include <linux/timex.h>
#include <linux/jiffies.h>
+#include <linux/plist.h>
#include <linux/rbtree.h>
#include <linux/thread_info.h>
#include <linux/cpumask.h>
#include <asm/processor.h>
+#define SCHED_ATTR_SIZE_VER0 48 /* sizeof first published struct */
+
+/*
+ * Extended scheduling parameters data structure.
+ *
+ * This is needed because the original struct sched_param can not be
+ * altered without introducing ABI issues with legacy applications
+ * (e.g., in sched_getparam()).
+ *
+ * However, the possibility of specifying more than just a priority for
+ * the tasks may be useful for a wide variety of application fields, e.g.,
+ * multimedia, streaming, automation and control, and many others.
+ *
+ * This variant (sched_attr) is meant at describing a so-called
+ * sporadic time-constrained task. In such model a task is specified by:
+ * - the activation period or minimum instance inter-arrival time;
+ * - the maximum (or average, depending on the actual scheduling
+ * discipline) computation time of all instances, a.k.a. runtime;
+ * - the deadline (relative to the actual activation time) of each
+ * instance.
+ * Very briefly, a periodic (sporadic) task asks for the execution of
+ * some specific computation --which is typically called an instance--
+ * (at most) every period. Moreover, each instance typically lasts no more
+ * than the runtime and must be completed by time instant t equal to
+ * the instance activation time + the deadline.
+ *
+ * This is reflected by the actual fields of the sched_attr structure:
+ *
+ * @size size of the structure, for fwd/bwd compat.
+ *
+ * @sched_policy task's scheduling policy
+ * @sched_flags for customizing the scheduler behaviour
+ * @sched_nice task's nice value (SCHED_NORMAL/BATCH)
+ * @sched_priority task's static priority (SCHED_FIFO/RR)
+ * @sched_deadline representative of the task's deadline
+ * @sched_runtime representative of the task's runtime
+ * @sched_period representative of the task's period
+ *
+ * Given this task model, there are a multiplicity of scheduling algorithms
+ * and policies, that can be used to ensure all the tasks will make their
+ * timing constraints.
+ *
+ * As of now, the SCHED_DEADLINE policy (sched_dl scheduling class) is the
+ * only user of this new interface. More information about the algorithm
+ * available in the scheduling class file or in Documentation/.
+ */
+struct sched_attr {
+ u32 size;
+
+ u32 sched_policy;
+ u64 sched_flags;
+
+ /* SCHED_NORMAL, SCHED_BATCH */
+ s32 sched_nice;
+
+ /* SCHED_FIFO, SCHED_RR */
+ u32 sched_priority;
+
+ /* SCHED_DEADLINE */
+ u64 sched_runtime;
+ u64 sched_deadline;
+ u64 sched_period;
+};
+
struct exec_domain;
struct futex_pi_state;
struct robust_list_head;
#define task_is_traced(task) ((task->state & __TASK_TRACED) != 0)
#define task_is_stopped(task) ((task->state & __TASK_STOPPED) != 0)
-#define task_is_dead(task) ((task)->exit_state != 0)
#define task_is_stopped_or_traced(task) \
((task->state & (__TASK_STOPPED | __TASK_TRACED)) != 0)
#define task_contributes_to_load(task) \
#endif
};
+struct sched_dl_entity {
+ struct rb_node rb_node;
+
+ /*
+ * Original scheduling parameters. Copied here from sched_attr
+ * during sched_setscheduler2(), they will remain the same until
+ * the next sched_setscheduler2().
+ */
+ u64 dl_runtime; /* maximum runtime for each instance */
+ u64 dl_deadline; /* relative deadline of each instance */
+ u64 dl_period; /* separation of two instances (period) */
+ u64 dl_bw; /* dl_runtime / dl_deadline */
+
+ /*
+ * Actual scheduling parameters. Initialized with the values above,
+ * they are continously updated during task execution. Note that
+ * the remaining runtime could be < 0 in case we are in overrun.
+ */
+ s64 runtime; /* remaining runtime for this instance */
+ u64 deadline; /* absolute deadline for this instance */
+ unsigned int flags; /* specifying the scheduler behaviour */
+
+ /*
+ * Some bool flags:
+ *
+ * @dl_throttled tells if we exhausted the runtime. If so, the
+ * task has to wait for a replenishment to be performed at the
+ * next firing of dl_timer.
+ *
+ * @dl_new tells if a new instance arrived. If so we must
+ * start executing it with full runtime and reset its absolute
+ * deadline;
+ *
+ * @dl_boosted tells if we are boosted due to DI. If so we are
+ * outside bandwidth enforcement mechanism (but only until we
+ * exit the critical section).
+ */
+ int dl_throttled, dl_new, dl_boosted;
+
+ /*
+ * Bandwidth enforcement timer. Each -deadline task has its
+ * own bandwidth to be enforced, thus we need one timer per task.
+ */
+ struct hrtimer dl_timer;
+};
struct rcu_node;
#ifdef CONFIG_CGROUP_SCHED
struct task_group *sched_task_group;
#endif
+ struct sched_dl_entity dl;
#ifdef CONFIG_PREEMPT_NOTIFIERS
/* list of struct preempt_notifier: */
struct list_head tasks;
#ifdef CONFIG_SMP
struct plist_node pushable_tasks;
+ struct rb_node pushable_dl_tasks;
#endif
struct mm_struct *mm, *active_mm;
#ifdef CONFIG_RT_MUTEXES
/* PI waiters blocked on a rt_mutex held by this task */
- struct plist_head pi_waiters;
+ struct rb_root pi_waiters;
+ struct rb_node *pi_waiters_leftmost;
/* Deadlock detection and priority inheritance handling */
struct rt_mutex_waiter *pi_blocked_on;
+ /* Top pi_waiters task */
+ struct task_struct *pi_top_task;
#endif
#ifdef CONFIG_DEBUG_MUTEXES
* but then during bootup it turns out that sched_clock()
* is reliable after all:
*/
-extern int sched_clock_stable;
+extern int sched_clock_stable(void);
+extern void set_sched_clock_stable(void);
+extern void clear_sched_clock_stable(void);
extern void sched_clock_tick(void);
extern void sched_clock_idle_sleep_event(void);
const struct sched_param *);
extern int sched_setscheduler_nocheck(struct task_struct *, int,
const struct sched_param *);
+extern int sched_setattr(struct task_struct *,
+ const struct sched_attr *);
extern struct task_struct *idle_task(int cpu);
/**
* is_idle_task - is the specified task an idle task?
#else
static inline void kick_process(struct task_struct *tsk) { }
#endif
-extern void sched_fork(unsigned long clone_flags, struct task_struct *p);
+extern int sched_fork(unsigned long clone_flags, struct task_struct *p);
extern void sched_dead(struct task_struct *p);
extern void proc_caches_init(void);
}
#endif
+static inline void current_clr_polling(void)
+{
+ __current_clr_polling();
+
+ /*
+ * Ensure we check TIF_NEED_RESCHED after we clear the polling bit.
+ * Once the bit is cleared, we'll get IPIs with every new
+ * TIF_NEED_RESCHED and the IPI handler, scheduler_ipi(), will also
+ * fold.
+ */
+ smp_mb(); /* paired with resched_task() */
+
+ preempt_fold_need_resched();
+}
+
static __always_inline bool need_resched(void)
{
return unlikely(tif_need_resched());
--- /dev/null
+#ifndef _SCHED_DEADLINE_H
+#define _SCHED_DEADLINE_H
+
+/*
+ * SCHED_DEADLINE tasks has negative priorities, reflecting
+ * the fact that any of them has higher prio than RT and
+ * NORMAL/BATCH tasks.
+ */
+
+#define MAX_DL_PRIO 0
+
+static inline int dl_prio(int prio)
+{
+ if (unlikely(prio < MAX_DL_PRIO))
+ return 1;
+ return 0;
+}
+
+static inline int dl_task(struct task_struct *p)
+{
+ return dl_prio(p->prio);
+}
+
+#endif /* _SCHED_DEADLINE_H */
#ifdef CONFIG_RT_MUTEXES
extern int rt_mutex_getprio(struct task_struct *p);
extern void rt_mutex_setprio(struct task_struct *p, int prio);
+extern struct task_struct *rt_mutex_get_top_task(struct task_struct *task);
extern void rt_mutex_adjust_pi(struct task_struct *p);
static inline bool tsk_is_pi_blocked(struct task_struct *tsk)
{
{
return p->normal_prio;
}
+static inline struct task_struct *rt_mutex_get_top_task(struct task_struct *task)
+{
+ return NULL;
+}
# define rt_mutex_adjust_pi(p) do { } while (0)
static inline bool tsk_is_pi_blocked(struct task_struct *tsk)
{
extern unsigned int sysctl_numa_balancing_scan_period_min;
extern unsigned int sysctl_numa_balancing_scan_period_max;
extern unsigned int sysctl_numa_balancing_scan_size;
-extern unsigned int sysctl_numa_balancing_settle_count;
#ifdef CONFIG_SCHED_DEBUG
extern unsigned int sysctl_sched_migration_cost;
}
/**
- * read_seqcount_begin_no_lockdep - start seq-read critical section w/o lockdep
+ * raw_read_seqcount_begin - start seq-read critical section w/o lockdep
* @s: pointer to seqcount_t
* Returns: count to be passed to read_seqcount_retry
*
- * read_seqcount_begin_no_lockdep opens a read critical section of the given
+ * raw_read_seqcount_begin opens a read critical section of the given
* seqcount, but without any lockdep checking. Validity of the critical
* section is tested by checking read_seqcount_retry function.
*/
-static inline unsigned read_seqcount_begin_no_lockdep(const seqcount_t *s)
+static inline unsigned raw_read_seqcount_begin(const seqcount_t *s)
{
unsigned ret = __read_seqcount_begin(s);
smp_rmb();
static inline unsigned read_seqcount_begin(const seqcount_t *s)
{
seqcount_lockdep_reader_access(s);
- return read_seqcount_begin_no_lockdep(s);
+ return raw_read_seqcount_begin(s);
}
/**
}
+
+static inline void raw_write_seqcount_begin(seqcount_t *s)
+{
+ s->sequence++;
+ smp_wmb();
+}
+
+static inline void raw_write_seqcount_end(seqcount_t *s)
+{
+ smp_wmb();
+ s->sequence++;
+}
+
/*
* Sequence counter only version assumes that callers are using their
* own mutexing.
*/
static inline void write_seqcount_begin_nested(seqcount_t *s, int subclass)
{
- s->sequence++;
- smp_wmb();
+ raw_write_seqcount_begin(s);
seqcount_acquire(&s->dep_map, subclass, 0, _RET_IP_);
}
static inline void write_seqcount_end(seqcount_t *s)
{
seqcount_release(&s->dep_map, 1, _RET_IP_);
- smp_wmb();
- s->sequence++;
+ raw_write_seqcount_end(s);
}
/**
#define smp_mb__before_spinlock() smp_wmb()
#endif
+/*
+ * Place this after a lock-acquisition primitive to guarantee that
+ * an UNLOCK+LOCK pair act as a full barrier. This guarantee applies
+ * if the UNLOCK and LOCK are executed by the same CPU or if the
+ * UNLOCK and LOCK operate on the same lock variable.
+ */
+#ifndef smp_mb__after_unlock_lock
+#define smp_mb__after_unlock_lock() do { } while (0)
+#endif
+
/**
* raw_spin_unlock_wait - wait until the spinlock gets unlocked
* @lock: the spinlock in question.
static inline void __raw_spin_lock_bh(raw_spinlock_t *lock)
{
- local_bh_disable();
- preempt_disable();
+ __local_bh_disable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
spin_acquire(&lock->dep_map, 0, 0, _RET_IP_);
LOCK_CONTENDED(lock, do_raw_spin_trylock, do_raw_spin_lock);
}
{
spin_release(&lock->dep_map, 1, _RET_IP_);
do_raw_spin_unlock(lock);
- preempt_enable_no_resched();
- local_bh_enable_ip((unsigned long)__builtin_return_address(0));
+ __local_bh_enable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
}
static inline int __raw_spin_trylock_bh(raw_spinlock_t *lock)
{
- local_bh_disable();
- preempt_disable();
+ __local_bh_disable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
if (do_raw_spin_trylock(lock)) {
spin_acquire(&lock->dep_map, 0, 1, _RET_IP_);
return 1;
}
- preempt_enable_no_resched();
- local_bh_enable_ip((unsigned long)__builtin_return_address(0));
+ __local_bh_enable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
return 0;
}
* flags straight, to suppress compiler warnings of unused lock
* variables, and to add the proper checker annotations:
*/
+#define ___LOCK(lock) \
+ do { __acquire(lock); (void)(lock); } while (0)
+
#define __LOCK(lock) \
- do { preempt_disable(); __acquire(lock); (void)(lock); } while (0)
+ do { preempt_disable(); ___LOCK(lock); } while (0)
#define __LOCK_BH(lock) \
- do { local_bh_disable(); __LOCK(lock); } while (0)
+ do { __local_bh_disable_ip(_THIS_IP_, SOFTIRQ_LOCK_OFFSET); ___LOCK(lock); } while (0)
#define __LOCK_IRQ(lock) \
do { local_irq_disable(); __LOCK(lock); } while (0)
#define __LOCK_IRQSAVE(lock, flags) \
do { local_irq_save(flags); __LOCK(lock); } while (0)
+#define ___UNLOCK(lock) \
+ do { __release(lock); (void)(lock); } while (0)
+
#define __UNLOCK(lock) \
- do { preempt_enable(); __release(lock); (void)(lock); } while (0)
+ do { preempt_enable(); ___UNLOCK(lock); } while (0)
#define __UNLOCK_BH(lock) \
- do { preempt_enable_no_resched(); local_bh_enable(); \
- __release(lock); (void)(lock); } while (0)
+ do { __local_bh_enable_ip(_THIS_IP_, SOFTIRQ_LOCK_OFFSET); \
+ ___UNLOCK(lock); } while (0)
#define __UNLOCK_IRQ(lock) \
do { local_irq_enable(); __UNLOCK(lock); } while (0)
struct rlimit64;
struct rusage;
struct sched_param;
+struct sched_attr;
struct sel_arg_struct;
struct semaphore;
struct sembuf;
struct sched_param __user *param);
asmlinkage long sys_sched_setparam(pid_t pid,
struct sched_param __user *param);
+asmlinkage long sys_sched_setattr(pid_t pid,
+ struct sched_attr __user *attr);
asmlinkage long sys_sched_getscheduler(pid_t pid);
asmlinkage long sys_sched_getparam(pid_t pid,
struct sched_param __user *param);
+asmlinkage long sys_sched_getattr(pid_t pid,
+ struct sched_attr __user *attr,
+ unsigned int size);
asmlinkage long sys_sched_setaffinity(pid_t pid, unsigned int len,
unsigned long __user *user_mask_ptr);
asmlinkage long sys_sched_getaffinity(pid_t pid, unsigned int len,
extern void tick_clock_notify(void);
extern int tick_check_oneshot_change(int allow_nohz);
extern struct tick_sched *tick_get_tick_sched(int cpu);
-extern void tick_check_idle(int cpu);
+extern void tick_check_idle(void);
extern int tick_oneshot_mode_active(void);
# ifndef arch_needs_cpu
# define arch_needs_cpu(cpu) (0)
# else
static inline void tick_clock_notify(void) { }
static inline int tick_check_oneshot_change(int allow_nohz) { return 0; }
-static inline void tick_check_idle(int cpu) { }
+static inline void tick_check_idle(void) { }
static inline int tick_oneshot_mode_active(void) { return 0; }
# endif
static inline void tick_cancel_sched_timer(int cpu) { }
static inline void tick_clock_notify(void) { }
static inline int tick_check_oneshot_change(int allow_nohz) { return 0; }
-static inline void tick_check_idle(int cpu) { }
+static inline void tick_check_idle(void) { }
static inline int tick_oneshot_mode_active(void) { return 0; }
#endif /* !CONFIG_GENERIC_CLOCKEVENTS */
static inline bool tick_nohz_full_enabled(void)
{
- if (!static_key_false(&context_tracking_enabled))
+ if (!context_tracking_is_enabled())
return false;
return tick_nohz_full_running;
static inline void pagefault_enable(void)
{
+#ifndef CONFIG_PREEMPT
/*
* make sure to issue those last loads/stores before enabling
* the pagefault handler again.
*/
barrier();
preempt_count_dec();
- preempt_check_resched();
+#else
+ preempt_enable();
+#endif
}
#ifndef ARCH_HAS_NOCACHE_UACCESS
#include <linux/errno.h>
#include <linux/rbtree.h>
+#include <linux/types.h>
struct vm_area_struct;
struct mm_struct;
struct inode;
struct notifier_block;
-#ifdef CONFIG_ARCH_SUPPORTS_UPROBES
-# include <asm/uprobes.h>
-#endif
-
#define UPROBE_HANDLER_REMOVE 1
#define UPROBE_HANDLER_MASK 1
};
#ifdef CONFIG_UPROBES
+#include <asm/uprobes.h>
+
enum uprobe_task_state {
UTASK_RUNNING,
UTASK_SSTEP,
*/
struct uprobe_task {
enum uprobe_task_state state;
- struct arch_uprobe_task autask;
- struct return_instance *return_instances;
- unsigned int depth;
- struct uprobe *active_uprobe;
+ union {
+ struct {
+ struct arch_uprobe_task autask;
+ unsigned long vaddr;
+ };
+ struct {
+ struct callback_head dup_xol_work;
+ unsigned long dup_xol_addr;
+ };
+ };
+
+ struct uprobe *active_uprobe;
unsigned long xol_vaddr;
- unsigned long vaddr;
-};
-/*
- * On a breakpoint hit, thread contests for a slot. It frees the
- * slot after singlestep. Currently a fixed number of slots are
- * allocated.
- */
-struct xol_area {
- wait_queue_head_t wq; /* if all slots are busy */
- atomic_t slot_count; /* number of in-use slots */
- unsigned long *bitmap; /* 0 = free slot */
- struct page *page;
-
- /*
- * We keep the vma's vm_start rather than a pointer to the vma
- * itself. The probed process or a naughty kernel module could make
- * the vma go away, and we must handle that reasonably gracefully.
- */
- unsigned long vaddr; /* Page(s) of instruction slots */
+ struct return_instance *return_instances;
+ unsigned int depth;
};
+struct xol_area;
+
struct uprobes_state {
struct xol_area *xol_area;
};
extern int __weak set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
extern bool __weak is_swbp_insn(uprobe_opcode_t *insn);
extern bool __weak is_trap_insn(uprobe_opcode_t *insn);
+extern unsigned long __weak uprobe_get_swbp_addr(struct pt_regs *regs);
extern int uprobe_write_opcode(struct mm_struct *mm, unsigned long vaddr, uprobe_opcode_t);
extern int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
extern int uprobe_apply(struct inode *inode, loff_t offset, struct uprobe_consumer *uc, bool);
extern void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm);
extern void uprobe_free_utask(struct task_struct *t);
extern void uprobe_copy_process(struct task_struct *t, unsigned long flags);
-extern unsigned long __weak uprobe_get_swbp_addr(struct pt_regs *regs);
extern int uprobe_post_sstep_notifier(struct pt_regs *regs);
extern int uprobe_pre_sstep_notifier(struct pt_regs *regs);
extern void uprobe_notify_resume(struct pt_regs *regs);
{
return false;
}
-static inline unsigned long uprobe_get_swbp_addr(struct pt_regs *regs)
-{
- return 0;
-}
static inline void uprobe_free_utask(struct task_struct *t)
{
}
#ifdef CONFIG_VIRT_CPU_ACCOUNTING_GEN
static inline bool vtime_accounting_enabled(void)
{
- if (static_key_false(&context_tracking_enabled)) {
- if (context_tracking_active())
+ if (context_tracking_is_enabled()) {
+ if (context_tracking_cpu_is_enabled())
return true;
}
#ifndef _LINUX_ZORRO_H
#define _LINUX_ZORRO_H
-#include <linux/device.h>
-
-
- /*
- * Each Zorro board has a 32-bit ID of the form
- *
- * mmmmmmmmmmmmmmmmppppppppeeeeeeee
- *
- * with
- *
- * mmmmmmmmmmmmmmmm 16-bit Manufacturer ID (assigned by CBM (sigh))
- * pppppppp 8-bit Product ID (assigned by manufacturer)
- * eeeeeeee 8-bit Extended Product ID (currently only used
- * for some GVP boards)
- */
-
-
-#define ZORRO_MANUF(id) ((id) >> 16)
-#define ZORRO_PROD(id) (((id) >> 8) & 0xff)
-#define ZORRO_EPC(id) ((id) & 0xff)
-
-#define ZORRO_ID(manuf, prod, epc) \
- ((ZORRO_MANUF_##manuf << 16) | ((prod) << 8) | (epc))
-
-typedef __u32 zorro_id;
-
-
-/* Include the ID list */
-#include <linux/zorro_ids.h>
-
- /*
- * GVP identifies most of its products through the 'extended product code'
- * (epc). The epc has to be ANDed with the GVP_PRODMASK before the
- * identification.
- */
-
-#define GVP_PRODMASK (0xf8)
-#define GVP_SCSICLKMASK (0x01)
-
-enum GVP_flags {
- GVP_IO = 0x01,
- GVP_ACCEL = 0x02,
- GVP_SCSI = 0x04,
- GVP_24BITDMA = 0x08,
- GVP_25BITDMA = 0x10,
- GVP_NOBANK = 0x20,
- GVP_14MHZ = 0x40,
-};
-
-
-struct Node {
- struct Node *ln_Succ; /* Pointer to next (successor) */
- struct Node *ln_Pred; /* Pointer to previous (predecessor) */
- __u8 ln_Type;
- __s8 ln_Pri; /* Priority, for sorting */
- __s8 *ln_Name; /* ID string, null terminated */
-} __attribute__ ((packed));
-
-struct ExpansionRom {
- /* -First 16 bytes of the expansion ROM */
- __u8 er_Type; /* Board type, size and flags */
- __u8 er_Product; /* Product number, assigned by manufacturer */
- __u8 er_Flags; /* Flags */
- __u8 er_Reserved03; /* Must be zero ($ff inverted) */
- __u16 er_Manufacturer; /* Unique ID, ASSIGNED BY COMMODORE-AMIGA! */
- __u32 er_SerialNumber; /* Available for use by manufacturer */
- __u16 er_InitDiagVec; /* Offset to optional "DiagArea" structure */
- __u8 er_Reserved0c;
- __u8 er_Reserved0d;
- __u8 er_Reserved0e;
- __u8 er_Reserved0f;
-} __attribute__ ((packed));
-
-/* er_Type board type bits */
-#define ERT_TYPEMASK 0xc0
-#define ERT_ZORROII 0xc0
-#define ERT_ZORROIII 0x80
-
-/* other bits defined in er_Type */
-#define ERTB_MEMLIST 5 /* Link RAM into free memory list */
-#define ERTF_MEMLIST (1<<5)
-
-struct ConfigDev {
- struct Node cd_Node;
- __u8 cd_Flags; /* (read/write) */
- __u8 cd_Pad; /* reserved */
- struct ExpansionRom cd_Rom; /* copy of board's expansion ROM */
- void *cd_BoardAddr; /* where in memory the board was placed */
- __u32 cd_BoardSize; /* size of board in bytes */
- __u16 cd_SlotAddr; /* which slot number (PRIVATE) */
- __u16 cd_SlotSize; /* number of slots (PRIVATE) */
- void *cd_Driver; /* pointer to node of driver */
- struct ConfigDev *cd_NextCD; /* linked list of drivers to config */
- __u32 cd_Unused[4]; /* for whatever the driver wants */
-} __attribute__ ((packed));
-
-#define ZORRO_NUM_AUTO 16
-
-#ifdef __KERNEL__
+#include <uapi/linux/zorro.h>
+#include <linux/device.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/mod_devicetable.h>
extern unsigned int zorro_num_autocon; /* # of autoconfig devices found */
-extern struct zorro_dev zorro_autocon[ZORRO_NUM_AUTO];
+extern struct zorro_dev *zorro_autocon;
+
+
+ /*
+ * Minimal information about a Zorro device, passed from bootinfo
+ * Only available temporarily, i.e. until initmem has been freed!
+ */
+
+struct zorro_dev_init {
+ struct ExpansionRom rom;
+ u16 slotaddr;
+ u16 slotsize;
+ u32 boardaddr;
+ u32 boardsize;
+};
+
+extern struct zorro_dev_init zorro_autocon_init[ZORRO_NUM_AUTO] __initdata;
/*
#define Z2RAM_CHUNKSHIFT (16)
-#endif /* __KERNEL__ */
-
#endif /* _LINUX_ZORRO_H */
+++ /dev/null
-/*
- * Zorro board IDs
- *
- * Please keep sorted.
- */
-
-
-#define ZORRO_MANUF_PACIFIC_PERIPHERALS 0x00D3
-#define ZORRO_PROD_PACIFIC_PERIPHERALS_SE_2000_A500 ZORRO_ID(PACIFIC_PERIPHERALS, 0x00, 0)
-#define ZORRO_PROD_PACIFIC_PERIPHERALS_SCSI ZORRO_ID(PACIFIC_PERIPHERALS, 0x0A, 0)
-
-#define ZORRO_MANUF_MACROSYSTEMS_USA_2 0x0100
-#define ZORRO_PROD_MACROSYSTEMS_WARP_ENGINE ZORRO_ID(MACROSYSTEMS_USA_2, 0x13, 0)
-
-#define ZORRO_MANUF_KUPKE_1 0x00DD
-#define ZORRO_PROD_KUPKE_GOLEM_RAM_BOX_2MB ZORRO_ID(KUPKE_1, 0x00, 0)
-
-#define ZORRO_MANUF_MEMPHIS 0x0100
-#define ZORRO_PROD_MEMPHIS_STORMBRINGER ZORRO_ID(MEMPHIS, 0x00, 0)
-
-#define ZORRO_MANUF_3_STATE 0x0200
-#define ZORRO_PROD_3_STATE_MEGAMIX_2000 ZORRO_ID(3_STATE, 0x02, 0)
-
-#define ZORRO_MANUF_COMMODORE_BRAUNSCHWEIG 0x0201
-#define ZORRO_PROD_CBM_A2088_A2286 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x01, 0)
-#define ZORRO_PROD_CBM_A2286 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x02, 0)
-#define ZORRO_PROD_CBM_A4091_1 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x54, 0)
-#define ZORRO_PROD_CBM_A2386SX_1 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x67, 0)
-
-#define ZORRO_MANUF_COMMODORE_WEST_CHESTER_1 0x0202
-#define ZORRO_PROD_CBM_A2090A ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x01, 0)
-#define ZORRO_PROD_CBM_A590_A2091_1 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x02, 0)
-#define ZORRO_PROD_CBM_A590_A2091_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x03, 0)
-#define ZORRO_PROD_CBM_A2090B ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x04, 0)
-#define ZORRO_PROD_CBM_A2060 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x09, 0)
-#define ZORRO_PROD_CBM_A590_A2052_A2058_A2091 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x0A, 0)
-#define ZORRO_PROD_CBM_A560_RAM ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x20, 0)
-#define ZORRO_PROD_CBM_A2232_PROTOTYPE ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x45, 0)
-#define ZORRO_PROD_CBM_A2232 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x46, 0)
-#define ZORRO_PROD_CBM_A2620 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x50, 0)
-#define ZORRO_PROD_CBM_A2630 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x51, 0)
-#define ZORRO_PROD_CBM_A4091_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x54, 0)
-#define ZORRO_PROD_CBM_A2065_1 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x5A, 0)
-#define ZORRO_PROD_CBM_ROMULATOR ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x60, 0)
-#define ZORRO_PROD_CBM_A3000_TEST_FIXTURE ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x61, 0)
-#define ZORRO_PROD_CBM_A2386SX_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x67, 0)
-#define ZORRO_PROD_CBM_A2065_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x70, 0)
-
-#define ZORRO_MANUF_COMMODORE_WEST_CHESTER_2 0x0203
-#define ZORRO_PROD_CBM_A2090A_CM ZORRO_ID(COMMODORE_WEST_CHESTER_2, 0x03, 0)
-
-#define ZORRO_MANUF_PROGRESSIVE_PERIPHERALS_AND_SYSTEMS_2 0x02F4
-#define ZORRO_PROD_PPS_EXP8000 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS_2, 0x02, 0)
-
-#define ZORRO_MANUF_KOLFF_COMPUTER_SUPPLIES 0x02FF
-#define ZORRO_PROD_KCS_POWER_PC_BOARD ZORRO_ID(KOLFF_COMPUTER_SUPPLIES, 0x00, 0)
-
-#define ZORRO_MANUF_CARDCO_1 0x03EC
-#define ZORRO_PROD_CARDCO_KRONOS_2000_1 ZORRO_ID(CARDCO_1, 0x04, 0)
-#define ZORRO_PROD_CARDCO_A1000_1 ZORRO_ID(CARDCO_1, 0x0C, 0)
-#define ZORRO_PROD_CARDCO_ESCORT ZORRO_ID(CARDCO_1, 0x0E, 0)
-#define ZORRO_PROD_CARDCO_A2410 ZORRO_ID(CARDCO_1, 0xF5, 0)
-
-#define ZORRO_MANUF_A_SQUARED 0x03ED
-#define ZORRO_PROD_A_SQUARED_LIVE_2000 ZORRO_ID(A_SQUARED, 0x01, 0)
-
-#define ZORRO_MANUF_COMSPEC_COMMUNICATIONS 0x03EE
-#define ZORRO_PROD_COMSPEC_COMMUNICATIONS_AX2000 ZORRO_ID(COMSPEC_COMMUNICATIONS, 0x01, 0)
-
-#define ZORRO_MANUF_ANAKIN_RESEARCH 0x03F1
-#define ZORRO_PROD_ANAKIN_RESEARCH_EASYL ZORRO_ID(ANAKIN_RESEARCH, 0x01, 0)
-
-#define ZORRO_MANUF_MICROBOTICS 0x03F2
-#define ZORRO_PROD_MICROBOTICS_STARBOARD_II ZORRO_ID(MICROBOTICS, 0x00, 0)
-#define ZORRO_PROD_MICROBOTICS_STARDRIVE ZORRO_ID(MICROBOTICS, 0x02, 0)
-#define ZORRO_PROD_MICROBOTICS_8_UP_A ZORRO_ID(MICROBOTICS, 0x03, 0)
-#define ZORRO_PROD_MICROBOTICS_8_UP_Z ZORRO_ID(MICROBOTICS, 0x04, 0)
-#define ZORRO_PROD_MICROBOTICS_DELTA_RAM ZORRO_ID(MICROBOTICS, 0x20, 0)
-#define ZORRO_PROD_MICROBOTICS_8_STAR_RAM ZORRO_ID(MICROBOTICS, 0x40, 0)
-#define ZORRO_PROD_MICROBOTICS_8_STAR ZORRO_ID(MICROBOTICS, 0x41, 0)
-#define ZORRO_PROD_MICROBOTICS_VXL_RAM_32 ZORRO_ID(MICROBOTICS, 0x44, 0)
-#define ZORRO_PROD_MICROBOTICS_VXL_68030 ZORRO_ID(MICROBOTICS, 0x45, 0)
-#define ZORRO_PROD_MICROBOTICS_DELTA ZORRO_ID(MICROBOTICS, 0x60, 0)
-#define ZORRO_PROD_MICROBOTICS_MBX_1200_1200Z_RAM ZORRO_ID(MICROBOTICS, 0x81, 0)
-#define ZORRO_PROD_MICROBOTICS_HARDFRAME_2000_1 ZORRO_ID(MICROBOTICS, 0x96, 0)
-#define ZORRO_PROD_MICROBOTICS_HARDFRAME_2000_2 ZORRO_ID(MICROBOTICS, 0x9E, 0)
-#define ZORRO_PROD_MICROBOTICS_MBX_1200_1200Z ZORRO_ID(MICROBOTICS, 0xC1, 0)
-
-#define ZORRO_MANUF_ACCESS_ASSOCIATES_ALEGRA 0x03F4
-
-#define ZORRO_MANUF_EXPANSION_TECHNOLOGIES 0x03F6
-
-#define ZORRO_MANUF_ASDG 0x03FF
-#define ZORRO_PROD_ASDG_MEMORY_1 ZORRO_ID(ASDG, 0x01, 0)
-#define ZORRO_PROD_ASDG_MEMORY_2 ZORRO_ID(ASDG, 0x02, 0)
-#define ZORRO_PROD_ASDG_EB920_LAN_ROVER ZORRO_ID(ASDG, 0xFE, 0)
-#define ZORRO_PROD_ASDG_GPIB_DUALIEEE488_TWIN_X ZORRO_ID(ASDG, 0xFF, 0)
-
-#define ZORRO_MANUF_IMTRONICS_1 0x0404
-#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_1 ZORRO_ID(IMTRONICS_1, 0x39, 0)
-#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_2 ZORRO_ID(IMTRONICS_1, 0x57, 0)
-
-#define ZORRO_MANUF_CBM_UNIVERSITY_OF_LOWELL 0x0406
-#define ZORRO_PROD_CBM_A2410 ZORRO_ID(CBM_UNIVERSITY_OF_LOWELL, 0x00, 0)
-
-#define ZORRO_MANUF_AMERISTAR 0x041D
-#define ZORRO_PROD_AMERISTAR_A2065 ZORRO_ID(AMERISTAR, 0x01, 0)
-#define ZORRO_PROD_AMERISTAR_A560 ZORRO_ID(AMERISTAR, 0x09, 0)
-#define ZORRO_PROD_AMERISTAR_A4066 ZORRO_ID(AMERISTAR, 0x0A, 0)
-
-#define ZORRO_MANUF_SUPRA 0x0420
-#define ZORRO_PROD_SUPRA_SUPRADRIVE_4x4 ZORRO_ID(SUPRA, 0x01, 0)
-#define ZORRO_PROD_SUPRA_1000_RAM ZORRO_ID(SUPRA, 0x02, 0)
-#define ZORRO_PROD_SUPRA_2000_DMA ZORRO_ID(SUPRA, 0x03, 0)
-#define ZORRO_PROD_SUPRA_500 ZORRO_ID(SUPRA, 0x05, 0)
-#define ZORRO_PROD_SUPRA_500_SCSI ZORRO_ID(SUPRA, 0x08, 0)
-#define ZORRO_PROD_SUPRA_500XP_2000_RAM ZORRO_ID(SUPRA, 0x09, 0)
-#define ZORRO_PROD_SUPRA_500RX_2000_RAM ZORRO_ID(SUPRA, 0x0A, 0)
-#define ZORRO_PROD_SUPRA_2400ZI ZORRO_ID(SUPRA, 0x0B, 0)
-#define ZORRO_PROD_SUPRA_500XP_SUPRADRIVE_WORDSYNC ZORRO_ID(SUPRA, 0x0C, 0)
-#define ZORRO_PROD_SUPRA_SUPRADRIVE_WORDSYNC_II ZORRO_ID(SUPRA, 0x0D, 0)
-#define ZORRO_PROD_SUPRA_2400ZIPLUS ZORRO_ID(SUPRA, 0x10, 0)
-
-#define ZORRO_MANUF_COMPUTER_SYSTEMS_ASSOCIATES 0x0422
-#define ZORRO_PROD_CSA_MAGNUM ZORRO_ID(COMPUTER_SYSTEMS_ASSOCIATES, 0x11, 0)
-#define ZORRO_PROD_CSA_12_GAUGE ZORRO_ID(COMPUTER_SYSTEMS_ASSOCIATES, 0x15, 0)
-
-#define ZORRO_MANUF_MARC_MICHAEL_GROTH 0x0439
-
-#define ZORRO_MANUF_M_TECH 0x0502
-#define ZORRO_PROD_MTEC_AT500_1 ZORRO_ID(M_TECH, 0x03, 0)
-
-#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_1 0x06E1
-#define ZORRO_PROD_GVP_IMPACT_SERIES_I ZORRO_ID(GREAT_VALLEY_PRODUCTS_1, 0x08, 0)
-
-#define ZORRO_MANUF_BYTEBOX 0x07DA
-#define ZORRO_PROD_BYTEBOX_A500 ZORRO_ID(BYTEBOX, 0x00, 0)
-
-#define ZORRO_MANUF_DKB_POWER_COMPUTING 0x07DC
-#define ZORRO_PROD_DKB_POWER_COMPUTING_SECUREKEY ZORRO_ID(DKB_POWER_COMPUTING, 0x09, 0)
-#define ZORRO_PROD_DKB_POWER_COMPUTING_DKM_3128 ZORRO_ID(DKB_POWER_COMPUTING, 0x0E, 0)
-#define ZORRO_PROD_DKB_POWER_COMPUTING_RAPID_FIRE ZORRO_ID(DKB_POWER_COMPUTING, 0x0F, 0)
-#define ZORRO_PROD_DKB_POWER_COMPUTING_DKM_1202 ZORRO_ID(DKB_POWER_COMPUTING, 0x10, 0)
-#define ZORRO_PROD_DKB_POWER_COMPUTING_COBRA_VIPER_II_68EC030 ZORRO_ID(DKB_POWER_COMPUTING, 0x12, 0)
-#define ZORRO_PROD_DKB_POWER_COMPUTING_WILDFIRE_060_1 ZORRO_ID(DKB_POWER_COMPUTING, 0x17, 0)
-#define ZORRO_PROD_DKB_POWER_COMPUTING_WILDFIRE_060_2 ZORRO_ID(DKB_POWER_COMPUTING, 0xFF, 0)
-
-#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_2 0x07E1
-#define ZORRO_PROD_GVP_IMPACT_SERIES_I_4K ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x01, 0)
-#define ZORRO_PROD_GVP_IMPACT_SERIES_I_16K_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x02, 0)
-#define ZORRO_PROD_GVP_IMPACT_SERIES_I_16K_3 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x03, 0)
-#define ZORRO_PROD_GVP_IMPACT_3001_IDE_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x08, 0)
-#define ZORRO_PROD_GVP_IMPACT_3001_RAM ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x09, 0)
-#define ZORRO_PROD_GVP_IMPACT_SERIES_II_RAM_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0A, 0)
-#define ZORRO_PROD_GVP_EPC_BASE ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0)
-#define ZORRO_PROD_GVP_GFORCE_040_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x20)
-#define ZORRO_PROD_GVP_GFORCE_040_SCSI_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x30)
-#define ZORRO_PROD_GVP_A1291 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x40)
-#define ZORRO_PROD_GVP_COMBO_030_R4 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x60)
-#define ZORRO_PROD_GVP_COMBO_030_R4_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x70)
-#define ZORRO_PROD_GVP_PHONEPAK ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x78)
-#define ZORRO_PROD_GVP_IO_EXTENDER ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x98)
-#define ZORRO_PROD_GVP_GFORCE_030 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xa0)
-#define ZORRO_PROD_GVP_GFORCE_030_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xb0)
-#define ZORRO_PROD_GVP_A530 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xc0)
-#define ZORRO_PROD_GVP_A530_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xd0)
-#define ZORRO_PROD_GVP_COMBO_030_R3 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xe0)
-#define ZORRO_PROD_GVP_COMBO_030_R3_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xf0)
-#define ZORRO_PROD_GVP_SERIES_II ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xf8)
-#define ZORRO_PROD_GVP_IMPACT_3001_IDE_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0D, 0)
-/*#define ZORRO_PROD_GVP_A2000_030 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0D, 0)*/
-/*#define ZORRO_PROD_GVP_GFORCE_040_SCSI_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0D, 0)*/
-#define ZORRO_PROD_GVP_GFORCE_040_060 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x16, 0)
-#define ZORRO_PROD_GVP_IMPACT_VISION_24 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x20, 0)
-#define ZORRO_PROD_GVP_GFORCE_040_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0xFF, 0)
-
-#define ZORRO_MANUF_CALIFORNIA_ACCESS_SYNERGY 0x07E5
-#define ZORRO_PROD_CALIFORNIA_ACCESS_SYNERGY_MALIBU ZORRO_ID(CALIFORNIA_ACCESS_SYNERGY, 0x01, 0)
-
-#define ZORRO_MANUF_XETEC 0x07E6
-#define ZORRO_PROD_XETEC_FASTCARD ZORRO_ID(XETEC, 0x01, 0)
-#define ZORRO_PROD_XETEC_FASTCARD_RAM ZORRO_ID(XETEC, 0x02, 0)
-#define ZORRO_PROD_XETEC_FASTCARD_PLUS ZORRO_ID(XETEC, 0x03, 0)
-
-#define ZORRO_MANUF_PROGRESSIVE_PERIPHERALS_AND_SYSTEMS 0x07EA
-#define ZORRO_PROD_PPS_MERCURY ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x00, 0)
-#define ZORRO_PROD_PPS_A3000_68040 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x01, 0)
-#define ZORRO_PROD_PPS_A2000_68040 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x69, 0)
-#define ZORRO_PROD_PPS_ZEUS ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x96, 0)
-#define ZORRO_PROD_PPS_A500_68040 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0xBB, 0)
-
-#define ZORRO_MANUF_XEBEC 0x07EC
-
-#define ZORRO_MANUF_SPIRIT_TECHNOLOGY 0x07F2
-#define ZORRO_PROD_SPIRIT_TECHNOLOGY_INSIDER_IN1000 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x01, 0)
-#define ZORRO_PROD_SPIRIT_TECHNOLOGY_INSIDER_IN500 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x02, 0)
-#define ZORRO_PROD_SPIRIT_TECHNOLOGY_SIN500 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x03, 0)
-#define ZORRO_PROD_SPIRIT_TECHNOLOGY_HDA_506 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x04, 0)
-#define ZORRO_PROD_SPIRIT_TECHNOLOGY_AX_S ZORRO_ID(SPIRIT_TECHNOLOGY, 0x05, 0)
-#define ZORRO_PROD_SPIRIT_TECHNOLOGY_OCTABYTE ZORRO_ID(SPIRIT_TECHNOLOGY, 0x06, 0)
-#define ZORRO_PROD_SPIRIT_TECHNOLOGY_INMATE ZORRO_ID(SPIRIT_TECHNOLOGY, 0x08, 0)
-
-#define ZORRO_MANUF_SPIRIT_TECHNOLOGY_2 0x07F3
-
-#define ZORRO_MANUF_BSC_ALFADATA_1 0x07FE
-#define ZORRO_PROD_BSC_ALF_3_1 ZORRO_ID(BSC_ALFADATA_1, 0x03, 0)
-
-#define ZORRO_MANUF_BSC_ALFADATA_2 0x0801
-#define ZORRO_PROD_BSC_ALF_2_1 ZORRO_ID(BSC_ALFADATA_2, 0x01, 0)
-#define ZORRO_PROD_BSC_ALF_2_2 ZORRO_ID(BSC_ALFADATA_2, 0x02, 0)
-#define ZORRO_PROD_BSC_ALF_3_2 ZORRO_ID(BSC_ALFADATA_2, 0x03, 0)
-
-#define ZORRO_MANUF_CARDCO_2 0x0802
-#define ZORRO_PROD_CARDCO_KRONOS_2000_2 ZORRO_ID(CARDCO_2, 0x04, 0)
-#define ZORRO_PROD_CARDCO_A1000_2 ZORRO_ID(CARDCO_2, 0x0C, 0)
-
-#define ZORRO_MANUF_JOCHHEIM 0x0804
-#define ZORRO_PROD_JOCHHEIM_RAM ZORRO_ID(JOCHHEIM, 0x01, 0)
-
-#define ZORRO_MANUF_CHECKPOINT_TECHNOLOGIES 0x0807
-#define ZORRO_PROD_CHECKPOINT_TECHNOLOGIES_SERIAL_SOLUTION ZORRO_ID(CHECKPOINT_TECHNOLOGIES, 0x00, 0)
-
-#define ZORRO_MANUF_EDOTRONIK 0x0810
-#define ZORRO_PROD_EDOTRONIK_IEEE_488 ZORRO_ID(EDOTRONIK, 0x01, 0)
-#define ZORRO_PROD_EDOTRONIK_8032 ZORRO_ID(EDOTRONIK, 0x02, 0)
-#define ZORRO_PROD_EDOTRONIK_MULTISERIAL ZORRO_ID(EDOTRONIK, 0x03, 0)
-#define ZORRO_PROD_EDOTRONIK_VIDEODIGITIZER ZORRO_ID(EDOTRONIK, 0x04, 0)
-#define ZORRO_PROD_EDOTRONIK_PARALLEL_IO ZORRO_ID(EDOTRONIK, 0x05, 0)
-#define ZORRO_PROD_EDOTRONIK_PIC_PROTOYPING ZORRO_ID(EDOTRONIK, 0x06, 0)
-#define ZORRO_PROD_EDOTRONIK_ADC ZORRO_ID(EDOTRONIK, 0x07, 0)
-#define ZORRO_PROD_EDOTRONIK_VME ZORRO_ID(EDOTRONIK, 0x08, 0)
-#define ZORRO_PROD_EDOTRONIK_DSP96000 ZORRO_ID(EDOTRONIK, 0x09, 0)
-
-#define ZORRO_MANUF_NES_INC 0x0813
-#define ZORRO_PROD_NES_INC_RAM ZORRO_ID(NES_INC, 0x00, 0)
-
-#define ZORRO_MANUF_ICD 0x0817
-#define ZORRO_PROD_ICD_ADVANTAGE_2000_SCSI ZORRO_ID(ICD, 0x01, 0)
-#define ZORRO_PROD_ICD_ADVANTAGE_IDE ZORRO_ID(ICD, 0x03, 0)
-#define ZORRO_PROD_ICD_ADVANTAGE_2080_RAM ZORRO_ID(ICD, 0x04, 0)
-
-#define ZORRO_MANUF_KUPKE_2 0x0819
-#define ZORRO_PROD_KUPKE_OMTI ZORRO_ID(KUPKE_2, 0x01, 0)
-#define ZORRO_PROD_KUPKE_SCSI_II ZORRO_ID(KUPKE_2, 0x02, 0)
-#define ZORRO_PROD_KUPKE_GOLEM_BOX ZORRO_ID(KUPKE_2, 0x03, 0)
-#define ZORRO_PROD_KUPKE_030_882 ZORRO_ID(KUPKE_2, 0x04, 0)
-#define ZORRO_PROD_KUPKE_SCSI_AT ZORRO_ID(KUPKE_2, 0x05, 0)
-
-#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_3 0x081D
-#define ZORRO_PROD_GVP_A2000_RAM8 ZORRO_ID(GREAT_VALLEY_PRODUCTS_3, 0x09, 0)
-#define ZORRO_PROD_GVP_IMPACT_SERIES_II_RAM_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_3, 0x0A, 0)
-
-#define ZORRO_MANUF_INTERWORKS_NETWORK 0x081E
-
-#define ZORRO_MANUF_HARDITAL_SYNTHESIS 0x0820
-#define ZORRO_PROD_HARDITAL_SYNTHESIS_TQM_68030_68882 ZORRO_ID(HARDITAL_SYNTHESIS, 0x14, 0)
-
-#define ZORRO_MANUF_APPLIED_ENGINEERING 0x0828
-#define ZORRO_PROD_APPLIED_ENGINEERING_DL2000 ZORRO_ID(APPLIED_ENGINEERING, 0x10, 0)
-#define ZORRO_PROD_APPLIED_ENGINEERING_RAM_WORKS ZORRO_ID(APPLIED_ENGINEERING, 0xE0, 0)
-
-#define ZORRO_MANUF_BSC_ALFADATA_3 0x082C
-#define ZORRO_PROD_BSC_OKTAGON_2008 ZORRO_ID(BSC_ALFADATA_3, 0x05, 0)
-#define ZORRO_PROD_BSC_TANDEM_AT_2008_508 ZORRO_ID(BSC_ALFADATA_3, 0x06, 0)
-#define ZORRO_PROD_BSC_ALFA_RAM_1200 ZORRO_ID(BSC_ALFADATA_3, 0x07, 0)
-#define ZORRO_PROD_BSC_OKTAGON_2008_RAM ZORRO_ID(BSC_ALFADATA_3, 0x08, 0)
-#define ZORRO_PROD_BSC_MULTIFACE_I ZORRO_ID(BSC_ALFADATA_3, 0x10, 0)
-#define ZORRO_PROD_BSC_MULTIFACE_II ZORRO_ID(BSC_ALFADATA_3, 0x11, 0)
-#define ZORRO_PROD_BSC_MULTIFACE_III ZORRO_ID(BSC_ALFADATA_3, 0x12, 0)
-#define ZORRO_PROD_BSC_FRAMEMASTER_II ZORRO_ID(BSC_ALFADATA_3, 0x20, 0)
-#define ZORRO_PROD_BSC_GRAFFITI_RAM ZORRO_ID(BSC_ALFADATA_3, 0x21, 0)
-#define ZORRO_PROD_BSC_GRAFFITI_REG ZORRO_ID(BSC_ALFADATA_3, 0x22, 0)
-#define ZORRO_PROD_BSC_ISDN_MASTERCARD ZORRO_ID(BSC_ALFADATA_3, 0x40, 0)
-#define ZORRO_PROD_BSC_ISDN_MASTERCARD_II ZORRO_ID(BSC_ALFADATA_3, 0x41, 0)
-
-#define ZORRO_MANUF_PHOENIX 0x0835
-#define ZORRO_PROD_PHOENIX_ST506 ZORRO_ID(PHOENIX, 0x21, 0)
-#define ZORRO_PROD_PHOENIX_SCSI ZORRO_ID(PHOENIX, 0x22, 0)
-#define ZORRO_PROD_PHOENIX_RAM ZORRO_ID(PHOENIX, 0xBE, 0)
-
-#define ZORRO_MANUF_ADVANCED_STORAGE_SYSTEMS 0x0836
-#define ZORRO_PROD_ADVANCED_STORAGE_SYSTEMS_NEXUS ZORRO_ID(ADVANCED_STORAGE_SYSTEMS, 0x01, 0)
-#define ZORRO_PROD_ADVANCED_STORAGE_SYSTEMS_NEXUS_RAM ZORRO_ID(ADVANCED_STORAGE_SYSTEMS, 0x08, 0)
-
-#define ZORRO_MANUF_IMPULSE 0x0838
-#define ZORRO_PROD_IMPULSE_FIRECRACKER_24 ZORRO_ID(IMPULSE, 0x00, 0)
-
-#define ZORRO_MANUF_IVS 0x0840
-#define ZORRO_PROD_IVS_GRANDSLAM_PIC_2 ZORRO_ID(IVS, 0x02, 0)
-#define ZORRO_PROD_IVS_GRANDSLAM_PIC_1 ZORRO_ID(IVS, 0x04, 0)
-#define ZORRO_PROD_IVS_OVERDRIVE ZORRO_ID(IVS, 0x10, 0)
-#define ZORRO_PROD_IVS_TRUMPCARD_CLASSIC ZORRO_ID(IVS, 0x30, 0)
-#define ZORRO_PROD_IVS_TRUMPCARD_PRO_GRANDSLAM ZORRO_ID(IVS, 0x34, 0)
-#define ZORRO_PROD_IVS_META_4 ZORRO_ID(IVS, 0x40, 0)
-#define ZORRO_PROD_IVS_WAVETOOLS ZORRO_ID(IVS, 0xBF, 0)
-#define ZORRO_PROD_IVS_VECTOR_1 ZORRO_ID(IVS, 0xF3, 0)
-#define ZORRO_PROD_IVS_VECTOR_2 ZORRO_ID(IVS, 0xF4, 0)
-
-#define ZORRO_MANUF_VECTOR_1 0x0841
-#define ZORRO_PROD_VECTOR_CONNECTION_1 ZORRO_ID(VECTOR_1, 0xE3, 0)
-
-#define ZORRO_MANUF_XPERT_PRODEV 0x0845
-#define ZORRO_PROD_XPERT_PRODEV_VISIONA_RAM ZORRO_ID(XPERT_PRODEV, 0x01, 0)
-#define ZORRO_PROD_XPERT_PRODEV_VISIONA_REG ZORRO_ID(XPERT_PRODEV, 0x02, 0)
-#define ZORRO_PROD_XPERT_PRODEV_MERLIN_RAM ZORRO_ID(XPERT_PRODEV, 0x03, 0)
-#define ZORRO_PROD_XPERT_PRODEV_MERLIN_REG_1 ZORRO_ID(XPERT_PRODEV, 0x04, 0)
-#define ZORRO_PROD_XPERT_PRODEV_MERLIN_REG_2 ZORRO_ID(XPERT_PRODEV, 0xC9, 0)
-
-#define ZORRO_MANUF_HYDRA_SYSTEMS 0x0849
-#define ZORRO_PROD_HYDRA_SYSTEMS_AMIGANET ZORRO_ID(HYDRA_SYSTEMS, 0x01, 0)
-
-#define ZORRO_MANUF_SUNRIZE_INDUSTRIES 0x084F
-#define ZORRO_PROD_SUNRIZE_INDUSTRIES_AD1012 ZORRO_ID(SUNRIZE_INDUSTRIES, 0x01, 0)
-#define ZORRO_PROD_SUNRIZE_INDUSTRIES_AD516 ZORRO_ID(SUNRIZE_INDUSTRIES, 0x02, 0)
-#define ZORRO_PROD_SUNRIZE_INDUSTRIES_DD512 ZORRO_ID(SUNRIZE_INDUSTRIES, 0x03, 0)
-
-#define ZORRO_MANUF_TRICERATOPS 0x0850
-#define ZORRO_PROD_TRICERATOPS_MULTI_IO ZORRO_ID(TRICERATOPS, 0x01, 0)
-
-#define ZORRO_MANUF_APPLIED_MAGIC 0x0851
-#define ZORRO_PROD_APPLIED_MAGIC_DMI_RESOLVER ZORRO_ID(APPLIED_MAGIC, 0x01, 0)
-#define ZORRO_PROD_APPLIED_MAGIC_DIGITAL_BROADCASTER ZORRO_ID(APPLIED_MAGIC, 0x06, 0)
-
-#define ZORRO_MANUF_GFX_BASE 0x085E
-#define ZORRO_PROD_GFX_BASE_GDA_1_VRAM ZORRO_ID(GFX_BASE, 0x00, 0)
-#define ZORRO_PROD_GFX_BASE_GDA_1 ZORRO_ID(GFX_BASE, 0x01, 0)
-
-#define ZORRO_MANUF_ROCTEC 0x0860
-#define ZORRO_PROD_ROCTEC_RH_800C ZORRO_ID(ROCTEC, 0x01, 0)
-#define ZORRO_PROD_ROCTEC_RH_800C_RAM ZORRO_ID(ROCTEC, 0x01, 0)
-
-#define ZORRO_MANUF_KATO 0x0861
-#define ZORRO_PROD_KATO_MELODY ZORRO_ID(KATO, 0x80, 0)
-/* ID clash!! */
-#define ZORRO_MANUF_HELFRICH_1 0x0861
-#define ZORRO_PROD_HELFRICH_RAINBOW_II ZORRO_ID(HELFRICH_1, 0x20, 0)
-#define ZORRO_PROD_HELFRICH_RAINBOW_III ZORRO_ID(HELFRICH_1, 0x21, 0)
-
-#define ZORRO_MANUF_ATLANTIS 0x0862
-
-#define ZORRO_MANUF_PROTAR 0x0864
-
-#define ZORRO_MANUF_ACS 0x0865
-
-#define ZORRO_MANUF_SOFTWARE_RESULTS_ENTERPRISES 0x0866
-#define ZORRO_PROD_SOFTWARE_RESULTS_ENTERPRISES_GOLDEN_GATE_2_BUS_PLUS ZORRO_ID(SOFTWARE_RESULTS_ENTERPRISES, 0x01, 0)
-
-#define ZORRO_MANUF_MASOBOSHI 0x086D
-#define ZORRO_PROD_MASOBOSHI_MASTER_CARD_SC201 ZORRO_ID(MASOBOSHI, 0x03, 0)
-#define ZORRO_PROD_MASOBOSHI_MASTER_CARD_MC702 ZORRO_ID(MASOBOSHI, 0x04, 0)
-#define ZORRO_PROD_MASOBOSHI_MVD_819 ZORRO_ID(MASOBOSHI, 0x07, 0)
-
-#define ZORRO_MANUF_MAINHATTAN_DATA 0x086F
-#define ZORRO_PROD_MAINHATTAN_DATA_IDE ZORRO_ID(MAINHATTAN_DATA, 0x01, 0)
-
-#define ZORRO_MANUF_VILLAGE_TRONIC 0x0877
-#define ZORRO_PROD_VILLAGE_TRONIC_DOMINO_RAM ZORRO_ID(VILLAGE_TRONIC, 0x01, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_DOMINO_REG ZORRO_ID(VILLAGE_TRONIC, 0x02, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_DOMINO_16M_PROTOTYPE ZORRO_ID(VILLAGE_TRONIC, 0x03, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_II_II_PLUS_RAM ZORRO_ID(VILLAGE_TRONIC, 0x0B, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_II_II_PLUS_REG ZORRO_ID(VILLAGE_TRONIC, 0x0C, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_II_II_PLUS_SEGMENTED_MODE ZORRO_ID(VILLAGE_TRONIC, 0x0D, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z2_RAM1 ZORRO_ID(VILLAGE_TRONIC, 0x15, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z2_RAM2 ZORRO_ID(VILLAGE_TRONIC, 0x16, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z2_REG ZORRO_ID(VILLAGE_TRONIC, 0x17, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z3 ZORRO_ID(VILLAGE_TRONIC, 0x18, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_ARIADNE ZORRO_ID(VILLAGE_TRONIC, 0xC9, 0)
-#define ZORRO_PROD_VILLAGE_TRONIC_ARIADNE2 ZORRO_ID(VILLAGE_TRONIC, 0xCA, 0)
-
-#define ZORRO_MANUF_UTILITIES_UNLIMITED 0x087B
-#define ZORRO_PROD_UTILITIES_UNLIMITED_EMPLANT_DELUXE ZORRO_ID(UTILITIES_UNLIMITED, 0x15, 0)
-#define ZORRO_PROD_UTILITIES_UNLIMITED_EMPLANT_DELUXE2 ZORRO_ID(UTILITIES_UNLIMITED, 0x20, 0)
-
-#define ZORRO_MANUF_AMITRIX 0x0880
-#define ZORRO_PROD_AMITRIX_MULTI_IO ZORRO_ID(AMITRIX, 0x01, 0)
-#define ZORRO_PROD_AMITRIX_CD_RAM ZORRO_ID(AMITRIX, 0x02, 0)
-
-#define ZORRO_MANUF_ARMAX 0x0885
-#define ZORRO_PROD_ARMAX_OMNIBUS ZORRO_ID(ARMAX, 0x00, 0)
-
-#define ZORRO_MANUF_ZEUS 0x088D
-#define ZORRO_PROD_ZEUS_SPIDER ZORRO_ID(ZEUS, 0x04, 0)
-
-#define ZORRO_MANUF_NEWTEK 0x088F
-#define ZORRO_PROD_NEWTEK_VIDEOTOASTER ZORRO_ID(NEWTEK, 0x00, 0)
-
-#define ZORRO_MANUF_M_TECH_GERMANY 0x0890
-#define ZORRO_PROD_MTEC_AT500_2 ZORRO_ID(M_TECH_GERMANY, 0x01, 0)
-#define ZORRO_PROD_MTEC_68030 ZORRO_ID(M_TECH_GERMANY, 0x03, 0)
-#define ZORRO_PROD_MTEC_68020I ZORRO_ID(M_TECH_GERMANY, 0x06, 0)
-#define ZORRO_PROD_MTEC_A1200_T68030_RTC ZORRO_ID(M_TECH_GERMANY, 0x20, 0)
-#define ZORRO_PROD_MTEC_VIPER_MK_V_E_MATRIX_530 ZORRO_ID(M_TECH_GERMANY, 0x21, 0)
-#define ZORRO_PROD_MTEC_8_MB_RAM ZORRO_ID(M_TECH_GERMANY, 0x22, 0)
-#define ZORRO_PROD_MTEC_VIPER_MK_V_E_MATRIX_530_SCSI_IDE ZORRO_ID(M_TECH_GERMANY, 0x24, 0)
-
-#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_4 0x0891
-#define ZORRO_PROD_GVP_EGS_28_24_SPECTRUM_RAM ZORRO_ID(GREAT_VALLEY_PRODUCTS_4, 0x01, 0)
-#define ZORRO_PROD_GVP_EGS_28_24_SPECTRUM_REG ZORRO_ID(GREAT_VALLEY_PRODUCTS_4, 0x02, 0)
-
-#define ZORRO_MANUF_APOLLO_1 0x0892
-#define ZORRO_PROD_APOLLO_A1200 ZORRO_ID(APOLLO_1, 0x01, 0)
-
-#define ZORRO_MANUF_HELFRICH_2 0x0893
-#define ZORRO_PROD_HELFRICH_PICCOLO_RAM ZORRO_ID(HELFRICH_2, 0x05, 0)
-#define ZORRO_PROD_HELFRICH_PICCOLO_REG ZORRO_ID(HELFRICH_2, 0x06, 0)
-#define ZORRO_PROD_HELFRICH_PEGGY_PLUS_MPEG ZORRO_ID(HELFRICH_2, 0x07, 0)
-#define ZORRO_PROD_HELFRICH_VIDEOCRUNCHER ZORRO_ID(HELFRICH_2, 0x08, 0)
-#define ZORRO_PROD_HELFRICH_SD64_RAM ZORRO_ID(HELFRICH_2, 0x0A, 0)
-#define ZORRO_PROD_HELFRICH_SD64_REG ZORRO_ID(HELFRICH_2, 0x0B, 0)
-
-#define ZORRO_MANUF_MACROSYSTEMS_USA 0x089B
-#define ZORRO_PROD_MACROSYSTEMS_WARP_ENGINE_40xx ZORRO_ID(MACROSYSTEMS_USA, 0x13, 0)
-
-#define ZORRO_MANUF_ELBOX_COMPUTER 0x089E
-#define ZORRO_PROD_ELBOX_COMPUTER_1200_4 ZORRO_ID(ELBOX_COMPUTER, 0x06, 0)
-
-#define ZORRO_MANUF_HARMS_PROFESSIONAL 0x0A00
-#define ZORRO_PROD_HARMS_PROFESSIONAL_030_PLUS ZORRO_ID(HARMS_PROFESSIONAL, 0x10, 0)
-#define ZORRO_PROD_HARMS_PROFESSIONAL_3500 ZORRO_ID(HARMS_PROFESSIONAL, 0xD0, 0)
-
-#define ZORRO_MANUF_MICRONIK 0x0A50
-#define ZORRO_PROD_MICRONIK_RCA_120 ZORRO_ID(MICRONIK, 0x0A, 0)
-
-#define ZORRO_MANUF_MICRONIK2 0x0F0F
-#define ZORRO_PROD_MICRONIK2_Z3I ZORRO_ID(MICRONIK2, 0x01, 0)
-
-#define ZORRO_MANUF_MEGAMICRO 0x1000
-#define ZORRO_PROD_MEGAMICRO_SCRAM_500 ZORRO_ID(MEGAMICRO, 0x03, 0)
-#define ZORRO_PROD_MEGAMICRO_SCRAM_500_RAM ZORRO_ID(MEGAMICRO, 0x04, 0)
-
-#define ZORRO_MANUF_IMTRONICS_2 0x1028
-#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_3 ZORRO_ID(IMTRONICS_2, 0x39, 0)
-#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_4 ZORRO_ID(IMTRONICS_2, 0x57, 0)
-
-/* unofficial ID */
-#define ZORRO_MANUF_INDIVIDUAL_COMPUTERS 0x1212
-#define ZORRO_PROD_INDIVIDUAL_COMPUTERS_BUDDHA ZORRO_ID(INDIVIDUAL_COMPUTERS, 0x00, 0)
-#define ZORRO_PROD_INDIVIDUAL_COMPUTERS_X_SURF ZORRO_ID(INDIVIDUAL_COMPUTERS, 0x17, 0)
-#define ZORRO_PROD_INDIVIDUAL_COMPUTERS_CATWEASEL ZORRO_ID(INDIVIDUAL_COMPUTERS, 0x2A, 0)
-
-#define ZORRO_MANUF_KUPKE_3 0x1248
-#define ZORRO_PROD_KUPKE_GOLEM_HD_3000 ZORRO_ID(KUPKE_3, 0x01, 0)
-
-#define ZORRO_MANUF_ITH 0x1388
-#define ZORRO_PROD_ITH_ISDN_MASTER_II ZORRO_ID(ITH, 0x01, 0)
-
-#define ZORRO_MANUF_VMC 0x1389
-#define ZORRO_PROD_VMC_ISDN_BLASTER_Z2 ZORRO_ID(VMC, 0x01, 0)
-#define ZORRO_PROD_VMC_HYPERCOM_4 ZORRO_ID(VMC, 0x02, 0)
-
-#define ZORRO_MANUF_INFORMATION 0x157C
-#define ZORRO_PROD_INFORMATION_ISDN_ENGINE_I ZORRO_ID(INFORMATION, 0x64, 0)
-
-#define ZORRO_MANUF_VORTEX 0x2017
-#define ZORRO_PROD_VORTEX_GOLDEN_GATE_80386SX ZORRO_ID(VORTEX, 0x07, 0)
-#define ZORRO_PROD_VORTEX_GOLDEN_GATE_RAM ZORRO_ID(VORTEX, 0x08, 0)
-#define ZORRO_PROD_VORTEX_GOLDEN_GATE_80486 ZORRO_ID(VORTEX, 0x09, 0)
-
-#define ZORRO_MANUF_EXPANSION_SYSTEMS 0x2062
-#define ZORRO_PROD_EXPANSION_SYSTEMS_DATAFLYER_4000SX ZORRO_ID(EXPANSION_SYSTEMS, 0x01, 0)
-#define ZORRO_PROD_EXPANSION_SYSTEMS_DATAFLYER_4000SX_RAM ZORRO_ID(EXPANSION_SYSTEMS, 0x02, 0)
-
-#define ZORRO_MANUF_READYSOFT 0x2100
-#define ZORRO_PROD_READYSOFT_AMAX_II_IV ZORRO_ID(READYSOFT, 0x01, 0)
-
-#define ZORRO_MANUF_PHASE5 0x2140
-#define ZORRO_PROD_PHASE5_BLIZZARD_RAM ZORRO_ID(PHASE5, 0x01, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD ZORRO_ID(PHASE5, 0x02, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD_1220_IV ZORRO_ID(PHASE5, 0x06, 0)
-#define ZORRO_PROD_PHASE5_FASTLANE_Z3_RAM ZORRO_ID(PHASE5, 0x0A, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD_1230_II_FASTLANE_Z3_CYBERSCSI_CYBERSTORM060 ZORRO_ID(PHASE5, 0x0B, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD_1220_CYBERSTORM ZORRO_ID(PHASE5, 0x0C, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD_1230 ZORRO_ID(PHASE5, 0x0D, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD_1230_IV_1260 ZORRO_ID(PHASE5, 0x11, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD_2060 ZORRO_ID(PHASE5, 0x18, 0)
-#define ZORRO_PROD_PHASE5_CYBERSTORM_MK_II ZORRO_ID(PHASE5, 0x19, 0)
-#define ZORRO_PROD_PHASE5_CYBERVISION64 ZORRO_ID(PHASE5, 0x22, 0)
-#define ZORRO_PROD_PHASE5_CYBERVISION64_3D_PROTOTYPE ZORRO_ID(PHASE5, 0x32, 0)
-#define ZORRO_PROD_PHASE5_CYBERVISION64_3D ZORRO_ID(PHASE5, 0x43, 0)
-#define ZORRO_PROD_PHASE5_CYBERSTORM_MK_III ZORRO_ID(PHASE5, 0x64, 0)
-#define ZORRO_PROD_PHASE5_BLIZZARD_603E_PLUS ZORRO_ID(PHASE5, 0x6e, 0)
-
-#define ZORRO_MANUF_DPS 0x2169
-#define ZORRO_PROD_DPS_PERSONAL_ANIMATION_RECORDER ZORRO_ID(DPS, 0x01, 0)
-
-#define ZORRO_MANUF_APOLLO_2 0x2200
-#define ZORRO_PROD_APOLLO_A620_68020_1 ZORRO_ID(APOLLO_2, 0x00, 0)
-#define ZORRO_PROD_APOLLO_A620_68020_2 ZORRO_ID(APOLLO_2, 0x01, 0)
-
-#define ZORRO_MANUF_APOLLO_3 0x2222
-#define ZORRO_PROD_APOLLO_AT_APOLLO ZORRO_ID(APOLLO_3, 0x22, 0)
-#define ZORRO_PROD_APOLLO_1230_1240_1260_2030_4040_4060 ZORRO_ID(APOLLO_3, 0x23, 0)
-
-#define ZORRO_MANUF_PETSOFF_LP 0x38A5
-#define ZORRO_PROD_PETSOFF_LP_DELFINA ZORRO_ID(PETSOFF_LP, 0x00, 0)
-#define ZORRO_PROD_PETSOFF_LP_DELFINA_LITE ZORRO_ID(PETSOFF_LP, 0x01, 0)
-
-#define ZORRO_MANUF_UWE_GERLACH 0x3FF7
-#define ZORRO_PROD_UWE_GERLACH_RAM_ROM ZORRO_ID(UWE_GERLACH, 0xd4, 0)
-
-#define ZORRO_MANUF_ACT 0x4231
-#define ZORRO_PROD_ACT_PRELUDE ZORRO_ID(ACT, 0x01, 0)
-
-#define ZORRO_MANUF_MACROSYSTEMS_GERMANY 0x4754
-#define ZORRO_PROD_MACROSYSTEMS_MAESTRO ZORRO_ID(MACROSYSTEMS_GERMANY, 0x03, 0)
-#define ZORRO_PROD_MACROSYSTEMS_VLAB ZORRO_ID(MACROSYSTEMS_GERMANY, 0x04, 0)
-#define ZORRO_PROD_MACROSYSTEMS_MAESTRO_PRO ZORRO_ID(MACROSYSTEMS_GERMANY, 0x05, 0)
-#define ZORRO_PROD_MACROSYSTEMS_RETINA ZORRO_ID(MACROSYSTEMS_GERMANY, 0x06, 0)
-#define ZORRO_PROD_MACROSYSTEMS_MULTI_EVOLUTION ZORRO_ID(MACROSYSTEMS_GERMANY, 0x08, 0)
-#define ZORRO_PROD_MACROSYSTEMS_TOCCATA ZORRO_ID(MACROSYSTEMS_GERMANY, 0x0C, 0)
-#define ZORRO_PROD_MACROSYSTEMS_RETINA_Z3 ZORRO_ID(MACROSYSTEMS_GERMANY, 0x10, 0)
-#define ZORRO_PROD_MACROSYSTEMS_VLAB_MOTION ZORRO_ID(MACROSYSTEMS_GERMANY, 0x12, 0)
-#define ZORRO_PROD_MACROSYSTEMS_ALTAIS ZORRO_ID(MACROSYSTEMS_GERMANY, 0x13, 0)
-#define ZORRO_PROD_MACROSYSTEMS_FALCON_040 ZORRO_ID(MACROSYSTEMS_GERMANY, 0xFD, 0)
-
-#define ZORRO_MANUF_COMBITEC 0x6766
-
-#define ZORRO_MANUF_SKI_PERIPHERALS 0x8000
-#define ZORRO_PROD_SKI_PERIPHERALS_MAST_FIREBALL ZORRO_ID(SKI_PERIPHERALS, 0x08, 0)
-#define ZORRO_PROD_SKI_PERIPHERALS_SCSI_DUAL_SERIAL ZORRO_ID(SKI_PERIPHERALS, 0x80, 0)
-
-#define ZORRO_MANUF_REIS_WARE_2 0xA9AD
-#define ZORRO_PROD_REIS_WARE_SCAN_KING ZORRO_ID(REIS_WARE_2, 0x11, 0)
-
-#define ZORRO_MANUF_CAMERON 0xAA01
-#define ZORRO_PROD_CAMERON_PERSONAL_A4 ZORRO_ID(CAMERON, 0x10, 0)
-
-#define ZORRO_MANUF_REIS_WARE 0xAA11
-#define ZORRO_PROD_REIS_WARE_HANDYSCANNER ZORRO_ID(REIS_WARE, 0x11, 0)
-
-#define ZORRO_MANUF_PHOENIX_2 0xB5A8
-#define ZORRO_PROD_PHOENIX_ST506_2 ZORRO_ID(PHOENIX_2, 0x21, 0)
-#define ZORRO_PROD_PHOENIX_SCSI_2 ZORRO_ID(PHOENIX_2, 0x22, 0)
-#define ZORRO_PROD_PHOENIX_RAM_2 ZORRO_ID(PHOENIX_2, 0xBE, 0)
-
-#define ZORRO_MANUF_COMBITEC_2 0xC008
-#define ZORRO_PROD_COMBITEC_HD ZORRO_ID(COMBITEC_2, 0x2A, 0)
-#define ZORRO_PROD_COMBITEC_SRAM ZORRO_ID(COMBITEC_2, 0x2B, 0)
-
-
- /*
- * Test and illegal Manufacturer IDs.
- */
-
-#define ZORRO_MANUF_HACKER 0x07DB
-#define ZORRO_PROD_GENERAL_PROTOTYPE ZORRO_ID(HACKER, 0x00, 0)
-#define ZORRO_PROD_HACKER_SCSI ZORRO_ID(HACKER, 0x01, 0)
-#define ZORRO_PROD_RESOURCE_MANAGEMENT_FORCE_QUICKNET_QN2000 ZORRO_ID(HACKER, 0x02, 0)
-#define ZORRO_PROD_VECTOR_CONNECTION_2 ZORRO_ID(HACKER, 0xE0, 0)
-#define ZORRO_PROD_VECTOR_CONNECTION_3 ZORRO_ID(HACKER, 0xE1, 0)
-#define ZORRO_PROD_VECTOR_CONNECTION_4 ZORRO_ID(HACKER, 0xE2, 0)
-#define ZORRO_PROD_VECTOR_CONNECTION_5 ZORRO_ID(HACKER, 0xE3, 0)
return sysctl_net_busy_poll;
}
-/* a wrapper to make debug_smp_processor_id() happy
- * we can use sched_clock() because we don't care much about precision
- * we only care that the average is bounded
- */
-#ifdef CONFIG_DEBUG_PREEMPT
-static inline u64 busy_loop_us_clock(void)
-{
- u64 rc;
-
- preempt_disable_notrace();
- rc = sched_clock();
- preempt_enable_no_resched_notrace();
-
- return rc >> 10;
-}
-#else /* CONFIG_DEBUG_PREEMPT */
static inline u64 busy_loop_us_clock(void)
{
- return sched_clock() >> 10;
+ return local_clock() >> 10;
}
-#endif /* CONFIG_DEBUG_PREEMPT */
static inline unsigned long sk_busy_loop_end_time(struct sock *sk)
{
struct net_device *dev;
struct list_head addr_list;
- int valid_ll_addr_cnt;
struct ifmcaddr6 *mc_list;
struct ifmcaddr6 *mc_tomb;
#define _TRACE_AER_H
#include <linux/tracepoint.h>
-#include <linux/edac.h>
+#include <linux/aer.h>
/*
TP_printk("%s PCIe Bus Error: severity=%s, %s\n",
__get_str(dev_name),
- __entry->severity == HW_EVENT_ERR_CORRECTED ? "Corrected" :
- __entry->severity == HW_EVENT_ERR_FATAL ?
- "Fatal" : "Uncorrected",
- __entry->severity == HW_EVENT_ERR_CORRECTED ?
+ __entry->severity == AER_CORRECTABLE ? "Corrected" :
+ __entry->severity == AER_FATAL ?
+ "Fatal" : "Uncorrected, non-fatal",
+ __entry->severity == AER_CORRECTABLE ?
__print_flags(__entry->status, "|", aer_correctable_errors) :
__print_flags(__entry->status, "|", aer_uncorrectable_errors))
);
*/
#ifndef __statfs_word
#if __BITS_PER_LONG == 64
-#define __statfs_word long
+#define __statfs_word __kernel_long_t
#else
#define __statfs_word __u32
#endif
header-y += xattr.h
header-y += xfrm.h
header-y += hw_breakpoint.h
+header-y += zorro.h
+header-y += zorro_ids.h
*/
#define KEXEC_ARCH_DEFAULT ( 0 << 16)
#define KEXEC_ARCH_386 ( 3 << 16)
+#define KEXEC_ARCH_68K ( 4 << 16)
#define KEXEC_ARCH_X86_64 (62 << 16)
#define KEXEC_ARCH_PPC (20 << 16)
#define KEXEC_ARCH_PPC64 (21 << 16)
#define PERF_FLAG_FD_NO_GROUP (1U << 0)
#define PERF_FLAG_FD_OUTPUT (1U << 1)
#define PERF_FLAG_PID_CGROUP (1U << 2) /* pid=cgroup id, per-cpu mode only */
+#define PERF_FLAG_FD_CLOEXEC (1U << 3) /* O_CLOEXEC */
union perf_mem_data_src {
__u64 val;
#define SCHED_BATCH 3
/* SCHED_ISO: reserved but not implemented yet */
#define SCHED_IDLE 5
+#define SCHED_DEADLINE 6
+
/* Can be ORed in to make sure the process is reverted back to SCHED_NORMAL on fork */
#define SCHED_RESET_ON_FORK 0x40000000
+/*
+ * For the sched_{set,get}attr() calls
+ */
+#define SCHED_FLAG_RESET_ON_FORK 0x01
#endif /* _UAPI_LINUX_SCHED_H */
--- /dev/null
+/*
+ * linux/zorro.h -- Amiga AutoConfig (Zorro) Bus Definitions
+ *
+ * Copyright (C) 1995--2003 Geert Uytterhoeven
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file COPYING in the main directory of this archive
+ * for more details.
+ */
+
+#ifndef _UAPI_LINUX_ZORRO_H
+#define _UAPI_LINUX_ZORRO_H
+
+#include <linux/types.h>
+
+
+ /*
+ * Each Zorro board has a 32-bit ID of the form
+ *
+ * mmmmmmmmmmmmmmmmppppppppeeeeeeee
+ *
+ * with
+ *
+ * mmmmmmmmmmmmmmmm 16-bit Manufacturer ID (assigned by CBM (sigh))
+ * pppppppp 8-bit Product ID (assigned by manufacturer)
+ * eeeeeeee 8-bit Extended Product ID (currently only used
+ * for some GVP boards)
+ */
+
+
+#define ZORRO_MANUF(id) ((id) >> 16)
+#define ZORRO_PROD(id) (((id) >> 8) & 0xff)
+#define ZORRO_EPC(id) ((id) & 0xff)
+
+#define ZORRO_ID(manuf, prod, epc) \
+ ((ZORRO_MANUF_##manuf << 16) | ((prod) << 8) | (epc))
+
+typedef __u32 zorro_id;
+
+
+/* Include the ID list */
+#include <linux/zorro_ids.h>
+
+
+ /*
+ * GVP identifies most of its products through the 'extended product code'
+ * (epc). The epc has to be ANDed with the GVP_PRODMASK before the
+ * identification.
+ */
+
+#define GVP_PRODMASK (0xf8)
+#define GVP_SCSICLKMASK (0x01)
+
+enum GVP_flags {
+ GVP_IO = 0x01,
+ GVP_ACCEL = 0x02,
+ GVP_SCSI = 0x04,
+ GVP_24BITDMA = 0x08,
+ GVP_25BITDMA = 0x10,
+ GVP_NOBANK = 0x20,
+ GVP_14MHZ = 0x40,
+};
+
+
+struct Node {
+ __be32 ln_Succ; /* Pointer to next (successor) */
+ __be32 ln_Pred; /* Pointer to previous (predecessor) */
+ __u8 ln_Type;
+ __s8 ln_Pri; /* Priority, for sorting */
+ __be32 ln_Name; /* ID string, null terminated */
+} __packed;
+
+struct ExpansionRom {
+ /* -First 16 bytes of the expansion ROM */
+ __u8 er_Type; /* Board type, size and flags */
+ __u8 er_Product; /* Product number, assigned by manufacturer */
+ __u8 er_Flags; /* Flags */
+ __u8 er_Reserved03; /* Must be zero ($ff inverted) */
+ __be16 er_Manufacturer; /* Unique ID, ASSIGNED BY COMMODORE-AMIGA! */
+ __be32 er_SerialNumber; /* Available for use by manufacturer */
+ __be16 er_InitDiagVec; /* Offset to optional "DiagArea" structure */
+ __u8 er_Reserved0c;
+ __u8 er_Reserved0d;
+ __u8 er_Reserved0e;
+ __u8 er_Reserved0f;
+} __packed;
+
+/* er_Type board type bits */
+#define ERT_TYPEMASK 0xc0
+#define ERT_ZORROII 0xc0
+#define ERT_ZORROIII 0x80
+
+/* other bits defined in er_Type */
+#define ERTB_MEMLIST 5 /* Link RAM into free memory list */
+#define ERTF_MEMLIST (1<<5)
+
+struct ConfigDev {
+ struct Node cd_Node;
+ __u8 cd_Flags; /* (read/write) */
+ __u8 cd_Pad; /* reserved */
+ struct ExpansionRom cd_Rom; /* copy of board's expansion ROM */
+ __be32 cd_BoardAddr; /* where in memory the board was placed */
+ __be32 cd_BoardSize; /* size of board in bytes */
+ __be16 cd_SlotAddr; /* which slot number (PRIVATE) */
+ __be16 cd_SlotSize; /* number of slots (PRIVATE) */
+ __be32 cd_Driver; /* pointer to node of driver */
+ __be32 cd_NextCD; /* linked list of drivers to config */
+ __be32 cd_Unused[4]; /* for whatever the driver wants */
+} __packed;
+
+#define ZORRO_NUM_AUTO 16
+
+#endif /* _UAPI_LINUX_ZORRO_H */
--- /dev/null
+/*
+ * Zorro board IDs
+ *
+ * Please keep sorted.
+ */
+
+
+#define ZORRO_MANUF_PACIFIC_PERIPHERALS 0x00D3
+#define ZORRO_PROD_PACIFIC_PERIPHERALS_SE_2000_A500 ZORRO_ID(PACIFIC_PERIPHERALS, 0x00, 0)
+#define ZORRO_PROD_PACIFIC_PERIPHERALS_SCSI ZORRO_ID(PACIFIC_PERIPHERALS, 0x0A, 0)
+
+#define ZORRO_MANUF_MACROSYSTEMS_USA_2 0x0100
+#define ZORRO_PROD_MACROSYSTEMS_WARP_ENGINE ZORRO_ID(MACROSYSTEMS_USA_2, 0x13, 0)
+
+#define ZORRO_MANUF_KUPKE_1 0x00DD
+#define ZORRO_PROD_KUPKE_GOLEM_RAM_BOX_2MB ZORRO_ID(KUPKE_1, 0x00, 0)
+
+#define ZORRO_MANUF_MEMPHIS 0x0100
+#define ZORRO_PROD_MEMPHIS_STORMBRINGER ZORRO_ID(MEMPHIS, 0x00, 0)
+
+#define ZORRO_MANUF_3_STATE 0x0200
+#define ZORRO_PROD_3_STATE_MEGAMIX_2000 ZORRO_ID(3_STATE, 0x02, 0)
+
+#define ZORRO_MANUF_COMMODORE_BRAUNSCHWEIG 0x0201
+#define ZORRO_PROD_CBM_A2088_A2286 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x01, 0)
+#define ZORRO_PROD_CBM_A2286 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x02, 0)
+#define ZORRO_PROD_CBM_A4091_1 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x54, 0)
+#define ZORRO_PROD_CBM_A2386SX_1 ZORRO_ID(COMMODORE_BRAUNSCHWEIG, 0x67, 0)
+
+#define ZORRO_MANUF_COMMODORE_WEST_CHESTER_1 0x0202
+#define ZORRO_PROD_CBM_A2090A ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x01, 0)
+#define ZORRO_PROD_CBM_A590_A2091_1 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x02, 0)
+#define ZORRO_PROD_CBM_A590_A2091_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x03, 0)
+#define ZORRO_PROD_CBM_A2090B ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x04, 0)
+#define ZORRO_PROD_CBM_A2060 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x09, 0)
+#define ZORRO_PROD_CBM_A590_A2052_A2058_A2091 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x0A, 0)
+#define ZORRO_PROD_CBM_A560_RAM ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x20, 0)
+#define ZORRO_PROD_CBM_A2232_PROTOTYPE ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x45, 0)
+#define ZORRO_PROD_CBM_A2232 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x46, 0)
+#define ZORRO_PROD_CBM_A2620 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x50, 0)
+#define ZORRO_PROD_CBM_A2630 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x51, 0)
+#define ZORRO_PROD_CBM_A4091_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x54, 0)
+#define ZORRO_PROD_CBM_A2065_1 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x5A, 0)
+#define ZORRO_PROD_CBM_ROMULATOR ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x60, 0)
+#define ZORRO_PROD_CBM_A3000_TEST_FIXTURE ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x61, 0)
+#define ZORRO_PROD_CBM_A2386SX_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x67, 0)
+#define ZORRO_PROD_CBM_A2065_2 ZORRO_ID(COMMODORE_WEST_CHESTER_1, 0x70, 0)
+
+#define ZORRO_MANUF_COMMODORE_WEST_CHESTER_2 0x0203
+#define ZORRO_PROD_CBM_A2090A_CM ZORRO_ID(COMMODORE_WEST_CHESTER_2, 0x03, 0)
+
+#define ZORRO_MANUF_PROGRESSIVE_PERIPHERALS_AND_SYSTEMS_2 0x02F4
+#define ZORRO_PROD_PPS_EXP8000 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS_2, 0x02, 0)
+
+#define ZORRO_MANUF_KOLFF_COMPUTER_SUPPLIES 0x02FF
+#define ZORRO_PROD_KCS_POWER_PC_BOARD ZORRO_ID(KOLFF_COMPUTER_SUPPLIES, 0x00, 0)
+
+#define ZORRO_MANUF_CARDCO_1 0x03EC
+#define ZORRO_PROD_CARDCO_KRONOS_2000_1 ZORRO_ID(CARDCO_1, 0x04, 0)
+#define ZORRO_PROD_CARDCO_A1000_1 ZORRO_ID(CARDCO_1, 0x0C, 0)
+#define ZORRO_PROD_CARDCO_ESCORT ZORRO_ID(CARDCO_1, 0x0E, 0)
+#define ZORRO_PROD_CARDCO_A2410 ZORRO_ID(CARDCO_1, 0xF5, 0)
+
+#define ZORRO_MANUF_A_SQUARED 0x03ED
+#define ZORRO_PROD_A_SQUARED_LIVE_2000 ZORRO_ID(A_SQUARED, 0x01, 0)
+
+#define ZORRO_MANUF_COMSPEC_COMMUNICATIONS 0x03EE
+#define ZORRO_PROD_COMSPEC_COMMUNICATIONS_AX2000 ZORRO_ID(COMSPEC_COMMUNICATIONS, 0x01, 0)
+
+#define ZORRO_MANUF_ANAKIN_RESEARCH 0x03F1
+#define ZORRO_PROD_ANAKIN_RESEARCH_EASYL ZORRO_ID(ANAKIN_RESEARCH, 0x01, 0)
+
+#define ZORRO_MANUF_MICROBOTICS 0x03F2
+#define ZORRO_PROD_MICROBOTICS_STARBOARD_II ZORRO_ID(MICROBOTICS, 0x00, 0)
+#define ZORRO_PROD_MICROBOTICS_STARDRIVE ZORRO_ID(MICROBOTICS, 0x02, 0)
+#define ZORRO_PROD_MICROBOTICS_8_UP_A ZORRO_ID(MICROBOTICS, 0x03, 0)
+#define ZORRO_PROD_MICROBOTICS_8_UP_Z ZORRO_ID(MICROBOTICS, 0x04, 0)
+#define ZORRO_PROD_MICROBOTICS_DELTA_RAM ZORRO_ID(MICROBOTICS, 0x20, 0)
+#define ZORRO_PROD_MICROBOTICS_8_STAR_RAM ZORRO_ID(MICROBOTICS, 0x40, 0)
+#define ZORRO_PROD_MICROBOTICS_8_STAR ZORRO_ID(MICROBOTICS, 0x41, 0)
+#define ZORRO_PROD_MICROBOTICS_VXL_RAM_32 ZORRO_ID(MICROBOTICS, 0x44, 0)
+#define ZORRO_PROD_MICROBOTICS_VXL_68030 ZORRO_ID(MICROBOTICS, 0x45, 0)
+#define ZORRO_PROD_MICROBOTICS_DELTA ZORRO_ID(MICROBOTICS, 0x60, 0)
+#define ZORRO_PROD_MICROBOTICS_MBX_1200_1200Z_RAM ZORRO_ID(MICROBOTICS, 0x81, 0)
+#define ZORRO_PROD_MICROBOTICS_HARDFRAME_2000_1 ZORRO_ID(MICROBOTICS, 0x96, 0)
+#define ZORRO_PROD_MICROBOTICS_HARDFRAME_2000_2 ZORRO_ID(MICROBOTICS, 0x9E, 0)
+#define ZORRO_PROD_MICROBOTICS_MBX_1200_1200Z ZORRO_ID(MICROBOTICS, 0xC1, 0)
+
+#define ZORRO_MANUF_ACCESS_ASSOCIATES_ALEGRA 0x03F4
+
+#define ZORRO_MANUF_EXPANSION_TECHNOLOGIES 0x03F6
+
+#define ZORRO_MANUF_ASDG 0x03FF
+#define ZORRO_PROD_ASDG_MEMORY_1 ZORRO_ID(ASDG, 0x01, 0)
+#define ZORRO_PROD_ASDG_MEMORY_2 ZORRO_ID(ASDG, 0x02, 0)
+#define ZORRO_PROD_ASDG_EB920_LAN_ROVER ZORRO_ID(ASDG, 0xFE, 0)
+#define ZORRO_PROD_ASDG_GPIB_DUALIEEE488_TWIN_X ZORRO_ID(ASDG, 0xFF, 0)
+
+#define ZORRO_MANUF_IMTRONICS_1 0x0404
+#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_1 ZORRO_ID(IMTRONICS_1, 0x39, 0)
+#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_2 ZORRO_ID(IMTRONICS_1, 0x57, 0)
+
+#define ZORRO_MANUF_CBM_UNIVERSITY_OF_LOWELL 0x0406
+#define ZORRO_PROD_CBM_A2410 ZORRO_ID(CBM_UNIVERSITY_OF_LOWELL, 0x00, 0)
+
+#define ZORRO_MANUF_AMERISTAR 0x041D
+#define ZORRO_PROD_AMERISTAR_A2065 ZORRO_ID(AMERISTAR, 0x01, 0)
+#define ZORRO_PROD_AMERISTAR_A560 ZORRO_ID(AMERISTAR, 0x09, 0)
+#define ZORRO_PROD_AMERISTAR_A4066 ZORRO_ID(AMERISTAR, 0x0A, 0)
+
+#define ZORRO_MANUF_SUPRA 0x0420
+#define ZORRO_PROD_SUPRA_SUPRADRIVE_4x4 ZORRO_ID(SUPRA, 0x01, 0)
+#define ZORRO_PROD_SUPRA_1000_RAM ZORRO_ID(SUPRA, 0x02, 0)
+#define ZORRO_PROD_SUPRA_2000_DMA ZORRO_ID(SUPRA, 0x03, 0)
+#define ZORRO_PROD_SUPRA_500 ZORRO_ID(SUPRA, 0x05, 0)
+#define ZORRO_PROD_SUPRA_500_SCSI ZORRO_ID(SUPRA, 0x08, 0)
+#define ZORRO_PROD_SUPRA_500XP_2000_RAM ZORRO_ID(SUPRA, 0x09, 0)
+#define ZORRO_PROD_SUPRA_500RX_2000_RAM ZORRO_ID(SUPRA, 0x0A, 0)
+#define ZORRO_PROD_SUPRA_2400ZI ZORRO_ID(SUPRA, 0x0B, 0)
+#define ZORRO_PROD_SUPRA_500XP_SUPRADRIVE_WORDSYNC ZORRO_ID(SUPRA, 0x0C, 0)
+#define ZORRO_PROD_SUPRA_SUPRADRIVE_WORDSYNC_II ZORRO_ID(SUPRA, 0x0D, 0)
+#define ZORRO_PROD_SUPRA_2400ZIPLUS ZORRO_ID(SUPRA, 0x10, 0)
+
+#define ZORRO_MANUF_COMPUTER_SYSTEMS_ASSOCIATES 0x0422
+#define ZORRO_PROD_CSA_MAGNUM ZORRO_ID(COMPUTER_SYSTEMS_ASSOCIATES, 0x11, 0)
+#define ZORRO_PROD_CSA_12_GAUGE ZORRO_ID(COMPUTER_SYSTEMS_ASSOCIATES, 0x15, 0)
+
+#define ZORRO_MANUF_MARC_MICHAEL_GROTH 0x0439
+
+#define ZORRO_MANUF_M_TECH 0x0502
+#define ZORRO_PROD_MTEC_AT500_1 ZORRO_ID(M_TECH, 0x03, 0)
+
+#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_1 0x06E1
+#define ZORRO_PROD_GVP_IMPACT_SERIES_I ZORRO_ID(GREAT_VALLEY_PRODUCTS_1, 0x08, 0)
+
+#define ZORRO_MANUF_BYTEBOX 0x07DA
+#define ZORRO_PROD_BYTEBOX_A500 ZORRO_ID(BYTEBOX, 0x00, 0)
+
+#define ZORRO_MANUF_DKB_POWER_COMPUTING 0x07DC
+#define ZORRO_PROD_DKB_POWER_COMPUTING_SECUREKEY ZORRO_ID(DKB_POWER_COMPUTING, 0x09, 0)
+#define ZORRO_PROD_DKB_POWER_COMPUTING_DKM_3128 ZORRO_ID(DKB_POWER_COMPUTING, 0x0E, 0)
+#define ZORRO_PROD_DKB_POWER_COMPUTING_RAPID_FIRE ZORRO_ID(DKB_POWER_COMPUTING, 0x0F, 0)
+#define ZORRO_PROD_DKB_POWER_COMPUTING_DKM_1202 ZORRO_ID(DKB_POWER_COMPUTING, 0x10, 0)
+#define ZORRO_PROD_DKB_POWER_COMPUTING_COBRA_VIPER_II_68EC030 ZORRO_ID(DKB_POWER_COMPUTING, 0x12, 0)
+#define ZORRO_PROD_DKB_POWER_COMPUTING_WILDFIRE_060_1 ZORRO_ID(DKB_POWER_COMPUTING, 0x17, 0)
+#define ZORRO_PROD_DKB_POWER_COMPUTING_WILDFIRE_060_2 ZORRO_ID(DKB_POWER_COMPUTING, 0xFF, 0)
+
+#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_2 0x07E1
+#define ZORRO_PROD_GVP_IMPACT_SERIES_I_4K ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x01, 0)
+#define ZORRO_PROD_GVP_IMPACT_SERIES_I_16K_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x02, 0)
+#define ZORRO_PROD_GVP_IMPACT_SERIES_I_16K_3 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x03, 0)
+#define ZORRO_PROD_GVP_IMPACT_3001_IDE_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x08, 0)
+#define ZORRO_PROD_GVP_IMPACT_3001_RAM ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x09, 0)
+#define ZORRO_PROD_GVP_IMPACT_SERIES_II_RAM_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0A, 0)
+#define ZORRO_PROD_GVP_EPC_BASE ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0)
+#define ZORRO_PROD_GVP_GFORCE_040_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x20)
+#define ZORRO_PROD_GVP_GFORCE_040_SCSI_1 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x30)
+#define ZORRO_PROD_GVP_A1291 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x40)
+#define ZORRO_PROD_GVP_COMBO_030_R4 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x60)
+#define ZORRO_PROD_GVP_COMBO_030_R4_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x70)
+#define ZORRO_PROD_GVP_PHONEPAK ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x78)
+#define ZORRO_PROD_GVP_IO_EXTENDER ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0x98)
+#define ZORRO_PROD_GVP_GFORCE_030 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xa0)
+#define ZORRO_PROD_GVP_GFORCE_030_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xb0)
+#define ZORRO_PROD_GVP_A530 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xc0)
+#define ZORRO_PROD_GVP_A530_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xd0)
+#define ZORRO_PROD_GVP_COMBO_030_R3 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xe0)
+#define ZORRO_PROD_GVP_COMBO_030_R3_SCSI ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xf0)
+#define ZORRO_PROD_GVP_SERIES_II ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0B, 0xf8)
+#define ZORRO_PROD_GVP_IMPACT_3001_IDE_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0D, 0)
+/*#define ZORRO_PROD_GVP_A2000_030 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0D, 0)*/
+/*#define ZORRO_PROD_GVP_GFORCE_040_SCSI_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x0D, 0)*/
+#define ZORRO_PROD_GVP_GFORCE_040_060 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x16, 0)
+#define ZORRO_PROD_GVP_IMPACT_VISION_24 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0x20, 0)
+#define ZORRO_PROD_GVP_GFORCE_040_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_2, 0xFF, 0)
+
+#define ZORRO_MANUF_CALIFORNIA_ACCESS_SYNERGY 0x07E5
+#define ZORRO_PROD_CALIFORNIA_ACCESS_SYNERGY_MALIBU ZORRO_ID(CALIFORNIA_ACCESS_SYNERGY, 0x01, 0)
+
+#define ZORRO_MANUF_XETEC 0x07E6
+#define ZORRO_PROD_XETEC_FASTCARD ZORRO_ID(XETEC, 0x01, 0)
+#define ZORRO_PROD_XETEC_FASTCARD_RAM ZORRO_ID(XETEC, 0x02, 0)
+#define ZORRO_PROD_XETEC_FASTCARD_PLUS ZORRO_ID(XETEC, 0x03, 0)
+
+#define ZORRO_MANUF_PROGRESSIVE_PERIPHERALS_AND_SYSTEMS 0x07EA
+#define ZORRO_PROD_PPS_MERCURY ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x00, 0)
+#define ZORRO_PROD_PPS_A3000_68040 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x01, 0)
+#define ZORRO_PROD_PPS_A2000_68040 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x69, 0)
+#define ZORRO_PROD_PPS_ZEUS ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0x96, 0)
+#define ZORRO_PROD_PPS_A500_68040 ZORRO_ID(PROGRESSIVE_PERIPHERALS_AND_SYSTEMS, 0xBB, 0)
+
+#define ZORRO_MANUF_XEBEC 0x07EC
+
+#define ZORRO_MANUF_SPIRIT_TECHNOLOGY 0x07F2
+#define ZORRO_PROD_SPIRIT_TECHNOLOGY_INSIDER_IN1000 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x01, 0)
+#define ZORRO_PROD_SPIRIT_TECHNOLOGY_INSIDER_IN500 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x02, 0)
+#define ZORRO_PROD_SPIRIT_TECHNOLOGY_SIN500 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x03, 0)
+#define ZORRO_PROD_SPIRIT_TECHNOLOGY_HDA_506 ZORRO_ID(SPIRIT_TECHNOLOGY, 0x04, 0)
+#define ZORRO_PROD_SPIRIT_TECHNOLOGY_AX_S ZORRO_ID(SPIRIT_TECHNOLOGY, 0x05, 0)
+#define ZORRO_PROD_SPIRIT_TECHNOLOGY_OCTABYTE ZORRO_ID(SPIRIT_TECHNOLOGY, 0x06, 0)
+#define ZORRO_PROD_SPIRIT_TECHNOLOGY_INMATE ZORRO_ID(SPIRIT_TECHNOLOGY, 0x08, 0)
+
+#define ZORRO_MANUF_SPIRIT_TECHNOLOGY_2 0x07F3
+
+#define ZORRO_MANUF_BSC_ALFADATA_1 0x07FE
+#define ZORRO_PROD_BSC_ALF_3_1 ZORRO_ID(BSC_ALFADATA_1, 0x03, 0)
+
+#define ZORRO_MANUF_BSC_ALFADATA_2 0x0801
+#define ZORRO_PROD_BSC_ALF_2_1 ZORRO_ID(BSC_ALFADATA_2, 0x01, 0)
+#define ZORRO_PROD_BSC_ALF_2_2 ZORRO_ID(BSC_ALFADATA_2, 0x02, 0)
+#define ZORRO_PROD_BSC_ALF_3_2 ZORRO_ID(BSC_ALFADATA_2, 0x03, 0)
+
+#define ZORRO_MANUF_CARDCO_2 0x0802
+#define ZORRO_PROD_CARDCO_KRONOS_2000_2 ZORRO_ID(CARDCO_2, 0x04, 0)
+#define ZORRO_PROD_CARDCO_A1000_2 ZORRO_ID(CARDCO_2, 0x0C, 0)
+
+#define ZORRO_MANUF_JOCHHEIM 0x0804
+#define ZORRO_PROD_JOCHHEIM_RAM ZORRO_ID(JOCHHEIM, 0x01, 0)
+
+#define ZORRO_MANUF_CHECKPOINT_TECHNOLOGIES 0x0807
+#define ZORRO_PROD_CHECKPOINT_TECHNOLOGIES_SERIAL_SOLUTION ZORRO_ID(CHECKPOINT_TECHNOLOGIES, 0x00, 0)
+
+#define ZORRO_MANUF_EDOTRONIK 0x0810
+#define ZORRO_PROD_EDOTRONIK_IEEE_488 ZORRO_ID(EDOTRONIK, 0x01, 0)
+#define ZORRO_PROD_EDOTRONIK_8032 ZORRO_ID(EDOTRONIK, 0x02, 0)
+#define ZORRO_PROD_EDOTRONIK_MULTISERIAL ZORRO_ID(EDOTRONIK, 0x03, 0)
+#define ZORRO_PROD_EDOTRONIK_VIDEODIGITIZER ZORRO_ID(EDOTRONIK, 0x04, 0)
+#define ZORRO_PROD_EDOTRONIK_PARALLEL_IO ZORRO_ID(EDOTRONIK, 0x05, 0)
+#define ZORRO_PROD_EDOTRONIK_PIC_PROTOYPING ZORRO_ID(EDOTRONIK, 0x06, 0)
+#define ZORRO_PROD_EDOTRONIK_ADC ZORRO_ID(EDOTRONIK, 0x07, 0)
+#define ZORRO_PROD_EDOTRONIK_VME ZORRO_ID(EDOTRONIK, 0x08, 0)
+#define ZORRO_PROD_EDOTRONIK_DSP96000 ZORRO_ID(EDOTRONIK, 0x09, 0)
+
+#define ZORRO_MANUF_NES_INC 0x0813
+#define ZORRO_PROD_NES_INC_RAM ZORRO_ID(NES_INC, 0x00, 0)
+
+#define ZORRO_MANUF_ICD 0x0817
+#define ZORRO_PROD_ICD_ADVANTAGE_2000_SCSI ZORRO_ID(ICD, 0x01, 0)
+#define ZORRO_PROD_ICD_ADVANTAGE_IDE ZORRO_ID(ICD, 0x03, 0)
+#define ZORRO_PROD_ICD_ADVANTAGE_2080_RAM ZORRO_ID(ICD, 0x04, 0)
+
+#define ZORRO_MANUF_KUPKE_2 0x0819
+#define ZORRO_PROD_KUPKE_OMTI ZORRO_ID(KUPKE_2, 0x01, 0)
+#define ZORRO_PROD_KUPKE_SCSI_II ZORRO_ID(KUPKE_2, 0x02, 0)
+#define ZORRO_PROD_KUPKE_GOLEM_BOX ZORRO_ID(KUPKE_2, 0x03, 0)
+#define ZORRO_PROD_KUPKE_030_882 ZORRO_ID(KUPKE_2, 0x04, 0)
+#define ZORRO_PROD_KUPKE_SCSI_AT ZORRO_ID(KUPKE_2, 0x05, 0)
+
+#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_3 0x081D
+#define ZORRO_PROD_GVP_A2000_RAM8 ZORRO_ID(GREAT_VALLEY_PRODUCTS_3, 0x09, 0)
+#define ZORRO_PROD_GVP_IMPACT_SERIES_II_RAM_2 ZORRO_ID(GREAT_VALLEY_PRODUCTS_3, 0x0A, 0)
+
+#define ZORRO_MANUF_INTERWORKS_NETWORK 0x081E
+
+#define ZORRO_MANUF_HARDITAL_SYNTHESIS 0x0820
+#define ZORRO_PROD_HARDITAL_SYNTHESIS_TQM_68030_68882 ZORRO_ID(HARDITAL_SYNTHESIS, 0x14, 0)
+
+#define ZORRO_MANUF_APPLIED_ENGINEERING 0x0828
+#define ZORRO_PROD_APPLIED_ENGINEERING_DL2000 ZORRO_ID(APPLIED_ENGINEERING, 0x10, 0)
+#define ZORRO_PROD_APPLIED_ENGINEERING_RAM_WORKS ZORRO_ID(APPLIED_ENGINEERING, 0xE0, 0)
+
+#define ZORRO_MANUF_BSC_ALFADATA_3 0x082C
+#define ZORRO_PROD_BSC_OKTAGON_2008 ZORRO_ID(BSC_ALFADATA_3, 0x05, 0)
+#define ZORRO_PROD_BSC_TANDEM_AT_2008_508 ZORRO_ID(BSC_ALFADATA_3, 0x06, 0)
+#define ZORRO_PROD_BSC_ALFA_RAM_1200 ZORRO_ID(BSC_ALFADATA_3, 0x07, 0)
+#define ZORRO_PROD_BSC_OKTAGON_2008_RAM ZORRO_ID(BSC_ALFADATA_3, 0x08, 0)
+#define ZORRO_PROD_BSC_MULTIFACE_I ZORRO_ID(BSC_ALFADATA_3, 0x10, 0)
+#define ZORRO_PROD_BSC_MULTIFACE_II ZORRO_ID(BSC_ALFADATA_3, 0x11, 0)
+#define ZORRO_PROD_BSC_MULTIFACE_III ZORRO_ID(BSC_ALFADATA_3, 0x12, 0)
+#define ZORRO_PROD_BSC_FRAMEMASTER_II ZORRO_ID(BSC_ALFADATA_3, 0x20, 0)
+#define ZORRO_PROD_BSC_GRAFFITI_RAM ZORRO_ID(BSC_ALFADATA_3, 0x21, 0)
+#define ZORRO_PROD_BSC_GRAFFITI_REG ZORRO_ID(BSC_ALFADATA_3, 0x22, 0)
+#define ZORRO_PROD_BSC_ISDN_MASTERCARD ZORRO_ID(BSC_ALFADATA_3, 0x40, 0)
+#define ZORRO_PROD_BSC_ISDN_MASTERCARD_II ZORRO_ID(BSC_ALFADATA_3, 0x41, 0)
+
+#define ZORRO_MANUF_PHOENIX 0x0835
+#define ZORRO_PROD_PHOENIX_ST506 ZORRO_ID(PHOENIX, 0x21, 0)
+#define ZORRO_PROD_PHOENIX_SCSI ZORRO_ID(PHOENIX, 0x22, 0)
+#define ZORRO_PROD_PHOENIX_RAM ZORRO_ID(PHOENIX, 0xBE, 0)
+
+#define ZORRO_MANUF_ADVANCED_STORAGE_SYSTEMS 0x0836
+#define ZORRO_PROD_ADVANCED_STORAGE_SYSTEMS_NEXUS ZORRO_ID(ADVANCED_STORAGE_SYSTEMS, 0x01, 0)
+#define ZORRO_PROD_ADVANCED_STORAGE_SYSTEMS_NEXUS_RAM ZORRO_ID(ADVANCED_STORAGE_SYSTEMS, 0x08, 0)
+
+#define ZORRO_MANUF_IMPULSE 0x0838
+#define ZORRO_PROD_IMPULSE_FIRECRACKER_24 ZORRO_ID(IMPULSE, 0x00, 0)
+
+#define ZORRO_MANUF_IVS 0x0840
+#define ZORRO_PROD_IVS_GRANDSLAM_PIC_2 ZORRO_ID(IVS, 0x02, 0)
+#define ZORRO_PROD_IVS_GRANDSLAM_PIC_1 ZORRO_ID(IVS, 0x04, 0)
+#define ZORRO_PROD_IVS_OVERDRIVE ZORRO_ID(IVS, 0x10, 0)
+#define ZORRO_PROD_IVS_TRUMPCARD_CLASSIC ZORRO_ID(IVS, 0x30, 0)
+#define ZORRO_PROD_IVS_TRUMPCARD_PRO_GRANDSLAM ZORRO_ID(IVS, 0x34, 0)
+#define ZORRO_PROD_IVS_META_4 ZORRO_ID(IVS, 0x40, 0)
+#define ZORRO_PROD_IVS_WAVETOOLS ZORRO_ID(IVS, 0xBF, 0)
+#define ZORRO_PROD_IVS_VECTOR_1 ZORRO_ID(IVS, 0xF3, 0)
+#define ZORRO_PROD_IVS_VECTOR_2 ZORRO_ID(IVS, 0xF4, 0)
+
+#define ZORRO_MANUF_VECTOR_1 0x0841
+#define ZORRO_PROD_VECTOR_CONNECTION_1 ZORRO_ID(VECTOR_1, 0xE3, 0)
+
+#define ZORRO_MANUF_XPERT_PRODEV 0x0845
+#define ZORRO_PROD_XPERT_PRODEV_VISIONA_RAM ZORRO_ID(XPERT_PRODEV, 0x01, 0)
+#define ZORRO_PROD_XPERT_PRODEV_VISIONA_REG ZORRO_ID(XPERT_PRODEV, 0x02, 0)
+#define ZORRO_PROD_XPERT_PRODEV_MERLIN_RAM ZORRO_ID(XPERT_PRODEV, 0x03, 0)
+#define ZORRO_PROD_XPERT_PRODEV_MERLIN_REG_1 ZORRO_ID(XPERT_PRODEV, 0x04, 0)
+#define ZORRO_PROD_XPERT_PRODEV_MERLIN_REG_2 ZORRO_ID(XPERT_PRODEV, 0xC9, 0)
+
+#define ZORRO_MANUF_HYDRA_SYSTEMS 0x0849
+#define ZORRO_PROD_HYDRA_SYSTEMS_AMIGANET ZORRO_ID(HYDRA_SYSTEMS, 0x01, 0)
+
+#define ZORRO_MANUF_SUNRIZE_INDUSTRIES 0x084F
+#define ZORRO_PROD_SUNRIZE_INDUSTRIES_AD1012 ZORRO_ID(SUNRIZE_INDUSTRIES, 0x01, 0)
+#define ZORRO_PROD_SUNRIZE_INDUSTRIES_AD516 ZORRO_ID(SUNRIZE_INDUSTRIES, 0x02, 0)
+#define ZORRO_PROD_SUNRIZE_INDUSTRIES_DD512 ZORRO_ID(SUNRIZE_INDUSTRIES, 0x03, 0)
+
+#define ZORRO_MANUF_TRICERATOPS 0x0850
+#define ZORRO_PROD_TRICERATOPS_MULTI_IO ZORRO_ID(TRICERATOPS, 0x01, 0)
+
+#define ZORRO_MANUF_APPLIED_MAGIC 0x0851
+#define ZORRO_PROD_APPLIED_MAGIC_DMI_RESOLVER ZORRO_ID(APPLIED_MAGIC, 0x01, 0)
+#define ZORRO_PROD_APPLIED_MAGIC_DIGITAL_BROADCASTER ZORRO_ID(APPLIED_MAGIC, 0x06, 0)
+
+#define ZORRO_MANUF_GFX_BASE 0x085E
+#define ZORRO_PROD_GFX_BASE_GDA_1_VRAM ZORRO_ID(GFX_BASE, 0x00, 0)
+#define ZORRO_PROD_GFX_BASE_GDA_1 ZORRO_ID(GFX_BASE, 0x01, 0)
+
+#define ZORRO_MANUF_ROCTEC 0x0860
+#define ZORRO_PROD_ROCTEC_RH_800C ZORRO_ID(ROCTEC, 0x01, 0)
+#define ZORRO_PROD_ROCTEC_RH_800C_RAM ZORRO_ID(ROCTEC, 0x01, 0)
+
+#define ZORRO_MANUF_KATO 0x0861
+#define ZORRO_PROD_KATO_MELODY ZORRO_ID(KATO, 0x80, 0)
+/* ID clash!! */
+#define ZORRO_MANUF_HELFRICH_1 0x0861
+#define ZORRO_PROD_HELFRICH_RAINBOW_II ZORRO_ID(HELFRICH_1, 0x20, 0)
+#define ZORRO_PROD_HELFRICH_RAINBOW_III ZORRO_ID(HELFRICH_1, 0x21, 0)
+
+#define ZORRO_MANUF_ATLANTIS 0x0862
+
+#define ZORRO_MANUF_PROTAR 0x0864
+
+#define ZORRO_MANUF_ACS 0x0865
+
+#define ZORRO_MANUF_SOFTWARE_RESULTS_ENTERPRISES 0x0866
+#define ZORRO_PROD_SOFTWARE_RESULTS_ENTERPRISES_GOLDEN_GATE_2_BUS_PLUS ZORRO_ID(SOFTWARE_RESULTS_ENTERPRISES, 0x01, 0)
+
+#define ZORRO_MANUF_MASOBOSHI 0x086D
+#define ZORRO_PROD_MASOBOSHI_MASTER_CARD_SC201 ZORRO_ID(MASOBOSHI, 0x03, 0)
+#define ZORRO_PROD_MASOBOSHI_MASTER_CARD_MC702 ZORRO_ID(MASOBOSHI, 0x04, 0)
+#define ZORRO_PROD_MASOBOSHI_MVD_819 ZORRO_ID(MASOBOSHI, 0x07, 0)
+
+#define ZORRO_MANUF_MAINHATTAN_DATA 0x086F
+#define ZORRO_PROD_MAINHATTAN_DATA_IDE ZORRO_ID(MAINHATTAN_DATA, 0x01, 0)
+
+#define ZORRO_MANUF_VILLAGE_TRONIC 0x0877
+#define ZORRO_PROD_VILLAGE_TRONIC_DOMINO_RAM ZORRO_ID(VILLAGE_TRONIC, 0x01, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_DOMINO_REG ZORRO_ID(VILLAGE_TRONIC, 0x02, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_DOMINO_16M_PROTOTYPE ZORRO_ID(VILLAGE_TRONIC, 0x03, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_II_II_PLUS_RAM ZORRO_ID(VILLAGE_TRONIC, 0x0B, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_II_II_PLUS_REG ZORRO_ID(VILLAGE_TRONIC, 0x0C, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_II_II_PLUS_SEGMENTED_MODE ZORRO_ID(VILLAGE_TRONIC, 0x0D, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z2_RAM1 ZORRO_ID(VILLAGE_TRONIC, 0x15, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z2_RAM2 ZORRO_ID(VILLAGE_TRONIC, 0x16, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z2_REG ZORRO_ID(VILLAGE_TRONIC, 0x17, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_PICASSO_IV_Z3 ZORRO_ID(VILLAGE_TRONIC, 0x18, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_ARIADNE ZORRO_ID(VILLAGE_TRONIC, 0xC9, 0)
+#define ZORRO_PROD_VILLAGE_TRONIC_ARIADNE2 ZORRO_ID(VILLAGE_TRONIC, 0xCA, 0)
+
+#define ZORRO_MANUF_UTILITIES_UNLIMITED 0x087B
+#define ZORRO_PROD_UTILITIES_UNLIMITED_EMPLANT_DELUXE ZORRO_ID(UTILITIES_UNLIMITED, 0x15, 0)
+#define ZORRO_PROD_UTILITIES_UNLIMITED_EMPLANT_DELUXE2 ZORRO_ID(UTILITIES_UNLIMITED, 0x20, 0)
+
+#define ZORRO_MANUF_AMITRIX 0x0880
+#define ZORRO_PROD_AMITRIX_MULTI_IO ZORRO_ID(AMITRIX, 0x01, 0)
+#define ZORRO_PROD_AMITRIX_CD_RAM ZORRO_ID(AMITRIX, 0x02, 0)
+
+#define ZORRO_MANUF_ARMAX 0x0885
+#define ZORRO_PROD_ARMAX_OMNIBUS ZORRO_ID(ARMAX, 0x00, 0)
+
+#define ZORRO_MANUF_ZEUS 0x088D
+#define ZORRO_PROD_ZEUS_SPIDER ZORRO_ID(ZEUS, 0x04, 0)
+
+#define ZORRO_MANUF_NEWTEK 0x088F
+#define ZORRO_PROD_NEWTEK_VIDEOTOASTER ZORRO_ID(NEWTEK, 0x00, 0)
+
+#define ZORRO_MANUF_M_TECH_GERMANY 0x0890
+#define ZORRO_PROD_MTEC_AT500_2 ZORRO_ID(M_TECH_GERMANY, 0x01, 0)
+#define ZORRO_PROD_MTEC_68030 ZORRO_ID(M_TECH_GERMANY, 0x03, 0)
+#define ZORRO_PROD_MTEC_68020I ZORRO_ID(M_TECH_GERMANY, 0x06, 0)
+#define ZORRO_PROD_MTEC_A1200_T68030_RTC ZORRO_ID(M_TECH_GERMANY, 0x20, 0)
+#define ZORRO_PROD_MTEC_VIPER_MK_V_E_MATRIX_530 ZORRO_ID(M_TECH_GERMANY, 0x21, 0)
+#define ZORRO_PROD_MTEC_8_MB_RAM ZORRO_ID(M_TECH_GERMANY, 0x22, 0)
+#define ZORRO_PROD_MTEC_VIPER_MK_V_E_MATRIX_530_SCSI_IDE ZORRO_ID(M_TECH_GERMANY, 0x24, 0)
+
+#define ZORRO_MANUF_GREAT_VALLEY_PRODUCTS_4 0x0891
+#define ZORRO_PROD_GVP_EGS_28_24_SPECTRUM_RAM ZORRO_ID(GREAT_VALLEY_PRODUCTS_4, 0x01, 0)
+#define ZORRO_PROD_GVP_EGS_28_24_SPECTRUM_REG ZORRO_ID(GREAT_VALLEY_PRODUCTS_4, 0x02, 0)
+
+#define ZORRO_MANUF_APOLLO_1 0x0892
+#define ZORRO_PROD_APOLLO_A1200 ZORRO_ID(APOLLO_1, 0x01, 0)
+
+#define ZORRO_MANUF_HELFRICH_2 0x0893
+#define ZORRO_PROD_HELFRICH_PICCOLO_RAM ZORRO_ID(HELFRICH_2, 0x05, 0)
+#define ZORRO_PROD_HELFRICH_PICCOLO_REG ZORRO_ID(HELFRICH_2, 0x06, 0)
+#define ZORRO_PROD_HELFRICH_PEGGY_PLUS_MPEG ZORRO_ID(HELFRICH_2, 0x07, 0)
+#define ZORRO_PROD_HELFRICH_VIDEOCRUNCHER ZORRO_ID(HELFRICH_2, 0x08, 0)
+#define ZORRO_PROD_HELFRICH_SD64_RAM ZORRO_ID(HELFRICH_2, 0x0A, 0)
+#define ZORRO_PROD_HELFRICH_SD64_REG ZORRO_ID(HELFRICH_2, 0x0B, 0)
+
+#define ZORRO_MANUF_MACROSYSTEMS_USA 0x089B
+#define ZORRO_PROD_MACROSYSTEMS_WARP_ENGINE_40xx ZORRO_ID(MACROSYSTEMS_USA, 0x13, 0)
+
+#define ZORRO_MANUF_ELBOX_COMPUTER 0x089E
+#define ZORRO_PROD_ELBOX_COMPUTER_1200_4 ZORRO_ID(ELBOX_COMPUTER, 0x06, 0)
+
+#define ZORRO_MANUF_HARMS_PROFESSIONAL 0x0A00
+#define ZORRO_PROD_HARMS_PROFESSIONAL_030_PLUS ZORRO_ID(HARMS_PROFESSIONAL, 0x10, 0)
+#define ZORRO_PROD_HARMS_PROFESSIONAL_3500 ZORRO_ID(HARMS_PROFESSIONAL, 0xD0, 0)
+
+#define ZORRO_MANUF_MICRONIK 0x0A50
+#define ZORRO_PROD_MICRONIK_RCA_120 ZORRO_ID(MICRONIK, 0x0A, 0)
+
+#define ZORRO_MANUF_MICRONIK2 0x0F0F
+#define ZORRO_PROD_MICRONIK2_Z3I ZORRO_ID(MICRONIK2, 0x01, 0)
+
+#define ZORRO_MANUF_MEGAMICRO 0x1000
+#define ZORRO_PROD_MEGAMICRO_SCRAM_500 ZORRO_ID(MEGAMICRO, 0x03, 0)
+#define ZORRO_PROD_MEGAMICRO_SCRAM_500_RAM ZORRO_ID(MEGAMICRO, 0x04, 0)
+
+#define ZORRO_MANUF_IMTRONICS_2 0x1028
+#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_3 ZORRO_ID(IMTRONICS_2, 0x39, 0)
+#define ZORRO_PROD_IMTRONICS_HURRICANE_2800_4 ZORRO_ID(IMTRONICS_2, 0x57, 0)
+
+/* unofficial ID */
+#define ZORRO_MANUF_INDIVIDUAL_COMPUTERS 0x1212
+#define ZORRO_PROD_INDIVIDUAL_COMPUTERS_BUDDHA ZORRO_ID(INDIVIDUAL_COMPUTERS, 0x00, 0)
+#define ZORRO_PROD_INDIVIDUAL_COMPUTERS_X_SURF ZORRO_ID(INDIVIDUAL_COMPUTERS, 0x17, 0)
+#define ZORRO_PROD_INDIVIDUAL_COMPUTERS_CATWEASEL ZORRO_ID(INDIVIDUAL_COMPUTERS, 0x2A, 0)
+
+#define ZORRO_MANUF_KUPKE_3 0x1248
+#define ZORRO_PROD_KUPKE_GOLEM_HD_3000 ZORRO_ID(KUPKE_3, 0x01, 0)
+
+#define ZORRO_MANUF_ITH 0x1388
+#define ZORRO_PROD_ITH_ISDN_MASTER_II ZORRO_ID(ITH, 0x01, 0)
+
+#define ZORRO_MANUF_VMC 0x1389
+#define ZORRO_PROD_VMC_ISDN_BLASTER_Z2 ZORRO_ID(VMC, 0x01, 0)
+#define ZORRO_PROD_VMC_HYPERCOM_4 ZORRO_ID(VMC, 0x02, 0)
+
+#define ZORRO_MANUF_INFORMATION 0x157C
+#define ZORRO_PROD_INFORMATION_ISDN_ENGINE_I ZORRO_ID(INFORMATION, 0x64, 0)
+
+#define ZORRO_MANUF_VORTEX 0x2017
+#define ZORRO_PROD_VORTEX_GOLDEN_GATE_80386SX ZORRO_ID(VORTEX, 0x07, 0)
+#define ZORRO_PROD_VORTEX_GOLDEN_GATE_RAM ZORRO_ID(VORTEX, 0x08, 0)
+#define ZORRO_PROD_VORTEX_GOLDEN_GATE_80486 ZORRO_ID(VORTEX, 0x09, 0)
+
+#define ZORRO_MANUF_EXPANSION_SYSTEMS 0x2062
+#define ZORRO_PROD_EXPANSION_SYSTEMS_DATAFLYER_4000SX ZORRO_ID(EXPANSION_SYSTEMS, 0x01, 0)
+#define ZORRO_PROD_EXPANSION_SYSTEMS_DATAFLYER_4000SX_RAM ZORRO_ID(EXPANSION_SYSTEMS, 0x02, 0)
+
+#define ZORRO_MANUF_READYSOFT 0x2100
+#define ZORRO_PROD_READYSOFT_AMAX_II_IV ZORRO_ID(READYSOFT, 0x01, 0)
+
+#define ZORRO_MANUF_PHASE5 0x2140
+#define ZORRO_PROD_PHASE5_BLIZZARD_RAM ZORRO_ID(PHASE5, 0x01, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD ZORRO_ID(PHASE5, 0x02, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD_1220_IV ZORRO_ID(PHASE5, 0x06, 0)
+#define ZORRO_PROD_PHASE5_FASTLANE_Z3_RAM ZORRO_ID(PHASE5, 0x0A, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD_1230_II_FASTLANE_Z3_CYBERSCSI_CYBERSTORM060 ZORRO_ID(PHASE5, 0x0B, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD_1220_CYBERSTORM ZORRO_ID(PHASE5, 0x0C, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD_1230 ZORRO_ID(PHASE5, 0x0D, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD_1230_IV_1260 ZORRO_ID(PHASE5, 0x11, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD_2060 ZORRO_ID(PHASE5, 0x18, 0)
+#define ZORRO_PROD_PHASE5_CYBERSTORM_MK_II ZORRO_ID(PHASE5, 0x19, 0)
+#define ZORRO_PROD_PHASE5_CYBERVISION64 ZORRO_ID(PHASE5, 0x22, 0)
+#define ZORRO_PROD_PHASE5_CYBERVISION64_3D_PROTOTYPE ZORRO_ID(PHASE5, 0x32, 0)
+#define ZORRO_PROD_PHASE5_CYBERVISION64_3D ZORRO_ID(PHASE5, 0x43, 0)
+#define ZORRO_PROD_PHASE5_CYBERSTORM_MK_III ZORRO_ID(PHASE5, 0x64, 0)
+#define ZORRO_PROD_PHASE5_BLIZZARD_603E_PLUS ZORRO_ID(PHASE5, 0x6e, 0)
+
+#define ZORRO_MANUF_DPS 0x2169
+#define ZORRO_PROD_DPS_PERSONAL_ANIMATION_RECORDER ZORRO_ID(DPS, 0x01, 0)
+
+#define ZORRO_MANUF_APOLLO_2 0x2200
+#define ZORRO_PROD_APOLLO_A620_68020_1 ZORRO_ID(APOLLO_2, 0x00, 0)
+#define ZORRO_PROD_APOLLO_A620_68020_2 ZORRO_ID(APOLLO_2, 0x01, 0)
+
+#define ZORRO_MANUF_APOLLO_3 0x2222
+#define ZORRO_PROD_APOLLO_AT_APOLLO ZORRO_ID(APOLLO_3, 0x22, 0)
+#define ZORRO_PROD_APOLLO_1230_1240_1260_2030_4040_4060 ZORRO_ID(APOLLO_3, 0x23, 0)
+
+#define ZORRO_MANUF_PETSOFF_LP 0x38A5
+#define ZORRO_PROD_PETSOFF_LP_DELFINA ZORRO_ID(PETSOFF_LP, 0x00, 0)
+#define ZORRO_PROD_PETSOFF_LP_DELFINA_LITE ZORRO_ID(PETSOFF_LP, 0x01, 0)
+
+#define ZORRO_MANUF_UWE_GERLACH 0x3FF7
+#define ZORRO_PROD_UWE_GERLACH_RAM_ROM ZORRO_ID(UWE_GERLACH, 0xd4, 0)
+
+#define ZORRO_MANUF_ACT 0x4231
+#define ZORRO_PROD_ACT_PRELUDE ZORRO_ID(ACT, 0x01, 0)
+
+#define ZORRO_MANUF_MACROSYSTEMS_GERMANY 0x4754
+#define ZORRO_PROD_MACROSYSTEMS_MAESTRO ZORRO_ID(MACROSYSTEMS_GERMANY, 0x03, 0)
+#define ZORRO_PROD_MACROSYSTEMS_VLAB ZORRO_ID(MACROSYSTEMS_GERMANY, 0x04, 0)
+#define ZORRO_PROD_MACROSYSTEMS_MAESTRO_PRO ZORRO_ID(MACROSYSTEMS_GERMANY, 0x05, 0)
+#define ZORRO_PROD_MACROSYSTEMS_RETINA ZORRO_ID(MACROSYSTEMS_GERMANY, 0x06, 0)
+#define ZORRO_PROD_MACROSYSTEMS_MULTI_EVOLUTION ZORRO_ID(MACROSYSTEMS_GERMANY, 0x08, 0)
+#define ZORRO_PROD_MACROSYSTEMS_TOCCATA ZORRO_ID(MACROSYSTEMS_GERMANY, 0x0C, 0)
+#define ZORRO_PROD_MACROSYSTEMS_RETINA_Z3 ZORRO_ID(MACROSYSTEMS_GERMANY, 0x10, 0)
+#define ZORRO_PROD_MACROSYSTEMS_VLAB_MOTION ZORRO_ID(MACROSYSTEMS_GERMANY, 0x12, 0)
+#define ZORRO_PROD_MACROSYSTEMS_ALTAIS ZORRO_ID(MACROSYSTEMS_GERMANY, 0x13, 0)
+#define ZORRO_PROD_MACROSYSTEMS_FALCON_040 ZORRO_ID(MACROSYSTEMS_GERMANY, 0xFD, 0)
+
+#define ZORRO_MANUF_COMBITEC 0x6766
+
+#define ZORRO_MANUF_SKI_PERIPHERALS 0x8000
+#define ZORRO_PROD_SKI_PERIPHERALS_MAST_FIREBALL ZORRO_ID(SKI_PERIPHERALS, 0x08, 0)
+#define ZORRO_PROD_SKI_PERIPHERALS_SCSI_DUAL_SERIAL ZORRO_ID(SKI_PERIPHERALS, 0x80, 0)
+
+#define ZORRO_MANUF_REIS_WARE_2 0xA9AD
+#define ZORRO_PROD_REIS_WARE_SCAN_KING ZORRO_ID(REIS_WARE_2, 0x11, 0)
+
+#define ZORRO_MANUF_CAMERON 0xAA01
+#define ZORRO_PROD_CAMERON_PERSONAL_A4 ZORRO_ID(CAMERON, 0x10, 0)
+
+#define ZORRO_MANUF_REIS_WARE 0xAA11
+#define ZORRO_PROD_REIS_WARE_HANDYSCANNER ZORRO_ID(REIS_WARE, 0x11, 0)
+
+#define ZORRO_MANUF_PHOENIX_2 0xB5A8
+#define ZORRO_PROD_PHOENIX_ST506_2 ZORRO_ID(PHOENIX_2, 0x21, 0)
+#define ZORRO_PROD_PHOENIX_SCSI_2 ZORRO_ID(PHOENIX_2, 0x22, 0)
+#define ZORRO_PROD_PHOENIX_RAM_2 ZORRO_ID(PHOENIX_2, 0xBE, 0)
+
+#define ZORRO_MANUF_COMBITEC_2 0xC008
+#define ZORRO_PROD_COMBITEC_HD ZORRO_ID(COMBITEC_2, 0x2A, 0)
+#define ZORRO_PROD_COMBITEC_SRAM ZORRO_ID(COMBITEC_2, 0x2B, 0)
+
+
+ /*
+ * Test and illegal Manufacturer IDs.
+ */
+
+#define ZORRO_MANUF_HACKER 0x07DB
+#define ZORRO_PROD_GENERAL_PROTOTYPE ZORRO_ID(HACKER, 0x00, 0)
+#define ZORRO_PROD_HACKER_SCSI ZORRO_ID(HACKER, 0x01, 0)
+#define ZORRO_PROD_RESOURCE_MANAGEMENT_FORCE_QUICKNET_QN2000 ZORRO_ID(HACKER, 0x02, 0)
+#define ZORRO_PROD_VECTOR_CONNECTION_2 ZORRO_ID(HACKER, 0xE0, 0)
+#define ZORRO_PROD_VECTOR_CONNECTION_3 ZORRO_ID(HACKER, 0xE1, 0)
+#define ZORRO_PROD_VECTOR_CONNECTION_4 ZORRO_ID(HACKER, 0xE2, 0)
+#define ZORRO_PROD_VECTOR_CONNECTION_5 ZORRO_ID(HACKER, 0xE3, 0)
* @extra_args == Operation-specific extra arguments (NULL if none).
*/
-/* ia64, x86: Callback for event delivery. */
+/* x86: Callback for event delivery. */
#define CALLBACKTYPE_event 0
/* x86: Failsafe callback when guest state cannot be restored by Xen. */
#define XEN_IO_PROTO_ABI_X86_32 "x86_32-abi"
#define XEN_IO_PROTO_ABI_X86_64 "x86_64-abi"
-#define XEN_IO_PROTO_ABI_IA64 "ia64-abi"
#define XEN_IO_PROTO_ABI_POWERPC64 "powerpc64-abi"
#define XEN_IO_PROTO_ABI_ARM "arm-abi"
# define XEN_IO_PROTO_ABI_NATIVE XEN_IO_PROTO_ABI_X86_32
#elif defined(__x86_64__)
# define XEN_IO_PROTO_ABI_NATIVE XEN_IO_PROTO_ABI_X86_64
-#elif defined(__ia64__)
-# define XEN_IO_PROTO_ABI_NATIVE XEN_IO_PROTO_ABI_IA64
#elif defined(__powerpc64__)
# define XEN_IO_PROTO_ABI_NATIVE XEN_IO_PROTO_ABI_POWERPC64
#elif defined(__arm__) || defined(__aarch64__)
dynticks subsystem by forcing the context tracking on all
CPUs in the system.
- Say Y only if you're working on the developpement of an
+ Say Y only if you're working on the development of an
architecture backend for the context tracking.
Say N otherwise, this option brings an overhead that you
/*
* Repeat the user_enter() check here because some archs may be calling
* this from asm and if no CPU needs context tracking, they shouldn't
- * go further. Repeat the check here until they support the static key
- * check.
+ * go further. Repeat the check here until they support the inline static
+ * key check.
*/
- if (!static_key_false(&context_tracking_enabled))
+ if (!context_tracking_is_enabled())
return;
/*
{
unsigned long flags;
- if (!static_key_false(&context_tracking_enabled))
+ if (!context_tracking_is_enabled())
return;
if (in_interrupt())
__current_set_polling();
}
arch_cpu_idle_exit();
- /*
- * We need to test and propagate the TIF_NEED_RESCHED
- * bit here because we might not have send the
- * reschedule IPI to idle tasks.
- */
- if (tif_need_resched())
- set_preempt_need_resched();
}
+
+ /*
+ * Since we fell out of the loop above, we know
+ * TIF_NEED_RESCHED must be set, propagate it into
+ * PREEMPT_NEED_RESCHED.
+ *
+ * This is required because for polling idle loops we will
+ * not have had an IPI to fold the state for us.
+ */
+ preempt_set_need_resched();
tick_nohz_idle_exit();
schedule_preempt_disabled();
}
#define PERF_FLAG_ALL (PERF_FLAG_FD_NO_GROUP |\
PERF_FLAG_FD_OUTPUT |\
- PERF_FLAG_PID_CGROUP)
+ PERF_FLAG_PID_CGROUP |\
+ PERF_FLAG_FD_CLOEXEC)
/*
* branch priv levels that need permission checks
static int perf_event_period(struct perf_event *event, u64 __user *arg)
{
struct perf_event_context *ctx = event->ctx;
- int ret = 0;
+ int ret = 0, active;
u64 value;
if (!is_sampling_event(event))
event->attr.sample_period = value;
event->hw.sample_period = value;
}
+
+ active = (event->state == PERF_EVENT_STATE_ACTIVE);
+ if (active) {
+ perf_pmu_disable(ctx->pmu);
+ event->pmu->stop(event, PERF_EF_UPDATE);
+ }
+
+ local64_set(&event->hw.period_left, 0);
+
+ if (active) {
+ event->pmu->start(event, PERF_EF_RELOAD);
+ perf_pmu_enable(ctx->pmu);
+ }
+
unlock:
raw_spin_unlock_irq(&ctx->lock);
INIT_LIST_HEAD(&event->event_entry);
INIT_LIST_HEAD(&event->sibling_list);
INIT_LIST_HEAD(&event->rb_entry);
+ INIT_LIST_HEAD(&event->active_entry);
+ INIT_HLIST_NODE(&event->hlist_entry);
+
init_waitqueue_head(&event->waitq);
init_irq_work(&event->pending, perf_pending_event);
int event_fd;
int move_group = 0;
int err;
+ int f_flags = O_RDWR;
/* for future expandability... */
if (flags & ~PERF_FLAG_ALL)
if ((flags & PERF_FLAG_PID_CGROUP) && (pid == -1 || cpu == -1))
return -EINVAL;
- event_fd = get_unused_fd();
+ if (flags & PERF_FLAG_FD_CLOEXEC)
+ f_flags |= O_CLOEXEC;
+
+ event_fd = get_unused_fd_flags(f_flags);
if (event_fd < 0)
return event_fd;
goto err_context;
}
- event_file = anon_inode_getfile("[perf_event]", &perf_fops, event, O_RDWR);
+ event_file = anon_inode_getfile("[perf_event]", &perf_fops, event,
+ f_flags);
if (IS_ERR(event_file)) {
err = PTR_ERR(event_file);
goto err_context;
*
* kernel user
*
- * READ ->data_tail READ ->data_head
- * smp_mb() (A) smp_rmb() (C)
- * WRITE $data READ $data
- * smp_wmb() (B) smp_mb() (D)
- * STORE ->data_head WRITE ->data_tail
+ * if (LOAD ->data_tail) { LOAD ->data_head
+ * (A) smp_rmb() (C)
+ * STORE $data LOAD $data
+ * smp_wmb() (B) smp_mb() (D)
+ * STORE ->data_head STORE ->data_tail
+ * }
*
* Where A pairs with D, and B pairs with C.
*
- * I don't think A needs to be a full barrier because we won't in fact
- * write data until we see the store from userspace. So we simply don't
- * issue the data WRITE until we observe it. Be conservative for now.
+ * In our case (A) is a control dependency that separates the load of
+ * the ->data_tail and the stores of $data. In case ->data_tail
+ * indicates there is no room in the buffer to store $data we do not.
*
- * OTOH, D needs to be a full barrier since it separates the data READ
+ * D needs to be a full barrier since it separates the data READ
* from the tail WRITE.
*
* For B a WMB is sufficient since it separates two WRITEs, and for C
*
* See perf_output_begin().
*/
- smp_wmb();
+ smp_wmb(); /* B, matches C */
rb->user_page->data_head = head;
/*
if (!rb->overwrite &&
unlikely(CIRC_SPACE(head, tail, perf_data_size(rb)) < size))
goto fail;
+
+ /*
+ * The above forms a control dependency barrier separating the
+ * @tail load above from the data stores below. Since the @tail
+ * load is required to compute the branch to fail below.
+ *
+ * A, matches D; the full memory barrier userspace SHOULD issue
+ * after reading the data and before storing the new tail
+ * position.
+ *
+ * See perf_output_put_handle().
+ */
+
head += size;
} while (local_cmpxchg(&rb->head, offset, head) != offset);
/*
- * Separate the userpage->tail read from the data stores below.
- * Matches the MB userspace SHOULD issue after reading the data
- * and before storing the new tail position.
- *
- * See perf_output_put_handle().
+ * We rely on the implied barrier() by local_cmpxchg() to ensure
+ * none of the data stores below can be lifted up by the compiler.
*/
- smp_mb();
if (unlikely(head - local_read(&rb->wakeup) > rb->watermark))
local_add(rb->watermark, &rb->wakeup);
struct inode *inode; /* Also hold a ref to inode */
loff_t offset;
unsigned long flags;
+
+ /*
+ * The generic code assumes that it has two members of unknown type
+ * owned by the arch-specific code:
+ *
+ * insn - copy_insn() saves the original instruction here for
+ * arch_uprobe_analyze_insn().
+ *
+ * ixol - potentially modified instruction to execute out of
+ * line, copied to xol_area by xol_get_insn_slot().
+ */
struct arch_uprobe arch;
};
struct return_instance *next; /* keep as stack */
};
+/*
+ * Execute out of line area: anonymous executable mapping installed
+ * by the probed task to execute the copy of the original instruction
+ * mangled by set_swbp().
+ *
+ * On a breakpoint hit, thread contests for a slot. It frees the
+ * slot after singlestep. Currently a fixed number of slots are
+ * allocated.
+ */
+struct xol_area {
+ wait_queue_head_t wq; /* if all slots are busy */
+ atomic_t slot_count; /* number of in-use slots */
+ unsigned long *bitmap; /* 0 = free slot */
+ struct page *page;
+
+ /*
+ * We keep the vma's vm_start rather than a pointer to the vma
+ * itself. The probed process or a naughty kernel module could make
+ * the vma go away, and we must handle that reasonably gracefully.
+ */
+ unsigned long vaddr; /* Page(s) of instruction slots */
+};
+
/*
* valid_vma: Verify if the specified vma is an executable vma
* Relax restrictions while unregistering: vm_flags might have
int __weak
set_orig_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, unsigned long vaddr)
{
- return uprobe_write_opcode(mm, vaddr, *(uprobe_opcode_t *)auprobe->insn);
+ return uprobe_write_opcode(mm, vaddr, *(uprobe_opcode_t *)&auprobe->insn);
}
static int match_uprobe(struct uprobe *l, struct uprobe *r)
{
struct address_space *mapping = uprobe->inode->i_mapping;
loff_t offs = uprobe->offset;
- void *insn = uprobe->arch.insn;
- int size = MAX_UINSN_BYTES;
+ void *insn = &uprobe->arch.insn;
+ int size = sizeof(uprobe->arch.insn);
int len, err = -EIO;
/* Copy only available bytes, -EIO if nothing was read */
goto out;
ret = -ENOTSUPP;
- if (is_trap_insn((uprobe_opcode_t *)uprobe->arch.insn))
+ if (is_trap_insn((uprobe_opcode_t *)&uprobe->arch.insn))
goto out;
ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
/* Initialize the slot */
copy_to_page(area->page, xol_vaddr,
- uprobe->arch.ixol, sizeof(uprobe->arch.ixol));
+ &uprobe->arch.ixol, sizeof(uprobe->arch.ixol));
/*
* We probably need flush_icache_user_range() but it needs vma.
* This should work on supported architectures too.
static void dup_xol_work(struct callback_head *work)
{
- kfree(work);
-
if (current->flags & PF_EXITING)
return;
- if (!__create_xol_area(current->utask->vaddr))
+ if (!__create_xol_area(current->utask->dup_xol_addr))
uprobe_warn(current, "dup xol area");
}
{
struct uprobe_task *utask = current->utask;
struct mm_struct *mm = current->mm;
- struct callback_head *work;
struct xol_area *area;
t->utask = NULL;
if (mm == t->mm)
return;
- /* TODO: move it into the union in uprobe_task */
- work = kmalloc(sizeof(*work), GFP_KERNEL);
- if (!work)
- return uprobe_warn(t, "dup xol area");
-
- t->utask->vaddr = area->vaddr;
- init_task_work(work, dup_xol_work);
- task_work_add(t, work, true);
+ t->utask->dup_xol_addr = area->vaddr;
+ init_task_work(&t->utask->dup_xol_work, dup_xol_work);
+ task_work_add(t, &t->utask->dup_xol_work, true);
}
/*
{
raw_spin_lock_init(&p->pi_lock);
#ifdef CONFIG_RT_MUTEXES
- plist_head_init(&p->pi_waiters);
+ p->pi_waiters = RB_ROOT;
+ p->pi_waiters_leftmost = NULL;
p->pi_blocked_on = NULL;
+ p->pi_top_task = NULL;
#endif
}
* do not allow it to share a thread group or signal handlers or
* parent with the forking task.
*/
- if (clone_flags & (CLONE_SIGHAND | CLONE_PARENT)) {
+ if (clone_flags & CLONE_SIGHAND) {
if ((clone_flags & (CLONE_NEWUSER | CLONE_NEWPID)) ||
(task_active_pid_ns(current) !=
current->nsproxy->pid_ns_for_children))
#endif
/* Perform scheduler related setup. Assign this task to a CPU. */
- sched_fork(clone_flags, p);
+ retval = sched_fork(clone_flags, p);
+ if (retval)
+ goto bad_fork_cleanup_policy;
retval = perf_event_init_task(p);
if (retval)
p->tgid = p->pid;
}
- p->pdeath_signal = 0;
- p->exit_state = 0;
-
p->nr_dirtied = 0;
p->nr_dirtied_pause = 128 >> (PAGE_SHIFT - 10);
p->dirty_paused_when = 0;
+ p->pdeath_signal = 0;
INIT_LIST_HEAD(&p->thread_group);
p->task_works = NULL;
#include <linux/sched/rt.h>
#include <linux/hugetlb.h>
#include <linux/freezer.h>
+#include <linux/bootmem.h>
#include <asm/futex.h>
#include "locking/rtmutex_common.h"
-int __read_mostly futex_cmpxchg_enabled;
+/*
+ * Basic futex operation and ordering guarantees:
+ *
+ * The waiter reads the futex value in user space and calls
+ * futex_wait(). This function computes the hash bucket and acquires
+ * the hash bucket lock. After that it reads the futex user space value
+ * again and verifies that the data has not changed. If it has not changed
+ * it enqueues itself into the hash bucket, releases the hash bucket lock
+ * and schedules.
+ *
+ * The waker side modifies the user space value of the futex and calls
+ * futex_wake(). This function computes the hash bucket and acquires the
+ * hash bucket lock. Then it looks for waiters on that futex in the hash
+ * bucket and wakes them.
+ *
+ * In futex wake up scenarios where no tasks are blocked on a futex, taking
+ * the hb spinlock can be avoided and simply return. In order for this
+ * optimization to work, ordering guarantees must exist so that the waiter
+ * being added to the list is acknowledged when the list is concurrently being
+ * checked by the waker, avoiding scenarios like the following:
+ *
+ * CPU 0 CPU 1
+ * val = *futex;
+ * sys_futex(WAIT, futex, val);
+ * futex_wait(futex, val);
+ * uval = *futex;
+ * *futex = newval;
+ * sys_futex(WAKE, futex);
+ * futex_wake(futex);
+ * if (queue_empty())
+ * return;
+ * if (uval == val)
+ * lock(hash_bucket(futex));
+ * queue();
+ * unlock(hash_bucket(futex));
+ * schedule();
+ *
+ * This would cause the waiter on CPU 0 to wait forever because it
+ * missed the transition of the user space value from val to newval
+ * and the waker did not find the waiter in the hash bucket queue.
+ *
+ * The correct serialization ensures that a waiter either observes
+ * the changed user space value before blocking or is woken by a
+ * concurrent waker:
+ *
+ * CPU 0 CPU 1
+ * val = *futex;
+ * sys_futex(WAIT, futex, val);
+ * futex_wait(futex, val);
+ *
+ * waiters++;
+ * mb(); (A) <-- paired with -.
+ * |
+ * lock(hash_bucket(futex)); |
+ * |
+ * uval = *futex; |
+ * | *futex = newval;
+ * | sys_futex(WAKE, futex);
+ * | futex_wake(futex);
+ * |
+ * `-------> mb(); (B)
+ * if (uval == val)
+ * queue();
+ * unlock(hash_bucket(futex));
+ * schedule(); if (waiters)
+ * lock(hash_bucket(futex));
+ * wake_waiters(futex);
+ * unlock(hash_bucket(futex));
+ *
+ * Where (A) orders the waiters increment and the futex value read -- this
+ * is guaranteed by the head counter in the hb spinlock; and where (B)
+ * orders the write to futex and the waiters read -- this is done by the
+ * barriers in get_futex_key_refs(), through either ihold or atomic_inc,
+ * depending on the futex type.
+ *
+ * This yields the following case (where X:=waiters, Y:=futex):
+ *
+ * X = Y = 0
+ *
+ * w[X]=1 w[Y]=1
+ * MB MB
+ * r[Y]=y r[X]=x
+ *
+ * Which guarantees that x==0 && y==0 is impossible; which translates back into
+ * the guarantee that we cannot both miss the futex variable change and the
+ * enqueue.
+ */
-#define FUTEX_HASHBITS (CONFIG_BASE_SMALL ? 4 : 8)
+int __read_mostly futex_cmpxchg_enabled;
/*
* Futex flags used to encode options to functions and preserve them across
struct futex_hash_bucket {
spinlock_t lock;
struct plist_head chain;
-};
+} ____cacheline_aligned_in_smp;
-static struct futex_hash_bucket futex_queues[1<<FUTEX_HASHBITS];
+static unsigned long __read_mostly futex_hashsize;
+
+static struct futex_hash_bucket *futex_queues;
+
+static inline void futex_get_mm(union futex_key *key)
+{
+ atomic_inc(&key->private.mm->mm_count);
+ /*
+ * Ensure futex_get_mm() implies a full barrier such that
+ * get_futex_key() implies a full barrier. This is relied upon
+ * as full barrier (B), see the ordering comment above.
+ */
+ smp_mb__after_atomic_inc();
+}
+
+static inline bool hb_waiters_pending(struct futex_hash_bucket *hb)
+{
+#ifdef CONFIG_SMP
+ /*
+ * Tasks trying to enter the critical region are most likely
+ * potential waiters that will be added to the plist. Ensure
+ * that wakers won't miss to-be-slept tasks in the window between
+ * the wait call and the actual plist_add.
+ */
+ if (spin_is_locked(&hb->lock))
+ return true;
+ smp_rmb(); /* Make sure we check the lock state first */
+
+ return !plist_head_empty(&hb->chain);
+#else
+ return true;
+#endif
+}
/*
* We hash on the keys returned from get_futex_key (see below).
u32 hash = jhash2((u32*)&key->both.word,
(sizeof(key->both.word)+sizeof(key->both.ptr))/4,
key->both.offset);
- return &futex_queues[hash & ((1 << FUTEX_HASHBITS)-1)];
+ return &futex_queues[hash & (futex_hashsize - 1)];
}
/*
switch (key->both.offset & (FUT_OFF_INODE|FUT_OFF_MMSHARED)) {
case FUT_OFF_INODE:
- ihold(key->shared.inode);
+ ihold(key->shared.inode); /* implies MB (B) */
break;
case FUT_OFF_MMSHARED:
- atomic_inc(&key->private.mm->mm_count);
+ futex_get_mm(key); /* implies MB (B) */
break;
}
}
if (!fshared) {
key->private.mm = mm;
key->private.address = address;
- get_futex_key_refs(key);
+ get_futex_key_refs(key); /* implies MB (B) */
return 0;
}
key->shared.pgoff = basepage_index(page);
}
- get_futex_key_refs(key);
+ get_futex_key_refs(key); /* implies MB (B) */
out:
unlock_page(page_head);
{
struct futex_pi_state *pi_state = NULL;
struct futex_q *this, *next;
- struct plist_head *head;
struct task_struct *p;
pid_t pid = uval & FUTEX_TID_MASK;
- head = &hb->chain;
-
- plist_for_each_entry_safe(this, next, head, list) {
+ plist_for_each_entry_safe(this, next, &hb->chain, list) {
if (match_futex(&this->key, key)) {
/*
* Another waiter already exists - bump up
{
struct futex_hash_bucket *hb;
struct futex_q *this, *next;
- struct plist_head *head;
union futex_key key = FUTEX_KEY_INIT;
int ret;
goto out;
hb = hash_futex(&key);
+
+ /* Make sure we really have tasks to wakeup */
+ if (!hb_waiters_pending(hb))
+ goto out_put_key;
+
spin_lock(&hb->lock);
- head = &hb->chain;
- plist_for_each_entry_safe(this, next, head, list) {
+ plist_for_each_entry_safe(this, next, &hb->chain, list) {
if (match_futex (&this->key, &key)) {
if (this->pi_state || this->rt_waiter) {
ret = -EINVAL;
}
spin_unlock(&hb->lock);
+out_put_key:
put_futex_key(&key);
out:
return ret;
{
union futex_key key1 = FUTEX_KEY_INIT, key2 = FUTEX_KEY_INIT;
struct futex_hash_bucket *hb1, *hb2;
- struct plist_head *head;
struct futex_q *this, *next;
int ret, op_ret;
goto retry;
}
- head = &hb1->chain;
-
- plist_for_each_entry_safe(this, next, head, list) {
+ plist_for_each_entry_safe(this, next, &hb1->chain, list) {
if (match_futex (&this->key, &key1)) {
if (this->pi_state || this->rt_waiter) {
ret = -EINVAL;
}
if (op_ret > 0) {
- head = &hb2->chain;
-
op_ret = 0;
- plist_for_each_entry_safe(this, next, head, list) {
+ plist_for_each_entry_safe(this, next, &hb2->chain, list) {
if (match_futex (&this->key, &key2)) {
if (this->pi_state || this->rt_waiter) {
ret = -EINVAL;
int drop_count = 0, task_count = 0, ret;
struct futex_pi_state *pi_state = NULL;
struct futex_hash_bucket *hb1, *hb2;
- struct plist_head *head1;
struct futex_q *this, *next;
u32 curval2;
}
}
- head1 = &hb1->chain;
- plist_for_each_entry_safe(this, next, head1, list) {
+ plist_for_each_entry_safe(this, next, &hb1->chain, list) {
if (task_count - nr_wake >= nr_requeue)
break;
hb = hash_futex(&q->key);
q->lock_ptr = &hb->lock;
- spin_lock(&hb->lock);
+ spin_lock(&hb->lock); /* implies MB (A) */
return hb;
}
static inline void
-queue_unlock(struct futex_q *q, struct futex_hash_bucket *hb)
+queue_unlock(struct futex_hash_bucket *hb)
__releases(&hb->lock)
{
spin_unlock(&hb->lock);
ret = get_futex_value_locked(&uval, uaddr);
if (ret) {
- queue_unlock(q, *hb);
+ queue_unlock(*hb);
ret = get_user(uval, uaddr);
if (ret)
}
if (uval != val) {
- queue_unlock(q, *hb);
+ queue_unlock(*hb);
ret = -EWOULDBLOCK;
}
* Task is exiting and we just wait for the
* exit to complete.
*/
- queue_unlock(&q, hb);
+ queue_unlock(hb);
put_futex_key(&q.key);
cond_resched();
goto retry;
goto out_put_key;
out_unlock_put_key:
- queue_unlock(&q, hb);
+ queue_unlock(hb);
out_put_key:
put_futex_key(&q.key);
return ret != -EINTR ? ret : -ERESTARTNOINTR;
uaddr_faulted:
- queue_unlock(&q, hb);
+ queue_unlock(hb);
ret = fault_in_user_writeable(uaddr);
if (ret)
{
struct futex_hash_bucket *hb;
struct futex_q *this, *next;
- struct plist_head *head;
union futex_key key = FUTEX_KEY_INIT;
u32 uval, vpid = task_pid_vnr(current);
int ret;
* Ok, other tasks may need to be woken up - check waiters
* and do the wakeup if necessary:
*/
- head = &hb->chain;
-
- plist_for_each_entry_safe(this, next, head, list) {
+ plist_for_each_entry_safe(this, next, &hb->chain, list) {
if (!match_futex (&this->key, &key))
continue;
ret = wake_futex_pi(uaddr, uval, this);
* code while we sleep on uaddr.
*/
debug_rt_mutex_init_waiter(&rt_waiter);
+ RB_CLEAR_NODE(&rt_waiter.pi_tree_entry);
+ RB_CLEAR_NODE(&rt_waiter.tree_entry);
rt_waiter.task = NULL;
ret = get_futex_key(uaddr2, flags & FLAGS_SHARED, &key2, VERIFY_WRITE);
static int __init futex_init(void)
{
u32 curval;
- int i;
+ unsigned int futex_shift;
+ unsigned long i;
+
+#if CONFIG_BASE_SMALL
+ futex_hashsize = 16;
+#else
+ futex_hashsize = roundup_pow_of_two(256 * num_possible_cpus());
+#endif
+ futex_queues = alloc_large_system_hash("futex", sizeof(*futex_queues),
+ futex_hashsize, 0,
+ futex_hashsize < 256 ? HASH_SMALL : 0,
+ &futex_shift, NULL,
+ futex_hashsize, futex_hashsize);
+ futex_hashsize = 1UL << futex_shift;
/*
* This will fail and we want it. Some arch implementations do
* runtime detection of the futex_atomic_cmpxchg_inatomic()
if (cmpxchg_futex_value_locked(&curval, NULL, 0, 0) == -EFAULT)
futex_cmpxchg_enabled = 1;
- for (i = 0; i < ARRAY_SIZE(futex_queues); i++) {
+ for (i = 0; i < futex_hashsize; i++) {
plist_head_init(&futex_queues[i].chain);
spin_lock_init(&futex_queues[i].lock);
}
#include <linux/sched.h>
#include <linux/sched/sysctl.h>
#include <linux/sched/rt.h>
+#include <linux/sched/deadline.h>
#include <linux/timer.h>
#include <linux/freezer.h>
unsigned long slack;
slack = current->timer_slack_ns;
- if (rt_task(current))
+ if (dl_task(current) || rt_task(current))
slack = 0;
hrtimer_init_on_stack(&t.timer, clockid, mode);
/*
* Is this the address of a static object:
*/
+#ifdef __KERNEL__
static int static_obj(void *obj)
{
unsigned long start = (unsigned long) &_stext,
*/
return is_module_address(addr) || is_module_percpu_address(addr);
}
+#endif
/*
* To make lock name printouts unique, we calculate a unique
}
EXPORT_SYMBOL_GPL(debug_check_no_locks_held);
+#ifdef __KERNEL__
void debug_show_all_locks(void)
{
struct task_struct *g, *p;
read_unlock(&tasklist_lock);
}
EXPORT_SYMBOL_GPL(debug_show_all_locks);
+#endif
/*
* Careful: only use this function if you are sure that
return;
DEBUG_LOCKS_WARN_ON(lock->magic != lock);
- DEBUG_LOCKS_WARN_ON(lock->owner != current);
+
+ if (!lock->owner)
+ DEBUG_LOCKS_WARN_ON(!lock->owner);
+ else
+ DEBUG_LOCKS_WARN_ON(lock->owner != current);
+
DEBUG_LOCKS_WARN_ON(!lock->wait_list.prev && !lock->wait_list.next);
mutex_clear_owner(lock);
}
#include <linux/kallsyms.h>
#include <linux/syscalls.h>
#include <linux/interrupt.h>
-#include <linux/plist.h>
+#include <linux/rbtree.h>
#include <linux/fs.h>
#include <linux/debug_locks.h>
void rt_mutex_debug_task_free(struct task_struct *task)
{
- DEBUG_LOCKS_WARN_ON(!plist_head_empty(&task->pi_waiters));
+ DEBUG_LOCKS_WARN_ON(!RB_EMPTY_ROOT(&task->pi_waiters));
DEBUG_LOCKS_WARN_ON(task->pi_blocked_on);
}
void debug_rt_mutex_init_waiter(struct rt_mutex_waiter *waiter)
{
memset(waiter, 0x11, sizeof(*waiter));
- plist_node_init(&waiter->list_entry, MAX_PRIO);
- plist_node_init(&waiter->pi_list_entry, MAX_PRIO);
waiter->deadlock_task_pid = NULL;
}
void debug_rt_mutex_free_waiter(struct rt_mutex_waiter *waiter)
{
put_pid(waiter->deadlock_task_pid);
- DEBUG_LOCKS_WARN_ON(!plist_node_empty(&waiter->list_entry));
- DEBUG_LOCKS_WARN_ON(!plist_node_empty(&waiter->pi_list_entry));
memset(waiter, 0x22, sizeof(*waiter));
}
#include <linux/export.h>
#include <linux/sched.h>
#include <linux/sched/rt.h>
+#include <linux/sched/deadline.h>
#include <linux/timer.h>
#include "rtmutex_common.h"
}
#endif
+static inline int
+rt_mutex_waiter_less(struct rt_mutex_waiter *left,
+ struct rt_mutex_waiter *right)
+{
+ if (left->prio < right->prio)
+ return 1;
+
+ /*
+ * If both waiters have dl_prio(), we check the deadlines of the
+ * associated tasks.
+ * If left waiter has a dl_prio(), and we didn't return 1 above,
+ * then right waiter has a dl_prio() too.
+ */
+ if (dl_prio(left->prio))
+ return (left->task->dl.deadline < right->task->dl.deadline);
+
+ return 0;
+}
+
+static void
+rt_mutex_enqueue(struct rt_mutex *lock, struct rt_mutex_waiter *waiter)
+{
+ struct rb_node **link = &lock->waiters.rb_node;
+ struct rb_node *parent = NULL;
+ struct rt_mutex_waiter *entry;
+ int leftmost = 1;
+
+ while (*link) {
+ parent = *link;
+ entry = rb_entry(parent, struct rt_mutex_waiter, tree_entry);
+ if (rt_mutex_waiter_less(waiter, entry)) {
+ link = &parent->rb_left;
+ } else {
+ link = &parent->rb_right;
+ leftmost = 0;
+ }
+ }
+
+ if (leftmost)
+ lock->waiters_leftmost = &waiter->tree_entry;
+
+ rb_link_node(&waiter->tree_entry, parent, link);
+ rb_insert_color(&waiter->tree_entry, &lock->waiters);
+}
+
+static void
+rt_mutex_dequeue(struct rt_mutex *lock, struct rt_mutex_waiter *waiter)
+{
+ if (RB_EMPTY_NODE(&waiter->tree_entry))
+ return;
+
+ if (lock->waiters_leftmost == &waiter->tree_entry)
+ lock->waiters_leftmost = rb_next(&waiter->tree_entry);
+
+ rb_erase(&waiter->tree_entry, &lock->waiters);
+ RB_CLEAR_NODE(&waiter->tree_entry);
+}
+
+static void
+rt_mutex_enqueue_pi(struct task_struct *task, struct rt_mutex_waiter *waiter)
+{
+ struct rb_node **link = &task->pi_waiters.rb_node;
+ struct rb_node *parent = NULL;
+ struct rt_mutex_waiter *entry;
+ int leftmost = 1;
+
+ while (*link) {
+ parent = *link;
+ entry = rb_entry(parent, struct rt_mutex_waiter, pi_tree_entry);
+ if (rt_mutex_waiter_less(waiter, entry)) {
+ link = &parent->rb_left;
+ } else {
+ link = &parent->rb_right;
+ leftmost = 0;
+ }
+ }
+
+ if (leftmost)
+ task->pi_waiters_leftmost = &waiter->pi_tree_entry;
+
+ rb_link_node(&waiter->pi_tree_entry, parent, link);
+ rb_insert_color(&waiter->pi_tree_entry, &task->pi_waiters);
+}
+
+static void
+rt_mutex_dequeue_pi(struct task_struct *task, struct rt_mutex_waiter *waiter)
+{
+ if (RB_EMPTY_NODE(&waiter->pi_tree_entry))
+ return;
+
+ if (task->pi_waiters_leftmost == &waiter->pi_tree_entry)
+ task->pi_waiters_leftmost = rb_next(&waiter->pi_tree_entry);
+
+ rb_erase(&waiter->pi_tree_entry, &task->pi_waiters);
+ RB_CLEAR_NODE(&waiter->pi_tree_entry);
+}
+
/*
- * Calculate task priority from the waiter list priority
+ * Calculate task priority from the waiter tree priority
*
- * Return task->normal_prio when the waiter list is empty or when
+ * Return task->normal_prio when the waiter tree is empty or when
* the waiter is not allowed to do priority boosting
*/
int rt_mutex_getprio(struct task_struct *task)
if (likely(!task_has_pi_waiters(task)))
return task->normal_prio;
- return min(task_top_pi_waiter(task)->pi_list_entry.prio,
+ return min(task_top_pi_waiter(task)->prio,
task->normal_prio);
}
+struct task_struct *rt_mutex_get_top_task(struct task_struct *task)
+{
+ if (likely(!task_has_pi_waiters(task)))
+ return NULL;
+
+ return task_top_pi_waiter(task)->task;
+}
+
/*
* Adjust the priority of a task, after its pi_waiters got modified.
*
{
int prio = rt_mutex_getprio(task);
- if (task->prio != prio)
+ if (task->prio != prio || dl_prio(prio))
rt_mutex_setprio(task, prio);
}
* When deadlock detection is off then we check, if further
* priority adjustment is necessary.
*/
- if (!detect_deadlock && waiter->list_entry.prio == task->prio)
+ if (!detect_deadlock && waiter->prio == task->prio)
goto out_unlock_pi;
lock = waiter->lock;
top_waiter = rt_mutex_top_waiter(lock);
/* Requeue the waiter */
- plist_del(&waiter->list_entry, &lock->wait_list);
- waiter->list_entry.prio = task->prio;
- plist_add(&waiter->list_entry, &lock->wait_list);
+ rt_mutex_dequeue(lock, waiter);
+ waiter->prio = task->prio;
+ rt_mutex_enqueue(lock, waiter);
/* Release the task */
raw_spin_unlock_irqrestore(&task->pi_lock, flags);
if (waiter == rt_mutex_top_waiter(lock)) {
/* Boost the owner */
- plist_del(&top_waiter->pi_list_entry, &task->pi_waiters);
- waiter->pi_list_entry.prio = waiter->list_entry.prio;
- plist_add(&waiter->pi_list_entry, &task->pi_waiters);
+ rt_mutex_dequeue_pi(task, top_waiter);
+ rt_mutex_enqueue_pi(task, waiter);
__rt_mutex_adjust_prio(task);
} else if (top_waiter == waiter) {
/* Deboost the owner */
- plist_del(&waiter->pi_list_entry, &task->pi_waiters);
+ rt_mutex_dequeue_pi(task, waiter);
waiter = rt_mutex_top_waiter(lock);
- waiter->pi_list_entry.prio = waiter->list_entry.prio;
- plist_add(&waiter->pi_list_entry, &task->pi_waiters);
+ rt_mutex_enqueue_pi(task, waiter);
__rt_mutex_adjust_prio(task);
}
* 3) it is top waiter
*/
if (rt_mutex_has_waiters(lock)) {
- if (task->prio >= rt_mutex_top_waiter(lock)->list_entry.prio) {
+ if (task->prio >= rt_mutex_top_waiter(lock)->prio) {
if (!waiter || waiter != rt_mutex_top_waiter(lock))
return 0;
}
/* remove the queued waiter. */
if (waiter) {
- plist_del(&waiter->list_entry, &lock->wait_list);
+ rt_mutex_dequeue(lock, waiter);
task->pi_blocked_on = NULL;
}
*/
if (rt_mutex_has_waiters(lock)) {
top = rt_mutex_top_waiter(lock);
- top->pi_list_entry.prio = top->list_entry.prio;
- plist_add(&top->pi_list_entry, &task->pi_waiters);
+ rt_mutex_enqueue_pi(task, top);
}
raw_spin_unlock_irqrestore(&task->pi_lock, flags);
}
__rt_mutex_adjust_prio(task);
waiter->task = task;
waiter->lock = lock;
- plist_node_init(&waiter->list_entry, task->prio);
- plist_node_init(&waiter->pi_list_entry, task->prio);
+ waiter->prio = task->prio;
/* Get the top priority waiter on the lock */
if (rt_mutex_has_waiters(lock))
top_waiter = rt_mutex_top_waiter(lock);
- plist_add(&waiter->list_entry, &lock->wait_list);
+ rt_mutex_enqueue(lock, waiter);
task->pi_blocked_on = waiter;
if (waiter == rt_mutex_top_waiter(lock)) {
raw_spin_lock_irqsave(&owner->pi_lock, flags);
- plist_del(&top_waiter->pi_list_entry, &owner->pi_waiters);
- plist_add(&waiter->pi_list_entry, &owner->pi_waiters);
+ rt_mutex_dequeue_pi(owner, top_waiter);
+ rt_mutex_enqueue_pi(owner, waiter);
__rt_mutex_adjust_prio(owner);
if (owner->pi_blocked_on)
* boosted mode and go back to normal after releasing
* lock->wait_lock.
*/
- plist_del(&waiter->pi_list_entry, ¤t->pi_waiters);
+ rt_mutex_dequeue_pi(current, waiter);
rt_mutex_set_owner(lock, NULL);
int chain_walk = 0;
raw_spin_lock_irqsave(¤t->pi_lock, flags);
- plist_del(&waiter->list_entry, &lock->wait_list);
+ rt_mutex_dequeue(lock, waiter);
current->pi_blocked_on = NULL;
raw_spin_unlock_irqrestore(¤t->pi_lock, flags);
raw_spin_lock_irqsave(&owner->pi_lock, flags);
- plist_del(&waiter->pi_list_entry, &owner->pi_waiters);
+ rt_mutex_dequeue_pi(owner, waiter);
if (rt_mutex_has_waiters(lock)) {
struct rt_mutex_waiter *next;
next = rt_mutex_top_waiter(lock);
- plist_add(&next->pi_list_entry, &owner->pi_waiters);
+ rt_mutex_enqueue_pi(owner, next);
}
__rt_mutex_adjust_prio(owner);
raw_spin_unlock_irqrestore(&owner->pi_lock, flags);
}
- WARN_ON(!plist_node_empty(&waiter->pi_list_entry));
-
if (!chain_walk)
return;
raw_spin_lock_irqsave(&task->pi_lock, flags);
waiter = task->pi_blocked_on;
- if (!waiter || waiter->list_entry.prio == task->prio) {
+ if (!waiter || (waiter->prio == task->prio &&
+ !dl_prio(task->prio))) {
raw_spin_unlock_irqrestore(&task->pi_lock, flags);
return;
}
int ret = 0;
debug_rt_mutex_init_waiter(&waiter);
+ RB_CLEAR_NODE(&waiter.pi_tree_entry);
+ RB_CLEAR_NODE(&waiter.tree_entry);
raw_spin_lock(&lock->wait_lock);
{
lock->owner = NULL;
raw_spin_lock_init(&lock->wait_lock);
- plist_head_init(&lock->wait_list);
+ lock->waiters = RB_ROOT;
+ lock->waiters_leftmost = NULL;
debug_rt_mutex_init(lock, name);
}
* This is the control structure for tasks blocked on a rt_mutex,
* which is allocated on the kernel stack on of the blocked task.
*
- * @list_entry: pi node to enqueue into the mutex waiters list
- * @pi_list_entry: pi node to enqueue into the mutex owner waiters list
+ * @tree_entry: pi node to enqueue into the mutex waiters tree
+ * @pi_tree_entry: pi node to enqueue into the mutex owner waiters tree
* @task: task reference to the blocked task
*/
struct rt_mutex_waiter {
- struct plist_node list_entry;
- struct plist_node pi_list_entry;
+ struct rb_node tree_entry;
+ struct rb_node pi_tree_entry;
struct task_struct *task;
struct rt_mutex *lock;
#ifdef CONFIG_DEBUG_RT_MUTEXES
struct pid *deadlock_task_pid;
struct rt_mutex *deadlock_lock;
#endif
+ int prio;
};
/*
- * Various helpers to access the waiters-plist:
+ * Various helpers to access the waiters-tree:
*/
static inline int rt_mutex_has_waiters(struct rt_mutex *lock)
{
- return !plist_head_empty(&lock->wait_list);
+ return !RB_EMPTY_ROOT(&lock->waiters);
}
static inline struct rt_mutex_waiter *
{
struct rt_mutex_waiter *w;
- w = plist_first_entry(&lock->wait_list, struct rt_mutex_waiter,
- list_entry);
+ w = rb_entry(lock->waiters_leftmost, struct rt_mutex_waiter,
+ tree_entry);
BUG_ON(w->lock != lock);
return w;
static inline int task_has_pi_waiters(struct task_struct *p)
{
- return !plist_head_empty(&p->pi_waiters);
+ return !RB_EMPTY_ROOT(&p->pi_waiters);
}
static inline struct rt_mutex_waiter *
task_top_pi_waiter(struct task_struct *p)
{
- return plist_first_entry(&p->pi_waiters, struct rt_mutex_waiter,
- pi_list_entry);
+ return rb_entry(p->pi_waiters_leftmost, struct rt_mutex_waiter,
+ pi_tree_entry);
}
/*
static int pause_on_oops_flag;
static DEFINE_SPINLOCK(pause_on_oops_lock);
-int panic_timeout;
+int panic_timeout = CONFIG_PANIC_TIMEOUT;
EXPORT_SYMBOL_GPL(panic_timeout);
ATOMIC_NOTIFIER_HEAD(panic_notifier_list);
/*
* Sample a process (thread group) clock for the given group_leader task.
- * Must be called with tasklist_lock held for reading.
+ * Must be called with task sighand lock held for safe while_each_thread()
+ * traversal.
*/
static int cpu_clock_sample_group(const clockid_t which_clock,
struct task_struct *p,
return 0;
}
+static int posix_cpu_clock_get_task(struct task_struct *tsk,
+ const clockid_t which_clock,
+ struct timespec *tp)
+{
+ int err = -EINVAL;
+ unsigned long long rtn;
+
+ if (CPUCLOCK_PERTHREAD(which_clock)) {
+ if (same_thread_group(tsk, current))
+ err = cpu_clock_sample(which_clock, tsk, &rtn);
+ } else {
+ unsigned long flags;
+ struct sighand_struct *sighand;
+
+ /*
+ * while_each_thread() is not yet entirely RCU safe,
+ * keep locking the group while sampling process
+ * clock for now.
+ */
+ sighand = lock_task_sighand(tsk, &flags);
+ if (!sighand)
+ return err;
+
+ if (tsk == current || thread_group_leader(tsk))
+ err = cpu_clock_sample_group(which_clock, tsk, &rtn);
+
+ unlock_task_sighand(tsk, &flags);
+ }
+
+ if (!err)
+ sample_to_timespec(which_clock, rtn, tp);
+
+ return err;
+}
+
static int posix_cpu_clock_get(const clockid_t which_clock, struct timespec *tp)
{
const pid_t pid = CPUCLOCK_PID(which_clock);
- int error = -EINVAL;
- unsigned long long rtn;
+ int err = -EINVAL;
if (pid == 0) {
/*
* Special case constant value for our own clocks.
* We don't have to do any lookup to find ourselves.
*/
- if (CPUCLOCK_PERTHREAD(which_clock)) {
- /*
- * Sampling just ourselves we can do with no locking.
- */
- error = cpu_clock_sample(which_clock,
- current, &rtn);
- } else {
- read_lock(&tasklist_lock);
- error = cpu_clock_sample_group(which_clock,
- current, &rtn);
- read_unlock(&tasklist_lock);
- }
+ err = posix_cpu_clock_get_task(current, which_clock, tp);
} else {
/*
* Find the given PID, and validate that the caller
struct task_struct *p;
rcu_read_lock();
p = find_task_by_vpid(pid);
- if (p) {
- if (CPUCLOCK_PERTHREAD(which_clock)) {
- if (same_thread_group(p, current)) {
- error = cpu_clock_sample(which_clock,
- p, &rtn);
- }
- } else {
- read_lock(&tasklist_lock);
- if (thread_group_leader(p) && p->sighand) {
- error =
- cpu_clock_sample_group(which_clock,
- p, &rtn);
- }
- read_unlock(&tasklist_lock);
- }
- }
+ if (p)
+ err = posix_cpu_clock_get_task(p, which_clock, tp);
rcu_read_unlock();
}
- if (error)
- return error;
- sample_to_timespec(which_clock, rtn, tp);
- return 0;
+ return err;
}
*/
static int posix_cpu_timer_del(struct k_itimer *timer)
{
- struct task_struct *p = timer->it.cpu.task;
int ret = 0;
+ unsigned long flags;
+ struct sighand_struct *sighand;
+ struct task_struct *p = timer->it.cpu.task;
- if (likely(p != NULL)) {
- read_lock(&tasklist_lock);
- if (unlikely(p->sighand == NULL)) {
- /*
- * We raced with the reaping of the task.
- * The deletion should have cleared us off the list.
- */
- BUG_ON(!list_empty(&timer->it.cpu.entry));
- } else {
- spin_lock(&p->sighand->siglock);
- if (timer->it.cpu.firing)
- ret = TIMER_RETRY;
- else
- list_del(&timer->it.cpu.entry);
- spin_unlock(&p->sighand->siglock);
- }
- read_unlock(&tasklist_lock);
+ WARN_ON_ONCE(p == NULL);
- if (!ret)
- put_task_struct(p);
+ /*
+ * Protect against sighand release/switch in exit/exec and process/
+ * thread timer list entry concurrent read/writes.
+ */
+ sighand = lock_task_sighand(p, &flags);
+ if (unlikely(sighand == NULL)) {
+ /*
+ * We raced with the reaping of the task.
+ * The deletion should have cleared us off the list.
+ */
+ WARN_ON_ONCE(!list_empty(&timer->it.cpu.entry));
+ } else {
+ if (timer->it.cpu.firing)
+ ret = TIMER_RETRY;
+ else
+ list_del(&timer->it.cpu.entry);
+
+ unlock_task_sighand(p, &flags);
}
+ if (!ret)
+ put_task_struct(p);
+
return ret;
}
-static void cleanup_timers_list(struct list_head *head,
- unsigned long long curr)
+static void cleanup_timers_list(struct list_head *head)
{
struct cpu_timer_list *timer, *next;
* time for later timer_gettime calls to return.
* This must be called with the siglock held.
*/
-static void cleanup_timers(struct list_head *head,
- cputime_t utime, cputime_t stime,
- unsigned long long sum_exec_runtime)
+static void cleanup_timers(struct list_head *head)
{
-
- cputime_t ptime = utime + stime;
-
- cleanup_timers_list(head, cputime_to_expires(ptime));
- cleanup_timers_list(++head, cputime_to_expires(utime));
- cleanup_timers_list(++head, sum_exec_runtime);
+ cleanup_timers_list(head);
+ cleanup_timers_list(++head);
+ cleanup_timers_list(++head);
}
/*
*/
void posix_cpu_timers_exit(struct task_struct *tsk)
{
- cputime_t utime, stime;
-
add_device_randomness((const void*) &tsk->se.sum_exec_runtime,
sizeof(unsigned long long));
- task_cputime(tsk, &utime, &stime);
- cleanup_timers(tsk->cpu_timers,
- utime, stime, tsk->se.sum_exec_runtime);
+ cleanup_timers(tsk->cpu_timers);
}
void posix_cpu_timers_exit_group(struct task_struct *tsk)
{
- struct signal_struct *const sig = tsk->signal;
- cputime_t utime, stime;
-
- task_cputime(tsk, &utime, &stime);
- cleanup_timers(tsk->signal->cpu_timers,
- utime + sig->utime, stime + sig->stime,
- tsk->se.sum_exec_runtime + sig->sum_sched_runtime);
-}
-
-static void clear_dead_task(struct k_itimer *itimer, unsigned long long now)
-{
- struct cpu_timer_list *timer = &itimer->it.cpu;
-
- /*
- * That's all for this thread or process.
- * We leave our residual in expires to be reported.
- */
- put_task_struct(timer->task);
- timer->task = NULL;
- if (timer->expires < now) {
- timer->expires = 0;
- } else {
- timer->expires -= now;
- }
+ cleanup_timers(tsk->signal->cpu_timers);
}
static inline int expires_gt(cputime_t expires, cputime_t new_exp)
/*
* Insert the timer on the appropriate list before any timers that
- * expire later. This must be called with the tasklist_lock held
- * for reading, interrupts disabled and p->sighand->siglock taken.
+ * expire later. This must be called with the sighand lock held.
*/
static void arm_timer(struct k_itimer *timer)
{
/*
* Sample a process (thread group) timer for the given group_leader task.
- * Must be called with tasklist_lock held for reading.
+ * Must be called with task sighand lock held for safe while_each_thread()
+ * traversal.
*/
static int cpu_timer_sample_group(const clockid_t which_clock,
struct task_struct *p,
*/
static void posix_cpu_timer_kick_nohz(void)
{
- schedule_work(&nohz_kick_work);
+ if (context_tracking_is_enabled())
+ schedule_work(&nohz_kick_work);
}
bool posix_cpu_timers_can_stop_tick(struct task_struct *tsk)
* If we return TIMER_RETRY, it's necessary to release the timer's lock
* and try again. (This happens when the timer is in the middle of firing.)
*/
-static int posix_cpu_timer_set(struct k_itimer *timer, int flags,
+static int posix_cpu_timer_set(struct k_itimer *timer, int timer_flags,
struct itimerspec *new, struct itimerspec *old)
{
+ unsigned long flags;
+ struct sighand_struct *sighand;
struct task_struct *p = timer->it.cpu.task;
unsigned long long old_expires, new_expires, old_incr, val;
int ret;
- if (unlikely(p == NULL)) {
- /*
- * Timer refers to a dead task's clock.
- */
- return -ESRCH;
- }
+ WARN_ON_ONCE(p == NULL);
new_expires = timespec_to_sample(timer->it_clock, &new->it_value);
- read_lock(&tasklist_lock);
/*
- * We need the tasklist_lock to protect against reaping that
- * clears p->sighand. If p has just been reaped, we can no
+ * Protect against sighand release/switch in exit/exec and p->cpu_timers
+ * and p->signal->cpu_timers read/write in arm_timer()
+ */
+ sighand = lock_task_sighand(p, &flags);
+ /*
+ * If p has just been reaped, we can no
* longer get any information about it at all.
*/
- if (unlikely(p->sighand == NULL)) {
- read_unlock(&tasklist_lock);
- put_task_struct(p);
- timer->it.cpu.task = NULL;
+ if (unlikely(sighand == NULL)) {
return -ESRCH;
}
/*
* Disarm any old timer after extracting its expiry time.
*/
- BUG_ON(!irqs_disabled());
+ WARN_ON_ONCE(!irqs_disabled());
ret = 0;
old_incr = timer->it.cpu.incr;
- spin_lock(&p->sighand->siglock);
old_expires = timer->it.cpu.expires;
if (unlikely(timer->it.cpu.firing)) {
timer->it.cpu.firing = -1;
* disable this firing since we are already reporting
* it as an overrun (thanks to bump_cpu_timer above).
*/
- spin_unlock(&p->sighand->siglock);
- read_unlock(&tasklist_lock);
+ unlock_task_sighand(p, &flags);
goto out;
}
- if (new_expires != 0 && !(flags & TIMER_ABSTIME)) {
+ if (new_expires != 0 && !(timer_flags & TIMER_ABSTIME)) {
new_expires += val;
}
arm_timer(timer);
}
- spin_unlock(&p->sighand->siglock);
- read_unlock(&tasklist_lock);
-
+ unlock_task_sighand(p, &flags);
/*
* Install the new reload setting, and
* set up the signal and overrun bookkeeping.
{
unsigned long long now;
struct task_struct *p = timer->it.cpu.task;
- int clear_dead;
+
+ WARN_ON_ONCE(p == NULL);
/*
* Easy part: convert the reload time.
return;
}
- if (unlikely(p == NULL)) {
- /*
- * This task already died and the timer will never fire.
- * In this case, expires is actually the dead value.
- */
- dead:
- sample_to_timespec(timer->it_clock, timer->it.cpu.expires,
- &itp->it_value);
- return;
- }
-
/*
* Sample the clock to take the difference with the expiry time.
*/
if (CPUCLOCK_PERTHREAD(timer->it_clock)) {
cpu_clock_sample(timer->it_clock, p, &now);
- clear_dead = p->exit_state;
} else {
- read_lock(&tasklist_lock);
- if (unlikely(p->sighand == NULL)) {
+ struct sighand_struct *sighand;
+ unsigned long flags;
+
+ /*
+ * Protect against sighand release/switch in exit/exec and
+ * also make timer sampling safe if it ends up calling
+ * thread_group_cputime().
+ */
+ sighand = lock_task_sighand(p, &flags);
+ if (unlikely(sighand == NULL)) {
/*
* The process has been reaped.
* We can't even collect a sample any more.
* Call the timer disarmed, nothing else to do.
*/
- put_task_struct(p);
- timer->it.cpu.task = NULL;
timer->it.cpu.expires = 0;
- read_unlock(&tasklist_lock);
- goto dead;
+ sample_to_timespec(timer->it_clock, timer->it.cpu.expires,
+ &itp->it_value);
} else {
cpu_timer_sample_group(timer->it_clock, p, &now);
- clear_dead = (unlikely(p->exit_state) &&
- thread_group_empty(p));
+ unlock_task_sighand(p, &flags);
}
- read_unlock(&tasklist_lock);
- }
-
- if (unlikely(clear_dead)) {
- /*
- * We've noticed that the thread is dead, but
- * not yet reaped. Take this opportunity to
- * drop our task ref.
- */
- clear_dead_task(timer, now);
- goto dead;
}
if (now < timer->it.cpu.expires) {
*/
void posix_cpu_timer_schedule(struct k_itimer *timer)
{
+ struct sighand_struct *sighand;
+ unsigned long flags;
struct task_struct *p = timer->it.cpu.task;
unsigned long long now;
- if (unlikely(p == NULL))
- /*
- * The task was cleaned up already, no future firings.
- */
- goto out;
+ WARN_ON_ONCE(p == NULL);
/*
* Fetch the current sample and update the timer's expiry time.
if (CPUCLOCK_PERTHREAD(timer->it_clock)) {
cpu_clock_sample(timer->it_clock, p, &now);
bump_cpu_timer(timer, now);
- if (unlikely(p->exit_state)) {
- clear_dead_task(timer, now);
+ if (unlikely(p->exit_state))
+ goto out;
+
+ /* Protect timer list r/w in arm_timer() */
+ sighand = lock_task_sighand(p, &flags);
+ if (!sighand)
goto out;
- }
- read_lock(&tasklist_lock); /* arm_timer needs it. */
- spin_lock(&p->sighand->siglock);
} else {
- read_lock(&tasklist_lock);
- if (unlikely(p->sighand == NULL)) {
+ /*
+ * Protect arm_timer() and timer sampling in case of call to
+ * thread_group_cputime().
+ */
+ sighand = lock_task_sighand(p, &flags);
+ if (unlikely(sighand == NULL)) {
/*
* The process has been reaped.
* We can't even collect a sample any more.
*/
- put_task_struct(p);
- timer->it.cpu.task = p = NULL;
timer->it.cpu.expires = 0;
- goto out_unlock;
+ goto out;
} else if (unlikely(p->exit_state) && thread_group_empty(p)) {
- /*
- * We've noticed that the thread is dead, but
- * not yet reaped. Take this opportunity to
- * drop our task ref.
- */
- cpu_timer_sample_group(timer->it_clock, p, &now);
- clear_dead_task(timer, now);
- goto out_unlock;
+ unlock_task_sighand(p, &flags);
+ /* Optimizations: if the process is dying, no need to rearm */
+ goto out;
}
- spin_lock(&p->sighand->siglock);
cpu_timer_sample_group(timer->it_clock, p, &now);
bump_cpu_timer(timer, now);
- /* Leave the tasklist_lock locked for the call below. */
+ /* Leave the sighand locked for the call below. */
}
/*
* Now re-arm for the new expiry time.
*/
- BUG_ON(!irqs_disabled());
+ WARN_ON_ONCE(!irqs_disabled());
arm_timer(timer);
- spin_unlock(&p->sighand->siglock);
-
-out_unlock:
- read_unlock(&tasklist_lock);
+ unlock_task_sighand(p, &flags);
+ /* Kick full dynticks CPUs in case they need to tick on the new timer */
+ posix_cpu_timer_kick_nohz();
out:
timer->it_overrun_last = timer->it_overrun;
timer->it_overrun = -1;
struct k_itimer *timer, *next;
unsigned long flags;
- BUG_ON(!irqs_disabled());
+ WARN_ON_ONCE(!irqs_disabled());
/*
* The fast path checks that there are no expired thread or thread
cpu_timer_fire(timer);
spin_unlock(&timer->it_lock);
}
-
- /*
- * In case some timers were rescheduled after the queue got emptied,
- * wake up full dynticks CPUs.
- */
- if (tsk->signal->cputimer.running)
- posix_cpu_timer_kick_nohz();
}
/*
{
unsigned long long now;
- BUG_ON(clock_idx == CPUCLOCK_SCHED);
+ WARN_ON_ONCE(clock_idx == CPUCLOCK_SCHED);
cpu_timer_sample_group(clock_idx, tsk, &now);
if (oldval) {
}
#endif /* #else !CONFIG_DEBUG_OBJECTS_RCU_HEAD */
-extern void kfree(const void *);
+void kfree(const void *);
static inline bool __rcu_reclaim(const char *rn, struct rcu_head *head)
{
unsigned long offset = (unsigned long)head->func;
+ rcu_lock_acquire(&rcu_callback_map);
if (__is_kfree_rcu_offset(offset)) {
RCU_TRACE(trace_rcu_invoke_kfree_callback(rn, head, offset));
kfree((void *)head - offset);
+ rcu_lock_release(&rcu_callback_map);
return 1;
} else {
RCU_TRACE(trace_rcu_invoke_callback(rn, head));
head->func(head);
+ rcu_lock_release(&rcu_callback_map);
return 0;
}
}
/*
* Enqueue an SRCU callback on the specified srcu_struct structure,
* initiating grace-period processing if it is not already running.
+ *
+ * Note that all CPUs must agree that the grace period extended beyond
+ * all pre-existing SRCU read-side critical section. On systems with
+ * more than one CPU, this means that when "func()" is invoked, each CPU
+ * is guaranteed to have executed a full memory barrier since the end of
+ * its last corresponding SRCU read-side critical section whose beginning
+ * preceded the call to call_rcu(). It also means that each CPU executing
+ * an SRCU read-side critical section that continues beyond the start of
+ * "func()" must have executed a memory barrier after the call_rcu()
+ * but before the beginning of that SRCU read-side critical section.
+ * Note that these guarantees include CPUs that are offline, idle, or
+ * executing in user mode, as well as CPUs that are executing in the kernel.
+ *
+ * Furthermore, if CPU A invoked call_rcu() and CPU B invoked the
+ * resulting SRCU callback function "func()", then both CPU A and CPU
+ * B are guaranteed to execute a full memory barrier during the time
+ * interval between the call to call_rcu() and the invocation of "func()".
+ * This guarantee applies even if CPU A and CPU B are the same CPU (but
+ * again only if the system has more than one CPU).
+ *
+ * Of course, these guarantees apply only for invocations of call_srcu(),
+ * srcu_read_lock(), and srcu_read_unlock() that are all passed the same
+ * srcu_struct structure.
*/
void call_srcu(struct srcu_struct *sp, struct rcu_head *head,
void (*func)(struct rcu_head *head))
* Note that it is illegal to call synchronize_srcu() from the corresponding
* SRCU read-side critical section; doing so will result in deadlock.
* However, it is perfectly legal to call synchronize_srcu() on one
- * srcu_struct from some other srcu_struct's read-side critical section.
+ * srcu_struct from some other srcu_struct's read-side critical section,
+ * as long as the resulting graph of srcu_structs is acyclic.
+ *
+ * There are memory-ordering constraints implied by synchronize_srcu().
+ * On systems with more than one CPU, when synchronize_srcu() returns,
+ * each CPU is guaranteed to have executed a full memory barrier since
+ * the end of its last corresponding SRCU-sched read-side critical section
+ * whose beginning preceded the call to synchronize_srcu(). In addition,
+ * each CPU having an SRCU read-side critical section that extends beyond
+ * the return from synchronize_srcu() is guaranteed to have executed a
+ * full memory barrier after the beginning of synchronize_srcu() and before
+ * the beginning of that SRCU read-side critical section. Note that these
+ * guarantees include CPUs that are offline, idle, or executing in user mode,
+ * as well as CPUs that are executing in the kernel.
+ *
+ * Furthermore, if CPU A invoked synchronize_srcu(), which returned
+ * to its caller on CPU B, then both CPU A and CPU B are guaranteed
+ * to have executed a full memory barrier during the execution of
+ * synchronize_srcu(). This guarantee applies even if CPU A and CPU B
+ * are the same CPU, but again only if the system has more than one CPU.
+ *
+ * Of course, these memory-ordering guarantees apply only when
+ * synchronize_srcu(), srcu_read_lock(), and srcu_read_unlock() are
+ * passed the same srcu_struct structure.
*/
void synchronize_srcu(struct srcu_struct *sp)
{
* Wait for an SRCU grace period to elapse, but be more aggressive about
* spinning rather than blocking when waiting.
*
- * Note that it is also illegal to call synchronize_srcu_expedited()
- * from the corresponding SRCU read-side critical section;
- * doing so will result in deadlock. However, it is perfectly legal
- * to call synchronize_srcu_expedited() on one srcu_struct from some
- * other srcu_struct's read-side critical section, as long as
- * the resulting graph of srcu_structs is acyclic.
+ * Note that synchronize_srcu_expedited() has the same deadlock and
+ * memory-ordering properties as does synchronize_srcu().
*/
void synchronize_srcu_expedited(struct srcu_struct *sp)
{
/**
* srcu_barrier - Wait until all in-flight call_srcu() callbacks complete.
+ * @sp: srcu_struct on which to wait for in-flight callbacks.
*/
void srcu_barrier(struct srcu_struct *sp)
{
#define VERBOSE_PRINTK_ERRSTRING(s) \
do { if (verbose) pr_alert("%s" TORTURE_FLAG "!!! " s "\n", torture_type); } while (0)
-static char printk_buf[4096];
-
static int nrealreaders;
static struct task_struct *writer_task;
static struct task_struct **fakewriter_tasks;
void (*call)(struct rcu_head *head, void (*func)(struct rcu_head *rcu));
void (*cb_barrier)(void);
void (*fqs)(void);
- int (*stats)(char *page);
+ void (*stats)(char *page);
int irq_capable;
int can_boost;
const char *name;
srcu_barrier(&srcu_ctl);
}
-static int srcu_torture_stats(char *page)
+static void srcu_torture_stats(char *page)
{
- int cnt = 0;
int cpu;
int idx = srcu_ctl.completed & 0x1;
- cnt += sprintf(&page[cnt], "%s%s per-CPU(idx=%d):",
+ page += sprintf(page, "%s%s per-CPU(idx=%d):",
torture_type, TORTURE_FLAG, idx);
for_each_possible_cpu(cpu) {
- cnt += sprintf(&page[cnt], " %d(%lu,%lu)", cpu,
+ page += sprintf(page, " %d(%lu,%lu)", cpu,
per_cpu_ptr(srcu_ctl.per_cpu_ref, cpu)->c[!idx],
per_cpu_ptr(srcu_ctl.per_cpu_ref, cpu)->c[idx]);
}
- cnt += sprintf(&page[cnt], "\n");
- return cnt;
+ sprintf(page, "\n");
}
static void srcu_torture_synchronize_expedited(void)
/*
* Create an RCU-torture statistics message in the specified buffer.
*/
-static int
+static void
rcu_torture_printk(char *page)
{
- int cnt = 0;
int cpu;
int i;
long pipesummary[RCU_TORTURE_PIPE_LEN + 1] = { 0 };
if (pipesummary[i] != 0)
break;
}
- cnt += sprintf(&page[cnt], "%s%s ", torture_type, TORTURE_FLAG);
- cnt += sprintf(&page[cnt],
+ page += sprintf(page, "%s%s ", torture_type, TORTURE_FLAG);
+ page += sprintf(page,
"rtc: %p ver: %lu tfle: %d rta: %d rtaf: %d rtf: %d ",
rcu_torture_current,
rcu_torture_current_version,
atomic_read(&n_rcu_torture_alloc),
atomic_read(&n_rcu_torture_alloc_fail),
atomic_read(&n_rcu_torture_free));
- cnt += sprintf(&page[cnt], "rtmbe: %d rtbke: %ld rtbre: %ld ",
+ page += sprintf(page, "rtmbe: %d rtbke: %ld rtbre: %ld ",
atomic_read(&n_rcu_torture_mberror),
n_rcu_torture_boost_ktrerror,
n_rcu_torture_boost_rterror);
- cnt += sprintf(&page[cnt], "rtbf: %ld rtb: %ld nt: %ld ",
+ page += sprintf(page, "rtbf: %ld rtb: %ld nt: %ld ",
n_rcu_torture_boost_failure,
n_rcu_torture_boosts,
n_rcu_torture_timers);
- cnt += sprintf(&page[cnt],
+ page += sprintf(page,
"onoff: %ld/%ld:%ld/%ld %d,%d:%d,%d %lu:%lu (HZ=%d) ",
n_online_successes, n_online_attempts,
n_offline_successes, n_offline_attempts,
min_online, max_online,
min_offline, max_offline,
sum_online, sum_offline, HZ);
- cnt += sprintf(&page[cnt], "barrier: %ld/%ld:%ld",
+ page += sprintf(page, "barrier: %ld/%ld:%ld",
n_barrier_successes,
n_barrier_attempts,
n_rcu_torture_barrier_error);
- cnt += sprintf(&page[cnt], "\n%s%s ", torture_type, TORTURE_FLAG);
+ page += sprintf(page, "\n%s%s ", torture_type, TORTURE_FLAG);
if (atomic_read(&n_rcu_torture_mberror) != 0 ||
n_rcu_torture_barrier_error != 0 ||
n_rcu_torture_boost_ktrerror != 0 ||
n_rcu_torture_boost_rterror != 0 ||
n_rcu_torture_boost_failure != 0 ||
i > 1) {
- cnt += sprintf(&page[cnt], "!!! ");
+ page += sprintf(page, "!!! ");
atomic_inc(&n_rcu_torture_error);
WARN_ON_ONCE(1);
}
- cnt += sprintf(&page[cnt], "Reader Pipe: ");
+ page += sprintf(page, "Reader Pipe: ");
for (i = 0; i < RCU_TORTURE_PIPE_LEN + 1; i++)
- cnt += sprintf(&page[cnt], " %ld", pipesummary[i]);
- cnt += sprintf(&page[cnt], "\n%s%s ", torture_type, TORTURE_FLAG);
- cnt += sprintf(&page[cnt], "Reader Batch: ");
+ page += sprintf(page, " %ld", pipesummary[i]);
+ page += sprintf(page, "\n%s%s ", torture_type, TORTURE_FLAG);
+ page += sprintf(page, "Reader Batch: ");
for (i = 0; i < RCU_TORTURE_PIPE_LEN + 1; i++)
- cnt += sprintf(&page[cnt], " %ld", batchsummary[i]);
- cnt += sprintf(&page[cnt], "\n%s%s ", torture_type, TORTURE_FLAG);
- cnt += sprintf(&page[cnt], "Free-Block Circulation: ");
+ page += sprintf(page, " %ld", batchsummary[i]);
+ page += sprintf(page, "\n%s%s ", torture_type, TORTURE_FLAG);
+ page += sprintf(page, "Free-Block Circulation: ");
for (i = 0; i < RCU_TORTURE_PIPE_LEN + 1; i++) {
- cnt += sprintf(&page[cnt], " %d",
+ page += sprintf(page, " %d",
atomic_read(&rcu_torture_wcount[i]));
}
- cnt += sprintf(&page[cnt], "\n");
+ page += sprintf(page, "\n");
if (cur_ops->stats)
- cnt += cur_ops->stats(&page[cnt]);
- return cnt;
+ cur_ops->stats(page);
}
/*
static void
rcu_torture_stats_print(void)
{
- int cnt;
+ int size = nr_cpu_ids * 200 + 8192;
+ char *buf;
- cnt = rcu_torture_printk(printk_buf);
- pr_alert("%s", printk_buf);
+ buf = kmalloc(size, GFP_KERNEL);
+ if (!buf) {
+ pr_err("rcu-torture: Out of memory, need: %d", size);
+ return;
+ }
+ rcu_torture_printk(buf);
+ pr_alert("%s", buf);
+ kfree(buf);
}
/*
{
long myid = (long)arg;
bool lastphase = 0;
+ bool newphase;
struct rcu_head rcu;
init_rcu_head_on_stack(&rcu);
set_user_nice(current, 19);
do {
wait_event(barrier_cbs_wq[myid],
- barrier_phase != lastphase ||
+ (newphase =
+ ACCESS_ONCE(barrier_phase)) != lastphase ||
kthread_should_stop() ||
fullstop != FULLSTOP_DONTSTOP);
- lastphase = barrier_phase;
+ lastphase = newphase;
smp_mb(); /* ensure barrier_phase load before ->call(). */
if (kthread_should_stop() || fullstop != FULLSTOP_DONTSTOP)
break;
if (kthread_should_stop() || fullstop != FULLSTOP_DONTSTOP)
break;
n_barrier_attempts++;
- cur_ops->cb_barrier();
+ cur_ops->cb_barrier(); /* Implies smp_mb() for wait_event(). */
if (atomic_read(&barrier_cbs_invoked) != n_barrier_cbs) {
n_rcu_torture_barrier_error++;
WARN_ON_ONCE(1);
static void rcu_eqs_enter_common(struct rcu_dynticks *rdtp, long long oldval,
bool user)
{
+ struct rcu_state *rsp;
+ struct rcu_data *rdp;
+
trace_rcu_dyntick(TPS("Start"), oldval, rdtp->dynticks_nesting);
if (!user && !is_idle_task(current)) {
struct task_struct *idle __maybe_unused =
current->pid, current->comm,
idle->pid, idle->comm); /* must be idle task! */
}
+ for_each_rcu_flavor(rsp) {
+ rdp = this_cpu_ptr(rsp->rda);
+ do_nocb_deferred_wakeup(rdp);
+ }
rcu_prepare_for_idle(smp_processor_id());
/* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
smp_mb__before_atomic_inc(); /* See above. */
rdtp = this_cpu_ptr(&rcu_dynticks);
oldval = rdtp->dynticks_nesting;
WARN_ON_ONCE((oldval & DYNTICK_TASK_NEST_MASK) == 0);
- if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE)
+ if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE) {
rdtp->dynticks_nesting = 0;
- else
+ rcu_eqs_enter_common(rdtp, oldval, user);
+ } else {
rdtp->dynticks_nesting -= DYNTICK_TASK_NEST_VALUE;
- rcu_eqs_enter_common(rdtp, oldval, user);
+ }
}
/**
rdtp = this_cpu_ptr(&rcu_dynticks);
oldval = rdtp->dynticks_nesting;
WARN_ON_ONCE(oldval < 0);
- if (oldval & DYNTICK_TASK_NEST_MASK)
+ if (oldval & DYNTICK_TASK_NEST_MASK) {
rdtp->dynticks_nesting += DYNTICK_TASK_NEST_VALUE;
- else
+ } else {
rdtp->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
- rcu_eqs_exit_common(rdtp, oldval, user);
+ rcu_eqs_exit_common(rdtp, oldval, user);
+ }
}
/**
bool ret;
if (in_nmi())
- return 1;
+ return true;
preempt_disable();
rdp = this_cpu_ptr(&rcu_sched_data);
rnp = rdp->mynode;
return (rdp->dynticks_snap & 0x1) == 0;
}
+/*
+ * This function really isn't for public consumption, but RCU is special in
+ * that context switches can allow the state machine to make progress.
+ */
+extern void resched_cpu(int cpu);
+
/*
* Return true if the specified CPU has passed through a quiescent
* state by virtue of being in or having passed through an dynticks
*/
rcu_kick_nohz_cpu(rdp->cpu);
+ /*
+ * Alternatively, the CPU might be running in the kernel
+ * for an extended period of time without a quiescent state.
+ * Attempt to force the CPU through the scheduler to gain the
+ * needed quiescent state, but only if the grace period has gone
+ * on for an uncommonly long time. If there are many stuck CPUs,
+ * we will beat on the first one until it gets unstuck, then move
+ * to the next. Only do this for the primary flavor of RCU.
+ */
+ if (rdp->rsp == rcu_state &&
+ ULONG_CMP_GE(ACCESS_ONCE(jiffies), rdp->rsp->jiffies_resched)) {
+ rdp->rsp->jiffies_resched += 5;
+ resched_cpu(rdp->cpu);
+ }
+
return 0;
}
static void record_gp_stall_check_time(struct rcu_state *rsp)
{
unsigned long j = ACCESS_ONCE(jiffies);
+ unsigned long j1;
rsp->gp_start = j;
smp_wmb(); /* Record start time before stall time. */
- rsp->jiffies_stall = j + rcu_jiffies_till_stall_check();
+ j1 = rcu_jiffies_till_stall_check();
+ rsp->jiffies_stall = j + j1;
+ rsp->jiffies_resched = j + j1 / 2;
}
/*
* hold it, acquire the root rcu_node structure's lock in order to
* start one (if needed).
*/
- if (rnp != rnp_root)
+ if (rnp != rnp_root) {
raw_spin_lock(&rnp_root->lock);
+ smp_mb__after_unlock_lock();
+ }
/*
* Get a new grace-period number. If there really is no grace
local_irq_restore(flags);
return;
}
+ smp_mb__after_unlock_lock();
__note_gp_changes(rsp, rnp, rdp);
raw_spin_unlock_irqrestore(&rnp->lock, flags);
}
rcu_bind_gp_kthread();
raw_spin_lock_irq(&rnp->lock);
+ smp_mb__after_unlock_lock();
if (rsp->gp_flags == 0) {
/* Spurious wakeup, tell caller to go back to sleep. */
raw_spin_unlock_irq(&rnp->lock);
*/
rcu_for_each_node_breadth_first(rsp, rnp) {
raw_spin_lock_irq(&rnp->lock);
+ smp_mb__after_unlock_lock();
rdp = this_cpu_ptr(rsp->rda);
rcu_preempt_check_blocked_tasks(rnp);
rnp->qsmask = rnp->qsmaskinit;
/* Clear flag to prevent immediate re-entry. */
if (ACCESS_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) {
raw_spin_lock_irq(&rnp->lock);
+ smp_mb__after_unlock_lock();
rsp->gp_flags &= ~RCU_GP_FLAG_FQS;
raw_spin_unlock_irq(&rnp->lock);
}
struct rcu_node *rnp = rcu_get_root(rsp);
raw_spin_lock_irq(&rnp->lock);
+ smp_mb__after_unlock_lock();
gp_duration = jiffies - rsp->gp_start;
if (gp_duration > rsp->gp_max)
rsp->gp_max = gp_duration;
*/
rcu_for_each_node_breadth_first(rsp, rnp) {
raw_spin_lock_irq(&rnp->lock);
+ smp_mb__after_unlock_lock();
ACCESS_ONCE(rnp->completed) = rsp->gpnum;
rdp = this_cpu_ptr(rsp->rda);
if (rnp == rdp->mynode)
__note_gp_changes(rsp, rnp, rdp);
+ /* smp_mb() provided by prior unlock-lock pair. */
nocb += rcu_future_gp_cleanup(rsp, rnp);
raw_spin_unlock_irq(&rnp->lock);
cond_resched();
}
rnp = rcu_get_root(rsp);
raw_spin_lock_irq(&rnp->lock);
+ smp_mb__after_unlock_lock();
rcu_nocb_gp_set(rnp, nocb);
rsp->completed = rsp->gpnum; /* Declare grace period done. */
wait_event_interruptible(rsp->gp_wq,
ACCESS_ONCE(rsp->gp_flags) &
RCU_GP_FLAG_INIT);
+ /* Locking provides needed memory barrier. */
if (rcu_gp_init(rsp))
break;
cond_resched();
(!ACCESS_ONCE(rnp->qsmask) &&
!rcu_preempt_blocked_readers_cgp(rnp)),
j);
+ /* Locking provides needed memory barriers. */
/* If grace period done, leave loop. */
if (!ACCESS_ONCE(rnp->qsmask) &&
!rcu_preempt_blocked_readers_cgp(rnp))
rnp_c = rnp;
rnp = rnp->parent;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
WARN_ON_ONCE(rnp_c->qsmask);
}
rnp = rdp->mynode;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
if (rdp->passed_quiesce == 0 || rdp->gpnum != rnp->gpnum ||
rnp->completed == rnp->gpnum) {
* Adopt the RCU callbacks from the specified rcu_state structure's
* orphanage. The caller must hold the ->orphan_lock.
*/
-static void rcu_adopt_orphan_cbs(struct rcu_state *rsp)
+static void rcu_adopt_orphan_cbs(struct rcu_state *rsp, unsigned long flags)
{
int i;
struct rcu_data *rdp = __this_cpu_ptr(rsp->rda);
/* No-CBs CPUs are handled specially. */
- if (rcu_nocb_adopt_orphan_cbs(rsp, rdp))
+ if (rcu_nocb_adopt_orphan_cbs(rsp, rdp, flags))
return;
/* Do the accounting first. */
/* Orphan the dead CPU's callbacks, and adopt them if appropriate. */
rcu_send_cbs_to_orphanage(cpu, rsp, rnp, rdp);
- rcu_adopt_orphan_cbs(rsp);
+ rcu_adopt_orphan_cbs(rsp, flags);
/* Remove the outgoing CPU from the masks in the rcu_node hierarchy. */
mask = rdp->grpmask; /* rnp->grplo is constant. */
do {
raw_spin_lock(&rnp->lock); /* irqs already disabled. */
+ smp_mb__after_unlock_lock();
rnp->qsmaskinit &= ~mask;
if (rnp->qsmaskinit != 0) {
if (rnp != rdp->mynode)
cond_resched();
mask = 0;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
if (!rcu_gp_in_progress(rsp)) {
raw_spin_unlock_irqrestore(&rnp->lock, flags);
return;
rnp = rcu_get_root(rsp);
if (rnp->qsmask == 0) {
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
rcu_initiate_boost(rnp, flags); /* releases rnp->lock. */
}
}
/* Reached the root of the rcu_node tree, acquire lock. */
raw_spin_lock_irqsave(&rnp_old->lock, flags);
+ smp_mb__after_unlock_lock();
raw_spin_unlock(&rnp_old->fqslock);
if (ACCESS_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) {
rsp->n_force_qs_lh++;
/* If there are callbacks ready, invoke them. */
if (cpu_has_callbacks_ready_to_invoke(rdp))
invoke_rcu_callbacks(rsp, rdp);
+
+ /* Do any needed deferred wakeups of rcuo kthreads. */
+ do_nocb_deferred_wakeup(rdp);
}
/*
struct rcu_node *rnp_root = rcu_get_root(rsp);
raw_spin_lock(&rnp_root->lock);
+ smp_mb__after_unlock_lock();
rcu_start_gp(rsp);
raw_spin_unlock(&rnp_root->lock);
} else {
if (cpu != -1)
rdp = per_cpu_ptr(rsp->rda, cpu);
- offline = !__call_rcu_nocb(rdp, head, lazy);
+ offline = !__call_rcu_nocb(rdp, head, lazy, flags);
WARN_ON_ONCE(offline);
/* _call_rcu() is illegal on offline CPU; leak the callback. */
local_irq_restore(flags);
/* Check for CPU stalls, if enabled. */
check_cpu_stall(rsp, rdp);
+ /* Is this CPU a NO_HZ_FULL CPU that should ignore RCU? */
+ if (rcu_nohz_full_cpu(rsp))
+ return 0;
+
/* Is the RCU core waiting for a quiescent state from this CPU? */
if (rcu_scheduler_fully_active &&
rdp->qs_pending && !rdp->passed_quiesce) {
return 1;
}
+ /* Does this CPU need a deferred NOCB wakeup? */
+ if (rcu_nocb_need_deferred_wakeup(rdp)) {
+ rdp->n_rp_nocb_defer_wakeup++;
+ return 1;
+ }
+
/* nothing to do */
rdp->n_rp_need_nothing++;
return 0;
{
int i;
- for (i = rcu_num_lvls - 1; i > 0; i--)
+ rsp->levelspread[rcu_num_lvls - 1] = rcu_fanout_leaf;
+ for (i = rcu_num_lvls - 2; i >= 0; i--)
rsp->levelspread[i] = CONFIG_RCU_FANOUT;
- rsp->levelspread[0] = rcu_fanout_leaf;
}
#else /* #ifdef CONFIG_RCU_FANOUT_EXACT */
static void __init rcu_init_levelspread(struct rcu_state *rsp)
if (rcu_fanout_leaf == CONFIG_RCU_FANOUT_LEAF &&
nr_cpu_ids == NR_CPUS)
return;
+ pr_info("RCU: Adjusting geometry for rcu_fanout_leaf=%d, nr_cpu_ids=%d\n",
+ rcu_fanout_leaf, nr_cpu_ids);
/*
* Compute number of nodes that can be handled an rcu_node tree
unsigned long n_rp_cpu_needs_gp;
unsigned long n_rp_gp_completed;
unsigned long n_rp_gp_started;
+ unsigned long n_rp_nocb_defer_wakeup;
unsigned long n_rp_need_nothing;
/* 6) _rcu_barrier() and OOM callbacks. */
int nocb_p_count_lazy; /* (approximate). */
wait_queue_head_t nocb_wq; /* For nocb kthreads to sleep on. */
struct task_struct *nocb_kthread;
+ bool nocb_defer_wakeup; /* Defer wakeup of nocb_kthread. */
#endif /* #ifdef CONFIG_RCU_NOCB_CPU */
/* 8) RCU CPU stall data. */
/* but in jiffies. */
unsigned long jiffies_stall; /* Time at which to check */
/* for CPU stalls. */
+ unsigned long jiffies_resched; /* Time at which to resched */
+ /* a reluctant CPU. */
unsigned long gp_max; /* Maximum GP duration in */
/* jiffies. */
const char *name; /* Name of structure. */
static void rcu_nocb_gp_cleanup(struct rcu_state *rsp, struct rcu_node *rnp);
static void rcu_init_one_nocb(struct rcu_node *rnp);
static bool __call_rcu_nocb(struct rcu_data *rdp, struct rcu_head *rhp,
- bool lazy);
+ bool lazy, unsigned long flags);
static bool rcu_nocb_adopt_orphan_cbs(struct rcu_state *rsp,
- struct rcu_data *rdp);
+ struct rcu_data *rdp,
+ unsigned long flags);
+static bool rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp);
+static void do_nocb_deferred_wakeup(struct rcu_data *rdp);
static void rcu_boot_init_nocb_percpu_data(struct rcu_data *rdp);
static void rcu_spawn_nocb_kthreads(struct rcu_state *rsp);
static void rcu_kick_nohz_cpu(int cpu);
unsigned long maxj);
static void rcu_bind_gp_kthread(void);
static void rcu_sysidle_init_percpu_data(struct rcu_dynticks *rdtp);
+static bool rcu_nohz_full_cpu(struct rcu_state *rsp);
#endif /* #ifndef RCU_TREE_NONCORE */
rdp = per_cpu_ptr(rcu_preempt_state.rda, cpu);
rnp = rdp->mynode;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
t->rcu_read_unlock_special |= RCU_READ_UNLOCK_BLOCKED;
t->rcu_blocked_node = rnp;
mask = rnp->grpmask;
raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
raw_spin_lock(&rnp_p->lock); /* irqs already disabled. */
+ smp_mb__after_unlock_lock();
rcu_report_qs_rnp(mask, &rcu_preempt_state, rnp_p, flags);
}
special = t->rcu_read_unlock_special;
if (special & RCU_READ_UNLOCK_NEED_QS) {
rcu_preempt_qs(smp_processor_id());
+ if (!t->rcu_read_unlock_special) {
+ local_irq_restore(flags);
+ return;
+ }
}
- /* Hardware IRQ handlers cannot block. */
- if (in_irq() || in_serving_softirq()) {
+ /* Hardware IRQ handlers cannot block, complain if they get here. */
+ if (WARN_ON_ONCE(in_irq() || in_serving_softirq())) {
local_irq_restore(flags);
return;
}
for (;;) {
rnp = t->rcu_blocked_node;
raw_spin_lock(&rnp->lock); /* irqs already disabled. */
+ smp_mb__after_unlock_lock();
if (rnp == t->rcu_blocked_node)
break;
raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
while (!list_empty(lp)) {
t = list_entry(lp->next, typeof(*t), rcu_node_entry);
raw_spin_lock(&rnp_root->lock); /* irqs already disabled */
+ smp_mb__after_unlock_lock();
list_del(&t->rcu_node_entry);
t->rcu_blocked_node = rnp_root;
list_add(&t->rcu_node_entry, lp_root);
* in this case.
*/
raw_spin_lock(&rnp_root->lock); /* irqs already disabled */
+ smp_mb__after_unlock_lock();
if (rnp_root->boost_tasks != NULL &&
rnp_root->boost_tasks != rnp_root->gp_tasks &&
rnp_root->boost_tasks != rnp_root->exp_tasks)
unsigned long mask;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
for (;;) {
if (!sync_rcu_preempt_exp_done(rnp)) {
raw_spin_unlock_irqrestore(&rnp->lock, flags);
}
if (rnp->parent == NULL) {
raw_spin_unlock_irqrestore(&rnp->lock, flags);
- if (wake)
+ if (wake) {
+ smp_mb(); /* EGP done before wake_up(). */
wake_up(&sync_rcu_preempt_exp_wq);
+ }
break;
}
mask = rnp->grpmask;
raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
rnp = rnp->parent;
raw_spin_lock(&rnp->lock); /* irqs already disabled */
+ smp_mb__after_unlock_lock();
rnp->expmask &= ~mask;
}
}
int must_wait = 0;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
if (list_empty(&rnp->blkd_tasks)) {
raw_spin_unlock_irqrestore(&rnp->lock, flags);
} else {
/* Initialize ->expmask for all non-leaf rcu_node structures. */
rcu_for_each_nonleaf_node_breadth_first(rsp, rnp) {
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
rnp->expmask = rnp->qsmaskinit;
raw_spin_unlock_irqrestore(&rnp->lock, flags);
}
return 0; /* Nothing left to boost. */
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
/*
* Recheck under the lock: all tasks in need of boosting
if (IS_ERR(t))
return PTR_ERR(t);
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
rnp->boost_kthread_task = t;
raw_spin_unlock_irqrestore(&rnp->lock, flags);
sp.sched_priority = RCU_BOOST_PRIO;
continue;
rnp = rdp->mynode;
raw_spin_lock(&rnp->lock); /* irqs already disabled. */
+ smp_mb__after_unlock_lock();
rcu_accelerate_cbs(rsp, rnp, rdp);
raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
}
/* Wait for callbacks from earlier instance to complete. */
wait_event(oom_callback_wq, atomic_read(&oom_callback_count) == 0);
+ smp_mb(); /* Ensure callback reuse happens after callback invocation. */
/*
* Prevent premature wakeup: ensure that all increments happen
static void __call_rcu_nocb_enqueue(struct rcu_data *rdp,
struct rcu_head *rhp,
struct rcu_head **rhtp,
- int rhcount, int rhcount_lazy)
+ int rhcount, int rhcount_lazy,
+ unsigned long flags)
{
int len;
struct rcu_head **old_rhpp;
}
len = atomic_long_read(&rdp->nocb_q_count);
if (old_rhpp == &rdp->nocb_head) {
- wake_up(&rdp->nocb_wq); /* ... only if queue was empty ... */
+ if (!irqs_disabled_flags(flags)) {
+ wake_up(&rdp->nocb_wq); /* ... if queue was empty ... */
+ trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
+ TPS("WakeEmpty"));
+ } else {
+ rdp->nocb_defer_wakeup = true;
+ trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
+ TPS("WakeEmptyIsDeferred"));
+ }
rdp->qlen_last_fqs_check = 0;
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, TPS("WakeEmpty"));
} else if (len > rdp->qlen_last_fqs_check + qhimark) {
wake_up_process(t); /* ... or if many callbacks queued. */
rdp->qlen_last_fqs_check = LONG_MAX / 2;
* "rcuo" kthread can find it.
*/
static bool __call_rcu_nocb(struct rcu_data *rdp, struct rcu_head *rhp,
- bool lazy)
+ bool lazy, unsigned long flags)
{
if (!rcu_is_nocb_cpu(rdp->cpu))
return 0;
- __call_rcu_nocb_enqueue(rdp, rhp, &rhp->next, 1, lazy);
+ __call_rcu_nocb_enqueue(rdp, rhp, &rhp->next, 1, lazy, flags);
if (__is_kfree_rcu_offset((unsigned long)rhp->func))
trace_rcu_kfree_callback(rdp->rsp->name, rhp,
(unsigned long)rhp->func,
* not a no-CBs CPU.
*/
static bool __maybe_unused rcu_nocb_adopt_orphan_cbs(struct rcu_state *rsp,
- struct rcu_data *rdp)
+ struct rcu_data *rdp,
+ unsigned long flags)
{
long ql = rsp->qlen;
long qll = rsp->qlen_lazy;
/* First, enqueue the donelist, if any. This preserves CB ordering. */
if (rsp->orphan_donelist != NULL) {
__call_rcu_nocb_enqueue(rdp, rsp->orphan_donelist,
- rsp->orphan_donetail, ql, qll);
+ rsp->orphan_donetail, ql, qll, flags);
ql = qll = 0;
rsp->orphan_donelist = NULL;
rsp->orphan_donetail = &rsp->orphan_donelist;
}
if (rsp->orphan_nxtlist != NULL) {
__call_rcu_nocb_enqueue(rdp, rsp->orphan_nxtlist,
- rsp->orphan_nxttail, ql, qll);
+ rsp->orphan_nxttail, ql, qll, flags);
ql = qll = 0;
rsp->orphan_nxtlist = NULL;
rsp->orphan_nxttail = &rsp->orphan_nxtlist;
struct rcu_node *rnp = rdp->mynode;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock();
c = rcu_start_future_gp(rnp, rdp);
raw_spin_unlock_irqrestore(&rnp->lock, flags);
trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
TPS("Sleep"));
wait_event_interruptible(rdp->nocb_wq, rdp->nocb_head);
+ /* Memory barrier provide by xchg() below. */
} else if (firsttime) {
firsttime = 0;
trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
return 0;
}
+/* Is a deferred wakeup of rcu_nocb_kthread() required? */
+static bool rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp)
+{
+ return ACCESS_ONCE(rdp->nocb_defer_wakeup);
+}
+
+/* Do a deferred wakeup of rcu_nocb_kthread(). */
+static void do_nocb_deferred_wakeup(struct rcu_data *rdp)
+{
+ if (!rcu_nocb_need_deferred_wakeup(rdp))
+ return;
+ ACCESS_ONCE(rdp->nocb_defer_wakeup) = false;
+ wake_up(&rdp->nocb_wq);
+ trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, TPS("DeferredWakeEmpty"));
+}
+
/* Initialize per-rcu_data variables for no-CBs CPUs. */
static void __init rcu_boot_init_nocb_percpu_data(struct rcu_data *rdp)
{
}
static bool __call_rcu_nocb(struct rcu_data *rdp, struct rcu_head *rhp,
- bool lazy)
+ bool lazy, unsigned long flags)
{
return 0;
}
static bool __maybe_unused rcu_nocb_adopt_orphan_cbs(struct rcu_state *rsp,
- struct rcu_data *rdp)
+ struct rcu_data *rdp,
+ unsigned long flags)
{
return 0;
}
{
}
+static bool rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp)
+{
+ return false;
+}
+
+static void do_nocb_deferred_wakeup(struct rcu_data *rdp)
+{
+}
+
static void __init rcu_spawn_nocb_kthreads(struct rcu_state *rsp)
{
}
}
#endif /* #else #ifdef CONFIG_NO_HZ_FULL_SYSIDLE */
+
+/*
+ * Is this CPU a NO_HZ_FULL CPU that should ignore RCU so that the
+ * grace-period kthread will do force_quiescent_state() processing?
+ * The idea is to avoid waking up RCU core processing on such a
+ * CPU unless the grace period has extended for too long.
+ *
+ * This code relies on the fact that all NO_HZ_FULL CPUs are also
+ * CONFIG_RCU_NOCB_CPUs.
+ */
+static bool rcu_nohz_full_cpu(struct rcu_state *rsp)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_cpu(smp_processor_id()) &&
+ (!rcu_gp_in_progress(rsp) ||
+ ULONG_CMP_LT(jiffies, ACCESS_ONCE(rsp->gp_start) + HZ)))
+ return 1;
+#endif /* #ifdef CONFIG_NO_HZ_FULL */
+ return 0;
+}
rdp->n_rp_report_qs,
rdp->n_rp_cb_ready,
rdp->n_rp_cpu_needs_gp);
- seq_printf(m, "gpc=%ld gps=%ld nn=%ld\n",
+ seq_printf(m, "gpc=%ld gps=%ld nn=%ld ndw%ld\n",
rdp->n_rp_gp_completed,
rdp->n_rp_gp_started,
+ rdp->n_rp_nocb_defer_wakeup,
rdp->n_rp_need_nothing);
}
STATIC_LOCKDEP_MAP_INIT("rcu_read_lock_sched", &rcu_sched_lock_key);
EXPORT_SYMBOL_GPL(rcu_sched_lock_map);
+static struct lock_class_key rcu_callback_key;
+struct lockdep_map rcu_callback_map =
+ STATIC_LOCKDEP_MAP_INIT("rcu_callback", &rcu_callback_key);
+EXPORT_SYMBOL_GPL(rcu_callback_map);
+
int notrace debug_lockdep_rcu_enabled(void)
{
return rcu_scheduler_active && debug_locks &&
CFLAGS_core.o := $(PROFILING) -fno-omit-frame-pointer
endif
-obj-y += core.o proc.o clock.o cputime.o idle_task.o fair.o rt.o stop_task.o
+obj-y += core.o proc.o clock.o cputime.o
+obj-y += idle_task.o fair.o rt.o deadline.o stop_task.o
obj-y += wait.o completion.o
-obj-$(CONFIG_SMP) += cpupri.o
+obj-$(CONFIG_SMP) += cpupri.o cpudeadline.o
obj-$(CONFIG_SCHED_AUTOGROUP) += auto_group.o
obj-$(CONFIG_SCHEDSTATS) += stats.o
obj-$(CONFIG_SCHED_DEBUG) += debug.o
* at 0 on boot (but people really shouldn't rely on that).
*
* cpu_clock(i) -- can be used from any context, including NMI.
- * sched_clock_cpu(i) -- must be used with local IRQs disabled (implied by NMI)
* local_clock() -- is cpu_clock() on the current cpu.
*
+ * sched_clock_cpu(i)
+ *
* How:
*
* The implementation either uses sched_clock() when
* Furthermore, explicit sleep and wakeup hooks allow us to account for time
* that is otherwise invisible (TSC gets stopped).
*
- *
- * Notes:
- *
- * The !IRQ-safetly of sched_clock() and sched_clock_cpu() comes from things
- * like cpufreq interrupts that can change the base clock (TSC) multiplier
- * and cause funny jumps in time -- although the filtering provided by
- * sched_clock_cpu() should mitigate serious artifacts we cannot rely on it
- * in general since for !CONFIG_HAVE_UNSTABLE_SCHED_CLOCK we fully rely on
- * sched_clock().
*/
#include <linux/spinlock.h>
#include <linux/hardirq.h>
#include <linux/percpu.h>
#include <linux/ktime.h>
#include <linux/sched.h>
+#include <linux/static_key.h>
+#include <linux/workqueue.h>
/*
* Scheduler clock - returns current time in nanosec units.
__read_mostly int sched_clock_running;
#ifdef CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
-__read_mostly int sched_clock_stable;
+static struct static_key __sched_clock_stable = STATIC_KEY_INIT;
+
+int sched_clock_stable(void)
+{
+ if (static_key_false(&__sched_clock_stable))
+ return false;
+ return true;
+}
+
+void set_sched_clock_stable(void)
+{
+ if (!sched_clock_stable())
+ static_key_slow_dec(&__sched_clock_stable);
+}
+
+static void __clear_sched_clock_stable(struct work_struct *work)
+{
+ /* XXX worry about clock continuity */
+ if (sched_clock_stable())
+ static_key_slow_inc(&__sched_clock_stable);
+}
+
+static DECLARE_WORK(sched_clock_work, __clear_sched_clock_stable);
+
+void clear_sched_clock_stable(void)
+{
+ if (keventd_up())
+ schedule_work(&sched_clock_work);
+ else
+ __clear_sched_clock_stable(&sched_clock_work);
+}
struct sched_clock_data {
u64 tick_raw;
struct sched_clock_data *scd;
u64 clock;
- WARN_ON_ONCE(!irqs_disabled());
-
- if (sched_clock_stable)
+ if (sched_clock_stable())
return sched_clock();
if (unlikely(!sched_clock_running))
return 0ull;
+ preempt_disable();
scd = cpu_sdc(cpu);
if (cpu != smp_processor_id())
clock = sched_clock_remote(scd);
else
clock = sched_clock_local(scd);
+ preempt_enable();
return clock;
}
struct sched_clock_data *scd;
u64 now, now_gtod;
- if (sched_clock_stable)
+ if (sched_clock_stable())
return;
if (unlikely(!sched_clock_running))
*/
u64 cpu_clock(int cpu)
{
- u64 clock;
- unsigned long flags;
-
- local_irq_save(flags);
- clock = sched_clock_cpu(cpu);
- local_irq_restore(flags);
+ if (static_key_false(&__sched_clock_stable))
+ return sched_clock_cpu(cpu);
- return clock;
+ return sched_clock();
}
/*
*/
u64 local_clock(void)
{
- u64 clock;
- unsigned long flags;
+ if (static_key_false(&__sched_clock_stable))
+ return sched_clock_cpu(raw_smp_processor_id());
- local_irq_save(flags);
- clock = sched_clock_cpu(smp_processor_id());
- local_irq_restore(flags);
-
- return clock;
+ return sched_clock();
}
#else /* CONFIG_HAVE_UNSTABLE_SCHED_CLOCK */
u64 cpu_clock(int cpu)
{
- return sched_clock_cpu(cpu);
+ return sched_clock();
}
u64 local_clock(void)
{
- return sched_clock_cpu(0);
+ return sched_clock();
}
#endif /* CONFIG_HAVE_UNSTABLE_SCHED_CLOCK */
*/
int sysctl_sched_rt_runtime = 950000;
-
-
/*
* __task_rq_lock - lock the rq @p resides on.
*/
{
int prio;
- if (task_has_rt_policy(p))
+ if (task_has_dl_policy(p))
+ prio = MAX_DL_PRIO-1;
+ else if (task_has_rt_policy(p))
prio = MAX_RT_PRIO-1 - p->rt_priority;
else
prio = __normal_prio(p);
if (prev_class->switched_from)
prev_class->switched_from(rq, p);
p->sched_class->switched_to(rq, p);
- } else if (oldprio != p->prio)
+ } else if (oldprio != p->prio || dl_task(p))
p->sched_class->prio_changed(rq, p, oldprio);
}
* TIF_NEED_RESCHED remotely (for the first time) will also send
* this IPI.
*/
- if (tif_need_resched())
- set_preempt_need_resched();
+ preempt_fold_need_resched();
if (llist_empty(&this_rq()->wake_list)
&& !tick_nohz_full_cpu(smp_processor_id())
memset(&p->se.statistics, 0, sizeof(p->se.statistics));
#endif
+ RB_CLEAR_NODE(&p->dl.rb_node);
+ hrtimer_init(&p->dl.dl_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ p->dl.dl_runtime = p->dl.runtime = 0;
+ p->dl.dl_deadline = p->dl.deadline = 0;
+ p->dl.dl_period = 0;
+ p->dl.flags = 0;
+
INIT_LIST_HEAD(&p->rt.run_list);
#ifdef CONFIG_PREEMPT_NOTIFIERS
/*
* fork()/clone()-time setup:
*/
-void sched_fork(unsigned long clone_flags, struct task_struct *p)
+int sched_fork(unsigned long clone_flags, struct task_struct *p)
{
unsigned long flags;
int cpu = get_cpu();
* Revert to default priority/policy on fork if requested.
*/
if (unlikely(p->sched_reset_on_fork)) {
- if (task_has_rt_policy(p)) {
+ if (task_has_dl_policy(p) || task_has_rt_policy(p)) {
p->policy = SCHED_NORMAL;
p->static_prio = NICE_TO_PRIO(0);
p->rt_priority = 0;
p->sched_reset_on_fork = 0;
}
- if (!rt_prio(p->prio))
+ if (dl_prio(p->prio)) {
+ put_cpu();
+ return -EAGAIN;
+ } else if (rt_prio(p->prio)) {
+ p->sched_class = &rt_sched_class;
+ } else {
p->sched_class = &fair_sched_class;
+ }
if (p->sched_class->task_fork)
p->sched_class->task_fork(p);
init_task_preempt_count(p);
#ifdef CONFIG_SMP
plist_node_init(&p->pushable_tasks, MAX_PRIO);
+ RB_CLEAR_NODE(&p->pushable_dl_tasks);
#endif
put_cpu();
+ return 0;
+}
+
+unsigned long to_ratio(u64 period, u64 runtime)
+{
+ if (runtime == RUNTIME_INF)
+ return 1ULL << 20;
+
+ /*
+ * Doing this here saves a lot of checks in all
+ * the calling paths, and returning zero seems
+ * safe for them anyway.
+ */
+ if (period == 0)
+ return 0;
+
+ return div64_u64(runtime << 20, period);
+}
+
+#ifdef CONFIG_SMP
+inline struct dl_bw *dl_bw_of(int i)
+{
+ return &cpu_rq(i)->rd->dl_bw;
}
+static inline int dl_bw_cpus(int i)
+{
+ struct root_domain *rd = cpu_rq(i)->rd;
+ int cpus = 0;
+
+ for_each_cpu_and(i, rd->span, cpu_active_mask)
+ cpus++;
+
+ return cpus;
+}
+#else
+inline struct dl_bw *dl_bw_of(int i)
+{
+ return &cpu_rq(i)->dl.dl_bw;
+}
+
+static inline int dl_bw_cpus(int i)
+{
+ return 1;
+}
+#endif
+
+static inline
+void __dl_clear(struct dl_bw *dl_b, u64 tsk_bw)
+{
+ dl_b->total_bw -= tsk_bw;
+}
+
+static inline
+void __dl_add(struct dl_bw *dl_b, u64 tsk_bw)
+{
+ dl_b->total_bw += tsk_bw;
+}
+
+static inline
+bool __dl_overflow(struct dl_bw *dl_b, int cpus, u64 old_bw, u64 new_bw)
+{
+ return dl_b->bw != -1 &&
+ dl_b->bw * cpus < dl_b->total_bw - old_bw + new_bw;
+}
+
+/*
+ * We must be sure that accepting a new task (or allowing changing the
+ * parameters of an existing one) is consistent with the bandwidth
+ * constraints. If yes, this function also accordingly updates the currently
+ * allocated bandwidth to reflect the new situation.
+ *
+ * This function is called while holding p's rq->lock.
+ */
+static int dl_overflow(struct task_struct *p, int policy,
+ const struct sched_attr *attr)
+{
+
+ struct dl_bw *dl_b = dl_bw_of(task_cpu(p));
+ u64 period = attr->sched_period;
+ u64 runtime = attr->sched_runtime;
+ u64 new_bw = dl_policy(policy) ? to_ratio(period, runtime) : 0;
+ int cpus, err = -1;
+
+ if (new_bw == p->dl.dl_bw)
+ return 0;
+
+ /*
+ * Either if a task, enters, leave, or stays -deadline but changes
+ * its parameters, we may need to update accordingly the total
+ * allocated bandwidth of the container.
+ */
+ raw_spin_lock(&dl_b->lock);
+ cpus = dl_bw_cpus(task_cpu(p));
+ if (dl_policy(policy) && !task_has_dl_policy(p) &&
+ !__dl_overflow(dl_b, cpus, 0, new_bw)) {
+ __dl_add(dl_b, new_bw);
+ err = 0;
+ } else if (dl_policy(policy) && task_has_dl_policy(p) &&
+ !__dl_overflow(dl_b, cpus, p->dl.dl_bw, new_bw)) {
+ __dl_clear(dl_b, p->dl.dl_bw);
+ __dl_add(dl_b, new_bw);
+ err = 0;
+ } else if (!dl_policy(policy) && task_has_dl_policy(p)) {
+ __dl_clear(dl_b, p->dl.dl_bw);
+ err = 0;
+ }
+ raw_spin_unlock(&dl_b->lock);
+
+ return err;
+}
+
+extern void init_dl_bw(struct dl_bw *dl_b);
+
/*
* wake_up_new_task - wake up a newly created task for the first time.
*
if (unlikely(prev_state == TASK_DEAD)) {
task_numa_free(prev);
+ if (prev->sched_class->task_dead)
+ prev->sched_class->task_dead(prev);
+
/*
* Remove function-return probe instances associated with this
* task and put them back on the free list.
#ifdef CONFIG_SMP
rq->idle_balance = idle_cpu(cpu);
- trigger_load_balance(rq, cpu);
+ trigger_load_balance(rq);
#endif
rq_last_tick_reset(rq);
}
{
/*
* Test if we are atomic. Since do_exit() needs to call into
- * schedule() atomically, we ignore that path for now.
- * Otherwise, whine if we are scheduling when we should not be.
+ * schedule() atomically, we ignore that path. Otherwise whine
+ * if we are scheduling when we should not.
*/
- if (unlikely(in_atomic_preempt_off() && !prev->exit_state))
+ if (unlikely(in_atomic_preempt_off() && prev->state != TASK_DEAD))
__schedule_bug(prev);
rcu_sleep_check();
*/
void rt_mutex_setprio(struct task_struct *p, int prio)
{
- int oldprio, on_rq, running;
+ int oldprio, on_rq, running, enqueue_flag = 0;
struct rq *rq;
const struct sched_class *prev_class;
- BUG_ON(prio < 0 || prio > MAX_PRIO);
+ BUG_ON(prio > MAX_PRIO);
rq = __task_rq_lock(p);
}
trace_sched_pi_setprio(p, prio);
+ p->pi_top_task = rt_mutex_get_top_task(p);
oldprio = p->prio;
prev_class = p->sched_class;
on_rq = p->on_rq;
if (running)
p->sched_class->put_prev_task(rq, p);
- if (rt_prio(prio))
+ /*
+ * Boosting condition are:
+ * 1. -rt task is running and holds mutex A
+ * --> -dl task blocks on mutex A
+ *
+ * 2. -dl task is running and holds mutex A
+ * --> -dl task blocks on mutex A and could preempt the
+ * running task
+ */
+ if (dl_prio(prio)) {
+ if (!dl_prio(p->normal_prio) || (p->pi_top_task &&
+ dl_entity_preempt(&p->pi_top_task->dl, &p->dl))) {
+ p->dl.dl_boosted = 1;
+ p->dl.dl_throttled = 0;
+ enqueue_flag = ENQUEUE_REPLENISH;
+ } else
+ p->dl.dl_boosted = 0;
+ p->sched_class = &dl_sched_class;
+ } else if (rt_prio(prio)) {
+ if (dl_prio(oldprio))
+ p->dl.dl_boosted = 0;
+ if (oldprio < prio)
+ enqueue_flag = ENQUEUE_HEAD;
p->sched_class = &rt_sched_class;
- else
+ } else {
+ if (dl_prio(oldprio))
+ p->dl.dl_boosted = 0;
p->sched_class = &fair_sched_class;
+ }
p->prio = prio;
if (running)
p->sched_class->set_curr_task(rq);
if (on_rq)
- enqueue_task(rq, p, oldprio < prio ? ENQUEUE_HEAD : 0);
+ enqueue_task(rq, p, enqueue_flag);
check_class_changed(rq, p, prev_class, oldprio);
out_unlock:
__task_rq_unlock(rq);
}
#endif
+
void set_user_nice(struct task_struct *p, long nice)
{
int old_prio, delta, on_rq;
* The RT priorities are set via sched_setscheduler(), but we still
* allow the 'normal' nice value to be set - but as expected
* it wont have any effect on scheduling until the task is
- * SCHED_FIFO/SCHED_RR:
+ * SCHED_DEADLINE, SCHED_FIFO or SCHED_RR:
*/
- if (task_has_rt_policy(p)) {
+ if (task_has_dl_policy(p) || task_has_rt_policy(p)) {
p->static_prio = NICE_TO_PRIO(nice);
goto out_unlock;
}
return pid ? find_task_by_vpid(pid) : current;
}
-/* Actually do priority change: must hold rq lock. */
+/*
+ * This function initializes the sched_dl_entity of a newly becoming
+ * SCHED_DEADLINE task.
+ *
+ * Only the static values are considered here, the actual runtime and the
+ * absolute deadline will be properly calculated when the task is enqueued
+ * for the first time with its new policy.
+ */
static void
-__setscheduler(struct rq *rq, struct task_struct *p, int policy, int prio)
+__setparam_dl(struct task_struct *p, const struct sched_attr *attr)
+{
+ struct sched_dl_entity *dl_se = &p->dl;
+
+ init_dl_task_timer(dl_se);
+ dl_se->dl_runtime = attr->sched_runtime;
+ dl_se->dl_deadline = attr->sched_deadline;
+ dl_se->dl_period = attr->sched_period ?: dl_se->dl_deadline;
+ dl_se->flags = attr->sched_flags;
+ dl_se->dl_bw = to_ratio(dl_se->dl_period, dl_se->dl_runtime);
+ dl_se->dl_throttled = 0;
+ dl_se->dl_new = 1;
+}
+
+/* Actually do priority change: must hold pi & rq lock. */
+static void __setscheduler(struct rq *rq, struct task_struct *p,
+ const struct sched_attr *attr)
{
+ int policy = attr->sched_policy;
+
+ if (policy == -1) /* setparam */
+ policy = p->policy;
+
p->policy = policy;
- p->rt_priority = prio;
+
+ if (dl_policy(policy))
+ __setparam_dl(p, attr);
+ else if (fair_policy(policy))
+ p->static_prio = NICE_TO_PRIO(attr->sched_nice);
+
+ /*
+ * __sched_setscheduler() ensures attr->sched_priority == 0 when
+ * !rt_policy. Always setting this ensures that things like
+ * getparam()/getattr() don't report silly values for !rt tasks.
+ */
+ p->rt_priority = attr->sched_priority;
+
p->normal_prio = normal_prio(p);
- /* we are holding p->pi_lock already */
p->prio = rt_mutex_getprio(p);
- if (rt_prio(p->prio))
+
+ if (dl_prio(p->prio))
+ p->sched_class = &dl_sched_class;
+ else if (rt_prio(p->prio))
p->sched_class = &rt_sched_class;
else
p->sched_class = &fair_sched_class;
+
set_load_weight(p);
}
+static void
+__getparam_dl(struct task_struct *p, struct sched_attr *attr)
+{
+ struct sched_dl_entity *dl_se = &p->dl;
+
+ attr->sched_priority = p->rt_priority;
+ attr->sched_runtime = dl_se->dl_runtime;
+ attr->sched_deadline = dl_se->dl_deadline;
+ attr->sched_period = dl_se->dl_period;
+ attr->sched_flags = dl_se->flags;
+}
+
+/*
+ * This function validates the new parameters of a -deadline task.
+ * We ask for the deadline not being zero, and greater or equal
+ * than the runtime, as well as the period of being zero or
+ * greater than deadline. Furthermore, we have to be sure that
+ * user parameters are above the internal resolution (1us); we
+ * check sched_runtime only since it is always the smaller one.
+ */
+static bool
+__checkparam_dl(const struct sched_attr *attr)
+{
+ return attr && attr->sched_deadline != 0 &&
+ (attr->sched_period == 0 ||
+ (s64)(attr->sched_period - attr->sched_deadline) >= 0) &&
+ (s64)(attr->sched_deadline - attr->sched_runtime ) >= 0 &&
+ attr->sched_runtime >= (2 << (DL_SCALE - 1));
+}
+
/*
* check the target process has a UID that matches the current process's
*/
return match;
}
-static int __sched_setscheduler(struct task_struct *p, int policy,
- const struct sched_param *param, bool user)
+static int __sched_setscheduler(struct task_struct *p,
+ const struct sched_attr *attr,
+ bool user)
{
int retval, oldprio, oldpolicy = -1, on_rq, running;
+ int policy = attr->sched_policy;
unsigned long flags;
const struct sched_class *prev_class;
struct rq *rq;
reset_on_fork = p->sched_reset_on_fork;
policy = oldpolicy = p->policy;
} else {
- reset_on_fork = !!(policy & SCHED_RESET_ON_FORK);
- policy &= ~SCHED_RESET_ON_FORK;
+ reset_on_fork = !!(attr->sched_flags & SCHED_FLAG_RESET_ON_FORK);
- if (policy != SCHED_FIFO && policy != SCHED_RR &&
+ if (policy != SCHED_DEADLINE &&
+ policy != SCHED_FIFO && policy != SCHED_RR &&
policy != SCHED_NORMAL && policy != SCHED_BATCH &&
policy != SCHED_IDLE)
return -EINVAL;
}
+ if (attr->sched_flags & ~(SCHED_FLAG_RESET_ON_FORK))
+ return -EINVAL;
+
/*
* Valid priorities for SCHED_FIFO and SCHED_RR are
* 1..MAX_USER_RT_PRIO-1, valid priority for SCHED_NORMAL,
* SCHED_BATCH and SCHED_IDLE is 0.
*/
- if (param->sched_priority < 0 ||
- (p->mm && param->sched_priority > MAX_USER_RT_PRIO-1) ||
- (!p->mm && param->sched_priority > MAX_RT_PRIO-1))
+ if ((p->mm && attr->sched_priority > MAX_USER_RT_PRIO-1) ||
+ (!p->mm && attr->sched_priority > MAX_RT_PRIO-1))
return -EINVAL;
- if (rt_policy(policy) != (param->sched_priority != 0))
+ if ((dl_policy(policy) && !__checkparam_dl(attr)) ||
+ (rt_policy(policy) != (attr->sched_priority != 0)))
return -EINVAL;
/*
* Allow unprivileged RT tasks to decrease priority:
*/
if (user && !capable(CAP_SYS_NICE)) {
+ if (fair_policy(policy)) {
+ if (attr->sched_nice < TASK_NICE(p) &&
+ !can_nice(p, attr->sched_nice))
+ return -EPERM;
+ }
+
if (rt_policy(policy)) {
unsigned long rlim_rtprio =
task_rlimit(p, RLIMIT_RTPRIO);
return -EPERM;
/* can't increase priority */
- if (param->sched_priority > p->rt_priority &&
- param->sched_priority > rlim_rtprio)
+ if (attr->sched_priority > p->rt_priority &&
+ attr->sched_priority > rlim_rtprio)
return -EPERM;
}
/*
* If not changing anything there's no need to proceed further:
*/
- if (unlikely(policy == p->policy && (!rt_policy(policy) ||
- param->sched_priority == p->rt_priority))) {
+ if (unlikely(policy == p->policy)) {
+ if (fair_policy(policy) && attr->sched_nice != TASK_NICE(p))
+ goto change;
+ if (rt_policy(policy) && attr->sched_priority != p->rt_priority)
+ goto change;
+ if (dl_policy(policy))
+ goto change;
+
task_rq_unlock(rq, p, &flags);
return 0;
}
+change:
-#ifdef CONFIG_RT_GROUP_SCHED
if (user) {
+#ifdef CONFIG_RT_GROUP_SCHED
/*
* Do not allow realtime tasks into groups that have no runtime
* assigned.
task_rq_unlock(rq, p, &flags);
return -EPERM;
}
- }
#endif
+#ifdef CONFIG_SMP
+ if (dl_bandwidth_enabled() && dl_policy(policy)) {
+ cpumask_t *span = rq->rd->span;
+
+ /*
+ * Don't allow tasks with an affinity mask smaller than
+ * the entire root_domain to become SCHED_DEADLINE. We
+ * will also fail if there's no bandwidth available.
+ */
+ if (!cpumask_subset(span, &p->cpus_allowed) ||
+ rq->rd->dl_bw.bw == 0) {
+ task_rq_unlock(rq, p, &flags);
+ return -EPERM;
+ }
+ }
+#endif
+ }
/* recheck policy now with rq lock held */
if (unlikely(oldpolicy != -1 && oldpolicy != p->policy)) {
task_rq_unlock(rq, p, &flags);
goto recheck;
}
+
+ /*
+ * If setscheduling to SCHED_DEADLINE (or changing the parameters
+ * of a SCHED_DEADLINE task) we need to check if enough bandwidth
+ * is available.
+ */
+ if ((dl_policy(policy) || dl_task(p)) && dl_overflow(p, policy, attr)) {
+ task_rq_unlock(rq, p, &flags);
+ return -EBUSY;
+ }
+
on_rq = p->on_rq;
running = task_current(rq, p);
if (on_rq)
oldprio = p->prio;
prev_class = p->sched_class;
- __setscheduler(rq, p, policy, param->sched_priority);
+ __setscheduler(rq, p, attr);
if (running)
p->sched_class->set_curr_task(rq);
return 0;
}
+static int _sched_setscheduler(struct task_struct *p, int policy,
+ const struct sched_param *param, bool check)
+{
+ struct sched_attr attr = {
+ .sched_policy = policy,
+ .sched_priority = param->sched_priority,
+ .sched_nice = PRIO_TO_NICE(p->static_prio),
+ };
+
+ /*
+ * Fixup the legacy SCHED_RESET_ON_FORK hack
+ */
+ if (policy & SCHED_RESET_ON_FORK) {
+ attr.sched_flags |= SCHED_FLAG_RESET_ON_FORK;
+ policy &= ~SCHED_RESET_ON_FORK;
+ attr.sched_policy = policy;
+ }
+
+ return __sched_setscheduler(p, &attr, check);
+}
/**
* sched_setscheduler - change the scheduling policy and/or RT priority of a thread.
* @p: the task in question.
int sched_setscheduler(struct task_struct *p, int policy,
const struct sched_param *param)
{
- return __sched_setscheduler(p, policy, param, true);
+ return _sched_setscheduler(p, policy, param, true);
}
EXPORT_SYMBOL_GPL(sched_setscheduler);
+int sched_setattr(struct task_struct *p, const struct sched_attr *attr)
+{
+ return __sched_setscheduler(p, attr, true);
+}
+EXPORT_SYMBOL_GPL(sched_setattr);
+
/**
* sched_setscheduler_nocheck - change the scheduling policy and/or RT priority of a thread from kernelspace.
* @p: the task in question.
int sched_setscheduler_nocheck(struct task_struct *p, int policy,
const struct sched_param *param)
{
- return __sched_setscheduler(p, policy, param, false);
+ return _sched_setscheduler(p, policy, param, false);
}
static int
return retval;
}
+/*
+ * Mimics kernel/events/core.c perf_copy_attr().
+ */
+static int sched_copy_attr(struct sched_attr __user *uattr,
+ struct sched_attr *attr)
+{
+ u32 size;
+ int ret;
+
+ if (!access_ok(VERIFY_WRITE, uattr, SCHED_ATTR_SIZE_VER0))
+ return -EFAULT;
+
+ /*
+ * zero the full structure, so that a short copy will be nice.
+ */
+ memset(attr, 0, sizeof(*attr));
+
+ ret = get_user(size, &uattr->size);
+ if (ret)
+ return ret;
+
+ if (size > PAGE_SIZE) /* silly large */
+ goto err_size;
+
+ if (!size) /* abi compat */
+ size = SCHED_ATTR_SIZE_VER0;
+
+ if (size < SCHED_ATTR_SIZE_VER0)
+ goto err_size;
+
+ /*
+ * If we're handed a bigger struct than we know of,
+ * ensure all the unknown bits are 0 - i.e. new
+ * user-space does not rely on any kernel feature
+ * extensions we dont know about yet.
+ */
+ if (size > sizeof(*attr)) {
+ unsigned char __user *addr;
+ unsigned char __user *end;
+ unsigned char val;
+
+ addr = (void __user *)uattr + sizeof(*attr);
+ end = (void __user *)uattr + size;
+
+ for (; addr < end; addr++) {
+ ret = get_user(val, addr);
+ if (ret)
+ return ret;
+ if (val)
+ goto err_size;
+ }
+ size = sizeof(*attr);
+ }
+
+ ret = copy_from_user(attr, uattr, size);
+ if (ret)
+ return -EFAULT;
+
+ /*
+ * XXX: do we want to be lenient like existing syscalls; or do we want
+ * to be strict and return an error on out-of-bounds values?
+ */
+ attr->sched_nice = clamp(attr->sched_nice, -20, 19);
+
+out:
+ return ret;
+
+err_size:
+ put_user(sizeof(*attr), &uattr->size);
+ ret = -E2BIG;
+ goto out;
+}
+
/**
* sys_sched_setscheduler - set/change the scheduler policy and RT priority
* @pid: the pid in question.
return do_sched_setscheduler(pid, -1, param);
}
+/**
+ * sys_sched_setattr - same as above, but with extended sched_attr
+ * @pid: the pid in question.
+ * @uattr: structure containing the extended parameters.
+ */
+SYSCALL_DEFINE2(sched_setattr, pid_t, pid, struct sched_attr __user *, uattr)
+{
+ struct sched_attr attr;
+ struct task_struct *p;
+ int retval;
+
+ if (!uattr || pid < 0)
+ return -EINVAL;
+
+ if (sched_copy_attr(uattr, &attr))
+ return -EFAULT;
+
+ rcu_read_lock();
+ retval = -ESRCH;
+ p = find_process_by_pid(pid);
+ if (p != NULL)
+ retval = sched_setattr(p, &attr);
+ rcu_read_unlock();
+
+ return retval;
+}
+
/**
* sys_sched_getscheduler - get the policy (scheduling class) of a thread
* @pid: the pid in question.
if (retval)
goto out_unlock;
+ if (task_has_dl_policy(p)) {
+ retval = -EINVAL;
+ goto out_unlock;
+ }
lp.sched_priority = p->rt_priority;
rcu_read_unlock();
return retval;
}
+static int sched_read_attr(struct sched_attr __user *uattr,
+ struct sched_attr *attr,
+ unsigned int usize)
+{
+ int ret;
+
+ if (!access_ok(VERIFY_WRITE, uattr, usize))
+ return -EFAULT;
+
+ /*
+ * If we're handed a smaller struct than we know of,
+ * ensure all the unknown bits are 0 - i.e. old
+ * user-space does not get uncomplete information.
+ */
+ if (usize < sizeof(*attr)) {
+ unsigned char *addr;
+ unsigned char *end;
+
+ addr = (void *)attr + usize;
+ end = (void *)attr + sizeof(*attr);
+
+ for (; addr < end; addr++) {
+ if (*addr)
+ goto err_size;
+ }
+
+ attr->size = usize;
+ }
+
+ ret = copy_to_user(uattr, attr, usize);
+ if (ret)
+ return -EFAULT;
+
+out:
+ return ret;
+
+err_size:
+ ret = -E2BIG;
+ goto out;
+}
+
+/**
+ * sys_sched_getattr - similar to sched_getparam, but with sched_attr
+ * @pid: the pid in question.
+ * @uattr: structure containing the extended parameters.
+ * @size: sizeof(attr) for fwd/bwd comp.
+ */
+SYSCALL_DEFINE3(sched_getattr, pid_t, pid, struct sched_attr __user *, uattr,
+ unsigned int, size)
+{
+ struct sched_attr attr = {
+ .size = sizeof(struct sched_attr),
+ };
+ struct task_struct *p;
+ int retval;
+
+ if (!uattr || pid < 0 || size > PAGE_SIZE ||
+ size < SCHED_ATTR_SIZE_VER0)
+ return -EINVAL;
+
+ rcu_read_lock();
+ p = find_process_by_pid(pid);
+ retval = -ESRCH;
+ if (!p)
+ goto out_unlock;
+
+ retval = security_task_getscheduler(p);
+ if (retval)
+ goto out_unlock;
+
+ attr.sched_policy = p->policy;
+ if (p->sched_reset_on_fork)
+ attr.sched_flags |= SCHED_FLAG_RESET_ON_FORK;
+ if (task_has_dl_policy(p))
+ __getparam_dl(p, &attr);
+ else if (task_has_rt_policy(p))
+ attr.sched_priority = p->rt_priority;
+ else
+ attr.sched_nice = TASK_NICE(p);
+
+ rcu_read_unlock();
+
+ retval = sched_read_attr(uattr, &attr, size);
+ return retval;
+
+out_unlock:
+ rcu_read_unlock();
+ return retval;
+}
+
long sched_setaffinity(pid_t pid, const struct cpumask *in_mask)
{
cpumask_var_t cpus_allowed, new_mask;
if (retval)
goto out_unlock;
+
cpuset_cpus_allowed(p, cpus_allowed);
cpumask_and(new_mask, in_mask, cpus_allowed);
+
+ /*
+ * Since bandwidth control happens on root_domain basis,
+ * if admission test is enabled, we only admit -deadline
+ * tasks allowed to run on all the CPUs in the task's
+ * root_domain.
+ */
+#ifdef CONFIG_SMP
+ if (task_has_dl_policy(p)) {
+ const struct cpumask *span = task_rq(p)->rd->span;
+
+ if (dl_bandwidth_enabled() && !cpumask_subset(span, new_mask)) {
+ retval = -EBUSY;
+ goto out_unlock;
+ }
+ }
+#endif
again:
retval = set_cpus_allowed_ptr(p, new_mask);
}
double_rq_lock(rq, p_rq);
- while (task_rq(p) != p_rq) {
+ if (task_rq(p) != p_rq) {
double_rq_unlock(rq, p_rq);
goto again;
}
case SCHED_RR:
ret = MAX_USER_RT_PRIO-1;
break;
+ case SCHED_DEADLINE:
case SCHED_NORMAL:
case SCHED_BATCH:
case SCHED_IDLE:
case SCHED_RR:
ret = 1;
break;
+ case SCHED_DEADLINE:
case SCHED_NORMAL:
case SCHED_BATCH:
case SCHED_IDLE:
static int sched_cpu_inactive(struct notifier_block *nfb,
unsigned long action, void *hcpu)
{
+ unsigned long flags;
+ long cpu = (long)hcpu;
+
switch (action & ~CPU_TASKS_FROZEN) {
case CPU_DOWN_PREPARE:
- set_cpu_active((long)hcpu, false);
+ set_cpu_active(cpu, false);
+
+ /* explicitly allow suspend */
+ if (!(action & CPU_TASKS_FROZEN)) {
+ struct dl_bw *dl_b = dl_bw_of(cpu);
+ bool overflow;
+ int cpus;
+
+ raw_spin_lock_irqsave(&dl_b->lock, flags);
+ cpus = dl_bw_cpus(cpu);
+ overflow = __dl_overflow(dl_b, cpus, 0, 0);
+ raw_spin_unlock_irqrestore(&dl_b->lock, flags);
+
+ if (overflow)
+ return notifier_from_errno(-EBUSY);
+ }
return NOTIFY_OK;
- default:
- return NOTIFY_DONE;
}
+
+ return NOTIFY_DONE;
}
static int __init migration_init(void)
struct root_domain *rd = container_of(rcu, struct root_domain, rcu);
cpupri_cleanup(&rd->cpupri);
+ cpudl_cleanup(&rd->cpudl);
+ free_cpumask_var(rd->dlo_mask);
free_cpumask_var(rd->rto_mask);
free_cpumask_var(rd->online);
free_cpumask_var(rd->span);
goto out;
if (!alloc_cpumask_var(&rd->online, GFP_KERNEL))
goto free_span;
- if (!alloc_cpumask_var(&rd->rto_mask, GFP_KERNEL))
+ if (!alloc_cpumask_var(&rd->dlo_mask, GFP_KERNEL))
goto free_online;
+ if (!alloc_cpumask_var(&rd->rto_mask, GFP_KERNEL))
+ goto free_dlo_mask;
+
+ init_dl_bw(&rd->dl_bw);
+ if (cpudl_init(&rd->cpudl) != 0)
+ goto free_dlo_mask;
if (cpupri_init(&rd->cpupri) != 0)
goto free_rto_mask;
free_rto_mask:
free_cpumask_var(rd->rto_mask);
+free_dlo_mask:
+ free_cpumask_var(rd->dlo_mask);
free_online:
free_cpumask_var(rd->online);
free_span:
free_cpumask_var(non_isolated_cpus);
init_sched_rt_class();
+ init_sched_dl_class();
}
#else
void __init sched_init_smp(void)
#endif /* CONFIG_CPUMASK_OFFSTACK */
}
+ init_rt_bandwidth(&def_rt_bandwidth,
+ global_rt_period(), global_rt_runtime());
+ init_dl_bandwidth(&def_dl_bandwidth,
+ global_rt_period(), global_rt_runtime());
+
#ifdef CONFIG_SMP
init_defrootdomain();
#endif
- init_rt_bandwidth(&def_rt_bandwidth,
- global_rt_period(), global_rt_runtime());
-
#ifdef CONFIG_RT_GROUP_SCHED
init_rt_bandwidth(&root_task_group.rt_bandwidth,
global_rt_period(), global_rt_runtime());
rq->calc_load_update = jiffies + LOAD_FREQ;
init_cfs_rq(&rq->cfs);
init_rt_rq(&rq->rt, rq);
+ init_dl_rq(&rq->dl, rq);
#ifdef CONFIG_FAIR_GROUP_SCHED
root_task_group.shares = ROOT_TASK_GROUP_LOAD;
INIT_LIST_HEAD(&rq->leaf_cfs_rq_list);
INIT_HLIST_HEAD(&init_task.preempt_notifiers);
#endif
-#ifdef CONFIG_RT_MUTEXES
- plist_head_init(&init_task.pi_waiters);
-#endif
-
/*
* The boot idle thread does lazy MMU switching as well:
*/
static void normalize_task(struct rq *rq, struct task_struct *p)
{
const struct sched_class *prev_class = p->sched_class;
+ struct sched_attr attr = {
+ .sched_policy = SCHED_NORMAL,
+ };
int old_prio = p->prio;
int on_rq;
on_rq = p->on_rq;
if (on_rq)
dequeue_task(rq, p, 0);
- __setscheduler(rq, p, SCHED_NORMAL, 0);
+ __setscheduler(rq, p, &attr);
if (on_rq) {
enqueue_task(rq, p, 0);
resched_task(rq->curr);
p->se.statistics.block_start = 0;
#endif
- if (!rt_task(p)) {
+ if (!dl_task(p) && !rt_task(p)) {
/*
* Renice negative nice level userspace
* tasks back to 0:
}
#endif /* CONFIG_CGROUP_SCHED */
-#if defined(CONFIG_RT_GROUP_SCHED) || defined(CONFIG_CFS_BANDWIDTH)
-static unsigned long to_ratio(u64 period, u64 runtime)
-{
- if (runtime == RUNTIME_INF)
- return 1ULL << 20;
-
- return div64_u64(runtime << 20, period);
-}
-#endif
-
#ifdef CONFIG_RT_GROUP_SCHED
/*
* Ensure that the real time constraints are schedulable.
do_div(rt_period_us, NSEC_PER_USEC);
return rt_period_us;
}
+#endif /* CONFIG_RT_GROUP_SCHED */
+#ifdef CONFIG_RT_GROUP_SCHED
static int sched_rt_global_constraints(void)
{
- u64 runtime, period;
int ret = 0;
- if (sysctl_sched_rt_period <= 0)
- return -EINVAL;
-
- runtime = global_rt_runtime();
- period = global_rt_period();
-
- /*
- * Sanity check on the sysctl variables.
- */
- if (runtime > period && runtime != RUNTIME_INF)
- return -EINVAL;
-
mutex_lock(&rt_constraints_mutex);
read_lock(&tasklist_lock);
ret = __rt_schedulable(NULL, 0, 0);
static int sched_rt_global_constraints(void)
{
unsigned long flags;
- int i;
-
- if (sysctl_sched_rt_period <= 0)
- return -EINVAL;
-
- /*
- * There's always some RT tasks in the root group
- * -- migration, kstopmachine etc..
- */
- if (sysctl_sched_rt_runtime == 0)
- return -EBUSY;
+ int i, ret = 0;
raw_spin_lock_irqsave(&def_rt_bandwidth.rt_runtime_lock, flags);
for_each_possible_cpu(i) {
}
raw_spin_unlock_irqrestore(&def_rt_bandwidth.rt_runtime_lock, flags);
- return 0;
+ return ret;
}
#endif /* CONFIG_RT_GROUP_SCHED */
-int sched_rr_handler(struct ctl_table *table, int write,
- void __user *buffer, size_t *lenp,
- loff_t *ppos)
+static int sched_dl_global_constraints(void)
{
- int ret;
- static DEFINE_MUTEX(mutex);
+ u64 runtime = global_rt_runtime();
+ u64 period = global_rt_period();
+ u64 new_bw = to_ratio(period, runtime);
+ int cpu, ret = 0;
- mutex_lock(&mutex);
- ret = proc_dointvec(table, write, buffer, lenp, ppos);
- /* make sure that internally we keep jiffies */
- /* also, writing zero resets timeslice to default */
- if (!ret && write) {
- sched_rr_timeslice = sched_rr_timeslice <= 0 ?
- RR_TIMESLICE : msecs_to_jiffies(sched_rr_timeslice);
+ /*
+ * Here we want to check the bandwidth not being set to some
+ * value smaller than the currently allocated bandwidth in
+ * any of the root_domains.
+ *
+ * FIXME: Cycling on all the CPUs is overdoing, but simpler than
+ * cycling on root_domains... Discussion on different/better
+ * solutions is welcome!
+ */
+ for_each_possible_cpu(cpu) {
+ struct dl_bw *dl_b = dl_bw_of(cpu);
+
+ raw_spin_lock(&dl_b->lock);
+ if (new_bw < dl_b->total_bw)
+ ret = -EBUSY;
+ raw_spin_unlock(&dl_b->lock);
+
+ if (ret)
+ break;
}
- mutex_unlock(&mutex);
+
return ret;
}
+static void sched_dl_do_global(void)
+{
+ u64 new_bw = -1;
+ int cpu;
+
+ def_dl_bandwidth.dl_period = global_rt_period();
+ def_dl_bandwidth.dl_runtime = global_rt_runtime();
+
+ if (global_rt_runtime() != RUNTIME_INF)
+ new_bw = to_ratio(global_rt_period(), global_rt_runtime());
+
+ /*
+ * FIXME: As above...
+ */
+ for_each_possible_cpu(cpu) {
+ struct dl_bw *dl_b = dl_bw_of(cpu);
+
+ raw_spin_lock(&dl_b->lock);
+ dl_b->bw = new_bw;
+ raw_spin_unlock(&dl_b->lock);
+ }
+}
+
+static int sched_rt_global_validate(void)
+{
+ if (sysctl_sched_rt_period <= 0)
+ return -EINVAL;
+
+ if (sysctl_sched_rt_runtime > sysctl_sched_rt_period)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void sched_rt_do_global(void)
+{
+ def_rt_bandwidth.rt_runtime = global_rt_runtime();
+ def_rt_bandwidth.rt_period = ns_to_ktime(global_rt_period());
+}
+
int sched_rt_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp,
loff_t *ppos)
{
- int ret;
int old_period, old_runtime;
static DEFINE_MUTEX(mutex);
+ int ret;
mutex_lock(&mutex);
old_period = sysctl_sched_rt_period;
ret = proc_dointvec(table, write, buffer, lenp, ppos);
if (!ret && write) {
+ ret = sched_rt_global_validate();
+ if (ret)
+ goto undo;
+
ret = sched_rt_global_constraints();
- if (ret) {
- sysctl_sched_rt_period = old_period;
- sysctl_sched_rt_runtime = old_runtime;
- } else {
- def_rt_bandwidth.rt_runtime = global_rt_runtime();
- def_rt_bandwidth.rt_period =
- ns_to_ktime(global_rt_period());
- }
+ if (ret)
+ goto undo;
+
+ ret = sched_dl_global_constraints();
+ if (ret)
+ goto undo;
+
+ sched_rt_do_global();
+ sched_dl_do_global();
+ }
+ if (0) {
+undo:
+ sysctl_sched_rt_period = old_period;
+ sysctl_sched_rt_runtime = old_runtime;
}
mutex_unlock(&mutex);
return ret;
}
+int sched_rr_handler(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp,
+ loff_t *ppos)
+{
+ int ret;
+ static DEFINE_MUTEX(mutex);
+
+ mutex_lock(&mutex);
+ ret = proc_dointvec(table, write, buffer, lenp, ppos);
+ /* make sure that internally we keep jiffies */
+ /* also, writing zero resets timeslice to default */
+ if (!ret && write) {
+ sched_rr_timeslice = sched_rr_timeslice <= 0 ?
+ RR_TIMESLICE : msecs_to_jiffies(sched_rr_timeslice);
+ }
+ mutex_unlock(&mutex);
+ return ret;
+}
+
#ifdef CONFIG_CGROUP_SCHED
static inline struct task_group *css_tg(struct cgroup_subsys_state *css)
--- /dev/null
+/*
+ * kernel/sched/cpudl.c
+ *
+ * Global CPU deadline management
+ *
+ * Author: Juri Lelli <j.lelli@sssup.it>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+
+#include <linux/gfp.h>
+#include <linux/kernel.h>
+#include "cpudeadline.h"
+
+static inline int parent(int i)
+{
+ return (i - 1) >> 1;
+}
+
+static inline int left_child(int i)
+{
+ return (i << 1) + 1;
+}
+
+static inline int right_child(int i)
+{
+ return (i << 1) + 2;
+}
+
+static inline int dl_time_before(u64 a, u64 b)
+{
+ return (s64)(a - b) < 0;
+}
+
+static void cpudl_exchange(struct cpudl *cp, int a, int b)
+{
+ int cpu_a = cp->elements[a].cpu, cpu_b = cp->elements[b].cpu;
+
+ swap(cp->elements[a], cp->elements[b]);
+ swap(cp->cpu_to_idx[cpu_a], cp->cpu_to_idx[cpu_b]);
+}
+
+static void cpudl_heapify(struct cpudl *cp, int idx)
+{
+ int l, r, largest;
+
+ /* adapted from lib/prio_heap.c */
+ while(1) {
+ l = left_child(idx);
+ r = right_child(idx);
+ largest = idx;
+
+ if ((l < cp->size) && dl_time_before(cp->elements[idx].dl,
+ cp->elements[l].dl))
+ largest = l;
+ if ((r < cp->size) && dl_time_before(cp->elements[largest].dl,
+ cp->elements[r].dl))
+ largest = r;
+ if (largest == idx)
+ break;
+
+ /* Push idx down the heap one level and bump one up */
+ cpudl_exchange(cp, largest, idx);
+ idx = largest;
+ }
+}
+
+static void cpudl_change_key(struct cpudl *cp, int idx, u64 new_dl)
+{
+ WARN_ON(idx > num_present_cpus() || idx == IDX_INVALID);
+
+ if (dl_time_before(new_dl, cp->elements[idx].dl)) {
+ cp->elements[idx].dl = new_dl;
+ cpudl_heapify(cp, idx);
+ } else {
+ cp->elements[idx].dl = new_dl;
+ while (idx > 0 && dl_time_before(cp->elements[parent(idx)].dl,
+ cp->elements[idx].dl)) {
+ cpudl_exchange(cp, idx, parent(idx));
+ idx = parent(idx);
+ }
+ }
+}
+
+static inline int cpudl_maximum(struct cpudl *cp)
+{
+ return cp->elements[0].cpu;
+}
+
+/*
+ * cpudl_find - find the best (later-dl) CPU in the system
+ * @cp: the cpudl max-heap context
+ * @p: the task
+ * @later_mask: a mask to fill in with the selected CPUs (or NULL)
+ *
+ * Returns: int - best CPU (heap maximum if suitable)
+ */
+int cpudl_find(struct cpudl *cp, struct task_struct *p,
+ struct cpumask *later_mask)
+{
+ int best_cpu = -1;
+ const struct sched_dl_entity *dl_se = &p->dl;
+
+ if (later_mask && cpumask_and(later_mask, cp->free_cpus,
+ &p->cpus_allowed) && cpumask_and(later_mask,
+ later_mask, cpu_active_mask)) {
+ best_cpu = cpumask_any(later_mask);
+ goto out;
+ } else if (cpumask_test_cpu(cpudl_maximum(cp), &p->cpus_allowed) &&
+ dl_time_before(dl_se->deadline, cp->elements[0].dl)) {
+ best_cpu = cpudl_maximum(cp);
+ if (later_mask)
+ cpumask_set_cpu(best_cpu, later_mask);
+ }
+
+out:
+ WARN_ON(best_cpu > num_present_cpus() && best_cpu != -1);
+
+ return best_cpu;
+}
+
+/*
+ * cpudl_set - update the cpudl max-heap
+ * @cp: the cpudl max-heap context
+ * @cpu: the target cpu
+ * @dl: the new earliest deadline for this cpu
+ *
+ * Notes: assumes cpu_rq(cpu)->lock is locked
+ *
+ * Returns: (void)
+ */
+void cpudl_set(struct cpudl *cp, int cpu, u64 dl, int is_valid)
+{
+ int old_idx, new_cpu;
+ unsigned long flags;
+
+ WARN_ON(cpu > num_present_cpus());
+
+ raw_spin_lock_irqsave(&cp->lock, flags);
+ old_idx = cp->cpu_to_idx[cpu];
+ if (!is_valid) {
+ /* remove item */
+ if (old_idx == IDX_INVALID) {
+ /*
+ * Nothing to remove if old_idx was invalid.
+ * This could happen if a rq_offline_dl is
+ * called for a CPU without -dl tasks running.
+ */
+ goto out;
+ }
+ new_cpu = cp->elements[cp->size - 1].cpu;
+ cp->elements[old_idx].dl = cp->elements[cp->size - 1].dl;
+ cp->elements[old_idx].cpu = new_cpu;
+ cp->size--;
+ cp->cpu_to_idx[new_cpu] = old_idx;
+ cp->cpu_to_idx[cpu] = IDX_INVALID;
+ while (old_idx > 0 && dl_time_before(
+ cp->elements[parent(old_idx)].dl,
+ cp->elements[old_idx].dl)) {
+ cpudl_exchange(cp, old_idx, parent(old_idx));
+ old_idx = parent(old_idx);
+ }
+ cpumask_set_cpu(cpu, cp->free_cpus);
+ cpudl_heapify(cp, old_idx);
+
+ goto out;
+ }
+
+ if (old_idx == IDX_INVALID) {
+ cp->size++;
+ cp->elements[cp->size - 1].dl = 0;
+ cp->elements[cp->size - 1].cpu = cpu;
+ cp->cpu_to_idx[cpu] = cp->size - 1;
+ cpudl_change_key(cp, cp->size - 1, dl);
+ cpumask_clear_cpu(cpu, cp->free_cpus);
+ } else {
+ cpudl_change_key(cp, old_idx, dl);
+ }
+
+out:
+ raw_spin_unlock_irqrestore(&cp->lock, flags);
+}
+
+/*
+ * cpudl_init - initialize the cpudl structure
+ * @cp: the cpudl max-heap context
+ */
+int cpudl_init(struct cpudl *cp)
+{
+ int i;
+
+ memset(cp, 0, sizeof(*cp));
+ raw_spin_lock_init(&cp->lock);
+ cp->size = 0;
+ for (i = 0; i < NR_CPUS; i++)
+ cp->cpu_to_idx[i] = IDX_INVALID;
+ if (!alloc_cpumask_var(&cp->free_cpus, GFP_KERNEL))
+ return -ENOMEM;
+ cpumask_setall(cp->free_cpus);
+
+ return 0;
+}
+
+/*
+ * cpudl_cleanup - clean up the cpudl structure
+ * @cp: the cpudl max-heap context
+ */
+void cpudl_cleanup(struct cpudl *cp)
+{
+ /*
+ * nothing to do for the moment
+ */
+}
--- /dev/null
+#ifndef _LINUX_CPUDL_H
+#define _LINUX_CPUDL_H
+
+#include <linux/sched.h>
+
+#define IDX_INVALID -1
+
+struct array_item {
+ u64 dl;
+ int cpu;
+};
+
+struct cpudl {
+ raw_spinlock_t lock;
+ int size;
+ int cpu_to_idx[NR_CPUS];
+ struct array_item elements[NR_CPUS];
+ cpumask_var_t free_cpus;
+};
+
+
+#ifdef CONFIG_SMP
+int cpudl_find(struct cpudl *cp, struct task_struct *p,
+ struct cpumask *later_mask);
+void cpudl_set(struct cpudl *cp, int cpu, u64 dl, int is_valid);
+int cpudl_init(struct cpudl *cp);
+void cpudl_cleanup(struct cpudl *cp);
+#else
+#define cpudl_set(cp, cpu, dl) do { } while (0)
+#define cpudl_init() do { } while (0)
+#endif /* CONFIG_SMP */
+
+#endif /* _LINUX_CPUDL_H */
--- /dev/null
+/*
+ * Deadline Scheduling Class (SCHED_DEADLINE)
+ *
+ * Earliest Deadline First (EDF) + Constant Bandwidth Server (CBS).
+ *
+ * Tasks that periodically executes their instances for less than their
+ * runtime won't miss any of their deadlines.
+ * Tasks that are not periodic or sporadic or that tries to execute more
+ * than their reserved bandwidth will be slowed down (and may potentially
+ * miss some of their deadlines), and won't affect any other task.
+ *
+ * Copyright (C) 2012 Dario Faggioli <raistlin@linux.it>,
+ * Juri Lelli <juri.lelli@gmail.com>,
+ * Michael Trimarchi <michael@amarulasolutions.com>,
+ * Fabio Checconi <fchecconi@gmail.com>
+ */
+#include "sched.h"
+
+#include <linux/slab.h>
+
+struct dl_bandwidth def_dl_bandwidth;
+
+static inline struct task_struct *dl_task_of(struct sched_dl_entity *dl_se)
+{
+ return container_of(dl_se, struct task_struct, dl);
+}
+
+static inline struct rq *rq_of_dl_rq(struct dl_rq *dl_rq)
+{
+ return container_of(dl_rq, struct rq, dl);
+}
+
+static inline struct dl_rq *dl_rq_of_se(struct sched_dl_entity *dl_se)
+{
+ struct task_struct *p = dl_task_of(dl_se);
+ struct rq *rq = task_rq(p);
+
+ return &rq->dl;
+}
+
+static inline int on_dl_rq(struct sched_dl_entity *dl_se)
+{
+ return !RB_EMPTY_NODE(&dl_se->rb_node);
+}
+
+static inline int is_leftmost(struct task_struct *p, struct dl_rq *dl_rq)
+{
+ struct sched_dl_entity *dl_se = &p->dl;
+
+ return dl_rq->rb_leftmost == &dl_se->rb_node;
+}
+
+void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime)
+{
+ raw_spin_lock_init(&dl_b->dl_runtime_lock);
+ dl_b->dl_period = period;
+ dl_b->dl_runtime = runtime;
+}
+
+extern unsigned long to_ratio(u64 period, u64 runtime);
+
+void init_dl_bw(struct dl_bw *dl_b)
+{
+ raw_spin_lock_init(&dl_b->lock);
+ raw_spin_lock(&def_dl_bandwidth.dl_runtime_lock);
+ if (global_rt_runtime() == RUNTIME_INF)
+ dl_b->bw = -1;
+ else
+ dl_b->bw = to_ratio(global_rt_period(), global_rt_runtime());
+ raw_spin_unlock(&def_dl_bandwidth.dl_runtime_lock);
+ dl_b->total_bw = 0;
+}
+
+void init_dl_rq(struct dl_rq *dl_rq, struct rq *rq)
+{
+ dl_rq->rb_root = RB_ROOT;
+
+#ifdef CONFIG_SMP
+ /* zero means no -deadline tasks */
+ dl_rq->earliest_dl.curr = dl_rq->earliest_dl.next = 0;
+
+ dl_rq->dl_nr_migratory = 0;
+ dl_rq->overloaded = 0;
+ dl_rq->pushable_dl_tasks_root = RB_ROOT;
+#else
+ init_dl_bw(&dl_rq->dl_bw);
+#endif
+}
+
+#ifdef CONFIG_SMP
+
+static inline int dl_overloaded(struct rq *rq)
+{
+ return atomic_read(&rq->rd->dlo_count);
+}
+
+static inline void dl_set_overload(struct rq *rq)
+{
+ if (!rq->online)
+ return;
+
+ cpumask_set_cpu(rq->cpu, rq->rd->dlo_mask);
+ /*
+ * Must be visible before the overload count is
+ * set (as in sched_rt.c).
+ *
+ * Matched by the barrier in pull_dl_task().
+ */
+ smp_wmb();
+ atomic_inc(&rq->rd->dlo_count);
+}
+
+static inline void dl_clear_overload(struct rq *rq)
+{
+ if (!rq->online)
+ return;
+
+ atomic_dec(&rq->rd->dlo_count);
+ cpumask_clear_cpu(rq->cpu, rq->rd->dlo_mask);
+}
+
+static void update_dl_migration(struct dl_rq *dl_rq)
+{
+ if (dl_rq->dl_nr_migratory && dl_rq->dl_nr_total > 1) {
+ if (!dl_rq->overloaded) {
+ dl_set_overload(rq_of_dl_rq(dl_rq));
+ dl_rq->overloaded = 1;
+ }
+ } else if (dl_rq->overloaded) {
+ dl_clear_overload(rq_of_dl_rq(dl_rq));
+ dl_rq->overloaded = 0;
+ }
+}
+
+static void inc_dl_migration(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
+{
+ struct task_struct *p = dl_task_of(dl_se);
+ dl_rq = &rq_of_dl_rq(dl_rq)->dl;
+
+ dl_rq->dl_nr_total++;
+ if (p->nr_cpus_allowed > 1)
+ dl_rq->dl_nr_migratory++;
+
+ update_dl_migration(dl_rq);
+}
+
+static void dec_dl_migration(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
+{
+ struct task_struct *p = dl_task_of(dl_se);
+ dl_rq = &rq_of_dl_rq(dl_rq)->dl;
+
+ dl_rq->dl_nr_total--;
+ if (p->nr_cpus_allowed > 1)
+ dl_rq->dl_nr_migratory--;
+
+ update_dl_migration(dl_rq);
+}
+
+/*
+ * The list of pushable -deadline task is not a plist, like in
+ * sched_rt.c, it is an rb-tree with tasks ordered by deadline.
+ */
+static void enqueue_pushable_dl_task(struct rq *rq, struct task_struct *p)
+{
+ struct dl_rq *dl_rq = &rq->dl;
+ struct rb_node **link = &dl_rq->pushable_dl_tasks_root.rb_node;
+ struct rb_node *parent = NULL;
+ struct task_struct *entry;
+ int leftmost = 1;
+
+ BUG_ON(!RB_EMPTY_NODE(&p->pushable_dl_tasks));
+
+ while (*link) {
+ parent = *link;
+ entry = rb_entry(parent, struct task_struct,
+ pushable_dl_tasks);
+ if (dl_entity_preempt(&p->dl, &entry->dl))
+ link = &parent->rb_left;
+ else {
+ link = &parent->rb_right;
+ leftmost = 0;
+ }
+ }
+
+ if (leftmost)
+ dl_rq->pushable_dl_tasks_leftmost = &p->pushable_dl_tasks;
+
+ rb_link_node(&p->pushable_dl_tasks, parent, link);
+ rb_insert_color(&p->pushable_dl_tasks, &dl_rq->pushable_dl_tasks_root);
+}
+
+static void dequeue_pushable_dl_task(struct rq *rq, struct task_struct *p)
+{
+ struct dl_rq *dl_rq = &rq->dl;
+
+ if (RB_EMPTY_NODE(&p->pushable_dl_tasks))
+ return;
+
+ if (dl_rq->pushable_dl_tasks_leftmost == &p->pushable_dl_tasks) {
+ struct rb_node *next_node;
+
+ next_node = rb_next(&p->pushable_dl_tasks);
+ dl_rq->pushable_dl_tasks_leftmost = next_node;
+ }
+
+ rb_erase(&p->pushable_dl_tasks, &dl_rq->pushable_dl_tasks_root);
+ RB_CLEAR_NODE(&p->pushable_dl_tasks);
+}
+
+static inline int has_pushable_dl_tasks(struct rq *rq)
+{
+ return !RB_EMPTY_ROOT(&rq->dl.pushable_dl_tasks_root);
+}
+
+static int push_dl_task(struct rq *rq);
+
+#else
+
+static inline
+void enqueue_pushable_dl_task(struct rq *rq, struct task_struct *p)
+{
+}
+
+static inline
+void dequeue_pushable_dl_task(struct rq *rq, struct task_struct *p)
+{
+}
+
+static inline
+void inc_dl_migration(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
+{
+}
+
+static inline
+void dec_dl_migration(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
+{
+}
+
+#endif /* CONFIG_SMP */
+
+static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags);
+static void __dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags);
+static void check_preempt_curr_dl(struct rq *rq, struct task_struct *p,
+ int flags);
+
+/*
+ * We are being explicitly informed that a new instance is starting,
+ * and this means that:
+ * - the absolute deadline of the entity has to be placed at
+ * current time + relative deadline;
+ * - the runtime of the entity has to be set to the maximum value.
+ *
+ * The capability of specifying such event is useful whenever a -deadline
+ * entity wants to (try to!) synchronize its behaviour with the scheduler's
+ * one, and to (try to!) reconcile itself with its own scheduling
+ * parameters.
+ */
+static inline void setup_new_dl_entity(struct sched_dl_entity *dl_se,
+ struct sched_dl_entity *pi_se)
+{
+ struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
+ struct rq *rq = rq_of_dl_rq(dl_rq);
+
+ WARN_ON(!dl_se->dl_new || dl_se->dl_throttled);
+
+ /*
+ * We use the regular wall clock time to set deadlines in the
+ * future; in fact, we must consider execution overheads (time
+ * spent on hardirq context, etc.).
+ */
+ dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
+ dl_se->runtime = pi_se->dl_runtime;
+ dl_se->dl_new = 0;
+}
+
+/*
+ * Pure Earliest Deadline First (EDF) scheduling does not deal with the
+ * possibility of a entity lasting more than what it declared, and thus
+ * exhausting its runtime.
+ *
+ * Here we are interested in making runtime overrun possible, but we do
+ * not want a entity which is misbehaving to affect the scheduling of all
+ * other entities.
+ * Therefore, a budgeting strategy called Constant Bandwidth Server (CBS)
+ * is used, in order to confine each entity within its own bandwidth.
+ *
+ * This function deals exactly with that, and ensures that when the runtime
+ * of a entity is replenished, its deadline is also postponed. That ensures
+ * the overrunning entity can't interfere with other entity in the system and
+ * can't make them miss their deadlines. Reasons why this kind of overruns
+ * could happen are, typically, a entity voluntarily trying to overcome its
+ * runtime, or it just underestimated it during sched_setscheduler_ex().
+ */
+static void replenish_dl_entity(struct sched_dl_entity *dl_se,
+ struct sched_dl_entity *pi_se)
+{
+ struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
+ struct rq *rq = rq_of_dl_rq(dl_rq);
+
+ BUG_ON(pi_se->dl_runtime <= 0);
+
+ /*
+ * This could be the case for a !-dl task that is boosted.
+ * Just go with full inherited parameters.
+ */
+ if (dl_se->dl_deadline == 0) {
+ dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
+ dl_se->runtime = pi_se->dl_runtime;
+ }
+
+ /*
+ * We keep moving the deadline away until we get some
+ * available runtime for the entity. This ensures correct
+ * handling of situations where the runtime overrun is
+ * arbitrary large.
+ */
+ while (dl_se->runtime <= 0) {
+ dl_se->deadline += pi_se->dl_period;
+ dl_se->runtime += pi_se->dl_runtime;
+ }
+
+ /*
+ * At this point, the deadline really should be "in
+ * the future" with respect to rq->clock. If it's
+ * not, we are, for some reason, lagging too much!
+ * Anyway, after having warn userspace abut that,
+ * we still try to keep the things running by
+ * resetting the deadline and the budget of the
+ * entity.
+ */
+ if (dl_time_before(dl_se->deadline, rq_clock(rq))) {
+ static bool lag_once = false;
+
+ if (!lag_once) {
+ lag_once = true;
+ printk_sched("sched: DL replenish lagged to much\n");
+ }
+ dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
+ dl_se->runtime = pi_se->dl_runtime;
+ }
+}
+
+/*
+ * Here we check if --at time t-- an entity (which is probably being
+ * [re]activated or, in general, enqueued) can use its remaining runtime
+ * and its current deadline _without_ exceeding the bandwidth it is
+ * assigned (function returns true if it can't). We are in fact applying
+ * one of the CBS rules: when a task wakes up, if the residual runtime
+ * over residual deadline fits within the allocated bandwidth, then we
+ * can keep the current (absolute) deadline and residual budget without
+ * disrupting the schedulability of the system. Otherwise, we should
+ * refill the runtime and set the deadline a period in the future,
+ * because keeping the current (absolute) deadline of the task would
+ * result in breaking guarantees promised to other tasks.
+ *
+ * This function returns true if:
+ *
+ * runtime / (deadline - t) > dl_runtime / dl_period ,
+ *
+ * IOW we can't recycle current parameters.
+ *
+ * Notice that the bandwidth check is done against the period. For
+ * task with deadline equal to period this is the same of using
+ * dl_deadline instead of dl_period in the equation above.
+ */
+static bool dl_entity_overflow(struct sched_dl_entity *dl_se,
+ struct sched_dl_entity *pi_se, u64 t)
+{
+ u64 left, right;
+
+ /*
+ * left and right are the two sides of the equation above,
+ * after a bit of shuffling to use multiplications instead
+ * of divisions.
+ *
+ * Note that none of the time values involved in the two
+ * multiplications are absolute: dl_deadline and dl_runtime
+ * are the relative deadline and the maximum runtime of each
+ * instance, runtime is the runtime left for the last instance
+ * and (deadline - t), since t is rq->clock, is the time left
+ * to the (absolute) deadline. Even if overflowing the u64 type
+ * is very unlikely to occur in both cases, here we scale down
+ * as we want to avoid that risk at all. Scaling down by 10
+ * means that we reduce granularity to 1us. We are fine with it,
+ * since this is only a true/false check and, anyway, thinking
+ * of anything below microseconds resolution is actually fiction
+ * (but still we want to give the user that illusion >;).
+ */
+ left = (pi_se->dl_period >> DL_SCALE) * (dl_se->runtime >> DL_SCALE);
+ right = ((dl_se->deadline - t) >> DL_SCALE) *
+ (pi_se->dl_runtime >> DL_SCALE);
+
+ return dl_time_before(right, left);
+}
+
+/*
+ * When a -deadline entity is queued back on the runqueue, its runtime and
+ * deadline might need updating.
+ *
+ * The policy here is that we update the deadline of the entity only if:
+ * - the current deadline is in the past,
+ * - using the remaining runtime with the current deadline would make
+ * the entity exceed its bandwidth.
+ */
+static void update_dl_entity(struct sched_dl_entity *dl_se,
+ struct sched_dl_entity *pi_se)
+{
+ struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
+ struct rq *rq = rq_of_dl_rq(dl_rq);
+
+ /*
+ * The arrival of a new instance needs special treatment, i.e.,
+ * the actual scheduling parameters have to be "renewed".
+ */
+ if (dl_se->dl_new) {
+ setup_new_dl_entity(dl_se, pi_se);
+ return;
+ }
+
+ if (dl_time_before(dl_se->deadline, rq_clock(rq)) ||
+ dl_entity_overflow(dl_se, pi_se, rq_clock(rq))) {
+ dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
+ dl_se->runtime = pi_se->dl_runtime;
+ }
+}
+
+/*
+ * If the entity depleted all its runtime, and if we want it to sleep
+ * while waiting for some new execution time to become available, we
+ * set the bandwidth enforcement timer to the replenishment instant
+ * and try to activate it.
+ *
+ * Notice that it is important for the caller to know if the timer
+ * actually started or not (i.e., the replenishment instant is in
+ * the future or in the past).
+ */
+static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
+{
+ struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
+ struct rq *rq = rq_of_dl_rq(dl_rq);
+ ktime_t now, act;
+ ktime_t soft, hard;
+ unsigned long range;
+ s64 delta;
+
+ if (boosted)
+ return 0;
+ /*
+ * We want the timer to fire at the deadline, but considering
+ * that it is actually coming from rq->clock and not from
+ * hrtimer's time base reading.
+ */
+ act = ns_to_ktime(dl_se->deadline);
+ now = hrtimer_cb_get_time(&dl_se->dl_timer);
+ delta = ktime_to_ns(now) - rq_clock(rq);
+ act = ktime_add_ns(act, delta);
+
+ /*
+ * If the expiry time already passed, e.g., because the value
+ * chosen as the deadline is too small, don't even try to
+ * start the timer in the past!
+ */
+ if (ktime_us_delta(act, now) < 0)
+ return 0;
+
+ hrtimer_set_expires(&dl_se->dl_timer, act);
+
+ soft = hrtimer_get_softexpires(&dl_se->dl_timer);
+ hard = hrtimer_get_expires(&dl_se->dl_timer);
+ range = ktime_to_ns(ktime_sub(hard, soft));
+ __hrtimer_start_range_ns(&dl_se->dl_timer, soft,
+ range, HRTIMER_MODE_ABS, 0);
+
+ return hrtimer_active(&dl_se->dl_timer);
+}
+
+/*
+ * This is the bandwidth enforcement timer callback. If here, we know
+ * a task is not on its dl_rq, since the fact that the timer was running
+ * means the task is throttled and needs a runtime replenishment.
+ *
+ * However, what we actually do depends on the fact the task is active,
+ * (it is on its rq) or has been removed from there by a call to
+ * dequeue_task_dl(). In the former case we must issue the runtime
+ * replenishment and add the task back to the dl_rq; in the latter, we just
+ * do nothing but clearing dl_throttled, so that runtime and deadline
+ * updating (and the queueing back to dl_rq) will be done by the
+ * next call to enqueue_task_dl().
+ */
+static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
+{
+ struct sched_dl_entity *dl_se = container_of(timer,
+ struct sched_dl_entity,
+ dl_timer);
+ struct task_struct *p = dl_task_of(dl_se);
+ struct rq *rq = task_rq(p);
+ raw_spin_lock(&rq->lock);
+
+ /*
+ * We need to take care of a possible races here. In fact, the
+ * task might have changed its scheduling policy to something
+ * different from SCHED_DEADLINE or changed its reservation
+ * parameters (through sched_setscheduler()).
+ */
+ if (!dl_task(p) || dl_se->dl_new)
+ goto unlock;
+
+ sched_clock_tick();
+ update_rq_clock(rq);
+ dl_se->dl_throttled = 0;
+ if (p->on_rq) {
+ enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
+ if (task_has_dl_policy(rq->curr))
+ check_preempt_curr_dl(rq, p, 0);
+ else
+ resched_task(rq->curr);
+#ifdef CONFIG_SMP
+ /*
+ * Queueing this task back might have overloaded rq,
+ * check if we need to kick someone away.
+ */
+ if (has_pushable_dl_tasks(rq))
+ push_dl_task(rq);
+#endif
+ }
+unlock:
+ raw_spin_unlock(&rq->lock);
+
+ return HRTIMER_NORESTART;
+}
+
+void init_dl_task_timer(struct sched_dl_entity *dl_se)
+{
+ struct hrtimer *timer = &dl_se->dl_timer;
+
+ if (hrtimer_active(timer)) {
+ hrtimer_try_to_cancel(timer);
+ return;
+ }
+
+ hrtimer_init(timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ timer->function = dl_task_timer;
+}
+
+static
+int dl_runtime_exceeded(struct rq *rq, struct sched_dl_entity *dl_se)
+{
+ int dmiss = dl_time_before(dl_se->deadline, rq_clock(rq));
+ int rorun = dl_se->runtime <= 0;
+
+ if (!rorun && !dmiss)
+ return 0;
+
+ /*
+ * If we are beyond our current deadline and we are still
+ * executing, then we have already used some of the runtime of
+ * the next instance. Thus, if we do not account that, we are
+ * stealing bandwidth from the system at each deadline miss!
+ */
+ if (dmiss) {
+ dl_se->runtime = rorun ? dl_se->runtime : 0;
+ dl_se->runtime -= rq_clock(rq) - dl_se->deadline;
+ }
+
+ return 1;
+}
+
+/*
+ * Update the current task's runtime statistics (provided it is still
+ * a -deadline task and has not been removed from the dl_rq).
+ */
+static void update_curr_dl(struct rq *rq)
+{
+ struct task_struct *curr = rq->curr;
+ struct sched_dl_entity *dl_se = &curr->dl;
+ u64 delta_exec;
+
+ if (!dl_task(curr) || !on_dl_rq(dl_se))
+ return;
+
+ /*
+ * Consumed budget is computed considering the time as
+ * observed by schedulable tasks (excluding time spent
+ * in hardirq context, etc.). Deadlines are instead
+ * computed using hard walltime. This seems to be the more
+ * natural solution, but the full ramifications of this
+ * approach need further study.
+ */
+ delta_exec = rq_clock_task(rq) - curr->se.exec_start;
+ if (unlikely((s64)delta_exec < 0))
+ delta_exec = 0;
+
+ schedstat_set(curr->se.statistics.exec_max,
+ max(curr->se.statistics.exec_max, delta_exec));
+
+ curr->se.sum_exec_runtime += delta_exec;
+ account_group_exec_runtime(curr, delta_exec);
+
+ curr->se.exec_start = rq_clock_task(rq);
+ cpuacct_charge(curr, delta_exec);
+
+ sched_rt_avg_update(rq, delta_exec);
+
+ dl_se->runtime -= delta_exec;
+ if (dl_runtime_exceeded(rq, dl_se)) {
+ __dequeue_task_dl(rq, curr, 0);
+ if (likely(start_dl_timer(dl_se, curr->dl.dl_boosted)))
+ dl_se->dl_throttled = 1;
+ else
+ enqueue_task_dl(rq, curr, ENQUEUE_REPLENISH);
+
+ if (!is_leftmost(curr, &rq->dl))
+ resched_task(curr);
+ }
+
+ /*
+ * Because -- for now -- we share the rt bandwidth, we need to
+ * account our runtime there too, otherwise actual rt tasks
+ * would be able to exceed the shared quota.
+ *
+ * Account to the root rt group for now.
+ *
+ * The solution we're working towards is having the RT groups scheduled
+ * using deadline servers -- however there's a few nasties to figure
+ * out before that can happen.
+ */
+ if (rt_bandwidth_enabled()) {
+ struct rt_rq *rt_rq = &rq->rt;
+
+ raw_spin_lock(&rt_rq->rt_runtime_lock);
+ rt_rq->rt_time += delta_exec;
+ /*
+ * We'll let actual RT tasks worry about the overflow here, we
+ * have our own CBS to keep us inline -- see above.
+ */
+ raw_spin_unlock(&rt_rq->rt_runtime_lock);
+ }
+}
+
+#ifdef CONFIG_SMP
+
+static struct task_struct *pick_next_earliest_dl_task(struct rq *rq, int cpu);
+
+static inline u64 next_deadline(struct rq *rq)
+{
+ struct task_struct *next = pick_next_earliest_dl_task(rq, rq->cpu);
+
+ if (next && dl_prio(next->prio))
+ return next->dl.deadline;
+ else
+ return 0;
+}
+
+static void inc_dl_deadline(struct dl_rq *dl_rq, u64 deadline)
+{
+ struct rq *rq = rq_of_dl_rq(dl_rq);
+
+ if (dl_rq->earliest_dl.curr == 0 ||
+ dl_time_before(deadline, dl_rq->earliest_dl.curr)) {
+ /*
+ * If the dl_rq had no -deadline tasks, or if the new task
+ * has shorter deadline than the current one on dl_rq, we
+ * know that the previous earliest becomes our next earliest,
+ * as the new task becomes the earliest itself.
+ */
+ dl_rq->earliest_dl.next = dl_rq->earliest_dl.curr;
+ dl_rq->earliest_dl.curr = deadline;
+ cpudl_set(&rq->rd->cpudl, rq->cpu, deadline, 1);
+ } else if (dl_rq->earliest_dl.next == 0 ||
+ dl_time_before(deadline, dl_rq->earliest_dl.next)) {
+ /*
+ * On the other hand, if the new -deadline task has a
+ * a later deadline than the earliest one on dl_rq, but
+ * it is earlier than the next (if any), we must
+ * recompute the next-earliest.
+ */
+ dl_rq->earliest_dl.next = next_deadline(rq);
+ }
+}
+
+static void dec_dl_deadline(struct dl_rq *dl_rq, u64 deadline)
+{
+ struct rq *rq = rq_of_dl_rq(dl_rq);
+
+ /*
+ * Since we may have removed our earliest (and/or next earliest)
+ * task we must recompute them.
+ */
+ if (!dl_rq->dl_nr_running) {
+ dl_rq->earliest_dl.curr = 0;
+ dl_rq->earliest_dl.next = 0;
+ cpudl_set(&rq->rd->cpudl, rq->cpu, 0, 0);
+ } else {
+ struct rb_node *leftmost = dl_rq->rb_leftmost;
+ struct sched_dl_entity *entry;
+
+ entry = rb_entry(leftmost, struct sched_dl_entity, rb_node);
+ dl_rq->earliest_dl.curr = entry->deadline;
+ dl_rq->earliest_dl.next = next_deadline(rq);
+ cpudl_set(&rq->rd->cpudl, rq->cpu, entry->deadline, 1);
+ }
+}
+
+#else
+
+static inline void inc_dl_deadline(struct dl_rq *dl_rq, u64 deadline) {}
+static inline void dec_dl_deadline(struct dl_rq *dl_rq, u64 deadline) {}
+
+#endif /* CONFIG_SMP */
+
+static inline
+void inc_dl_tasks(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
+{
+ int prio = dl_task_of(dl_se)->prio;
+ u64 deadline = dl_se->deadline;
+
+ WARN_ON(!dl_prio(prio));
+ dl_rq->dl_nr_running++;
+
+ inc_dl_deadline(dl_rq, deadline);
+ inc_dl_migration(dl_se, dl_rq);
+}
+
+static inline
+void dec_dl_tasks(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
+{
+ int prio = dl_task_of(dl_se)->prio;
+
+ WARN_ON(!dl_prio(prio));
+ WARN_ON(!dl_rq->dl_nr_running);
+ dl_rq->dl_nr_running--;
+
+ dec_dl_deadline(dl_rq, dl_se->deadline);
+ dec_dl_migration(dl_se, dl_rq);
+}
+
+static void __enqueue_dl_entity(struct sched_dl_entity *dl_se)
+{
+ struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
+ struct rb_node **link = &dl_rq->rb_root.rb_node;
+ struct rb_node *parent = NULL;
+ struct sched_dl_entity *entry;
+ int leftmost = 1;
+
+ BUG_ON(!RB_EMPTY_NODE(&dl_se->rb_node));
+
+ while (*link) {
+ parent = *link;
+ entry = rb_entry(parent, struct sched_dl_entity, rb_node);
+ if (dl_time_before(dl_se->deadline, entry->deadline))
+ link = &parent->rb_left;
+ else {
+ link = &parent->rb_right;
+ leftmost = 0;
+ }
+ }
+
+ if (leftmost)
+ dl_rq->rb_leftmost = &dl_se->rb_node;
+
+ rb_link_node(&dl_se->rb_node, parent, link);
+ rb_insert_color(&dl_se->rb_node, &dl_rq->rb_root);
+
+ inc_dl_tasks(dl_se, dl_rq);
+}
+
+static void __dequeue_dl_entity(struct sched_dl_entity *dl_se)
+{
+ struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
+
+ if (RB_EMPTY_NODE(&dl_se->rb_node))
+ return;
+
+ if (dl_rq->rb_leftmost == &dl_se->rb_node) {
+ struct rb_node *next_node;
+
+ next_node = rb_next(&dl_se->rb_node);
+ dl_rq->rb_leftmost = next_node;
+ }
+
+ rb_erase(&dl_se->rb_node, &dl_rq->rb_root);
+ RB_CLEAR_NODE(&dl_se->rb_node);
+
+ dec_dl_tasks(dl_se, dl_rq);
+}
+
+static void
+enqueue_dl_entity(struct sched_dl_entity *dl_se,
+ struct sched_dl_entity *pi_se, int flags)
+{
+ BUG_ON(on_dl_rq(dl_se));
+
+ /*
+ * If this is a wakeup or a new instance, the scheduling
+ * parameters of the task might need updating. Otherwise,
+ * we want a replenishment of its runtime.
+ */
+ if (!dl_se->dl_new && flags & ENQUEUE_REPLENISH)
+ replenish_dl_entity(dl_se, pi_se);
+ else
+ update_dl_entity(dl_se, pi_se);
+
+ __enqueue_dl_entity(dl_se);
+}
+
+static void dequeue_dl_entity(struct sched_dl_entity *dl_se)
+{
+ __dequeue_dl_entity(dl_se);
+}
+
+static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags)
+{
+ struct task_struct *pi_task = rt_mutex_get_top_task(p);
+ struct sched_dl_entity *pi_se = &p->dl;
+
+ /*
+ * Use the scheduling parameters of the top pi-waiter
+ * task if we have one and its (relative) deadline is
+ * smaller than our one... OTW we keep our runtime and
+ * deadline.
+ */
+ if (pi_task && p->dl.dl_boosted && dl_prio(pi_task->normal_prio))
+ pi_se = &pi_task->dl;
+
+ /*
+ * If p is throttled, we do nothing. In fact, if it exhausted
+ * its budget it needs a replenishment and, since it now is on
+ * its rq, the bandwidth timer callback (which clearly has not
+ * run yet) will take care of this.
+ */
+ if (p->dl.dl_throttled)
+ return;
+
+ enqueue_dl_entity(&p->dl, pi_se, flags);
+
+ if (!task_current(rq, p) && p->nr_cpus_allowed > 1)
+ enqueue_pushable_dl_task(rq, p);
+
+ inc_nr_running(rq);
+}
+
+static void __dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags)
+{
+ dequeue_dl_entity(&p->dl);
+ dequeue_pushable_dl_task(rq, p);
+}
+
+static void dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags)
+{
+ update_curr_dl(rq);
+ __dequeue_task_dl(rq, p, flags);
+
+ dec_nr_running(rq);
+}
+
+/*
+ * Yield task semantic for -deadline tasks is:
+ *
+ * get off from the CPU until our next instance, with
+ * a new runtime. This is of little use now, since we
+ * don't have a bandwidth reclaiming mechanism. Anyway,
+ * bandwidth reclaiming is planned for the future, and
+ * yield_task_dl will indicate that some spare budget
+ * is available for other task instances to use it.
+ */
+static void yield_task_dl(struct rq *rq)
+{
+ struct task_struct *p = rq->curr;
+
+ /*
+ * We make the task go to sleep until its current deadline by
+ * forcing its runtime to zero. This way, update_curr_dl() stops
+ * it and the bandwidth timer will wake it up and will give it
+ * new scheduling parameters (thanks to dl_new=1).
+ */
+ if (p->dl.runtime > 0) {
+ rq->curr->dl.dl_new = 1;
+ p->dl.runtime = 0;
+ }
+ update_curr_dl(rq);
+}
+
+#ifdef CONFIG_SMP
+
+static int find_later_rq(struct task_struct *task);
+
+static int
+select_task_rq_dl(struct task_struct *p, int cpu, int sd_flag, int flags)
+{
+ struct task_struct *curr;
+ struct rq *rq;
+
+ if (sd_flag != SD_BALANCE_WAKE && sd_flag != SD_BALANCE_FORK)
+ goto out;
+
+ rq = cpu_rq(cpu);
+
+ rcu_read_lock();
+ curr = ACCESS_ONCE(rq->curr); /* unlocked access */
+
+ /*
+ * If we are dealing with a -deadline task, we must
+ * decide where to wake it up.
+ * If it has a later deadline and the current task
+ * on this rq can't move (provided the waking task
+ * can!) we prefer to send it somewhere else. On the
+ * other hand, if it has a shorter deadline, we
+ * try to make it stay here, it might be important.
+ */
+ if (unlikely(dl_task(curr)) &&
+ (curr->nr_cpus_allowed < 2 ||
+ !dl_entity_preempt(&p->dl, &curr->dl)) &&
+ (p->nr_cpus_allowed > 1)) {
+ int target = find_later_rq(p);
+
+ if (target != -1)
+ cpu = target;
+ }
+ rcu_read_unlock();
+
+out:
+ return cpu;
+}
+
+static void check_preempt_equal_dl(struct rq *rq, struct task_struct *p)
+{
+ /*
+ * Current can't be migrated, useless to reschedule,
+ * let's hope p can move out.
+ */
+ if (rq->curr->nr_cpus_allowed == 1 ||
+ cpudl_find(&rq->rd->cpudl, rq->curr, NULL) == -1)
+ return;
+
+ /*
+ * p is migratable, so let's not schedule it and
+ * see if it is pushed or pulled somewhere else.
+ */
+ if (p->nr_cpus_allowed != 1 &&
+ cpudl_find(&rq->rd->cpudl, p, NULL) != -1)
+ return;
+
+ resched_task(rq->curr);
+}
+
+#endif /* CONFIG_SMP */
+
+/*
+ * Only called when both the current and waking task are -deadline
+ * tasks.
+ */
+static void check_preempt_curr_dl(struct rq *rq, struct task_struct *p,
+ int flags)
+{
+ if (dl_entity_preempt(&p->dl, &rq->curr->dl)) {
+ resched_task(rq->curr);
+ return;
+ }
+
+#ifdef CONFIG_SMP
+ /*
+ * In the unlikely case current and p have the same deadline
+ * let us try to decide what's the best thing to do...
+ */
+ if ((p->dl.deadline == rq->curr->dl.deadline) &&
+ !test_tsk_need_resched(rq->curr))
+ check_preempt_equal_dl(rq, p);
+#endif /* CONFIG_SMP */
+}
+
+#ifdef CONFIG_SCHED_HRTICK
+static void start_hrtick_dl(struct rq *rq, struct task_struct *p)
+{
+ s64 delta = p->dl.dl_runtime - p->dl.runtime;
+
+ if (delta > 10000)
+ hrtick_start(rq, p->dl.runtime);
+}
+#endif
+
+static struct sched_dl_entity *pick_next_dl_entity(struct rq *rq,
+ struct dl_rq *dl_rq)
+{
+ struct rb_node *left = dl_rq->rb_leftmost;
+
+ if (!left)
+ return NULL;
+
+ return rb_entry(left, struct sched_dl_entity, rb_node);
+}
+
+struct task_struct *pick_next_task_dl(struct rq *rq)
+{
+ struct sched_dl_entity *dl_se;
+ struct task_struct *p;
+ struct dl_rq *dl_rq;
+
+ dl_rq = &rq->dl;
+
+ if (unlikely(!dl_rq->dl_nr_running))
+ return NULL;
+
+ dl_se = pick_next_dl_entity(rq, dl_rq);
+ BUG_ON(!dl_se);
+
+ p = dl_task_of(dl_se);
+ p->se.exec_start = rq_clock_task(rq);
+
+ /* Running task will never be pushed. */
+ dequeue_pushable_dl_task(rq, p);
+
+#ifdef CONFIG_SCHED_HRTICK
+ if (hrtick_enabled(rq))
+ start_hrtick_dl(rq, p);
+#endif
+
+#ifdef CONFIG_SMP
+ rq->post_schedule = has_pushable_dl_tasks(rq);
+#endif /* CONFIG_SMP */
+
+ return p;
+}
+
+static void put_prev_task_dl(struct rq *rq, struct task_struct *p)
+{
+ update_curr_dl(rq);
+
+ if (on_dl_rq(&p->dl) && p->nr_cpus_allowed > 1)
+ enqueue_pushable_dl_task(rq, p);
+}
+
+static void task_tick_dl(struct rq *rq, struct task_struct *p, int queued)
+{
+ update_curr_dl(rq);
+
+#ifdef CONFIG_SCHED_HRTICK
+ if (hrtick_enabled(rq) && queued && p->dl.runtime > 0)
+ start_hrtick_dl(rq, p);
+#endif
+}
+
+static void task_fork_dl(struct task_struct *p)
+{
+ /*
+ * SCHED_DEADLINE tasks cannot fork and this is achieved through
+ * sched_fork()
+ */
+}
+
+static void task_dead_dl(struct task_struct *p)
+{
+ struct hrtimer *timer = &p->dl.dl_timer;
+ struct dl_bw *dl_b = dl_bw_of(task_cpu(p));
+
+ /*
+ * Since we are TASK_DEAD we won't slip out of the domain!
+ */
+ raw_spin_lock_irq(&dl_b->lock);
+ dl_b->total_bw -= p->dl.dl_bw;
+ raw_spin_unlock_irq(&dl_b->lock);
+
+ hrtimer_cancel(timer);
+}
+
+static void set_curr_task_dl(struct rq *rq)
+{
+ struct task_struct *p = rq->curr;
+
+ p->se.exec_start = rq_clock_task(rq);
+
+ /* You can't push away the running task */
+ dequeue_pushable_dl_task(rq, p);
+}
+
+#ifdef CONFIG_SMP
+
+/* Only try algorithms three times */
+#define DL_MAX_TRIES 3
+
+static int pick_dl_task(struct rq *rq, struct task_struct *p, int cpu)
+{
+ if (!task_running(rq, p) &&
+ (cpu < 0 || cpumask_test_cpu(cpu, &p->cpus_allowed)) &&
+ (p->nr_cpus_allowed > 1))
+ return 1;
+
+ return 0;
+}
+
+/* Returns the second earliest -deadline task, NULL otherwise */
+static struct task_struct *pick_next_earliest_dl_task(struct rq *rq, int cpu)
+{
+ struct rb_node *next_node = rq->dl.rb_leftmost;
+ struct sched_dl_entity *dl_se;
+ struct task_struct *p = NULL;
+
+next_node:
+ next_node = rb_next(next_node);
+ if (next_node) {
+ dl_se = rb_entry(next_node, struct sched_dl_entity, rb_node);
+ p = dl_task_of(dl_se);
+
+ if (pick_dl_task(rq, p, cpu))
+ return p;
+
+ goto next_node;
+ }
+
+ return NULL;
+}
+
+static DEFINE_PER_CPU(cpumask_var_t, local_cpu_mask_dl);
+
+static int find_later_rq(struct task_struct *task)
+{
+ struct sched_domain *sd;
+ struct cpumask *later_mask = __get_cpu_var(local_cpu_mask_dl);
+ int this_cpu = smp_processor_id();
+ int best_cpu, cpu = task_cpu(task);
+
+ /* Make sure the mask is initialized first */
+ if (unlikely(!later_mask))
+ return -1;
+
+ if (task->nr_cpus_allowed == 1)
+ return -1;
+
+ best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
+ task, later_mask);
+ if (best_cpu == -1)
+ return -1;
+
+ /*
+ * If we are here, some target has been found,
+ * the most suitable of which is cached in best_cpu.
+ * This is, among the runqueues where the current tasks
+ * have later deadlines than the task's one, the rq
+ * with the latest possible one.
+ *
+ * Now we check how well this matches with task's
+ * affinity and system topology.
+ *
+ * The last cpu where the task run is our first
+ * guess, since it is most likely cache-hot there.
+ */
+ if (cpumask_test_cpu(cpu, later_mask))
+ return cpu;
+ /*
+ * Check if this_cpu is to be skipped (i.e., it is
+ * not in the mask) or not.
+ */
+ if (!cpumask_test_cpu(this_cpu, later_mask))
+ this_cpu = -1;
+
+ rcu_read_lock();
+ for_each_domain(cpu, sd) {
+ if (sd->flags & SD_WAKE_AFFINE) {
+
+ /*
+ * If possible, preempting this_cpu is
+ * cheaper than migrating.
+ */
+ if (this_cpu != -1 &&
+ cpumask_test_cpu(this_cpu, sched_domain_span(sd))) {
+ rcu_read_unlock();
+ return this_cpu;
+ }
+
+ /*
+ * Last chance: if best_cpu is valid and is
+ * in the mask, that becomes our choice.
+ */
+ if (best_cpu < nr_cpu_ids &&
+ cpumask_test_cpu(best_cpu, sched_domain_span(sd))) {
+ rcu_read_unlock();
+ return best_cpu;
+ }
+ }
+ }
+ rcu_read_unlock();
+
+ /*
+ * At this point, all our guesses failed, we just return
+ * 'something', and let the caller sort the things out.
+ */
+ if (this_cpu != -1)
+ return this_cpu;
+
+ cpu = cpumask_any(later_mask);
+ if (cpu < nr_cpu_ids)
+ return cpu;
+
+ return -1;
+}
+
+/* Locks the rq it finds */
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq)
+{
+ struct rq *later_rq = NULL;
+ int tries;
+ int cpu;
+
+ for (tries = 0; tries < DL_MAX_TRIES; tries++) {
+ cpu = find_later_rq(task);
+
+ if ((cpu == -1) || (cpu == rq->cpu))
+ break;
+
+ later_rq = cpu_rq(cpu);
+
+ /* Retry if something changed. */
+ if (double_lock_balance(rq, later_rq)) {
+ if (unlikely(task_rq(task) != rq ||
+ !cpumask_test_cpu(later_rq->cpu,
+ &task->cpus_allowed) ||
+ task_running(rq, task) || !task->on_rq)) {
+ double_unlock_balance(rq, later_rq);
+ later_rq = NULL;
+ break;
+ }
+ }
+
+ /*
+ * If the rq we found has no -deadline task, or
+ * its earliest one has a later deadline than our
+ * task, the rq is a good one.
+ */
+ if (!later_rq->dl.dl_nr_running ||
+ dl_time_before(task->dl.deadline,
+ later_rq->dl.earliest_dl.curr))
+ break;
+
+ /* Otherwise we try again. */
+ double_unlock_balance(rq, later_rq);
+ later_rq = NULL;
+ }
+
+ return later_rq;
+}
+
+static struct task_struct *pick_next_pushable_dl_task(struct rq *rq)
+{
+ struct task_struct *p;
+
+ if (!has_pushable_dl_tasks(rq))
+ return NULL;
+
+ p = rb_entry(rq->dl.pushable_dl_tasks_leftmost,
+ struct task_struct, pushable_dl_tasks);
+
+ BUG_ON(rq->cpu != task_cpu(p));
+ BUG_ON(task_current(rq, p));
+ BUG_ON(p->nr_cpus_allowed <= 1);
+
+ BUG_ON(!p->on_rq);
+ BUG_ON(!dl_task(p));
+
+ return p;
+}
+
+/*
+ * See if the non running -deadline tasks on this rq
+ * can be sent to some other CPU where they can preempt
+ * and start executing.
+ */
+static int push_dl_task(struct rq *rq)
+{
+ struct task_struct *next_task;
+ struct rq *later_rq;
+
+ if (!rq->dl.overloaded)
+ return 0;
+
+ next_task = pick_next_pushable_dl_task(rq);
+ if (!next_task)
+ return 0;
+
+retry:
+ if (unlikely(next_task == rq->curr)) {
+ WARN_ON(1);
+ return 0;
+ }
+
+ /*
+ * If next_task preempts rq->curr, and rq->curr
+ * can move away, it makes sense to just reschedule
+ * without going further in pushing next_task.
+ */
+ if (dl_task(rq->curr) &&
+ dl_time_before(next_task->dl.deadline, rq->curr->dl.deadline) &&
+ rq->curr->nr_cpus_allowed > 1) {
+ resched_task(rq->curr);
+ return 0;
+ }
+
+ /* We might release rq lock */
+ get_task_struct(next_task);
+
+ /* Will lock the rq it'll find */
+ later_rq = find_lock_later_rq(next_task, rq);
+ if (!later_rq) {
+ struct task_struct *task;
+
+ /*
+ * We must check all this again, since
+ * find_lock_later_rq releases rq->lock and it is
+ * then possible that next_task has migrated.
+ */
+ task = pick_next_pushable_dl_task(rq);
+ if (task_cpu(next_task) == rq->cpu && task == next_task) {
+ /*
+ * The task is still there. We don't try
+ * again, some other cpu will pull it when ready.
+ */
+ dequeue_pushable_dl_task(rq, next_task);
+ goto out;
+ }
+
+ if (!task)
+ /* No more tasks */
+ goto out;
+
+ put_task_struct(next_task);
+ next_task = task;
+ goto retry;
+ }
+
+ deactivate_task(rq, next_task, 0);
+ set_task_cpu(next_task, later_rq->cpu);
+ activate_task(later_rq, next_task, 0);
+
+ resched_task(later_rq->curr);
+
+ double_unlock_balance(rq, later_rq);
+
+out:
+ put_task_struct(next_task);
+
+ return 1;
+}
+
+static void push_dl_tasks(struct rq *rq)
+{
+ /* Terminates as it moves a -deadline task */
+ while (push_dl_task(rq))
+ ;
+}
+
+static int pull_dl_task(struct rq *this_rq)
+{
+ int this_cpu = this_rq->cpu, ret = 0, cpu;
+ struct task_struct *p;
+ struct rq *src_rq;
+ u64 dmin = LONG_MAX;
+
+ if (likely(!dl_overloaded(this_rq)))
+ return 0;
+
+ /*
+ * Match the barrier from dl_set_overloaded; this guarantees that if we
+ * see overloaded we must also see the dlo_mask bit.
+ */
+ smp_rmb();
+
+ for_each_cpu(cpu, this_rq->rd->dlo_mask) {
+ if (this_cpu == cpu)
+ continue;
+
+ src_rq = cpu_rq(cpu);
+
+ /*
+ * It looks racy, abd it is! However, as in sched_rt.c,
+ * we are fine with this.
+ */
+ if (this_rq->dl.dl_nr_running &&
+ dl_time_before(this_rq->dl.earliest_dl.curr,
+ src_rq->dl.earliest_dl.next))
+ continue;
+
+ /* Might drop this_rq->lock */
+ double_lock_balance(this_rq, src_rq);
+
+ /*
+ * If there are no more pullable tasks on the
+ * rq, we're done with it.
+ */
+ if (src_rq->dl.dl_nr_running <= 1)
+ goto skip;
+
+ p = pick_next_earliest_dl_task(src_rq, this_cpu);
+
+ /*
+ * We found a task to be pulled if:
+ * - it preempts our current (if there's one),
+ * - it will preempt the last one we pulled (if any).
+ */
+ if (p && dl_time_before(p->dl.deadline, dmin) &&
+ (!this_rq->dl.dl_nr_running ||
+ dl_time_before(p->dl.deadline,
+ this_rq->dl.earliest_dl.curr))) {
+ WARN_ON(p == src_rq->curr);
+ WARN_ON(!p->on_rq);
+
+ /*
+ * Then we pull iff p has actually an earlier
+ * deadline than the current task of its runqueue.
+ */
+ if (dl_time_before(p->dl.deadline,
+ src_rq->curr->dl.deadline))
+ goto skip;
+
+ ret = 1;
+
+ deactivate_task(src_rq, p, 0);
+ set_task_cpu(p, this_cpu);
+ activate_task(this_rq, p, 0);
+ dmin = p->dl.deadline;
+
+ /* Is there any other task even earlier? */
+ }
+skip:
+ double_unlock_balance(this_rq, src_rq);
+ }
+
+ return ret;
+}
+
+static void pre_schedule_dl(struct rq *rq, struct task_struct *prev)
+{
+ /* Try to pull other tasks here */
+ if (dl_task(prev))
+ pull_dl_task(rq);
+}
+
+static void post_schedule_dl(struct rq *rq)
+{
+ push_dl_tasks(rq);
+}
+
+/*
+ * Since the task is not running and a reschedule is not going to happen
+ * anytime soon on its runqueue, we try pushing it away now.
+ */
+static void task_woken_dl(struct rq *rq, struct task_struct *p)
+{
+ if (!task_running(rq, p) &&
+ !test_tsk_need_resched(rq->curr) &&
+ has_pushable_dl_tasks(rq) &&
+ p->nr_cpus_allowed > 1 &&
+ dl_task(rq->curr) &&
+ (rq->curr->nr_cpus_allowed < 2 ||
+ dl_entity_preempt(&rq->curr->dl, &p->dl))) {
+ push_dl_tasks(rq);
+ }
+}
+
+static void set_cpus_allowed_dl(struct task_struct *p,
+ const struct cpumask *new_mask)
+{
+ struct rq *rq;
+ int weight;
+
+ BUG_ON(!dl_task(p));
+
+ /*
+ * Update only if the task is actually running (i.e.,
+ * it is on the rq AND it is not throttled).
+ */
+ if (!on_dl_rq(&p->dl))
+ return;
+
+ weight = cpumask_weight(new_mask);
+
+ /*
+ * Only update if the process changes its state from whether it
+ * can migrate or not.
+ */
+ if ((p->nr_cpus_allowed > 1) == (weight > 1))
+ return;
+
+ rq = task_rq(p);
+
+ /*
+ * The process used to be able to migrate OR it can now migrate
+ */
+ if (weight <= 1) {
+ if (!task_current(rq, p))
+ dequeue_pushable_dl_task(rq, p);
+ BUG_ON(!rq->dl.dl_nr_migratory);
+ rq->dl.dl_nr_migratory--;
+ } else {
+ if (!task_current(rq, p))
+ enqueue_pushable_dl_task(rq, p);
+ rq->dl.dl_nr_migratory++;
+ }
+
+ update_dl_migration(&rq->dl);
+}
+
+/* Assumes rq->lock is held */
+static void rq_online_dl(struct rq *rq)
+{
+ if (rq->dl.overloaded)
+ dl_set_overload(rq);
+
+ if (rq->dl.dl_nr_running > 0)
+ cpudl_set(&rq->rd->cpudl, rq->cpu, rq->dl.earliest_dl.curr, 1);
+}
+
+/* Assumes rq->lock is held */
+static void rq_offline_dl(struct rq *rq)
+{
+ if (rq->dl.overloaded)
+ dl_clear_overload(rq);
+
+ cpudl_set(&rq->rd->cpudl, rq->cpu, 0, 0);
+}
+
+void init_sched_dl_class(void)
+{
+ unsigned int i;
+
+ for_each_possible_cpu(i)
+ zalloc_cpumask_var_node(&per_cpu(local_cpu_mask_dl, i),
+ GFP_KERNEL, cpu_to_node(i));
+}
+
+#endif /* CONFIG_SMP */
+
+static void switched_from_dl(struct rq *rq, struct task_struct *p)
+{
+ if (hrtimer_active(&p->dl.dl_timer) && !dl_policy(p->policy))
+ hrtimer_try_to_cancel(&p->dl.dl_timer);
+
+#ifdef CONFIG_SMP
+ /*
+ * Since this might be the only -deadline task on the rq,
+ * this is the right place to try to pull some other one
+ * from an overloaded cpu, if any.
+ */
+ if (!rq->dl.dl_nr_running)
+ pull_dl_task(rq);
+#endif
+}
+
+/*
+ * When switching to -deadline, we may overload the rq, then
+ * we try to push someone off, if possible.
+ */
+static void switched_to_dl(struct rq *rq, struct task_struct *p)
+{
+ int check_resched = 1;
+
+ /*
+ * If p is throttled, don't consider the possibility
+ * of preempting rq->curr, the check will be done right
+ * after its runtime will get replenished.
+ */
+ if (unlikely(p->dl.dl_throttled))
+ return;
+
+ if (p->on_rq || rq->curr != p) {
+#ifdef CONFIG_SMP
+ if (rq->dl.overloaded && push_dl_task(rq) && rq != task_rq(p))
+ /* Only reschedule if pushing failed */
+ check_resched = 0;
+#endif /* CONFIG_SMP */
+ if (check_resched && task_has_dl_policy(rq->curr))
+ check_preempt_curr_dl(rq, p, 0);
+ }
+}
+
+/*
+ * If the scheduling parameters of a -deadline task changed,
+ * a push or pull operation might be needed.
+ */
+static void prio_changed_dl(struct rq *rq, struct task_struct *p,
+ int oldprio)
+{
+ if (p->on_rq || rq->curr == p) {
+#ifdef CONFIG_SMP
+ /*
+ * This might be too much, but unfortunately
+ * we don't have the old deadline value, and
+ * we can't argue if the task is increasing
+ * or lowering its prio, so...
+ */
+ if (!rq->dl.overloaded)
+ pull_dl_task(rq);
+
+ /*
+ * If we now have a earlier deadline task than p,
+ * then reschedule, provided p is still on this
+ * runqueue.
+ */
+ if (dl_time_before(rq->dl.earliest_dl.curr, p->dl.deadline) &&
+ rq->curr == p)
+ resched_task(p);
+#else
+ /*
+ * Again, we don't know if p has a earlier
+ * or later deadline, so let's blindly set a
+ * (maybe not needed) rescheduling point.
+ */
+ resched_task(p);
+#endif /* CONFIG_SMP */
+ } else
+ switched_to_dl(rq, p);
+}
+
+const struct sched_class dl_sched_class = {
+ .next = &rt_sched_class,
+ .enqueue_task = enqueue_task_dl,
+ .dequeue_task = dequeue_task_dl,
+ .yield_task = yield_task_dl,
+
+ .check_preempt_curr = check_preempt_curr_dl,
+
+ .pick_next_task = pick_next_task_dl,
+ .put_prev_task = put_prev_task_dl,
+
+#ifdef CONFIG_SMP
+ .select_task_rq = select_task_rq_dl,
+ .set_cpus_allowed = set_cpus_allowed_dl,
+ .rq_online = rq_online_dl,
+ .rq_offline = rq_offline_dl,
+ .pre_schedule = pre_schedule_dl,
+ .post_schedule = post_schedule_dl,
+ .task_woken = task_woken_dl,
+#endif
+
+ .set_curr_task = set_curr_task_dl,
+ .task_tick = task_tick_dl,
+ .task_fork = task_fork_dl,
+ .task_dead = task_dead_dl,
+
+ .prio_changed = prio_changed_dl,
+ .switched_from = switched_from_dl,
+ .switched_to = switched_to_dl,
+};
0LL, 0LL, 0LL, 0L, 0LL, 0L, 0LL, 0L);
#endif
#ifdef CONFIG_NUMA_BALANCING
- SEQ_printf(m, " %d", cpu_to_node(task_cpu(p)));
+ SEQ_printf(m, " %d", task_node(p));
#endif
#ifdef CONFIG_CGROUP_SCHED
SEQ_printf(m, " %s", task_group_path(task_group(p)));
PN(cpu_clk);
P(jiffies);
#ifdef CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
- P(sched_clock_stable);
+ P(sched_clock_stable());
#endif
#undef PN
#undef P
return max(smin, smax);
}
-/*
- * Once a preferred node is selected the scheduler balancer will prefer moving
- * a task to that node for sysctl_numa_balancing_settle_count number of PTE
- * scans. This will give the process the chance to accumulate more faults on
- * the preferred node but still allow the scheduler to move the task again if
- * the nodes CPUs are overloaded.
- */
-unsigned int sysctl_numa_balancing_settle_count __read_mostly = 4;
-
static void account_numa_enqueue(struct rq *rq, struct task_struct *p)
{
rq->nr_numa_running += (p->numa_preferred_nid != -1);
if (!p->numa_group)
return 0;
- return p->numa_group->faults[2*nid] + p->numa_group->faults[2*nid+1];
+ return p->numa_group->faults[task_faults_idx(nid, 0)] +
+ p->numa_group->faults[task_faults_idx(nid, 1)];
}
/*
struct numa_stats src_stats, dst_stats;
- int imbalance_pct, idx;
+ int imbalance_pct;
struct task_struct *best_task;
long best_imp;
* elsewhere, so there is no point in (re)trying.
*/
if (unlikely(!sd)) {
- p->numa_preferred_nid = cpu_to_node(task_cpu(p));
+ p->numa_preferred_nid = task_node(p);
return -EINVAL;
}
p->numa_migrate_retry = jiffies + HZ;
/* Success if task is already running on preferred CPU */
- if (cpu_to_node(task_cpu(p)) == p->numa_preferred_nid)
+ if (task_node(p) == p->numa_preferred_nid)
return;
/* Otherwise, try migrate to a CPU on the preferred node */
* scanning faster if shared accesses dominate as it may
* simply bounce migrations uselessly
*/
- period_slot = DIV_ROUND_UP(diff, NUMA_PERIOD_SLOTS);
ratio = DIV_ROUND_UP(private * NUMA_PERIOD_SLOTS, (private + shared));
diff = (diff * ratio) / NUMA_PERIOD_SLOTS;
}
{
struct sched_entity *se = tg->se[cpu];
- if (!tg->parent || !wl) /* the trivial, non-cgroup case */
+ if (!tg->parent) /* the trivial, non-cgroup case */
return wl;
for_each_sched_entity(se) {
*/
static struct sched_group *
find_idlest_group(struct sched_domain *sd, struct task_struct *p,
- int this_cpu, int load_idx)
+ int this_cpu, int sd_flag)
{
struct sched_group *idlest = NULL, *group = sd->groups;
unsigned long min_load = ULONG_MAX, this_load = 0;
+ int load_idx = sd->forkexec_idx;
int imbalance = 100 + (sd->imbalance_pct-100)/2;
+ if (sd_flag & SD_BALANCE_WAKE)
+ load_idx = sd->wake_idx;
+
do {
unsigned long load, avg_load;
int local_group;
}
while (sd) {
- int load_idx = sd->forkexec_idx;
struct sched_group *group;
int weight;
continue;
}
- if (sd_flag & SD_BALANCE_WAKE)
- load_idx = sd->wake_idx;
-
- group = find_idlest_group(sd, p, cpu, load_idx);
+ group = find_idlest_group(sd, p, cpu, sd_flag);
if (!group) {
sd = sd->child;
continue;
struct sched_group *group, int load_idx,
int local_group, struct sg_lb_stats *sgs)
{
- unsigned long nr_running;
unsigned long load;
int i;
for_each_cpu_and(i, sched_group_cpus(group), env->cpus) {
struct rq *rq = cpu_rq(i);
- nr_running = rq->nr_running;
-
/* Bias balancing toward cpus of our domain */
if (local_group)
load = target_load(i, load_idx);
load = source_load(i, load_idx);
sgs->group_load += load;
- sgs->sum_nr_running += nr_running;
+ sgs->sum_nr_running += rq->nr_running;
#ifdef CONFIG_NUMA_BALANCING
sgs->nr_numa_running += rq->nr_numa_running;
sgs->nr_preferred_running += rq->nr_preferred_running;
unsigned long next_balance; /* in jiffy units */
} nohz ____cacheline_aligned;
-static inline int find_new_ilb(int call_cpu)
+static inline int find_new_ilb(void)
{
int ilb = cpumask_first(nohz.idle_cpus_mask);
* nohz_load_balancer CPU (if there is one) otherwise fallback to any idle
* CPU (if there is one).
*/
-static void nohz_balancer_kick(int cpu)
+static void nohz_balancer_kick(void)
{
int ilb_cpu;
nohz.next_balance++;
- ilb_cpu = find_new_ilb(cpu);
+ ilb_cpu = find_new_ilb();
if (ilb_cpu >= nr_cpu_ids)
return;
*
* Balancing parameters are set up in init_sched_domains.
*/
-static void rebalance_domains(int cpu, enum cpu_idle_type idle)
+static void rebalance_domains(struct rq *rq, enum cpu_idle_type idle)
{
int continue_balancing = 1;
- struct rq *rq = cpu_rq(cpu);
+ int cpu = rq->cpu;
unsigned long interval;
struct sched_domain *sd;
/* Earliest time when we have to do rebalance again */
* In CONFIG_NO_HZ_COMMON case, the idle balance kickee will do the
* rebalancing for all the cpus for whom scheduler ticks are stopped.
*/
-static void nohz_idle_balance(int this_cpu, enum cpu_idle_type idle)
+static void nohz_idle_balance(struct rq *this_rq, enum cpu_idle_type idle)
{
- struct rq *this_rq = cpu_rq(this_cpu);
+ int this_cpu = this_rq->cpu;
struct rq *rq;
int balance_cpu;
update_idle_cpu_load(rq);
raw_spin_unlock_irq(&rq->lock);
- rebalance_domains(balance_cpu, CPU_IDLE);
+ rebalance_domains(rq, CPU_IDLE);
if (time_after(this_rq->next_balance, rq->next_balance))
this_rq->next_balance = rq->next_balance;
* - For SD_ASYM_PACKING, if the lower numbered cpu's in the scheduler
* domain span are idle.
*/
-static inline int nohz_kick_needed(struct rq *rq, int cpu)
+static inline int nohz_kick_needed(struct rq *rq)
{
unsigned long now = jiffies;
struct sched_domain *sd;
struct sched_group_power *sgp;
- int nr_busy;
+ int nr_busy, cpu = rq->cpu;
- if (unlikely(idle_cpu(cpu)))
+ if (unlikely(rq->idle_balance))
return 0;
/*
return 1;
}
#else
-static void nohz_idle_balance(int this_cpu, enum cpu_idle_type idle) { }
+static void nohz_idle_balance(struct rq *this_rq, enum cpu_idle_type idle) { }
#endif
/*
*/
static void run_rebalance_domains(struct softirq_action *h)
{
- int this_cpu = smp_processor_id();
- struct rq *this_rq = cpu_rq(this_cpu);
+ struct rq *this_rq = this_rq();
enum cpu_idle_type idle = this_rq->idle_balance ?
CPU_IDLE : CPU_NOT_IDLE;
- rebalance_domains(this_cpu, idle);
+ rebalance_domains(this_rq, idle);
/*
* If this cpu has a pending nohz_balance_kick, then do the
* balancing on behalf of the other idle cpus whose ticks are
* stopped.
*/
- nohz_idle_balance(this_cpu, idle);
+ nohz_idle_balance(this_rq, idle);
}
-static inline int on_null_domain(int cpu)
+static inline int on_null_domain(struct rq *rq)
{
- return !rcu_dereference_sched(cpu_rq(cpu)->sd);
+ return !rcu_dereference_sched(rq->sd);
}
/*
* Trigger the SCHED_SOFTIRQ if it is time to do periodic load balancing.
*/
-void trigger_load_balance(struct rq *rq, int cpu)
+void trigger_load_balance(struct rq *rq)
{
/* Don't need to rebalance while attached to NULL domain */
- if (time_after_eq(jiffies, rq->next_balance) &&
- likely(!on_null_domain(cpu)))
+ if (unlikely(on_null_domain(rq)))
+ return;
+
+ if (time_after_eq(jiffies, rq->next_balance))
raise_softirq(SCHED_SOFTIRQ);
#ifdef CONFIG_NO_HZ_COMMON
- if (nohz_kick_needed(rq, cpu) && likely(!on_null_domain(cpu)))
- nohz_balancer_kick(cpu);
+ if (nohz_kick_needed(rq))
+ nohz_balancer_kick();
#endif
}
!test_tsk_need_resched(rq->curr) &&
has_pushable_tasks(rq) &&
p->nr_cpus_allowed > 1 &&
- rt_task(rq->curr) &&
+ (dl_task(rq->curr) || rt_task(rq->curr)) &&
(rq->curr->nr_cpus_allowed < 2 ||
rq->curr->prio <= p->prio))
push_rt_tasks(rq);
#include <linux/sched.h>
#include <linux/sched/sysctl.h>
#include <linux/sched/rt.h>
+#include <linux/sched/deadline.h>
#include <linux/mutex.h>
#include <linux/spinlock.h>
#include <linux/stop_machine.h>
#include <linux/slab.h>
#include "cpupri.h"
+#include "cpudeadline.h"
#include "cpuacct.h"
struct rq;
#define NICE_0_LOAD SCHED_LOAD_SCALE
#define NICE_0_SHIFT SCHED_LOAD_SHIFT
+/*
+ * Single value that decides SCHED_DEADLINE internal math precision.
+ * 10 -> just above 1us
+ * 9 -> just above 0.5us
+ */
+#define DL_SCALE (10)
+
/*
* These are the 'tuning knobs' of the scheduler:
*/
*/
#define RUNTIME_INF ((u64)~0ULL)
+static inline int fair_policy(int policy)
+{
+ return policy == SCHED_NORMAL || policy == SCHED_BATCH;
+}
+
static inline int rt_policy(int policy)
{
- if (policy == SCHED_FIFO || policy == SCHED_RR)
- return 1;
- return 0;
+ return policy == SCHED_FIFO || policy == SCHED_RR;
+}
+
+static inline int dl_policy(int policy)
+{
+ return policy == SCHED_DEADLINE;
}
static inline int task_has_rt_policy(struct task_struct *p)
return rt_policy(p->policy);
}
+static inline int task_has_dl_policy(struct task_struct *p)
+{
+ return dl_policy(p->policy);
+}
+
+static inline bool dl_time_before(u64 a, u64 b)
+{
+ return (s64)(a - b) < 0;
+}
+
+/*
+ * Tells if entity @a should preempt entity @b.
+ */
+static inline bool
+dl_entity_preempt(struct sched_dl_entity *a, struct sched_dl_entity *b)
+{
+ return dl_time_before(a->deadline, b->deadline);
+}
+
/*
* This is the priority-queue data structure of the RT scheduling class:
*/
u64 rt_runtime;
struct hrtimer rt_period_timer;
};
+/*
+ * To keep the bandwidth of -deadline tasks and groups under control
+ * we need some place where:
+ * - store the maximum -deadline bandwidth of the system (the group);
+ * - cache the fraction of that bandwidth that is currently allocated.
+ *
+ * This is all done in the data structure below. It is similar to the
+ * one used for RT-throttling (rt_bandwidth), with the main difference
+ * that, since here we are only interested in admission control, we
+ * do not decrease any runtime while the group "executes", neither we
+ * need a timer to replenish it.
+ *
+ * With respect to SMP, the bandwidth is given on a per-CPU basis,
+ * meaning that:
+ * - dl_bw (< 100%) is the bandwidth of the system (group) on each CPU;
+ * - dl_total_bw array contains, in the i-eth element, the currently
+ * allocated bandwidth on the i-eth CPU.
+ * Moreover, groups consume bandwidth on each CPU, while tasks only
+ * consume bandwidth on the CPU they're running on.
+ * Finally, dl_total_bw_cpu is used to cache the index of dl_total_bw
+ * that will be shown the next time the proc or cgroup controls will
+ * be red. It on its turn can be changed by writing on its own
+ * control.
+ */
+struct dl_bandwidth {
+ raw_spinlock_t dl_runtime_lock;
+ u64 dl_runtime;
+ u64 dl_period;
+};
+
+static inline int dl_bandwidth_enabled(void)
+{
+ return sysctl_sched_rt_runtime >= 0;
+}
+
+extern struct dl_bw *dl_bw_of(int i);
+
+struct dl_bw {
+ raw_spinlock_t lock;
+ u64 bw, total_bw;
+};
extern struct mutex sched_domains_mutex;
#endif
};
+/* Deadline class' related fields in a runqueue */
+struct dl_rq {
+ /* runqueue is an rbtree, ordered by deadline */
+ struct rb_root rb_root;
+ struct rb_node *rb_leftmost;
+
+ unsigned long dl_nr_running;
+
+#ifdef CONFIG_SMP
+ /*
+ * Deadline values of the currently executing and the
+ * earliest ready task on this rq. Caching these facilitates
+ * the decision wether or not a ready but not running task
+ * should migrate somewhere else.
+ */
+ struct {
+ u64 curr;
+ u64 next;
+ } earliest_dl;
+
+ unsigned long dl_nr_migratory;
+ unsigned long dl_nr_total;
+ int overloaded;
+
+ /*
+ * Tasks on this rq that can be pushed away. They are kept in
+ * an rb-tree, ordered by tasks' deadlines, with caching
+ * of the leftmost (earliest deadline) element.
+ */
+ struct rb_root pushable_dl_tasks_root;
+ struct rb_node *pushable_dl_tasks_leftmost;
+#else
+ struct dl_bw dl_bw;
+#endif
+};
+
#ifdef CONFIG_SMP
/*
cpumask_var_t span;
cpumask_var_t online;
+ /*
+ * The bit corresponding to a CPU gets set here if such CPU has more
+ * than one runnable -deadline task (as it is below for RT tasks).
+ */
+ cpumask_var_t dlo_mask;
+ atomic_t dlo_count;
+ struct dl_bw dl_bw;
+ struct cpudl cpudl;
+
/*
* The "RT overload" flag: it gets set if a CPU has more than
* one runnable RT task.
struct cfs_rq cfs;
struct rt_rq rt;
+ struct dl_rq dl;
#ifdef CONFIG_FAIR_GROUP_SCHED
/* list of leaf cfs_rq on this cpu: */
return (u64)sysctl_sched_rt_runtime * NSEC_PER_USEC;
}
-
-
static inline int task_current(struct rq *rq, struct task_struct *p)
{
return rq->curr == p;
#else
#define ENQUEUE_WAKING 0
#endif
+#define ENQUEUE_REPLENISH 8
#define DEQUEUE_SLEEP 1
void (*set_curr_task) (struct rq *rq);
void (*task_tick) (struct rq *rq, struct task_struct *p, int queued);
void (*task_fork) (struct task_struct *p);
+ void (*task_dead) (struct task_struct *p);
void (*switched_from) (struct rq *this_rq, struct task_struct *task);
void (*switched_to) (struct rq *this_rq, struct task_struct *task);
for (class = sched_class_highest; class; class = class->next)
extern const struct sched_class stop_sched_class;
+extern const struct sched_class dl_sched_class;
extern const struct sched_class rt_sched_class;
extern const struct sched_class fair_sched_class;
extern const struct sched_class idle_sched_class;
extern void update_group_power(struct sched_domain *sd, int cpu);
-extern void trigger_load_balance(struct rq *rq, int cpu);
+extern void trigger_load_balance(struct rq *rq);
extern void idle_balance(int this_cpu, struct rq *this_rq);
extern void idle_enter_fair(struct rq *this_rq);
extern void sysrq_sched_debug_show(void);
extern void sched_init_granularity(void);
extern void update_max_interval(void);
+
+extern void init_sched_dl_class(void);
extern void init_sched_rt_class(void);
extern void init_sched_fair_class(void);
+extern void init_sched_dl_class(void);
extern void resched_task(struct task_struct *p);
extern void resched_cpu(int cpu);
extern struct rt_bandwidth def_rt_bandwidth;
extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime);
+extern struct dl_bandwidth def_dl_bandwidth;
+extern void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime);
+extern void init_dl_task_timer(struct sched_dl_entity *dl_se);
+
+unsigned long to_ratio(u64 period, u64 runtime);
+
extern void update_idle_cpu_load(struct rq *this_rq);
extern void init_task_runnable_average(struct task_struct *p);
extern void init_cfs_rq(struct cfs_rq *cfs_rq);
extern void init_rt_rq(struct rt_rq *rt_rq, struct rq *rq);
+extern void init_dl_rq(struct dl_rq *dl_rq, struct rq *rq);
extern void cfs_bandwidth_usage_inc(void);
extern void cfs_bandwidth_usage_dec(void);
* Simple, special scheduling class for the per-CPU stop tasks:
*/
const struct sched_class stop_sched_class = {
- .next = &rt_sched_class,
+ .next = &dl_sched_class,
.enqueue_task = enqueue_task_stop,
.dequeue_task = dequeue_task_stop,
* where hardirqs are disabled legitimately:
*/
#ifdef CONFIG_TRACE_IRQFLAGS
-static void __local_bh_disable(unsigned long ip, unsigned int cnt)
+void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
{
unsigned long flags;
/*
* Were softirqs turned off above:
*/
- if (softirq_count() == cnt)
+ if (softirq_count() == (cnt & SOFTIRQ_MASK))
trace_softirqs_off(ip);
raw_local_irq_restore(flags);
if (preempt_count() == cnt)
trace_preempt_off(CALLER_ADDR0, get_parent_ip(CALLER_ADDR1));
}
-#else /* !CONFIG_TRACE_IRQFLAGS */
-static inline void __local_bh_disable(unsigned long ip, unsigned int cnt)
-{
- preempt_count_add(cnt);
- barrier();
-}
+EXPORT_SYMBOL(__local_bh_disable_ip);
#endif /* CONFIG_TRACE_IRQFLAGS */
-void local_bh_disable(void)
-{
- __local_bh_disable(_RET_IP_, SOFTIRQ_DISABLE_OFFSET);
-}
-
-EXPORT_SYMBOL(local_bh_disable);
-
static void __local_bh_enable(unsigned int cnt)
{
WARN_ON_ONCE(!irqs_disabled());
- if (softirq_count() == cnt)
+ if (softirq_count() == (cnt & SOFTIRQ_MASK))
trace_softirqs_on(_RET_IP_);
preempt_count_sub(cnt);
}
EXPORT_SYMBOL(_local_bh_enable);
-static inline void _local_bh_enable_ip(unsigned long ip)
+void __local_bh_enable_ip(unsigned long ip, unsigned int cnt)
{
WARN_ON_ONCE(in_irq() || irqs_disabled());
#ifdef CONFIG_TRACE_IRQFLAGS
* Keep preemption disabled until we are done with
* softirq processing:
*/
- preempt_count_sub(SOFTIRQ_DISABLE_OFFSET - 1);
+ preempt_count_sub(cnt - 1);
if (unlikely(!in_interrupt() && local_softirq_pending())) {
/*
#endif
preempt_check_resched();
}
-
-void local_bh_enable(void)
-{
- _local_bh_enable_ip(_RET_IP_);
-}
-EXPORT_SYMBOL(local_bh_enable);
-
-void local_bh_enable_ip(unsigned long ip)
-{
- _local_bh_enable_ip(ip);
-}
-EXPORT_SYMBOL(local_bh_enable_ip);
+EXPORT_SYMBOL(__local_bh_enable_ip);
/*
* We restart softirq processing for at most MAX_SOFTIRQ_RESTART times,
#define MAX_SOFTIRQ_TIME msecs_to_jiffies(2)
#define MAX_SOFTIRQ_RESTART 10
+#ifdef CONFIG_TRACE_IRQFLAGS
+/*
+ * When we run softirqs from irq_exit() and thus on the hardirq stack we need
+ * to keep the lockdep irq context tracking as tight as possible in order to
+ * not miss-qualify lock contexts and miss possible deadlocks.
+ */
+
+static inline bool lockdep_softirq_start(void)
+{
+ bool in_hardirq = false;
+
+ if (trace_hardirq_context(current)) {
+ in_hardirq = true;
+ trace_hardirq_exit();
+ }
+
+ lockdep_softirq_enter();
+
+ return in_hardirq;
+}
+
+static inline void lockdep_softirq_end(bool in_hardirq)
+{
+ lockdep_softirq_exit();
+
+ if (in_hardirq)
+ trace_hardirq_enter();
+}
+#else
+static inline bool lockdep_softirq_start(void) { return false; }
+static inline void lockdep_softirq_end(bool in_hardirq) { }
+#endif
+
asmlinkage void __do_softirq(void)
{
- struct softirq_action *h;
- __u32 pending;
unsigned long end = jiffies + MAX_SOFTIRQ_TIME;
- int cpu;
unsigned long old_flags = current->flags;
int max_restart = MAX_SOFTIRQ_RESTART;
+ struct softirq_action *h;
+ bool in_hardirq;
+ __u32 pending;
+ int cpu;
/*
* Mask out PF_MEMALLOC s current task context is borrowed for the
pending = local_softirq_pending();
account_irq_enter_time(current);
- __local_bh_disable(_RET_IP_, SOFTIRQ_OFFSET);
- lockdep_softirq_enter();
+ __local_bh_disable_ip(_RET_IP_, SOFTIRQ_OFFSET);
+ in_hardirq = lockdep_softirq_start();
cpu = smp_processor_id();
restart:
wakeup_softirqd();
}
- lockdep_softirq_exit();
-
+ lockdep_softirq_end(in_hardirq);
account_irq_exit_time(current);
__local_bh_enable(SOFTIRQ_OFFSET);
WARN_ON_ONCE(in_interrupt());
tsk_restore_flags(current, old_flags, PF_MEMALLOC);
}
-
-
asmlinkage void do_softirq(void)
{
__u32 pending;
*/
void irq_enter(void)
{
- int cpu = smp_processor_id();
-
rcu_irq_enter();
if (is_idle_task(current) && !in_interrupt()) {
/*
* here, as softirq will be serviced on return from interrupt.
*/
local_bh_disable();
- tick_check_idle(cpu);
+ tick_check_idle();
_local_bh_enable();
}
#endif
account_irq_exit_time(current);
- trace_hardirq_exit();
preempt_count_sub(HARDIRQ_OFFSET);
if (!in_interrupt() && local_softirq_pending())
invoke_softirq();
tick_irq_exit();
rcu_irq_exit();
+ trace_hardirq_exit(); /* must be last! */
}
/*
.mode = 0644,
.proc_handler = proc_dointvec,
},
- {
- .procname = "numa_balancing_settle_count",
- .data = &sysctl_numa_balancing_settle_count,
- .maxlen = sizeof(unsigned int),
- .mode = 0644,
- .proc_handler = proc_dointvec,
- },
{
.procname = "numa_balancing_migrate_deferred",
.data = &sysctl_numa_balancing_migrate_deferred,
return cd.epoch_ns;
do {
- seq = read_seqcount_begin(&cd.seq);
+ seq = raw_read_seqcount_begin(&cd.seq);
epoch_cyc = cd.epoch_cyc;
epoch_ns = cd.epoch_ns;
} while (read_seqcount_retry(&cd.seq, seq));
cd.mult, cd.shift);
raw_local_irq_save(flags);
- write_seqcount_begin(&cd.seq);
+ raw_write_seqcount_begin(&cd.seq);
cd.epoch_ns = ns;
cd.epoch_cyc = cyc;
- write_seqcount_end(&cd.seq);
+ raw_write_seqcount_end(&cd.seq);
raw_local_irq_restore(flags);
}
* Called from irq_enter() when idle was interrupted to reenable the
* per cpu device.
*/
-void tick_check_oneshot_broadcast(int cpu)
+void tick_check_oneshot_broadcast_this_cpu(void)
{
- if (cpumask_test_cpu(cpu, tick_broadcast_oneshot_mask)) {
- struct tick_device *td = &per_cpu(tick_cpu_device, cpu);
+ if (cpumask_test_cpu(smp_processor_id(), tick_broadcast_oneshot_mask)) {
+ struct tick_device *td = &__get_cpu_var(tick_cpu_device);
/*
* We might be in the middle of switching over from
do_timer(1);
write_sequnlock(&jiffies_lock);
+ update_wall_time();
}
update_process_times(user_mode(get_irq_regs()));
extern void tick_shutdown_broadcast_oneshot(unsigned int *cpup);
extern int tick_resume_broadcast_oneshot(struct clock_event_device *bc);
extern int tick_broadcast_oneshot_active(void);
-extern void tick_check_oneshot_broadcast(int cpu);
+extern void tick_check_oneshot_broadcast_this_cpu(void);
bool tick_broadcast_oneshot_available(void);
# else /* BROADCAST */
static inline void tick_broadcast_setup_oneshot(struct clock_event_device *bc)
static inline void tick_broadcast_switch_to_oneshot(void) { }
static inline void tick_shutdown_broadcast_oneshot(unsigned int *cpup) { }
static inline int tick_broadcast_oneshot_active(void) { return 0; }
-static inline void tick_check_oneshot_broadcast(int cpu) { }
+static inline void tick_check_oneshot_broadcast_this_cpu(void) { }
static inline bool tick_broadcast_oneshot_available(void) { return true; }
# endif /* !BROADCAST */
#endif
extern void do_timer(unsigned long ticks);
+extern void update_wall_time(void);
tick_next_period = ktime_add(last_jiffies_update, tick_period);
}
write_sequnlock(&jiffies_lock);
+ update_wall_time();
}
/*
* TODO: kick full dynticks CPUs when
* sched_clock_stable is set.
*/
- if (!sched_clock_stable) {
+ if (!sched_clock_stable()) {
trace_tick_stop(0, "unstable sched clock\n");
/*
* Don't allow the user to think they can get
*/
static void tick_nohz_update_jiffies(ktime_t now)
{
- int cpu = smp_processor_id();
- struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu);
unsigned long flags;
- ts->idle_waketime = now;
+ __this_cpu_write(tick_cpu_sched.idle_waketime, now);
local_irq_save(flags);
tick_do_update_jiffies64(now);
}
-static void tick_nohz_stop_idle(int cpu, ktime_t now)
+static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now)
{
- struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu);
-
- update_ts_time_stats(cpu, ts, now, NULL);
+ update_ts_time_stats(smp_processor_id(), ts, now, NULL);
ts->idle_active = 0;
sched_clock_idle_wakeup_event(0);
}
-static ktime_t tick_nohz_start_idle(int cpu, struct tick_sched *ts)
+static ktime_t tick_nohz_start_idle(struct tick_sched *ts)
{
ktime_t now = ktime_get();
ktime_t now, expires;
int cpu = smp_processor_id();
- now = tick_nohz_start_idle(cpu, ts);
+ now = tick_nohz_start_idle(ts);
if (can_stop_idle_tick(cpu, ts)) {
int was_stopped = ts->tick_stopped;
*/
void tick_nohz_idle_exit(void)
{
- int cpu = smp_processor_id();
- struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu);
+ struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
ktime_t now;
local_irq_disable();
now = ktime_get();
if (ts->idle_active)
- tick_nohz_stop_idle(cpu, now);
+ tick_nohz_stop_idle(ts, now);
if (ts->tick_stopped) {
tick_nohz_restart_sched_tick(ts, now);
* timer and do not touch the other magic bits which need to be done
* when idle is left.
*/
-static void tick_nohz_kick_tick(int cpu, ktime_t now)
+static void tick_nohz_kick_tick(struct tick_sched *ts, ktime_t now)
{
#if 0
/* Switch back to 2.6.27 behaviour */
-
- struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu);
ktime_t delta;
/*
#endif
}
-static inline void tick_check_nohz(int cpu)
+static inline void tick_check_nohz_this_cpu(void)
{
- struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu);
+ struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
ktime_t now;
if (!ts->idle_active && !ts->tick_stopped)
return;
now = ktime_get();
if (ts->idle_active)
- tick_nohz_stop_idle(cpu, now);
+ tick_nohz_stop_idle(ts, now);
if (ts->tick_stopped) {
tick_nohz_update_jiffies(now);
- tick_nohz_kick_tick(cpu, now);
+ tick_nohz_kick_tick(ts, now);
}
}
#else
static inline void tick_nohz_switch_to_nohz(void) { }
-static inline void tick_check_nohz(int cpu) { }
+static inline void tick_check_nohz_this_cpu(void) { }
#endif /* CONFIG_NO_HZ_COMMON */
/*
* Called from irq_enter to notify about the possible interruption of idle()
*/
-void tick_check_idle(int cpu)
+void tick_check_idle(void)
{
- tick_check_oneshot_broadcast(cpu);
- tick_check_nohz(cpu);
+ tick_check_oneshot_broadcast_this_cpu();
+ tick_check_nohz_this_cpu();
}
/*
tk->wall_to_monotonic = wtm;
set_normalized_timespec(&tmp, -wtm.tv_sec, -wtm.tv_nsec);
tk->offs_real = timespec_to_ktime(tmp);
- tk->offs_tai = ktime_sub(tk->offs_real, ktime_set(tk->tai_offset, 0));
+ tk->offs_tai = ktime_add(tk->offs_real, ktime_set(tk->tai_offset, 0));
}
static void tk_set_sleep_time(struct timekeeper *tk, struct timespec t)
}
/**
- * timekeeper_setup_internals - Set up internals to use clocksource clock.
+ * tk_setup_internals - Set up internals to use clocksource clock.
*
+ * @tk: The target timekeeper to setup.
* @clock: Pointer to clocksource.
*
* Calculates a fixed cycle/nsec interval for a given clocksource/adjustment
static void __timekeeping_set_tai_offset(struct timekeeper *tk, s32 tai_offset)
{
tk->tai_offset = tai_offset;
- tk->offs_tai = ktime_sub(tk->offs_real, ktime_set(tai_offset, 0));
+ tk->offs_tai = ktime_add(tk->offs_real, ktime_set(tai_offset, 0));
}
/**
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&timekeeper_seq);
__timekeeping_set_tai_offset(tk, tai_offset);
+ timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
write_seqcount_end(&timekeeper_seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
clock_was_set();
timekeeping_suspend_time =
timespec_add(timekeeping_suspend_time, delta_delta);
}
+
+ timekeeping_update(tk, TK_MIRROR);
write_seqcount_end(&timekeeper_seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
* we can adjust by 1.
*/
error >>= 2;
- /*
- * XXX - In update_wall_time, we round up to the next
- * nanosecond, and store the amount rounded up into
- * the error. This causes the likely below to be unlikely.
- *
- * The proper fix is to avoid rounding up by using
- * the high precision tk->xtime_nsec instead of
- * xtime.tv_nsec everywhere. Fixing this will take some
- * time.
- */
if (likely(error <= interval))
adj = 1;
else
static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk)
{
u64 nsecps = (u64)NSEC_PER_SEC << tk->shift;
- unsigned int action = 0;
+ unsigned int clock_set = 0;
while (tk->xtime_nsec >= nsecps) {
int leap;
__timekeeping_set_tai_offset(tk, tk->tai_offset - leap);
- clock_was_set_delayed();
- action = TK_CLOCK_WAS_SET;
+ clock_set = TK_CLOCK_WAS_SET;
}
}
- return action;
+ return clock_set;
}
/**
* Returns the unconsumed cycles.
*/
static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,
- u32 shift)
+ u32 shift,
+ unsigned int *clock_set)
{
cycle_t interval = tk->cycle_interval << shift;
u64 raw_nsecs;
tk->cycle_last += interval;
tk->xtime_nsec += tk->xtime_interval << shift;
- accumulate_nsecs_to_secs(tk);
+ *clock_set |= accumulate_nsecs_to_secs(tk);
/* Accumulate raw time */
raw_nsecs = (u64)tk->raw_interval << shift;
* update_wall_time - Uses the current clocksource to increment the wall time
*
*/
-static void update_wall_time(void)
+void update_wall_time(void)
{
struct clocksource *clock;
struct timekeeper *real_tk = &timekeeper;
struct timekeeper *tk = &shadow_timekeeper;
cycle_t offset;
int shift = 0, maxshift;
- unsigned int action;
+ unsigned int clock_set = 0;
unsigned long flags;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1;
shift = min(shift, maxshift);
while (offset >= tk->cycle_interval) {
- offset = logarithmic_accumulation(tk, offset, shift);
+ offset = logarithmic_accumulation(tk, offset, shift,
+ &clock_set);
if (offset < tk->cycle_interval<<shift)
shift--;
}
* Finally, make sure that after the rounding
* xtime_nsec isn't larger than NSEC_PER_SEC
*/
- action = accumulate_nsecs_to_secs(tk);
+ clock_set |= accumulate_nsecs_to_secs(tk);
write_seqcount_begin(&timekeeper_seq);
/* Update clock->cycle_last with the new value */
* updating.
*/
memcpy(real_tk, tk, sizeof(*tk));
- timekeeping_update(real_tk, action);
+ timekeeping_update(real_tk, clock_set);
write_seqcount_end(&timekeeper_seq);
out:
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
+ if (clock_set)
+ clock_was_set();
}
/**
void do_timer(unsigned long ticks)
{
jiffies_64 += ticks;
- update_wall_time();
calc_global_load(ticks);
}
if (tai != orig_tai) {
__timekeeping_set_tai_offset(tk, tai);
- update_pvclock_gtod(tk, true);
- clock_was_set_delayed();
+ timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
}
write_seqcount_end(&timekeeper_seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
+ if (tai != orig_tai)
+ clock_was_set();
+
ntp_notify_cmos_timer();
return ret;
write_seqlock(&jiffies_lock);
do_timer(ticks);
write_sequnlock(&jiffies_lock);
+ update_wall_time();
}
if (unlikely(test_time_stamp(delta))) {
int local_clock_stable = 1;
#ifdef CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
- local_clock_stable = sched_clock_stable;
+ local_clock_stable = sched_clock_stable();
#endif
WARN_ONCE(delta > (1ULL << 59),
KERN_WARNING "Delta way too big! %llu ts=%llu write stamp = %llu\n%s",
#include <linux/uaccess.h>
#include <linux/ftrace.h>
#include <linux/sched/rt.h>
+#include <linux/sched/deadline.h>
#include <trace/events/sched.h>
#include "trace.h"
static int wakeup_current_cpu;
static unsigned wakeup_prio = -1;
static int wakeup_rt;
+static int wakeup_dl;
+static int tracing_dl = 0;
static arch_spinlock_t wakeup_lock =
(arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED;
{
wakeup_cpu = -1;
wakeup_prio = -1;
+ tracing_dl = 0;
if (wakeup_task)
put_task_struct(wakeup_task);
tracing_record_cmdline(p);
tracing_record_cmdline(current);
- if ((wakeup_rt && !rt_task(p)) ||
- p->prio >= wakeup_prio ||
- p->prio >= current->prio)
+ /*
+ * Semantic is like this:
+ * - wakeup tracer handles all tasks in the system, independently
+ * from their scheduling class;
+ * - wakeup_rt tracer handles tasks belonging to sched_dl and
+ * sched_rt class;
+ * - wakeup_dl handles tasks belonging to sched_dl class only.
+ */
+ if (tracing_dl || (wakeup_dl && !dl_task(p)) ||
+ (wakeup_rt && !dl_task(p) && !rt_task(p)) ||
+ (!dl_task(p) && (p->prio >= wakeup_prio || p->prio >= current->prio)))
return;
pc = preempt_count();
arch_spin_lock(&wakeup_lock);
/* check for races. */
- if (!tracer_enabled || p->prio >= wakeup_prio)
+ if (!tracer_enabled || tracing_dl ||
+ (!dl_task(p) && p->prio >= wakeup_prio))
goto out_locked;
/* reset the trace */
wakeup_current_cpu = wakeup_cpu;
wakeup_prio = p->prio;
+ /*
+ * Once you start tracing a -deadline task, don't bother tracing
+ * another task until the first one wakes up.
+ */
+ if (dl_task(p))
+ tracing_dl = 1;
+ else
+ tracing_dl = 0;
+
wakeup_task = p;
get_task_struct(wakeup_task);
static int wakeup_tracer_init(struct trace_array *tr)
{
+ wakeup_dl = 0;
wakeup_rt = 0;
return __wakeup_tracer_init(tr);
}
static int wakeup_rt_tracer_init(struct trace_array *tr)
{
+ wakeup_dl = 0;
wakeup_rt = 1;
return __wakeup_tracer_init(tr);
}
+static int wakeup_dl_tracer_init(struct trace_array *tr)
+{
+ wakeup_dl = 1;
+ wakeup_rt = 0;
+ return __wakeup_tracer_init(tr);
+}
+
static void wakeup_tracer_reset(struct trace_array *tr)
{
int lat_flag = save_flags & TRACE_ITER_LATENCY_FMT;
.use_max_tr = true,
};
+static struct tracer wakeup_dl_tracer __read_mostly =
+{
+ .name = "wakeup_dl",
+ .init = wakeup_dl_tracer_init,
+ .reset = wakeup_tracer_reset,
+ .start = wakeup_tracer_start,
+ .stop = wakeup_tracer_stop,
+ .wait_pipe = poll_wait_pipe,
+ .print_max = true,
+ .print_header = wakeup_print_header,
+ .print_line = wakeup_print_line,
+ .flags = &tracer_flags,
+ .set_flag = wakeup_set_flag,
+ .flag_changed = wakeup_flag_changed,
+#ifdef CONFIG_FTRACE_SELFTEST
+ .selftest = trace_selftest_startup_wakeup,
+#endif
+ .open = wakeup_trace_open,
+ .close = wakeup_trace_close,
+ .use_max_tr = true,
+};
+
__init static int init_wakeup_tracer(void)
{
int ret;
if (ret)
return ret;
+ ret = register_tracer(&wakeup_dl_tracer);
+ if (ret)
+ return ret;
+
return 0;
}
core_initcall(init_wakeup_tracer);
#ifdef CONFIG_SCHED_TRACER
static int trace_wakeup_test_thread(void *data)
{
- /* Make this a RT thread, doesn't need to be too high */
- static const struct sched_param param = { .sched_priority = 5 };
+ /* Make this a -deadline thread */
+ static const struct sched_attr attr = {
+ .sched_policy = SCHED_DEADLINE,
+ .sched_runtime = 100000ULL,
+ .sched_deadline = 10000000ULL,
+ .sched_period = 10000000ULL
+ };
struct completion *x = data;
- sched_setscheduler(current, SCHED_FIFO, ¶m);
+ sched_setattr(current, &attr);
/* Make it know we have a new prio */
complete(x);
/* we are awake, now wait to disappear */
while (!kthread_should_stop()) {
/*
- * This is an RT task, do short sleeps to let
- * others run.
+ * This will likely be the system top priority
+ * task, do short sleeps to let others run.
*/
msleep(100);
}
{
unsigned long save_max = tracing_max_latency;
struct task_struct *p;
- struct completion isrt;
+ struct completion is_ready;
unsigned long count;
int ret;
- init_completion(&isrt);
+ init_completion(&is_ready);
- /* create a high prio thread */
- p = kthread_run(trace_wakeup_test_thread, &isrt, "ftrace-test");
+ /* create a -deadline thread */
+ p = kthread_run(trace_wakeup_test_thread, &is_ready, "ftrace-test");
if (IS_ERR(p)) {
printk(KERN_CONT "Failed to create ftrace wakeup test thread ");
return -1;
}
- /* make sure the thread is running at an RT prio */
- wait_for_completion(&isrt);
+ /* make sure the thread is running at -deadline policy */
+ wait_for_completion(&is_ready);
/* start the tracing */
ret = tracer_init(trace, tr);
while (p->on_rq) {
/*
- * Sleep to make sure the RT thread is asleep too.
+ * Sleep to make sure the -deadline thread is asleep too.
* On virtual machines we can't rely on timings,
* but we want to make sure this test still works.
*/
msleep(100);
}
- init_completion(&isrt);
+ init_completion(&is_ready);
wake_up_process(p);
/* Wait for the task to wake up */
- wait_for_completion(&isrt);
+ wait_for_completion(&is_ready);
/* stop the tracing. */
tracing_stop();
default 0 if !PANIC_ON_OOPS
default 1 if PANIC_ON_OOPS
+config PANIC_TIMEOUT
+ int "panic timeout"
+ default 0
+ help
+ Set the timeout value (in seconds) until a reboot occurs when the
+ the kernel panics. If n = 0, then we wait forever. A timeout
+ value n > 0 will wait n seconds before rebooting, while a timeout
+ value n < 0 will reboot immediately.
+
config SCHED_DEBUG
bool "Collect scheduler debugging info"
depends on DEBUG_KERNEL && PROC_FS
unsigned long flags;
raw_spin_lock_irqsave(&fbc->lock, flags);
fbc->count += count;
+ __this_cpu_sub(*fbc->counters, count - amount);
raw_spin_unlock_irqrestore(&fbc->lock, flags);
- __this_cpu_write(*fbc->counters, 0);
} else {
- __this_cpu_write(*fbc->counters, count);
+ this_cpu_add(*fbc->counters, amount);
}
preempt_enable();
}
{
struct address_space *mapping = page->mapping;
- VM_BUG_ON(PageSlab(page));
+ /* This happens if someone calls flush_dcache_page on slab page */
+ if (unlikely(PageSlab(page)))
+ return NULL;
+
if (unlikely(PageSwapCache(page))) {
swp_entry_t entry;
sizeof(struct batadv_coded_packet));
#endif
- return header_len;
+ return header_len + ETH_HLEN;
}
/**
#include <asm/uaccess.h>
#include <asm/unaligned.h>
#include <linux/filter.h>
-#include <linux/reciprocal_div.h>
#include <linux/ratelimit.h>
#include <linux/seccomp.h>
#include <linux/if_vlan.h>
A /= X;
continue;
case BPF_S_ALU_DIV_K:
- A = reciprocal_divide(A, K);
+ A /= K;
continue;
case BPF_S_ALU_MOD_X:
if (X == 0)
/* Some instructions need special checks */
switch (code) {
case BPF_S_ALU_DIV_K:
- /* check for division by zero */
- if (ftest->k == 0)
- return -EINVAL;
- ftest->k = reciprocal_value(ftest->k);
- break;
case BPF_S_ALU_MOD_K:
/* check for division by zero */
if (ftest->k == 0)
to->code = decodes[code];
to->jt = filt->jt;
to->jf = filt->jf;
-
- if (code == BPF_S_ALU_DIV_K) {
- /*
- * When loaded this rule user gave us X, which was
- * translated into R = r(X). Now we calculate the
- * RR = r(R) and report it back. If next time this
- * value is loaded and RRR = r(RR) is calculated
- * then the R == RRR will be true.
- *
- * One exception. X == 1 translates into R == 0 and
- * we can't calculate RR out of it with r().
- */
-
- if (filt->k == 0)
- to->k = 1;
- else
- to->k = reciprocal_value(filt->k);
-
- BUG_ON(reciprocal_value(to->k) != filt->k);
- } else
- to->k = filt->k;
+ to->k = filt->k;
}
int sk_get_filter(struct sock *sk, struct sock_filter __user *ubuf, unsigned int len)
if (info->attrs[IEEE802154_ATTR_DEV_TYPE]) {
type = nla_get_u8(info->attrs[IEEE802154_ATTR_DEV_TYPE]);
- if (type >= __IEEE802154_DEV_MAX)
- return -EINVAL;
+ if (type >= __IEEE802154_DEV_MAX) {
+ rc = -EINVAL;
+ goto nla_put_failure;
+ }
}
dev = phy->add_iface(phy, devname, type);
spin_lock_bh(lock);
sk_nulls_for_each(sk, node, &head->chain) {
int res;
+ int state;
if (!net_eq(sock_net(sk), net))
continue;
if (num < s_num)
goto next_normal;
- if (!(r->idiag_states & (1 << sk->sk_state)))
+ state = (sk->sk_state == TCP_TIME_WAIT) ?
+ inet_twsk(sk)->tw_substate : sk->sk_state;
+ if (!(r->idiag_states & (1 << state)))
goto next_normal;
if (r->sdiag_family != AF_UNSPEC &&
sk->sk_family != r->sdiag_family)
static int ipmr_fib_lookup(struct net *net, struct flowi4 *flp4,
struct mr_table **mrt)
{
- struct ipmr_result res;
- struct fib_lookup_arg arg = { .result = &res, };
int err;
+ struct ipmr_result res;
+ struct fib_lookup_arg arg = {
+ .result = &res,
+ .flags = FIB_LOOKUP_NOREF,
+ };
err = fib_rules_lookup(net->ipv4.mr_rules_ops,
flowi4_to_flowi(flp4), 0, &arg);
(len > sysctl_tcp_dma_copybreak) && !(flags & MSG_PEEK) &&
!sysctl_tcp_low_latency &&
net_dma_find_channel()) {
- preempt_enable_no_resched();
+ preempt_enable();
tp->ucopy.pinned_list =
dma_pin_iovec_pages(msg->msg_iov, len);
} else {
- preempt_enable_no_resched();
+ preempt_enable();
}
}
#endif
int sysctl_tcp_nometrics_save __read_mostly;
+static struct tcp_metrics_block *__tcp_get_metrics(const struct inetpeer_addr *addr,
+ struct net *net, unsigned int hash);
+
struct tcp_fastopen_metrics {
u16 mss;
u16 syn_loss:10; /* Recurring Fast Open SYN losses */
}
}
+#define TCP_METRICS_TIMEOUT (60 * 60 * HZ)
+
+static void tcpm_check_stamp(struct tcp_metrics_block *tm, struct dst_entry *dst)
+{
+ if (tm && unlikely(time_after(jiffies, tm->tcpm_stamp + TCP_METRICS_TIMEOUT)))
+ tcpm_suck_dst(tm, dst, false);
+}
+
+#define TCP_METRICS_RECLAIM_DEPTH 5
+#define TCP_METRICS_RECLAIM_PTR (struct tcp_metrics_block *) 0x1UL
+
static struct tcp_metrics_block *tcpm_new(struct dst_entry *dst,
struct inetpeer_addr *addr,
- unsigned int hash,
- bool reclaim)
+ unsigned int hash)
{
struct tcp_metrics_block *tm;
struct net *net;
+ bool reclaim = false;
spin_lock_bh(&tcp_metrics_lock);
net = dev_net(dst->dev);
+
+ /* While waiting for the spin-lock the cache might have been populated
+ * with this entry and so we have to check again.
+ */
+ tm = __tcp_get_metrics(addr, net, hash);
+ if (tm == TCP_METRICS_RECLAIM_PTR) {
+ reclaim = true;
+ tm = NULL;
+ }
+ if (tm) {
+ tcpm_check_stamp(tm, dst);
+ goto out_unlock;
+ }
+
if (unlikely(reclaim)) {
struct tcp_metrics_block *oldest;
return tm;
}
-#define TCP_METRICS_TIMEOUT (60 * 60 * HZ)
-
-static void tcpm_check_stamp(struct tcp_metrics_block *tm, struct dst_entry *dst)
-{
- if (tm && unlikely(time_after(jiffies, tm->tcpm_stamp + TCP_METRICS_TIMEOUT)))
- tcpm_suck_dst(tm, dst, false);
-}
-
-#define TCP_METRICS_RECLAIM_DEPTH 5
-#define TCP_METRICS_RECLAIM_PTR (struct tcp_metrics_block *) 0x1UL
-
static struct tcp_metrics_block *tcp_get_encode(struct tcp_metrics_block *tm, int depth)
{
if (tm)
struct inetpeer_addr addr;
unsigned int hash;
struct net *net;
- bool reclaim;
addr.family = sk->sk_family;
switch (addr.family) {
hash = hash_32(hash, net->ipv4.tcp_metrics_hash_log);
tm = __tcp_get_metrics(&addr, net, hash);
- reclaim = false;
- if (tm == TCP_METRICS_RECLAIM_PTR) {
- reclaim = true;
+ if (tm == TCP_METRICS_RECLAIM_PTR)
tm = NULL;
- }
if (!tm && create)
- tm = tcpm_new(dst, &addr, hash, reclaim);
+ tm = tcpm_new(dst, &addr, hash);
else
tcpm_check_stamp(tm, dst);
in6_ifa_put(ifp);
}
+/* ifp->idev must be at least read locked */
+static bool ipv6_lonely_lladdr(struct inet6_ifaddr *ifp)
+{
+ struct inet6_ifaddr *ifpiter;
+ struct inet6_dev *idev = ifp->idev;
+
+ list_for_each_entry(ifpiter, &idev->addr_list, if_list) {
+ if (ifp != ifpiter && ifpiter->scope == IFA_LINK &&
+ (ifpiter->flags & (IFA_F_PERMANENT|IFA_F_TENTATIVE|
+ IFA_F_OPTIMISTIC|IFA_F_DADFAILED)) ==
+ IFA_F_PERMANENT)
+ return false;
+ }
+ return true;
+}
+
static void addrconf_dad_completed(struct inet6_ifaddr *ifp)
{
struct net_device *dev = ifp->idev->dev;
*/
read_lock_bh(&ifp->idev->lock);
- spin_lock(&ifp->lock);
- send_mld = ipv6_addr_type(&ifp->addr) & IPV6_ADDR_LINKLOCAL &&
- ifp->idev->valid_ll_addr_cnt == 1;
+ send_mld = ifp->scope == IFA_LINK && ipv6_lonely_lladdr(ifp);
send_rs = send_mld &&
ipv6_accept_ra(ifp->idev) &&
ifp->idev->cnf.rtr_solicits > 0 &&
(dev->flags&IFF_LOOPBACK) == 0;
- spin_unlock(&ifp->lock);
read_unlock_bh(&ifp->idev->lock);
/* While dad is in progress mld report's source address is in6_addrany.
rtnl_set_sk_err(net, RTNLGRP_IPV6_PREFIX, err);
}
-static void update_valid_ll_addr_cnt(struct inet6_ifaddr *ifp, int count)
-{
- write_lock_bh(&ifp->idev->lock);
- spin_lock(&ifp->lock);
- if (((ifp->flags & (IFA_F_PERMANENT|IFA_F_TENTATIVE|IFA_F_OPTIMISTIC|
- IFA_F_DADFAILED)) == IFA_F_PERMANENT) &&
- (ipv6_addr_type(&ifp->addr) & IPV6_ADDR_LINKLOCAL))
- ifp->idev->valid_ll_addr_cnt += count;
- WARN_ON(ifp->idev->valid_ll_addr_cnt < 0);
- spin_unlock(&ifp->lock);
- write_unlock_bh(&ifp->idev->lock);
-}
-
static void __ipv6_ifa_notify(int event, struct inet6_ifaddr *ifp)
{
struct net *net = dev_net(ifp->idev->dev);
switch (event) {
case RTM_NEWADDR:
- update_valid_ll_addr_cnt(ifp, 1);
-
/*
* If the address was optimistic
* we inserted the route at the start of
ifp->idev->dev, 0, 0);
break;
case RTM_DELADDR:
- update_valid_ll_addr_cnt(ifp, -1);
-
if (ifp->idev->cnf.forwarding)
addrconf_leave_anycast(ifp);
addrconf_leave_solict(ifp->idev, &ifp->addr);
static int ip6mr_fib_lookup(struct net *net, struct flowi6 *flp6,
struct mr6_table **mrt)
{
- struct ip6mr_result res;
- struct fib_lookup_arg arg = { .result = &res, };
int err;
+ struct ip6mr_result res;
+ struct fib_lookup_arg arg = {
+ .result = &res,
+ .flags = FIB_LOOKUP_NOREF,
+ };
err = fib_rules_lookup(net->ipv6.mr6_rules_ops,
flowi6_to_flowi(flp6), 0, &arg);
struct rds_ib_refill_cache *cache)
{
unsigned long flags;
- struct list_head *old;
- struct list_head __percpu *chpfirst;
+ struct list_head *old, *chpfirst;
local_irq_save(flags);
else /* put on front */
list_add_tail(new_item, chpfirst);
- __this_cpu_write(chpfirst, new_item);
+ __this_cpu_write(cache->percpu->first, new_item);
__this_cpu_inc(cache->percpu->count);
if (__this_cpu_read(cache->percpu->count) < RDS_IB_RECYCLE_BATCH_COUNT)
} while (old);
- __this_cpu_write(chpfirst, NULL);
+ __this_cpu_write(cache->percpu->first, NULL);
__this_cpu_write(cache->percpu->count, 0);
end:
local_irq_restore(flags);
$depends{$config} = $1;
} elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
$depends{$config} .= " " . $1;
+ } elsif ($state eq "DEP" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
+ my $dep = $3;
+ if ($dep !~ /^\s*(y|m|n)\s*$/) {
+ $dep =~ s/.*\sif\s+//;
+ $depends{$config} .= " " . $dep;
+ dprint "Added default depends $dep to $config\n";
+ }
# Get the configs that select this config
} elsif ($state ne "NONE" && /^\s*select\s+(\S+)/) {
cgroup firewire guest usb virtio vm net: FORCE
$(call descend,$@)
-liblk: FORCE
- $(call descend,lib/lk)
+libapikfs: FORCE
+ $(call descend,lib/api)
-perf: liblk FORCE
+perf: libapikfs FORCE
$(call descend,$@)
selftests: FORCE
cgroup_clean firewire_clean lguest_clean usb_clean virtio_clean vm_clean net_clean:
$(call descend,$(@:_clean=),clean)
-liblk_clean:
- $(call descend,lib/lk,clean)
+libapikfs_clean:
+ $(call descend,lib/api,clean)
-perf_clean: liblk_clean
+perf_clean: libapikfs_clean
$(call descend,$(@:_clean=),clean)
selftests_clean:
--- /dev/null
+#ifndef _TOOLS_ASM_BUG_H
+#define _TOOLS_ASM_BUG_H
+
+#include <linux/compiler.h>
+
+#define __WARN_printf(arg...) do { fprintf(stderr, arg); } while (0)
+
+#define WARN(condition, format...) ({ \
+ int __ret_warn_on = !!(condition); \
+ if (unlikely(__ret_warn_on)) \
+ __WARN_printf(format); \
+ unlikely(__ret_warn_on); \
+})
+
+#define WARN_ONCE(condition, format...) ({ \
+ static int __warned; \
+ int __ret_warn_once = !!(condition); \
+ \
+ if (unlikely(__ret_warn_once)) \
+ if (WARN(!__warned, format)) \
+ __warned = 1; \
+ unlikely(__ret_warn_once); \
+})
+
+#endif /* _TOOLS_ASM_BUG_H */
--- /dev/null
+#ifndef _TOOLS_LINUX_COMPILER_H_
+#define _TOOLS_LINUX_COMPILER_H_
+
+#ifndef __always_inline
+# define __always_inline inline __attribute__((always_inline))
+#endif
+
+#define __user
+
+#ifndef __attribute_const__
+# define __attribute_const__
+#endif
+
+#ifndef __maybe_unused
+# define __maybe_unused __attribute__((unused))
+#endif
+
+#ifndef __packed
+# define __packed __attribute__((__packed__))
+#endif
+
+#ifndef __force
+# define __force
+#endif
+
+#ifndef __weak
+# define __weak __attribute__((weak))
+#endif
+
+#ifndef likely
+# define likely(x) __builtin_expect(!!(x), 1)
+#endif
+
+#ifndef unlikely
+# define unlikely(x) __builtin_expect(!!(x), 0)
+#endif
+
+#endif /* _TOOLS_LINUX_COMPILER_H */
--- /dev/null
+include ../../scripts/Makefile.include
+include ../../perf/config/utilities.mak # QUIET_CLEAN
+
+CC = $(CROSS_COMPILE)gcc
+AR = $(CROSS_COMPILE)ar
+
+# guard against environment variables
+LIB_H=
+LIB_OBJS=
+
+LIB_H += fs/debugfs.h
+
+LIB_OBJS += $(OUTPUT)fs/debugfs.o
+
+LIBFILE = libapikfs.a
+
+CFLAGS = -ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -D_FORTIFY_SOURCE=2 $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) -fPIC
+EXTLIBS = -lelf -lpthread -lrt -lm
+ALL_CFLAGS = $(CFLAGS) $(BASIC_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
+ALL_LDFLAGS = $(LDFLAGS)
+
+RM = rm -f
+
+$(LIBFILE): $(LIB_OBJS)
+ $(QUIET_AR)$(RM) $@ && $(AR) rcs $(OUTPUT)$@ $(LIB_OBJS)
+
+$(LIB_OBJS): $(LIB_H)
+
+libapi_dirs:
+ $(QUIET_MKDIR)mkdir -p $(OUTPUT)fs/
+
+$(OUTPUT)%.o: %.c libapi_dirs
+ $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
+$(OUTPUT)%.s: %.c libapi_dirs
+ $(QUIET_CC)$(CC) -S $(ALL_CFLAGS) $<
+$(OUTPUT)%.o: %.S libapi_dirs
+ $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
+
+clean:
+ $(call QUIET_CLEAN, libapi) $(RM) $(LIB_OBJS) $(LIBFILE)
+
+.PHONY: clean
--- /dev/null
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdbool.h>
+#include <sys/vfs.h>
+#include <sys/mount.h>
+#include <linux/kernel.h>
+
+#include "debugfs.h"
+
+char debugfs_mountpoint[PATH_MAX + 1] = "/sys/kernel/debug";
+
+static const char * const debugfs_known_mountpoints[] = {
+ "/sys/kernel/debug/",
+ "/debug/",
+ 0,
+};
+
+static bool debugfs_found;
+
+/* find the path to the mounted debugfs */
+const char *debugfs_find_mountpoint(void)
+{
+ const char * const *ptr;
+ char type[100];
+ FILE *fp;
+
+ if (debugfs_found)
+ return (const char *)debugfs_mountpoint;
+
+ ptr = debugfs_known_mountpoints;
+ while (*ptr) {
+ if (debugfs_valid_mountpoint(*ptr) == 0) {
+ debugfs_found = true;
+ strcpy(debugfs_mountpoint, *ptr);
+ return debugfs_mountpoint;
+ }
+ ptr++;
+ }
+
+ /* give up and parse /proc/mounts */
+ fp = fopen("/proc/mounts", "r");
+ if (fp == NULL)
+ return NULL;
+
+ while (fscanf(fp, "%*s %" STR(PATH_MAX) "s %99s %*s %*d %*d\n",
+ debugfs_mountpoint, type) == 2) {
+ if (strcmp(type, "debugfs") == 0)
+ break;
+ }
+ fclose(fp);
+
+ if (strcmp(type, "debugfs") != 0)
+ return NULL;
+
+ debugfs_found = true;
+
+ return debugfs_mountpoint;
+}
+
+/* verify that a mountpoint is actually a debugfs instance */
+
+int debugfs_valid_mountpoint(const char *debugfs)
+{
+ struct statfs st_fs;
+
+ if (statfs(debugfs, &st_fs) < 0)
+ return -ENOENT;
+ else if (st_fs.f_type != (long) DEBUGFS_MAGIC)
+ return -ENOENT;
+
+ return 0;
+}
+
+/* mount the debugfs somewhere if it's not mounted */
+char *debugfs_mount(const char *mountpoint)
+{
+ /* see if it's already mounted */
+ if (debugfs_find_mountpoint())
+ goto out;
+
+ /* if not mounted and no argument */
+ if (mountpoint == NULL) {
+ /* see if environment variable set */
+ mountpoint = getenv(PERF_DEBUGFS_ENVIRONMENT);
+ /* if no environment variable, use default */
+ if (mountpoint == NULL)
+ mountpoint = "/sys/kernel/debug";
+ }
+
+ if (mount(NULL, mountpoint, "debugfs", 0, NULL) < 0)
+ return NULL;
+
+ /* save the mountpoint */
+ debugfs_found = true;
+ strncpy(debugfs_mountpoint, mountpoint, sizeof(debugfs_mountpoint));
+out:
+ return debugfs_mountpoint;
+}
--- /dev/null
+#ifndef __API_DEBUGFS_H__
+#define __API_DEBUGFS_H__
+
+#define _STR(x) #x
+#define STR(x) _STR(x)
+
+/*
+ * On most systems <limits.h> would have given us this, but not on some systems
+ * (e.g. GNU/Hurd).
+ */
+#ifndef PATH_MAX
+#define PATH_MAX 4096
+#endif
+
+#ifndef DEBUGFS_MAGIC
+#define DEBUGFS_MAGIC 0x64626720
+#endif
+
+#ifndef PERF_DEBUGFS_ENVIRONMENT
+#define PERF_DEBUGFS_ENVIRONMENT "PERF_DEBUGFS_DIR"
+#endif
+
+const char *debugfs_find_mountpoint(void);
+int debugfs_valid_mountpoint(const char *debugfs);
+char *debugfs_mount(const char *mountpoint);
+
+extern char debugfs_mountpoint[];
+
+#endif /* __API_DEBUGFS_H__ */
+++ /dev/null
-include ../../scripts/Makefile.include
-
-CC = $(CROSS_COMPILE)gcc
-AR = $(CROSS_COMPILE)ar
-
-# guard against environment variables
-LIB_H=
-LIB_OBJS=
-
-LIB_H += debugfs.h
-
-LIB_OBJS += $(OUTPUT)debugfs.o
-
-LIBFILE = liblk.a
-
-CFLAGS = -ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -D_FORTIFY_SOURCE=2 $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) -fPIC
-EXTLIBS = -lelf -lpthread -lrt -lm
-ALL_CFLAGS = $(CFLAGS) $(BASIC_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
-ALL_LDFLAGS = $(LDFLAGS)
-
-RM = rm -f
-
-$(LIBFILE): $(LIB_OBJS)
- $(QUIET_AR)$(RM) $@ && $(AR) rcs $(OUTPUT)$@ $(LIB_OBJS)
-
-$(LIB_OBJS): $(LIB_H)
-
-$(OUTPUT)%.o: %.c
- $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
-$(OUTPUT)%.s: %.c
- $(QUIET_CC)$(CC) -S $(ALL_CFLAGS) $<
-$(OUTPUT)%.o: %.S
- $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
-
-clean:
- $(RM) $(LIB_OBJS) $(LIBFILE)
-
-.PHONY: clean
+++ /dev/null
-#include <errno.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <stdbool.h>
-#include <sys/vfs.h>
-#include <sys/mount.h>
-#include <linux/kernel.h>
-
-#include "debugfs.h"
-
-char debugfs_mountpoint[PATH_MAX + 1] = "/sys/kernel/debug";
-
-static const char * const debugfs_known_mountpoints[] = {
- "/sys/kernel/debug/",
- "/debug/",
- 0,
-};
-
-static bool debugfs_found;
-
-/* find the path to the mounted debugfs */
-const char *debugfs_find_mountpoint(void)
-{
- const char * const *ptr;
- char type[100];
- FILE *fp;
-
- if (debugfs_found)
- return (const char *)debugfs_mountpoint;
-
- ptr = debugfs_known_mountpoints;
- while (*ptr) {
- if (debugfs_valid_mountpoint(*ptr) == 0) {
- debugfs_found = true;
- strcpy(debugfs_mountpoint, *ptr);
- return debugfs_mountpoint;
- }
- ptr++;
- }
-
- /* give up and parse /proc/mounts */
- fp = fopen("/proc/mounts", "r");
- if (fp == NULL)
- return NULL;
-
- while (fscanf(fp, "%*s %" STR(PATH_MAX) "s %99s %*s %*d %*d\n",
- debugfs_mountpoint, type) == 2) {
- if (strcmp(type, "debugfs") == 0)
- break;
- }
- fclose(fp);
-
- if (strcmp(type, "debugfs") != 0)
- return NULL;
-
- debugfs_found = true;
-
- return debugfs_mountpoint;
-}
-
-/* verify that a mountpoint is actually a debugfs instance */
-
-int debugfs_valid_mountpoint(const char *debugfs)
-{
- struct statfs st_fs;
-
- if (statfs(debugfs, &st_fs) < 0)
- return -ENOENT;
- else if (st_fs.f_type != (long) DEBUGFS_MAGIC)
- return -ENOENT;
-
- return 0;
-}
-
-/* mount the debugfs somewhere if it's not mounted */
-char *debugfs_mount(const char *mountpoint)
-{
- /* see if it's already mounted */
- if (debugfs_find_mountpoint())
- goto out;
-
- /* if not mounted and no argument */
- if (mountpoint == NULL) {
- /* see if environment variable set */
- mountpoint = getenv(PERF_DEBUGFS_ENVIRONMENT);
- /* if no environment variable, use default */
- if (mountpoint == NULL)
- mountpoint = "/sys/kernel/debug";
- }
-
- if (mount(NULL, mountpoint, "debugfs", 0, NULL) < 0)
- return NULL;
-
- /* save the mountpoint */
- debugfs_found = true;
- strncpy(debugfs_mountpoint, mountpoint, sizeof(debugfs_mountpoint));
-out:
- return debugfs_mountpoint;
-}
+++ /dev/null
-#ifndef __LK_DEBUGFS_H__
-#define __LK_DEBUGFS_H__
-
-#define _STR(x) #x
-#define STR(x) _STR(x)
-
-/*
- * On most systems <limits.h> would have given us this, but not on some systems
- * (e.g. GNU/Hurd).
- */
-#ifndef PATH_MAX
-#define PATH_MAX 4096
-#endif
-
-#ifndef DEBUGFS_MAGIC
-#define DEBUGFS_MAGIC 0x64626720
-#endif
-
-#ifndef PERF_DEBUGFS_ENVIRONMENT
-#define PERF_DEBUGFS_ENVIRONMENT "PERF_DEBUGFS_DIR"
-#endif
-
-const char *debugfs_find_mountpoint(void);
-int debugfs_valid_mountpoint(const char *debugfs);
-char *debugfs_mount(const char *mountpoint);
-
-extern char debugfs_mountpoint[];
-
-#endif /* __LK_DEBUGFS_H__ */
--- /dev/null
+# liblockdep version
+LL_VERSION = 0
+LL_PATCHLEVEL = 0
+LL_EXTRAVERSION = 1
+
+# file format version
+FILE_VERSION = 1
+
+MAKEFLAGS += --no-print-directory
+
+
+# Makefiles suck: This macro sets a default value of $(2) for the
+# variable named by $(1), unless the variable has been set by
+# environment or command line. This is necessary for CC and AR
+# because make sets default values, so the simpler ?= approach
+# won't work as expected.
+define allow-override
+ $(if $(or $(findstring environment,$(origin $(1))),\
+ $(findstring command line,$(origin $(1)))),,\
+ $(eval $(1) = $(2)))
+endef
+
+# Allow setting CC and AR, or setting CROSS_COMPILE as a prefix.
+$(call allow-override,CC,$(CROSS_COMPILE)gcc)
+$(call allow-override,AR,$(CROSS_COMPILE)ar)
+
+INSTALL = install
+
+# Use DESTDIR for installing into a different root directory.
+# This is useful for building a package. The program will be
+# installed in this directory as if it was the root directory.
+# Then the build tool can move it later.
+DESTDIR ?=
+DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
+
+prefix ?= /usr/local
+libdir_relative = lib
+libdir = $(prefix)/$(libdir_relative)
+bindir_relative = bin
+bindir = $(prefix)/$(bindir_relative)
+
+export DESTDIR DESTDIR_SQ INSTALL
+
+# copy a bit from Linux kbuild
+
+ifeq ("$(origin V)", "command line")
+ VERBOSE = $(V)
+endif
+ifndef VERBOSE
+ VERBOSE = 0
+endif
+
+ifeq ("$(origin O)", "command line")
+ BUILD_OUTPUT := $(O)
+endif
+
+ifeq ($(BUILD_SRC),)
+ifneq ($(BUILD_OUTPUT),)
+
+define build_output
+ $(if $(VERBOSE:1=),@)$(MAKE) -C $(BUILD_OUTPUT) \
+ BUILD_SRC=$(CURDIR) -f $(CURDIR)/Makefile $1
+endef
+
+saved-output := $(BUILD_OUTPUT)
+BUILD_OUTPUT := $(shell cd $(BUILD_OUTPUT) && /bin/pwd)
+$(if $(BUILD_OUTPUT),, \
+ $(error output directory "$(saved-output)" does not exist))
+
+all: sub-make
+
+gui: force
+ $(call build_output, all_cmd)
+
+$(filter-out gui,$(MAKECMDGOALS)): sub-make
+
+sub-make: force
+ $(call build_output, $(MAKECMDGOALS))
+
+
+# Leave processing to above invocation of make
+skip-makefile := 1
+
+endif # BUILD_OUTPUT
+endif # BUILD_SRC
+
+# We process the rest of the Makefile if this is the final invocation of make
+ifeq ($(skip-makefile),)
+
+srctree := $(if $(BUILD_SRC),$(BUILD_SRC),$(CURDIR))
+objtree := $(CURDIR)
+src := $(srctree)
+obj := $(objtree)
+
+export prefix libdir bindir src obj
+
+# Shell quotes
+libdir_SQ = $(subst ','\'',$(libdir))
+bindir_SQ = $(subst ','\'',$(bindir))
+
+LIB_FILE = liblockdep.a liblockdep.so
+BIN_FILE = lockdep
+
+CONFIG_INCLUDES =
+CONFIG_LIBS =
+CONFIG_FLAGS =
+
+OBJ = $@
+N =
+
+export Q VERBOSE
+
+LIBLOCKDEP_VERSION = $(LL_VERSION).$(LL_PATCHLEVEL).$(LL_EXTRAVERSION)
+
+INCLUDES = -I. -I/usr/local/include -I./uinclude $(CONFIG_INCLUDES)
+
+# Set compile option CFLAGS if not set elsewhere
+CFLAGS ?= -g -DCONFIG_LOCKDEP -DCONFIG_STACKTRACE -DCONFIG_PROVE_LOCKING -DBITS_PER_LONG=__WORDSIZE -DLIBLOCKDEP_VERSION='"$(LIBLOCKDEP_VERSION)"' -rdynamic -O0 -g
+
+override CFLAGS += $(CONFIG_FLAGS) $(INCLUDES) $(PLUGIN_DIR_SQ)
+
+ifeq ($(VERBOSE),1)
+ Q =
+ print_compile =
+ print_app_build =
+ print_fpic_compile =
+ print_shared_lib_compile =
+ print_install =
+else
+ Q = @
+ print_compile = echo ' CC '$(OBJ);
+ print_app_build = echo ' BUILD '$(OBJ);
+ print_fpic_compile = echo ' CC FPIC '$(OBJ);
+ print_shared_lib_compile = echo ' BUILD SHARED LIB '$(OBJ);
+ print_static_lib_build = echo ' BUILD STATIC LIB '$(OBJ);
+ print_install = echo ' INSTALL '$1' to $(DESTDIR_SQ)$2';
+endif
+
+do_fpic_compile = \
+ ($(print_fpic_compile) \
+ $(CC) -c $(CFLAGS) $(EXT) -fPIC $< -o $@)
+
+do_app_build = \
+ ($(print_app_build) \
+ $(CC) $^ -rdynamic -o $@ $(CONFIG_LIBS) $(LIBS))
+
+do_compile_shared_library = \
+ ($(print_shared_lib_compile) \
+ $(CC) --shared $^ -o $@ -lpthread -ldl)
+
+do_build_static_lib = \
+ ($(print_static_lib_build) \
+ $(RM) $@; $(AR) rcs $@ $^)
+
+
+define do_compile
+ $(print_compile) \
+ $(CC) -c $(CFLAGS) $(EXT) $< -o $(obj)/$@;
+endef
+
+$(obj)/%.o: $(src)/%.c
+ $(Q)$(call do_compile)
+
+%.o: $(src)/%.c
+ $(Q)$(call do_compile)
+
+PEVENT_LIB_OBJS = common.o lockdep.o preload.o rbtree.o
+
+ALL_OBJS = $(PEVENT_LIB_OBJS)
+
+CMD_TARGETS = $(LIB_FILE)
+
+TARGETS = $(CMD_TARGETS)
+
+
+all: all_cmd
+
+all_cmd: $(CMD_TARGETS)
+
+liblockdep.so: $(PEVENT_LIB_OBJS)
+ $(Q)$(do_compile_shared_library)
+
+liblockdep.a: $(PEVENT_LIB_OBJS)
+ $(Q)$(do_build_static_lib)
+
+$(PEVENT_LIB_OBJS): %.o: $(src)/%.c
+ $(Q)$(do_fpic_compile)
+
+## make deps
+
+all_objs := $(sort $(ALL_OBJS))
+all_deps := $(all_objs:%.o=.%.d)
+
+# let .d file also depends on the source and header files
+define check_deps
+ @set -e; $(RM) $@; \
+ $(CC) -MM $(CFLAGS) $< > $@.$$$$; \
+ sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@; \
+ $(RM) $@.$$$$
+endef
+
+$(all_deps): .%.d: $(src)/%.c
+ $(Q)$(call check_deps)
+
+$(all_objs) : %.o : .%.d
+
+dep_includes := $(wildcard $(all_deps))
+
+ifneq ($(dep_includes),)
+ include $(dep_includes)
+endif
+
+### Detect environment changes
+TRACK_CFLAGS = $(subst ','\'',$(CFLAGS)):$(ARCH):$(CROSS_COMPILE)
+
+tags: force
+ $(RM) tags
+ find . -name '*.[ch]' | xargs ctags --extra=+f --c-kinds=+px \
+ --regex-c++='/_PE\(([^,)]*).*/PEVENT_ERRNO__\1/'
+
+TAGS: force
+ $(RM) TAGS
+ find . -name '*.[ch]' | xargs etags \
+ --regex='/_PE(\([^,)]*\).*/PEVENT_ERRNO__\1/'
+
+define do_install
+ $(print_install) \
+ if [ ! -d '$(DESTDIR_SQ)$2' ]; then \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$2'; \
+ fi; \
+ $(INSTALL) $1 '$(DESTDIR_SQ)$2'
+endef
+
+install_lib: all_cmd
+ $(Q)$(call do_install,$(LIB_FILE),$(libdir_SQ))
+ $(Q)$(call do_install,$(BIN_FILE),$(bindir_SQ))
+
+install: install_lib
+
+clean:
+ $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d
+ $(RM) tags TAGS
+
+endif # skip-makefile
+
+PHONY += force
+force:
+
+# Declare the contents of the .PHONY variable as phony. We keep that
+# information in a variable so we can use it in if_changed and friends.
+.PHONY: $(PHONY)
--- /dev/null
+#include <stddef.h>
+#include <stdbool.h>
+#include <linux/compiler.h>
+#include <linux/lockdep.h>
+#include <unistd.h>
+#include <sys/syscall.h>
+
+static __thread struct task_struct current_obj;
+
+/* lockdep wants these */
+bool debug_locks = true;
+bool debug_locks_silent;
+
+__attribute__((constructor)) static void liblockdep_init(void)
+{
+ lockdep_init();
+}
+
+__attribute__((destructor)) static void liblockdep_exit(void)
+{
+ debug_check_no_locks_held(¤t_obj);
+}
+
+struct task_struct *__curr(void)
+{
+ if (current_obj.pid == 0) {
+ /* Makes lockdep output pretty */
+ prctl(PR_GET_NAME, current_obj.comm);
+ current_obj.pid = syscall(__NR_gettid);
+ }
+
+ return ¤t_obj;
+}
--- /dev/null
+#ifndef _LIBLOCKDEP_COMMON_H
+#define _LIBLOCKDEP_COMMON_H
+
+#include <pthread.h>
+
+#define NR_LOCKDEP_CACHING_CLASSES 2
+#define MAX_LOCKDEP_SUBCLASSES 8UL
+
+#ifndef CALLER_ADDR0
+#define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
+#endif
+
+#ifndef _RET_IP_
+#define _RET_IP_ CALLER_ADDR0
+#endif
+
+#ifndef _THIS_IP_
+#define _THIS_IP_ ({ __label__ __here; __here: (unsigned long)&&__here; })
+#endif
+
+struct lockdep_subclass_key {
+ char __one_byte;
+};
+
+struct lock_class_key {
+ struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES];
+};
+
+struct lockdep_map {
+ struct lock_class_key *key;
+ struct lock_class *class_cache[NR_LOCKDEP_CACHING_CLASSES];
+ const char *name;
+#ifdef CONFIG_LOCK_STAT
+ int cpu;
+ unsigned long ip;
+#endif
+};
+
+void lockdep_init_map(struct lockdep_map *lock, const char *name,
+ struct lock_class_key *key, int subclass);
+void lock_acquire(struct lockdep_map *lock, unsigned int subclass,
+ int trylock, int read, int check,
+ struct lockdep_map *nest_lock, unsigned long ip);
+void lock_release(struct lockdep_map *lock, int nested,
+ unsigned long ip);
+
+#define STATIC_LOCKDEP_MAP_INIT(_name, _key) \
+ { .name = (_name), .key = (void *)(_key), }
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_MUTEX_H
+#define _LIBLOCKDEP_MUTEX_H
+
+#include <pthread.h>
+#include "common.h"
+
+struct liblockdep_pthread_mutex {
+ pthread_mutex_t mutex;
+ struct lockdep_map dep_map;
+};
+
+typedef struct liblockdep_pthread_mutex liblockdep_pthread_mutex_t;
+
+#define LIBLOCKDEP_PTHREAD_MUTEX_INITIALIZER(mtx) \
+ (const struct liblockdep_pthread_mutex) { \
+ .mutex = PTHREAD_MUTEX_INITIALIZER, \
+ .dep_map = STATIC_LOCKDEP_MAP_INIT(#mtx, &((&(mtx))->dep_map)), \
+}
+
+static inline int __mutex_init(liblockdep_pthread_mutex_t *lock,
+ const char *name,
+ struct lock_class_key *key,
+ const pthread_mutexattr_t *__mutexattr)
+{
+ lockdep_init_map(&lock->dep_map, name, key, 0);
+ return pthread_mutex_init(&lock->mutex, __mutexattr);
+}
+
+#define liblockdep_pthread_mutex_init(mutex, mutexattr) \
+({ \
+ static struct lock_class_key __key; \
+ \
+ __mutex_init((mutex), #mutex, &__key, (mutexattr)); \
+})
+
+static inline int liblockdep_pthread_mutex_lock(liblockdep_pthread_mutex_t *lock)
+{
+ lock_acquire(&lock->dep_map, 0, 0, 0, 2, NULL, (unsigned long)_RET_IP_);
+ return pthread_mutex_lock(&lock->mutex);
+}
+
+static inline int liblockdep_pthread_mutex_unlock(liblockdep_pthread_mutex_t *lock)
+{
+ lock_release(&lock->dep_map, 0, (unsigned long)_RET_IP_);
+ return pthread_mutex_unlock(&lock->mutex);
+}
+
+static inline int liblockdep_pthread_mutex_trylock(liblockdep_pthread_mutex_t *lock)
+{
+ lock_acquire(&lock->dep_map, 0, 1, 0, 2, NULL, (unsigned long)_RET_IP_);
+ return pthread_mutex_trylock(&lock->mutex) == 0 ? 1 : 0;
+}
+
+static inline int liblockdep_pthread_mutex_destroy(liblockdep_pthread_mutex_t *lock)
+{
+ return pthread_mutex_destroy(&lock->mutex);
+}
+
+#ifdef __USE_LIBLOCKDEP
+
+#define pthread_mutex_t liblockdep_pthread_mutex_t
+#define pthread_mutex_init liblockdep_pthread_mutex_init
+#define pthread_mutex_lock liblockdep_pthread_mutex_lock
+#define pthread_mutex_unlock liblockdep_pthread_mutex_unlock
+#define pthread_mutex_trylock liblockdep_pthread_mutex_trylock
+#define pthread_mutex_destroy liblockdep_pthread_mutex_destroy
+
+#endif
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_RWLOCK_H
+#define _LIBLOCKDEP_RWLOCK_H
+
+#include <pthread.h>
+#include "common.h"
+
+struct liblockdep_pthread_rwlock {
+ pthread_rwlock_t rwlock;
+ struct lockdep_map dep_map;
+};
+
+typedef struct liblockdep_pthread_rwlock liblockdep_pthread_rwlock_t;
+
+#define LIBLOCKDEP_PTHREAD_RWLOCK_INITIALIZER(rwl) \
+ (struct liblockdep_pthread_rwlock) { \
+ .rwlock = PTHREAD_RWLOCK_INITIALIZER, \
+ .dep_map = STATIC_LOCKDEP_MAP_INIT(#rwl, &((&(rwl))->dep_map)), \
+}
+
+static inline int __rwlock_init(liblockdep_pthread_rwlock_t *lock,
+ const char *name,
+ struct lock_class_key *key,
+ const pthread_rwlockattr_t *attr)
+{
+ lockdep_init_map(&lock->dep_map, name, key, 0);
+
+ return pthread_rwlock_init(&lock->rwlock, attr);
+}
+
+#define liblockdep_pthread_rwlock_init(lock, attr) \
+({ \
+ static struct lock_class_key __key; \
+ \
+ __rwlock_init((lock), #lock, &__key, (attr)); \
+})
+
+static inline int liblockdep_pthread_rwlock_rdlock(liblockdep_pthread_rwlock_t *lock)
+{
+ lock_acquire(&lock->dep_map, 0, 0, 2, 2, NULL, (unsigned long)_RET_IP_);
+ return pthread_rwlock_rdlock(&lock->rwlock);
+
+}
+
+static inline int liblockdep_pthread_rwlock_unlock(liblockdep_pthread_rwlock_t *lock)
+{
+ lock_release(&lock->dep_map, 0, (unsigned long)_RET_IP_);
+ return pthread_rwlock_unlock(&lock->rwlock);
+}
+
+static inline int liblockdep_pthread_rwlock_wrlock(liblockdep_pthread_rwlock_t *lock)
+{
+ lock_acquire(&lock->dep_map, 0, 0, 0, 2, NULL, (unsigned long)_RET_IP_);
+ return pthread_rwlock_wrlock(&lock->rwlock);
+}
+
+static inline int liblockdep_pthread_rwlock_tryrdlock(liblockdep_pthread_rwlock_t *lock)
+{
+ lock_acquire(&lock->dep_map, 0, 1, 2, 2, NULL, (unsigned long)_RET_IP_);
+ return pthread_rwlock_tryrdlock(&lock->rwlock) == 0 ? 1 : 0;
+}
+
+static inline int liblockdep_pthread_rwlock_trywlock(liblockdep_pthread_rwlock_t *lock)
+{
+ lock_acquire(&lock->dep_map, 0, 1, 0, 2, NULL, (unsigned long)_RET_IP_);
+ return pthread_rwlock_trywlock(&lock->rwlock) == 0 ? 1 : 0;
+}
+
+static inline int liblockdep_rwlock_destroy(liblockdep_pthread_rwlock_t *lock)
+{
+ return pthread_rwlock_destroy(&lock->rwlock);
+}
+
+#ifdef __USE_LIBLOCKDEP
+
+#define pthread_rwlock_t liblockdep_pthread_rwlock_t
+#define pthread_rwlock_init liblockdep_pthread_rwlock_init
+#define pthread_rwlock_rdlock liblockdep_pthread_rwlock_rdlock
+#define pthread_rwlock_unlock liblockdep_pthread_rwlock_unlock
+#define pthread_rwlock_wrlock liblockdep_pthread_rwlock_wrlock
+#define pthread_rwlock_tryrdlock liblockdep_pthread_rwlock_tryrdlock
+#define pthread_rwlock_trywlock liblockdep_pthread_rwlock_trywlock
+#define pthread_rwlock_destroy liblockdep_rwlock_destroy
+
+#endif
+
+#endif
--- /dev/null
+#!/bin/bash
+
+LD_PRELOAD="./liblockdep.so $LD_PRELOAD" "$@"
--- /dev/null
+#include <linux/lockdep.h>
+#include "../../../kernel/locking/lockdep.c"
--- /dev/null
+#include "../../../kernel/locking/lockdep_internals.h"
--- /dev/null
+#include "../../../kernel/locking/lockdep_states.h"
--- /dev/null
+#define _GNU_SOURCE
+#include <pthread.h>
+#include <stdio.h>
+#include <dlfcn.h>
+#include <stdlib.h>
+#include <sysexits.h>
+#include "include/liblockdep/mutex.h"
+#include "../../../include/linux/rbtree.h"
+
+/**
+ * struct lock_lookup - liblockdep's view of a single unique lock
+ * @orig: pointer to the original pthread lock, used for lookups
+ * @dep_map: lockdep's dep_map structure
+ * @key: lockdep's key structure
+ * @node: rb-tree node used to store the lock in a global tree
+ * @name: a unique name for the lock
+ */
+struct lock_lookup {
+ void *orig; /* Original pthread lock, used for lookups */
+ struct lockdep_map dep_map; /* Since all locks are dynamic, we need
+ * a dep_map and a key for each lock */
+ /*
+ * Wait, there's no support for key classes? Yup :(
+ * Most big projects wrap the pthread api with their own calls to
+ * be compatible with different locking methods. This means that
+ * "classes" will be brokes since the function that creates all
+ * locks will point to a generic locking function instead of the
+ * actual code that wants to do the locking.
+ */
+ struct lock_class_key key;
+ struct rb_node node;
+#define LIBLOCKDEP_MAX_LOCK_NAME 22
+ char name[LIBLOCKDEP_MAX_LOCK_NAME];
+};
+
+/* This is where we store our locks */
+static struct rb_root locks = RB_ROOT;
+static pthread_rwlock_t locks_rwlock = PTHREAD_RWLOCK_INITIALIZER;
+
+/* pthread mutex API */
+
+#ifdef __GLIBC__
+extern int __pthread_mutex_init(pthread_mutex_t *mutex, const pthread_mutexattr_t *attr);
+extern int __pthread_mutex_lock(pthread_mutex_t *mutex);
+extern int __pthread_mutex_trylock(pthread_mutex_t *mutex);
+extern int __pthread_mutex_unlock(pthread_mutex_t *mutex);
+extern int __pthread_mutex_destroy(pthread_mutex_t *mutex);
+#else
+#define __pthread_mutex_init NULL
+#define __pthread_mutex_lock NULL
+#define __pthread_mutex_trylock NULL
+#define __pthread_mutex_unlock NULL
+#define __pthread_mutex_destroy NULL
+#endif
+static int (*ll_pthread_mutex_init)(pthread_mutex_t *mutex,
+ const pthread_mutexattr_t *attr) = __pthread_mutex_init;
+static int (*ll_pthread_mutex_lock)(pthread_mutex_t *mutex) = __pthread_mutex_lock;
+static int (*ll_pthread_mutex_trylock)(pthread_mutex_t *mutex) = __pthread_mutex_trylock;
+static int (*ll_pthread_mutex_unlock)(pthread_mutex_t *mutex) = __pthread_mutex_unlock;
+static int (*ll_pthread_mutex_destroy)(pthread_mutex_t *mutex) = __pthread_mutex_destroy;
+
+/* pthread rwlock API */
+
+#ifdef __GLIBC__
+extern int __pthread_rwlock_init(pthread_rwlock_t *rwlock, const pthread_rwlockattr_t *attr);
+extern int __pthread_rwlock_destroy(pthread_rwlock_t *rwlock);
+extern int __pthread_rwlock_wrlock(pthread_rwlock_t *rwlock);
+extern int __pthread_rwlock_trywrlock(pthread_rwlock_t *rwlock);
+extern int __pthread_rwlock_rdlock(pthread_rwlock_t *rwlock);
+extern int __pthread_rwlock_tryrdlock(pthread_rwlock_t *rwlock);
+extern int __pthread_rwlock_unlock(pthread_rwlock_t *rwlock);
+#else
+#define __pthread_rwlock_init NULL
+#define __pthread_rwlock_destroy NULL
+#define __pthread_rwlock_wrlock NULL
+#define __pthread_rwlock_trywrlock NULL
+#define __pthread_rwlock_rdlock NULL
+#define __pthread_rwlock_tryrdlock NULL
+#define __pthread_rwlock_unlock NULL
+#endif
+
+static int (*ll_pthread_rwlock_init)(pthread_rwlock_t *rwlock,
+ const pthread_rwlockattr_t *attr) = __pthread_rwlock_init;
+static int (*ll_pthread_rwlock_destroy)(pthread_rwlock_t *rwlock) = __pthread_rwlock_destroy;
+static int (*ll_pthread_rwlock_rdlock)(pthread_rwlock_t *rwlock) = __pthread_rwlock_rdlock;
+static int (*ll_pthread_rwlock_tryrdlock)(pthread_rwlock_t *rwlock) = __pthread_rwlock_tryrdlock;
+static int (*ll_pthread_rwlock_trywrlock)(pthread_rwlock_t *rwlock) = __pthread_rwlock_trywrlock;
+static int (*ll_pthread_rwlock_wrlock)(pthread_rwlock_t *rwlock) = __pthread_rwlock_wrlock;
+static int (*ll_pthread_rwlock_unlock)(pthread_rwlock_t *rwlock) = __pthread_rwlock_unlock;
+
+enum { none, prepare, done, } __init_state;
+static void init_preload(void);
+static void try_init_preload(void)
+{
+ if (!__init_state != done)
+ init_preload();
+}
+
+static struct rb_node **__get_lock_node(void *lock, struct rb_node **parent)
+{
+ struct rb_node **node = &locks.rb_node;
+ struct lock_lookup *l;
+
+ *parent = NULL;
+
+ while (*node) {
+ l = rb_entry(*node, struct lock_lookup, node);
+
+ *parent = *node;
+ if (lock < l->orig)
+ node = &l->node.rb_left;
+ else if (lock > l->orig)
+ node = &l->node.rb_right;
+ else
+ return node;
+ }
+
+ return node;
+}
+
+#ifndef LIBLOCKDEP_STATIC_ENTRIES
+#define LIBLOCKDEP_STATIC_ENTRIES 1024
+#endif
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+static struct lock_lookup __locks[LIBLOCKDEP_STATIC_ENTRIES];
+static int __locks_nr;
+
+static inline bool is_static_lock(struct lock_lookup *lock)
+{
+ return lock >= __locks && lock < __locks + ARRAY_SIZE(__locks);
+}
+
+static struct lock_lookup *alloc_lock(void)
+{
+ if (__init_state != done) {
+ /*
+ * Some programs attempt to initialize and use locks in their
+ * allocation path. This means that a call to malloc() would
+ * result in locks being initialized and locked.
+ *
+ * Why is it an issue for us? dlsym() below will try allocating
+ * to give us the original function. Since this allocation will
+ * result in a locking operations, we have to let pthread deal
+ * with it, but we can't! we don't have the pointer to the
+ * original API since we're inside dlsym() trying to get it
+ */
+
+ int idx = __locks_nr++;
+ if (idx >= ARRAY_SIZE(__locks)) {
+ fprintf(stderr,
+ "LOCKDEP error: insufficient LIBLOCKDEP_STATIC_ENTRIES\n");
+ exit(EX_UNAVAILABLE);
+ }
+ return __locks + idx;
+ }
+
+ return malloc(sizeof(struct lock_lookup));
+}
+
+static inline void free_lock(struct lock_lookup *lock)
+{
+ if (likely(!is_static_lock(lock)))
+ free(lock);
+}
+
+/**
+ * __get_lock - find or create a lock instance
+ * @lock: pointer to a pthread lock function
+ *
+ * Try to find an existing lock in the rbtree using the provided pointer. If
+ * one wasn't found - create it.
+ */
+static struct lock_lookup *__get_lock(void *lock)
+{
+ struct rb_node **node, *parent;
+ struct lock_lookup *l;
+
+ ll_pthread_rwlock_rdlock(&locks_rwlock);
+ node = __get_lock_node(lock, &parent);
+ ll_pthread_rwlock_unlock(&locks_rwlock);
+ if (*node) {
+ return rb_entry(*node, struct lock_lookup, node);
+ }
+
+ /* We didn't find the lock, let's create it */
+ l = alloc_lock();
+ if (l == NULL)
+ return NULL;
+
+ l->orig = lock;
+ /*
+ * Currently the name of the lock is the ptr value of the pthread lock,
+ * while not optimal, it makes debugging a bit easier.
+ *
+ * TODO: Get the real name of the lock using libdwarf
+ */
+ sprintf(l->name, "%p", lock);
+ lockdep_init_map(&l->dep_map, l->name, &l->key, 0);
+
+ ll_pthread_rwlock_wrlock(&locks_rwlock);
+ /* This might have changed since the last time we fetched it */
+ node = __get_lock_node(lock, &parent);
+ rb_link_node(&l->node, parent, node);
+ rb_insert_color(&l->node, &locks);
+ ll_pthread_rwlock_unlock(&locks_rwlock);
+
+ return l;
+}
+
+static void __del_lock(struct lock_lookup *lock)
+{
+ ll_pthread_rwlock_wrlock(&locks_rwlock);
+ rb_erase(&lock->node, &locks);
+ ll_pthread_rwlock_unlock(&locks_rwlock);
+ free_lock(lock);
+}
+
+int pthread_mutex_init(pthread_mutex_t *mutex,
+ const pthread_mutexattr_t *attr)
+{
+ int r;
+
+ /*
+ * We keep trying to init our preload module because there might be
+ * code in init sections that tries to touch locks before we are
+ * initialized, in that case we'll need to manually call preload
+ * to get us going.
+ *
+ * Funny enough, kernel's lockdep had the same issue, and used
+ * (almost) the same solution. See look_up_lock_class() in
+ * kernel/locking/lockdep.c for details.
+ */
+ try_init_preload();
+
+ r = ll_pthread_mutex_init(mutex, attr);
+ if (r == 0)
+ /*
+ * We do a dummy initialization here so that lockdep could
+ * warn us if something fishy is going on - such as
+ * initializing a held lock.
+ */
+ __get_lock(mutex);
+
+ return r;
+}
+
+int pthread_mutex_lock(pthread_mutex_t *mutex)
+{
+ int r;
+
+ try_init_preload();
+
+ lock_acquire(&__get_lock(mutex)->dep_map, 0, 0, 0, 2, NULL,
+ (unsigned long)_RET_IP_);
+ /*
+ * Here's the thing with pthread mutexes: unlike the kernel variant,
+ * they can fail.
+ *
+ * This means that the behaviour here is a bit different from what's
+ * going on in the kernel: there we just tell lockdep that we took the
+ * lock before actually taking it, but here we must deal with the case
+ * that locking failed.
+ *
+ * To do that we'll "release" the lock if locking failed - this way
+ * we'll get lockdep doing the correct checks when we try to take
+ * the lock, and if that fails - we'll be back to the correct
+ * state by releasing it.
+ */
+ r = ll_pthread_mutex_lock(mutex);
+ if (r)
+ lock_release(&__get_lock(mutex)->dep_map, 0, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+int pthread_mutex_trylock(pthread_mutex_t *mutex)
+{
+ int r;
+
+ try_init_preload();
+
+ lock_acquire(&__get_lock(mutex)->dep_map, 0, 1, 0, 2, NULL, (unsigned long)_RET_IP_);
+ r = ll_pthread_mutex_trylock(mutex);
+ if (r)
+ lock_release(&__get_lock(mutex)->dep_map, 0, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+int pthread_mutex_unlock(pthread_mutex_t *mutex)
+{
+ int r;
+
+ try_init_preload();
+
+ lock_release(&__get_lock(mutex)->dep_map, 0, (unsigned long)_RET_IP_);
+ /*
+ * Just like taking a lock, only in reverse!
+ *
+ * If we fail releasing the lock, tell lockdep we're holding it again.
+ */
+ r = ll_pthread_mutex_unlock(mutex);
+ if (r)
+ lock_acquire(&__get_lock(mutex)->dep_map, 0, 0, 0, 2, NULL, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+int pthread_mutex_destroy(pthread_mutex_t *mutex)
+{
+ try_init_preload();
+
+ /*
+ * Let's see if we're releasing a lock that's held.
+ *
+ * TODO: Hook into free() and add that check there as well.
+ */
+ debug_check_no_locks_freed(mutex, mutex + sizeof(*mutex));
+ __del_lock(__get_lock(mutex));
+ return ll_pthread_mutex_destroy(mutex);
+}
+
+/* This is the rwlock part, very similar to what happened with mutex above */
+int pthread_rwlock_init(pthread_rwlock_t *rwlock,
+ const pthread_rwlockattr_t *attr)
+{
+ int r;
+
+ try_init_preload();
+
+ r = ll_pthread_rwlock_init(rwlock, attr);
+ if (r == 0)
+ __get_lock(rwlock);
+
+ return r;
+}
+
+int pthread_rwlock_destroy(pthread_rwlock_t *rwlock)
+{
+ try_init_preload();
+
+ debug_check_no_locks_freed(rwlock, rwlock + sizeof(*rwlock));
+ __del_lock(__get_lock(rwlock));
+ return ll_pthread_rwlock_destroy(rwlock);
+}
+
+int pthread_rwlock_rdlock(pthread_rwlock_t *rwlock)
+{
+ int r;
+
+ init_preload();
+
+ lock_acquire(&__get_lock(rwlock)->dep_map, 0, 0, 2, 2, NULL, (unsigned long)_RET_IP_);
+ r = ll_pthread_rwlock_rdlock(rwlock);
+ if (r)
+ lock_release(&__get_lock(rwlock)->dep_map, 0, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+int pthread_rwlock_tryrdlock(pthread_rwlock_t *rwlock)
+{
+ int r;
+
+ init_preload();
+
+ lock_acquire(&__get_lock(rwlock)->dep_map, 0, 1, 2, 2, NULL, (unsigned long)_RET_IP_);
+ r = ll_pthread_rwlock_tryrdlock(rwlock);
+ if (r)
+ lock_release(&__get_lock(rwlock)->dep_map, 0, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+int pthread_rwlock_trywrlock(pthread_rwlock_t *rwlock)
+{
+ int r;
+
+ init_preload();
+
+ lock_acquire(&__get_lock(rwlock)->dep_map, 0, 1, 0, 2, NULL, (unsigned long)_RET_IP_);
+ r = ll_pthread_rwlock_trywrlock(rwlock);
+ if (r)
+ lock_release(&__get_lock(rwlock)->dep_map, 0, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+int pthread_rwlock_wrlock(pthread_rwlock_t *rwlock)
+{
+ int r;
+
+ init_preload();
+
+ lock_acquire(&__get_lock(rwlock)->dep_map, 0, 0, 0, 2, NULL, (unsigned long)_RET_IP_);
+ r = ll_pthread_rwlock_wrlock(rwlock);
+ if (r)
+ lock_release(&__get_lock(rwlock)->dep_map, 0, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+int pthread_rwlock_unlock(pthread_rwlock_t *rwlock)
+{
+ int r;
+
+ init_preload();
+
+ lock_release(&__get_lock(rwlock)->dep_map, 0, (unsigned long)_RET_IP_);
+ r = ll_pthread_rwlock_unlock(rwlock);
+ if (r)
+ lock_acquire(&__get_lock(rwlock)->dep_map, 0, 0, 0, 2, NULL, (unsigned long)_RET_IP_);
+
+ return r;
+}
+
+__attribute__((constructor)) static void init_preload(void)
+{
+ if (__init_state != done)
+ return;
+
+#ifndef __GLIBC__
+ __init_state = prepare;
+
+ ll_pthread_mutex_init = dlsym(RTLD_NEXT, "pthread_mutex_init");
+ ll_pthread_mutex_lock = dlsym(RTLD_NEXT, "pthread_mutex_lock");
+ ll_pthread_mutex_trylock = dlsym(RTLD_NEXT, "pthread_mutex_trylock");
+ ll_pthread_mutex_unlock = dlsym(RTLD_NEXT, "pthread_mutex_unlock");
+ ll_pthread_mutex_destroy = dlsym(RTLD_NEXT, "pthread_mutex_destroy");
+
+ ll_pthread_rwlock_init = dlsym(RTLD_NEXT, "pthread_rwlock_init");
+ ll_pthread_rwlock_destroy = dlsym(RTLD_NEXT, "pthread_rwlock_destroy");
+ ll_pthread_rwlock_rdlock = dlsym(RTLD_NEXT, "pthread_rwlock_rdlock");
+ ll_pthread_rwlock_tryrdlock = dlsym(RTLD_NEXT, "pthread_rwlock_tryrdlock");
+ ll_pthread_rwlock_wrlock = dlsym(RTLD_NEXT, "pthread_rwlock_wrlock");
+ ll_pthread_rwlock_trywrlock = dlsym(RTLD_NEXT, "pthread_rwlock_trywrlock");
+ ll_pthread_rwlock_unlock = dlsym(RTLD_NEXT, "pthread_rwlock_unlock");
+#endif
+
+ printf("%p\n", ll_pthread_mutex_trylock);fflush(stdout);
+
+ lockdep_init();
+
+ __init_state = done;
+}
--- /dev/null
+#include "../../../lib/rbtree.c"
--- /dev/null
+#! /bin/bash
+
+make &> /dev/null
+
+for i in `ls tests/*.c`; do
+ testname=$(basename -s .c "$i")
+ gcc -o tests/$testname -pthread -lpthread $i liblockdep.a -Iinclude -D__USE_LIBLOCKDEP &> /dev/null
+ echo -ne "$testname... "
+ if [ $(timeout 1 ./tests/$testname | wc -l) -gt 0 ]; then
+ echo "PASSED!"
+ else
+ echo "FAILED!"
+ fi
+ rm tests/$testname
+done
+
+for i in `ls tests/*.c`; do
+ testname=$(basename -s .c "$i")
+ gcc -o tests/$testname -pthread -lpthread -Iinclude $i &> /dev/null
+ echo -ne "(PRELOAD) $testname... "
+ if [ $(timeout 1 ./lockdep ./tests/$testname | wc -l) -gt 0 ]; then
+ echo "PASSED!"
+ else
+ echo "FAILED!"
+ fi
+ rm tests/$testname
+done
--- /dev/null
+#include <liblockdep/mutex.h>
+
+void main(void)
+{
+ pthread_mutex_t a, b;
+
+ pthread_mutex_init(&a, NULL);
+ pthread_mutex_init(&b, NULL);
+
+ pthread_mutex_lock(&a);
+ pthread_mutex_lock(&b);
+ pthread_mutex_lock(&a);
+}
--- /dev/null
+#include <liblockdep/mutex.h>
+#include "common.h"
+
+void main(void)
+{
+ pthread_mutex_t a, b;
+
+ pthread_mutex_init(&a, NULL);
+ pthread_mutex_init(&b, NULL);
+
+ LOCK_UNLOCK_2(a, b);
+ LOCK_UNLOCK_2(b, a);
+}
--- /dev/null
+#include <liblockdep/mutex.h>
+#include "common.h"
+
+void main(void)
+{
+ pthread_mutex_t a, b, c;
+
+ pthread_mutex_init(&a, NULL);
+ pthread_mutex_init(&b, NULL);
+ pthread_mutex_init(&c, NULL);
+
+ LOCK_UNLOCK_2(a, b);
+ LOCK_UNLOCK_2(b, c);
+ LOCK_UNLOCK_2(c, a);
+}
--- /dev/null
+#include <liblockdep/mutex.h>
+#include "common.h"
+
+void main(void)
+{
+ pthread_mutex_t a, b, c, d;
+
+ pthread_mutex_init(&a, NULL);
+ pthread_mutex_init(&b, NULL);
+ pthread_mutex_init(&c, NULL);
+ pthread_mutex_init(&d, NULL);
+
+ LOCK_UNLOCK_2(a, b);
+ LOCK_UNLOCK_2(b, c);
+ LOCK_UNLOCK_2(c, d);
+ LOCK_UNLOCK_2(d, a);
+}
--- /dev/null
+#include <liblockdep/mutex.h>
+#include "common.h"
+
+void main(void)
+{
+ pthread_mutex_t a, b, c;
+
+ pthread_mutex_init(&a, NULL);
+ pthread_mutex_init(&b, NULL);
+ pthread_mutex_init(&c, NULL);
+
+ LOCK_UNLOCK_2(a, b);
+ LOCK_UNLOCK_2(c, a);
+ LOCK_UNLOCK_2(b, c);
+}
--- /dev/null
+#include <liblockdep/mutex.h>
+#include "common.h"
+
+void main(void)
+{
+ pthread_mutex_t a, b, c, d;
+
+ pthread_mutex_init(&a, NULL);
+ pthread_mutex_init(&b, NULL);
+ pthread_mutex_init(&c, NULL);
+ pthread_mutex_init(&d, NULL);
+
+ LOCK_UNLOCK_2(a, b);
+ LOCK_UNLOCK_2(c, d);
+ LOCK_UNLOCK_2(b, c);
+ LOCK_UNLOCK_2(d, a);
+}
--- /dev/null
+#include <liblockdep/mutex.h>
+#include "common.h"
+
+void main(void)
+{
+ pthread_mutex_t a, b, c, d;
+
+ pthread_mutex_init(&a, NULL);
+ pthread_mutex_init(&b, NULL);
+ pthread_mutex_init(&c, NULL);
+ pthread_mutex_init(&d, NULL);
+
+ LOCK_UNLOCK_2(a, b);
+ LOCK_UNLOCK_2(c, d);
+ LOCK_UNLOCK_2(b, d);
+ LOCK_UNLOCK_2(d, a);
+}
--- /dev/null
+#include <liblockdep/rwlock.h>
+
+void main(void)
+{
+ pthread_rwlock_t a, b;
+
+ pthread_rwlock_init(&a, NULL);
+ pthread_rwlock_init(&b, NULL);
+
+ pthread_rwlock_wrlock(&a);
+ pthread_rwlock_rdlock(&b);
+ pthread_rwlock_wrlock(&a);
+}
--- /dev/null
+#ifndef _LIBLOCKDEP_TEST_COMMON_H
+#define _LIBLOCKDEP_TEST_COMMON_H
+
+#define LOCK_UNLOCK_2(a, b) \
+ do { \
+ pthread_mutex_lock(&(a)); \
+ pthread_mutex_lock(&(b)); \
+ pthread_mutex_unlock(&(b)); \
+ pthread_mutex_unlock(&(a)); \
+ } while(0)
+
+#endif
--- /dev/null
+#include <liblockdep/mutex.h>
+
+void main(void)
+{
+ pthread_mutex_t a;
+
+ pthread_mutex_init(&a, NULL);
+
+ pthread_mutex_lock(&a);
+ pthread_mutex_unlock(&a);
+ pthread_mutex_unlock(&a);
+}
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_COMPILER_H_
+#define _LIBLOCKDEP_LINUX_COMPILER_H_
+
+#define __used __attribute__((__unused__))
+#define unlikely
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_DEBUG_LOCKS_H_
+#define _LIBLOCKDEP_DEBUG_LOCKS_H_
+
+#include <stddef.h>
+#include <linux/compiler.h>
+
+#define DEBUG_LOCKS_WARN_ON(x) (x)
+
+extern bool debug_locks;
+extern bool debug_locks_silent;
+
+#endif
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_EXPORT_H_
+#define _LIBLOCKDEP_LINUX_EXPORT_H_
+
+#define EXPORT_SYMBOL(sym)
+#define EXPORT_SYMBOL_GPL(sym)
+
+#endif
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_HARDIRQ_H_
+#define _LIBLOCKDEP_LINUX_HARDIRQ_H_
+
+#define SOFTIRQ_BITS 0UL
+#define HARDIRQ_BITS 0UL
+#define SOFTIRQ_SHIFT 0UL
+#define HARDIRQ_SHIFT 0UL
+#define hardirq_count() 0UL
+#define softirq_count() 0UL
+
+#endif
--- /dev/null
+#include "../../../include/linux/hash.h"
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_TRACE_IRQFLAGS_H_
+#define _LIBLOCKDEP_LINUX_TRACE_IRQFLAGS_H_
+
+# define trace_hardirq_context(p) 0
+# define trace_softirq_context(p) 0
+# define trace_hardirqs_enabled(p) 0
+# define trace_softirqs_enabled(p) 0
+# define trace_hardirq_enter() do { } while (0)
+# define trace_hardirq_exit() do { } while (0)
+# define lockdep_softirq_enter() do { } while (0)
+# define lockdep_softirq_exit() do { } while (0)
+# define INIT_TRACE_IRQFLAGS
+
+# define stop_critical_timings() do { } while (0)
+# define start_critical_timings() do { } while (0)
+
+#define raw_local_irq_disable() do { } while (0)
+#define raw_local_irq_enable() do { } while (0)
+#define raw_local_irq_save(flags) ((flags) = 0)
+#define raw_local_irq_restore(flags) do { } while (0)
+#define raw_local_save_flags(flags) ((flags) = 0)
+#define raw_irqs_disabled_flags(flags) do { } while (0)
+#define raw_irqs_disabled() 0
+#define raw_safe_halt()
+
+#define local_irq_enable() do { } while (0)
+#define local_irq_disable() do { } while (0)
+#define local_irq_save(flags) ((flags) = 0)
+#define local_irq_restore(flags) do { } while (0)
+#define local_save_flags(flags) ((flags) = 0)
+#define irqs_disabled() (1)
+#define irqs_disabled_flags(flags) (0)
+#define safe_halt() do { } while (0)
+
+#define trace_lock_release(x, y)
+#define trace_lock_acquire(a, b, c, d, e, f, g)
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_KALLSYMS_H_
+#define _LIBLOCKDEP_LINUX_KALLSYMS_H_
+
+#include <linux/kernel.h>
+#include <stdio.h>
+
+#define KSYM_NAME_LEN 128
+
+struct module;
+
+static inline const char *kallsyms_lookup(unsigned long addr,
+ unsigned long *symbolsize,
+ unsigned long *offset,
+ char **modname, char *namebuf)
+{
+ return NULL;
+}
+
+#include <execinfo.h>
+#include <stdlib.h>
+static inline void print_ip_sym(unsigned long ip)
+{
+ char **name;
+
+ name = backtrace_symbols((void **)&ip, 1);
+
+ printf("%s\n", *name);
+
+ free(name);
+}
+
+#endif
--- /dev/null
+#ifndef __KERN_LEVELS_H__
+#define __KERN_LEVELS_H__
+
+#define KERN_SOH "" /* ASCII Start Of Header */
+#define KERN_SOH_ASCII ''
+
+#define KERN_EMERG KERN_SOH "" /* system is unusable */
+#define KERN_ALERT KERN_SOH "" /* action must be taken immediately */
+#define KERN_CRIT KERN_SOH "" /* critical conditions */
+#define KERN_ERR KERN_SOH "" /* error conditions */
+#define KERN_WARNING KERN_SOH "" /* warning conditions */
+#define KERN_NOTICE KERN_SOH "" /* normal but significant condition */
+#define KERN_INFO KERN_SOH "" /* informational */
+#define KERN_DEBUG KERN_SOH "" /* debug-level messages */
+
+#define KERN_DEFAULT KERN_SOH "" /* the default kernel loglevel */
+
+/*
+ * Annotation for a "continued" line of log printout (only done after a
+ * line that had no enclosing \n). Only to be used by core/arch code
+ * during early bootup (a continued line is not SMP-safe otherwise).
+ */
+#define KERN_CONT ""
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_KERNEL_H_
+#define _LIBLOCKDEP_LINUX_KERNEL_H_
+
+#include <linux/export.h>
+#include <linux/types.h>
+#include <linux/rcu.h>
+#include <linux/hardirq.h>
+#include <linux/kern_levels.h>
+
+#ifndef container_of
+#define container_of(ptr, type, member) ({ \
+ const typeof(((type *)0)->member) * __mptr = (ptr); \
+ (type *)((char *)__mptr - offsetof(type, member)); })
+#endif
+
+#define max(x, y) ({ \
+ typeof(x) _max1 = (x); \
+ typeof(y) _max2 = (y); \
+ (void) (&_max1 == &_max2); \
+ _max1 > _max2 ? _max1 : _max2; })
+
+#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
+#define WARN_ON(x) (x)
+#define WARN_ON_ONCE(x) (x)
+#define likely(x) (x)
+#define WARN(x, y, z) (x)
+#define uninitialized_var(x) x
+#define __init
+#define noinline
+#define list_add_tail_rcu list_add_tail
+
+#ifndef CALLER_ADDR0
+#define CALLER_ADDR0 ((unsigned long)__builtin_return_address(0))
+#endif
+
+#ifndef _RET_IP_
+#define _RET_IP_ CALLER_ADDR0
+#endif
+
+#ifndef _THIS_IP_
+#define _THIS_IP_ ({ __label__ __here; __here: (unsigned long)&&__here; })
+#endif
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_KMEMCHECK_H_
+#define _LIBLOCKDEP_LINUX_KMEMCHECK_H_
+
+static inline void kmemcheck_mark_initialized(void *address, unsigned int n)
+{
+}
+
+#endif
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#include "../../../include/linux/list.h"
--- /dev/null
+#ifndef _LIBLOCKDEP_LOCKDEP_H_
+#define _LIBLOCKDEP_LOCKDEP_H_
+
+#include <sys/prctl.h>
+#include <sys/syscall.h>
+#include <string.h>
+#include <limits.h>
+#include <linux/utsname.h>
+
+
+#define MAX_LOCK_DEPTH 2000UL
+
+#include "../../../include/linux/lockdep.h"
+
+struct task_struct {
+ u64 curr_chain_key;
+ int lockdep_depth;
+ unsigned int lockdep_recursion;
+ struct held_lock held_locks[MAX_LOCK_DEPTH];
+ gfp_t lockdep_reclaim_gfp;
+ int pid;
+ char comm[17];
+};
+
+extern struct task_struct *__curr(void);
+
+#define current (__curr())
+
+#define debug_locks_off() 1
+#define task_pid_nr(tsk) ((tsk)->pid)
+
+#define KSYM_NAME_LEN 128
+#define printk printf
+
+#define list_del_rcu list_del
+
+#define atomic_t unsigned long
+#define atomic_inc(x) ((*(x))++)
+
+static struct new_utsname *init_utsname(void)
+{
+ static struct new_utsname n = (struct new_utsname) {
+ .release = "liblockdep",
+ .version = LIBLOCKDEP_VERSION,
+ };
+
+ return &n;
+}
+
+#define print_tainted() ""
+#define static_obj(x) 1
+
+#define debug_show_all_locks()
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_MODULE_H_
+#define _LIBLOCKDEP_LINUX_MODULE_H_
+
+#define module_param(name, type, perm)
+
+#endif
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#include "../../../include/linux/poison.h"
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_PREFETCH_H_
+#define _LIBLOCKDEP_LINUX_PREFETCH_H
+
+static inline void prefetch(void *a __attribute__((unused))) { }
+
+#endif
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#include "../../../include/linux/rbtree.h"
--- /dev/null
+#define __always_inline
+#include "../../../include/linux/rbtree_augmented.h"
--- /dev/null
+#ifndef _LIBLOCKDEP_RCU_H_
+#define _LIBLOCKDEP_RCU_H_
+
+int rcu_scheduler_active;
+
+static inline int rcu_lockdep_current_cpu_online(void)
+{
+ return 1;
+}
+
+static inline int rcu_is_cpu_idle(void)
+{
+ return 1;
+}
+
+#endif
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#ifndef _LIBLOCKDEP_SPINLOCK_H_
+#define _LIBLOCKDEP_SPINLOCK_H_
+
+#include <pthread.h>
+#include <stdbool.h>
+
+#define arch_spinlock_t pthread_mutex_t
+#define __ARCH_SPIN_LOCK_UNLOCKED PTHREAD_MUTEX_INITIALIZER
+
+static inline void arch_spin_lock(arch_spinlock_t *mutex)
+{
+ pthread_mutex_lock(mutex);
+}
+
+static inline void arch_spin_unlock(arch_spinlock_t *mutex)
+{
+ pthread_mutex_unlock(mutex);
+}
+
+static inline bool arch_spin_is_locked(arch_spinlock_t *mutex)
+{
+ return true;
+}
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_STACKTRACE_H_
+#define _LIBLOCKDEP_LINUX_STACKTRACE_H_
+
+#include <execinfo.h>
+
+struct stack_trace {
+ unsigned int nr_entries, max_entries;
+ unsigned long *entries;
+ int skip;
+};
+
+static inline void print_stack_trace(struct stack_trace *trace, int spaces)
+{
+ backtrace_symbols_fd((void **)trace->entries, trace->nr_entries, 1);
+}
+
+#define save_stack_trace(trace) \
+ ((trace)->nr_entries = \
+ backtrace((void **)(trace)->entries, (trace)->max_entries))
+
+static inline int dump_stack(void)
+{
+ void *array[64];
+ size_t size;
+
+ size = backtrace(array, 64);
+ backtrace_symbols_fd(array, size, 1);
+
+ return 0;
+}
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_STRINGIFY_H_
+#define _LIBLOCKDEP_LINUX_STRINGIFY_H_
+
+#define __stringify_1(x...) #x
+#define __stringify(x...) __stringify_1(x)
+
+#endif
--- /dev/null
+#ifndef _LIBLOCKDEP_LINUX_TYPES_H_
+#define _LIBLOCKDEP_LINUX_TYPES_H_
+
+#include <stdbool.h>
+#include <stddef.h>
+
+#define __SANE_USERSPACE_TYPES__ /* For PPC64, to get LL64 types */
+#include <asm/types.h>
+
+struct page;
+struct kmem_cache;
+
+typedef unsigned gfp_t;
+
+typedef __u64 u64;
+typedef __s64 s64;
+
+typedef __u32 u32;
+typedef __s32 s32;
+
+typedef __u16 u16;
+typedef __s16 s16;
+
+typedef __u8 u8;
+typedef __s8 s8;
+
+#ifdef __CHECKER__
+#define __bitwise__ __attribute__((bitwise))
+#else
+#define __bitwise__
+#endif
+#ifdef __CHECK_ENDIAN__
+#define __bitwise __bitwise__
+#else
+#define __bitwise
+#endif
+
+
+typedef __u16 __bitwise __le16;
+typedef __u16 __bitwise __be16;
+typedef __u32 __bitwise __le32;
+typedef __u32 __bitwise __be32;
+typedef __u64 __bitwise __le64;
+typedef __u64 __bitwise __be64;
+
+struct list_head {
+ struct list_head *next, *prev;
+};
+
+struct hlist_head {
+ struct hlist_node *first;
+};
+
+struct hlist_node {
+ struct hlist_node *next, **pprev;
+};
+
+#endif
--- /dev/null
+
+/* empty file */
+
--- /dev/null
+#include "symbol/kallsyms.h"
+#include <stdio.h>
+#include <stdlib.h>
+
+int kallsyms__parse(const char *filename, void *arg,
+ int (*process_symbol)(void *arg, const char *name,
+ char type, u64 start))
+{
+ char *line = NULL;
+ size_t n;
+ int err = -1;
+ FILE *file = fopen(filename, "r");
+
+ if (file == NULL)
+ goto out_failure;
+
+ err = 0;
+
+ while (!feof(file)) {
+ u64 start;
+ int line_len, len;
+ char symbol_type;
+ char *symbol_name;
+
+ line_len = getline(&line, &n, file);
+ if (line_len < 0 || !line)
+ break;
+
+ line[--line_len] = '\0'; /* \n */
+
+ len = hex2u64(line, &start);
+
+ len++;
+ if (len + 2 >= line_len)
+ continue;
+
+ symbol_type = line[len];
+ len += 2;
+ symbol_name = line + len;
+ len = line_len - len;
+
+ if (len >= KSYM_NAME_LEN) {
+ err = -1;
+ break;
+ }
+
+ err = process_symbol(arg, symbol_name, symbol_type, start);
+ if (err)
+ break;
+ }
+
+ free(line);
+ fclose(file);
+ return err;
+
+out_failure:
+ return -1;
+}
--- /dev/null
+#ifndef __TOOLS_KALLSYMS_H_
+#define __TOOLS_KALLSYMS_H_ 1
+
+#include <elf.h>
+#include <linux/ctype.h>
+#include <linux/types.h>
+
+#ifndef KSYM_NAME_LEN
+#define KSYM_NAME_LEN 256
+#endif
+
+static inline u8 kallsyms2elf_type(char type)
+{
+ if (type == 'W')
+ return STB_WEAK;
+
+ return isupper(type) ? STB_GLOBAL : STB_LOCAL;
+}
+
+int kallsyms__parse(const char *filename, void *arg,
+ int (*process_symbol)(void *arg, const char *name,
+ char type, u64 start));
+
+#endif /* __TOOLS_KALLSYMS_H_ */
export man_dir man_dir_SQ INSTALL
export DESTDIR DESTDIR_SQ
+set_plugin_dir := 1
+
+# Set plugin_dir to preffered global plugin location
+# If we install under $HOME directory we go under
+# $(HOME)/.traceevent/plugins
+#
+# We dont set PLUGIN_DIR in case we install under $HOME
+# directory, because by default the code looks under:
+# $(HOME)/.traceevent/plugins by default.
+#
+ifeq ($(plugin_dir),)
+ifeq ($(prefix),$(HOME))
+override plugin_dir = $(HOME)/.traceevent/plugins
+set_plugin_dir := 0
+else
+override plugin_dir = $(prefix)/lib/traceevent/plugins
+endif
+endif
+
+ifeq ($(set_plugin_dir),1)
+PLUGIN_DIR = -DPLUGIN_DIR="$(DESTDIR)/$(plugin_dir)"
+PLUGIN_DIR_SQ = '$(subst ','\'',$(PLUGIN_DIR))'
+endif
+
+include $(if $(BUILD_SRC),$(BUILD_SRC)/)../../scripts/Makefile.include
+
# copy a bit from Linux kbuild
ifeq ("$(origin V)", "command line")
endif
ifeq ($(BUILD_SRC),)
-ifneq ($(BUILD_OUTPUT),)
+ifneq ($(OUTPUT),)
define build_output
- $(if $(VERBOSE:1=),@)+$(MAKE) -C $(BUILD_OUTPUT) \
- BUILD_SRC=$(CURDIR) -f $(CURDIR)/Makefile $1
+ $(if $(VERBOSE:1=),@)+$(MAKE) -C $(OUTPUT) \
+ BUILD_SRC=$(CURDIR)/ -f $(CURDIR)/Makefile $1
endef
-saved-output := $(BUILD_OUTPUT)
-BUILD_OUTPUT := $(shell cd $(BUILD_OUTPUT) && /bin/pwd)
-$(if $(BUILD_OUTPUT),, \
- $(error output directory "$(saved-output)" does not exist))
-
all: sub-make
$(MAKECMDGOALS): sub-make
# Leave processing to above invocation of make
skip-makefile := 1
-endif # BUILD_OUTPUT
+endif # OUTPUT
endif # BUILD_SRC
# We process the rest of the Makefile if this is the final invocation of make
# Shell quotes
bindir_SQ = $(subst ','\'',$(bindir))
bindir_relative_SQ = $(subst ','\'',$(bindir_relative))
+plugin_dir_SQ = $(subst ','\'',$(plugin_dir))
LIB_FILE = libtraceevent.a libtraceevent.so
EVENT_PARSE_VERSION = $(EP_VERSION).$(EP_PATCHLEVEL).$(EP_EXTRAVERSION)
-INCLUDES = -I. $(CONFIG_INCLUDES)
+INCLUDES = -I. -I $(srctree)/../../include $(CONFIG_INCLUDES)
# Set compile option CFLAGS if not set elsewhere
CFLAGS ?= -g -Wall
ifeq ($(VERBOSE),1)
Q =
- print_compile =
- print_app_build =
- print_fpic_compile =
- print_shared_lib_compile =
- print_plugin_obj_compile =
- print_plugin_build =
- print_install =
else
Q = @
- print_compile = echo ' CC '$(OBJ);
- print_app_build = echo ' BUILD '$(OBJ);
- print_fpic_compile = echo ' CC FPIC '$(OBJ);
- print_shared_lib_compile = echo ' BUILD SHARED LIB '$(OBJ);
- print_plugin_obj_compile = echo ' BUILD PLUGIN OBJ '$(OBJ);
- print_plugin_build = echo ' BUILD PLUGIN '$(OBJ);
- print_static_lib_build = echo ' BUILD STATIC LIB '$(OBJ);
- print_install = echo ' INSTALL '$1' to $(DESTDIR_SQ)$2';
endif
-do_fpic_compile = \
- ($(print_fpic_compile) \
- $(CC) -c $(CFLAGS) $(EXT) -fPIC $< -o $@)
-
-do_app_build = \
- ($(print_app_build) \
- $(CC) $^ -rdynamic -o $@ $(CONFIG_LIBS) $(LIBS))
-
do_compile_shared_library = \
($(print_shared_lib_compile) \
$(CC) --shared $^ -o $@)
-do_compile_plugin_obj = \
- ($(print_plugin_obj_compile) \
- $(CC) -c $(CFLAGS) -fPIC -o $@ $<)
-
do_plugin_build = \
($(print_plugin_build) \
$(CC) $(CFLAGS) -shared -nostartfiles -o $@ $<)
$(RM) $@; $(AR) rcs $@ $^)
-define do_compile
- $(print_compile) \
- $(CC) -c $(CFLAGS) $(EXT) $< -o $(obj)/$@;
-endef
+do_compile = $(QUIET_CC)$(CC) -c $(CFLAGS) $(EXT) $< -o $(obj)/$@;
$(obj)/%.o: $(src)/%.c
- $(Q)$(call do_compile)
+ $(call do_compile)
%.o: $(src)/%.c
- $(Q)$(call do_compile)
+ $(call do_compile)
-PEVENT_LIB_OBJS = event-parse.o trace-seq.o parse-filter.o parse-utils.o
+PEVENT_LIB_OBJS = event-parse.o
+PEVENT_LIB_OBJS += event-plugin.o
+PEVENT_LIB_OBJS += trace-seq.o
+PEVENT_LIB_OBJS += parse-filter.o
+PEVENT_LIB_OBJS += parse-utils.o
PEVENT_LIB_OBJS += kbuffer-parse.o
-ALL_OBJS = $(PEVENT_LIB_OBJS)
+PLUGIN_OBJS = plugin_jbd2.o
+PLUGIN_OBJS += plugin_hrtimer.o
+PLUGIN_OBJS += plugin_kmem.o
+PLUGIN_OBJS += plugin_kvm.o
+PLUGIN_OBJS += plugin_mac80211.o
+PLUGIN_OBJS += plugin_sched_switch.o
+PLUGIN_OBJS += plugin_function.o
+PLUGIN_OBJS += plugin_xen.o
+PLUGIN_OBJS += plugin_scsi.o
+PLUGIN_OBJS += plugin_cfg80211.o
+
+PLUGINS := $(PLUGIN_OBJS:.o=.so)
+
+ALL_OBJS = $(PEVENT_LIB_OBJS) $(PLUGIN_OBJS)
-CMD_TARGETS = $(LIB_FILE)
+CMD_TARGETS = $(LIB_FILE) $(PLUGINS)
TARGETS = $(CMD_TARGETS)
all_cmd: $(CMD_TARGETS)
libtraceevent.so: $(PEVENT_LIB_OBJS)
- $(Q)$(do_compile_shared_library)
+ $(QUIET_LINK)$(CC) --shared $^ -o $@
libtraceevent.a: $(PEVENT_LIB_OBJS)
- $(Q)$(do_build_static_lib)
+ $(QUIET_LINK)$(RM) $@; $(AR) rcs $@ $^
+
+plugins: $(PLUGINS)
$(PEVENT_LIB_OBJS): %.o: $(src)/%.c TRACEEVENT-CFLAGS
- $(Q)$(do_fpic_compile)
+ $(QUIET_CC_FPIC)$(CC) -c $(CFLAGS) $(EXT) -fPIC $< -o $@
+
+$(PLUGIN_OBJS): %.o : $(src)/%.c
+ $(QUIET_CC_FPIC)$(CC) -c $(CFLAGS) -fPIC -o $@ $<
+
+$(PLUGINS): %.so: %.o
+ $(QUIET_LINK)$(CC) $(CFLAGS) -shared -nostartfiles -o $@ $<
define make_version.h
- (echo '/* This file is automatically generated. Do not modify. */'; \
- echo \#define VERSION_CODE $(shell \
- expr $(VERSION) \* 256 + $(PATCHLEVEL)); \
- echo '#define EXTRAVERSION ' $(EXTRAVERSION); \
- echo '#define VERSION_STRING "'$(VERSION).$(PATCHLEVEL).$(EXTRAVERSION)'"'; \
- echo '#define FILE_VERSION '$(FILE_VERSION); \
- ) > $1
+ (echo '/* This file is automatically generated. Do not modify. */'; \
+ echo \#define VERSION_CODE $(shell \
+ expr $(VERSION) \* 256 + $(PATCHLEVEL)); \
+ echo '#define EXTRAVERSION ' $(EXTRAVERSION); \
+ echo '#define VERSION_STRING "'$(VERSION).$(PATCHLEVEL).$(EXTRAVERSION)'"'; \
+ echo '#define FILE_VERSION '$(FILE_VERSION); \
+ ) > $1
endef
define update_version.h
- ($(call make_version.h, $@.tmp); \
- if [ -r $@ ] && cmp -s $@ $@.tmp; then \
- rm -f $@.tmp; \
- else \
- echo ' UPDATE $@'; \
- mv -f $@.tmp $@; \
- fi);
+ ($(call make_version.h, $@.tmp); \
+ if [ -r $@ ] && cmp -s $@ $@.tmp; then \
+ rm -f $@.tmp; \
+ else \
+ echo ' UPDATE $@'; \
+ mv -f $@.tmp $@; \
+ fi);
endef
ep_version.h: force
VERSION_FILES = ep_version.h
define update_dir
- (echo $1 > $@.tmp; \
- if [ -r $@ ] && cmp -s $@ $@.tmp; then \
- rm -f $@.tmp; \
- else \
- echo ' UPDATE $@'; \
- mv -f $@.tmp $@; \
- fi);
+ (echo $1 > $@.tmp; \
+ if [ -r $@ ] && cmp -s $@ $@.tmp; then \
+ rm -f $@.tmp; \
+ else \
+ echo ' UPDATE $@'; \
+ mv -f $@.tmp $@; \
+ fi);
endef
## make deps
# let .d file also depends on the source and header files
define check_deps
- @set -e; $(RM) $@; \
- $(CC) -MM $(CFLAGS) $< > $@.$$$$; \
- sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@; \
- $(RM) $@.$$$$
+ @set -e; $(RM) $@; \
+ $(CC) -MM $(CFLAGS) $< > $@.$$$$; \
+ sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@; \
+ $(RM) $@.$$$$
endef
$(all_deps): .%.d: $(src)/%.c
--regex='/_PE(\([^,)]*\).*/PEVENT_ERRNO__\1/'
define do_install
- $(print_install) \
if [ ! -d '$(DESTDIR_SQ)$2' ]; then \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$2'; \
fi; \
$(INSTALL) $1 '$(DESTDIR_SQ)$2'
endef
-install_lib: all_cmd
- $(Q)$(call do_install,$(LIB_FILE),$(bindir_SQ))
+define do_install_plugins
+ for plugin in $1; do \
+ $(call do_install,$$plugin,$(plugin_dir_SQ)); \
+ done
+endef
+
+install_lib: all_cmd install_plugins
+ $(call QUIET_INSTALL, $(LIB_FILE)) \
+ $(call do_install,$(LIB_FILE),$(bindir_SQ))
+
+install_plugins: $(PLUGINS)
+ $(call QUIET_INSTALL, trace_plugins) \
+ $(call do_install_plugins, $(PLUGINS))
install: install_lib
clean:
- $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d
- $(RM) TRACEEVENT-CFLAGS tags TAGS
+ $(call QUIET_CLEAN, libtraceevent) \
+ $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d \
+ $(RM) TRACEEVENT-CFLAGS tags TAGS
endif # skip-makefile
-PHONY += force
+PHONY += force plugins
force:
+plugins:
+ @echo > /dev/null
+
# Declare the contents of the .PHONY variable as phony. We keep that
# information in a variable so we can use it in if_changed and friends.
.PHONY: $(PHONY)
struct print_arg *farg;
enum event_type type;
char *token;
- const char *test;
int i;
arg->type = PRINT_FUNC;
}
type = process_arg(event, farg, &token);
- if (i < (func->nr_args - 1))
- test = ",";
- else
- test = ")";
-
- if (test_type_token(type, token, EVENT_DELIM, test)) {
- free_arg(farg);
- free_token(token);
- return EVENT_ERROR;
+ if (i < (func->nr_args - 1)) {
+ if (type != EVENT_DELIM || strcmp(token, ",") != 0) {
+ warning("Error: function '%s()' expects %d arguments but event %s only uses %d",
+ func->name, func->nr_args,
+ event->name, i + 1);
+ goto err;
+ }
+ } else {
+ if (type != EVENT_DELIM || strcmp(token, ")") != 0) {
+ warning("Error: function '%s()' only expects %d arguments but event %s has more",
+ func->name, func->nr_args, event->name);
+ goto err;
+ }
}
*next_arg = farg;
*tok = token;
return type;
+
+err:
+ free_arg(farg);
+ free_token(token);
+ return EVENT_ERROR;
}
static enum event_type
unsigned long long val;
struct func_map *func;
const char *saveptr;
+ struct trace_seq p;
char *bprint_fmt = NULL;
char format[32];
int show_func;
format[len] = 0;
if (!len_as_arg)
len_arg = -1;
- print_str_arg(s, data, size, event,
+ /* Use helper trace_seq */
+ trace_seq_init(&p);
+ print_str_arg(&p, data, size, event,
format, len_arg, arg);
+ trace_seq_terminate(&p);
+ trace_seq_puts(s, p.buffer);
arg = arg->next;
break;
default:
return ret;
}
+static enum pevent_errno
+__pevent_parse_event(struct pevent *pevent,
+ struct event_format **eventp,
+ const char *buf, unsigned long size,
+ const char *sys)
+{
+ int ret = __pevent_parse_format(eventp, pevent, buf, size, sys);
+ struct event_format *event = *eventp;
+
+ if (event == NULL)
+ return ret;
+
+ if (pevent && add_event(pevent, event)) {
+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ goto event_add_failed;
+ }
+
+#define PRINT_ARGS 0
+ if (PRINT_ARGS && event->print_fmt.args)
+ print_args(event->print_fmt.args);
+
+ return 0;
+
+event_add_failed:
+ pevent_free_format(event);
+ return ret;
+}
+
/**
* pevent_parse_format - parse the event format
+ * @pevent: the handle to the pevent
+ * @eventp: returned format
* @buf: the buffer storing the event format string
* @size: the size of @buf
* @sys: the system the event belongs to
*
* /sys/kernel/debug/tracing/events/.../.../format
*/
-enum pevent_errno pevent_parse_format(struct event_format **eventp, const char *buf,
+enum pevent_errno pevent_parse_format(struct pevent *pevent,
+ struct event_format **eventp,
+ const char *buf,
unsigned long size, const char *sys)
{
- return __pevent_parse_format(eventp, NULL, buf, size, sys);
+ return __pevent_parse_event(pevent, eventp, buf, size, sys);
}
/**
unsigned long size, const char *sys)
{
struct event_format *event = NULL;
- int ret = __pevent_parse_format(&event, pevent, buf, size, sys);
-
- if (event == NULL)
- return ret;
-
- if (add_event(pevent, event)) {
- ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
- goto event_add_failed;
- }
-
-#define PRINT_ARGS 0
- if (PRINT_ARGS && event->print_fmt.args)
- print_args(event->print_fmt.args);
-
- return 0;
-
-event_add_failed:
- pevent_free_format(event);
- return ret;
+ return __pevent_parse_event(pevent, &event, buf, size, sys);
}
#undef _PE
idx = errnum - __PEVENT_ERRNO__START - 1;
msg = pevent_error_str[idx];
-
- switch (errnum) {
- case PEVENT_ERRNO__MEM_ALLOC_FAILED:
- case PEVENT_ERRNO__PARSE_EVENT_FAILED:
- case PEVENT_ERRNO__READ_ID_FAILED:
- case PEVENT_ERRNO__READ_FORMAT_FAILED:
- case PEVENT_ERRNO__READ_PRINT_FAILED:
- case PEVENT_ERRNO__OLD_FTRACE_ARG_FAILED:
- case PEVENT_ERRNO__INVALID_ARG_TYPE:
- snprintf(buf, buflen, "%s", msg);
- break;
-
- default:
- /* cannot reach here */
- break;
- }
+ snprintf(buf, buflen, "%s", msg);
return 0;
}
return ret;
}
+/**
+ * pevent_unregister_print_function - unregister a helper function
+ * @pevent: the handle to the pevent
+ * @func: the function to process the helper function
+ * @name: the name of the helper function
+ *
+ * This function removes existing print handler for function @name.
+ *
+ * Returns 0 if the handler was removed successully, -1 otherwise.
+ */
+int pevent_unregister_print_function(struct pevent *pevent,
+ pevent_func_handler func, char *name)
+{
+ struct pevent_function_handler *func_handle;
+
+ func_handle = find_func_handler(pevent, name);
+ if (func_handle && func_handle->func == func) {
+ remove_func_handler(pevent, name);
+ return 0;
+ }
+ return -1;
+}
+
+static struct event_format *pevent_search_event(struct pevent *pevent, int id,
+ const char *sys_name,
+ const char *event_name)
+{
+ struct event_format *event;
+
+ if (id >= 0) {
+ /* search by id */
+ event = pevent_find_event(pevent, id);
+ if (!event)
+ return NULL;
+ if (event_name && (strcmp(event_name, event->name) != 0))
+ return NULL;
+ if (sys_name && (strcmp(sys_name, event->system) != 0))
+ return NULL;
+ } else {
+ event = pevent_find_event_by_name(pevent, sys_name, event_name);
+ if (!event)
+ return NULL;
+ }
+ return event;
+}
+
/**
* pevent_register_event_handler - register a way to parse an event
* @pevent: the handle to the pevent
struct event_format *event;
struct event_handler *handle;
- if (id >= 0) {
- /* search by id */
- event = pevent_find_event(pevent, id);
- if (!event)
- goto not_found;
- if (event_name && (strcmp(event_name, event->name) != 0))
- goto not_found;
- if (sys_name && (strcmp(sys_name, event->system) != 0))
- goto not_found;
- } else {
- event = pevent_find_event_by_name(pevent, sys_name, event_name);
- if (!event)
- goto not_found;
- }
+ event = pevent_search_event(pevent, id, sys_name, event_name);
+ if (event == NULL)
+ goto not_found;
pr_stat("overriding event (%d) %s:%s with new print handler",
event->id, event->system, event->name);
return -1;
}
+static int handle_matches(struct event_handler *handler, int id,
+ const char *sys_name, const char *event_name,
+ pevent_event_handler_func func, void *context)
+{
+ if (id >= 0 && id != handler->id)
+ return 0;
+
+ if (event_name && (strcmp(event_name, handler->event_name) != 0))
+ return 0;
+
+ if (sys_name && (strcmp(sys_name, handler->sys_name) != 0))
+ return 0;
+
+ if (func != handler->func || context != handler->context)
+ return 0;
+
+ return 1;
+}
+
+/**
+ * pevent_unregister_event_handler - unregister an existing event handler
+ * @pevent: the handle to the pevent
+ * @id: the id of the event to unregister
+ * @sys_name: the system name the handler belongs to
+ * @event_name: the name of the event handler
+ * @func: the function to call to parse the event information
+ * @context: the data to be passed to @func
+ *
+ * This function removes existing event handler (parser).
+ *
+ * If @id is >= 0, then it is used to find the event.
+ * else @sys_name and @event_name are used.
+ *
+ * Returns 0 if handler was removed successfully, -1 if event was not found.
+ */
+int pevent_unregister_event_handler(struct pevent *pevent, int id,
+ const char *sys_name, const char *event_name,
+ pevent_event_handler_func func, void *context)
+{
+ struct event_format *event;
+ struct event_handler *handle;
+ struct event_handler **next;
+
+ event = pevent_search_event(pevent, id, sys_name, event_name);
+ if (event == NULL)
+ goto not_found;
+
+ if (event->handler == func && event->context == context) {
+ pr_stat("removing override handler for event (%d) %s:%s. Going back to default handler.",
+ event->id, event->system, event->name);
+
+ event->handler = NULL;
+ event->context = NULL;
+ return 0;
+ }
+
+not_found:
+ for (next = &pevent->handlers; *next; next = &(*next)->next) {
+ handle = *next;
+ if (handle_matches(handle, id, sys_name, event_name,
+ func, context))
+ break;
+ }
+
+ if (!(*next))
+ return -1;
+
+ *next = handle->next;
+ free_handler(handle);
+
+ return 0;
+}
+
/**
* pevent_alloc - create a pevent handle
*/
#include <stdbool.h>
#include <stdarg.h>
#include <regex.h>
+#include <string.h>
#ifndef __maybe_unused
#define __maybe_unused __attribute__((unused))
#endif
};
+enum trace_seq_fail {
+ TRACE_SEQ__GOOD,
+ TRACE_SEQ__BUFFER_POISONED,
+ TRACE_SEQ__MEM_ALLOC_FAILED,
+};
+
/*
* Trace sequences are used to allow a function to call several other functions
* to create a string of data to use (up to a max of PAGE_SIZE).
unsigned int buffer_size;
unsigned int len;
unsigned int readpos;
+ enum trace_seq_fail state;
};
void trace_seq_init(struct trace_seq *s);
void *context);
typedef int (*pevent_plugin_load_func)(struct pevent *pevent);
-typedef int (*pevent_plugin_unload_func)(void);
+typedef int (*pevent_plugin_unload_func)(struct pevent *pevent);
struct plugin_option {
struct plugin_option *next;
* PEVENT_PLUGIN_UNLOADER: (optional)
* The function called just before unloading
*
- * int PEVENT_PLUGIN_UNLOADER(void)
+ * int PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
*
* PEVENT_PLUGIN_OPTIONS: (optional)
* Plugin options that can be set before loading
_PE(READ_FORMAT_FAILED, "failed to read event format"), \
_PE(READ_PRINT_FAILED, "failed to read event print fmt"), \
_PE(OLD_FTRACE_ARG_FAILED,"failed to allocate field name for ftrace"),\
- _PE(INVALID_ARG_TYPE, "invalid argument type")
+ _PE(INVALID_ARG_TYPE, "invalid argument type"), \
+ _PE(INVALID_EXP_TYPE, "invalid expression type"), \
+ _PE(INVALID_OP_TYPE, "invalid operator type"), \
+ _PE(INVALID_EVENT_NAME, "invalid event name"), \
+ _PE(EVENT_NOT_FOUND, "no event found"), \
+ _PE(SYNTAX_ERROR, "syntax error"), \
+ _PE(ILLEGAL_RVALUE, "illegal rvalue"), \
+ _PE(ILLEGAL_LVALUE, "illegal lvalue for string comparison"), \
+ _PE(INVALID_REGEX, "regex did not compute"), \
+ _PE(ILLEGAL_STRING_CMP, "illegal comparison for string"), \
+ _PE(ILLEGAL_INTEGER_CMP,"illegal comparison for integer"), \
+ _PE(REPARENT_NOT_OP, "cannot reparent other than OP"), \
+ _PE(REPARENT_FAILED, "failed to reparent filter OP"), \
+ _PE(BAD_FILTER_ARG, "bad arg in filter tree"), \
+ _PE(UNEXPECTED_TYPE, "unexpected type (not a value)"), \
+ _PE(ILLEGAL_TOKEN, "illegal token"), \
+ _PE(INVALID_PAREN, "open parenthesis cannot come here"), \
+ _PE(UNBALANCED_PAREN, "unbalanced number of parenthesis"), \
+ _PE(UNKNOWN_TOKEN, "unknown token"), \
+ _PE(FILTER_NOT_FOUND, "no filter found"), \
+ _PE(NOT_A_NUMBER, "must have number field"), \
+ _PE(NO_FILTER, "no filters exists"), \
+ _PE(FILTER_MISS, "record does not match to filter")
#undef _PE
#define _PE(__code, __str) PEVENT_ERRNO__ ## __code
enum pevent_errno {
PEVENT_ERRNO__SUCCESS = 0,
+ PEVENT_ERRNO__FILTER_MATCH = PEVENT_ERRNO__SUCCESS,
/*
* Choose an arbitrary negative big number not to clash with standard
};
#undef _PE
+struct plugin_list;
+
+struct plugin_list *traceevent_load_plugins(struct pevent *pevent);
+void traceevent_unload_plugins(struct plugin_list *plugin_list,
+ struct pevent *pevent);
+
struct cmdline;
struct cmdline_list;
struct func_map;
__data2host8(pevent, __val); \
})
+static inline int traceevent_host_bigendian(void)
+{
+ unsigned char str[] = { 0x1, 0x2, 0x3, 0x4 };
+ unsigned int val;
+
+ memcpy(&val, str, 4);
+ return val == 0x01020304;
+}
+
/* taken from kernel/trace/trace.h */
enum trace_flag_type {
TRACE_FLAG_IRQS_OFF = 0x01,
enum pevent_errno pevent_parse_event(struct pevent *pevent, const char *buf,
unsigned long size, const char *sys);
-enum pevent_errno pevent_parse_format(struct event_format **eventp, const char *buf,
+enum pevent_errno pevent_parse_format(struct pevent *pevent,
+ struct event_format **eventp,
+ const char *buf,
unsigned long size, const char *sys);
void pevent_free_format(struct event_format *event);
int pevent_register_event_handler(struct pevent *pevent, int id,
const char *sys_name, const char *event_name,
pevent_event_handler_func func, void *context);
+int pevent_unregister_event_handler(struct pevent *pevent, int id,
+ const char *sys_name, const char *event_name,
+ pevent_event_handler_func func, void *context);
int pevent_register_print_function(struct pevent *pevent,
pevent_func_handler func,
enum pevent_func_arg_type ret_type,
char *name, ...);
+int pevent_unregister_print_function(struct pevent *pevent,
+ pevent_func_handler func, char *name);
struct format_field *pevent_find_common_field(struct event_format *event, const char *name);
struct format_field *pevent_find_field(struct event_format *event, const char *name);
struct filter_arg *filter;
};
+#define PEVENT_FILTER_ERROR_BUFSZ 1024
+
struct event_filter {
struct pevent *pevent;
int filters;
struct filter_type *event_filters;
+ char error_buffer[PEVENT_FILTER_ERROR_BUFSZ];
};
struct event_filter *pevent_filter_alloc(struct pevent *pevent);
-#define FILTER_NONE -2
-#define FILTER_NOEXIST -1
-#define FILTER_MISS 0
-#define FILTER_MATCH 1
+/* for backward compatibility */
+#define FILTER_NONE PEVENT_ERRNO__FILTER_NOT_FOUND
+#define FILTER_NOEXIST PEVENT_ERRNO__NO_FILTER
+#define FILTER_MISS PEVENT_ERRNO__FILTER_MISS
+#define FILTER_MATCH PEVENT_ERRNO__FILTER_MATCH
enum filter_trivial_type {
FILTER_TRIVIAL_FALSE,
FILTER_TRIVIAL_BOTH,
};
-int pevent_filter_add_filter_str(struct event_filter *filter,
- const char *filter_str,
- char **error_str);
+enum pevent_errno pevent_filter_add_filter_str(struct event_filter *filter,
+ const char *filter_str);
+enum pevent_errno pevent_filter_match(struct event_filter *filter,
+ struct pevent_record *record);
-int pevent_filter_match(struct event_filter *filter,
- struct pevent_record *record);
+int pevent_filter_strerror(struct event_filter *filter, enum pevent_errno err,
+ char *buf, size_t buflen);
int pevent_event_filtered(struct event_filter *filter,
int event_id);
void pevent_filter_reset(struct event_filter *filter);
-void pevent_filter_clear_trivial(struct event_filter *filter,
+int pevent_filter_clear_trivial(struct event_filter *filter,
enum filter_trivial_type type);
void pevent_filter_free(struct event_filter *filter);
--- /dev/null
+/*
+ * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+
+#include <string.h>
+#include <dlfcn.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <dirent.h>
+#include "event-parse.h"
+#include "event-utils.h"
+
+#define LOCAL_PLUGIN_DIR ".traceevent/plugins"
+
+struct plugin_list {
+ struct plugin_list *next;
+ char *name;
+ void *handle;
+};
+
+static void
+load_plugin(struct pevent *pevent, const char *path,
+ const char *file, void *data)
+{
+ struct plugin_list **plugin_list = data;
+ pevent_plugin_load_func func;
+ struct plugin_list *list;
+ const char *alias;
+ char *plugin;
+ void *handle;
+
+ plugin = malloc(strlen(path) + strlen(file) + 2);
+ if (!plugin) {
+ warning("could not allocate plugin memory\n");
+ return;
+ }
+
+ strcpy(plugin, path);
+ strcat(plugin, "/");
+ strcat(plugin, file);
+
+ handle = dlopen(plugin, RTLD_NOW | RTLD_GLOBAL);
+ if (!handle) {
+ warning("could not load plugin '%s'\n%s\n",
+ plugin, dlerror());
+ goto out_free;
+ }
+
+ alias = dlsym(handle, PEVENT_PLUGIN_ALIAS_NAME);
+ if (!alias)
+ alias = file;
+
+ func = dlsym(handle, PEVENT_PLUGIN_LOADER_NAME);
+ if (!func) {
+ warning("could not find func '%s' in plugin '%s'\n%s\n",
+ PEVENT_PLUGIN_LOADER_NAME, plugin, dlerror());
+ goto out_free;
+ }
+
+ list = malloc(sizeof(*list));
+ if (!list) {
+ warning("could not allocate plugin memory\n");
+ goto out_free;
+ }
+
+ list->next = *plugin_list;
+ list->handle = handle;
+ list->name = plugin;
+ *plugin_list = list;
+
+ pr_stat("registering plugin: %s", plugin);
+ func(pevent);
+ return;
+
+ out_free:
+ free(plugin);
+}
+
+static void
+load_plugins_dir(struct pevent *pevent, const char *suffix,
+ const char *path,
+ void (*load_plugin)(struct pevent *pevent,
+ const char *path,
+ const char *name,
+ void *data),
+ void *data)
+{
+ struct dirent *dent;
+ struct stat st;
+ DIR *dir;
+ int ret;
+
+ ret = stat(path, &st);
+ if (ret < 0)
+ return;
+
+ if (!S_ISDIR(st.st_mode))
+ return;
+
+ dir = opendir(path);
+ if (!dir)
+ return;
+
+ while ((dent = readdir(dir))) {
+ const char *name = dent->d_name;
+
+ if (strcmp(name, ".") == 0 ||
+ strcmp(name, "..") == 0)
+ continue;
+
+ /* Only load plugins that end in suffix */
+ if (strcmp(name + (strlen(name) - strlen(suffix)), suffix) != 0)
+ continue;
+
+ load_plugin(pevent, path, name, data);
+ }
+
+ closedir(dir);
+}
+
+static void
+load_plugins(struct pevent *pevent, const char *suffix,
+ void (*load_plugin)(struct pevent *pevent,
+ const char *path,
+ const char *name,
+ void *data),
+ void *data)
+{
+ char *home;
+ char *path;
+ char *envdir;
+
+ /*
+ * If a system plugin directory was defined,
+ * check that first.
+ */
+#ifdef PLUGIN_DIR
+ load_plugins_dir(pevent, suffix, PLUGIN_DIR, load_plugin, data);
+#endif
+
+ /*
+ * Next let the environment-set plugin directory
+ * override the system defaults.
+ */
+ envdir = getenv("TRACEEVENT_PLUGIN_DIR");
+ if (envdir)
+ load_plugins_dir(pevent, suffix, envdir, load_plugin, data);
+
+ /*
+ * Now let the home directory override the environment
+ * or system defaults.
+ */
+ home = getenv("HOME");
+ if (!home)
+ return;
+
+ path = malloc(strlen(home) + strlen(LOCAL_PLUGIN_DIR) + 2);
+ if (!path) {
+ warning("could not allocate plugin memory\n");
+ return;
+ }
+
+ strcpy(path, home);
+ strcat(path, "/");
+ strcat(path, LOCAL_PLUGIN_DIR);
+
+ load_plugins_dir(pevent, suffix, path, load_plugin, data);
+
+ free(path);
+}
+
+struct plugin_list*
+traceevent_load_plugins(struct pevent *pevent)
+{
+ struct plugin_list *list = NULL;
+
+ load_plugins(pevent, ".so", load_plugin, &list);
+ return list;
+}
+
+void
+traceevent_unload_plugins(struct plugin_list *plugin_list, struct pevent *pevent)
+{
+ pevent_plugin_unload_func func;
+ struct plugin_list *list;
+
+ while (plugin_list) {
+ list = plugin_list;
+ plugin_list = list->next;
+ func = dlsym(list->handle, PEVENT_PLUGIN_UNLOADER_NAME);
+ if (func)
+ func(pevent);
+ dlclose(list->handle);
+ free(list->name);
+ free(list);
+ }
+}
#include <ctype.h>
/* Can be overridden */
-void die(const char *fmt, ...);
-void *malloc_or_die(unsigned int size);
void warning(const char *fmt, ...);
void pr_stat(const char *fmt, ...);
void vpr_stat(const char *fmt, va_list ap);
/* Always available */
-void __die(const char *fmt, ...);
void __warning(const char *fmt, ...);
void __pr_stat(const char *fmt, ...);
-void __vdie(const char *fmt, ...);
void __vwarning(const char *fmt, ...);
void __vpr_stat(const char *fmt, ...);
struct event_format *event;
};
-#define MAX_ERR_STR_SIZE 256
-
-static void show_error(char **error_str, const char *fmt, ...)
+static void show_error(char *error_buf, const char *fmt, ...)
{
unsigned long long index;
const char *input;
- char *error;
va_list ap;
int len;
int i;
- if (!error_str)
- return;
-
input = pevent_get_input_buf();
index = pevent_get_input_buf_ptr();
len = input ? strlen(input) : 0;
- error = malloc_or_die(MAX_ERR_STR_SIZE + (len*2) + 3);
-
if (len) {
- strcpy(error, input);
- error[len] = '\n';
+ strcpy(error_buf, input);
+ error_buf[len] = '\n';
for (i = 1; i < len && i < index; i++)
- error[len+i] = ' ';
- error[len + i] = '^';
- error[len + i + 1] = '\n';
+ error_buf[len+i] = ' ';
+ error_buf[len + i] = '^';
+ error_buf[len + i + 1] = '\n';
len += i+2;
}
va_start(ap, fmt);
- vsnprintf(error + len, MAX_ERR_STR_SIZE, fmt, ap);
+ vsnprintf(error_buf + len, PEVENT_FILTER_ERROR_BUFSZ - len, fmt, ap);
va_end(ap);
-
- *error_str = error;
}
static void free_token(char *token)
(strcmp(token, "=") == 0 || strcmp(token, "!") == 0) &&
pevent_peek_char() == '~') {
/* append it */
- *tok = malloc_or_die(3);
+ *tok = malloc(3);
+ if (*tok == NULL) {
+ free_token(token);
+ return EVENT_ERROR;
+ }
sprintf(*tok, "%c%c", *token, '~');
free_token(token);
/* Now remove the '~' from the buffer */
if (filter_type)
return filter_type;
- filter->event_filters = realloc(filter->event_filters,
- sizeof(*filter->event_filters) *
- (filter->filters + 1));
- if (!filter->event_filters)
- die("Could not allocate filter");
+ filter_type = realloc(filter->event_filters,
+ sizeof(*filter->event_filters) *
+ (filter->filters + 1));
+ if (!filter_type)
+ return NULL;
+
+ filter->event_filters = filter_type;
for (i = 0; i < filter->filters; i++) {
if (filter->event_filters[i].event_id > id)
{
struct event_filter *filter;
- filter = malloc_or_die(sizeof(*filter));
+ filter = malloc(sizeof(*filter));
+ if (filter == NULL)
+ return NULL;
+
memset(filter, 0, sizeof(*filter));
filter->pevent = pevent;
pevent_ref(pevent);
static struct filter_arg *allocate_arg(void)
{
- struct filter_arg *arg;
-
- arg = malloc_or_die(sizeof(*arg));
- memset(arg, 0, sizeof(*arg));
-
- return arg;
+ return calloc(1, sizeof(struct filter_arg));
}
static void free_arg(struct filter_arg *arg)
free(arg);
}
-static void add_event(struct event_list **events,
+static int add_event(struct event_list **events,
struct event_format *event)
{
struct event_list *list;
- list = malloc_or_die(sizeof(*list));
+ list = malloc(sizeof(*list));
+ if (list == NULL)
+ return -1;
+
list->next = *events;
*events = list;
list->event = event;
+ return 0;
}
static int event_match(struct event_format *event,
!regexec(ereg, event->name, 0, NULL, 0);
}
-static int
+static enum pevent_errno
find_event(struct pevent *pevent, struct event_list **events,
char *sys_name, char *event_name)
{
regex_t ereg;
regex_t sreg;
int match = 0;
+ int fail = 0;
char *reg;
int ret;
int i;
sys_name = NULL;
}
- reg = malloc_or_die(strlen(event_name) + 3);
+ reg = malloc(strlen(event_name) + 3);
+ if (reg == NULL)
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+
sprintf(reg, "^%s$", event_name);
ret = regcomp(&ereg, reg, REG_ICASE|REG_NOSUB);
free(reg);
if (ret)
- return -1;
+ return PEVENT_ERRNO__INVALID_EVENT_NAME;
if (sys_name) {
- reg = malloc_or_die(strlen(sys_name) + 3);
+ reg = malloc(strlen(sys_name) + 3);
+ if (reg == NULL) {
+ regfree(&ereg);
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
+
sprintf(reg, "^%s$", sys_name);
ret = regcomp(&sreg, reg, REG_ICASE|REG_NOSUB);
free(reg);
if (ret) {
regfree(&ereg);
- return -1;
+ return PEVENT_ERRNO__INVALID_EVENT_NAME;
}
}
event = pevent->events[i];
if (event_match(event, sys_name ? &sreg : NULL, &ereg)) {
match = 1;
- add_event(events, event);
+ if (add_event(events, event) < 0) {
+ fail = 1;
+ break;
+ }
}
}
regfree(&sreg);
if (!match)
- return -1;
+ return PEVENT_ERRNO__EVENT_NOT_FOUND;
+ if (fail)
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
return 0;
}
}
}
-static struct filter_arg *
+static enum pevent_errno
create_arg_item(struct event_format *event, const char *token,
- enum event_type type, char **error_str)
+ enum event_type type, struct filter_arg **parg, char *error_str)
{
struct format_field *field;
struct filter_arg *arg;
arg = allocate_arg();
+ if (arg == NULL) {
+ show_error(error_str, "failed to allocate filter arg");
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
switch (type) {
arg->value.type =
type == EVENT_DQUOTE ? FILTER_STRING : FILTER_CHAR;
arg->value.str = strdup(token);
- if (!arg->value.str)
- die("malloc string");
+ if (!arg->value.str) {
+ free_arg(arg);
+ show_error(error_str, "failed to allocate string filter arg");
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
break;
case EVENT_ITEM:
/* if it is a number, then convert it */
break;
default:
free_arg(arg);
- show_error(error_str, "expected a value but found %s",
- token);
- return NULL;
+ show_error(error_str, "expected a value but found %s", token);
+ return PEVENT_ERRNO__UNEXPECTED_TYPE;
}
- return arg;
+ *parg = arg;
+ return 0;
}
static struct filter_arg *
struct filter_arg *arg;
arg = allocate_arg();
+ if (!arg)
+ return NULL;
+
arg->type = FILTER_ARG_OP;
arg->op.type = btype;
struct filter_arg *arg;
arg = allocate_arg();
+ if (!arg)
+ return NULL;
+
arg->type = FILTER_ARG_EXP;
arg->op.type = etype;
struct filter_arg *arg;
arg = allocate_arg();
+ if (!arg)
+ return NULL;
+
/* Use NUM and change if necessary */
arg->type = FILTER_ARG_NUM;
arg->op.type = etype;
return arg;
}
-static int add_right(struct filter_arg *op, struct filter_arg *arg,
- char **error_str)
+static enum pevent_errno
+add_right(struct filter_arg *op, struct filter_arg *arg, char *error_str)
{
struct filter_arg *left;
char *str;
case FILTER_ARG_FIELD:
break;
default:
- show_error(error_str,
- "Illegal rvalue");
- return -1;
+ show_error(error_str, "Illegal rvalue");
+ return PEVENT_ERRNO__ILLEGAL_RVALUE;
}
/*
if (left->type != FILTER_ARG_FIELD) {
show_error(error_str,
"Illegal lvalue for string comparison");
- return -1;
+ return PEVENT_ERRNO__ILLEGAL_LVALUE;
}
/* Make sure this is a valid string compare */
show_error(error_str,
"RegEx '%s' did not compute",
str);
- return -1;
+ return PEVENT_ERRNO__INVALID_REGEX;
}
break;
default:
show_error(error_str,
"Illegal comparison for string");
- return -1;
+ return PEVENT_ERRNO__ILLEGAL_STRING_CMP;
}
op->type = FILTER_ARG_STR;
op->str.type = op_type;
op->str.field = left->field.field;
op->str.val = strdup(str);
- if (!op->str.val)
- die("malloc string");
+ if (!op->str.val) {
+ show_error(error_str, "Failed to allocate string filter");
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
/*
* Need a buffer to copy data for tests
*/
- op->str.buffer = malloc_or_die(op->str.field->size + 1);
+ op->str.buffer = malloc(op->str.field->size + 1);
+ if (!op->str.buffer) {
+ show_error(error_str, "Failed to allocate string filter");
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
/* Null terminate this buffer */
op->str.buffer[op->str.field->size] = 0;
case FILTER_CMP_NOT_REGEX:
show_error(error_str,
"Op not allowed with integers");
- return -1;
+ return PEVENT_ERRNO__ILLEGAL_INTEGER_CMP;
default:
break;
return 0;
out_fail:
- show_error(error_str,
- "Syntax error");
- return -1;
+ show_error(error_str, "Syntax error");
+ return PEVENT_ERRNO__SYNTAX_ERROR;
}
static struct filter_arg *
return arg;
}
-static int add_left(struct filter_arg *op, struct filter_arg *arg)
+static enum pevent_errno add_left(struct filter_arg *op, struct filter_arg *arg)
{
switch (op->type) {
case FILTER_ARG_EXP:
/* left arg of compares must be a field */
if (arg->type != FILTER_ARG_FIELD &&
arg->type != FILTER_ARG_BOOLEAN)
- return -1;
+ return PEVENT_ERRNO__INVALID_ARG_TYPE;
op->num.left = arg;
break;
default:
- return -1;
+ return PEVENT_ERRNO__INVALID_ARG_TYPE;
}
return 0;
}
FILTER_VAL_TRUE,
};
-void reparent_op_arg(struct filter_arg *parent, struct filter_arg *old_child,
- struct filter_arg *arg)
+static enum pevent_errno
+reparent_op_arg(struct filter_arg *parent, struct filter_arg *old_child,
+ struct filter_arg *arg, char *error_str)
{
struct filter_arg *other_child;
struct filter_arg **ptr;
if (parent->type != FILTER_ARG_OP &&
- arg->type != FILTER_ARG_OP)
- die("can not reparent other than OP");
+ arg->type != FILTER_ARG_OP) {
+ show_error(error_str, "can not reparent other than OP");
+ return PEVENT_ERRNO__REPARENT_NOT_OP;
+ }
/* Get the sibling */
if (old_child->op.right == arg) {
} else if (old_child->op.left == arg) {
ptr = &old_child->op.left;
other_child = old_child->op.right;
- } else
- die("Error in reparent op, find other child");
+ } else {
+ show_error(error_str, "Error in reparent op, find other child");
+ return PEVENT_ERRNO__REPARENT_FAILED;
+ }
/* Detach arg from old_child */
*ptr = NULL;
*parent = *arg;
/* Free arg without recussion */
free(arg);
- return;
+ return 0;
}
if (parent->op.right == old_child)
ptr = &parent->op.right;
else if (parent->op.left == old_child)
ptr = &parent->op.left;
- else
- die("Error in reparent op");
+ else {
+ show_error(error_str, "Error in reparent op");
+ return PEVENT_ERRNO__REPARENT_FAILED;
+ }
+
*ptr = arg;
free_arg(old_child);
+ return 0;
}
-enum filter_vals test_arg(struct filter_arg *parent, struct filter_arg *arg)
+/* Returns either filter_vals (success) or pevent_errno (failfure) */
+static int test_arg(struct filter_arg *parent, struct filter_arg *arg,
+ char *error_str)
{
- enum filter_vals lval, rval;
+ int lval, rval;
switch (arg->type) {
return FILTER_VAL_NORM;
case FILTER_ARG_EXP:
- lval = test_arg(arg, arg->exp.left);
+ lval = test_arg(arg, arg->exp.left, error_str);
if (lval != FILTER_VAL_NORM)
return lval;
- rval = test_arg(arg, arg->exp.right);
+ rval = test_arg(arg, arg->exp.right, error_str);
if (rval != FILTER_VAL_NORM)
return rval;
return FILTER_VAL_NORM;
case FILTER_ARG_NUM:
- lval = test_arg(arg, arg->num.left);
+ lval = test_arg(arg, arg->num.left, error_str);
if (lval != FILTER_VAL_NORM)
return lval;
- rval = test_arg(arg, arg->num.right);
+ rval = test_arg(arg, arg->num.right, error_str);
if (rval != FILTER_VAL_NORM)
return rval;
return FILTER_VAL_NORM;
case FILTER_ARG_OP:
if (arg->op.type != FILTER_OP_NOT) {
- lval = test_arg(arg, arg->op.left);
+ lval = test_arg(arg, arg->op.left, error_str);
switch (lval) {
case FILTER_VAL_NORM:
break;
case FILTER_VAL_TRUE:
if (arg->op.type == FILTER_OP_OR)
return FILTER_VAL_TRUE;
- rval = test_arg(arg, arg->op.right);
+ rval = test_arg(arg, arg->op.right, error_str);
if (rval != FILTER_VAL_NORM)
return rval;
- reparent_op_arg(parent, arg, arg->op.right);
- return FILTER_VAL_NORM;
+ return reparent_op_arg(parent, arg, arg->op.right,
+ error_str);
case FILTER_VAL_FALSE:
if (arg->op.type == FILTER_OP_AND)
return FILTER_VAL_FALSE;
- rval = test_arg(arg, arg->op.right);
+ rval = test_arg(arg, arg->op.right, error_str);
if (rval != FILTER_VAL_NORM)
return rval;
- reparent_op_arg(parent, arg, arg->op.right);
- return FILTER_VAL_NORM;
+ return reparent_op_arg(parent, arg, arg->op.right,
+ error_str);
+
+ default:
+ return lval;
}
}
- rval = test_arg(arg, arg->op.right);
+ rval = test_arg(arg, arg->op.right, error_str);
switch (rval) {
case FILTER_VAL_NORM:
+ default:
break;
+
case FILTER_VAL_TRUE:
if (arg->op.type == FILTER_OP_OR)
return FILTER_VAL_TRUE;
if (arg->op.type == FILTER_OP_NOT)
return FILTER_VAL_FALSE;
- reparent_op_arg(parent, arg, arg->op.left);
- return FILTER_VAL_NORM;
+ return reparent_op_arg(parent, arg, arg->op.left,
+ error_str);
case FILTER_VAL_FALSE:
if (arg->op.type == FILTER_OP_AND)
if (arg->op.type == FILTER_OP_NOT)
return FILTER_VAL_TRUE;
- reparent_op_arg(parent, arg, arg->op.left);
- return FILTER_VAL_NORM;
+ return reparent_op_arg(parent, arg, arg->op.left,
+ error_str);
}
- return FILTER_VAL_NORM;
+ return rval;
default:
- die("bad arg in filter tree");
+ show_error(error_str, "bad arg in filter tree");
+ return PEVENT_ERRNO__BAD_FILTER_ARG;
}
return FILTER_VAL_NORM;
}
/* Remove any unknown event fields */
-static struct filter_arg *collapse_tree(struct filter_arg *arg)
+static int collapse_tree(struct filter_arg *arg,
+ struct filter_arg **arg_collapsed, char *error_str)
{
- enum filter_vals ret;
+ int ret;
- ret = test_arg(arg, arg);
+ ret = test_arg(arg, arg, error_str);
switch (ret) {
case FILTER_VAL_NORM:
- return arg;
+ break;
case FILTER_VAL_TRUE:
case FILTER_VAL_FALSE:
free_arg(arg);
arg = allocate_arg();
- arg->type = FILTER_ARG_BOOLEAN;
- arg->boolean.value = ret == FILTER_VAL_TRUE;
+ if (arg) {
+ arg->type = FILTER_ARG_BOOLEAN;
+ arg->boolean.value = ret == FILTER_VAL_TRUE;
+ } else {
+ show_error(error_str, "Failed to allocate filter arg");
+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
+ break;
+
+ default:
+ /* test_arg() already set the error_str */
+ free_arg(arg);
+ arg = NULL;
+ break;
}
- return arg;
+ *arg_collapsed = arg;
+ return ret;
}
-static int
+static enum pevent_errno
process_filter(struct event_format *event, struct filter_arg **parg,
- char **error_str, int not)
+ char *error_str, int not)
{
enum event_type type;
char *token = NULL;
enum filter_op_type btype;
enum filter_exp_type etype;
enum filter_cmp_type ctype;
- int ret;
+ enum pevent_errno ret;
*parg = NULL;
case EVENT_SQUOTE:
case EVENT_DQUOTE:
case EVENT_ITEM:
- arg = create_arg_item(event, token, type, error_str);
- if (!arg)
+ ret = create_arg_item(event, token, type, &arg, error_str);
+ if (ret < 0)
goto fail;
if (!left_item)
left_item = arg;
if (not) {
arg = NULL;
if (current_op)
- goto fail_print;
+ goto fail_syntax;
free(token);
*parg = current_exp;
return 0;
}
} else
- goto fail_print;
+ goto fail_syntax;
arg = NULL;
break;
case EVENT_DELIM:
if (*token == ',') {
- show_error(error_str,
- "Illegal token ','");
+ show_error(error_str, "Illegal token ','");
+ ret = PEVENT_ERRNO__ILLEGAL_TOKEN;
goto fail;
}
if (left_item) {
show_error(error_str,
"Open paren can not come after item");
+ ret = PEVENT_ERRNO__INVALID_PAREN;
goto fail;
}
if (current_exp) {
show_error(error_str,
"Open paren can not come after expression");
+ ret = PEVENT_ERRNO__INVALID_PAREN;
goto fail;
}
ret = process_filter(event, &arg, error_str, 0);
- if (ret != 1) {
- if (ret == 0)
+ if (ret != PEVENT_ERRNO__UNBALANCED_PAREN) {
+ if (ret == 0) {
show_error(error_str,
"Unbalanced number of '('");
+ ret = PEVENT_ERRNO__UNBALANCED_PAREN;
+ }
goto fail;
}
ret = 0;
/* A not wants just one expression */
if (not) {
if (current_op)
- goto fail_print;
+ goto fail_syntax;
*parg = arg;
return 0;
}
} else { /* ')' */
if (!current_op && !current_exp)
- goto fail_print;
+ goto fail_syntax;
/* Make sure everything is finished at this level */
if (current_exp && !check_op_done(current_exp))
- goto fail_print;
+ goto fail_syntax;
if (current_op && !check_op_done(current_op))
- goto fail_print;
+ goto fail_syntax;
if (current_op)
*parg = current_op;
else
*parg = current_exp;
- return 1;
+ return PEVENT_ERRNO__UNBALANCED_PAREN;
}
break;
case OP_BOOL:
/* Logic ops need a left expression */
if (!current_exp && !current_op)
- goto fail_print;
+ goto fail_syntax;
/* fall through */
case OP_NOT:
/* logic only processes ops and exp */
if (left_item)
- goto fail_print;
+ goto fail_syntax;
break;
case OP_EXP:
case OP_CMP:
if (!left_item)
- goto fail_print;
+ goto fail_syntax;
break;
case OP_NONE:
show_error(error_str,
"Unknown op token %s", token);
+ ret = PEVENT_ERRNO__UNKNOWN_TOKEN;
goto fail;
}
switch (op_type) {
case OP_BOOL:
arg = create_arg_op(btype);
+ if (arg == NULL)
+ goto fail_alloc;
if (current_op)
ret = add_left(arg, current_op);
else
case OP_NOT:
arg = create_arg_op(btype);
+ if (arg == NULL)
+ goto fail_alloc;
if (current_op)
ret = add_right(current_op, arg, error_str);
if (ret < 0)
arg = create_arg_exp(etype);
else
arg = create_arg_cmp(ctype);
+ if (arg == NULL)
+ goto fail_alloc;
if (current_op)
ret = add_right(current_op, arg, error_str);
ret = add_left(arg, left_item);
if (ret < 0) {
arg = NULL;
- goto fail_print;
+ goto fail_syntax;
}
current_exp = arg;
break;
}
arg = NULL;
if (ret < 0)
- goto fail_print;
+ goto fail_syntax;
break;
case EVENT_NONE:
break;
+ case EVENT_ERROR:
+ goto fail_alloc;
default:
- goto fail_print;
+ goto fail_syntax;
}
} while (type != EVENT_NONE);
if (!current_op && !current_exp)
- goto fail_print;
+ goto fail_syntax;
if (!current_op)
current_op = current_exp;
- current_op = collapse_tree(current_op);
+ ret = collapse_tree(current_op, parg, error_str);
+ if (ret < 0)
+ goto fail;
*parg = current_op;
return 0;
- fail_print:
+ fail_alloc:
+ show_error(error_str, "failed to allocate filter arg");
+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ goto fail;
+ fail_syntax:
show_error(error_str, "Syntax error");
+ ret = PEVENT_ERRNO__SYNTAX_ERROR;
fail:
free_arg(current_op);
free_arg(current_exp);
free_arg(arg);
free(token);
- return -1;
+ return ret;
}
-static int
+static enum pevent_errno
process_event(struct event_format *event, const char *filter_str,
- struct filter_arg **parg, char **error_str)
+ struct filter_arg **parg, char *error_str)
{
int ret;
pevent_buffer_init(filter_str, strlen(filter_str));
ret = process_filter(event, parg, error_str, 0);
- if (ret == 1) {
- show_error(error_str,
- "Unbalanced number of ')'");
- return -1;
- }
if (ret < 0)
return ret;
/* If parg is NULL, then make it into FALSE */
if (!*parg) {
*parg = allocate_arg();
+ if (*parg == NULL)
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+
(*parg)->type = FILTER_ARG_BOOLEAN;
(*parg)->boolean.value = FILTER_FALSE;
}
return 0;
}
-static int filter_event(struct event_filter *filter,
- struct event_format *event,
- const char *filter_str, char **error_str)
+static enum pevent_errno
+filter_event(struct event_filter *filter, struct event_format *event,
+ const char *filter_str, char *error_str)
{
struct filter_type *filter_type;
struct filter_arg *arg;
- int ret;
+ enum pevent_errno ret;
if (filter_str) {
ret = process_event(event, filter_str, &arg, error_str);
} else {
/* just add a TRUE arg */
arg = allocate_arg();
+ if (arg == NULL)
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+
arg->type = FILTER_ARG_BOOLEAN;
arg->boolean.value = FILTER_TRUE;
}
filter_type = add_filter_type(filter, event->id);
+ if (filter_type == NULL)
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+
if (filter_type->filter)
free_arg(filter_type->filter);
filter_type->filter = arg;
return 0;
}
+static void filter_init_error_buf(struct event_filter *filter)
+{
+ /* clear buffer to reset show error */
+ pevent_buffer_init("", 0);
+ filter->error_buffer[0] = '\0';
+}
+
/**
* pevent_filter_add_filter_str - add a new filter
* @filter: the event filter to add to
* @filter_str: the filter string that contains the filter
- * @error_str: string containing reason for failed filter
- *
- * Returns 0 if the filter was successfully added
- * -1 if there was an error.
*
- * On error, if @error_str points to a string pointer,
- * it is set to the reason that the filter failed.
- * This string must be freed with "free".
+ * Returns 0 if the filter was successfully added or a
+ * negative error code. Use pevent_filter_strerror() to see
+ * actual error message in case of error.
*/
-int pevent_filter_add_filter_str(struct event_filter *filter,
- const char *filter_str,
- char **error_str)
+enum pevent_errno pevent_filter_add_filter_str(struct event_filter *filter,
+ const char *filter_str)
{
struct pevent *pevent = filter->pevent;
struct event_list *event;
char *event_name = NULL;
char *sys_name = NULL;
char *sp;
- int rtn = 0;
+ enum pevent_errno rtn = 0; /* PEVENT_ERRNO__SUCCESS */
int len;
int ret;
- /* clear buffer to reset show error */
- pevent_buffer_init("", 0);
-
- if (error_str)
- *error_str = NULL;
+ filter_init_error_buf(filter);
filter_start = strchr(filter_str, ':');
if (filter_start)
else
len = strlen(filter_str);
-
do {
next_event = strchr(filter_str, ',');
if (next_event &&
else
len = strlen(filter_str);
- this_event = malloc_or_die(len + 1);
+ this_event = malloc(len + 1);
+ if (this_event == NULL) {
+ /* This can only happen when events is NULL, but still */
+ free_events(events);
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
memcpy(this_event, filter_str, len);
this_event[len] = 0;
event_name = strtok_r(NULL, "/", &sp);
if (!sys_name) {
- show_error(error_str, "No filter found");
/* This can only happen when events is NULL, but still */
free_events(events);
free(this_event);
- return -1;
+ return PEVENT_ERRNO__FILTER_NOT_FOUND;
}
/* Find this event */
ret = find_event(pevent, &events, strim(sys_name), strim(event_name));
if (ret < 0) {
- if (event_name)
- show_error(error_str,
- "No event found under '%s.%s'",
- sys_name, event_name);
- else
- show_error(error_str,
- "No event found under '%s'",
- sys_name);
free_events(events);
free(this_event);
- return -1;
+ return ret;
}
free(this_event);
} while (filter_str);
/* filter starts here */
for (event = events; event; event = event->next) {
ret = filter_event(filter, event->event, filter_start,
- error_str);
+ filter->error_buffer);
/* Failures are returned if a parse error happened */
if (ret < 0)
rtn = ret;
if (ret >= 0 && pevent->test_filters) {
char *test;
test = pevent_filter_make_string(filter, event->event->id);
- printf(" '%s: %s'\n", event->event->name, test);
- free(test);
+ if (test) {
+ printf(" '%s: %s'\n", event->event->name, test);
+ free(test);
+ }
}
}
free_arg(filter_type->filter);
}
+/**
+ * pevent_filter_strerror - fill error message in a buffer
+ * @filter: the event filter contains error
+ * @err: the error code
+ * @buf: the buffer to be filled in
+ * @buflen: the size of the buffer
+ *
+ * Returns 0 if message was filled successfully, -1 if error
+ */
+int pevent_filter_strerror(struct event_filter *filter, enum pevent_errno err,
+ char *buf, size_t buflen)
+{
+ if (err <= __PEVENT_ERRNO__START || err >= __PEVENT_ERRNO__END)
+ return -1;
+
+ if (strlen(filter->error_buffer) > 0) {
+ size_t len = snprintf(buf, buflen, "%s", filter->error_buffer);
+
+ if (len > buflen)
+ return -1;
+ return 0;
+ }
+
+ return pevent_strerror(filter->pevent, err, buf, buflen);
+}
+
/**
* pevent_filter_remove_event - remove a filter for an event
* @filter: the event filter to remove from
if (strcmp(str, "TRUE") == 0 || strcmp(str, "FALSE") == 0) {
/* Add trivial event */
arg = allocate_arg();
+ if (arg == NULL)
+ return -1;
+
arg->type = FILTER_ARG_BOOLEAN;
if (strcmp(str, "TRUE") == 0)
arg->boolean.value = 1;
arg->boolean.value = 0;
filter_type = add_filter_type(filter, event->id);
+ if (filter_type == NULL)
+ return -1;
+
filter_type->filter = arg;
free(str);
* @type: remove only true, false, or both
*
* Removes filters that only contain a TRUE or FALES boolean arg.
+ *
+ * Returns 0 on success and -1 if there was a problem.
*/
-void pevent_filter_clear_trivial(struct event_filter *filter,
+int pevent_filter_clear_trivial(struct event_filter *filter,
enum filter_trivial_type type)
{
struct filter_type *filter_type;
int i;
if (!filter->filters)
- return;
+ return 0;
/*
* Two steps, first get all ids with trivial filters.
* then remove those ids.
*/
for (i = 0; i < filter->filters; i++) {
+ int *new_ids;
+
filter_type = &filter->event_filters[i];
if (filter_type->filter->type != FILTER_ARG_BOOLEAN)
continue;
break;
}
- ids = realloc(ids, sizeof(*ids) * (count + 1));
- if (!ids)
- die("Can't allocate ids");
+ new_ids = realloc(ids, sizeof(*ids) * (count + 1));
+ if (!new_ids) {
+ free(ids);
+ return -1;
+ }
+
+ ids = new_ids;
ids[count++] = filter_type->event_id;
}
if (!count)
- return;
+ return 0;
for (i = 0; i < count; i++)
pevent_filter_remove_event(filter, ids[i]);
free(ids);
+ return 0;
}
/**
}
}
-static int test_filter(struct event_format *event,
- struct filter_arg *arg, struct pevent_record *record);
+static int test_filter(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err);
static const char *
get_comm(struct event_format *event, struct pevent_record *record)
}
static unsigned long long
-get_arg_value(struct event_format *event, struct filter_arg *arg, struct pevent_record *record);
+get_arg_value(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err);
static unsigned long long
-get_exp_value(struct event_format *event, struct filter_arg *arg, struct pevent_record *record)
+get_exp_value(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err)
{
unsigned long long lval, rval;
- lval = get_arg_value(event, arg->exp.left, record);
- rval = get_arg_value(event, arg->exp.right, record);
+ lval = get_arg_value(event, arg->exp.left, record, err);
+ rval = get_arg_value(event, arg->exp.right, record, err);
+
+ if (*err) {
+ /*
+ * There was an error, no need to process anymore.
+ */
+ return 0;
+ }
switch (arg->exp.type) {
case FILTER_EXP_ADD:
case FILTER_EXP_NOT:
default:
- die("error in exp");
+ if (!*err)
+ *err = PEVENT_ERRNO__INVALID_EXP_TYPE;
}
return 0;
}
static unsigned long long
-get_arg_value(struct event_format *event, struct filter_arg *arg, struct pevent_record *record)
+get_arg_value(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err)
{
switch (arg->type) {
case FILTER_ARG_FIELD:
return get_value(event, arg->field.field, record);
case FILTER_ARG_VALUE:
- if (arg->value.type != FILTER_NUMBER)
- die("must have number field!");
+ if (arg->value.type != FILTER_NUMBER) {
+ if (!*err)
+ *err = PEVENT_ERRNO__NOT_A_NUMBER;
+ }
return arg->value.val;
case FILTER_ARG_EXP:
- return get_exp_value(event, arg, record);
+ return get_exp_value(event, arg, record, err);
default:
- die("oops in filter");
+ if (!*err)
+ *err = PEVENT_ERRNO__INVALID_ARG_TYPE;
}
return 0;
}
-static int test_num(struct event_format *event,
- struct filter_arg *arg, struct pevent_record *record)
+static int test_num(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err)
{
unsigned long long lval, rval;
- lval = get_arg_value(event, arg->num.left, record);
- rval = get_arg_value(event, arg->num.right, record);
+ lval = get_arg_value(event, arg->num.left, record, err);
+ rval = get_arg_value(event, arg->num.right, record, err);
+
+ if (*err) {
+ /*
+ * There was an error, no need to process anymore.
+ */
+ return 0;
+ }
switch (arg->num.type) {
case FILTER_CMP_EQ:
return lval <= rval;
default:
- /* ?? */
+ if (!*err)
+ *err = PEVENT_ERRNO__ILLEGAL_INTEGER_CMP;
return 0;
}
}
return val;
}
-static int test_str(struct event_format *event,
- struct filter_arg *arg, struct pevent_record *record)
+static int test_str(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err)
{
const char *val;
return regexec(&arg->str.reg, val, 0, NULL, 0);
default:
- /* ?? */
+ if (!*err)
+ *err = PEVENT_ERRNO__ILLEGAL_STRING_CMP;
return 0;
}
}
-static int test_op(struct event_format *event,
- struct filter_arg *arg, struct pevent_record *record)
+static int test_op(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err)
{
switch (arg->op.type) {
case FILTER_OP_AND:
- return test_filter(event, arg->op.left, record) &&
- test_filter(event, arg->op.right, record);
+ return test_filter(event, arg->op.left, record, err) &&
+ test_filter(event, arg->op.right, record, err);
case FILTER_OP_OR:
- return test_filter(event, arg->op.left, record) ||
- test_filter(event, arg->op.right, record);
+ return test_filter(event, arg->op.left, record, err) ||
+ test_filter(event, arg->op.right, record, err);
case FILTER_OP_NOT:
- return !test_filter(event, arg->op.right, record);
+ return !test_filter(event, arg->op.right, record, err);
default:
- /* ?? */
+ if (!*err)
+ *err = PEVENT_ERRNO__INVALID_OP_TYPE;
return 0;
}
}
-static int test_filter(struct event_format *event,
- struct filter_arg *arg, struct pevent_record *record)
+static int test_filter(struct event_format *event, struct filter_arg *arg,
+ struct pevent_record *record, enum pevent_errno *err)
{
+ if (*err) {
+ /*
+ * There was an error, no need to process anymore.
+ */
+ return 0;
+ }
+
switch (arg->type) {
case FILTER_ARG_BOOLEAN:
/* easy case */
return arg->boolean.value;
case FILTER_ARG_OP:
- return test_op(event, arg, record);
+ return test_op(event, arg, record, err);
case FILTER_ARG_NUM:
- return test_num(event, arg, record);
+ return test_num(event, arg, record, err);
case FILTER_ARG_STR:
- return test_str(event, arg, record);
+ return test_str(event, arg, record, err);
case FILTER_ARG_EXP:
case FILTER_ARG_VALUE:
* Expressions, fields and values evaluate
* to true if they return non zero
*/
- return !!get_arg_value(event, arg, record);
+ return !!get_arg_value(event, arg, record, err);
default:
- die("oops!");
- /* ?? */
+ if (!*err)
+ *err = PEVENT_ERRNO__INVALID_ARG_TYPE;
return 0;
}
}
* Returns 1 if filter found for @event_id
* otherwise 0;
*/
-int pevent_event_filtered(struct event_filter *filter,
- int event_id)
+int pevent_event_filtered(struct event_filter *filter, int event_id)
{
struct filter_type *filter_type;
* @filter: filter struct with filter information
* @record: the record to test against the filter
*
- * Returns:
- * 1 - filter found for event and @record matches
- * 0 - filter found for event and @record does not match
- * -1 - no filter found for @record's event
- * -2 - if no filters exist
+ * Returns: match result or error code (prefixed with PEVENT_ERRNO__)
+ * FILTER_MATCH - filter found for event and @record matches
+ * FILTER_MISS - filter found for event and @record does not match
+ * FILTER_NOT_FOUND - no filter found for @record's event
+ * NO_FILTER - if no filters exist
+ * otherwise - error occurred during test
*/
-int pevent_filter_match(struct event_filter *filter,
- struct pevent_record *record)
+enum pevent_errno pevent_filter_match(struct event_filter *filter,
+ struct pevent_record *record)
{
struct pevent *pevent = filter->pevent;
struct filter_type *filter_type;
int event_id;
+ int ret;
+ enum pevent_errno err = 0;
+
+ filter_init_error_buf(filter);
if (!filter->filters)
- return FILTER_NONE;
+ return PEVENT_ERRNO__NO_FILTER;
event_id = pevent_data_type(pevent, record);
filter_type = find_filter_type(filter, event_id);
-
if (!filter_type)
- return FILTER_NOEXIST;
+ return PEVENT_ERRNO__FILTER_NOT_FOUND;
+
+ ret = test_filter(filter_type->event, filter_type->filter, record, &err);
+ if (err)
+ return err;
- return test_filter(filter_type->event, filter_type->filter, record) ?
- FILTER_MATCH : FILTER_MISS;
+ return ret ? PEVENT_ERRNO__FILTER_MATCH : PEVENT_ERRNO__FILTER_MISS;
}
static char *op_to_str(struct event_filter *filter, struct filter_arg *arg)
int left_val = -1;
int right_val = -1;
int val;
- int len;
switch (arg->op.type) {
case FILTER_OP_AND:
default:
break;
}
- str = malloc_or_die(6);
- if (val)
- strcpy(str, "TRUE");
- else
- strcpy(str, "FALSE");
+ asprintf(&str, val ? "TRUE" : "FALSE");
break;
}
}
break;
}
- len = strlen(left) + strlen(right) + strlen(op) + 10;
- str = malloc_or_die(len);
- snprintf(str, len, "(%s) %s (%s)",
- left, op, right);
+ asprintf(&str, "(%s) %s (%s)", left, op, right);
break;
case FILTER_OP_NOT:
right_val = 0;
if (right_val >= 0) {
/* just return the opposite */
- str = malloc_or_die(6);
- if (right_val)
- strcpy(str, "FALSE");
- else
- strcpy(str, "TRUE");
+ asprintf(&str, right_val ? "FALSE" : "TRUE");
break;
}
- len = strlen(right) + strlen(op) + 3;
- str = malloc_or_die(len);
- snprintf(str, len, "%s(%s)", op, right);
+ asprintf(&str, "%s(%s)", op, right);
break;
default:
static char *val_to_str(struct event_filter *filter, struct filter_arg *arg)
{
- char *str;
-
- str = malloc_or_die(30);
+ char *str = NULL;
- snprintf(str, 30, "%lld", arg->value.val);
+ asprintf(&str, "%lld", arg->value.val);
return str;
}
char *rstr;
char *op;
char *str = NULL;
- int len;
lstr = arg_to_str(filter, arg->exp.left);
rstr = arg_to_str(filter, arg->exp.right);
op = "^";
break;
default:
- die("oops in exp");
+ op = "[ERROR IN EXPRESSION TYPE]";
+ break;
}
- len = strlen(op) + strlen(lstr) + strlen(rstr) + 4;
- str = malloc_or_die(len);
- snprintf(str, len, "%s %s %s", lstr, op, rstr);
+ asprintf(&str, "%s %s %s", lstr, op, rstr);
out:
free(lstr);
free(rstr);
char *rstr;
char *str = NULL;
char *op = NULL;
- int len;
lstr = arg_to_str(filter, arg->num.left);
rstr = arg_to_str(filter, arg->num.right);
if (!op)
op = "<=";
- len = strlen(lstr) + strlen(op) + strlen(rstr) + 4;
- str = malloc_or_die(len);
- sprintf(str, "%s %s %s", lstr, op, rstr);
-
+ asprintf(&str, "%s %s %s", lstr, op, rstr);
break;
default:
{
char *str = NULL;
char *op = NULL;
- int len;
switch (arg->str.type) {
case FILTER_CMP_MATCH:
if (!op)
op = "!~";
- len = strlen(arg->str.field->name) + strlen(op) +
- strlen(arg->str.val) + 6;
- str = malloc_or_die(len);
- snprintf(str, len, "%s %s \"%s\"",
- arg->str.field->name,
- op, arg->str.val);
+ asprintf(&str, "%s %s \"%s\"",
+ arg->str.field->name, op, arg->str.val);
break;
default:
static char *arg_to_str(struct event_filter *filter, struct filter_arg *arg)
{
- char *str;
+ char *str = NULL;
switch (arg->type) {
case FILTER_ARG_BOOLEAN:
- str = malloc_or_die(6);
- if (arg->boolean.value)
- strcpy(str, "TRUE");
- else
- strcpy(str, "FALSE");
+ asprintf(&str, arg->boolean.value ? "TRUE" : "FALSE");
return str;
case FILTER_ARG_OP:
*
* Returns a string that displays the filter contents.
* This string must be freed with free(str).
- * NULL is returned if no filter is found.
+ * NULL is returned if no filter is found or allocation failed.
*/
char *
pevent_filter_make_string(struct event_filter *filter, int event_id)
#define __weak __attribute__((weak))
-void __vdie(const char *fmt, va_list ap)
-{
- int ret = errno;
-
- if (errno)
- perror("trace-cmd");
- else
- ret = -1;
-
- fprintf(stderr, " ");
- vfprintf(stderr, fmt, ap);
-
- fprintf(stderr, "\n");
- exit(ret);
-}
-
-void __die(const char *fmt, ...)
-{
- va_list ap;
-
- va_start(ap, fmt);
- __vdie(fmt, ap);
- va_end(ap);
-}
-
-void __weak die(const char *fmt, ...)
-{
- va_list ap;
-
- va_start(ap, fmt);
- __vdie(fmt, ap);
- va_end(ap);
-}
-
void __vwarning(const char *fmt, va_list ap)
{
if (errno)
__vpr_stat(fmt, ap);
va_end(ap);
}
-
-void __weak *malloc_or_die(unsigned int size)
-{
- void *data;
-
- data = malloc(size);
- if (!data)
- die("malloc");
- return data;
-}
--- /dev/null
+#include <stdio.h>
+#include <string.h>
+#include <inttypes.h>
+#include <endian.h>
+#include "event-parse.h"
+
+static unsigned long long
+process___le16_to_cpup(struct trace_seq *s,
+ unsigned long long *args)
+{
+ uint16_t *val = (uint16_t *) (unsigned long) args[0];
+ return val ? (long long) le16toh(*val) : 0;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_print_function(pevent,
+ process___le16_to_cpup,
+ PEVENT_FUNC_ARG_INT,
+ "__le16_to_cpup",
+ PEVENT_FUNC_ARG_PTR,
+ PEVENT_FUNC_ARG_VOID);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_print_function(pevent, process___le16_to_cpup,
+ "__le16_to_cpup");
+}
--- /dev/null
+/*
+ * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "event-parse.h"
+#include "event-utils.h"
+
+static struct func_stack {
+ int size;
+ char **stack;
+} *fstack;
+
+static int cpus = -1;
+
+#define STK_BLK 10
+
+static void add_child(struct func_stack *stack, const char *child, int pos)
+{
+ int i;
+
+ if (!child)
+ return;
+
+ if (pos < stack->size)
+ free(stack->stack[pos]);
+ else {
+ char **ptr;
+
+ ptr = realloc(stack->stack, sizeof(char *) *
+ (stack->size + STK_BLK));
+ if (!ptr) {
+ warning("could not allocate plugin memory\n");
+ return;
+ }
+
+ stack->stack = ptr;
+
+ for (i = stack->size; i < stack->size + STK_BLK; i++)
+ stack->stack[i] = NULL;
+ stack->size += STK_BLK;
+ }
+
+ stack->stack[pos] = strdup(child);
+}
+
+static int add_and_get_index(const char *parent, const char *child, int cpu)
+{
+ int i;
+
+ if (cpu < 0)
+ return 0;
+
+ if (cpu > cpus) {
+ struct func_stack *ptr;
+
+ ptr = realloc(fstack, sizeof(*fstack) * (cpu + 1));
+ if (!ptr) {
+ warning("could not allocate plugin memory\n");
+ return 0;
+ }
+
+ fstack = ptr;
+
+ /* Account for holes in the cpu count */
+ for (i = cpus + 1; i <= cpu; i++)
+ memset(&fstack[i], 0, sizeof(fstack[i]));
+ cpus = cpu;
+ }
+
+ for (i = 0; i < fstack[cpu].size && fstack[cpu].stack[i]; i++) {
+ if (strcmp(parent, fstack[cpu].stack[i]) == 0) {
+ add_child(&fstack[cpu], child, i+1);
+ return i;
+ }
+ }
+
+ /* Not found */
+ add_child(&fstack[cpu], parent, 0);
+ add_child(&fstack[cpu], child, 1);
+ return 0;
+}
+
+static int function_handler(struct trace_seq *s, struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ struct pevent *pevent = event->pevent;
+ unsigned long long function;
+ unsigned long long pfunction;
+ const char *func;
+ const char *parent;
+ int index;
+
+ if (pevent_get_field_val(s, event, "ip", record, &function, 1))
+ return trace_seq_putc(s, '!');
+
+ func = pevent_find_function(pevent, function);
+
+ if (pevent_get_field_val(s, event, "parent_ip", record, &pfunction, 1))
+ return trace_seq_putc(s, '!');
+
+ parent = pevent_find_function(pevent, pfunction);
+
+ index = add_and_get_index(parent, func, record->cpu);
+
+ trace_seq_printf(s, "%*s", index*3, "");
+
+ if (func)
+ trace_seq_printf(s, "%s", func);
+ else
+ trace_seq_printf(s, "0x%llx", function);
+
+ trace_seq_printf(s, " <-- ");
+ if (parent)
+ trace_seq_printf(s, "%s", parent);
+ else
+ trace_seq_printf(s, "0x%llx", pfunction);
+
+ return 0;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_event_handler(pevent, -1, "ftrace", "function",
+ function_handler, NULL);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ int i, x;
+
+ pevent_unregister_event_handler(pevent, -1, "ftrace", "function",
+ function_handler, NULL);
+
+ for (i = 0; i <= cpus; i++) {
+ for (x = 0; x < fstack[i].size && fstack[i].stack[x]; x++)
+ free(fstack[i].stack[x]);
+ free(fstack[i].stack);
+ }
+
+ free(fstack);
+ fstack = NULL;
+ cpus = -1;
+}
--- /dev/null
+/*
+ * Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ * Copyright (C) 2009 Johannes Berg <johannes@sipsolutions.net>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "event-parse.h"
+
+static int timer_expire_handler(struct trace_seq *s,
+ struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ trace_seq_printf(s, "hrtimer=");
+
+ if (pevent_print_num_field(s, "0x%llx", event, "timer",
+ record, 0) == -1)
+ pevent_print_num_field(s, "0x%llx", event, "hrtimer",
+ record, 1);
+
+ trace_seq_printf(s, " now=");
+
+ pevent_print_num_field(s, "%llu", event, "now", record, 1);
+
+ pevent_print_func_field(s, " function=%s", event, "function",
+ record, 0);
+ return 0;
+}
+
+static int timer_start_handler(struct trace_seq *s,
+ struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ trace_seq_printf(s, "hrtimer=");
+
+ if (pevent_print_num_field(s, "0x%llx", event, "timer",
+ record, 0) == -1)
+ pevent_print_num_field(s, "0x%llx", event, "hrtimer",
+ record, 1);
+
+ pevent_print_func_field(s, " function=%s", event, "function",
+ record, 0);
+
+ trace_seq_printf(s, " expires=");
+ pevent_print_num_field(s, "%llu", event, "expires", record, 1);
+
+ trace_seq_printf(s, " softexpires=");
+ pevent_print_num_field(s, "%llu", event, "softexpires", record, 1);
+ return 0;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_event_handler(pevent, -1,
+ "timer", "hrtimer_expire_entry",
+ timer_expire_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "timer", "hrtimer_start",
+ timer_start_handler, NULL);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_event_handler(pevent, -1,
+ "timer", "hrtimer_expire_entry",
+ timer_expire_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "timer", "hrtimer_start",
+ timer_start_handler, NULL);
+}
--- /dev/null
+/*
+ * Copyright (C) 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "event-parse.h"
+
+#define MINORBITS 20
+#define MINORMASK ((1U << MINORBITS) - 1)
+
+#define MAJOR(dev) ((unsigned int) ((dev) >> MINORBITS))
+#define MINOR(dev) ((unsigned int) ((dev) & MINORMASK))
+
+static unsigned long long
+process_jbd2_dev_to_name(struct trace_seq *s,
+ unsigned long long *args)
+{
+ unsigned int dev = args[0];
+
+ trace_seq_printf(s, "%d:%d", MAJOR(dev), MINOR(dev));
+ return 0;
+}
+
+static unsigned long long
+process_jiffies_to_msecs(struct trace_seq *s,
+ unsigned long long *args)
+{
+ unsigned long long jiffies = args[0];
+
+ trace_seq_printf(s, "%lld", jiffies);
+ return jiffies;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_print_function(pevent,
+ process_jbd2_dev_to_name,
+ PEVENT_FUNC_ARG_STRING,
+ "jbd2_dev_to_name",
+ PEVENT_FUNC_ARG_INT,
+ PEVENT_FUNC_ARG_VOID);
+
+ pevent_register_print_function(pevent,
+ process_jiffies_to_msecs,
+ PEVENT_FUNC_ARG_LONG,
+ "jiffies_to_msecs",
+ PEVENT_FUNC_ARG_LONG,
+ PEVENT_FUNC_ARG_VOID);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_print_function(pevent, process_jbd2_dev_to_name,
+ "jbd2_dev_to_name");
+
+ pevent_unregister_print_function(pevent, process_jiffies_to_msecs,
+ "jiffies_to_msecs");
+}
--- /dev/null
+/*
+ * Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "event-parse.h"
+
+static int call_site_handler(struct trace_seq *s, struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ struct format_field *field;
+ unsigned long long val, addr;
+ void *data = record->data;
+ const char *func;
+
+ field = pevent_find_field(event, "call_site");
+ if (!field)
+ return 1;
+
+ if (pevent_read_number_field(field, data, &val))
+ return 1;
+
+ func = pevent_find_function(event->pevent, val);
+ if (!func)
+ return 1;
+
+ addr = pevent_find_function_address(event->pevent, val);
+
+ trace_seq_printf(s, "(%s+0x%x) ", func, (int)(val - addr));
+ return 1;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_event_handler(pevent, -1, "kmem", "kfree",
+ call_site_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kmem", "kmalloc",
+ call_site_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kmem", "kmalloc_node",
+ call_site_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kmem", "kmem_cache_alloc",
+ call_site_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kmem",
+ "kmem_cache_alloc_node",
+ call_site_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kmem", "kmem_cache_free",
+ call_site_handler, NULL);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_event_handler(pevent, -1, "kmem", "kfree",
+ call_site_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kmem", "kmalloc",
+ call_site_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kmem", "kmalloc_node",
+ call_site_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kmem", "kmem_cache_alloc",
+ call_site_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kmem",
+ "kmem_cache_alloc_node",
+ call_site_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kmem", "kmem_cache_free",
+ call_site_handler, NULL);
+}
--- /dev/null
+/*
+ * Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdint.h>
+
+#include "event-parse.h"
+
+#ifdef HAVE_UDIS86
+
+#include <udis86.h>
+
+static ud_t ud;
+
+static void init_disassembler(void)
+{
+ ud_init(&ud);
+ ud_set_syntax(&ud, UD_SYN_ATT);
+}
+
+static const char *disassemble(unsigned char *insn, int len, uint64_t rip,
+ int cr0_pe, int eflags_vm,
+ int cs_d, int cs_l)
+{
+ int mode;
+
+ if (!cr0_pe)
+ mode = 16;
+ else if (eflags_vm)
+ mode = 16;
+ else if (cs_l)
+ mode = 64;
+ else if (cs_d)
+ mode = 32;
+ else
+ mode = 16;
+
+ ud_set_pc(&ud, rip);
+ ud_set_mode(&ud, mode);
+ ud_set_input_buffer(&ud, insn, len);
+ ud_disassemble(&ud);
+ return ud_insn_asm(&ud);
+}
+
+#else
+
+static void init_disassembler(void)
+{
+}
+
+static const char *disassemble(unsigned char *insn, int len, uint64_t rip,
+ int cr0_pe, int eflags_vm,
+ int cs_d, int cs_l)
+{
+ static char out[15*3+1];
+ int i;
+
+ for (i = 0; i < len; ++i)
+ sprintf(out + i * 3, "%02x ", insn[i]);
+ out[len*3-1] = '\0';
+ return out;
+}
+
+#endif
+
+
+#define VMX_EXIT_REASONS \
+ _ER(EXCEPTION_NMI, 0) \
+ _ER(EXTERNAL_INTERRUPT, 1) \
+ _ER(TRIPLE_FAULT, 2) \
+ _ER(PENDING_INTERRUPT, 7) \
+ _ER(NMI_WINDOW, 8) \
+ _ER(TASK_SWITCH, 9) \
+ _ER(CPUID, 10) \
+ _ER(HLT, 12) \
+ _ER(INVD, 13) \
+ _ER(INVLPG, 14) \
+ _ER(RDPMC, 15) \
+ _ER(RDTSC, 16) \
+ _ER(VMCALL, 18) \
+ _ER(VMCLEAR, 19) \
+ _ER(VMLAUNCH, 20) \
+ _ER(VMPTRLD, 21) \
+ _ER(VMPTRST, 22) \
+ _ER(VMREAD, 23) \
+ _ER(VMRESUME, 24) \
+ _ER(VMWRITE, 25) \
+ _ER(VMOFF, 26) \
+ _ER(VMON, 27) \
+ _ER(CR_ACCESS, 28) \
+ _ER(DR_ACCESS, 29) \
+ _ER(IO_INSTRUCTION, 30) \
+ _ER(MSR_READ, 31) \
+ _ER(MSR_WRITE, 32) \
+ _ER(MWAIT_INSTRUCTION, 36) \
+ _ER(MONITOR_INSTRUCTION, 39) \
+ _ER(PAUSE_INSTRUCTION, 40) \
+ _ER(MCE_DURING_VMENTRY, 41) \
+ _ER(TPR_BELOW_THRESHOLD, 43) \
+ _ER(APIC_ACCESS, 44) \
+ _ER(EOI_INDUCED, 45) \
+ _ER(EPT_VIOLATION, 48) \
+ _ER(EPT_MISCONFIG, 49) \
+ _ER(INVEPT, 50) \
+ _ER(PREEMPTION_TIMER, 52) \
+ _ER(WBINVD, 54) \
+ _ER(XSETBV, 55) \
+ _ER(APIC_WRITE, 56) \
+ _ER(INVPCID, 58)
+
+#define SVM_EXIT_REASONS \
+ _ER(EXIT_READ_CR0, 0x000) \
+ _ER(EXIT_READ_CR3, 0x003) \
+ _ER(EXIT_READ_CR4, 0x004) \
+ _ER(EXIT_READ_CR8, 0x008) \
+ _ER(EXIT_WRITE_CR0, 0x010) \
+ _ER(EXIT_WRITE_CR3, 0x013) \
+ _ER(EXIT_WRITE_CR4, 0x014) \
+ _ER(EXIT_WRITE_CR8, 0x018) \
+ _ER(EXIT_READ_DR0, 0x020) \
+ _ER(EXIT_READ_DR1, 0x021) \
+ _ER(EXIT_READ_DR2, 0x022) \
+ _ER(EXIT_READ_DR3, 0x023) \
+ _ER(EXIT_READ_DR4, 0x024) \
+ _ER(EXIT_READ_DR5, 0x025) \
+ _ER(EXIT_READ_DR6, 0x026) \
+ _ER(EXIT_READ_DR7, 0x027) \
+ _ER(EXIT_WRITE_DR0, 0x030) \
+ _ER(EXIT_WRITE_DR1, 0x031) \
+ _ER(EXIT_WRITE_DR2, 0x032) \
+ _ER(EXIT_WRITE_DR3, 0x033) \
+ _ER(EXIT_WRITE_DR4, 0x034) \
+ _ER(EXIT_WRITE_DR5, 0x035) \
+ _ER(EXIT_WRITE_DR6, 0x036) \
+ _ER(EXIT_WRITE_DR7, 0x037) \
+ _ER(EXIT_EXCP_BASE, 0x040) \
+ _ER(EXIT_INTR, 0x060) \
+ _ER(EXIT_NMI, 0x061) \
+ _ER(EXIT_SMI, 0x062) \
+ _ER(EXIT_INIT, 0x063) \
+ _ER(EXIT_VINTR, 0x064) \
+ _ER(EXIT_CR0_SEL_WRITE, 0x065) \
+ _ER(EXIT_IDTR_READ, 0x066) \
+ _ER(EXIT_GDTR_READ, 0x067) \
+ _ER(EXIT_LDTR_READ, 0x068) \
+ _ER(EXIT_TR_READ, 0x069) \
+ _ER(EXIT_IDTR_WRITE, 0x06a) \
+ _ER(EXIT_GDTR_WRITE, 0x06b) \
+ _ER(EXIT_LDTR_WRITE, 0x06c) \
+ _ER(EXIT_TR_WRITE, 0x06d) \
+ _ER(EXIT_RDTSC, 0x06e) \
+ _ER(EXIT_RDPMC, 0x06f) \
+ _ER(EXIT_PUSHF, 0x070) \
+ _ER(EXIT_POPF, 0x071) \
+ _ER(EXIT_CPUID, 0x072) \
+ _ER(EXIT_RSM, 0x073) \
+ _ER(EXIT_IRET, 0x074) \
+ _ER(EXIT_SWINT, 0x075) \
+ _ER(EXIT_INVD, 0x076) \
+ _ER(EXIT_PAUSE, 0x077) \
+ _ER(EXIT_HLT, 0x078) \
+ _ER(EXIT_INVLPG, 0x079) \
+ _ER(EXIT_INVLPGA, 0x07a) \
+ _ER(EXIT_IOIO, 0x07b) \
+ _ER(EXIT_MSR, 0x07c) \
+ _ER(EXIT_TASK_SWITCH, 0x07d) \
+ _ER(EXIT_FERR_FREEZE, 0x07e) \
+ _ER(EXIT_SHUTDOWN, 0x07f) \
+ _ER(EXIT_VMRUN, 0x080) \
+ _ER(EXIT_VMMCALL, 0x081) \
+ _ER(EXIT_VMLOAD, 0x082) \
+ _ER(EXIT_VMSAVE, 0x083) \
+ _ER(EXIT_STGI, 0x084) \
+ _ER(EXIT_CLGI, 0x085) \
+ _ER(EXIT_SKINIT, 0x086) \
+ _ER(EXIT_RDTSCP, 0x087) \
+ _ER(EXIT_ICEBP, 0x088) \
+ _ER(EXIT_WBINVD, 0x089) \
+ _ER(EXIT_MONITOR, 0x08a) \
+ _ER(EXIT_MWAIT, 0x08b) \
+ _ER(EXIT_MWAIT_COND, 0x08c) \
+ _ER(EXIT_NPF, 0x400) \
+ _ER(EXIT_ERR, -1)
+
+#define _ER(reason, val) { #reason, val },
+struct str_values {
+ const char *str;
+ int val;
+};
+
+static struct str_values vmx_exit_reasons[] = {
+ VMX_EXIT_REASONS
+ { NULL, -1}
+};
+
+static struct str_values svm_exit_reasons[] = {
+ SVM_EXIT_REASONS
+ { NULL, -1}
+};
+
+static struct isa_exit_reasons {
+ unsigned isa;
+ struct str_values *strings;
+} isa_exit_reasons[] = {
+ { .isa = 1, .strings = vmx_exit_reasons },
+ { .isa = 2, .strings = svm_exit_reasons },
+ { }
+};
+
+static const char *find_exit_reason(unsigned isa, int val)
+{
+ struct str_values *strings = NULL;
+ int i;
+
+ for (i = 0; isa_exit_reasons[i].strings; ++i)
+ if (isa_exit_reasons[i].isa == isa) {
+ strings = isa_exit_reasons[i].strings;
+ break;
+ }
+ if (!strings)
+ return "UNKNOWN-ISA";
+ for (i = 0; strings[i].val >= 0; i++)
+ if (strings[i].val == val)
+ break;
+ if (strings[i].str)
+ return strings[i].str;
+ return "UNKNOWN";
+}
+
+static int kvm_exit_handler(struct trace_seq *s, struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ unsigned long long isa;
+ unsigned long long val;
+ unsigned long long info1 = 0, info2 = 0;
+
+ if (pevent_get_field_val(s, event, "exit_reason", record, &val, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "isa", record, &isa, 0) < 0)
+ isa = 1;
+
+ trace_seq_printf(s, "reason %s", find_exit_reason(isa, val));
+
+ pevent_print_num_field(s, " rip 0x%lx", event, "guest_rip", record, 1);
+
+ if (pevent_get_field_val(s, event, "info1", record, &info1, 0) >= 0
+ && pevent_get_field_val(s, event, "info2", record, &info2, 0) >= 0)
+ trace_seq_printf(s, " info %llx %llx", info1, info2);
+
+ return 0;
+}
+
+#define KVM_EMUL_INSN_F_CR0_PE (1 << 0)
+#define KVM_EMUL_INSN_F_EFL_VM (1 << 1)
+#define KVM_EMUL_INSN_F_CS_D (1 << 2)
+#define KVM_EMUL_INSN_F_CS_L (1 << 3)
+
+static int kvm_emulate_insn_handler(struct trace_seq *s,
+ struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ unsigned long long rip, csbase, len, flags, failed;
+ int llen;
+ uint8_t *insn;
+ const char *disasm;
+
+ if (pevent_get_field_val(s, event, "rip", record, &rip, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "csbase", record, &csbase, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "len", record, &len, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "flags", record, &flags, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "failed", record, &failed, 1) < 0)
+ return -1;
+
+ insn = pevent_get_field_raw(s, event, "insn", record, &llen, 1);
+ if (!insn)
+ return -1;
+
+ disasm = disassemble(insn, len, rip,
+ flags & KVM_EMUL_INSN_F_CR0_PE,
+ flags & KVM_EMUL_INSN_F_EFL_VM,
+ flags & KVM_EMUL_INSN_F_CS_D,
+ flags & KVM_EMUL_INSN_F_CS_L);
+
+ trace_seq_printf(s, "%llx:%llx: %s%s", csbase, rip, disasm,
+ failed ? " FAIL" : "");
+ return 0;
+}
+
+union kvm_mmu_page_role {
+ unsigned word;
+ struct {
+ unsigned glevels:4;
+ unsigned level:4;
+ unsigned quadrant:2;
+ unsigned pad_for_nice_hex_output:6;
+ unsigned direct:1;
+ unsigned access:3;
+ unsigned invalid:1;
+ unsigned cr4_pge:1;
+ unsigned nxe:1;
+ };
+};
+
+static int kvm_mmu_print_role(struct trace_seq *s, struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ unsigned long long val;
+ static const char *access_str[] = {
+ "---", "--x", "w--", "w-x", "-u-", "-ux", "wu-", "wux"
+ };
+ union kvm_mmu_page_role role;
+
+ if (pevent_get_field_val(s, event, "role", record, &val, 1) < 0)
+ return -1;
+
+ role.word = (int)val;
+
+ /*
+ * We can only use the structure if file is of the same
+ * endianess.
+ */
+ if (pevent_is_file_bigendian(event->pevent) ==
+ pevent_is_host_bigendian(event->pevent)) {
+
+ trace_seq_printf(s, "%u/%u q%u%s %s%s %spge %snxe",
+ role.level,
+ role.glevels,
+ role.quadrant,
+ role.direct ? " direct" : "",
+ access_str[role.access],
+ role.invalid ? " invalid" : "",
+ role.cr4_pge ? "" : "!",
+ role.nxe ? "" : "!");
+ } else
+ trace_seq_printf(s, "WORD: %08x", role.word);
+
+ pevent_print_num_field(s, " root %u ", event,
+ "root_count", record, 1);
+
+ if (pevent_get_field_val(s, event, "unsync", record, &val, 1) < 0)
+ return -1;
+
+ trace_seq_printf(s, "%s%c", val ? "unsync" : "sync", 0);
+ return 0;
+}
+
+static int kvm_mmu_get_page_handler(struct trace_seq *s,
+ struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ unsigned long long val;
+
+ if (pevent_get_field_val(s, event, "created", record, &val, 1) < 0)
+ return -1;
+
+ trace_seq_printf(s, "%s ", val ? "new" : "existing");
+
+ if (pevent_get_field_val(s, event, "gfn", record, &val, 1) < 0)
+ return -1;
+
+ trace_seq_printf(s, "sp gfn %llx ", val);
+ return kvm_mmu_print_role(s, record, event, context);
+}
+
+#define PT_WRITABLE_SHIFT 1
+#define PT_WRITABLE_MASK (1ULL << PT_WRITABLE_SHIFT)
+
+static unsigned long long
+process_is_writable_pte(struct trace_seq *s, unsigned long long *args)
+{
+ unsigned long pte = args[0];
+ return pte & PT_WRITABLE_MASK;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ init_disassembler();
+
+ pevent_register_event_handler(pevent, -1, "kvm", "kvm_exit",
+ kvm_exit_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kvm", "kvm_emulate_insn",
+ kvm_emulate_insn_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kvmmmu", "kvm_mmu_get_page",
+ kvm_mmu_get_page_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kvmmmu", "kvm_mmu_sync_page",
+ kvm_mmu_print_role, NULL);
+
+ pevent_register_event_handler(pevent, -1,
+ "kvmmmu", "kvm_mmu_unsync_page",
+ kvm_mmu_print_role, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kvmmmu", "kvm_mmu_zap_page",
+ kvm_mmu_print_role, NULL);
+
+ pevent_register_event_handler(pevent, -1, "kvmmmu",
+ "kvm_mmu_prepare_zap_page", kvm_mmu_print_role,
+ NULL);
+
+ pevent_register_print_function(pevent,
+ process_is_writable_pte,
+ PEVENT_FUNC_ARG_INT,
+ "is_writable_pte",
+ PEVENT_FUNC_ARG_LONG,
+ PEVENT_FUNC_ARG_VOID);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_event_handler(pevent, -1, "kvm", "kvm_exit",
+ kvm_exit_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kvm", "kvm_emulate_insn",
+ kvm_emulate_insn_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kvmmmu", "kvm_mmu_get_page",
+ kvm_mmu_get_page_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kvmmmu", "kvm_mmu_sync_page",
+ kvm_mmu_print_role, NULL);
+
+ pevent_unregister_event_handler(pevent, -1,
+ "kvmmmu", "kvm_mmu_unsync_page",
+ kvm_mmu_print_role, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kvmmmu", "kvm_mmu_zap_page",
+ kvm_mmu_print_role, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "kvmmmu",
+ "kvm_mmu_prepare_zap_page", kvm_mmu_print_role,
+ NULL);
+
+ pevent_unregister_print_function(pevent, process_is_writable_pte,
+ "is_writable_pte");
+}
--- /dev/null
+/*
+ * Copyright (C) 2009 Johannes Berg <johannes@sipsolutions.net>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "event-parse.h"
+
+#define INDENT 65
+
+static void print_string(struct trace_seq *s, struct event_format *event,
+ const char *name, const void *data)
+{
+ struct format_field *f = pevent_find_field(event, name);
+ int offset;
+ int length;
+
+ if (!f) {
+ trace_seq_printf(s, "NOTFOUND:%s", name);
+ return;
+ }
+
+ offset = f->offset;
+ length = f->size;
+
+ if (!strncmp(f->type, "__data_loc", 10)) {
+ unsigned long long v;
+ if (pevent_read_number_field(f, data, &v)) {
+ trace_seq_printf(s, "invalid_data_loc");
+ return;
+ }
+ offset = v & 0xffff;
+ length = v >> 16;
+ }
+
+ trace_seq_printf(s, "%.*s", length, (char *)data + offset);
+}
+
+#define SF(fn) pevent_print_num_field(s, fn ":%d", event, fn, record, 0)
+#define SFX(fn) pevent_print_num_field(s, fn ":%#x", event, fn, record, 0)
+#define SP() trace_seq_putc(s, ' ')
+
+static int drv_bss_info_changed(struct trace_seq *s,
+ struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ void *data = record->data;
+
+ print_string(s, event, "wiphy_name", data);
+ trace_seq_printf(s, " vif:");
+ print_string(s, event, "vif_name", data);
+ pevent_print_num_field(s, "(%d)", event, "vif_type", record, 1);
+
+ trace_seq_printf(s, "\n%*s", INDENT, "");
+ SF("assoc"); SP();
+ SF("aid"); SP();
+ SF("cts"); SP();
+ SF("shortpre"); SP();
+ SF("shortslot"); SP();
+ SF("dtimper"); SP();
+ trace_seq_printf(s, "\n%*s", INDENT, "");
+ SF("bcnint"); SP();
+ SFX("assoc_cap"); SP();
+ SFX("basic_rates"); SP();
+ SF("enable_beacon");
+ trace_seq_printf(s, "\n%*s", INDENT, "");
+ SF("ht_operation_mode");
+
+ return 0;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_event_handler(pevent, -1, "mac80211",
+ "drv_bss_info_changed",
+ drv_bss_info_changed, NULL);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_event_handler(pevent, -1, "mac80211",
+ "drv_bss_info_changed",
+ drv_bss_info_changed, NULL);
+}
--- /dev/null
+/*
+ * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License (not later!)
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this program; if not, see <http://www.gnu.org/licenses>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "event-parse.h"
+
+static void write_state(struct trace_seq *s, int val)
+{
+ const char states[] = "SDTtZXxW";
+ int found = 0;
+ int i;
+
+ for (i = 0; i < (sizeof(states) - 1); i++) {
+ if (!(val & (1 << i)))
+ continue;
+
+ if (found)
+ trace_seq_putc(s, '|');
+
+ found = 1;
+ trace_seq_putc(s, states[i]);
+ }
+
+ if (!found)
+ trace_seq_putc(s, 'R');
+}
+
+static void write_and_save_comm(struct format_field *field,
+ struct pevent_record *record,
+ struct trace_seq *s, int pid)
+{
+ const char *comm;
+ int len;
+
+ comm = (char *)(record->data + field->offset);
+ len = s->len;
+ trace_seq_printf(s, "%.*s",
+ field->size, comm);
+
+ /* make sure the comm has a \0 at the end. */
+ trace_seq_terminate(s);
+ comm = &s->buffer[len];
+
+ /* Help out the comm to ids. This will handle dups */
+ pevent_register_comm(field->event->pevent, comm, pid);
+}
+
+static int sched_wakeup_handler(struct trace_seq *s,
+ struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ struct format_field *field;
+ unsigned long long val;
+
+ if (pevent_get_field_val(s, event, "pid", record, &val, 1))
+ return trace_seq_putc(s, '!');
+
+ field = pevent_find_any_field(event, "comm");
+ if (field) {
+ write_and_save_comm(field, record, s, val);
+ trace_seq_putc(s, ':');
+ }
+ trace_seq_printf(s, "%lld", val);
+
+ if (pevent_get_field_val(s, event, "prio", record, &val, 0) == 0)
+ trace_seq_printf(s, " [%lld]", val);
+
+ if (pevent_get_field_val(s, event, "success", record, &val, 1) == 0)
+ trace_seq_printf(s, " success=%lld", val);
+
+ if (pevent_get_field_val(s, event, "target_cpu", record, &val, 0) == 0)
+ trace_seq_printf(s, " CPU:%03llu", val);
+
+ return 0;
+}
+
+static int sched_switch_handler(struct trace_seq *s,
+ struct pevent_record *record,
+ struct event_format *event, void *context)
+{
+ struct format_field *field;
+ unsigned long long val;
+
+ if (pevent_get_field_val(s, event, "prev_pid", record, &val, 1))
+ return trace_seq_putc(s, '!');
+
+ field = pevent_find_any_field(event, "prev_comm");
+ if (field) {
+ write_and_save_comm(field, record, s, val);
+ trace_seq_putc(s, ':');
+ }
+ trace_seq_printf(s, "%lld ", val);
+
+ if (pevent_get_field_val(s, event, "prev_prio", record, &val, 0) == 0)
+ trace_seq_printf(s, "[%lld] ", val);
+
+ if (pevent_get_field_val(s, event, "prev_state", record, &val, 0) == 0)
+ write_state(s, val);
+
+ trace_seq_puts(s, " ==> ");
+
+ if (pevent_get_field_val(s, event, "next_pid", record, &val, 1))
+ return trace_seq_putc(s, '!');
+
+ field = pevent_find_any_field(event, "next_comm");
+ if (field) {
+ write_and_save_comm(field, record, s, val);
+ trace_seq_putc(s, ':');
+ }
+ trace_seq_printf(s, "%lld", val);
+
+ if (pevent_get_field_val(s, event, "next_prio", record, &val, 0) == 0)
+ trace_seq_printf(s, " [%lld]", val);
+
+ return 0;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_event_handler(pevent, -1, "sched", "sched_switch",
+ sched_switch_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "sched", "sched_wakeup",
+ sched_wakeup_handler, NULL);
+
+ pevent_register_event_handler(pevent, -1, "sched", "sched_wakeup_new",
+ sched_wakeup_handler, NULL);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_event_handler(pevent, -1, "sched", "sched_switch",
+ sched_switch_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "sched", "sched_wakeup",
+ sched_wakeup_handler, NULL);
+
+ pevent_unregister_event_handler(pevent, -1, "sched", "sched_wakeup_new",
+ sched_wakeup_handler, NULL);
+}
--- /dev/null
+#include <stdio.h>
+#include <string.h>
+#include <inttypes.h>
+#include "event-parse.h"
+
+typedef unsigned long sector_t;
+typedef uint64_t u64;
+typedef unsigned int u32;
+
+/*
+ * SCSI opcodes
+ */
+#define TEST_UNIT_READY 0x00
+#define REZERO_UNIT 0x01
+#define REQUEST_SENSE 0x03
+#define FORMAT_UNIT 0x04
+#define READ_BLOCK_LIMITS 0x05
+#define REASSIGN_BLOCKS 0x07
+#define INITIALIZE_ELEMENT_STATUS 0x07
+#define READ_6 0x08
+#define WRITE_6 0x0a
+#define SEEK_6 0x0b
+#define READ_REVERSE 0x0f
+#define WRITE_FILEMARKS 0x10
+#define SPACE 0x11
+#define INQUIRY 0x12
+#define RECOVER_BUFFERED_DATA 0x14
+#define MODE_SELECT 0x15
+#define RESERVE 0x16
+#define RELEASE 0x17
+#define COPY 0x18
+#define ERASE 0x19
+#define MODE_SENSE 0x1a
+#define START_STOP 0x1b
+#define RECEIVE_DIAGNOSTIC 0x1c
+#define SEND_DIAGNOSTIC 0x1d
+#define ALLOW_MEDIUM_REMOVAL 0x1e
+
+#define READ_FORMAT_CAPACITIES 0x23
+#define SET_WINDOW 0x24
+#define READ_CAPACITY 0x25
+#define READ_10 0x28
+#define WRITE_10 0x2a
+#define SEEK_10 0x2b
+#define POSITION_TO_ELEMENT 0x2b
+#define WRITE_VERIFY 0x2e
+#define VERIFY 0x2f
+#define SEARCH_HIGH 0x30
+#define SEARCH_EQUAL 0x31
+#define SEARCH_LOW 0x32
+#define SET_LIMITS 0x33
+#define PRE_FETCH 0x34
+#define READ_POSITION 0x34
+#define SYNCHRONIZE_CACHE 0x35
+#define LOCK_UNLOCK_CACHE 0x36
+#define READ_DEFECT_DATA 0x37
+#define MEDIUM_SCAN 0x38
+#define COMPARE 0x39
+#define COPY_VERIFY 0x3a
+#define WRITE_BUFFER 0x3b
+#define READ_BUFFER 0x3c
+#define UPDATE_BLOCK 0x3d
+#define READ_LONG 0x3e
+#define WRITE_LONG 0x3f
+#define CHANGE_DEFINITION 0x40
+#define WRITE_SAME 0x41
+#define UNMAP 0x42
+#define READ_TOC 0x43
+#define READ_HEADER 0x44
+#define GET_EVENT_STATUS_NOTIFICATION 0x4a
+#define LOG_SELECT 0x4c
+#define LOG_SENSE 0x4d
+#define XDWRITEREAD_10 0x53
+#define MODE_SELECT_10 0x55
+#define RESERVE_10 0x56
+#define RELEASE_10 0x57
+#define MODE_SENSE_10 0x5a
+#define PERSISTENT_RESERVE_IN 0x5e
+#define PERSISTENT_RESERVE_OUT 0x5f
+#define VARIABLE_LENGTH_CMD 0x7f
+#define REPORT_LUNS 0xa0
+#define SECURITY_PROTOCOL_IN 0xa2
+#define MAINTENANCE_IN 0xa3
+#define MAINTENANCE_OUT 0xa4
+#define MOVE_MEDIUM 0xa5
+#define EXCHANGE_MEDIUM 0xa6
+#define READ_12 0xa8
+#define WRITE_12 0xaa
+#define READ_MEDIA_SERIAL_NUMBER 0xab
+#define WRITE_VERIFY_12 0xae
+#define VERIFY_12 0xaf
+#define SEARCH_HIGH_12 0xb0
+#define SEARCH_EQUAL_12 0xb1
+#define SEARCH_LOW_12 0xb2
+#define SECURITY_PROTOCOL_OUT 0xb5
+#define READ_ELEMENT_STATUS 0xb8
+#define SEND_VOLUME_TAG 0xb6
+#define WRITE_LONG_2 0xea
+#define EXTENDED_COPY 0x83
+#define RECEIVE_COPY_RESULTS 0x84
+#define ACCESS_CONTROL_IN 0x86
+#define ACCESS_CONTROL_OUT 0x87
+#define READ_16 0x88
+#define WRITE_16 0x8a
+#define READ_ATTRIBUTE 0x8c
+#define WRITE_ATTRIBUTE 0x8d
+#define VERIFY_16 0x8f
+#define SYNCHRONIZE_CACHE_16 0x91
+#define WRITE_SAME_16 0x93
+#define SERVICE_ACTION_IN 0x9e
+/* values for service action in */
+#define SAI_READ_CAPACITY_16 0x10
+#define SAI_GET_LBA_STATUS 0x12
+/* values for VARIABLE_LENGTH_CMD service action codes
+ * see spc4r17 Section D.3.5, table D.7 and D.8 */
+#define VLC_SA_RECEIVE_CREDENTIAL 0x1800
+/* values for maintenance in */
+#define MI_REPORT_IDENTIFYING_INFORMATION 0x05
+#define MI_REPORT_TARGET_PGS 0x0a
+#define MI_REPORT_ALIASES 0x0b
+#define MI_REPORT_SUPPORTED_OPERATION_CODES 0x0c
+#define MI_REPORT_SUPPORTED_TASK_MANAGEMENT_FUNCTIONS 0x0d
+#define MI_REPORT_PRIORITY 0x0e
+#define MI_REPORT_TIMESTAMP 0x0f
+#define MI_MANAGEMENT_PROTOCOL_IN 0x10
+/* value for MI_REPORT_TARGET_PGS ext header */
+#define MI_EXT_HDR_PARAM_FMT 0x20
+/* values for maintenance out */
+#define MO_SET_IDENTIFYING_INFORMATION 0x06
+#define MO_SET_TARGET_PGS 0x0a
+#define MO_CHANGE_ALIASES 0x0b
+#define MO_SET_PRIORITY 0x0e
+#define MO_SET_TIMESTAMP 0x0f
+#define MO_MANAGEMENT_PROTOCOL_OUT 0x10
+/* values for variable length command */
+#define XDREAD_32 0x03
+#define XDWRITE_32 0x04
+#define XPWRITE_32 0x06
+#define XDWRITEREAD_32 0x07
+#define READ_32 0x09
+#define VERIFY_32 0x0a
+#define WRITE_32 0x0b
+#define WRITE_SAME_32 0x0d
+
+#define SERVICE_ACTION16(cdb) (cdb[1] & 0x1f)
+#define SERVICE_ACTION32(cdb) ((cdb[8] << 8) | cdb[9])
+
+static const char *
+scsi_trace_misc(struct trace_seq *, unsigned char *, int);
+
+static const char *
+scsi_trace_rw6(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len;
+ sector_t lba = 0, txlen = 0;
+
+ lba |= ((cdb[1] & 0x1F) << 16);
+ lba |= (cdb[2] << 8);
+ lba |= cdb[3];
+ txlen = cdb[4];
+
+ trace_seq_printf(p, "lba=%llu txlen=%llu",
+ (unsigned long long)lba, (unsigned long long)txlen);
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+static const char *
+scsi_trace_rw10(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len;
+ sector_t lba = 0, txlen = 0;
+
+ lba |= (cdb[2] << 24);
+ lba |= (cdb[3] << 16);
+ lba |= (cdb[4] << 8);
+ lba |= cdb[5];
+ txlen |= (cdb[7] << 8);
+ txlen |= cdb[8];
+
+ trace_seq_printf(p, "lba=%llu txlen=%llu protect=%u",
+ (unsigned long long)lba, (unsigned long long)txlen,
+ cdb[1] >> 5);
+
+ if (cdb[0] == WRITE_SAME)
+ trace_seq_printf(p, " unmap=%u", cdb[1] >> 3 & 1);
+
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+static const char *
+scsi_trace_rw12(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len;
+ sector_t lba = 0, txlen = 0;
+
+ lba |= (cdb[2] << 24);
+ lba |= (cdb[3] << 16);
+ lba |= (cdb[4] << 8);
+ lba |= cdb[5];
+ txlen |= (cdb[6] << 24);
+ txlen |= (cdb[7] << 16);
+ txlen |= (cdb[8] << 8);
+ txlen |= cdb[9];
+
+ trace_seq_printf(p, "lba=%llu txlen=%llu protect=%u",
+ (unsigned long long)lba, (unsigned long long)txlen,
+ cdb[1] >> 5);
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+static const char *
+scsi_trace_rw16(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len;
+ sector_t lba = 0, txlen = 0;
+
+ lba |= ((u64)cdb[2] << 56);
+ lba |= ((u64)cdb[3] << 48);
+ lba |= ((u64)cdb[4] << 40);
+ lba |= ((u64)cdb[5] << 32);
+ lba |= (cdb[6] << 24);
+ lba |= (cdb[7] << 16);
+ lba |= (cdb[8] << 8);
+ lba |= cdb[9];
+ txlen |= (cdb[10] << 24);
+ txlen |= (cdb[11] << 16);
+ txlen |= (cdb[12] << 8);
+ txlen |= cdb[13];
+
+ trace_seq_printf(p, "lba=%llu txlen=%llu protect=%u",
+ (unsigned long long)lba, (unsigned long long)txlen,
+ cdb[1] >> 5);
+
+ if (cdb[0] == WRITE_SAME_16)
+ trace_seq_printf(p, " unmap=%u", cdb[1] >> 3 & 1);
+
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+static const char *
+scsi_trace_rw32(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len, *cmd;
+ sector_t lba = 0, txlen = 0;
+ u32 ei_lbrt = 0;
+
+ switch (SERVICE_ACTION32(cdb)) {
+ case READ_32:
+ cmd = "READ";
+ break;
+ case VERIFY_32:
+ cmd = "VERIFY";
+ break;
+ case WRITE_32:
+ cmd = "WRITE";
+ break;
+ case WRITE_SAME_32:
+ cmd = "WRITE_SAME";
+ break;
+ default:
+ trace_seq_printf(p, "UNKNOWN");
+ goto out;
+ }
+
+ lba |= ((u64)cdb[12] << 56);
+ lba |= ((u64)cdb[13] << 48);
+ lba |= ((u64)cdb[14] << 40);
+ lba |= ((u64)cdb[15] << 32);
+ lba |= (cdb[16] << 24);
+ lba |= (cdb[17] << 16);
+ lba |= (cdb[18] << 8);
+ lba |= cdb[19];
+ ei_lbrt |= (cdb[20] << 24);
+ ei_lbrt |= (cdb[21] << 16);
+ ei_lbrt |= (cdb[22] << 8);
+ ei_lbrt |= cdb[23];
+ txlen |= (cdb[28] << 24);
+ txlen |= (cdb[29] << 16);
+ txlen |= (cdb[30] << 8);
+ txlen |= cdb[31];
+
+ trace_seq_printf(p, "%s_32 lba=%llu txlen=%llu protect=%u ei_lbrt=%u",
+ cmd, (unsigned long long)lba,
+ (unsigned long long)txlen, cdb[10] >> 5, ei_lbrt);
+
+ if (SERVICE_ACTION32(cdb) == WRITE_SAME_32)
+ trace_seq_printf(p, " unmap=%u", cdb[10] >> 3 & 1);
+
+out:
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+static const char *
+scsi_trace_unmap(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len;
+ unsigned int regions = cdb[7] << 8 | cdb[8];
+
+ trace_seq_printf(p, "regions=%u", (regions - 8) / 16);
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+static const char *
+scsi_trace_service_action_in(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len, *cmd;
+ sector_t lba = 0;
+ u32 alloc_len = 0;
+
+ switch (SERVICE_ACTION16(cdb)) {
+ case SAI_READ_CAPACITY_16:
+ cmd = "READ_CAPACITY_16";
+ break;
+ case SAI_GET_LBA_STATUS:
+ cmd = "GET_LBA_STATUS";
+ break;
+ default:
+ trace_seq_printf(p, "UNKNOWN");
+ goto out;
+ }
+
+ lba |= ((u64)cdb[2] << 56);
+ lba |= ((u64)cdb[3] << 48);
+ lba |= ((u64)cdb[4] << 40);
+ lba |= ((u64)cdb[5] << 32);
+ lba |= (cdb[6] << 24);
+ lba |= (cdb[7] << 16);
+ lba |= (cdb[8] << 8);
+ lba |= cdb[9];
+ alloc_len |= (cdb[10] << 24);
+ alloc_len |= (cdb[11] << 16);
+ alloc_len |= (cdb[12] << 8);
+ alloc_len |= cdb[13];
+
+ trace_seq_printf(p, "%s lba=%llu alloc_len=%u", cmd,
+ (unsigned long long)lba, alloc_len);
+
+out:
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+static const char *
+scsi_trace_varlen(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ switch (SERVICE_ACTION32(cdb)) {
+ case READ_32:
+ case VERIFY_32:
+ case WRITE_32:
+ case WRITE_SAME_32:
+ return scsi_trace_rw32(p, cdb, len);
+ default:
+ return scsi_trace_misc(p, cdb, len);
+ }
+}
+
+static const char *
+scsi_trace_misc(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = p->buffer + p->len;
+
+ trace_seq_printf(p, "-");
+ trace_seq_putc(p, 0);
+ return ret;
+}
+
+const char *
+scsi_trace_parse_cdb(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ switch (cdb[0]) {
+ case READ_6:
+ case WRITE_6:
+ return scsi_trace_rw6(p, cdb, len);
+ case READ_10:
+ case VERIFY:
+ case WRITE_10:
+ case WRITE_SAME:
+ return scsi_trace_rw10(p, cdb, len);
+ case READ_12:
+ case VERIFY_12:
+ case WRITE_12:
+ return scsi_trace_rw12(p, cdb, len);
+ case READ_16:
+ case VERIFY_16:
+ case WRITE_16:
+ case WRITE_SAME_16:
+ return scsi_trace_rw16(p, cdb, len);
+ case UNMAP:
+ return scsi_trace_unmap(p, cdb, len);
+ case SERVICE_ACTION_IN:
+ return scsi_trace_service_action_in(p, cdb, len);
+ case VARIABLE_LENGTH_CMD:
+ return scsi_trace_varlen(p, cdb, len);
+ default:
+ return scsi_trace_misc(p, cdb, len);
+ }
+}
+
+unsigned long long process_scsi_trace_parse_cdb(struct trace_seq *s,
+ unsigned long long *args)
+{
+ scsi_trace_parse_cdb(s, (unsigned char *) (unsigned long) args[1], args[2]);
+ return 0;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_print_function(pevent,
+ process_scsi_trace_parse_cdb,
+ PEVENT_FUNC_ARG_STRING,
+ "scsi_trace_parse_cdb",
+ PEVENT_FUNC_ARG_PTR,
+ PEVENT_FUNC_ARG_PTR,
+ PEVENT_FUNC_ARG_INT,
+ PEVENT_FUNC_ARG_VOID);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_print_function(pevent, process_scsi_trace_parse_cdb,
+ "scsi_trace_parse_cdb");
+}
--- /dev/null
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include "event-parse.h"
+
+#define __HYPERVISOR_set_trap_table 0
+#define __HYPERVISOR_mmu_update 1
+#define __HYPERVISOR_set_gdt 2
+#define __HYPERVISOR_stack_switch 3
+#define __HYPERVISOR_set_callbacks 4
+#define __HYPERVISOR_fpu_taskswitch 5
+#define __HYPERVISOR_sched_op_compat 6
+#define __HYPERVISOR_dom0_op 7
+#define __HYPERVISOR_set_debugreg 8
+#define __HYPERVISOR_get_debugreg 9
+#define __HYPERVISOR_update_descriptor 10
+#define __HYPERVISOR_memory_op 12
+#define __HYPERVISOR_multicall 13
+#define __HYPERVISOR_update_va_mapping 14
+#define __HYPERVISOR_set_timer_op 15
+#define __HYPERVISOR_event_channel_op_compat 16
+#define __HYPERVISOR_xen_version 17
+#define __HYPERVISOR_console_io 18
+#define __HYPERVISOR_physdev_op_compat 19
+#define __HYPERVISOR_grant_table_op 20
+#define __HYPERVISOR_vm_assist 21
+#define __HYPERVISOR_update_va_mapping_otherdomain 22
+#define __HYPERVISOR_iret 23 /* x86 only */
+#define __HYPERVISOR_vcpu_op 24
+#define __HYPERVISOR_set_segment_base 25 /* x86/64 only */
+#define __HYPERVISOR_mmuext_op 26
+#define __HYPERVISOR_acm_op 27
+#define __HYPERVISOR_nmi_op 28
+#define __HYPERVISOR_sched_op 29
+#define __HYPERVISOR_callback_op 30
+#define __HYPERVISOR_xenoprof_op 31
+#define __HYPERVISOR_event_channel_op 32
+#define __HYPERVISOR_physdev_op 33
+#define __HYPERVISOR_hvm_op 34
+#define __HYPERVISOR_tmem_op 38
+
+/* Architecture-specific hypercall definitions. */
+#define __HYPERVISOR_arch_0 48
+#define __HYPERVISOR_arch_1 49
+#define __HYPERVISOR_arch_2 50
+#define __HYPERVISOR_arch_3 51
+#define __HYPERVISOR_arch_4 52
+#define __HYPERVISOR_arch_5 53
+#define __HYPERVISOR_arch_6 54
+#define __HYPERVISOR_arch_7 55
+
+#define N(x) [__HYPERVISOR_##x] = "("#x")"
+static const char *xen_hypercall_names[] = {
+ N(set_trap_table),
+ N(mmu_update),
+ N(set_gdt),
+ N(stack_switch),
+ N(set_callbacks),
+ N(fpu_taskswitch),
+ N(sched_op_compat),
+ N(dom0_op),
+ N(set_debugreg),
+ N(get_debugreg),
+ N(update_descriptor),
+ N(memory_op),
+ N(multicall),
+ N(update_va_mapping),
+ N(set_timer_op),
+ N(event_channel_op_compat),
+ N(xen_version),
+ N(console_io),
+ N(physdev_op_compat),
+ N(grant_table_op),
+ N(vm_assist),
+ N(update_va_mapping_otherdomain),
+ N(iret),
+ N(vcpu_op),
+ N(set_segment_base),
+ N(mmuext_op),
+ N(acm_op),
+ N(nmi_op),
+ N(sched_op),
+ N(callback_op),
+ N(xenoprof_op),
+ N(event_channel_op),
+ N(physdev_op),
+ N(hvm_op),
+
+/* Architecture-specific hypercall definitions. */
+ N(arch_0),
+ N(arch_1),
+ N(arch_2),
+ N(arch_3),
+ N(arch_4),
+ N(arch_5),
+ N(arch_6),
+ N(arch_7),
+};
+#undef N
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+static const char *xen_hypercall_name(unsigned op)
+{
+ if (op < ARRAY_SIZE(xen_hypercall_names) &&
+ xen_hypercall_names[op] != NULL)
+ return xen_hypercall_names[op];
+
+ return "";
+}
+
+unsigned long long process_xen_hypercall_name(struct trace_seq *s,
+ unsigned long long *args)
+{
+ unsigned int op = args[0];
+
+ trace_seq_printf(s, "%s", xen_hypercall_name(op));
+ return 0;
+}
+
+int PEVENT_PLUGIN_LOADER(struct pevent *pevent)
+{
+ pevent_register_print_function(pevent,
+ process_xen_hypercall_name,
+ PEVENT_FUNC_ARG_STRING,
+ "xen_hypercall_name",
+ PEVENT_FUNC_ARG_INT,
+ PEVENT_FUNC_ARG_VOID);
+ return 0;
+}
+
+void PEVENT_PLUGIN_UNLOADER(struct pevent *pevent)
+{
+ pevent_unregister_print_function(pevent, process_xen_hypercall_name,
+ "xen_hypercall_name");
+}
#include <string.h>
#include <stdarg.h>
+#include <asm/bug.h>
#include "event-parse.h"
#include "event-utils.h"
#define TRACE_SEQ_POISON ((void *)0xdeadbeef)
#define TRACE_SEQ_CHECK(s) \
do { \
- if ((s)->buffer == TRACE_SEQ_POISON) \
- die("Usage of trace_seq after it was destroyed"); \
+ if (WARN_ONCE((s)->buffer == TRACE_SEQ_POISON, \
+ "Usage of trace_seq after it was destroyed")) \
+ (s)->state = TRACE_SEQ__BUFFER_POISONED; \
} while (0)
+#define TRACE_SEQ_CHECK_RET_N(s, n) \
+do { \
+ TRACE_SEQ_CHECK(s); \
+ if ((s)->state != TRACE_SEQ__GOOD) \
+ return n; \
+} while (0)
+
+#define TRACE_SEQ_CHECK_RET(s) TRACE_SEQ_CHECK_RET_N(s, )
+#define TRACE_SEQ_CHECK_RET0(s) TRACE_SEQ_CHECK_RET_N(s, 0)
+
/**
* trace_seq_init - initialize the trace_seq structure
* @s: a pointer to the trace_seq structure to initialize
s->len = 0;
s->readpos = 0;
s->buffer_size = TRACE_SEQ_BUF_SIZE;
- s->buffer = malloc_or_die(s->buffer_size);
+ s->buffer = malloc(s->buffer_size);
+ if (s->buffer != NULL)
+ s->state = TRACE_SEQ__GOOD;
+ else
+ s->state = TRACE_SEQ__MEM_ALLOC_FAILED;
}
/**
{
if (!s)
return;
- TRACE_SEQ_CHECK(s);
+ TRACE_SEQ_CHECK_RET(s);
free(s->buffer);
s->buffer = TRACE_SEQ_POISON;
}
static void expand_buffer(struct trace_seq *s)
{
+ char *buf;
+
+ buf = realloc(s->buffer, s->buffer_size + TRACE_SEQ_BUF_SIZE);
+ if (WARN_ONCE(!buf, "Can't allocate trace_seq buffer memory")) {
+ s->state = TRACE_SEQ__MEM_ALLOC_FAILED;
+ return;
+ }
+
+ s->buffer = buf;
s->buffer_size += TRACE_SEQ_BUF_SIZE;
- s->buffer = realloc(s->buffer, s->buffer_size);
- if (!s->buffer)
- die("Can't allocate trace_seq buffer memory");
}
/**
int len;
int ret;
- TRACE_SEQ_CHECK(s);
-
try_again:
+ TRACE_SEQ_CHECK_RET0(s);
+
len = (s->buffer_size - 1) - s->len;
va_start(ap, fmt);
int len;
int ret;
- TRACE_SEQ_CHECK(s);
-
try_again:
+ TRACE_SEQ_CHECK_RET0(s);
+
len = (s->buffer_size - 1) - s->len;
ret = vsnprintf(s->buffer + s->len, len, fmt, args);
{
int len;
- TRACE_SEQ_CHECK(s);
+ TRACE_SEQ_CHECK_RET0(s);
len = strlen(str);
while (len > ((s->buffer_size - 1) - s->len))
expand_buffer(s);
+ TRACE_SEQ_CHECK_RET0(s);
+
memcpy(s->buffer + s->len, str, len);
s->len += len;
int trace_seq_putc(struct trace_seq *s, unsigned char c)
{
- TRACE_SEQ_CHECK(s);
+ TRACE_SEQ_CHECK_RET0(s);
while (s->len >= (s->buffer_size - 1))
expand_buffer(s);
+ TRACE_SEQ_CHECK_RET0(s);
+
s->buffer[s->len++] = c;
return 1;
void trace_seq_terminate(struct trace_seq *s)
{
- TRACE_SEQ_CHECK(s);
+ TRACE_SEQ_CHECK_RET(s);
/* There's always one character left on the buffer */
s->buffer[s->len] = 0;
int trace_seq_do_printf(struct trace_seq *s)
{
TRACE_SEQ_CHECK(s);
- return printf("%.*s", s->len, s->buffer);
+
+ switch (s->state) {
+ case TRACE_SEQ__GOOD:
+ return printf("%.*s", s->len, s->buffer);
+ case TRACE_SEQ__BUFFER_POISONED:
+ puts("Usage of trace_seq after it was destroyed");
+ break;
+ case TRACE_SEQ__MEM_ALLOC_FAILED:
+ puts("Can't allocate trace_seq buffer memory");
+ break;
+ }
+ return -1;
}
DESCRIPTION
-----------
-This command runs runs perf-buildid-list --with-hits, and collects the files
-with the buildids found so that analysis of perf.data contents can be possible
-on another machine.
+This command runs perf-buildid-list --with-hits, and collects the files with the
+buildids found so that analysis of perf.data contents can be possible on another
+machine.
SEE ALSO
[verse]
'perf kvm' [--host] [--guest] [--guestmount=<path>
[--guestkallsyms=<path> --guestmodules=<path> | --guestvmlinux=<path>]]
- {top|record|report|diff|buildid-list}
+ {top|record|report|diff|buildid-list} [<options>]
'perf kvm' [--host] [--guest] [--guestkallsyms=<path> --guestmodules=<path>
- | --guestvmlinux=<path>] {top|record|report|diff|buildid-list|stat}
+ | --guestvmlinux=<path>] {top|record|report|diff|buildid-list|stat} [<options>]
'perf kvm stat [record|report|live] [<options>]
DESCRIPTION
of an arbitrary workload.
'perf kvm record <command>' to record the performance counter profile
- of an arbitrary workload and save it into a perf data file. If both
- --host and --guest are input, the perf data file name is perf.data.kvm.
- If there is no --host but --guest, the file name is perf.data.guest.
- If there is no --guest but --host, the file name is perf.data.host.
+ of an arbitrary workload and save it into a perf data file. We set the
+ default behavior of perf kvm as --guest, so if neither --host nor --guest
+ is input, the perf data file name is perf.data.guest. If --host is input,
+ the perf data file name is perf.data.kvm. If you want to record data into
+ perf.data.host, please input --host --no-guest. The behaviors are shown as
+ following:
+ Default('') -> perf.data.guest
+ --host -> perf.data.kvm
+ --guest -> perf.data.guest
+ --host --guest -> perf.data.kvm
+ --host --no-guest -> perf.data.host
'perf kvm report' to display the performance counter profile information
recorded via perf kvm record.
'perf kvm buildid-list' to display the buildids found in a perf data file,
so that other tools can be used to fetch packages with matching symbol tables
- for use by perf report.
+ for use by perf report. As buildid is read from /sys/kernel/notes in os, then
+ if you want to list the buildid for guest, please make sure your perf data file
+ was captured with --guestmount in perf kvm record.
'perf kvm stat <command>' to run a command and gather performance counter
statistics.
OPTIONS
-------
-i::
---input=::
+--input=<path>::
Input file name.
-o::
---output::
+--output=<path>::
Output file name.
---host=::
+--host::
Collect host side performance profile.
---guest=::
+--guest::
Collect guest side performance profile.
--guestmount=<path>::
Guest os root file system mount directory. Users mounts guest os
kernel module information. Users copy it out from guest os.
--guestvmlinux=<path>::
Guest os kernel vmlinux.
+-v::
+--verbose::
+ Be more verbose (show counter open errors, etc).
STAT REPORT OPTIONS
-------------------
-t::
--tid=::
Record events on existing thread ID (comma separated list).
+ This option also disables inheritance by default. Enable it by adding
+ --inherit.
-u::
--uid=::
--realtime=::
Collect data with this RT SCHED_FIFO priority.
--D::
---no-delay::
+--no-buffering::
Collect data without buffering.
-c::
--transaction::
Record transaction flags for transaction related events.
---force-per-cpu::
-Force the use of per-cpu mmaps. By default, when tasks are specified (i.e. -p,
--t or -u options) per-thread mmaps are created. This option overrides that and
-forces per-cpu mmaps. A side-effect of that is that inheritance is
-automatically enabled. Add the -i option also to disable inheritance.
+--per-thread::
+Use per-thread mmaps. By default per-cpu mmaps are created. This option
+overrides that and uses per-thread mmaps. A side-effect of that is that
+inheritance is automatically disabled. --per-thread is ignored with a warning
+if combined with -a or -C options.
+
+-D::
+--delay=::
+After starting the program, wait msecs before measuring. This is useful to
+filter out the startup phase of the program, which is often very different.
SEE ALSO
--------
Do not show entries which have an overhead under that percent.
(Default: 0).
+--header::
+ Show header information in the perf.data file. This includes
+ various information like hostname, OS and perf version, cpu/mem
+ info, perf command line, event list and so on. Currently only
+ --stdio output supports this feature.
+
+--header-only::
+ Show only perf.data header (forces --stdio).
+
SEE ALSO
--------
linkperf:perf-stat[1], linkperf:perf-annotate[1]
-f::
--fields::
Comma separated list of fields to print. Options are:
- comm, tid, pid, time, cpu, event, trace, ip, sym, dso, addr, symoff.
+ comm, tid, pid, time, cpu, event, trace, ip, sym, dso, addr, symoff, srcline.
Field list can be prepended with the type, trace, sw or hw,
to indicate to which event type the field list applies.
e.g., -f sw:comm,tid,time,ip,sym and -f trace:time,cpu,trace
--show-kernel-path::
Try to resolve the path of [kernel.kallsyms]
+--show-task-events
+ Display task related events (e.g. FORK, COMM, EXIT).
+
+--show-mmap-events
+ Display mmap related events (e.g. MMAP, MMAP2).
+
+--header
+ Show perf.data header.
+
+--header-only
+ Show only perf.data header.
+
SEE ALSO
--------
linkperf:perf-record[1], linkperf:perf-script-perl[1],
core number and the number of online logical processors on that physical processor.
-D msecs::
---initial-delay msecs::
+--delay msecs::
After starting the program, wait msecs before measuring. This is useful to
filter out the startup phase of the program, which is often very different.
SYNOPSIS
--------
[verse]
-'perf timechart' record <command>
-'perf timechart' [<options>]
+'perf timechart' [<timechart options>] {record} [<record options>]
DESCRIPTION
-----------
'perf timechart' to turn a trace into a Scalable Vector Graphics file,
that can be viewed with popular SVG viewers such as 'Inkscape'.
-OPTIONS
--------
+TIMECHART OPTIONS
+-----------------
-o::
--output=::
Select the output file (default: output.svg)
-P::
--power-only::
Only output the CPU power section of the diagram
+-T::
+--tasks-only::
+ Don't output processor state transitions
-p::
--process::
Select the processes to display, by name or PID
Written 10.2 seconds of trace to output.svg.
+Record system-wide timechart:
+
+ $ perf timechart record
+
+ then generate timechart and highlight 'gcc' tasks:
+
+ $ perf timechart --highlight gcc
+
+-n::
+--proc-num::
+ Print task info for at least given number of tasks.
+-t::
+--topology::
+ Sort CPUs according to topology.
+--highlight=<duration_nsecs|task_name>::
+ Highlight tasks (using different color) that run more than given
+ duration or tasks with given name. If number is given it's interpreted
+ as number of nanoseconds. If non-numeric string is given it's
+ interpreted as task name.
+
+RECORD OPTIONS
+--------------
+-P::
+--power-only::
+ Record only power-related events
+-T::
+--tasks-only::
+ Record only tasks-related events
+-g::
+--callchain::
+ Do call-graph (stack chain/backtrace) recording
+
SEE ALSO
--------
linkperf:perf-record[1]
--count-filter=<count>::
Only display functions with more events than this.
--g::
--group::
Put the counters into a counter group.
--asm-raw::
Show raw instruction encoding of assembly instructions.
--G::
+-g::
Enables call-graph (stack chain/backtrace) recording.
--call-graph::
Setup and enable call-graph (stack chain/backtrace) recording,
- implies -G.
+ implies -g.
--max-stack::
Set the stack depth limit when parsing the callchain, anything
tools/perf
tools/scripts
tools/lib/traceevent
-tools/lib/lk
+tools/lib/api
+tools/lib/symbol/kallsyms.c
+tools/lib/symbol/kallsyms.h
+tools/include/asm/bug.h
+tools/include/linux/compiler.h
include/linux/const.h
include/linux/perf_event.h
include/linux/rbtree.h
#
# Needed if no target specified:
+# (Except for tags and TAGS targets. The reason is that the
+# Makefile does not treat tags/TAGS as targets but as files
+# and thus won't rebuilt them once they are in place.)
#
-all:
+all tags TAGS:
$(print_msg)
$(make)
clean:
$(make)
+#
+# The build-test target is not really parallel, don't print the jobs info:
+#
+build-test:
+ @$(MAKE) -f tests/make --no-print-directory
+
#
# All other targets get passed through:
#
%:
$(print_msg)
$(make)
+
+.PHONY: tags TAGS
CC = $(CROSS_COMPILE)gcc
AR = $(CROSS_COMPILE)ar
+PKG_CONFIG = $(CROSS_COMPILE)pkg-config
RM = rm -f
LN = ln -f
BISON = bison
STRIP = strip
-LK_DIR = $(srctree)/tools/lib/lk/
+LIB_DIR = $(srctree)/tools/lib/api/
TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
# include config/Makefile by default and rule out
include config/Makefile
endif
-export prefix bindir sharedir sysconfdir
+export prefix bindir sharedir sysconfdir DESTDIR
# sparse is architecture-neutral, which means that we need to tell it
# explicitly what architecture to check for. Fix this up for yours..
ifneq ($(OUTPUT),)
TE_PATH=$(OUTPUT)
ifneq ($(subdir),)
- LK_PATH=$(OUTPUT)/../lib/lk/
+ LIB_PATH=$(OUTPUT)/../lib/api/
else
- LK_PATH=$(OUTPUT)
+ LIB_PATH=$(OUTPUT)
endif
else
TE_PATH=$(TRACE_EVENT_DIR)
- LK_PATH=$(LK_DIR)
+ LIB_PATH=$(LIB_DIR)
endif
LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
export LIBTRACEEVENT
-LIBLK = $(LK_PATH)liblk.a
-export LIBLK
+LIBAPIKFS = $(LIB_PATH)libapikfs.a
+export LIBAPIKFS
# python extension build directories
PYTHON_EXTBUILD := $(OUTPUT)python_ext_build/
python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf.so
PYTHON_EXT_SRCS := $(shell grep -v ^\# util/python-ext-sources)
-PYTHON_EXT_DEPS := util/python-ext-sources util/setup.py $(LIBTRACEEVENT) $(LIBLK)
+PYTHON_EXT_DEPS := util/python-ext-sources util/setup.py $(LIBTRACEEVENT) $(LIBAPIKFS)
$(OUTPUT)python/perf.so: $(PYTHON_EXT_SRCS) $(PYTHON_EXT_DEPS)
$(QUIET_GEN)CFLAGS='$(CFLAGS)' $(PYTHON_WORD) util/setup.py \
LIB_FILE=$(OUTPUT)libperf.a
+LIB_H += ../lib/symbol/kallsyms.h
LIB_H += ../../include/uapi/linux/perf_event.h
LIB_H += ../../include/linux/rbtree.h
LIB_H += ../../include/linux/list.h
LIB_H += ../../include/linux/stringify.h
LIB_H += util/include/linux/bitmap.h
LIB_H += util/include/linux/bitops.h
-LIB_H += util/include/linux/compiler.h
+LIB_H += ../include/linux/compiler.h
LIB_H += util/include/linux/const.h
LIB_H += util/include/linux/ctype.h
LIB_H += util/include/linux/kernel.h
LIB_H += util/include/linux/types.h
LIB_H += util/include/linux/linkage.h
LIB_H += util/include/asm/asm-offsets.h
-LIB_H += util/include/asm/bug.h
+LIB_H += ../include/asm/bug.h
LIB_H += util/include/asm/byteorder.h
LIB_H += util/include/asm/hweight.h
LIB_H += util/include/asm/swab.h
LIB_OBJS += $(OUTPUT)util/evsel.o
LIB_OBJS += $(OUTPUT)util/exec_cmd.o
LIB_OBJS += $(OUTPUT)util/help.o
+LIB_OBJS += $(OUTPUT)util/kallsyms.o
LIB_OBJS += $(OUTPUT)util/levenshtein.o
LIB_OBJS += $(OUTPUT)util/parse-options.o
LIB_OBJS += $(OUTPUT)util/parse-events.o
LIB_OBJS += $(OUTPUT)util/trace-event-read.o
LIB_OBJS += $(OUTPUT)util/trace-event-info.o
LIB_OBJS += $(OUTPUT)util/trace-event-scripting.o
+LIB_OBJS += $(OUTPUT)util/trace-event.o
LIB_OBJS += $(OUTPUT)util/svghelper.o
LIB_OBJS += $(OUTPUT)util/sort.o
LIB_OBJS += $(OUTPUT)util/hist.o
BUILTIN_OBJS += $(OUTPUT)tests/builtin-test.o
BUILTIN_OBJS += $(OUTPUT)builtin-mem.o
-PERFLIBS = $(LIB_FILE) $(LIBLK) $(LIBTRACEEVENT)
+PERFLIBS = $(LIB_FILE) $(LIBAPIKFS) $(LIBTRACEEVENT)
# We choose to avoid "if .. else if .. else .. endif endif"
# because maintaining the nesting to match is a pain. If
LIB_OBJS += $(OUTPUT)ui/browsers/hists.o
LIB_OBJS += $(OUTPUT)ui/browsers/map.o
LIB_OBJS += $(OUTPUT)ui/browsers/scripts.o
+ LIB_OBJS += $(OUTPUT)ui/browsers/header.o
LIB_OBJS += $(OUTPUT)ui/tui/setup.o
LIB_OBJS += $(OUTPUT)ui/tui/util.o
LIB_OBJS += $(OUTPUT)ui/tui/helpline.o
$(OUTPUT)ui/browsers/scripts.o: ui/browsers/scripts.c $(OUTPUT)PERF-CFLAGS
$(QUIET_CC)$(CC) -o $@ -c $(CFLAGS) -DENABLE_SLFUTURE_CONST $<
+$(OUTPUT)util/kallsyms.o: ../lib/symbol/kallsyms.c $(OUTPUT)PERF-CFLAGS
+ $(QUIET_CC)$(CC) -o $@ -c $(CFLAGS) $<
+
$(OUTPUT)util/rbtree.o: ../../lib/rbtree.c $(OUTPUT)PERF-CFLAGS
$(QUIET_CC)$(CC) -o $@ -c $(CFLAGS) -Wno-unused-parameter -DETC_PERFCONFIG='"$(ETC_PERFCONFIG_SQ)"' $<
# libtraceevent.a
TE_SOURCES = $(wildcard $(TRACE_EVENT_DIR)*.[ch])
-$(LIBTRACEEVENT): $(TE_SOURCES)
- $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) CFLAGS="-g -Wall $(EXTRA_CFLAGS)" libtraceevent.a
+LIBTRACEEVENT_FLAGS = $(QUIET_SUBDIR1) O=$(OUTPUT)
+LIBTRACEEVENT_FLAGS += CFLAGS="-g -Wall $(EXTRA_CFLAGS)"
+LIBTRACEEVENT_FLAGS += plugin_dir=$(plugindir_SQ)
+
+$(LIBTRACEEVENT): $(TE_SOURCES) $(OUTPUT)PERF-CFLAGS
+ $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) libtraceevent.a plugins
$(LIBTRACEEVENT)-clean:
$(call QUIET_CLEAN, libtraceevent)
@$(MAKE) -C $(TRACE_EVENT_DIR) O=$(OUTPUT) clean >/dev/null
-LIBLK_SOURCES = $(wildcard $(LK_PATH)*.[ch])
+install-traceevent-plugins: $(LIBTRACEEVENT)
+ $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) install_plugins
+
+LIBAPIKFS_SOURCES = $(wildcard $(LIB_PATH)fs/*.[ch])
# if subdir is set, we've been called from above so target has been built
# already
-$(LIBLK): $(LIBLK_SOURCES)
+$(LIBAPIKFS): $(LIBAPIKFS_SOURCES)
ifeq ($(subdir),)
- $(QUIET_SUBDIR0)$(LK_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) liblk.a
+ $(QUIET_SUBDIR0)$(LIB_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) libapikfs.a
endif
-$(LIBLK)-clean:
+$(LIBAPIKFS)-clean:
ifeq ($(subdir),)
- $(call QUIET_CLEAN, liblk)
- @$(MAKE) -C $(LK_DIR) O=$(OUTPUT) clean >/dev/null
+ $(call QUIET_CLEAN, libapikfs)
+ @$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
endif
help:
### Detect prefix changes
TRACK_CFLAGS = $(subst ','\'',$(CFLAGS)):\
- $(bindir_SQ):$(perfexecdir_SQ):$(template_dir_SQ):$(prefix_SQ)
+ $(bindir_SQ):$(perfexecdir_SQ):$(template_dir_SQ):$(prefix_SQ):$(plugindir_SQ)
$(OUTPUT)PERF-CFLAGS: .FORCE-PERF-CFLAGS
@FLAGS='$(TRACK_CFLAGS)'; \
$(INSTALL) scripts/python/*.py -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
$(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
endif
- $(call QUIET_INSTALL, bash_completion-script) \
+ $(call QUIET_INSTALL, perf_completion-script) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(sysconfdir_SQ)/bash_completion.d'; \
- $(INSTALL) bash_completion '$(DESTDIR_SQ)$(sysconfdir_SQ)/bash_completion.d/perf'
+ $(INSTALL) perf-completion.sh '$(DESTDIR_SQ)$(sysconfdir_SQ)/bash_completion.d/perf'
$(call QUIET_INSTALL, tests) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests'; \
$(INSTALL) tests/attr.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests'; \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'; \
$(INSTALL) tests/attr/* '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'
-install: install-bin try-install-man
+install: install-bin try-install-man install-traceevent-plugins
install-python_ext:
$(PYTHON_WORD) util/setup.py --quiet install --root='/$(DESTDIR_SQ)'
$(call QUIET_CLEAN, config)
@$(MAKE) -C config/feature-checks clean >/dev/null
-clean: $(LIBTRACEEVENT)-clean $(LIBLK)-clean config-clean
+clean: $(LIBTRACEEVENT)-clean $(LIBAPIKFS)-clean config-clean
$(call QUIET_CLEAN, core-objs) $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS) $(GTK_OBJS)
$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf
$(call QUIET_CLEAN, core-gen) $(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)PERF-CFLAGS $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex*
- $(call QUIET_CLEAN, Documentation)
- @$(MAKE) -C Documentation O=$(OUTPUT) clean >/dev/null
+ $(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) clean
$(python-clean)
#
}
if (lookup_path(buf))
goto out;
- free(buf);
- buf = NULL;
+ zfree(&buf);
}
if (!strcmp(arch, "arm"))
+++ /dev/null
-# perf completion
-
-# Taken from git.git's completion script.
-__my_reassemble_comp_words_by_ref()
-{
- local exclude i j first
- # Which word separators to exclude?
- exclude="${1//[^$COMP_WORDBREAKS]}"
- cword_=$COMP_CWORD
- if [ -z "$exclude" ]; then
- words_=("${COMP_WORDS[@]}")
- return
- fi
- # List of word completion separators has shrunk;
- # re-assemble words to complete.
- for ((i=0, j=0; i < ${#COMP_WORDS[@]}; i++, j++)); do
- # Append each nonempty word consisting of just
- # word separator characters to the current word.
- first=t
- while
- [ $i -gt 0 ] &&
- [ -n "${COMP_WORDS[$i]}" ] &&
- # word consists of excluded word separators
- [ "${COMP_WORDS[$i]//[^$exclude]}" = "${COMP_WORDS[$i]}" ]
- do
- # Attach to the previous token,
- # unless the previous token is the command name.
- if [ $j -ge 2 ] && [ -n "$first" ]; then
- ((j--))
- fi
- first=
- words_[$j]=${words_[j]}${COMP_WORDS[i]}
- if [ $i = $COMP_CWORD ]; then
- cword_=$j
- fi
- if (($i < ${#COMP_WORDS[@]} - 1)); then
- ((i++))
- else
- # Done.
- return
- fi
- done
- words_[$j]=${words_[j]}${COMP_WORDS[i]}
- if [ $i = $COMP_CWORD ]; then
- cword_=$j
- fi
- done
-}
-
-type _get_comp_words_by_ref &>/dev/null ||
-_get_comp_words_by_ref()
-{
- local exclude cur_ words_ cword_
- if [ "$1" = "-n" ]; then
- exclude=$2
- shift 2
- fi
- __my_reassemble_comp_words_by_ref "$exclude"
- cur_=${words_[cword_]}
- while [ $# -gt 0 ]; do
- case "$1" in
- cur)
- cur=$cur_
- ;;
- prev)
- prev=${words_[$cword_-1]}
- ;;
- words)
- words=("${words_[@]}")
- ;;
- cword)
- cword=$cword_
- ;;
- esac
- shift
- done
-}
-
-type __ltrim_colon_completions &>/dev/null ||
-__ltrim_colon_completions()
-{
- if [[ "$1" == *:* && "$COMP_WORDBREAKS" == *:* ]]; then
- # Remove colon-word prefix from COMPREPLY items
- local colon_word=${1%"${1##*:}"}
- local i=${#COMPREPLY[*]}
- while [[ $((--i)) -ge 0 ]]; do
- COMPREPLY[$i]=${COMPREPLY[$i]#"$colon_word"}
- done
- fi
-}
-
-type perf &>/dev/null &&
-_perf()
-{
- local cur words cword prev cmd
-
- COMPREPLY=()
- _get_comp_words_by_ref -n =: cur words cword prev
-
- cmd=${words[0]}
-
- # List perf subcommands or long options
- if [ $cword -eq 1 ]; then
- if [[ $cur == --* ]]; then
- COMPREPLY=( $( compgen -W '--help --version \
- --exec-path --html-path --paginate --no-pager \
- --perf-dir --work-tree --debugfs-dir' -- "$cur" ) )
- else
- cmds=$($cmd --list-cmds)
- COMPREPLY=( $( compgen -W '$cmds' -- "$cur" ) )
- fi
- # List possible events for -e option
- elif [[ $prev == "-e" && "${words[1]}" == @(record|stat|top) ]]; then
- evts=$($cmd list --raw-dump)
- COMPREPLY=( $( compgen -W '$evts' -- "$cur" ) )
- __ltrim_colon_completions $cur
- # List long option names
- elif [[ $cur == --* ]]; then
- subcmd=${words[1]}
- opts=$($cmd $subcmd --list-opts)
- COMPREPLY=( $( compgen -W '$opts' -- "$cur" ) )
- fi
-} &&
-
-complete -o bashdefault -o default -o nospace -F _perf perf 2>/dev/null \
- || complete -o default -o nospace -F _perf perf
if (he == NULL)
return -ENOMEM;
- ret = 0;
- if (he->ms.sym != NULL) {
- struct annotation *notes = symbol__annotation(he->ms.sym);
- if (notes->src == NULL && symbol__alloc_hist(he->ms.sym) < 0)
- return -ENOMEM;
-
- ret = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
- }
-
+ ret = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
evsel->hists.stats.total_period += sample->period;
hists__inc_nr_events(&evsel->hists, PERF_RECORD_SAMPLE);
return ret;
* symbol, free he->ms.sym->src to signal we already
* processed this symbol.
*/
- free(notes->src);
- notes->src = NULL;
+ zfree(¬es->src);
}
}
}
perf_session__fprintf_dsos(session, stdout);
total_nr_samples = 0;
- list_for_each_entry(pos, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, pos) {
struct hists *hists = &pos->hists;
u32 nr_samples = hists->stats.nr_events[PERF_RECORD_SAMPLE];
if (argc) {
/*
- * Special case: if there's an argument left then assume tha
+ * Special case: if there's an argument left then assume that
* it's a symbol filter:
*/
if (argc > 1)
{
struct perf_evsel *e;
- list_for_each_entry(e, &evlist->entries, node)
+ evlist__for_each(evlist, e) {
if (perf_evsel__match2(evsel, e))
return e;
+ }
return NULL;
}
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
struct hists *hists = &evsel->hists;
hists__collapse_resort(hists, NULL);
struct perf_evsel *evsel_base;
bool first = true;
- list_for_each_entry(evsel_base, &evlist_base->entries, node) {
+ evlist__for_each(evlist_base, evsel_base) {
struct data__file *d;
int i;
for (col = 0; col < PERF_HPP_DIFF__MAX_INDEX; col++) {
struct diff_hpp_fmt *fmt = &d->fmt[col];
- free(fmt->header);
+ zfree(&fmt->header);
}
}
return ret;
}
+static int __hpp__color_compare(struct perf_hpp_fmt *fmt,
+ struct perf_hpp *hpp, struct hist_entry *he,
+ int comparison_method)
+{
+ struct diff_hpp_fmt *dfmt =
+ container_of(fmt, struct diff_hpp_fmt, fmt);
+ struct hist_entry *pair = get_pair_fmt(he, dfmt);
+ double diff;
+ s64 wdiff;
+ char pfmt[20] = " ";
+
+ if (!pair)
+ goto dummy_print;
+
+ switch (comparison_method) {
+ case COMPUTE_DELTA:
+ if (pair->diff.computed)
+ diff = pair->diff.period_ratio_delta;
+ else
+ diff = compute_delta(he, pair);
+
+ if (fabs(diff) < 0.01)
+ goto dummy_print;
+ scnprintf(pfmt, 20, "%%%+d.2f%%%%", dfmt->header_width - 1);
+ return percent_color_snprintf(hpp->buf, hpp->size,
+ pfmt, diff);
+ case COMPUTE_RATIO:
+ if (he->dummy)
+ goto dummy_print;
+ if (pair->diff.computed)
+ diff = pair->diff.period_ratio;
+ else
+ diff = compute_ratio(he, pair);
+
+ scnprintf(pfmt, 20, "%%%d.6f", dfmt->header_width);
+ return value_color_snprintf(hpp->buf, hpp->size,
+ pfmt, diff);
+ case COMPUTE_WEIGHTED_DIFF:
+ if (he->dummy)
+ goto dummy_print;
+ if (pair->diff.computed)
+ wdiff = pair->diff.wdiff;
+ else
+ wdiff = compute_wdiff(he, pair);
+
+ scnprintf(pfmt, 20, "%%14ld", dfmt->header_width);
+ return color_snprintf(hpp->buf, hpp->size,
+ get_percent_color(wdiff),
+ pfmt, wdiff);
+ default:
+ BUG_ON(1);
+ }
+dummy_print:
+ return scnprintf(hpp->buf, hpp->size, "%*s",
+ dfmt->header_width, pfmt);
+}
+
+static int hpp__color_delta(struct perf_hpp_fmt *fmt,
+ struct perf_hpp *hpp, struct hist_entry *he)
+{
+ return __hpp__color_compare(fmt, hpp, he, COMPUTE_DELTA);
+}
+
+static int hpp__color_ratio(struct perf_hpp_fmt *fmt,
+ struct perf_hpp *hpp, struct hist_entry *he)
+{
+ return __hpp__color_compare(fmt, hpp, he, COMPUTE_RATIO);
+}
+
+static int hpp__color_wdiff(struct perf_hpp_fmt *fmt,
+ struct perf_hpp *hpp, struct hist_entry *he)
+{
+ return __hpp__color_compare(fmt, hpp, he, COMPUTE_WEIGHTED_DIFF);
+}
+
static void
hpp__entry_unpair(struct hist_entry *he, int idx, char *buf, size_t size)
{
fmt->entry = hpp__entry_global;
/* TODO more colors */
- if (idx == PERF_HPP_DIFF__BASELINE)
+ switch (idx) {
+ case PERF_HPP_DIFF__BASELINE:
fmt->color = hpp__color_baseline;
+ break;
+ case PERF_HPP_DIFF__DELTA:
+ fmt->color = hpp__color_delta;
+ break;
+ case PERF_HPP_DIFF__RATIO:
+ fmt->color = hpp__color_ratio;
+ break;
+ case PERF_HPP_DIFF__WEIGHTED_DIFF:
+ fmt->color = hpp__color_wdiff;
+ break;
+ default:
+ break;
+ }
init_header(d, dfmt);
perf_hpp__column_register(fmt);
data__files_cnt = argc;
use_default = false;
}
- } else if (symbol_conf.default_guest_vmlinux_name ||
- symbol_conf.default_guest_kallsyms) {
+ } else if (perf_guest) {
defaults[0] = "perf.data.host";
defaults[1] = "perf.data.guest";
}
if (session == NULL)
return -ENOMEM;
- list_for_each_entry(pos, &session->evlist->entries, node)
+ evlist__for_each(session->evlist, pos)
perf_evsel__fprintf(pos, details, stdout);
perf_session__delete(session);
#include <linux/list.h>
struct perf_inject {
- struct perf_tool tool;
- bool build_ids;
- bool sched_stat;
- const char *input_name;
- int pipe_output,
- output;
- u64 bytes_written;
- struct list_head samples;
+ struct perf_tool tool;
+ bool build_ids;
+ bool sched_stat;
+ const char *input_name;
+ struct perf_data_file output;
+ u64 bytes_written;
+ struct list_head samples;
};
struct event_entry {
union perf_event *event)
{
struct perf_inject *inject = container_of(tool, struct perf_inject, tool);
- uint32_t size;
- void *buf = event;
+ ssize_t size;
- size = event->header.size;
-
- while (size) {
- int ret = write(inject->output, buf, size);
- if (ret < 0)
- return -errno;
-
- size -= ret;
- buf += ret;
- inject->bytes_written += ret;
- }
+ size = perf_data_file__write(&inject->output, event,
+ event->header.size);
+ if (size < 0)
+ return -errno;
+ inject->bytes_written += size;
return 0;
}
if (ret)
return ret;
- if (!inject->pipe_output)
+ if (&inject->output.is_pipe)
return 0;
return perf_event__repipe_synth(tool, event);
.path = inject->input_name,
.mode = PERF_DATA_MODE_READ,
};
+ struct perf_data_file *file_out = &inject->output;
signal(SIGINT, sig_handler);
inject->tool.ordered_samples = true;
- list_for_each_entry(evsel, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, evsel) {
const char *name = perf_evsel__name(evsel);
if (!strcmp(name, "sched:sched_switch")) {
}
}
- if (!inject->pipe_output)
- lseek(inject->output, session->header.data_offset, SEEK_SET);
+ if (!file_out->is_pipe)
+ lseek(file_out->fd, session->header.data_offset, SEEK_SET);
ret = perf_session__process_events(session, &inject->tool);
- if (!inject->pipe_output) {
+ if (!file_out->is_pipe) {
session->header.data_size = inject->bytes_written;
- perf_session__write_header(session, session->evlist, inject->output, true);
+ perf_session__write_header(session, session->evlist, file_out->fd, true);
}
perf_session__delete(session);
},
.input_name = "-",
.samples = LIST_HEAD_INIT(inject.samples),
+ .output = {
+ .path = "-",
+ .mode = PERF_DATA_MODE_WRITE,
+ },
};
- const char *output_name = "-";
const struct option options[] = {
OPT_BOOLEAN('b', "build-ids", &inject.build_ids,
"Inject build-ids into the output stream"),
OPT_STRING('i', "input", &inject.input_name, "file",
"input file name"),
- OPT_STRING('o', "output", &output_name, "file",
+ OPT_STRING('o', "output", &inject.output.path, "file",
"output file name"),
OPT_BOOLEAN('s', "sched-stat", &inject.sched_stat,
"Merge sched-stat and sched-switch for getting events "
if (argc)
usage_with_options(inject_usage, options);
- if (!strcmp(output_name, "-")) {
- inject.pipe_output = 1;
- inject.output = STDOUT_FILENO;
- } else {
- inject.output = open(output_name, O_CREAT | O_WRONLY | O_TRUNC,
- S_IRUSR | S_IWUSR);
- if (inject.output < 0) {
- perror("failed to create output file");
- return -1;
- }
+ if (perf_data_file__open(&inject.output)) {
+ perror("failed to create output file");
+ return -1;
}
if (symbol__init() < 0)
#include "util/parse-options.h"
#include "util/trace-event.h"
#include "util/debug.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include "util/tool.h"
#include "util/stat.h"
#include "util/top.h"
struct perf_kvm_stat {
struct perf_tool tool;
- struct perf_record_opts opts;
+ struct record_opts opts;
struct perf_evlist *evlist;
struct perf_session *session;
if (kvm->timerfd >= 0)
close(kvm->timerfd);
- if (pollfds)
- free(pollfds);
-
+ free(pollfds);
return err;
}
* Note: exclude_{guest,host} do not apply here.
* This command processes KVM tracepoints from host only
*/
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
struct perf_event_attr *attr = &pos->attr;
/* make sure these *are* set */
.ordered_samples = true,
};
struct perf_data_file file = {
- .path = input_name,
+ .path = kvm->file_name,
.mode = PERF_DATA_MODE_READ,
};
if (kvm->session)
perf_session__delete(kvm->session);
kvm->session = NULL;
- if (kvm->evlist) {
- perf_evlist__delete_maps(kvm->evlist);
+ if (kvm->evlist)
perf_evlist__delete(kvm->evlist);
- }
return err;
}
"file", "file saving guest os /proc/kallsyms"),
OPT_STRING(0, "guestmodules", &symbol_conf.default_guest_modules,
"file", "file saving guest os /proc/modules"),
+ OPT_INCR('v', "verbose", &verbose,
+ "be more verbose (show counter open errors, etc)"),
OPT_END()
};
perf_guest = 1;
if (!file_name) {
- if (perf_host && !perf_guest)
- file_name = strdup("perf.data.host");
- else if (!perf_host && perf_guest)
- file_name = strdup("perf.data.guest");
- else
- file_name = strdup("perf.data.kvm");
+ file_name = get_filename_for_perf_kvm();
if (!file_name) {
pr_err("Failed to allocate memory for filename\n");
dump_raw_samples(struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel __maybe_unused,
struct machine *machine)
{
struct perf_mem *mem = container_of(tool, struct perf_mem, tool);
static int process_sample_event(struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel,
+ struct perf_evsel *evsel __maybe_unused,
struct machine *machine)
{
- return dump_raw_samples(tool, event, sample, evsel, machine);
+ return dump_raw_samples(tool, event, sample, machine);
}
static int report_raw_events(struct perf_mem *mem)
#include "util/strfilter.h"
#include "util/symbol.h"
#include "util/debug.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include "util/parse-options.h"
#include "util/probe-finder.h"
#include "util/probe-event.h"
struct perf_probe_event events[MAX_PROBES];
struct strlist *dellist;
struct line_range line_range;
- const char *target;
+ char *target;
int max_probe_points;
struct strfilter *filter;
} params;
* short module name.
*/
if (!params.target && ptr && *ptr == '/') {
- params.target = ptr;
+ params.target = strdup(ptr);
+ if (!params.target)
+ return -ENOMEM;
+
found = 1;
buf = ptr + (strlen(ptr) - 3);
char *buf;
found_target = set_target(argv[0]);
+ if (found_target < 0)
+ return found_target;
+
if (found_target && argc == 1)
return 0;
int unset __maybe_unused)
{
int ret = -ENOENT;
+ char *tmp;
if (str && !params.target) {
if (!strcmp(opt->long_name, "exec"))
else
return ret;
- params.target = str;
+ /* Expand given path to absolute path, except for modulename */
+ if (params.uprobes || strchr(str, '/')) {
+ tmp = realpath(str, NULL);
+ if (!tmp) {
+ pr_warning("Failed to get the absolute path of %s: %m\n", str);
+ return ret;
+ }
+ } else {
+ tmp = strdup(str);
+ if (!tmp)
+ return -ENOMEM;
+ }
+ params.target = tmp;
ret = 0;
}
params.show_lines = true;
ret = parse_line_range_desc(str, ¶ms.line_range);
- INIT_LIST_HEAD(¶ms.line_range.line_list);
return ret;
}
return 0;
}
-int cmd_probe(int argc, const char **argv, const char *prefix __maybe_unused)
+static void init_params(void)
+{
+ line_range__init(¶ms.line_range);
+}
+
+static void cleanup_params(void)
+{
+ int i;
+
+ for (i = 0; i < params.nevents; i++)
+ clear_perf_probe_event(params.events + i);
+ if (params.dellist)
+ strlist__delete(params.dellist);
+ line_range__clear(¶ms.line_range);
+ free(params.target);
+ if (params.filter)
+ strfilter__delete(params.filter);
+ memset(¶ms, 0, sizeof(params));
+}
+
+static int
+__cmd_probe(int argc, const char **argv, const char *prefix __maybe_unused)
{
const char * const probe_usage[] = {
"perf probe [<options>] 'PROBEDEF' ['PROBEDEF' ...]",
ret = show_available_funcs(params.target, params.filter,
params.uprobes);
strfilter__delete(params.filter);
+ params.filter = NULL;
if (ret < 0)
pr_err(" Error: Failed to show functions."
" (%d)\n", ret);
}
#ifdef HAVE_DWARF_SUPPORT
- if (params.show_lines && !params.uprobes) {
+ if (params.show_lines) {
if (params.mod_events) {
pr_err(" Error: Don't use --line with"
" --add/--del.\n");
params.filter,
params.show_ext_vars);
strfilter__delete(params.filter);
+ params.filter = NULL;
if (ret < 0)
pr_err(" Error: Failed to show vars. (%d)\n", ret);
return ret;
if (params.dellist) {
ret = del_perf_probe_events(params.dellist);
- strlist__delete(params.dellist);
if (ret < 0) {
pr_err(" Error: Failed to delete events. (%d)\n", ret);
return ret;
}
return 0;
}
+
+int cmd_probe(int argc, const char **argv, const char *prefix)
+{
+ int ret;
+
+ init_params();
+ ret = __cmd_probe(argc, argv, prefix);
+ cleanup_params();
+
+ return ret;
+}
}
#endif
-struct perf_record {
+struct record {
struct perf_tool tool;
- struct perf_record_opts opts;
+ struct record_opts opts;
u64 bytes_written;
struct perf_data_file file;
struct perf_evlist *evlist;
long samples;
};
-static int do_write_output(struct perf_record *rec, void *buf, size_t size)
+static int record__write(struct record *rec, void *bf, size_t size)
{
- struct perf_data_file *file = &rec->file;
-
- while (size) {
- ssize_t ret = write(file->fd, buf, size);
-
- if (ret < 0) {
- pr_err("failed to write perf data, error: %m\n");
- return -1;
- }
-
- size -= ret;
- buf += ret;
-
- rec->bytes_written += ret;
+ if (perf_data_file__write(rec->session->file, bf, size) < 0) {
+ pr_err("failed to write perf data, error: %m\n");
+ return -1;
}
+ rec->bytes_written += size;
return 0;
}
-static int write_output(struct perf_record *rec, void *buf, size_t size)
-{
- return do_write_output(rec, buf, size);
-}
-
static int process_synthesized_event(struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample __maybe_unused,
struct machine *machine __maybe_unused)
{
- struct perf_record *rec = container_of(tool, struct perf_record, tool);
- if (write_output(rec, event, event->header.size) < 0)
- return -1;
-
- return 0;
+ struct record *rec = container_of(tool, struct record, tool);
+ return record__write(rec, event, event->header.size);
}
-static int perf_record__mmap_read(struct perf_record *rec,
- struct perf_mmap *md)
+static int record__mmap_read(struct record *rec, struct perf_mmap *md)
{
unsigned int head = perf_mmap__read_head(md);
unsigned int old = md->prev;
size = md->mask + 1 - (old & md->mask);
old += size;
- if (write_output(rec, buf, size) < 0) {
+ if (record__write(rec, buf, size) < 0) {
rc = -1;
goto out;
}
size = head - old;
old += size;
- if (write_output(rec, buf, size) < 0) {
+ if (record__write(rec, buf, size) < 0) {
rc = -1;
goto out;
}
signr = sig;
}
-static void perf_record__sig_exit(int exit_status __maybe_unused, void *arg)
+static void record__sig_exit(int exit_status __maybe_unused, void *arg)
{
- struct perf_record *rec = arg;
+ struct record *rec = arg;
int status;
if (rec->evlist->workload.pid > 0) {
signal(signr, SIG_DFL);
}
-static int perf_record__open(struct perf_record *rec)
+static int record__open(struct record *rec)
{
char msg[512];
struct perf_evsel *pos;
struct perf_evlist *evlist = rec->evlist;
struct perf_session *session = rec->session;
- struct perf_record_opts *opts = &rec->opts;
+ struct record_opts *opts = &rec->opts;
int rc = 0;
perf_evlist__config(evlist, opts);
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
try_again:
if (perf_evsel__open(pos, evlist->cpus, evlist->threads) < 0) {
if (perf_evsel__fallback(pos, errno, msg, sizeof(msg))) {
"Consider increasing "
"/proc/sys/kernel/perf_event_mlock_kb,\n"
"or try again with a smaller value of -m/--mmap_pages.\n"
- "(current value: %d)\n", opts->mmap_pages);
+ "(current value: %u)\n", opts->mmap_pages);
rc = -errno;
} else {
pr_err("failed to mmap with %d (%s)\n", errno, strerror(errno));
return rc;
}
-static int process_buildids(struct perf_record *rec)
+static int process_buildids(struct record *rec)
{
struct perf_data_file *file = &rec->file;
struct perf_session *session = rec->session;
size, &build_id__mark_dso_hit_ops);
}
-static void perf_record__exit(int status, void *arg)
+static void record__exit(int status, void *arg)
{
- struct perf_record *rec = arg;
+ struct record *rec = arg;
struct perf_data_file *file = &rec->file;
if (status != 0)
.type = PERF_RECORD_FINISHED_ROUND,
};
-static int perf_record__mmap_read_all(struct perf_record *rec)
+static int record__mmap_read_all(struct record *rec)
{
int i;
int rc = 0;
for (i = 0; i < rec->evlist->nr_mmaps; i++) {
if (rec->evlist->mmap[i].base) {
- if (perf_record__mmap_read(rec, &rec->evlist->mmap[i]) != 0) {
+ if (record__mmap_read(rec, &rec->evlist->mmap[i]) != 0) {
rc = -1;
goto out;
}
}
if (perf_header__has_feat(&rec->session->header, HEADER_TRACING_DATA))
- rc = write_output(rec, &finished_round_event,
- sizeof(finished_round_event));
+ rc = record__write(rec, &finished_round_event, sizeof(finished_round_event));
out:
return rc;
}
-static void perf_record__init_features(struct perf_record *rec)
+static void record__init_features(struct record *rec)
{
- struct perf_evlist *evsel_list = rec->evlist;
struct perf_session *session = rec->session;
int feat;
if (rec->no_buildid)
perf_header__clear_feat(&session->header, HEADER_BUILD_ID);
- if (!have_tracepoints(&evsel_list->entries))
+ if (!have_tracepoints(&rec->evlist->entries))
perf_header__clear_feat(&session->header, HEADER_TRACING_DATA);
if (!rec->opts.branch_stack)
perf_header__clear_feat(&session->header, HEADER_BRANCH_STACK);
}
-static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
+static volatile int workload_exec_errno;
+
+/*
+ * perf_evlist__prepare_workload will send a SIGUSR1
+ * if the fork fails, since we asked by setting its
+ * want_signal to true.
+ */
+static void workload_exec_failed_signal(int signo, siginfo_t *info,
+ void *ucontext __maybe_unused)
+{
+ workload_exec_errno = info->si_value.sival_int;
+ done = 1;
+ signr = signo;
+ child_finished = 1;
+}
+
+static int __cmd_record(struct record *rec, int argc, const char **argv)
{
int err;
unsigned long waking = 0;
const bool forks = argc > 0;
struct machine *machine;
struct perf_tool *tool = &rec->tool;
- struct perf_record_opts *opts = &rec->opts;
- struct perf_evlist *evsel_list = rec->evlist;
+ struct record_opts *opts = &rec->opts;
struct perf_data_file *file = &rec->file;
struct perf_session *session;
bool disabled = false;
rec->progname = argv[0];
- on_exit(perf_record__sig_exit, rec);
+ on_exit(record__sig_exit, rec);
signal(SIGCHLD, sig_handler);
signal(SIGINT, sig_handler);
- signal(SIGUSR1, sig_handler);
signal(SIGTERM, sig_handler);
session = perf_session__new(file, false, NULL);
rec->session = session;
- perf_record__init_features(rec);
+ record__init_features(rec);
if (forks) {
- err = perf_evlist__prepare_workload(evsel_list, &opts->target,
+ err = perf_evlist__prepare_workload(rec->evlist, &opts->target,
argv, file->is_pipe,
- true);
+ workload_exec_failed_signal);
if (err < 0) {
pr_err("Couldn't run the workload!\n");
goto out_delete_session;
}
}
- if (perf_record__open(rec) != 0) {
+ if (record__open(rec) != 0) {
err = -1;
goto out_delete_session;
}
- if (!evsel_list->nr_groups)
+ if (!rec->evlist->nr_groups)
perf_header__clear_feat(&session->header, HEADER_GROUP_DESC);
/*
- * perf_session__delete(session) will be called at perf_record__exit()
+ * perf_session__delete(session) will be called at record__exit()
*/
- on_exit(perf_record__exit, rec);
+ on_exit(record__exit, rec);
if (file->is_pipe) {
err = perf_header__write_pipe(file->fd);
if (err < 0)
goto out_delete_session;
} else {
- err = perf_session__write_header(session, evsel_list,
+ err = perf_session__write_header(session, rec->evlist,
file->fd, false);
if (err < 0)
goto out_delete_session;
goto out_delete_session;
}
- if (have_tracepoints(&evsel_list->entries)) {
+ if (have_tracepoints(&rec->evlist->entries)) {
/*
* FIXME err <= 0 here actually means that
* there were no tracepoints so its not really
* return this more properly and also
* propagate errors that now are calling die()
*/
- err = perf_event__synthesize_tracing_data(tool, file->fd, evsel_list,
+ err = perf_event__synthesize_tracing_data(tool, file->fd, rec->evlist,
process_synthesized_event);
if (err <= 0) {
pr_err("Couldn't record tracing data.\n");
perf_event__synthesize_guest_os, tool);
}
- err = __machine__synthesize_threads(machine, tool, &opts->target, evsel_list->threads,
+ err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
process_synthesized_event, opts->sample_address);
if (err != 0)
goto out_delete_session;
* (apart from group members) have enable_on_exec=1 set,
* so don't spoil it by prematurely enabling them.
*/
- if (!target__none(&opts->target))
- perf_evlist__enable(evsel_list);
+ if (!target__none(&opts->target) && !opts->initial_delay)
+ perf_evlist__enable(rec->evlist);
/*
* Let the child rip
*/
if (forks)
- perf_evlist__start_workload(evsel_list);
+ perf_evlist__start_workload(rec->evlist);
+
+ if (opts->initial_delay) {
+ usleep(opts->initial_delay * 1000);
+ perf_evlist__enable(rec->evlist);
+ }
for (;;) {
int hits = rec->samples;
- if (perf_record__mmap_read_all(rec) < 0) {
+ if (record__mmap_read_all(rec) < 0) {
err = -1;
goto out_delete_session;
}
if (hits == rec->samples) {
if (done)
break;
- err = poll(evsel_list->pollfd, evsel_list->nr_fds, -1);
+ err = poll(rec->evlist->pollfd, rec->evlist->nr_fds, -1);
waking++;
}
* disable events in this case.
*/
if (done && !disabled && !target__none(&opts->target)) {
- perf_evlist__disable(evsel_list);
+ perf_evlist__disable(rec->evlist);
disabled = true;
}
}
+ if (forks && workload_exec_errno) {
+ char msg[512];
+ const char *emsg = strerror_r(workload_exec_errno, msg, sizeof(msg));
+ pr_err("Workload failed: %s\n", emsg);
+ err = -1;
+ goto out_delete_session;
+ }
+
if (quiet || signr == SIGUSR1)
return 0;
}
#endif /* HAVE_LIBUNWIND_SUPPORT */
-int record_parse_callchain(const char *arg, struct perf_record_opts *opts)
+int record_parse_callchain(const char *arg, struct record_opts *opts)
{
char *tok, *name, *saveptr = NULL;
char *buf;
return ret;
}
-static void callchain_debug(struct perf_record_opts *opts)
+static void callchain_debug(struct record_opts *opts)
{
pr_debug("callchain: type %d\n", opts->call_graph);
const char *arg,
int unset)
{
- struct perf_record_opts *opts = opt->value;
+ struct record_opts *opts = opt->value;
int ret;
/* --no-call-graph */
const char *arg __maybe_unused,
int unset __maybe_unused)
{
- struct perf_record_opts *opts = opt->value;
+ struct record_opts *opts = opt->value;
if (opts->call_graph == CALLCHAIN_NONE)
opts->call_graph = CALLCHAIN_FP;
};
/*
- * XXX Ideally would be local to cmd_record() and passed to a perf_record__new
- * because we need to have access to it in perf_record__exit, that is called
+ * XXX Ideally would be local to cmd_record() and passed to a record__new
+ * because we need to have access to it in record__exit, that is called
* after cmd_record() exits, but since record_options need to be accessible to
* builtin-script, leave it here.
*
*
* Just say no to tons of global variables, sigh.
*/
-static struct perf_record record = {
+static struct record record = {
.opts = {
.mmap_pages = UINT_MAX,
.user_freq = UINT_MAX,
.freq = 4000,
.target = {
.uses_mmap = true,
+ .default_per_cpu = true,
},
},
};
/*
* XXX Will stay a global variable till we fix builtin-script.c to stop messing
* with it and switch to use the library functions in perf_evlist that came
- * from builtin-record.c, i.e. use perf_record_opts,
+ * from builtin-record.c, i.e. use record_opts,
* perf_evlist__prepare_workload, etc instead of fork+exec'in 'perf record',
* using pipes, etc.
*/
"record events on existing thread id"),
OPT_INTEGER('r', "realtime", &record.realtime_prio,
"collect data with this RT SCHED_FIFO priority"),
- OPT_BOOLEAN('D', "no-delay", &record.opts.no_delay,
+ OPT_BOOLEAN(0, "no-buffering", &record.opts.no_buffering,
"collect data without buffering"),
OPT_BOOLEAN('R', "raw-samples", &record.opts.raw_samples,
"collect raw sample records from all opened counters"),
OPT_U64('c', "count", &record.opts.user_interval, "event period to sample"),
OPT_STRING('o', "output", &record.file.path, "file",
"output file name"),
- OPT_BOOLEAN('i', "no-inherit", &record.opts.no_inherit,
- "child tasks do not inherit counters"),
+ OPT_BOOLEAN_SET('i', "no-inherit", &record.opts.no_inherit,
+ &record.opts.no_inherit_set,
+ "child tasks do not inherit counters"),
OPT_UINTEGER('F', "freq", &record.opts.user_freq, "profile at this frequency"),
OPT_CALLBACK('m', "mmap-pages", &record.opts.mmap_pages, "pages",
"number of mmap data pages",
OPT_CALLBACK('G', "cgroup", &record.evlist, "name",
"monitor event in cgroup name only",
parse_cgroups),
+ OPT_UINTEGER('D', "delay", &record.opts.initial_delay,
+ "ms to wait before starting measurement after program start"),
OPT_STRING('u', "uid", &record.opts.target.uid_str, "user",
"user to profile"),
"sample by weight (on special events only)"),
OPT_BOOLEAN(0, "transaction", &record.opts.sample_transaction,
"sample transaction flags (special events only)"),
- OPT_BOOLEAN(0, "force-per-cpu", &record.opts.target.force_per_cpu,
- "force the use of per-cpu mmaps"),
+ OPT_BOOLEAN(0, "per-thread", &record.opts.target.per_thread,
+ "use per-thread mmaps"),
OPT_END()
};
int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
{
int err = -ENOMEM;
- struct perf_evlist *evsel_list;
- struct perf_record *rec = &record;
+ struct record *rec = &record;
char errbuf[BUFSIZ];
- evsel_list = perf_evlist__new();
- if (evsel_list == NULL)
+ rec->evlist = perf_evlist__new();
+ if (rec->evlist == NULL)
return -ENOMEM;
- rec->evlist = evsel_list;
-
argc = parse_options(argc, argv, record_options, record_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
if (!argc && target__none(&rec->opts.target))
if (rec->no_buildid_cache || rec->no_buildid)
disable_buildid_cache();
- if (evsel_list->nr_entries == 0 &&
- perf_evlist__add_default(evsel_list) < 0) {
+ if (rec->evlist->nr_entries == 0 &&
+ perf_evlist__add_default(rec->evlist) < 0) {
pr_err("Not enough memory for event selector list\n");
goto out_symbol_exit;
}
+ if (rec->opts.target.tid && !rec->opts.no_inherit_set)
+ rec->opts.no_inherit = true;
+
err = target__validate(&rec->opts.target);
if (err) {
target__strerror(&rec->opts.target, err, errbuf, BUFSIZ);
}
err = -ENOMEM;
- if (perf_evlist__create_maps(evsel_list, &rec->opts.target) < 0)
+ if (perf_evlist__create_maps(rec->evlist, &rec->opts.target) < 0)
usage_with_options(record_usage, record_options);
- if (perf_record_opts__config(&rec->opts)) {
+ if (record_opts__config(&rec->opts)) {
err = -EINVAL;
- goto out_free_fd;
+ goto out_symbol_exit;
}
err = __cmd_record(&record, argc, argv);
-
- perf_evlist__munmap(evsel_list);
- perf_evlist__close(evsel_list);
-out_free_fd:
- perf_evlist__delete_maps(evsel_list);
out_symbol_exit:
symbol__exit();
return err;
#include <dlfcn.h>
#include <linux/bitmap.h>
-struct perf_report {
+struct report {
struct perf_tool tool;
struct perf_session *session;
bool force, use_tui, use_gtk, use_stdio;
bool show_threads;
bool inverted_callchain;
bool mem_mode;
+ bool header;
+ bool header_only;
int max_stack;
struct perf_read_values show_threads_values;
const char *pretty_printing_style;
DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
};
-static int perf_report_config(const char *var, const char *value, void *cb)
+static int report__config(const char *var, const char *value, void *cb)
{
if (!strcmp(var, "report.group")) {
symbol_conf.event_group = perf_config_bool(var, value);
return 0;
}
if (!strcmp(var, "report.percent-limit")) {
- struct perf_report *rep = cb;
+ struct report *rep = cb;
rep->min_percent = strtof(value, NULL);
return 0;
}
return perf_default_config(var, value, cb);
}
-static int perf_report__add_mem_hist_entry(struct perf_tool *tool,
- struct addr_location *al,
- struct perf_sample *sample,
- struct perf_evsel *evsel,
- struct machine *machine,
- union perf_event *event)
+static int report__add_mem_hist_entry(struct perf_tool *tool, struct addr_location *al,
+ struct perf_sample *sample, struct perf_evsel *evsel,
+ union perf_event *event)
{
- struct perf_report *rep = container_of(tool, struct perf_report, tool);
+ struct report *rep = container_of(tool, struct report, tool);
struct symbol *parent = NULL;
u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
- int err = 0;
struct hist_entry *he;
struct mem_info *mi, *mx;
uint64_t cost;
+ int err = sample__resolve_callchain(sample, &parent, evsel, al, rep->max_stack);
- if ((sort__has_parent || symbol_conf.use_callchain) &&
- sample->callchain) {
- err = machine__resolve_callchain(machine, evsel, al->thread,
- sample, &parent, al,
- rep->max_stack);
- if (err)
- return err;
- }
+ if (err)
+ return err;
- mi = machine__resolve_mem(machine, al->thread, sample, cpumode);
+ mi = machine__resolve_mem(al->machine, al->thread, sample, cpumode);
if (!mi)
return -ENOMEM;
if (!he)
return -ENOMEM;
- /*
- * In the TUI browser, we are doing integrated annotation,
- * so we don't allocate the extra space needed because the stdio
- * code will not use it.
- */
- if (sort__has_sym && he->ms.sym && use_browser > 0) {
- struct annotation *notes = symbol__annotation(he->ms.sym);
-
- assert(evsel != NULL);
-
- if (notes->src == NULL && symbol__alloc_hist(he->ms.sym) < 0)
- goto out;
-
- err = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
- if (err)
- goto out;
- }
+ err = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
+ if (err)
+ goto out;
- if (sort__has_sym && he->mem_info->daddr.sym && use_browser > 0) {
- struct annotation *notes;
-
- mx = he->mem_info;
-
- notes = symbol__annotation(mx->daddr.sym);
- if (notes->src == NULL && symbol__alloc_hist(mx->daddr.sym) < 0)
- goto out;
-
- err = symbol__inc_addr_samples(mx->daddr.sym,
- mx->daddr.map,
- evsel->idx,
- mx->daddr.al_addr);
- if (err)
- goto out;
- }
+ mx = he->mem_info;
+ err = addr_map_symbol__inc_samples(&mx->daddr, evsel->idx);
+ if (err)
+ goto out;
evsel->hists.stats.total_period += cost;
hists__inc_nr_events(&evsel->hists, PERF_RECORD_SAMPLE);
- err = 0;
-
- if (symbol_conf.use_callchain) {
- err = callchain_append(he->callchain,
- &callchain_cursor,
- sample->period);
- }
+ err = hist_entry__append_callchain(he, sample);
out:
return err;
}
-static int perf_report__add_branch_hist_entry(struct perf_tool *tool,
- struct addr_location *al,
- struct perf_sample *sample,
- struct perf_evsel *evsel,
- struct machine *machine)
+static int report__add_branch_hist_entry(struct perf_tool *tool, struct addr_location *al,
+ struct perf_sample *sample, struct perf_evsel *evsel)
{
- struct perf_report *rep = container_of(tool, struct perf_report, tool);
+ struct report *rep = container_of(tool, struct report, tool);
struct symbol *parent = NULL;
- int err = 0;
unsigned i;
struct hist_entry *he;
struct branch_info *bi, *bx;
+ int err = sample__resolve_callchain(sample, &parent, evsel, al, rep->max_stack);
- if ((sort__has_parent || symbol_conf.use_callchain)
- && sample->callchain) {
- err = machine__resolve_callchain(machine, evsel, al->thread,
- sample, &parent, al,
- rep->max_stack);
- if (err)
- return err;
- }
+ if (err)
+ return err;
- bi = machine__resolve_bstack(machine, al->thread,
+ bi = machine__resolve_bstack(al->machine, al->thread,
sample->branch_stack);
if (!bi)
return -ENOMEM;
he = __hists__add_entry(&evsel->hists, al, parent, &bi[i], NULL,
1, 1, 0);
if (he) {
- struct annotation *notes;
bx = he->branch_info;
- if (bx->from.sym && use_browser == 1 && sort__has_sym) {
- notes = symbol__annotation(bx->from.sym);
- if (!notes->src
- && symbol__alloc_hist(bx->from.sym) < 0)
- goto out;
-
- err = symbol__inc_addr_samples(bx->from.sym,
- bx->from.map,
- evsel->idx,
- bx->from.al_addr);
- if (err)
- goto out;
- }
+ err = addr_map_symbol__inc_samples(&bx->from, evsel->idx);
+ if (err)
+ goto out;
+
+ err = addr_map_symbol__inc_samples(&bx->to, evsel->idx);
+ if (err)
+ goto out;
- if (bx->to.sym && use_browser == 1 && sort__has_sym) {
- notes = symbol__annotation(bx->to.sym);
- if (!notes->src
- && symbol__alloc_hist(bx->to.sym) < 0)
- goto out;
-
- err = symbol__inc_addr_samples(bx->to.sym,
- bx->to.map,
- evsel->idx,
- bx->to.al_addr);
- if (err)
- goto out;
- }
evsel->hists.stats.total_period += 1;
hists__inc_nr_events(&evsel->hists, PERF_RECORD_SAMPLE);
} else
return err;
}
-static int perf_evsel__add_hist_entry(struct perf_tool *tool,
- struct perf_evsel *evsel,
- struct addr_location *al,
- struct perf_sample *sample,
- struct machine *machine)
+static int report__add_hist_entry(struct perf_tool *tool, struct perf_evsel *evsel,
+ struct addr_location *al, struct perf_sample *sample)
{
- struct perf_report *rep = container_of(tool, struct perf_report, tool);
+ struct report *rep = container_of(tool, struct report, tool);
struct symbol *parent = NULL;
- int err = 0;
struct hist_entry *he;
+ int err = sample__resolve_callchain(sample, &parent, evsel, al, rep->max_stack);
- if ((sort__has_parent || symbol_conf.use_callchain) && sample->callchain) {
- err = machine__resolve_callchain(machine, evsel, al->thread,
- sample, &parent, al,
- rep->max_stack);
- if (err)
- return err;
- }
+ if (err)
+ return err;
he = __hists__add_entry(&evsel->hists, al, parent, NULL, NULL,
sample->period, sample->weight,
if (he == NULL)
return -ENOMEM;
- if (symbol_conf.use_callchain) {
- err = callchain_append(he->callchain,
- &callchain_cursor,
- sample->period);
- if (err)
- return err;
- }
- /*
- * Only in the TUI browser we are doing integrated annotation,
- * so we don't allocated the extra space needed because the stdio
- * code will not use it.
- */
- if (he->ms.sym != NULL && use_browser == 1 && sort__has_sym) {
- struct annotation *notes = symbol__annotation(he->ms.sym);
-
- assert(evsel != NULL);
-
- err = -ENOMEM;
- if (notes->src == NULL && symbol__alloc_hist(he->ms.sym) < 0)
- goto out;
-
- err = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
- }
+ err = hist_entry__append_callchain(he, sample);
+ if (err)
+ goto out;
+ err = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
evsel->hists.stats.total_period += sample->period;
hists__inc_nr_events(&evsel->hists, PERF_RECORD_SAMPLE);
out:
struct perf_evsel *evsel,
struct machine *machine)
{
- struct perf_report *rep = container_of(tool, struct perf_report, tool);
+ struct report *rep = container_of(tool, struct report, tool);
struct addr_location al;
int ret;
if (perf_event__preprocess_sample(event, machine, &al, sample) < 0) {
- fprintf(stderr, "problem processing %d event, skipping it.\n",
- event->header.type);
+ pr_debug("problem processing %d event, skipping it.\n",
+ event->header.type);
return -1;
}
return 0;
if (sort__mode == SORT_MODE__BRANCH) {
- ret = perf_report__add_branch_hist_entry(tool, &al, sample,
- evsel, machine);
+ ret = report__add_branch_hist_entry(tool, &al, sample, evsel);
if (ret < 0)
pr_debug("problem adding lbr entry, skipping event\n");
} else if (rep->mem_mode == 1) {
- ret = perf_report__add_mem_hist_entry(tool, &al, sample,
- evsel, machine, event);
+ ret = report__add_mem_hist_entry(tool, &al, sample, evsel, event);
if (ret < 0)
pr_debug("problem adding mem entry, skipping event\n");
} else {
if (al.map != NULL)
al.map->dso->hit = 1;
- ret = perf_evsel__add_hist_entry(tool, evsel, &al, sample,
- machine);
+ ret = report__add_hist_entry(tool, evsel, &al, sample);
if (ret < 0)
pr_debug("problem incrementing symbol period, skipping event\n");
}
struct perf_evsel *evsel,
struct machine *machine __maybe_unused)
{
- struct perf_report *rep = container_of(tool, struct perf_report, tool);
+ struct report *rep = container_of(tool, struct report, tool);
if (rep->show_threads) {
const char *name = evsel ? perf_evsel__name(evsel) : "unknown";
}
/* For pipe mode, sample_type is not currently set */
-static int perf_report__setup_sample_type(struct perf_report *rep)
+static int report__setup_sample_type(struct report *rep)
{
struct perf_session *session = rep->session;
u64 sample_type = perf_evlist__combined_sample_type(session->evlist);
session_done = 1;
}
-static size_t hists__fprintf_nr_sample_events(struct perf_report *rep,
- struct hists *hists,
+static size_t hists__fprintf_nr_sample_events(struct hists *hists, struct report *rep,
const char *evname, FILE *fp)
{
size_t ret;
}
static int perf_evlist__tty_browse_hists(struct perf_evlist *evlist,
- struct perf_report *rep,
+ struct report *rep,
const char *help)
{
struct perf_evsel *pos;
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
struct hists *hists = &pos->hists;
const char *evname = perf_evsel__name(pos);
!perf_evsel__is_group_leader(pos))
continue;
- hists__fprintf_nr_sample_events(rep, hists, evname, stdout);
+ hists__fprintf_nr_sample_events(hists, rep, evname, stdout);
hists__fprintf(hists, true, 0, 0, rep->min_percent, stdout);
fprintf(stdout, "\n\n");
}
return 0;
}
-static int __cmd_report(struct perf_report *rep)
+static void report__warn_kptr_restrict(const struct report *rep)
{
- int ret = -EINVAL;
- u64 nr_samples;
- struct perf_session *session = rep->session;
- struct perf_evsel *pos;
- struct map *kernel_map;
- struct kmap *kernel_kmap;
- const char *help = "For a higher level overview, try: perf report --sort comm,dso";
- struct ui_progress prog;
- struct perf_data_file *file = session->file;
-
- signal(SIGINT, sig_handler);
+ struct map *kernel_map = rep->session->machines.host.vmlinux_maps[MAP__FUNCTION];
+ struct kmap *kernel_kmap = map__kmap(kernel_map);
- if (rep->cpu_list) {
- ret = perf_session__cpu_bitmap(session, rep->cpu_list,
- rep->cpu_bitmap);
- if (ret)
- return ret;
- }
-
- if (use_browser <= 0)
- perf_session__fprintf_info(session, stdout, rep->show_full_info);
-
- if (rep->show_threads)
- perf_read_values_init(&rep->show_threads_values);
-
- ret = perf_report__setup_sample_type(rep);
- if (ret)
- return ret;
-
- ret = perf_session__process_events(session, &rep->tool);
- if (ret)
- return ret;
-
- kernel_map = session->machines.host.vmlinux_maps[MAP__FUNCTION];
- kernel_kmap = map__kmap(kernel_map);
if (kernel_map == NULL ||
(kernel_map->dso->hit &&
(kernel_kmap->ref_reloc_sym == NULL ||
"Samples in kernel modules can't be resolved as well.\n\n",
desc);
}
+}
- if (verbose > 3)
- perf_session__fprintf(session, stdout);
+static int report__gtk_browse_hists(struct report *rep, const char *help)
+{
+ int (*hist_browser)(struct perf_evlist *evlist, const char *help,
+ struct hist_browser_timer *timer, float min_pcnt);
- if (verbose > 2)
- perf_session__fprintf_dsos(session, stdout);
+ hist_browser = dlsym(perf_gtk_handle, "perf_evlist__gtk_browse_hists");
- if (dump_trace) {
- perf_session__fprintf_nr_events(session, stdout);
- return 0;
+ if (hist_browser == NULL) {
+ ui__error("GTK browser not found!\n");
+ return -1;
}
- nr_samples = 0;
- list_for_each_entry(pos, &session->evlist->entries, node)
+ return hist_browser(rep->session->evlist, help, NULL, rep->min_percent);
+}
+
+static int report__browse_hists(struct report *rep)
+{
+ int ret;
+ struct perf_session *session = rep->session;
+ struct perf_evlist *evlist = session->evlist;
+ const char *help = "For a higher level overview, try: perf report --sort comm,dso";
+
+ switch (use_browser) {
+ case 1:
+ ret = perf_evlist__tui_browse_hists(evlist, help, NULL,
+ rep->min_percent,
+ &session->header.env);
+ /*
+ * Usually "ret" is the last pressed key, and we only
+ * care if the key notifies us to switch data file.
+ */
+ if (ret != K_SWITCH_INPUT_DATA)
+ ret = 0;
+ break;
+ case 2:
+ ret = report__gtk_browse_hists(rep, help);
+ break;
+ default:
+ ret = perf_evlist__tty_browse_hists(evlist, rep, help);
+ break;
+ }
+
+ return ret;
+}
+
+static u64 report__collapse_hists(struct report *rep)
+{
+ struct ui_progress prog;
+ struct perf_evsel *pos;
+ u64 nr_samples = 0;
+ /*
+ * Count number of histogram entries to use when showing progress,
+ * reusing nr_samples variable.
+ */
+ evlist__for_each(rep->session->evlist, pos)
nr_samples += pos->hists.nr_entries;
ui_progress__init(&prog, nr_samples, "Merging related events...");
-
+ /*
+ * Count total number of samples, will be used to check if this
+ * session had any.
+ */
nr_samples = 0;
- list_for_each_entry(pos, &session->evlist->entries, node) {
+
+ evlist__for_each(rep->session->evlist, pos) {
struct hists *hists = &pos->hists;
if (pos->idx == 0)
hists__link(leader_hists, hists);
}
}
+
ui_progress__finish();
+ return nr_samples;
+}
+
+static int __cmd_report(struct report *rep)
+{
+ int ret;
+ u64 nr_samples;
+ struct perf_session *session = rep->session;
+ struct perf_evsel *pos;
+ struct perf_data_file *file = session->file;
+
+ signal(SIGINT, sig_handler);
+
+ if (rep->cpu_list) {
+ ret = perf_session__cpu_bitmap(session, rep->cpu_list,
+ rep->cpu_bitmap);
+ if (ret)
+ return ret;
+ }
+
+ if (rep->show_threads)
+ perf_read_values_init(&rep->show_threads_values);
+
+ ret = report__setup_sample_type(rep);
+ if (ret)
+ return ret;
+
+ ret = perf_session__process_events(session, &rep->tool);
+ if (ret)
+ return ret;
+
+ report__warn_kptr_restrict(rep);
+
+ if (use_browser == 0) {
+ if (verbose > 3)
+ perf_session__fprintf(session, stdout);
+
+ if (verbose > 2)
+ perf_session__fprintf_dsos(session, stdout);
+
+ if (dump_trace) {
+ perf_session__fprintf_nr_events(session, stdout);
+ return 0;
+ }
+ }
+
+ nr_samples = report__collapse_hists(rep);
+
if (session_done())
return 0;
return 0;
}
- list_for_each_entry(pos, &session->evlist->entries, node)
+ evlist__for_each(session->evlist, pos)
hists__output_resort(&pos->hists);
- if (use_browser > 0) {
- if (use_browser == 1) {
- ret = perf_evlist__tui_browse_hists(session->evlist,
- help, NULL,
- rep->min_percent,
- &session->header.env);
- /*
- * Usually "ret" is the last pressed key, and we only
- * care if the key notifies us to switch data file.
- */
- if (ret != K_SWITCH_INPUT_DATA)
- ret = 0;
-
- } else if (use_browser == 2) {
- int (*hist_browser)(struct perf_evlist *,
- const char *,
- struct hist_browser_timer *,
- float min_pcnt);
-
- hist_browser = dlsym(perf_gtk_handle,
- "perf_evlist__gtk_browse_hists");
- if (hist_browser == NULL) {
- ui__error("GTK browser not found!\n");
- return ret;
- }
- hist_browser(session->evlist, help, NULL,
- rep->min_percent);
- }
- } else
- perf_evlist__tty_browse_hists(session->evlist, rep, help);
-
- return ret;
+ return report__browse_hists(rep);
}
static int
parse_callchain_opt(const struct option *opt, const char *arg, int unset)
{
- struct perf_report *rep = (struct perf_report *)opt->value;
+ struct report *rep = (struct report *)opt->value;
char *tok, *tok2;
char *endptr;
return -1;
setup:
if (callchain_register_param(&callchain_param) < 0) {
- fprintf(stderr, "Can't register callchain params\n");
+ pr_err("Can't register callchain params\n");
return -1;
}
return 0;
parse_percent_limit(const struct option *opt, const char *str,
int unset __maybe_unused)
{
- struct perf_report *rep = opt->value;
+ struct report *rep = opt->value;
rep->min_percent = strtof(str, NULL);
return 0;
"perf report [<options>]",
NULL
};
- struct perf_report report = {
+ struct report report = {
.tool = {
.sample = process_sample_event,
.mmap = perf_event__process_mmap,
OPT_BOOLEAN(0, "gtk", &report.use_gtk, "Use the GTK2 interface"),
OPT_BOOLEAN(0, "stdio", &report.use_stdio,
"Use the stdio interface"),
+ OPT_BOOLEAN(0, "header", &report.header, "Show data header."),
+ OPT_BOOLEAN(0, "header-only", &report.header_only,
+ "Show only data header."),
OPT_STRING('s', "sort", &sort_order, "key[,key2...]",
"sort by key(s): pid, comm, dso, symbol, parent, cpu, srcline,"
" dso_to, dso_from, symbol_to, symbol_from, mispredict,"
.mode = PERF_DATA_MODE_READ,
};
- perf_config(perf_report_config, &report);
+ perf_config(report__config, &report);
argc = parse_options(argc, argv, options, report_usage, 0);
}
if (report.mem_mode) {
if (sort__mode == SORT_MODE__BRANCH) {
- fprintf(stderr, "branch and mem mode incompatible\n");
+ pr_err("branch and mem mode incompatible\n");
goto error;
}
sort__mode = SORT_MODE__MEMORY;
goto error;
}
+ /* Force tty output for header output. */
+ if (report.header || report.header_only)
+ use_browser = 0;
+
if (strcmp(input_name, "-") != 0)
setup_browser(true);
else {
perf_hpp__init();
}
+ if (report.header || report.header_only) {
+ perf_session__fprintf_info(session, stdout,
+ report.show_full_info);
+ if (report.header_only)
+ return 0;
+ } else if (use_browser == 0) {
+ fputs("# To display the perf.data header info, please use --header/--header-only options.\n#\n",
+ stdout);
+ }
+
/*
* Only in the TUI browser we are doing integrated annotation,
* so don't allocate extra space that won't be used in the stdio
char comm2[22];
int fd;
- free(parms);
+ zfree(&parms);
sprintf(comm2, ":%s", this_task->comm);
prctl(PR_SET_NAME, comm2);
PERF_OUTPUT_DSO = 1U << 9,
PERF_OUTPUT_ADDR = 1U << 10,
PERF_OUTPUT_SYMOFFSET = 1U << 11,
+ PERF_OUTPUT_SRCLINE = 1U << 12,
};
struct output_option {
{.str = "dso", .field = PERF_OUTPUT_DSO},
{.str = "addr", .field = PERF_OUTPUT_ADDR},
{.str = "symoff", .field = PERF_OUTPUT_SYMOFFSET},
+ {.str = "srcline", .field = PERF_OUTPUT_SRCLINE},
};
/* default set to maintain compatibility with current format */
"to DSO.\n");
return -EINVAL;
}
+ if (PRINT_FIELD(SRCLINE) && !PRINT_FIELD(IP)) {
+ pr_err("Display of source line number requested but sample IP is not\n"
+ "selected. Hence, no address to lookup the source line number.\n");
+ return -EINVAL;
+ }
if ((PRINT_FIELD(PID) || PRINT_FIELD(TID)) &&
perf_evsel__check_stype(evsel, PERF_SAMPLE_TID, "TID",
if (PRINT_FIELD(SYMOFFSET))
output[type].print_ip_opts |= PRINT_IP_OPT_SYMOFFSET;
+
+ if (PRINT_FIELD(SRCLINE))
+ output[type].print_ip_opts |= PRINT_IP_OPT_SRCLINE;
}
/*
set_print_ip_opts(&evsel->attr);
}
+ /*
+ * set default for tracepoints to print symbols only
+ * if callchains are present
+ */
+ if (symbol_conf.use_callchain &&
+ !output[PERF_TYPE_TRACEPOINT].user_set) {
+ struct perf_event_attr *attr;
+
+ j = PERF_TYPE_TRACEPOINT;
+ evsel = perf_session__find_first_evtype(session, j);
+ if (evsel == NULL)
+ goto out;
+
+ attr = &evsel->attr;
+
+ if (attr->sample_type & PERF_SAMPLE_CALLCHAIN) {
+ output[j].fields |= PERF_OUTPUT_IP;
+ output[j].fields |= PERF_OUTPUT_SYM;
+ output[j].fields |= PERF_OUTPUT_DSO;
+ set_print_ip_opts(attr);
+ }
+ }
+
+out:
return 0;
}
struct perf_evsel *evsel)
{
struct perf_event_attr *attr = &evsel->attr;
- const char *evname = NULL;
unsigned long secs;
unsigned long usecs;
unsigned long long nsecs;
usecs = nsecs / NSECS_PER_USEC;
printf("%5lu.%06lu: ", secs, usecs);
}
-
- if (PRINT_FIELD(EVNAME)) {
- evname = perf_evsel__name(evsel);
- printf("%s: ", evname ? evname : "[unknown]");
- }
}
static bool is_bts_event(struct perf_event_attr *attr)
static void print_sample_bts(union perf_event *event,
struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine,
- struct thread *thread)
+ struct thread *thread,
+ struct addr_location *al)
{
struct perf_event_attr *attr = &evsel->attr;
printf(" ");
else
printf("\n");
- perf_evsel__print_ip(evsel, event, sample, machine,
+ perf_evsel__print_ip(evsel, sample, al,
output[attr->type].print_ip_opts,
PERF_MAX_STACK_DEPTH);
}
if (PRINT_FIELD(ADDR) ||
((evsel->attr.sample_type & PERF_SAMPLE_ADDR) &&
!output[attr->type].user_set))
- print_sample_addr(event, sample, machine, thread, attr);
+ print_sample_addr(event, sample, al->machine, thread, attr);
printf("\n");
}
static void process_event(union perf_event *event, struct perf_sample *sample,
- struct perf_evsel *evsel, struct machine *machine,
- struct thread *thread,
- struct addr_location *al __maybe_unused)
+ struct perf_evsel *evsel, struct thread *thread,
+ struct addr_location *al)
{
struct perf_event_attr *attr = &evsel->attr;
print_sample_start(sample, thread, evsel);
+ if (PRINT_FIELD(EVNAME)) {
+ const char *evname = perf_evsel__name(evsel);
+ printf("%s: ", evname ? evname : "[unknown]");
+ }
+
if (is_bts_event(attr)) {
- print_sample_bts(event, sample, evsel, machine, thread);
+ print_sample_bts(event, sample, evsel, thread, al);
return;
}
event_format__print(evsel->tp_format, sample->cpu,
sample->raw_data, sample->raw_size);
if (PRINT_FIELD(ADDR))
- print_sample_addr(event, sample, machine, thread, attr);
+ print_sample_addr(event, sample, al->machine, thread, attr);
if (PRINT_FIELD(IP)) {
if (!symbol_conf.use_callchain)
else
printf("\n");
- perf_evsel__print_ip(evsel, event, sample, machine,
+ perf_evsel__print_ip(evsel, sample, al,
output[attr->type].print_ip_opts,
PERF_MAX_STACK_DEPTH);
}
if (cpu_list && !test_bit(sample->cpu, cpu_bitmap))
return 0;
- scripting_ops->process_event(event, sample, evsel, machine, thread, &al);
+ scripting_ops->process_event(event, sample, evsel, thread, &al);
evsel->hists.stats.total_period += sample->period;
return 0;
struct perf_script {
struct perf_tool tool;
struct perf_session *session;
+ bool show_task_events;
+ bool show_mmap_events;
};
static int process_attr(struct perf_tool *tool, union perf_event *event,
if (evsel->attr.type >= PERF_TYPE_MAX)
return 0;
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
if (pos->attr.type == evsel->attr.type && pos != evsel)
return 0;
}
return perf_evsel__check_attr(evsel, scr->session);
}
+static int process_comm_event(struct perf_tool *tool,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct thread *thread;
+ struct perf_script *script = container_of(tool, struct perf_script, tool);
+ struct perf_session *session = script->session;
+ struct perf_evsel *evsel = perf_evlist__first(session->evlist);
+ int ret = -1;
+
+ thread = machine__findnew_thread(machine, event->comm.pid, event->comm.tid);
+ if (thread == NULL) {
+ pr_debug("problem processing COMM event, skipping it.\n");
+ return -1;
+ }
+
+ if (perf_event__process_comm(tool, event, sample, machine) < 0)
+ goto out;
+
+ if (!evsel->attr.sample_id_all) {
+ sample->cpu = 0;
+ sample->time = 0;
+ sample->tid = event->comm.tid;
+ sample->pid = event->comm.pid;
+ }
+ print_sample_start(sample, thread, evsel);
+ perf_event__fprintf(event, stdout);
+ ret = 0;
+
+out:
+ return ret;
+}
+
+static int process_fork_event(struct perf_tool *tool,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct thread *thread;
+ struct perf_script *script = container_of(tool, struct perf_script, tool);
+ struct perf_session *session = script->session;
+ struct perf_evsel *evsel = perf_evlist__first(session->evlist);
+
+ if (perf_event__process_fork(tool, event, sample, machine) < 0)
+ return -1;
+
+ thread = machine__findnew_thread(machine, event->fork.pid, event->fork.tid);
+ if (thread == NULL) {
+ pr_debug("problem processing FORK event, skipping it.\n");
+ return -1;
+ }
+
+ if (!evsel->attr.sample_id_all) {
+ sample->cpu = 0;
+ sample->time = event->fork.time;
+ sample->tid = event->fork.tid;
+ sample->pid = event->fork.pid;
+ }
+ print_sample_start(sample, thread, evsel);
+ perf_event__fprintf(event, stdout);
+
+ return 0;
+}
+static int process_exit_event(struct perf_tool *tool,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct thread *thread;
+ struct perf_script *script = container_of(tool, struct perf_script, tool);
+ struct perf_session *session = script->session;
+ struct perf_evsel *evsel = perf_evlist__first(session->evlist);
+
+ thread = machine__findnew_thread(machine, event->fork.pid, event->fork.tid);
+ if (thread == NULL) {
+ pr_debug("problem processing EXIT event, skipping it.\n");
+ return -1;
+ }
+
+ if (!evsel->attr.sample_id_all) {
+ sample->cpu = 0;
+ sample->time = 0;
+ sample->tid = event->comm.tid;
+ sample->pid = event->comm.pid;
+ }
+ print_sample_start(sample, thread, evsel);
+ perf_event__fprintf(event, stdout);
+
+ if (perf_event__process_exit(tool, event, sample, machine) < 0)
+ return -1;
+
+ return 0;
+}
+
+static int process_mmap_event(struct perf_tool *tool,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct thread *thread;
+ struct perf_script *script = container_of(tool, struct perf_script, tool);
+ struct perf_session *session = script->session;
+ struct perf_evsel *evsel = perf_evlist__first(session->evlist);
+
+ if (perf_event__process_mmap(tool, event, sample, machine) < 0)
+ return -1;
+
+ thread = machine__findnew_thread(machine, event->mmap.pid, event->mmap.tid);
+ if (thread == NULL) {
+ pr_debug("problem processing MMAP event, skipping it.\n");
+ return -1;
+ }
+
+ if (!evsel->attr.sample_id_all) {
+ sample->cpu = 0;
+ sample->time = 0;
+ sample->tid = event->mmap.tid;
+ sample->pid = event->mmap.pid;
+ }
+ print_sample_start(sample, thread, evsel);
+ perf_event__fprintf(event, stdout);
+
+ return 0;
+}
+
+static int process_mmap2_event(struct perf_tool *tool,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct thread *thread;
+ struct perf_script *script = container_of(tool, struct perf_script, tool);
+ struct perf_session *session = script->session;
+ struct perf_evsel *evsel = perf_evlist__first(session->evlist);
+
+ if (perf_event__process_mmap2(tool, event, sample, machine) < 0)
+ return -1;
+
+ thread = machine__findnew_thread(machine, event->mmap2.pid, event->mmap2.tid);
+ if (thread == NULL) {
+ pr_debug("problem processing MMAP2 event, skipping it.\n");
+ return -1;
+ }
+
+ if (!evsel->attr.sample_id_all) {
+ sample->cpu = 0;
+ sample->time = 0;
+ sample->tid = event->mmap2.tid;
+ sample->pid = event->mmap2.pid;
+ }
+ print_sample_start(sample, thread, evsel);
+ perf_event__fprintf(event, stdout);
+
+ return 0;
+}
+
static void sig_handler(int sig __maybe_unused)
{
session_done = 1;
signal(SIGINT, sig_handler);
+ /* override event processing functions */
+ if (script->show_task_events) {
+ script->tool.comm = process_comm_event;
+ script->tool.fork = process_fork_event;
+ script->tool.exit = process_exit_event;
+ }
+ if (script->show_mmap_events) {
+ script->tool.mmap = process_mmap_event;
+ script->tool.mmap2 = process_mmap2_event;
+ }
+
ret = perf_session__process_events(script->session, &script->tool);
if (debug_mode)
static void script_desc__delete(struct script_desc *s)
{
- free(s->name);
- free(s->half_liner);
- free(s->args);
+ zfree(&s->name);
+ zfree(&s->half_liner);
+ zfree(&s->args);
free(s);
}
snprintf(evname, len + 1, "%s", p);
match = 0;
- list_for_each_entry(pos,
- &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, pos) {
if (!strcmp(perf_evsel__name(pos), evname)) {
match = 1;
break;
int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
{
bool show_full_info = false;
+ bool header = false;
+ bool header_only = false;
char *rec_script_path = NULL;
char *rep_script_path = NULL;
struct perf_session *session;
OPT_STRING('i', "input", &input_name, "file", "input file name"),
OPT_BOOLEAN('d', "debug-mode", &debug_mode,
"do various checks like samples ordering and lost events"),
+ OPT_BOOLEAN(0, "header", &header, "Show data header."),
+ OPT_BOOLEAN(0, "header-only", &header_only, "Show only data header."),
OPT_STRING('k', "vmlinux", &symbol_conf.vmlinux_name,
"file", "vmlinux pathname"),
OPT_STRING(0, "kallsyms", &symbol_conf.kallsyms_name,
"display extended information from perf.data file"),
OPT_BOOLEAN('\0', "show-kernel-path", &symbol_conf.show_kernel_path,
"Show the path of [kernel.kallsyms]"),
+ OPT_BOOLEAN('\0', "show-task-events", &script.show_task_events,
+ "Show the fork/comm/exit events"),
+ OPT_BOOLEAN('\0', "show-mmap-events", &script.show_mmap_events,
+ "Show the mmap events"),
OPT_END()
};
const char * const script_usage[] = {
if (session == NULL)
return -ENOMEM;
+ if (header || header_only) {
+ perf_session__fprintf_info(session, stdout, show_full_info);
+ if (header_only)
+ return 0;
+ }
+
script.session = session;
if (cpu_list) {
return -1;
}
- if (!script_name && !generate_script_lang)
- perf_session__fprintf_info(session, stdout, show_full_info);
-
if (!no_callchain)
symbol_conf.use_callchain = true;
else
return -1;
}
- err = scripting_ops->generate_script(session->pevent,
+ err = scripting_ops->generate_script(session->tevent.pevent,
"perf-script");
goto out;
}
static bool sync_run = false;
static unsigned int interval = 0;
static unsigned int initial_delay = 0;
+static unsigned int unit_width = 4; /* strlen("unit") */
static bool forever = false;
static struct timespec ref_time;
static struct cpu_map *aggr_map;
static void perf_evsel__free_stat_priv(struct perf_evsel *evsel)
{
- free(evsel->priv);
- evsel->priv = NULL;
+ zfree(&evsel->priv);
}
static int perf_evsel__alloc_prev_raw_counts(struct perf_evsel *evsel)
static void perf_evsel__free_prev_raw_counts(struct perf_evsel *evsel)
{
- free(evsel->prev_raw_counts);
- evsel->prev_raw_counts = NULL;
+ zfree(&evsel->prev_raw_counts);
}
static void perf_evlist__free_stats(struct perf_evlist *evlist)
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
perf_evsel__free_stat_priv(evsel);
perf_evsel__free_counts(evsel);
perf_evsel__free_prev_raw_counts(evsel);
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (perf_evsel__alloc_stat_priv(evsel) < 0 ||
perf_evsel__alloc_counts(evsel, perf_evsel__nr_cpus(evsel)) < 0 ||
(alloc_raw && perf_evsel__alloc_prev_raw_counts(evsel) < 0))
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
perf_evsel__reset_stat_priv(evsel);
perf_evsel__reset_counts(evsel, perf_evsel__nr_cpus(evsel));
}
/* Assumes this only called when evsel_list does not change anymore. */
if (!array) {
- list_for_each_entry(ev, &evsel_list->entries, node)
+ evlist__for_each(evsel_list, ev)
array_len++;
array = malloc(array_len * sizeof(void *));
if (!array)
exit(ENOMEM);
j = 0;
- list_for_each_entry(ev, &evsel_list->entries, node)
+ evlist__for_each(evsel_list, ev)
array[j++] = ev;
}
if (n < array_len)
char prefix[64];
if (aggr_mode == AGGR_GLOBAL) {
- list_for_each_entry(counter, &evsel_list->entries, node) {
+ evlist__for_each(evsel_list, counter) {
ps = counter->priv;
memset(ps->res_stats, 0, sizeof(ps->res_stats));
read_counter_aggr(counter);
}
} else {
- list_for_each_entry(counter, &evsel_list->entries, node) {
+ evlist__for_each(evsel_list, counter) {
ps = counter->priv;
memset(ps->res_stats, 0, sizeof(ps->res_stats));
read_counter(counter);
if (num_print_interval == 0 && !csv_output) {
switch (aggr_mode) {
case AGGR_SOCKET:
- fprintf(output, "# time socket cpus counts events\n");
+ fprintf(output, "# time socket cpus counts %*s events\n", unit_width, "unit");
break;
case AGGR_CORE:
- fprintf(output, "# time core cpus counts events\n");
+ fprintf(output, "# time core cpus counts %*s events\n", unit_width, "unit");
break;
case AGGR_NONE:
- fprintf(output, "# time CPU counts events\n");
+ fprintf(output, "# time CPU counts %*s events\n", unit_width, "unit");
break;
case AGGR_GLOBAL:
default:
- fprintf(output, "# time counts events\n");
+ fprintf(output, "# time counts %*s events\n", unit_width, "unit");
}
}
print_aggr(prefix);
break;
case AGGR_NONE:
- list_for_each_entry(counter, &evsel_list->entries, node)
+ evlist__for_each(evsel_list, counter)
print_counter(counter, prefix);
break;
case AGGR_GLOBAL:
default:
- list_for_each_entry(counter, &evsel_list->entries, node)
+ evlist__for_each(evsel_list, counter)
print_counter_aggr(counter, prefix);
}
nthreads = thread_map__nr(evsel_list->threads);
usleep(initial_delay * 1000);
- list_for_each_entry(counter, &evsel_list->entries, node)
+ evlist__for_each(evsel_list, counter)
perf_evsel__enable(counter, ncpus, nthreads);
}
}
+static volatile int workload_exec_errno;
+
+/*
+ * perf_evlist__prepare_workload will send a SIGUSR1
+ * if the fork fails, since we asked by setting its
+ * want_signal to true.
+ */
+static void workload_exec_failed_signal(int signo __maybe_unused, siginfo_t *info,
+ void *ucontext __maybe_unused)
+{
+ workload_exec_errno = info->si_value.sival_int;
+}
+
static int __run_perf_stat(int argc, const char **argv)
{
char msg[512];
unsigned long long t0, t1;
struct perf_evsel *counter;
struct timespec ts;
+ size_t l;
int status = 0;
const bool forks = (argc > 0);
}
if (forks) {
- if (perf_evlist__prepare_workload(evsel_list, &target, argv,
- false, false) < 0) {
+ if (perf_evlist__prepare_workload(evsel_list, &target, argv, false,
+ workload_exec_failed_signal) < 0) {
perror("failed to prepare workload");
return -1;
}
if (group)
perf_evlist__set_leader(evsel_list);
- list_for_each_entry(counter, &evsel_list->entries, node) {
+ evlist__for_each(evsel_list, counter) {
if (create_perf_stat_counter(counter) < 0) {
/*
* PPC returns ENXIO for HW counters until 2.6.37
return -1;
}
counter->supported = true;
+
+ l = strlen(counter->unit);
+ if (l > unit_width)
+ unit_width = l;
}
if (perf_evlist__apply_filters(evsel_list)) {
}
}
wait(&status);
+
+ if (workload_exec_errno) {
+ const char *emsg = strerror_r(workload_exec_errno, msg, sizeof(msg));
+ pr_err("Workload failed: %s\n", emsg);
+ return -1;
+ }
+
if (WIFSIGNALED(status))
psignal(WTERMSIG(status), argv[0]);
} else {
update_stats(&walltime_nsecs_stats, t1 - t0);
if (aggr_mode == AGGR_GLOBAL) {
- list_for_each_entry(counter, &evsel_list->entries, node) {
+ evlist__for_each(evsel_list, counter) {
read_counter_aggr(counter);
perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter),
thread_map__nr(evsel_list->threads));
}
} else {
- list_for_each_entry(counter, &evsel_list->entries, node) {
+ evlist__for_each(evsel_list, counter) {
read_counter(counter);
perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter), 1);
}
return WEXITSTATUS(status);
}
-static int run_perf_stat(int argc __maybe_unused, const char **argv)
+static int run_perf_stat(int argc, const char **argv)
{
int ret;
static void nsec_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
{
double msecs = avg / 1e6;
- const char *fmt = csv_output ? "%.6f%s%s" : "%18.6f%s%-25s";
+ const char *fmt_v, *fmt_n;
char name[25];
+ fmt_v = csv_output ? "%.6f%s" : "%18.6f%s";
+ fmt_n = csv_output ? "%s" : "%-25s";
+
aggr_printout(evsel, cpu, nr);
scnprintf(name, sizeof(name), "%s%s",
perf_evsel__name(evsel), csv_output ? "" : " (msec)");
- fprintf(output, fmt, msecs, csv_sep, name);
+
+ fprintf(output, fmt_v, msecs, csv_sep);
+
+ if (csv_output)
+ fprintf(output, "%s%s", evsel->unit, csv_sep);
+ else
+ fprintf(output, "%-*s%s", unit_width, evsel->unit, csv_sep);
+
+ fprintf(output, fmt_n, name);
if (evsel->cgrp)
fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
{
double total, ratio = 0.0, total2;
+ double sc = evsel->scale;
const char *fmt;
- if (csv_output)
- fmt = "%.0f%s%s";
- else if (big_num)
- fmt = "%'18.0f%s%-25s";
- else
- fmt = "%18.0f%s%-25s";
+ if (csv_output) {
+ fmt = sc != 1.0 ? "%.2f%s" : "%.0f%s";
+ } else {
+ if (big_num)
+ fmt = sc != 1.0 ? "%'18.2f%s" : "%'18.0f%s";
+ else
+ fmt = sc != 1.0 ? "%18.2f%s" : "%18.0f%s";
+ }
aggr_printout(evsel, cpu, nr);
if (aggr_mode == AGGR_GLOBAL)
cpu = 0;
- fprintf(output, fmt, avg, csv_sep, perf_evsel__name(evsel));
+ fprintf(output, fmt, avg, csv_sep);
+
+ if (evsel->unit)
+ fprintf(output, "%-*s%s",
+ csv_output ? 0 : unit_width,
+ evsel->unit, csv_sep);
+
+ fprintf(output, "%-*s", csv_output ? 0 : 25, perf_evsel__name(evsel));
if (evsel->cgrp)
fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
if (total && avg) {
ratio = total / avg;
- fprintf(output, "\n # %5.2f stalled cycles per insn", ratio);
+ fprintf(output, "\n");
+ if (aggr_mode == AGGR_NONE)
+ fprintf(output, " ");
+ fprintf(output, " # %5.2f stalled cycles per insn", ratio);
}
} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES) &&
{
struct perf_evsel *counter;
int cpu, cpu2, s, s2, id, nr;
+ double uval;
u64 ena, run, val;
if (!(aggr_map || aggr_get_id))
for (s = 0; s < aggr_map->nr; s++) {
id = aggr_map->map[s];
- list_for_each_entry(counter, &evsel_list->entries, node) {
+ evlist__for_each(evsel_list, counter) {
val = ena = run = 0;
nr = 0;
for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
if (run == 0 || ena == 0) {
aggr_printout(counter, id, nr);
- fprintf(output, "%*s%s%*s",
+ fprintf(output, "%*s%s",
csv_output ? 0 : 18,
counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
- csv_sep,
- csv_output ? 0 : -24,
+ csv_sep);
+
+ fprintf(output, "%-*s%s",
+ csv_output ? 0 : unit_width,
+ counter->unit, csv_sep);
+
+ fprintf(output, "%*s",
+ csv_output ? 0 : -25,
perf_evsel__name(counter));
if (counter->cgrp)
fputc('\n', output);
continue;
}
+ uval = val * counter->scale;
if (nsec_counter(counter))
- nsec_printout(id, nr, counter, val);
+ nsec_printout(id, nr, counter, uval);
else
- abs_printout(id, nr, counter, val);
+ abs_printout(id, nr, counter, uval);
if (!csv_output) {
print_noise(counter, 1.0);
struct perf_stat *ps = counter->priv;
double avg = avg_stats(&ps->res_stats[0]);
int scaled = counter->counts->scaled;
+ double uval;
if (prefix)
fprintf(output, "%s", prefix);
if (scaled == -1) {
- fprintf(output, "%*s%s%*s",
+ fprintf(output, "%*s%s",
csv_output ? 0 : 18,
counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
- csv_sep,
- csv_output ? 0 : -24,
+ csv_sep);
+ fprintf(output, "%-*s%s",
+ csv_output ? 0 : unit_width,
+ counter->unit, csv_sep);
+ fprintf(output, "%*s",
+ csv_output ? 0 : -25,
perf_evsel__name(counter));
if (counter->cgrp)
return;
}
+ uval = avg * counter->scale;
+
if (nsec_counter(counter))
- nsec_printout(-1, 0, counter, avg);
+ nsec_printout(-1, 0, counter, uval);
else
- abs_printout(-1, 0, counter, avg);
+ abs_printout(-1, 0, counter, uval);
print_noise(counter, avg);
static void print_counter(struct perf_evsel *counter, char *prefix)
{
u64 ena, run, val;
+ double uval;
int cpu;
for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
fprintf(output, "%s", prefix);
if (run == 0 || ena == 0) {
- fprintf(output, "CPU%*d%s%*s%s%*s",
+ fprintf(output, "CPU%*d%s%*s%s",
csv_output ? 0 : -4,
perf_evsel__cpus(counter)->map[cpu], csv_sep,
csv_output ? 0 : 18,
counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
- csv_sep,
- csv_output ? 0 : -24,
- perf_evsel__name(counter));
+ csv_sep);
+
+ fprintf(output, "%-*s%s",
+ csv_output ? 0 : unit_width,
+ counter->unit, csv_sep);
+
+ fprintf(output, "%*s",
+ csv_output ? 0 : -25,
+ perf_evsel__name(counter));
if (counter->cgrp)
fprintf(output, "%s%s",
continue;
}
+ uval = val * counter->scale;
+
if (nsec_counter(counter))
- nsec_printout(cpu, 0, counter, val);
+ nsec_printout(cpu, 0, counter, uval);
else
- abs_printout(cpu, 0, counter, val);
+ abs_printout(cpu, 0, counter, uval);
if (!csv_output) {
print_noise(counter, 1.0);
print_aggr(NULL);
break;
case AGGR_GLOBAL:
- list_for_each_entry(counter, &evsel_list->entries, node)
+ evlist__for_each(evsel_list, counter)
print_counter_aggr(counter, NULL);
break;
case AGGR_NONE:
- list_for_each_entry(counter, &evsel_list->entries, node)
+ evlist__for_each(evsel_list, counter)
print_counter(counter, NULL);
break;
default:
if (interval && interval < 100) {
pr_err("print interval must be >= 100ms\n");
parse_options_usage(stat_usage, options, "I", 1);
- goto out_free_maps;
+ goto out;
}
if (perf_evlist__alloc_stats(evsel_list, interval))
- goto out_free_maps;
+ goto out;
if (perf_stat_init_aggr_mode())
- goto out_free_maps;
+ goto out;
/*
* We dont want to block the signals - that would cause
print_stat(argc, argv);
perf_evlist__free_stats(evsel_list);
-out_free_maps:
- perf_evlist__delete_maps(evsel_list);
out:
perf_evlist__delete(evsel_list);
return status;
#define SUPPORT_OLD_POWER_EVENTS 1
#define PWR_EVENT_EXIT -1
-
-static unsigned int numcpus;
-static u64 min_freq; /* Lowest CPU frequency seen */
-static u64 max_freq; /* Highest CPU frequency seen */
-static u64 turbo_frequency;
-
-static u64 first_time, last_time;
-
-static bool power_only;
-
-
struct per_pid;
-struct per_pidcomm;
-
-struct cpu_sample;
struct power_event;
struct wake_event;
-struct sample_wrapper;
+struct timechart {
+ struct perf_tool tool;
+ struct per_pid *all_data;
+ struct power_event *power_events;
+ struct wake_event *wake_events;
+ int proc_num;
+ unsigned int numcpus;
+ u64 min_freq, /* Lowest CPU frequency seen */
+ max_freq, /* Highest CPU frequency seen */
+ turbo_frequency,
+ first_time, last_time;
+ bool power_only,
+ tasks_only,
+ with_backtrace,
+ topology;
+};
+
+struct per_pidcomm;
+struct cpu_sample;
/*
* Datastructure layout:
u64 end_time;
int type;
int cpu;
+ const char *backtrace;
};
-static struct per_pid *all_data;
-
#define CSTATE 1
#define PSTATE 2
int waker;
int wakee;
u64 time;
+ const char *backtrace;
};
-static struct power_event *power_events;
-static struct wake_event *wake_events;
-
-struct process_filter;
struct process_filter {
char *name;
int pid;
static struct process_filter *process_filter;
-static struct per_pid *find_create_pid(int pid)
+static struct per_pid *find_create_pid(struct timechart *tchart, int pid)
{
- struct per_pid *cursor = all_data;
+ struct per_pid *cursor = tchart->all_data;
while (cursor) {
if (cursor->pid == pid)
cursor = zalloc(sizeof(*cursor));
assert(cursor != NULL);
cursor->pid = pid;
- cursor->next = all_data;
- all_data = cursor;
+ cursor->next = tchart->all_data;
+ tchart->all_data = cursor;
return cursor;
}
-static void pid_set_comm(int pid, char *comm)
+static void pid_set_comm(struct timechart *tchart, int pid, char *comm)
{
struct per_pid *p;
struct per_pidcomm *c;
- p = find_create_pid(pid);
+ p = find_create_pid(tchart, pid);
c = p->all;
while (c) {
if (c->comm && strcmp(c->comm, comm) == 0) {
p->all = c;
}
-static void pid_fork(int pid, int ppid, u64 timestamp)
+static void pid_fork(struct timechart *tchart, int pid, int ppid, u64 timestamp)
{
struct per_pid *p, *pp;
- p = find_create_pid(pid);
- pp = find_create_pid(ppid);
+ p = find_create_pid(tchart, pid);
+ pp = find_create_pid(tchart, ppid);
p->ppid = ppid;
if (pp->current && pp->current->comm && !p->current)
- pid_set_comm(pid, pp->current->comm);
+ pid_set_comm(tchart, pid, pp->current->comm);
p->start_time = timestamp;
if (p->current) {
}
}
-static void pid_exit(int pid, u64 timestamp)
+static void pid_exit(struct timechart *tchart, int pid, u64 timestamp)
{
struct per_pid *p;
- p = find_create_pid(pid);
+ p = find_create_pid(tchart, pid);
p->end_time = timestamp;
if (p->current)
p->current->end_time = timestamp;
}
-static void
-pid_put_sample(int pid, int type, unsigned int cpu, u64 start, u64 end)
+static void pid_put_sample(struct timechart *tchart, int pid, int type,
+ unsigned int cpu, u64 start, u64 end,
+ const char *backtrace)
{
struct per_pid *p;
struct per_pidcomm *c;
struct cpu_sample *sample;
- p = find_create_pid(pid);
+ p = find_create_pid(tchart, pid);
c = p->current;
if (!c) {
c = zalloc(sizeof(*c));
sample->type = type;
sample->next = c->samples;
sample->cpu = cpu;
+ sample->backtrace = backtrace;
c->samples = sample;
if (sample->type == TYPE_RUNNING && end > start && start > 0) {
static u64 cpus_pstate_start_times[MAX_CPUS];
static u64 cpus_pstate_state[MAX_CPUS];
-static int process_comm_event(struct perf_tool *tool __maybe_unused,
+static int process_comm_event(struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample __maybe_unused,
struct machine *machine __maybe_unused)
{
- pid_set_comm(event->comm.tid, event->comm.comm);
+ struct timechart *tchart = container_of(tool, struct timechart, tool);
+ pid_set_comm(tchart, event->comm.tid, event->comm.comm);
return 0;
}
-static int process_fork_event(struct perf_tool *tool __maybe_unused,
+static int process_fork_event(struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample __maybe_unused,
struct machine *machine __maybe_unused)
{
- pid_fork(event->fork.pid, event->fork.ppid, event->fork.time);
+ struct timechart *tchart = container_of(tool, struct timechart, tool);
+ pid_fork(tchart, event->fork.pid, event->fork.ppid, event->fork.time);
return 0;
}
-static int process_exit_event(struct perf_tool *tool __maybe_unused,
+static int process_exit_event(struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample __maybe_unused,
struct machine *machine __maybe_unused)
{
- pid_exit(event->fork.pid, event->fork.time);
+ struct timechart *tchart = container_of(tool, struct timechart, tool);
+ pid_exit(tchart, event->fork.pid, event->fork.time);
return 0;
}
-struct trace_entry {
- unsigned short type;
- unsigned char flags;
- unsigned char preempt_count;
- int pid;
- int lock_depth;
-};
-
#ifdef SUPPORT_OLD_POWER_EVENTS
static int use_old_power_events;
-struct power_entry_old {
- struct trace_entry te;
- u64 type;
- u64 value;
- u64 cpu_id;
-};
#endif
-struct power_processor_entry {
- struct trace_entry te;
- u32 state;
- u32 cpu_id;
-};
-
-#define TASK_COMM_LEN 16
-struct wakeup_entry {
- struct trace_entry te;
- char comm[TASK_COMM_LEN];
- int pid;
- int prio;
- int success;
-};
-
-struct sched_switch {
- struct trace_entry te;
- char prev_comm[TASK_COMM_LEN];
- int prev_pid;
- int prev_prio;
- long prev_state; /* Arjan weeps. */
- char next_comm[TASK_COMM_LEN];
- int next_pid;
- int next_prio;
-};
-
static void c_state_start(int cpu, u64 timestamp, int state)
{
cpus_cstate_start_times[cpu] = timestamp;
cpus_cstate_state[cpu] = state;
}
-static void c_state_end(int cpu, u64 timestamp)
+static void c_state_end(struct timechart *tchart, int cpu, u64 timestamp)
{
struct power_event *pwr = zalloc(sizeof(*pwr));
pwr->end_time = timestamp;
pwr->cpu = cpu;
pwr->type = CSTATE;
- pwr->next = power_events;
+ pwr->next = tchart->power_events;
- power_events = pwr;
+ tchart->power_events = pwr;
}
-static void p_state_change(int cpu, u64 timestamp, u64 new_freq)
+static void p_state_change(struct timechart *tchart, int cpu, u64 timestamp, u64 new_freq)
{
struct power_event *pwr;
pwr->end_time = timestamp;
pwr->cpu = cpu;
pwr->type = PSTATE;
- pwr->next = power_events;
+ pwr->next = tchart->power_events;
if (!pwr->start_time)
- pwr->start_time = first_time;
+ pwr->start_time = tchart->first_time;
- power_events = pwr;
+ tchart->power_events = pwr;
cpus_pstate_state[cpu] = new_freq;
cpus_pstate_start_times[cpu] = timestamp;
- if ((u64)new_freq > max_freq)
- max_freq = new_freq;
+ if ((u64)new_freq > tchart->max_freq)
+ tchart->max_freq = new_freq;
- if (new_freq < min_freq || min_freq == 0)
- min_freq = new_freq;
+ if (new_freq < tchart->min_freq || tchart->min_freq == 0)
+ tchart->min_freq = new_freq;
- if (new_freq == max_freq - 1000)
- turbo_frequency = max_freq;
+ if (new_freq == tchart->max_freq - 1000)
+ tchart->turbo_frequency = tchart->max_freq;
}
-static void
-sched_wakeup(int cpu, u64 timestamp, int pid, struct trace_entry *te)
+static void sched_wakeup(struct timechart *tchart, int cpu, u64 timestamp,
+ int waker, int wakee, u8 flags, const char *backtrace)
{
struct per_pid *p;
- struct wakeup_entry *wake = (void *)te;
struct wake_event *we = zalloc(sizeof(*we));
if (!we)
return;
we->time = timestamp;
- we->waker = pid;
+ we->waker = waker;
+ we->backtrace = backtrace;
- if ((te->flags & TRACE_FLAG_HARDIRQ) || (te->flags & TRACE_FLAG_SOFTIRQ))
+ if ((flags & TRACE_FLAG_HARDIRQ) || (flags & TRACE_FLAG_SOFTIRQ))
we->waker = -1;
- we->wakee = wake->pid;
- we->next = wake_events;
- wake_events = we;
- p = find_create_pid(we->wakee);
+ we->wakee = wakee;
+ we->next = tchart->wake_events;
+ tchart->wake_events = we;
+ p = find_create_pid(tchart, we->wakee);
if (p && p->current && p->current->state == TYPE_NONE) {
p->current->state_since = timestamp;
p->current->state = TYPE_WAITING;
}
if (p && p->current && p->current->state == TYPE_BLOCKED) {
- pid_put_sample(p->pid, p->current->state, cpu, p->current->state_since, timestamp);
+ pid_put_sample(tchart, p->pid, p->current->state, cpu,
+ p->current->state_since, timestamp, NULL);
p->current->state_since = timestamp;
p->current->state = TYPE_WAITING;
}
}
-static void sched_switch(int cpu, u64 timestamp, struct trace_entry *te)
+static void sched_switch(struct timechart *tchart, int cpu, u64 timestamp,
+ int prev_pid, int next_pid, u64 prev_state,
+ const char *backtrace)
{
struct per_pid *p = NULL, *prev_p;
- struct sched_switch *sw = (void *)te;
-
- prev_p = find_create_pid(sw->prev_pid);
+ prev_p = find_create_pid(tchart, prev_pid);
- p = find_create_pid(sw->next_pid);
+ p = find_create_pid(tchart, next_pid);
if (prev_p->current && prev_p->current->state != TYPE_NONE)
- pid_put_sample(sw->prev_pid, TYPE_RUNNING, cpu, prev_p->current->state_since, timestamp);
+ pid_put_sample(tchart, prev_pid, TYPE_RUNNING, cpu,
+ prev_p->current->state_since, timestamp,
+ backtrace);
if (p && p->current) {
if (p->current->state != TYPE_NONE)
- pid_put_sample(sw->next_pid, p->current->state, cpu, p->current->state_since, timestamp);
+ pid_put_sample(tchart, next_pid, p->current->state, cpu,
+ p->current->state_since, timestamp,
+ backtrace);
p->current->state_since = timestamp;
p->current->state = TYPE_RUNNING;
if (prev_p->current) {
prev_p->current->state = TYPE_NONE;
prev_p->current->state_since = timestamp;
- if (sw->prev_state & 2)
+ if (prev_state & 2)
prev_p->current->state = TYPE_BLOCKED;
- if (sw->prev_state == 0)
+ if (prev_state == 0)
prev_p->current->state = TYPE_WAITING;
}
}
-typedef int (*tracepoint_handler)(struct perf_evsel *evsel,
- struct perf_sample *sample);
+static const char *cat_backtrace(union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct addr_location al;
+ unsigned int i;
+ char *p = NULL;
+ size_t p_len;
+ u8 cpumode = PERF_RECORD_MISC_USER;
+ struct addr_location tal;
+ struct ip_callchain *chain = sample->callchain;
+ FILE *f = open_memstream(&p, &p_len);
+
+ if (!f) {
+ perror("open_memstream error");
+ return NULL;
+ }
+
+ if (!chain)
+ goto exit;
-static int process_sample_event(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
+ if (perf_event__preprocess_sample(event, machine, &al, sample) < 0) {
+ fprintf(stderr, "problem processing %d event, skipping it.\n",
+ event->header.type);
+ goto exit;
+ }
+
+ for (i = 0; i < chain->nr; i++) {
+ u64 ip;
+
+ if (callchain_param.order == ORDER_CALLEE)
+ ip = chain->ips[i];
+ else
+ ip = chain->ips[chain->nr - i - 1];
+
+ if (ip >= PERF_CONTEXT_MAX) {
+ switch (ip) {
+ case PERF_CONTEXT_HV:
+ cpumode = PERF_RECORD_MISC_HYPERVISOR;
+ break;
+ case PERF_CONTEXT_KERNEL:
+ cpumode = PERF_RECORD_MISC_KERNEL;
+ break;
+ case PERF_CONTEXT_USER:
+ cpumode = PERF_RECORD_MISC_USER;
+ break;
+ default:
+ pr_debug("invalid callchain context: "
+ "%"PRId64"\n", (s64) ip);
+
+ /*
+ * It seems the callchain is corrupted.
+ * Discard all.
+ */
+ zfree(&p);
+ goto exit;
+ }
+ continue;
+ }
+
+ tal.filtered = false;
+ thread__find_addr_location(al.thread, machine, cpumode,
+ MAP__FUNCTION, ip, &tal);
+
+ if (tal.sym)
+ fprintf(f, "..... %016" PRIx64 " %s\n", ip,
+ tal.sym->name);
+ else
+ fprintf(f, "..... %016" PRIx64 "\n", ip);
+ }
+
+exit:
+ fclose(f);
+
+ return p;
+}
+
+typedef int (*tracepoint_handler)(struct timechart *tchart,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *backtrace);
+
+static int process_sample_event(struct perf_tool *tool,
+ union perf_event *event,
struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine __maybe_unused)
+ struct machine *machine)
{
+ struct timechart *tchart = container_of(tool, struct timechart, tool);
+
if (evsel->attr.sample_type & PERF_SAMPLE_TIME) {
- if (!first_time || first_time > sample->time)
- first_time = sample->time;
- if (last_time < sample->time)
- last_time = sample->time;
+ if (!tchart->first_time || tchart->first_time > sample->time)
+ tchart->first_time = sample->time;
+ if (tchart->last_time < sample->time)
+ tchart->last_time = sample->time;
}
- if (sample->cpu > numcpus)
- numcpus = sample->cpu;
-
if (evsel->handler != NULL) {
tracepoint_handler f = evsel->handler;
- return f(evsel, sample);
+ return f(tchart, evsel, sample,
+ cat_backtrace(event, sample, machine));
}
return 0;
}
static int
-process_sample_cpu_idle(struct perf_evsel *evsel __maybe_unused,
- struct perf_sample *sample)
+process_sample_cpu_idle(struct timechart *tchart __maybe_unused,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *backtrace __maybe_unused)
{
- struct power_processor_entry *ppe = sample->raw_data;
+ u32 state = perf_evsel__intval(evsel, sample, "state");
+ u32 cpu_id = perf_evsel__intval(evsel, sample, "cpu_id");
- if (ppe->state == (u32) PWR_EVENT_EXIT)
- c_state_end(ppe->cpu_id, sample->time);
+ if (state == (u32)PWR_EVENT_EXIT)
+ c_state_end(tchart, cpu_id, sample->time);
else
- c_state_start(ppe->cpu_id, sample->time, ppe->state);
+ c_state_start(cpu_id, sample->time, state);
return 0;
}
static int
-process_sample_cpu_frequency(struct perf_evsel *evsel __maybe_unused,
- struct perf_sample *sample)
+process_sample_cpu_frequency(struct timechart *tchart,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *backtrace __maybe_unused)
{
- struct power_processor_entry *ppe = sample->raw_data;
+ u32 state = perf_evsel__intval(evsel, sample, "state");
+ u32 cpu_id = perf_evsel__intval(evsel, sample, "cpu_id");
- p_state_change(ppe->cpu_id, sample->time, ppe->state);
+ p_state_change(tchart, cpu_id, sample->time, state);
return 0;
}
static int
-process_sample_sched_wakeup(struct perf_evsel *evsel __maybe_unused,
- struct perf_sample *sample)
+process_sample_sched_wakeup(struct timechart *tchart,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *backtrace)
{
- struct trace_entry *te = sample->raw_data;
+ u8 flags = perf_evsel__intval(evsel, sample, "common_flags");
+ int waker = perf_evsel__intval(evsel, sample, "common_pid");
+ int wakee = perf_evsel__intval(evsel, sample, "pid");
- sched_wakeup(sample->cpu, sample->time, sample->pid, te);
+ sched_wakeup(tchart, sample->cpu, sample->time, waker, wakee, flags, backtrace);
return 0;
}
static int
-process_sample_sched_switch(struct perf_evsel *evsel __maybe_unused,
- struct perf_sample *sample)
+process_sample_sched_switch(struct timechart *tchart,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *backtrace)
{
- struct trace_entry *te = sample->raw_data;
+ int prev_pid = perf_evsel__intval(evsel, sample, "prev_pid");
+ int next_pid = perf_evsel__intval(evsel, sample, "next_pid");
+ u64 prev_state = perf_evsel__intval(evsel, sample, "prev_state");
- sched_switch(sample->cpu, sample->time, te);
+ sched_switch(tchart, sample->cpu, sample->time, prev_pid, next_pid,
+ prev_state, backtrace);
return 0;
}
#ifdef SUPPORT_OLD_POWER_EVENTS
static int
-process_sample_power_start(struct perf_evsel *evsel __maybe_unused,
- struct perf_sample *sample)
+process_sample_power_start(struct timechart *tchart __maybe_unused,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *backtrace __maybe_unused)
{
- struct power_entry_old *peo = sample->raw_data;
+ u64 cpu_id = perf_evsel__intval(evsel, sample, "cpu_id");
+ u64 value = perf_evsel__intval(evsel, sample, "value");
- c_state_start(peo->cpu_id, sample->time, peo->value);
+ c_state_start(cpu_id, sample->time, value);
return 0;
}
static int
-process_sample_power_end(struct perf_evsel *evsel __maybe_unused,
- struct perf_sample *sample)
+process_sample_power_end(struct timechart *tchart,
+ struct perf_evsel *evsel __maybe_unused,
+ struct perf_sample *sample,
+ const char *backtrace __maybe_unused)
{
- c_state_end(sample->cpu, sample->time);
+ c_state_end(tchart, sample->cpu, sample->time);
return 0;
}
static int
-process_sample_power_frequency(struct perf_evsel *evsel __maybe_unused,
- struct perf_sample *sample)
+process_sample_power_frequency(struct timechart *tchart,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *backtrace __maybe_unused)
{
- struct power_entry_old *peo = sample->raw_data;
+ u64 cpu_id = perf_evsel__intval(evsel, sample, "cpu_id");
+ u64 value = perf_evsel__intval(evsel, sample, "value");
- p_state_change(peo->cpu_id, sample->time, peo->value);
+ p_state_change(tchart, cpu_id, sample->time, value);
return 0;
}
#endif /* SUPPORT_OLD_POWER_EVENTS */
* After the last sample we need to wrap up the current C/P state
* and close out each CPU for these.
*/
-static void end_sample_processing(void)
+static void end_sample_processing(struct timechart *tchart)
{
u64 cpu;
struct power_event *pwr;
- for (cpu = 0; cpu <= numcpus; cpu++) {
+ for (cpu = 0; cpu <= tchart->numcpus; cpu++) {
/* C state */
#if 0
pwr = zalloc(sizeof(*pwr));
pwr->state = cpus_cstate_state[cpu];
pwr->start_time = cpus_cstate_start_times[cpu];
- pwr->end_time = last_time;
+ pwr->end_time = tchart->last_time;
pwr->cpu = cpu;
pwr->type = CSTATE;
- pwr->next = power_events;
+ pwr->next = tchart->power_events;
- power_events = pwr;
+ tchart->power_events = pwr;
#endif
/* P state */
pwr->state = cpus_pstate_state[cpu];
pwr->start_time = cpus_pstate_start_times[cpu];
- pwr->end_time = last_time;
+ pwr->end_time = tchart->last_time;
pwr->cpu = cpu;
pwr->type = PSTATE;
- pwr->next = power_events;
+ pwr->next = tchart->power_events;
if (!pwr->start_time)
- pwr->start_time = first_time;
+ pwr->start_time = tchart->first_time;
if (!pwr->state)
- pwr->state = min_freq;
- power_events = pwr;
+ pwr->state = tchart->min_freq;
+ tchart->power_events = pwr;
}
}
/*
* Sort the pid datastructure
*/
-static void sort_pids(void)
+static void sort_pids(struct timechart *tchart)
{
struct per_pid *new_list, *p, *cursor, *prev;
/* sort by ppid first, then by pid, lowest to highest */
new_list = NULL;
- while (all_data) {
- p = all_data;
- all_data = p->next;
+ while (tchart->all_data) {
+ p = tchart->all_data;
+ tchart->all_data = p->next;
p->next = NULL;
if (new_list == NULL) {
prev->next = p;
}
}
- all_data = new_list;
+ tchart->all_data = new_list;
}
-static void draw_c_p_states(void)
+static void draw_c_p_states(struct timechart *tchart)
{
struct power_event *pwr;
- pwr = power_events;
+ pwr = tchart->power_events;
/*
* two pass drawing so that the P state bars are on top of the C state blocks
pwr = pwr->next;
}
- pwr = power_events;
+ pwr = tchart->power_events;
while (pwr) {
if (pwr->type == PSTATE) {
if (!pwr->state)
- pwr->state = min_freq;
+ pwr->state = tchart->min_freq;
svg_pstate(pwr->cpu, pwr->start_time, pwr->end_time, pwr->state);
}
pwr = pwr->next;
}
}
-static void draw_wakeups(void)
+static void draw_wakeups(struct timechart *tchart)
{
struct wake_event *we;
struct per_pid *p;
struct per_pidcomm *c;
- we = wake_events;
+ we = tchart->wake_events;
while (we) {
int from = 0, to = 0;
char *task_from = NULL, *task_to = NULL;
/* locate the column of the waker and wakee */
- p = all_data;
+ p = tchart->all_data;
while (p) {
if (p->pid == we->waker || p->pid == we->wakee) {
c = p->all;
}
if (we->waker == -1)
- svg_interrupt(we->time, to);
+ svg_interrupt(we->time, to, we->backtrace);
else if (from && to && abs(from - to) == 1)
- svg_wakeline(we->time, from, to);
+ svg_wakeline(we->time, from, to, we->backtrace);
else
- svg_partial_wakeline(we->time, from, task_from, to, task_to);
+ svg_partial_wakeline(we->time, from, task_from, to,
+ task_to, we->backtrace);
we = we->next;
free(task_from);
}
}
-static void draw_cpu_usage(void)
+static void draw_cpu_usage(struct timechart *tchart)
{
struct per_pid *p;
struct per_pidcomm *c;
struct cpu_sample *sample;
- p = all_data;
+ p = tchart->all_data;
while (p) {
c = p->all;
while (c) {
sample = c->samples;
while (sample) {
- if (sample->type == TYPE_RUNNING)
- svg_process(sample->cpu, sample->start_time, sample->end_time, "sample", c->comm);
+ if (sample->type == TYPE_RUNNING) {
+ svg_process(sample->cpu,
+ sample->start_time,
+ sample->end_time,
+ p->pid,
+ c->comm,
+ sample->backtrace);
+ }
sample = sample->next;
}
}
}
-static void draw_process_bars(void)
+static void draw_process_bars(struct timechart *tchart)
{
struct per_pid *p;
struct per_pidcomm *c;
struct cpu_sample *sample;
int Y = 0;
- Y = 2 * numcpus + 2;
+ Y = 2 * tchart->numcpus + 2;
- p = all_data;
+ p = tchart->all_data;
while (p) {
c = p->all;
while (c) {
sample = c->samples;
while (sample) {
if (sample->type == TYPE_RUNNING)
- svg_sample(Y, sample->cpu, sample->start_time, sample->end_time);
+ svg_running(Y, sample->cpu,
+ sample->start_time,
+ sample->end_time,
+ sample->backtrace);
if (sample->type == TYPE_BLOCKED)
- svg_box(Y, sample->start_time, sample->end_time, "blocked");
+ svg_blocked(Y, sample->cpu,
+ sample->start_time,
+ sample->end_time,
+ sample->backtrace);
if (sample->type == TYPE_WAITING)
- svg_waiting(Y, sample->start_time, sample->end_time);
+ svg_waiting(Y, sample->cpu,
+ sample->start_time,
+ sample->end_time,
+ sample->backtrace);
sample = sample->next;
}
return 0;
}
-static int determine_display_tasks_filtered(void)
+static int determine_display_tasks_filtered(struct timechart *tchart)
{
struct per_pid *p;
struct per_pidcomm *c;
int count = 0;
- p = all_data;
+ p = tchart->all_data;
while (p) {
p->display = 0;
if (p->start_time == 1)
- p->start_time = first_time;
+ p->start_time = tchart->first_time;
/* no exit marker, task kept running to the end */
if (p->end_time == 0)
- p->end_time = last_time;
+ p->end_time = tchart->last_time;
c = p->all;
c->display = 0;
if (c->start_time == 1)
- c->start_time = first_time;
+ c->start_time = tchart->first_time;
if (passes_filter(p, c)) {
c->display = 1;
}
if (c->end_time == 0)
- c->end_time = last_time;
+ c->end_time = tchart->last_time;
c = c->next;
}
return count;
}
-static int determine_display_tasks(u64 threshold)
+static int determine_display_tasks(struct timechart *tchart, u64 threshold)
{
struct per_pid *p;
struct per_pidcomm *c;
int count = 0;
if (process_filter)
- return determine_display_tasks_filtered();
+ return determine_display_tasks_filtered(tchart);
- p = all_data;
+ p = tchart->all_data;
while (p) {
p->display = 0;
if (p->start_time == 1)
- p->start_time = first_time;
+ p->start_time = tchart->first_time;
/* no exit marker, task kept running to the end */
if (p->end_time == 0)
- p->end_time = last_time;
- if (p->total_time >= threshold && !power_only)
+ p->end_time = tchart->last_time;
+ if (p->total_time >= threshold)
p->display = 1;
c = p->all;
c->display = 0;
if (c->start_time == 1)
- c->start_time = first_time;
+ c->start_time = tchart->first_time;
- if (c->total_time >= threshold && !power_only) {
+ if (c->total_time >= threshold) {
c->display = 1;
count++;
}
if (c->end_time == 0)
- c->end_time = last_time;
+ c->end_time = tchart->last_time;
c = c->next;
}
#define TIME_THRESH 10000000
-static void write_svg_file(const char *filename)
+static void write_svg_file(struct timechart *tchart, const char *filename)
{
u64 i;
int count;
+ int thresh = TIME_THRESH;
- numcpus++;
-
+ if (tchart->power_only)
+ tchart->proc_num = 0;
- count = determine_display_tasks(TIME_THRESH);
+ /* We'd like to show at least proc_num tasks;
+ * be less picky if we have fewer */
+ do {
+ count = determine_display_tasks(tchart, thresh);
+ thresh /= 10;
+ } while (!process_filter && thresh && count < tchart->proc_num);
- /* We'd like to show at least 15 tasks; be less picky if we have fewer */
- if (count < 15)
- count = determine_display_tasks(TIME_THRESH / 10);
-
- open_svg(filename, numcpus, count, first_time, last_time);
+ open_svg(filename, tchart->numcpus, count, tchart->first_time, tchart->last_time);
svg_time_grid();
svg_legenda();
- for (i = 0; i < numcpus; i++)
- svg_cpu_box(i, max_freq, turbo_frequency);
+ for (i = 0; i < tchart->numcpus; i++)
+ svg_cpu_box(i, tchart->max_freq, tchart->turbo_frequency);
- draw_cpu_usage();
- draw_process_bars();
- draw_c_p_states();
- draw_wakeups();
+ draw_cpu_usage(tchart);
+ if (tchart->proc_num)
+ draw_process_bars(tchart);
+ if (!tchart->tasks_only)
+ draw_c_p_states(tchart);
+ if (tchart->proc_num)
+ draw_wakeups(tchart);
svg_close();
}
-static int __cmd_timechart(const char *output_name)
+static int process_header(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph,
+ int feat,
+ int fd __maybe_unused,
+ void *data)
+{
+ struct timechart *tchart = data;
+
+ switch (feat) {
+ case HEADER_NRCPUS:
+ tchart->numcpus = ph->env.nr_cpus_avail;
+ break;
+
+ case HEADER_CPU_TOPOLOGY:
+ if (!tchart->topology)
+ break;
+
+ if (svg_build_topology_map(ph->env.sibling_cores,
+ ph->env.nr_sibling_cores,
+ ph->env.sibling_threads,
+ ph->env.nr_sibling_threads))
+ fprintf(stderr, "problem building topology\n");
+ break;
+
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+static int __cmd_timechart(struct timechart *tchart, const char *output_name)
{
- struct perf_tool perf_timechart = {
- .comm = process_comm_event,
- .fork = process_fork_event,
- .exit = process_exit_event,
- .sample = process_sample_event,
- .ordered_samples = true,
- };
const struct perf_evsel_str_handler power_tracepoints[] = {
{ "power:cpu_idle", process_sample_cpu_idle },
{ "power:cpu_frequency", process_sample_cpu_frequency },
};
struct perf_session *session = perf_session__new(&file, false,
- &perf_timechart);
+ &tchart->tool);
int ret = -EINVAL;
if (session == NULL)
return -ENOMEM;
+ (void)perf_header__process_sections(&session->header,
+ perf_data_file__fd(session->file),
+ tchart,
+ process_header);
+
if (!perf_session__has_traces(session, "timechart record"))
goto out_delete;
goto out_delete;
}
- ret = perf_session__process_events(session, &perf_timechart);
+ ret = perf_session__process_events(session, &tchart->tool);
if (ret)
goto out_delete;
- end_sample_processing();
+ end_sample_processing(tchart);
- sort_pids();
+ sort_pids(tchart);
- write_svg_file(output_name);
+ write_svg_file(tchart, output_name);
pr_info("Written %2.1f seconds of trace to %s.\n",
- (last_time - first_time) / 1000000000.0, output_name);
+ (tchart->last_time - tchart->first_time) / 1000000000.0, output_name);
out_delete:
perf_session__delete(session);
return ret;
}
-static int __cmd_record(int argc, const char **argv)
+static int timechart__record(struct timechart *tchart, int argc, const char **argv)
{
-#ifdef SUPPORT_OLD_POWER_EVENTS
- const char * const record_old_args[] = {
+ unsigned int rec_argc, i, j;
+ const char **rec_argv;
+ const char **p;
+ unsigned int record_elems;
+
+ const char * const common_args[] = {
"record", "-a", "-R", "-c", "1",
+ };
+ unsigned int common_args_nr = ARRAY_SIZE(common_args);
+
+ const char * const backtrace_args[] = {
+ "-g",
+ };
+ unsigned int backtrace_args_no = ARRAY_SIZE(backtrace_args);
+
+ const char * const power_args[] = {
+ "-e", "power:cpu_frequency",
+ "-e", "power:cpu_idle",
+ };
+ unsigned int power_args_nr = ARRAY_SIZE(power_args);
+
+ const char * const old_power_args[] = {
+#ifdef SUPPORT_OLD_POWER_EVENTS
"-e", "power:power_start",
"-e", "power:power_end",
"-e", "power:power_frequency",
- "-e", "sched:sched_wakeup",
- "-e", "sched:sched_switch",
- };
#endif
- const char * const record_new_args[] = {
- "record", "-a", "-R", "-c", "1",
- "-e", "power:cpu_frequency",
- "-e", "power:cpu_idle",
+ };
+ unsigned int old_power_args_nr = ARRAY_SIZE(old_power_args);
+
+ const char * const tasks_args[] = {
"-e", "sched:sched_wakeup",
"-e", "sched:sched_switch",
};
- unsigned int rec_argc, i, j;
- const char **rec_argv;
- const char * const *record_args = record_new_args;
- unsigned int record_elems = ARRAY_SIZE(record_new_args);
+ unsigned int tasks_args_nr = ARRAY_SIZE(tasks_args);
#ifdef SUPPORT_OLD_POWER_EVENTS
if (!is_valid_tracepoint("power:cpu_idle") &&
is_valid_tracepoint("power:power_start")) {
use_old_power_events = 1;
- record_args = record_old_args;
- record_elems = ARRAY_SIZE(record_old_args);
+ power_args_nr = 0;
+ } else {
+ old_power_args_nr = 0;
}
#endif
- rec_argc = record_elems + argc - 1;
+ if (tchart->power_only)
+ tasks_args_nr = 0;
+
+ if (tchart->tasks_only) {
+ power_args_nr = 0;
+ old_power_args_nr = 0;
+ }
+
+ if (!tchart->with_backtrace)
+ backtrace_args_no = 0;
+
+ record_elems = common_args_nr + tasks_args_nr +
+ power_args_nr + old_power_args_nr + backtrace_args_no;
+
+ rec_argc = record_elems + argc;
rec_argv = calloc(rec_argc + 1, sizeof(char *));
if (rec_argv == NULL)
return -ENOMEM;
- for (i = 0; i < record_elems; i++)
- rec_argv[i] = strdup(record_args[i]);
+ p = rec_argv;
+ for (i = 0; i < common_args_nr; i++)
+ *p++ = strdup(common_args[i]);
+
+ for (i = 0; i < backtrace_args_no; i++)
+ *p++ = strdup(backtrace_args[i]);
+
+ for (i = 0; i < tasks_args_nr; i++)
+ *p++ = strdup(tasks_args[i]);
+
+ for (i = 0; i < power_args_nr; i++)
+ *p++ = strdup(power_args[i]);
- for (j = 1; j < (unsigned int)argc; j++, i++)
- rec_argv[i] = argv[j];
+ for (i = 0; i < old_power_args_nr; i++)
+ *p++ = strdup(old_power_args[i]);
- return cmd_record(i, rec_argv, NULL);
+ for (j = 1; j < (unsigned int)argc; j++)
+ *p++ = argv[j];
+
+ return cmd_record(rec_argc, rec_argv, NULL);
}
static int
return 0;
}
+static int
+parse_highlight(const struct option *opt __maybe_unused, const char *arg,
+ int __maybe_unused unset)
+{
+ unsigned long duration = strtoul(arg, NULL, 0);
+
+ if (svg_highlight || svg_highlight_name)
+ return -1;
+
+ if (duration)
+ svg_highlight = duration;
+ else
+ svg_highlight_name = strdup(arg);
+
+ return 0;
+}
+
int cmd_timechart(int argc, const char **argv,
const char *prefix __maybe_unused)
{
+ struct timechart tchart = {
+ .tool = {
+ .comm = process_comm_event,
+ .fork = process_fork_event,
+ .exit = process_exit_event,
+ .sample = process_sample_event,
+ .ordered_samples = true,
+ },
+ .proc_num = 15,
+ };
const char *output_name = "output.svg";
- const struct option options[] = {
+ const struct option timechart_options[] = {
OPT_STRING('i', "input", &input_name, "file", "input file name"),
OPT_STRING('o', "output", &output_name, "file", "output file name"),
OPT_INTEGER('w', "width", &svg_page_width, "page width"),
- OPT_BOOLEAN('P', "power-only", &power_only, "output power data only"),
+ OPT_CALLBACK(0, "highlight", NULL, "duration or task name",
+ "highlight tasks. Pass duration in ns or process name.",
+ parse_highlight),
+ OPT_BOOLEAN('P', "power-only", &tchart.power_only, "output power data only"),
+ OPT_BOOLEAN('T', "tasks-only", &tchart.tasks_only,
+ "output processes data only"),
OPT_CALLBACK('p', "process", NULL, "process",
"process selector. Pass a pid or process name.",
parse_process),
OPT_STRING(0, "symfs", &symbol_conf.symfs, "directory",
"Look for files with symbols relative to this directory"),
+ OPT_INTEGER('n', "proc-num", &tchart.proc_num,
+ "min. number of tasks to print"),
+ OPT_BOOLEAN('t', "topology", &tchart.topology,
+ "sort CPUs according to topology"),
OPT_END()
};
const char * const timechart_usage[] = {
NULL
};
- argc = parse_options(argc, argv, options, timechart_usage,
+ const struct option record_options[] = {
+ OPT_BOOLEAN('P', "power-only", &tchart.power_only, "output power data only"),
+ OPT_BOOLEAN('T', "tasks-only", &tchart.tasks_only,
+ "output processes data only"),
+ OPT_BOOLEAN('g', "callchain", &tchart.with_backtrace, "record callchain"),
+ OPT_END()
+ };
+ const char * const record_usage[] = {
+ "perf timechart record [<options>]",
+ NULL
+ };
+ argc = parse_options(argc, argv, timechart_options, timechart_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
+ if (tchart.power_only && tchart.tasks_only) {
+ pr_err("-P and -T options cannot be used at the same time.\n");
+ return -1;
+ }
+
symbol__init();
- if (argc && !strncmp(argv[0], "rec", 3))
- return __cmd_record(argc, argv);
- else if (argc)
- usage_with_options(timechart_usage, options);
+ if (argc && !strncmp(argv[0], "rec", 3)) {
+ argc = parse_options(argc, argv, record_options, record_usage,
+ PARSE_OPT_STOP_AT_NON_OPTION);
+
+ if (tchart.power_only && tchart.tasks_only) {
+ pr_err("-P and -T options cannot be used at the same time.\n");
+ return -1;
+ }
+
+ return timechart__record(&tchart, argc, argv);
+ } else if (argc)
+ usage_with_options(timechart_usage, timechart_options);
setup_pager();
- return __cmd_timechart(output_name);
+ return __cmd_timechart(&tchart, output_name);
}
if (pthread_mutex_trylock(¬es->lock))
return;
- if (notes->src == NULL && symbol__alloc_hist(sym) < 0) {
- pthread_mutex_unlock(¬es->lock);
- pr_err("Not enough memory for annotating '%s' symbol!\n",
- sym->name);
- sleep(1);
- return;
- }
-
ip = he->ms.map->map_ip(he->ms.map, ip);
- err = symbol__inc_addr_samples(sym, he->ms.map, counter, ip);
+ err = hist_entry__inc_addr_samples(he, counter, ip);
pthread_mutex_unlock(¬es->lock);
if (err == -ERANGE && !he->ms.map->erange_warned)
ui__warn_map_erange(he->ms.map, sym, ip);
+ else if (err == -ENOMEM) {
+ pr_err("Not enough memory for annotating '%s' symbol!\n",
+ sym->name);
+ sleep(1);
+ }
}
static void perf_top__show_details(struct perf_top *top)
fprintf(stderr, "\nAvailable events:");
- list_for_each_entry(top->sym_evsel, &top->evlist->entries, node)
+ evlist__for_each(top->evlist, top->sym_evsel)
fprintf(stderr, "\n\t%d %s", top->sym_evsel->idx, perf_evsel__name(top->sym_evsel));
prompt_integer(&counter, "Enter details event counter");
sleep(1);
break;
}
- list_for_each_entry(top->sym_evsel, &top->evlist->entries, node)
+ evlist__for_each(top->evlist, top->sym_evsel)
if (top->sym_evsel->idx == counter)
break;
} else
* Zooming in/out UIDs. For now juse use whatever the user passed
* via --uid.
*/
- list_for_each_entry(pos, &top->evlist->entries, node)
+ evlist__for_each(top->evlist, pos)
pos->hists.uid_filter_str = top->record_opts.target.uid_str;
perf_evlist__tui_browse_hists(top->evlist, help, &hbt, top->min_percent,
return NULL;
}
-/* Tag samples to be skipped. */
-static const char *skip_symbols[] = {
- "intel_idle",
- "default_idle",
- "native_safe_halt",
- "cpu_idle",
- "enter_idle",
- "exit_idle",
- "mwait_idle",
- "mwait_idle_with_hints",
- "poll_idle",
- "ppc64_runlatch_off",
- "pseries_dedicated_idle_sleep",
- NULL
-};
-
static int symbol_filter(struct map *map __maybe_unused, struct symbol *sym)
{
const char *name = sym->name;
- int i;
/*
* ppc64 uses function descriptors and appends a '.' to the
strstr(name, "_text_end"))
return 1;
- for (i = 0; skip_symbols[i]; i++) {
- if (!strcmp(skip_symbols[i], name)) {
- sym->ignore = true;
- break;
- }
- }
+ if (symbol__is_idle(sym))
+ sym->ignore = true;
return 0;
}
if (al.sym == NULL || !al.sym->ignore) {
struct hist_entry *he;
- if ((sort__has_parent || symbol_conf.use_callchain) &&
- sample->callchain) {
- err = machine__resolve_callchain(machine, evsel,
- al.thread, sample,
- &parent, &al,
- top->max_stack);
- if (err)
- return;
- }
+ err = sample__resolve_callchain(sample, &parent, evsel, &al,
+ top->max_stack);
+ if (err)
+ return;
he = perf_evsel__add_hist_entry(evsel, &al, sample);
if (he == NULL) {
return;
}
- if (symbol_conf.use_callchain) {
- err = callchain_append(he->callchain, &callchain_cursor,
- sample->period);
- if (err)
- return;
- }
+ err = hist_entry__append_callchain(he, sample);
+ if (err)
+ return;
if (sort__has_sym)
perf_top__record_precise_ip(top, he, evsel->idx, ip);
char msg[512];
struct perf_evsel *counter;
struct perf_evlist *evlist = top->evlist;
- struct perf_record_opts *opts = &top->record_opts;
+ struct record_opts *opts = &top->record_opts;
perf_evlist__config(evlist, opts);
- list_for_each_entry(counter, &evlist->entries, node) {
+ evlist__for_each(evlist, counter) {
try_again:
if (perf_evsel__open(counter, top->evlist->cpus,
top->evlist->threads) < 0) {
static int __cmd_top(struct perf_top *top)
{
- struct perf_record_opts *opts = &top->record_opts;
+ struct record_opts *opts = &top->record_opts;
pthread_t thread;
int ret;
.max_stack = PERF_MAX_STACK_DEPTH,
.sym_pcnt_filter = 5,
};
- struct perf_record_opts *opts = &top.record_opts;
+ struct record_opts *opts = &top.record_opts;
struct target *target = &opts->target;
const struct option options[] = {
OPT_CALLBACK('e', "event", &top.evlist, "event",
"dump the symbol table used for profiling"),
OPT_INTEGER('f', "count-filter", &top.count_filter,
"only display functions with more events than this"),
- OPT_BOOLEAN('g', "group", &opts->group,
+ OPT_BOOLEAN(0, "group", &opts->group,
"put the counters into a counter group"),
OPT_BOOLEAN('i', "no-inherit", &opts->no_inherit,
"child tasks do not inherit counters"),
" abort, in_tx, transaction"),
OPT_BOOLEAN('n', "show-nr-samples", &symbol_conf.show_nr_samples,
"Show a column with the number of samples"),
- OPT_CALLBACK_NOOPT('G', NULL, &top.record_opts,
+ OPT_CALLBACK_NOOPT('g', NULL, &top.record_opts,
NULL, "enables call-graph recording",
&callchain_opt),
OPT_CALLBACK(0, "call-graph", &top.record_opts,
if (!top.evlist->nr_entries &&
perf_evlist__add_default(top.evlist) < 0) {
ui__error("Not enough memory for event selector list\n");
- goto out_delete_maps;
+ goto out_delete_evlist;
}
symbol_conf.nr_events = top.evlist->nr_entries;
if (top.delay_secs < 1)
top.delay_secs = 1;
- if (perf_record_opts__config(opts)) {
+ if (record_opts__config(opts)) {
status = -EINVAL;
- goto out_delete_maps;
+ goto out_delete_evlist;
}
top.sym_evsel = perf_evlist__first(top.evlist);
status = __cmd_top(&top);
-out_delete_maps:
- perf_evlist__delete_maps(top.evlist);
out_delete_evlist:
perf_evlist__delete(top.evlist);
#include "util/intlist.h"
#include "util/thread_map.h"
#include "util/stat.h"
+#include "trace-event.h"
+#include "util/parse-events.h"
#include <libaudit.h>
#include <stdlib.h>
static void perf_evsel__delete_priv(struct perf_evsel *evsel)
{
- free(evsel->priv);
- evsel->priv = NULL;
+ zfree(&evsel->priv);
perf_evsel__delete(evsel);
}
return -ENOMEM;
out_delete:
- free(evsel->priv);
- evsel->priv = NULL;
+ zfree(&evsel->priv);
return -ENOENT;
}
{
struct perf_evsel *evsel = perf_evsel__newtp("raw_syscalls", direction);
+ /* older kernel (e.g., RHEL6) use syscalls:{enter,exit} */
+ if (evsel == NULL)
+ evsel = perf_evsel__newtp("syscalls", direction);
+
if (evsel) {
if (perf_evsel__init_syscall_tp(evsel, handler))
goto out_delete;
int max;
struct syscall *table;
} syscalls;
- struct perf_record_opts opts;
+ struct record_opts opts;
struct machine *host;
u64 base_time;
- bool full_time;
FILE *output;
unsigned long nr_events;
struct strlist *ev_qualifier;
- bool not_ev_qualifier;
- bool live;
const char *last_vfs_getname;
struct intlist *tid_list;
struct intlist *pid_list;
+ double duration_filter;
+ double runtime_ms;
+ struct {
+ u64 vfs_getname,
+ proc_getname;
+ } stats;
+ bool not_ev_qualifier;
+ bool live;
+ bool full_time;
bool sched;
bool multiple_threads;
bool summary;
bool summary_only;
bool show_comm;
bool show_tool_stats;
- double duration_filter;
- double runtime_ms;
- struct {
- u64 vfs_getname, proc_getname;
- } stats;
};
static int trace__set_fd_pathname(struct thread *thread, int fd, const char *pathname)
size_t printed = syscall_arg__scnprintf_fd(bf, size, arg);
struct thread_trace *ttrace = arg->thread->priv;
- if (ttrace && fd >= 0 && fd <= ttrace->paths.max) {
- free(ttrace->paths.table[fd]);
- ttrace->paths.table[fd] = NULL;
- }
+ if (ttrace && fd >= 0 && fd <= ttrace->paths.max)
+ zfree(&ttrace->paths.table[fd]);
return printed;
}
sc->fmt = syscall_fmt__find(sc->name);
snprintf(tp_name, sizeof(tp_name), "sys_enter_%s", sc->name);
- sc->tp_format = event_format__new("syscalls", tp_name);
+ sc->tp_format = trace_event__tp_format("syscalls", tp_name);
if (sc->tp_format == NULL && sc->fmt && sc->fmt->alias) {
snprintf(tp_name, sizeof(tp_name), "sys_enter_%s", sc->fmt->alias);
- sc->tp_format = event_format__new("syscalls", tp_name);
+ sc->tp_format = trace_event__tp_format("syscalls", tp_name);
}
if (sc->tp_format == NULL)
if (!trace->full_time && trace->base_time == 0)
trace->base_time = sample->time;
- if (handler)
+ if (handler) {
+ ++trace->nr_events;
handler(trace, evsel, sample);
+ }
return err;
}
"-R",
"-m", "1024",
"-c", "1",
- "-e", "raw_syscalls:sys_enter,raw_syscalls:sys_exit",
+ "-e",
};
- rec_argc = ARRAY_SIZE(record_args) + argc;
+ /* +1 is for the event string below */
+ rec_argc = ARRAY_SIZE(record_args) + 1 + argc;
rec_argv = calloc(rec_argc + 1, sizeof(char *));
if (rec_argv == NULL)
for (i = 0; i < ARRAY_SIZE(record_args); i++)
rec_argv[i] = record_args[i];
+ /* event string may be different for older kernels - e.g., RHEL6 */
+ if (is_valid_tracepoint("raw_syscalls:sys_enter"))
+ rec_argv[i] = "raw_syscalls:sys_enter,raw_syscalls:sys_exit";
+ else if (is_valid_tracepoint("syscalls:sys_enter"))
+ rec_argv[i] = "syscalls:sys_enter,syscalls:sys_exit";
+ else {
+ pr_err("Neither raw_syscalls nor syscalls events exist.\n");
+ return -1;
+ }
+ i++;
+
for (j = 0; j < (unsigned int)argc; j++, i++)
rec_argv[i] = argv[j];
err = trace__symbols_init(trace, evlist);
if (err < 0) {
fprintf(trace->output, "Problems initializing symbol libraries!\n");
- goto out_delete_maps;
+ goto out_delete_evlist;
}
perf_evlist__config(evlist, &trace->opts);
if (forks) {
err = perf_evlist__prepare_workload(evlist, &trace->opts.target,
- argv, false, false);
+ argv, false, NULL);
if (err < 0) {
fprintf(trace->output, "Couldn't run the workload!\n");
- goto out_delete_maps;
+ goto out_delete_evlist;
}
}
if (err < 0)
goto out_error_open;
- err = perf_evlist__mmap(evlist, UINT_MAX, false);
+ err = perf_evlist__mmap(evlist, trace->opts.mmap_pages, false);
if (err < 0) {
fprintf(trace->output, "Couldn't mmap the events: %s\n", strerror(errno));
- goto out_close_evlist;
+ goto out_delete_evlist;
}
perf_evlist__enable(evlist);
}
}
- perf_evlist__munmap(evlist);
-out_close_evlist:
- perf_evlist__close(evlist);
-out_delete_maps:
- perf_evlist__delete_maps(evlist);
out_delete_evlist:
perf_evlist__delete(evlist);
out:
evsel = perf_evlist__find_tracepoint_by_name(session->evlist,
"raw_syscalls:sys_enter");
+ /* older kernels have syscalls tp versus raw_syscalls */
+ if (evsel == NULL)
+ evsel = perf_evlist__find_tracepoint_by_name(session->evlist,
+ "syscalls:sys_enter");
if (evsel == NULL) {
pr_err("Data file does not have raw_syscalls:sys_enter event\n");
goto out;
evsel = perf_evlist__find_tracepoint_by_name(session->evlist,
"raw_syscalls:sys_exit");
+ if (evsel == NULL)
+ evsel = perf_evlist__find_tracepoint_by_name(session->evlist,
+ "syscalls:sys_exit");
if (evsel == NULL) {
pr_err("Data file does not have raw_syscalls:sys_exit event\n");
goto out;
size_t printed = data->printed;
struct trace *trace = data->trace;
struct thread_trace *ttrace = thread->priv;
- const char *color;
double ratio;
if (ttrace == NULL)
ratio = (double)ttrace->nr_events / trace->nr_events * 100.0;
- color = PERF_COLOR_NORMAL;
- if (ratio > 50.0)
- color = PERF_COLOR_RED;
- else if (ratio > 25.0)
- color = PERF_COLOR_GREEN;
- else if (ratio > 5.0)
- color = PERF_COLOR_YELLOW;
-
- printed += color_fprintf(fp, color, " %s (%d), ", thread__comm_str(thread), thread->tid);
+ printed += fprintf(fp, " %s (%d), ", thread__comm_str(thread), thread->tid);
printed += fprintf(fp, "%lu events, ", ttrace->nr_events);
- printed += color_fprintf(fp, color, "%.1f%%", ratio);
+ printed += fprintf(fp, "%.1f%%", ratio);
printed += fprintf(fp, ", %.3f msec\n", ttrace->runtime_ms);
printed += thread__dump_stats(ttrace, trace, fp);
},
.user_freq = UINT_MAX,
.user_interval = ULLONG_MAX,
- .no_delay = true,
+ .no_buffering = true,
.mmap_pages = 1024,
},
.output = stdout,
-uname_M := $(shell uname -m 2>/dev/null || echo not)
-ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/i386/ -e s/sun4u/sparc64/ \
- -e s/arm.*/arm/ -e s/sa110/arm/ \
- -e s/s390x/s390/ -e s/parisc64/parisc/ \
- -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
- -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ )
-NO_PERF_REGS := 1
-CFLAGS := $(EXTRA_CFLAGS) $(EXTRA_WARNINGS)
+ifeq ($(src-perf),)
+src-perf := $(srctree)/tools/perf
+endif
-# Additional ARCH settings for x86
-ifeq ($(ARCH),i386)
- override ARCH := x86
- NO_PERF_REGS := 0
- LIBUNWIND_LIBS = -lunwind -lunwind-x86
+ifeq ($(obj-perf),)
+obj-perf := $(OUTPUT)
endif
-ifeq ($(ARCH),x86_64)
- override ARCH := x86
- IS_X86_64 := 0
- ifeq (, $(findstring m32,$(CFLAGS)))
- IS_X86_64 := $(shell echo __x86_64__ | ${CC} -E -x c - | tail -n 1)
- endif
+ifneq ($(obj-perf),)
+obj-perf := $(abspath $(obj-perf))/
+endif
+
+LIB_INCLUDE := $(srctree)/tools/lib/
+CFLAGS := $(EXTRA_CFLAGS) $(EXTRA_WARNINGS)
+
+include $(src-perf)/config/Makefile.arch
+
+NO_PERF_REGS := 1
+
+# Additional ARCH settings for x86
+ifeq ($(ARCH),x86)
ifeq (${IS_X86_64}, 1)
- RAW_ARCH := x86_64
CFLAGS += -DHAVE_ARCH_X86_64_SUPPORT
ARCH_INCLUDE = ../../arch/x86/lib/memcpy_64.S ../../arch/x86/lib/memset_64.S
LIBUNWIND_LIBS = -lunwind -lunwind-x86_64
LIBUNWIND_LIBS = -lunwind -lunwind-arm
endif
-ifeq ($(NO_PERF_REGS),0)
- CFLAGS += -DHAVE_PERF_REGS_SUPPORT
-endif
-
-ifeq ($(src-perf),)
-src-perf := $(srctree)/tools/perf
-endif
+ifeq ($(LIBUNWIND_LIBS),)
+ NO_LIBUNWIND := 1
+else
+ #
+ # For linking with debug library, run like:
+ #
+ # make DEBUG=1 LIBUNWIND_DIR=/opt/libunwind/
+ #
+ ifdef LIBUNWIND_DIR
+ LIBUNWIND_CFLAGS = -I$(LIBUNWIND_DIR)/include
+ LIBUNWIND_LDFLAGS = -L$(LIBUNWIND_DIR)/lib
+ endif
+ LIBUNWIND_LDFLAGS += $(LIBUNWIND_LIBS)
-ifeq ($(obj-perf),)
-obj-perf := $(OUTPUT)
+ # Set per-feature check compilation flags
+ FEATURE_CHECK_CFLAGS-libunwind = $(LIBUNWIND_CFLAGS)
+ FEATURE_CHECK_LDFLAGS-libunwind = $(LIBUNWIND_LDFLAGS)
+ FEATURE_CHECK_CFLAGS-libunwind-debug-frame = $(LIBUNWIND_CFLAGS)
+ FEATURE_CHECK_LDFLAGS-libunwind-debug-frame = $(LIBUNWIND_LDFLAGS)
endif
-ifneq ($(obj-perf),)
-obj-perf := $(abspath $(obj-perf))/
+ifeq ($(NO_PERF_REGS),0)
+ CFLAGS += -DHAVE_PERF_REGS_SUPPORT
endif
-LIB_INCLUDE := $(srctree)/tools/lib/
-
# include ARCH specific config
-include $(src-perf)/arch/$(ARCH)/Makefile
feature_check = $(eval $(feature_check_code))
define feature_check_code
- feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS)" LDFLAGS="$(LDFLAGS)" LIBUNWIND_LIBS="$(LIBUNWIND_LIBS)" -C config/feature-checks test-$1 >/dev/null 2>/dev/null && echo 1 || echo 0)
+ feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C config/feature-checks test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
endef
feature_set = $(eval $(feature_set_code))
libslang \
libunwind \
on-exit \
- stackprotector \
stackprotector-all \
timerfd
+# Set FEATURE_CHECK_(C|LD)FLAGS-all for all CORE_FEATURE_TESTS features.
+# If in the future we need per-feature checks/flags for features not
+# mentioned in this list we need to refactor this ;-).
+set_test_all_flags = $(eval $(set_test_all_flags_code))
+define set_test_all_flags_code
+ FEATURE_CHECK_CFLAGS-all += $(FEATURE_CHECK_CFLAGS-$(1))
+ FEATURE_CHECK_LDFLAGS-all += $(FEATURE_CHECK_LDFLAGS-$(1))
+endef
+
+$(foreach feat,$(CORE_FEATURE_TESTS),$(call set_test_all_flags,$(feat)))
+
#
# So here we detect whether test-all was rebuilt, to be able
# to skip the print-out of the long features list if the file
# existed before and after it was built:
#
-ifeq ($(wildcard $(OUTPUT)config/feature-checks/test-all),)
+ifeq ($(wildcard $(OUTPUT)config/feature-checks/test-all.bin),)
test-all-failed := 1
else
test-all-failed := 0
#
$(foreach feat,$(CORE_FEATURE_TESTS),$(call feature_set,$(feat)))
else
- $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS)" LDFLAGS=$(LDFLAGS) -i -j -C config/feature-checks $(CORE_FEATURE_TESTS) >/dev/null 2>&1)
+ $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS)" LDFLAGS=$(LDFLAGS) -i -j -C config/feature-checks $(addsuffix .bin,$(CORE_FEATURE_TESTS)) >/dev/null 2>&1)
$(foreach feat,$(CORE_FEATURE_TESTS),$(call feature_check,$(feat)))
endif
CFLAGS += -fstack-protector-all
endif
-ifeq ($(feature-stackprotector), 1)
- CFLAGS += -Wstack-protector
-endif
-
ifeq ($(DEBUG),0)
ifeq ($(feature-fortify-source), 1)
CFLAGS += -D_FORTIFY_SOURCE=2
CFLAGS += -I$(src-perf)/util/include
CFLAGS += -I$(src-perf)/arch/$(ARCH)/include
+CFLAGS += -I$(srctree)/tools/include/
CFLAGS += -I$(srctree)/arch/$(ARCH)/include/uapi
CFLAGS += -I$(srctree)/arch/$(ARCH)/include
CFLAGS += -I$(srctree)/include/uapi
endif # NO_DWARF
endif # NO_LIBELF
-ifeq ($(LIBUNWIND_LIBS),)
- NO_LIBUNWIND := 1
-endif
-
ifndef NO_LIBUNWIND
- #
- # For linking with debug library, run like:
- #
- # make DEBUG=1 LIBUNWIND_DIR=/opt/libunwind/
- #
- ifdef LIBUNWIND_DIR
- LIBUNWIND_CFLAGS := -I$(LIBUNWIND_DIR)/include
- LIBUNWIND_LDFLAGS := -L$(LIBUNWIND_DIR)/lib
- endif
-
ifneq ($(feature-libunwind), 1)
msg := $(warning No libunwind found, disabling post unwind support. Please install libunwind-dev[el] >= 1.1);
NO_LIBUNWIND := 1
# non-ARM has no dwarf_find_debug_frame() function:
CFLAGS += -DNO_LIBUNWIND_DEBUG_FRAME
endif
- endif
-endif
-ifndef NO_LIBUNWIND
- CFLAGS += -DHAVE_LIBUNWIND_SUPPORT
- EXTLIBS += $(LIBUNWIND_LIBS)
- CFLAGS += $(LIBUNWIND_CFLAGS)
- LDFLAGS += $(LIBUNWIND_LDFLAGS)
+ CFLAGS += -DHAVE_LIBUNWIND_SUPPORT
+ EXTLIBS += $(LIBUNWIND_LIBS)
+ CFLAGS += $(LIBUNWIND_CFLAGS)
+ LDFLAGS += $(LIBUNWIND_LDFLAGS)
+ endif # ifneq ($(feature-libunwind), 1)
endif
ifndef NO_LIBAUDIT
endif
ifndef NO_GTK2
- FLAGS_GTK2=$(CFLAGS) $(LDFLAGS) $(EXTLIBS) $(shell pkg-config --libs --cflags gtk+-2.0 2>/dev/null)
+ FLAGS_GTK2=$(CFLAGS) $(LDFLAGS) $(EXTLIBS) $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null)
ifneq ($(feature-gtk2), 1)
msg := $(warning GTK2 not found, disables GTK2 support. Please install gtk2-devel or libgtk2.0-dev);
NO_GTK2 := 1
GTK_CFLAGS := -DHAVE_GTK_INFO_BAR_SUPPORT
endif
CFLAGS += -DHAVE_GTK2_SUPPORT
- GTK_CFLAGS += $(shell pkg-config --cflags gtk+-2.0 2>/dev/null)
- GTK_LIBS := $(shell pkg-config --libs gtk+-2.0 2>/dev/null)
+ GTK_CFLAGS += $(shell $(PKG_CONFIG) --cflags gtk+-2.0 2>/dev/null)
+ GTK_LIBS := $(shell $(PKG_CONFIG) --libs gtk+-2.0 2>/dev/null)
EXTLIBS += -ldl
endif
endif
ifndef NO_LIBNUMA
ifeq ($(feature-libnuma), 0)
- msg := $(warning No numa.h found, disables 'perf bench numa mem' benchmark, please install numa-libs-devel or libnuma-dev);
+ msg := $(warning No numa.h found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev);
NO_LIBNUMA := 1
else
CFLAGS += -DHAVE_LIBNUMA_SUPPORT
perfexec_instdir = $(prefix)/$(perfexecdir)
endif
perfexec_instdir_SQ = $(subst ','\'',$(perfexec_instdir))
+
+# If we install to $(HOME) we keep the traceevent default:
+# $(HOME)/.traceevent/plugins
+# Otherwise we install plugins into the global $(libdir).
+ifdef DESTDIR
+plugindir=$(libdir)/traceevent/plugins
+plugindir_SQ= $(subst ','\'',$(prefix)/$(plugindir))
+endif
--- /dev/null
+
+uname_M := $(shell uname -m 2>/dev/null || echo not)
+
+ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/i386/ -e s/sun4u/sparc64/ \
+ -e s/arm.*/arm/ -e s/sa110/arm/ \
+ -e s/s390x/s390/ -e s/parisc64/parisc/ \
+ -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
+ -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ )
+
+# Additional ARCH settings for x86
+ifeq ($(ARCH),i386)
+ override ARCH := x86
+endif
+
+ifeq ($(ARCH),x86_64)
+ override ARCH := x86
+ IS_X86_64 := 0
+ ifeq (, $(findstring m32,$(CFLAGS)))
+ IS_X86_64 := $(shell echo __x86_64__ | ${CC} -E -x c - | tail -n 1)
+ RAW_ARCH := x86_64
+ endif
+endif
--- /dev/null
+*.d
+*.bin
FILES= \
- test-all \
- test-backtrace \
- test-bionic \
- test-dwarf \
- test-fortify-source \
- test-glibc \
- test-gtk2 \
- test-gtk2-infobar \
- test-hello \
- test-libaudit \
- test-libbfd \
- test-liberty \
- test-liberty-z \
- test-cplus-demangle \
- test-libelf \
- test-libelf-getphdrnum \
- test-libelf-mmap \
- test-libnuma \
- test-libperl \
- test-libpython \
- test-libpython-version \
- test-libslang \
- test-libunwind \
- test-libunwind-debug-frame \
- test-on-exit \
- test-stackprotector-all \
- test-stackprotector \
- test-timerfd
-
-CC := $(CC) -MD
+ test-all.bin \
+ test-backtrace.bin \
+ test-bionic.bin \
+ test-dwarf.bin \
+ test-fortify-source.bin \
+ test-glibc.bin \
+ test-gtk2.bin \
+ test-gtk2-infobar.bin \
+ test-hello.bin \
+ test-libaudit.bin \
+ test-libbfd.bin \
+ test-liberty.bin \
+ test-liberty-z.bin \
+ test-cplus-demangle.bin \
+ test-libelf.bin \
+ test-libelf-getphdrnum.bin \
+ test-libelf-mmap.bin \
+ test-libnuma.bin \
+ test-libperl.bin \
+ test-libpython.bin \
+ test-libpython-version.bin \
+ test-libslang.bin \
+ test-libunwind.bin \
+ test-libunwind-debug-frame.bin \
+ test-on-exit.bin \
+ test-stackprotector-all.bin \
+ test-timerfd.bin
+
+CC := $(CROSS_COMPILE)gcc -MD
+PKG_CONFIG := $(CROSS_COMPILE)pkg-config
all: $(FILES)
-BUILD = $(CC) $(CFLAGS) $(LDFLAGS) -o $(OUTPUT)$@ $@.c
+BUILD = $(CC) $(CFLAGS) -o $(OUTPUT)$@ $(patsubst %.bin,%.c,$@) $(LDFLAGS)
###############################
-test-all:
- $(BUILD) -Werror -fstack-protector -fstack-protector-all -O2 -Werror -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma $(LIBUNWIND_LIBS) -lelf -laudit -I/usr/include/slang -lslang $(shell pkg-config --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl
+test-all.bin:
+ $(BUILD) -Werror -fstack-protector-all -O2 -Werror -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -laudit -I/usr/include/slang -lslang $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl
-test-hello:
+test-hello.bin:
$(BUILD)
-test-stackprotector-all:
+test-stackprotector-all.bin:
$(BUILD) -Werror -fstack-protector-all
-test-stackprotector:
- $(BUILD) -Werror -fstack-protector -Wstack-protector
-
-test-fortify-source:
+test-fortify-source.bin:
$(BUILD) -O2 -Werror -D_FORTIFY_SOURCE=2
-test-bionic:
+test-bionic.bin:
$(BUILD)
-test-libelf:
+test-libelf.bin:
$(BUILD) -lelf
-test-glibc:
+test-glibc.bin:
$(BUILD)
-test-dwarf:
+test-dwarf.bin:
$(BUILD) -ldw
-test-libelf-mmap:
+test-libelf-mmap.bin:
$(BUILD) -lelf
-test-libelf-getphdrnum:
+test-libelf-getphdrnum.bin:
$(BUILD) -lelf
-test-libnuma:
+test-libnuma.bin:
$(BUILD) -lnuma
-test-libunwind:
- $(BUILD) $(LIBUNWIND_LIBS) -lelf
+test-libunwind.bin:
+ $(BUILD) -lelf
-test-libunwind-debug-frame:
- $(BUILD) $(LIBUNWIND_LIBS) -lelf
+test-libunwind-debug-frame.bin:
+ $(BUILD) -lelf
-test-libaudit:
+test-libaudit.bin:
$(BUILD) -laudit
-test-libslang:
+test-libslang.bin:
$(BUILD) -I/usr/include/slang -lslang
-test-gtk2:
- $(BUILD) $(shell pkg-config --libs --cflags gtk+-2.0 2>/dev/null)
+test-gtk2.bin:
+ $(BUILD) $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null)
-test-gtk2-infobar:
- $(BUILD) $(shell pkg-config --libs --cflags gtk+-2.0 2>/dev/null)
+test-gtk2-infobar.bin:
+ $(BUILD) $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null)
grep-libs = $(filter -l%,$(1))
strip-libs = $(filter-out -l%,$(1))
PERL_EMBED_CCOPTS = `perl -MExtUtils::Embed -e ccopts 2>/dev/null`
FLAGS_PERL_EMBED=$(PERL_EMBED_CCOPTS) $(PERL_EMBED_LDOPTS)
-test-libperl:
+test-libperl.bin:
$(BUILD) $(FLAGS_PERL_EMBED)
override PYTHON := python
PYTHON_EMBED_CCOPTS = $(shell $(PYTHON_CONFIG_SQ) --cflags 2>/dev/null)
FLAGS_PYTHON_EMBED = $(PYTHON_EMBED_CCOPTS) $(PYTHON_EMBED_LDOPTS)
-test-libpython:
+test-libpython.bin:
$(BUILD) $(FLAGS_PYTHON_EMBED)
-test-libpython-version:
+test-libpython-version.bin:
$(BUILD) $(FLAGS_PYTHON_EMBED)
-test-libbfd:
+test-libbfd.bin:
$(BUILD) -DPACKAGE='"perf"' -lbfd -ldl
-test-liberty:
+test-liberty.bin:
$(CC) -o $(OUTPUT)$@ test-libbfd.c -DPACKAGE='"perf"' -lbfd -ldl -liberty
-test-liberty-z:
+test-liberty-z.bin:
$(CC) -o $(OUTPUT)$@ test-libbfd.c -DPACKAGE='"perf"' -lbfd -ldl -liberty -lz
-test-cplus-demangle:
+test-cplus-demangle.bin:
$(BUILD) -liberty
-test-on-exit:
+test-on-exit.bin:
$(BUILD)
-test-backtrace:
+test-backtrace.bin:
$(BUILD)
-test-timerfd:
+test-timerfd.bin:
$(BUILD)
-include *.d
# include "test-timerfd.c"
#undef main
+#define main main_test_stackprotector_all
+# include "test-stackprotector-all.c"
+#undef main
+
int main(int argc, char *argv[])
{
main_test_libpython();
main_test_backtrace();
main_test_libnuma();
main_test_timerfd();
+ main_test_stackprotector_all();
return 0;
}
+++ /dev/null
-#include <stdio.h>
-
-int main(void)
-{
- return puts("hi");
-}
+++ /dev/null
-#include <stdio.h>
-
-int main(void)
-{
- return puts("hi");
-}
_ge_attempt = $(if $(get-executable),$(get-executable),$(_gea_warn)$(call _gea_err,$(2)))
_gea_warn = $(warning The path '$(1)' is not executable.)
_gea_err = $(if $(1),$(error Please set '$(1)' appropriately))
-
-ifneq ($(findstring $(MAKEFLAGS),s),s)
- ifneq ($(V),1)
- QUIET_CLEAN = @printf ' CLEAN %s\n' $1;
- QUIET_INSTALL = @printf ' INSTALL %s\n' $1;
- endif
-endif
--- /dev/null
+# perf bash and zsh completion
+
+# Taken from git.git's completion script.
+__my_reassemble_comp_words_by_ref()
+{
+ local exclude i j first
+ # Which word separators to exclude?
+ exclude="${1//[^$COMP_WORDBREAKS]}"
+ cword_=$COMP_CWORD
+ if [ -z "$exclude" ]; then
+ words_=("${COMP_WORDS[@]}")
+ return
+ fi
+ # List of word completion separators has shrunk;
+ # re-assemble words to complete.
+ for ((i=0, j=0; i < ${#COMP_WORDS[@]}; i++, j++)); do
+ # Append each nonempty word consisting of just
+ # word separator characters to the current word.
+ first=t
+ while
+ [ $i -gt 0 ] &&
+ [ -n "${COMP_WORDS[$i]}" ] &&
+ # word consists of excluded word separators
+ [ "${COMP_WORDS[$i]//[^$exclude]}" = "${COMP_WORDS[$i]}" ]
+ do
+ # Attach to the previous token,
+ # unless the previous token is the command name.
+ if [ $j -ge 2 ] && [ -n "$first" ]; then
+ ((j--))
+ fi
+ first=
+ words_[$j]=${words_[j]}${COMP_WORDS[i]}
+ if [ $i = $COMP_CWORD ]; then
+ cword_=$j
+ fi
+ if (($i < ${#COMP_WORDS[@]} - 1)); then
+ ((i++))
+ else
+ # Done.
+ return
+ fi
+ done
+ words_[$j]=${words_[j]}${COMP_WORDS[i]}
+ if [ $i = $COMP_CWORD ]; then
+ cword_=$j
+ fi
+ done
+}
+
+type _get_comp_words_by_ref &>/dev/null ||
+_get_comp_words_by_ref()
+{
+ local exclude cur_ words_ cword_
+ if [ "$1" = "-n" ]; then
+ exclude=$2
+ shift 2
+ fi
+ __my_reassemble_comp_words_by_ref "$exclude"
+ cur_=${words_[cword_]}
+ while [ $# -gt 0 ]; do
+ case "$1" in
+ cur)
+ cur=$cur_
+ ;;
+ prev)
+ prev=${words_[$cword_-1]}
+ ;;
+ words)
+ words=("${words_[@]}")
+ ;;
+ cword)
+ cword=$cword_
+ ;;
+ esac
+ shift
+ done
+}
+
+type __ltrim_colon_completions &>/dev/null ||
+__ltrim_colon_completions()
+{
+ if [[ "$1" == *:* && "$COMP_WORDBREAKS" == *:* ]]; then
+ # Remove colon-word prefix from COMPREPLY items
+ local colon_word=${1%"${1##*:}"}
+ local i=${#COMPREPLY[*]}
+ while [[ $((--i)) -ge 0 ]]; do
+ COMPREPLY[$i]=${COMPREPLY[$i]#"$colon_word"}
+ done
+ fi
+}
+
+__perfcomp ()
+{
+ COMPREPLY=( $( compgen -W "$1" -- "$2" ) )
+}
+
+__perfcomp_colon ()
+{
+ __perfcomp "$1" "$2"
+ __ltrim_colon_completions $cur
+}
+
+__perf_main ()
+{
+ local cmd
+
+ cmd=${words[0]}
+ COMPREPLY=()
+
+ # List perf subcommands or long options
+ if [ $cword -eq 1 ]; then
+ if [[ $cur == --* ]]; then
+ __perfcomp '--help --version \
+ --exec-path --html-path --paginate --no-pager \
+ --perf-dir --work-tree --debugfs-dir' -- "$cur"
+ else
+ cmds=$($cmd --list-cmds)
+ __perfcomp "$cmds" "$cur"
+ fi
+ # List possible events for -e option
+ elif [[ $prev == "-e" && "${words[1]}" == @(record|stat|top) ]]; then
+ evts=$($cmd list --raw-dump)
+ __perfcomp_colon "$evts" "$cur"
+ # List subcommands for 'perf kvm'
+ elif [[ $prev == "kvm" ]]; then
+ subcmds="top record report diff buildid-list stat"
+ __perfcomp_colon "$subcmds" "$cur"
+ # List long option names
+ elif [[ $cur == --* ]]; then
+ subcmd=${words[1]}
+ opts=$($cmd $subcmd --list-opts)
+ __perfcomp "$opts" "$cur"
+ fi
+}
+
+if [[ -n ${ZSH_VERSION-} ]]; then
+ autoload -U +X compinit && compinit
+
+ __perfcomp ()
+ {
+ emulate -L zsh
+
+ local c IFS=$' \t\n'
+ local -a array
+
+ for c in ${=1}; do
+ case $c in
+ --*=*|*.) ;;
+ *) c="$c " ;;
+ esac
+ array[${#array[@]}+1]="$c"
+ done
+
+ compset -P '*[=:]'
+ compadd -Q -S '' -a -- array && _ret=0
+ }
+
+ __perfcomp_colon ()
+ {
+ emulate -L zsh
+
+ local cur_="${2-$cur}"
+ local c IFS=$' \t\n'
+ local -a array
+
+ if [[ "$cur_" == *:* ]]; then
+ local colon_word=${cur_%"${cur_##*:}"}
+ fi
+
+ for c in ${=1}; do
+ case $c in
+ --*=*|*.) ;;
+ *) c="$c " ;;
+ esac
+ array[$#array+1]=${c#"$colon_word"}
+ done
+
+ compset -P '*[=:]'
+ compadd -Q -S '' -a -- array && _ret=0
+ }
+
+ _perf ()
+ {
+ local _ret=1 cur cword prev
+ cur=${words[CURRENT]}
+ prev=${words[CURRENT-1]}
+ let cword=CURRENT-1
+ emulate ksh -c __perf_main
+ let _ret && _default && _ret=0
+ return _ret
+ }
+
+ compdef _perf perf
+ return
+fi
+
+type perf &>/dev/null &&
+_perf()
+{
+ local cur words cword prev
+ _get_comp_words_by_ref -n =: cur words cword prev
+ __perf_main
+} &&
+
+complete -o bashdefault -o default -o nospace -F _perf perf 2>/dev/null \
+ || complete -o default -o nospace -F _perf perf
#include "util/quote.h"
#include "util/run-command.h"
#include "util/parse-events.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include <pthread.h>
const char perf_usage_string[] =
CALLCHAIN_DWARF
};
-struct perf_record_opts {
+struct record_opts {
struct target target;
int call_graph;
bool group;
bool inherit_stat;
- bool no_delay;
+ bool no_buffering;
bool no_inherit;
+ bool no_inherit_set;
bool no_samples;
bool raw_samples;
bool sample_address;
u64 user_interval;
u16 stack_dump_size;
bool sample_transaction;
+ unsigned initial_delay;
};
#endif
args = -i kill >/dev/null 2>&1
[event:base-record]
-sample_type=259
+sample_type=263
inherit=0
struct machines machines;
struct machine *machine;
struct thread *thread;
- struct perf_record_opts opts = {
+ struct record_opts opts = {
.mmap_pages = UINT_MAX,
.user_freq = UINT_MAX,
.user_interval = ULLONG_MAX,
err = TEST_CODE_READING_OK;
out_err:
if (evlist) {
- perf_evlist__munmap(evlist);
- perf_evlist__close(evlist);
perf_evlist__delete(evlist);
- }
- if (cpus)
+ } else {
cpu_map__delete(cpus);
- if (threads)
thread_map__delete(threads);
+ }
machines__destroy_kernel_maps(&machines);
machine__delete_threads(machine);
machines__exit(&machines);
}
err = 0;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (strcmp(perf_evsel__name(evsel), names[evsel->idx])) {
--err;
pr_debug("%s != %s\n", perf_evsel__name(evsel), names[evsel->idx]);
* However the second evsel also has a collapsed entry for
* "bash [libc] malloc" so total 9 entries will be in the tree.
*/
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
for (k = 0; k < ARRAY_SIZE(fake_common_samples); k++) {
const union perf_event event = {
.header = {
if (err < 0)
goto out;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
hists__collapse_resort(&evsel->hists, NULL);
if (verbose > 2)
*/
int test__keep_tracking(void)
{
- struct perf_record_opts opts = {
+ struct record_opts opts = {
.mmap_pages = UINT_MAX,
.user_freq = UINT_MAX,
.user_interval = ULLONG_MAX,
out_err:
if (evlist) {
perf_evlist__disable(evlist);
- perf_evlist__munmap(evlist);
- perf_evlist__close(evlist);
perf_evlist__delete(evlist);
- }
- if (cpus)
+ } else {
cpu_map__delete(cpus);
- if (threads)
thread_map__delete(threads);
+ }
return err;
}
PERF := .
MK := Makefile
+include config/Makefile.arch
+
+# FIXME looks like x86 is the only arch running tests ;-)
+# we need some IS_(32/64) flag to make this generic
+ifeq ($(IS_X86_64),1)
+lib = lib64
+else
+lib = lib
+endif
+
has = $(shell which $1 2>/dev/null)
# standard single make variable specified
test_make_perf_o := test -f $(PERF)/perf.o
test_make_util_map_o := test -f $(PERF)/util/map.o
-test_make_install := test -x $$TMP_DEST/bin/perf
-test_make_install_O := $(test_make_install)
-test_make_install_bin := $(test_make_install)
-test_make_install_bin_O := $(test_make_install)
+define test_dest_files
+ for file in $(1); do \
+ if [ ! -x $$TMP_DEST/$$file ]; then \
+ echo " failed to find: $$file"; \
+ fi \
+ done
+endef
+
+installed_files_bin := bin/perf
+installed_files_bin += etc/bash_completion.d/perf
+installed_files_bin += libexec/perf-core/perf-archive
+
+installed_files_plugins := $(lib)/traceevent/plugins/plugin_cfg80211.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_scsi.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_xen.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_function.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_sched_switch.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_mac80211.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_kvm.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_kmem.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_hrtimer.so
+installed_files_plugins += $(lib)/traceevent/plugins/plugin_jbd2.so
+
+installed_files_all := $(installed_files_bin)
+installed_files_all += $(installed_files_plugins)
+
+test_make_install := $(call test_dest_files,$(installed_files_all))
+test_make_install_O := $(call test_dest_files,$(installed_files_all))
+test_make_install_bin := $(call test_dest_files,$(installed_files_bin))
+test_make_install_bin_O := $(call test_dest_files,$(installed_files_bin))
# FIXME nothing gets installed
test_make_install_man := test -f $$TMP_DEST/share/man/man1/perf.1
cmd="cd $(PERF) && make -f $(MK) DESTDIR=$$TMP_DEST $($@)"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1; \
- echo " test: $(call test,$@)"; \
+ echo " test: $(call test,$@)" >> $@ 2>&1; \
$(call test,$@) && \
rm -f $@ \
rm -rf $$TMP_DEST
cmd="cd $(PERF) && make -f $(MK) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1 && \
- echo " test: $(call test_O,$@)"; \
+ echo " test: $(call test_O,$@)" >> $@ 2>&1; \
$(call test_O,$@) && \
rm -f $@ && \
rm -rf $$TMP_O \
rm -rf $$TMP_DEST
-all: $(run) $(run_O)
+tarpkg:
+ @cmd="$(PERF)/tests/perf-targz-src-pkg $(PERF)"; \
+ echo "- $@: $$cmd" && echo $$cmd > $@ && \
+ ( eval $$cmd ) >> $@ 2>&1
+
+
+all: $(run) $(run_O) tarpkg
@echo OK
out: $(run_O)
@echo OK
-.PHONY: all $(run) $(run_O) clean
+.PHONY: all $(run) $(run_O) tarpkg clean
evsels[i] = perf_evsel__newtp("syscalls", name);
if (evsels[i] == NULL) {
pr_debug("perf_evsel__new\n");
- goto out_free_evlist;
+ goto out_delete_evlist;
}
evsels[i]->attr.wakeup_events = 1;
pr_debug("failed to open counter: %s, "
"tweak /proc/sys/kernel/perf_event_paranoid?\n",
strerror(errno));
- goto out_close_fd;
+ goto out_delete_evlist;
}
nr_events[i] = 0;
if (perf_evlist__mmap(evlist, 128, true) < 0) {
pr_debug("failed to mmap events: %d (%s)\n", errno,
strerror(errno));
- goto out_close_fd;
+ goto out_delete_evlist;
}
for (i = 0; i < nsyscalls; ++i)
if (event->header.type != PERF_RECORD_SAMPLE) {
pr_debug("unexpected %s event\n",
perf_event__name(event->header.type));
- goto out_munmap;
+ goto out_delete_evlist;
}
err = perf_evlist__parse_sample(evlist, event, &sample);
if (err) {
pr_err("Can't parse sample, err = %d\n", err);
- goto out_munmap;
+ goto out_delete_evlist;
}
err = -1;
if (evsel == NULL) {
pr_debug("event with id %" PRIu64
" doesn't map to an evsel\n", sample.id);
- goto out_munmap;
+ goto out_delete_evlist;
}
nr_events[evsel->idx]++;
perf_evlist__mmap_consume(evlist, 0);
}
err = 0;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (nr_events[evsel->idx] != expected_nr_events[evsel->idx]) {
pr_debug("expected %d %s events, got %d\n",
expected_nr_events[evsel->idx],
perf_evsel__name(evsel), nr_events[evsel->idx]);
err = -1;
- goto out_munmap;
+ goto out_delete_evlist;
}
}
-out_munmap:
- perf_evlist__munmap(evlist);
-out_close_fd:
- for (i = 0; i < nsyscalls; ++i)
- perf_evsel__close_fd(evsels[i], 1, threads->nr);
-out_free_evlist:
+out_delete_evlist:
perf_evlist__delete(evlist);
+ cpus = NULL;
+ threads = NULL;
out_free_cpus:
cpu_map__delete(cpus);
out_free_threads:
int test__syscall_open_tp_fields(void)
{
- struct perf_record_opts opts = {
+ struct record_opts opts = {
.target = {
.uid = UINT_MAX,
.uses_mmap = true,
},
- .no_delay = true,
- .freq = 1,
- .mmap_pages = 256,
- .raw_samples = true,
+ .no_buffering = true,
+ .freq = 1,
+ .mmap_pages = 256,
+ .raw_samples = true,
};
const char *filename = "/etc/passwd";
int flags = O_RDONLY | O_DIRECTORY;
err = perf_evlist__open(evlist);
if (err < 0) {
pr_debug("perf_evlist__open: %s\n", strerror(errno));
- goto out_delete_maps;
+ goto out_delete_evlist;
}
err = perf_evlist__mmap(evlist, UINT_MAX, false);
if (err < 0) {
pr_debug("perf_evlist__mmap: %s\n", strerror(errno));
- goto out_close_evlist;
+ goto out_delete_evlist;
}
perf_evlist__enable(evlist);
err = perf_evsel__parse_sample(evsel, event, &sample);
if (err) {
pr_err("Can't parse sample, err = %d\n", err);
- goto out_munmap;
+ goto out_delete_evlist;
}
tp_flags = perf_evsel__intval(evsel, &sample, "flags");
if (flags != tp_flags) {
pr_debug("%s: Expected flags=%#x, got %#x\n",
__func__, flags, tp_flags);
- goto out_munmap;
+ goto out_delete_evlist;
}
goto out_ok;
if (++nr_polls > 5) {
pr_debug("%s: no events!\n", __func__);
- goto out_munmap;
+ goto out_delete_evlist;
}
}
out_ok:
err = 0;
-out_munmap:
- perf_evlist__munmap(evlist);
-out_close_evlist:
- perf_evlist__close(evlist);
-out_delete_maps:
- perf_evlist__delete_maps(evlist);
out_delete_evlist:
perf_evlist__delete(evlist);
out:
#include "evsel.h"
#include "evlist.h"
#include "fs.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include "tests.h"
#include <linux/hw_breakpoint.h>
TEST_ASSERT_VAL("wrong number of entries", evlist->nr_entries > 1);
TEST_ASSERT_VAL("wrong number of groups", 0 == evlist->nr_groups);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
TEST_ASSERT_VAL("wrong type",
PERF_TYPE_TRACEPOINT == evsel->attr.type);
TEST_ASSERT_VAL("wrong sample_type",
TEST_ASSERT_VAL("wrong number of entries", evlist->nr_entries > 1);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
TEST_ASSERT_VAL("wrong exclude_user",
!evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel",
if (ret) {
pr_debug("failed to parse event '%s', err %d\n",
e->name, ret);
- return ret;
+ } else {
+ ret = e->check(evlist);
}
-
- ret = e->check(evlist);
+
perf_evlist__delete(evlist);
return ret;
int test__PERF_RECORD(void)
{
- struct perf_record_opts opts = {
+ struct record_opts opts = {
.target = {
.uid = UINT_MAX,
.uses_mmap = true,
},
- .no_delay = true,
- .freq = 10,
- .mmap_pages = 256,
+ .no_buffering = true,
+ .freq = 10,
+ .mmap_pages = 256,
};
cpu_set_t cpu_mask;
size_t cpu_mask_size = sizeof(cpu_mask);
* so that we have time to open the evlist (calling sys_perf_event_open
* on all the fds) and then mmap them.
*/
- err = perf_evlist__prepare_workload(evlist, &opts.target, argv,
- false, false);
+ err = perf_evlist__prepare_workload(evlist, &opts.target, argv, false, NULL);
if (err < 0) {
pr_debug("Couldn't run the workload!\n");
- goto out_delete_maps;
+ goto out_delete_evlist;
}
/*
err = sched__get_first_possible_cpu(evlist->workload.pid, &cpu_mask);
if (err < 0) {
pr_debug("sched__get_first_possible_cpu: %s\n", strerror(errno));
- goto out_delete_maps;
+ goto out_delete_evlist;
}
cpu = err;
*/
if (sched_setaffinity(evlist->workload.pid, cpu_mask_size, &cpu_mask) < 0) {
pr_debug("sched_setaffinity: %s\n", strerror(errno));
- goto out_delete_maps;
+ goto out_delete_evlist;
}
/*
err = perf_evlist__open(evlist);
if (err < 0) {
pr_debug("perf_evlist__open: %s\n", strerror(errno));
- goto out_delete_maps;
+ goto out_delete_evlist;
}
/*
err = perf_evlist__mmap(evlist, opts.mmap_pages, false);
if (err < 0) {
pr_debug("perf_evlist__mmap: %s\n", strerror(errno));
- goto out_close_evlist;
+ goto out_delete_evlist;
}
/*
if (verbose)
perf_event__fprintf(event, stderr);
pr_debug("Couldn't parse sample\n");
- goto out_err;
+ goto out_delete_evlist;
}
if (verbose) {
pr_debug("PERF_RECORD_MMAP for %s missing!\n", "[vdso]");
++errs;
}
-out_err:
- perf_evlist__munmap(evlist);
-out_close_evlist:
- perf_evlist__close(evlist);
-out_delete_maps:
- perf_evlist__delete_maps(evlist);
out_delete_evlist:
perf_evlist__delete(evlist);
out:
--- /dev/null
+#!/bin/sh
+# Test one of the main kernel Makefile targets to generate a perf sources tarball
+# suitable for build outside the full kernel sources.
+#
+# This is to test that the tools/perf/MANIFEST file lists all the files needed to
+# be in such tarball, which sometimes gets broken when we move files around,
+# like when we made some files that were in tools/perf/ available to other tools/
+# codebases by moving it to tools/include/, etc.
+
+PERF=$1
+cd ${PERF}/../..
+make perf-targz-src-pkg > /dev/null
+TARBALL=$(ls -rt perf-*.tar.gz)
+TMP_DEST=$(mktemp -d)
+tar xf ${TARBALL} -C $TMP_DEST
+rm -f ${TARBALL}
+cd - > /dev/null
+make -C $TMP_DEST/perf*/tools/perf > /dev/null 2>&1
+RC=$?
+rm -rf ${TMP_DEST}
+exit $RC
*/
int test__perf_time_to_tsc(void)
{
- struct perf_record_opts opts = {
+ struct record_opts opts = {
.mmap_pages = UINT_MAX,
.user_freq = UINT_MAX,
.user_interval = ULLONG_MAX,
out_err:
if (evlist) {
perf_evlist__disable(evlist);
- perf_evlist__munmap(evlist);
- perf_evlist__close(evlist);
perf_evlist__delete(evlist);
}
- if (cpus)
- cpu_map__delete(cpus);
- if (threads)
- thread_map__delete(threads);
return err;
}
evsel = perf_evsel__new(&attr);
if (evsel == NULL) {
pr_debug("perf_evsel__new\n");
- goto out_free_evlist;
+ goto out_delete_evlist;
}
perf_evlist__add(evlist, evsel);
if (!evlist->cpus || !evlist->threads) {
err = -ENOMEM;
pr_debug("Not enough memory to create thread/cpu maps\n");
- goto out_delete_maps;
+ goto out_delete_evlist;
}
if (perf_evlist__open(evlist)) {
err = -errno;
pr_debug("Couldn't open evlist: %s\nHint: check %s, using %" PRIu64 " in this test.\n",
strerror(errno), knob, (u64)attr.sample_freq);
- goto out_delete_maps;
+ goto out_delete_evlist;
}
err = perf_evlist__mmap(evlist, 128, true);
if (err < 0) {
pr_debug("failed to mmap event: %d (%s)\n", errno,
strerror(errno));
- goto out_close_evlist;
+ goto out_delete_evlist;
}
perf_evlist__enable(evlist);
err = perf_evlist__parse_sample(evlist, event, &sample);
if (err < 0) {
pr_debug("Error during parse sample\n");
- goto out_unmap_evlist;
+ goto out_delete_evlist;
}
total_periods += sample.period;
err = -1;
}
-out_unmap_evlist:
- perf_evlist__munmap(evlist);
-out_close_evlist:
- perf_evlist__close(evlist);
-out_delete_maps:
- perf_evlist__delete_maps(evlist);
-out_free_evlist:
+out_delete_evlist:
perf_evlist__delete(evlist);
return err;
}
static int exited;
static int nr_exit;
-static void sig_handler(int sig)
+static void sig_handler(int sig __maybe_unused)
{
exited = 1;
+}
- if (sig == SIGUSR1)
- nr_exit = -1;
+/*
+ * perf_evlist__prepare_workload will send a SIGUSR1 if the fork fails, since
+ * we asked by setting its exec_error to this handler.
+ */
+static void workload_exec_failed_signal(int signo __maybe_unused,
+ siginfo_t *info __maybe_unused,
+ void *ucontext __maybe_unused)
+{
+ exited = 1;
+ nr_exit = -1;
}
/*
const char *argv[] = { "true", NULL };
signal(SIGCHLD, sig_handler);
- signal(SIGUSR1, sig_handler);
evlist = perf_evlist__new_default();
if (evlist == NULL) {
if (!evlist->cpus || !evlist->threads) {
err = -ENOMEM;
pr_debug("Not enough memory to create thread/cpu maps\n");
- goto out_delete_maps;
+ goto out_delete_evlist;
}
- err = perf_evlist__prepare_workload(evlist, &target, argv, false, true);
+ err = perf_evlist__prepare_workload(evlist, &target, argv, false,
+ workload_exec_failed_signal);
if (err < 0) {
pr_debug("Couldn't run the workload!\n");
- goto out_delete_maps;
+ goto out_delete_evlist;
}
evsel = perf_evlist__first(evlist);
err = perf_evlist__open(evlist);
if (err < 0) {
pr_debug("Couldn't open the evlist: %s\n", strerror(-err));
- goto out_delete_maps;
+ goto out_delete_evlist;
}
if (perf_evlist__mmap(evlist, 128, true) < 0) {
pr_debug("failed to mmap events: %d (%s)\n", errno,
strerror(errno));
- goto out_close_evlist;
+ goto out_delete_evlist;
}
perf_evlist__start_workload(evlist);
err = -1;
}
- perf_evlist__munmap(evlist);
-out_close_evlist:
- perf_evlist__close(evlist);
-out_delete_maps:
- perf_evlist__delete_maps(evlist);
+out_delete_evlist:
perf_evlist__delete(evlist);
return err;
}
__ui_browser__show_title(browser, title);
browser->title = title;
- free(browser->helpline);
- browser->helpline = NULL;
+ zfree(&browser->helpline);
va_start(ap, helpline);
err = vasprintf(&browser->helpline, helpline, ap);
return err ? 0 : -1;
}
-void ui_browser__hide(struct ui_browser *browser __maybe_unused)
+void ui_browser__hide(struct ui_browser *browser)
{
pthread_mutex_lock(&ui__lock);
ui_helpline__pop();
- free(browser->helpline);
- browser->helpline = NULL;
+ zfree(&browser->helpline);
pthread_mutex_unlock(&ui__lock);
}
bool ui_browser__dialog_yesno(struct ui_browser *browser, const char *text);
int ui_browser__input_window(const char *title, const char *text, char *input,
const char *exit_msg, int delay_sec);
+struct perf_session_env;
+int tui__header_window(struct perf_session_env *env);
void ui_browser__argv_seek(struct ui_browser *browser, off_t offset, int whence);
unsigned int ui_browser__argv_refresh(struct ui_browser *browser);
--- /dev/null
+#include "util/cache.h"
+#include "util/debug.h"
+#include "ui/browser.h"
+#include "ui/ui.h"
+#include "ui/util.h"
+#include "ui/libslang.h"
+#include "util/header.h"
+#include "util/session.h"
+
+static void ui_browser__argv_write(struct ui_browser *browser,
+ void *entry, int row)
+{
+ char **arg = entry;
+ char *str = *arg;
+ char empty[] = " ";
+ bool current_entry = ui_browser__is_current_entry(browser, row);
+ unsigned long offset = (unsigned long)browser->priv;
+
+ if (offset >= strlen(str))
+ str = empty;
+ else
+ str = str + offset;
+
+ ui_browser__set_color(browser, current_entry ? HE_COLORSET_SELECTED :
+ HE_COLORSET_NORMAL);
+
+ slsmg_write_nstring(str, browser->width);
+}
+
+static int list_menu__run(struct ui_browser *menu)
+{
+ int key;
+ unsigned long offset;
+ const char help[] =
+ "h/?/F1 Show this window\n"
+ "UP/DOWN/PGUP\n"
+ "PGDN/SPACE\n"
+ "LEFT/RIGHT Navigate\n"
+ "q/ESC/CTRL+C Exit browser";
+
+ if (ui_browser__show(menu, "Header information", "Press 'q' to exit") < 0)
+ return -1;
+
+ while (1) {
+ key = ui_browser__run(menu, 0);
+
+ switch (key) {
+ case K_RIGHT:
+ offset = (unsigned long)menu->priv;
+ offset += 10;
+ menu->priv = (void *)offset;
+ continue;
+ case K_LEFT:
+ offset = (unsigned long)menu->priv;
+ if (offset >= 10)
+ offset -= 10;
+ menu->priv = (void *)offset;
+ continue;
+ case K_F1:
+ case 'h':
+ case '?':
+ ui_browser__help_window(menu, help);
+ continue;
+ case K_ESC:
+ case 'q':
+ case CTRL('c'):
+ key = -1;
+ break;
+ default:
+ continue;
+ }
+
+ break;
+ }
+
+ ui_browser__hide(menu);
+ return key;
+}
+
+static int ui__list_menu(int argc, char * const argv[])
+{
+ struct ui_browser menu = {
+ .entries = (void *)argv,
+ .refresh = ui_browser__argv_refresh,
+ .seek = ui_browser__argv_seek,
+ .write = ui_browser__argv_write,
+ .nr_entries = argc,
+ };
+
+ return list_menu__run(&menu);
+}
+
+int tui__header_window(struct perf_session_env *env)
+{
+ int i, argc = 0;
+ char **argv;
+ struct perf_session *session;
+ char *ptr, *pos;
+ size_t size;
+ FILE *fp = open_memstream(&ptr, &size);
+
+ session = container_of(env, struct perf_session, header.env);
+ perf_header__fprintf_info(session, fp, true);
+ fclose(fp);
+
+ for (pos = ptr, argc = 0; (pos = strchr(pos, '\n')) != NULL; pos++)
+ argc++;
+
+ argv = calloc(argc + 1, sizeof(*argv));
+ if (argv == NULL)
+ goto out;
+
+ argv[0] = pos = ptr;
+ for (i = 1; (pos = strchr(pos, '\n')) != NULL; i++) {
+ *pos++ = '\0';
+ argv[i] = pos;
+ }
+
+ BUG_ON(i != argc + 1);
+
+ ui__list_menu(argc, argv);
+
+out:
+ free(argv);
+ free(ptr);
+ return 0;
+}
{
int i;
- for (i = 0; i < n; ++i) {
- free(options[i]);
- options[i] = NULL;
- }
+ for (i = 0; i < n; ++i)
+ zfree(&options[i]);
}
/* Check whether the browser is for 'top' or 'report' */
abs_path[nr_options] = strdup(path);
if (!abs_path[nr_options]) {
- free(options[nr_options]);
+ zfree(&options[nr_options]);
ui__warning("Can't search all data files due to memory shortage.\n");
fclose(file);
break;
char script_opt[64];
int delay_secs = hbt ? hbt->refresh : 0;
+#define HIST_BROWSER_HELP_COMMON \
+ "h/?/F1 Show this window\n" \
+ "UP/DOWN/PGUP\n" \
+ "PGDN/SPACE Navigate\n" \
+ "q/ESC/CTRL+C Exit browser\n\n" \
+ "For multiple event sessions:\n\n" \
+ "TAB/UNTAB Switch events\n\n" \
+ "For symbolic views (--sort has sym):\n\n" \
+ "-> Zoom into DSO/Threads & Annotate current symbol\n" \
+ "<- Zoom out\n" \
+ "a Annotate current symbol\n" \
+ "C Collapse all callchains\n" \
+ "d Zoom into current DSO\n" \
+ "E Expand all callchains\n" \
+
+ /* help messages are sorted by lexical order of the hotkey */
+ const char report_help[] = HIST_BROWSER_HELP_COMMON
+ "i Show header information\n"
+ "P Print histograms to perf.hist.N\n"
+ "r Run available scripts\n"
+ "s Switch to another data file in PWD\n"
+ "t Zoom into current Thread\n"
+ "V Verbose (DSO names in callchains, etc)\n"
+ "/ Filter symbol by name";
+ const char top_help[] = HIST_BROWSER_HELP_COMMON
+ "P Print histograms to perf.hist.N\n"
+ "t Zoom into current Thread\n"
+ "V Verbose (DSO names in callchains, etc)\n"
+ "/ Filter symbol by name";
+
if (browser == NULL)
return -1;
if (is_report_browser(hbt))
goto do_data_switch;
continue;
+ case 'i':
+ /* env->arch is NULL for live-mode (i.e. perf top) */
+ if (env->arch)
+ tui__header_window(env);
+ continue;
case K_F1:
case 'h':
case '?':
ui_browser__help_window(&browser->b,
- "h/?/F1 Show this window\n"
- "UP/DOWN/PGUP\n"
- "PGDN/SPACE Navigate\n"
- "q/ESC/CTRL+C Exit browser\n\n"
- "For multiple event sessions:\n\n"
- "TAB/UNTAB Switch events\n\n"
- "For symbolic views (--sort has sym):\n\n"
- "-> Zoom into DSO/Threads & Annotate current symbol\n"
- "<- Zoom out\n"
- "a Annotate current symbol\n"
- "C Collapse all callchains\n"
- "E Expand all callchains\n"
- "d Zoom into current DSO\n"
- "t Zoom into current Thread\n"
- "r Run available scripts('perf report' only)\n"
- "s Switch to another data file in PWD ('perf report' only)\n"
- "P Print histograms to perf.hist.N\n"
- "V Verbose (DSO names in callchains, etc)\n"
- "/ Filter symbol by name");
+ is_report_browser(hbt) ? report_help : top_help);
continue;
case K_ENTER:
case K_RIGHT:
ui_helpline__push("Press ESC to exit");
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
const char *ev_name = perf_evsel__name(pos);
size_t line_len = strlen(ev_name) + 7;
struct perf_evsel *pos;
nr_entries = 0;
- list_for_each_entry(pos, &evlist->entries, node)
+ evlist__for_each(evlist, pos) {
if (perf_evsel__is_group_leader(pos))
nr_entries++;
+ }
if (nr_entries == 1)
goto single_entry;
if (script.b.width > AVERAGE_LINE_LEN)
script.b.width = AVERAGE_LINE_LEN;
- if (line)
- free(line);
+ free(line);
pclose(fp);
script.nr_lines = nr_entries;
gtk_container_add(GTK_CONTAINER(window), vbox);
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
struct hists *hists = &pos->hists;
const char *evname = perf_evsel__name(pos);
GtkWidget *scrolled_window;
if (!perf_gtk__is_active_context(*ctx))
return -1;
- free(*ctx);
- *ctx = NULL;
+ zfree(ctx);
return 0;
}
free(line);
out:
- free(rem_sq_bracket);
+ zfree(&rem_sq_bracket);
return ret;
}
t = sep + 1;
}
+ pthread_mutex_lock(&ui__lock);
+
max_len += 2;
nr_lines += 8;
y = SLtt_Screen_Rows / 2 - nr_lines / 2;
SLsmg_write_nstring((char *)exit_msg, max_len);
SLsmg_refresh();
+ pthread_mutex_unlock(&ui__lock);
+
x += 2;
len = 0;
key = ui__getch(delay_secs);
while (key != K_TIMER && key != K_ENTER && key != K_ESC) {
+ pthread_mutex_lock(&ui__lock);
+
if (key == K_BKSPC) {
- if (len == 0)
+ if (len == 0) {
+ pthread_mutex_unlock(&ui__lock);
goto next_key;
+ }
SLsmg_gotorc(y, x + --len);
SLsmg_write_char(' ');
} else {
}
SLsmg_refresh();
+ pthread_mutex_unlock(&ui__lock);
+
/* XXX more graceful overflow handling needed */
if (len == sizeof(buf) - 1) {
ui_helpline__push("maximum size of symbol name reached!");
t = sep + 1;
}
+ pthread_mutex_lock(&ui__lock);
+
max_len += 2;
nr_lines += 4;
y = SLtt_Screen_Rows / 2 - nr_lines / 2,
SLsmg_gotorc(y + nr_lines - 1, x);
SLsmg_write_nstring((char *)exit_msg, max_len);
SLsmg_refresh();
+
+ pthread_mutex_unlock(&ui__lock);
+
return ui__getch(delay_secs);
}
if (vasprintf(&s, format, args) > 0) {
int key;
- pthread_mutex_lock(&ui__lock);
key = ui__question_window(title, s, "Press any key...", 0);
- pthread_mutex_unlock(&ui__lock);
free(s);
return key;
}
src++;
c = cmdline[src];
if (!c) {
- free(*argv);
- *argv = NULL;
+ zfree(argv);
return error("cmdline ends with \\");
}
}
cmdline[dst] = 0;
if (quoted) {
- free(*argv);
- *argv = NULL;
+ zfree(argv);
return error("unclosed quote");
}
static void ins__delete(struct ins_operands *ops)
{
- free(ops->source.raw);
- free(ops->source.name);
- free(ops->target.raw);
- free(ops->target.name);
+ zfree(&ops->source.raw);
+ zfree(&ops->source.name);
+ zfree(&ops->target.raw);
+ zfree(&ops->target.name);
}
static int ins__raw_scnprintf(struct ins *ins, char *bf, size_t size,
return 0;
out_free_ops:
- free(ops->locked.ops);
- ops->locked.ops = NULL;
+ zfree(&ops->locked.ops);
return 0;
}
static void lock__delete(struct ins_operands *ops)
{
- free(ops->locked.ops);
- free(ops->target.raw);
- free(ops->target.name);
+ zfree(&ops->locked.ops);
+ zfree(&ops->target.raw);
+ zfree(&ops->target.name);
}
static struct ins_ops lock_ops = {
return 0;
out_free_source:
- free(ops->source.raw);
- ops->source.raw = NULL;
+ zfree(&ops->source.raw);
return -1;
}
pthread_mutex_unlock(¬es->lock);
}
-int symbol__inc_addr_samples(struct symbol *sym, struct map *map,
- int evidx, u64 addr)
+static int __symbol__inc_addr_samples(struct symbol *sym, struct map *map,
+ struct annotation *notes, int evidx, u64 addr)
{
unsigned offset;
- struct annotation *notes;
struct sym_hist *h;
- notes = symbol__annotation(sym);
- if (notes->src == NULL)
- return -ENOMEM;
-
pr_debug3("%s: addr=%#" PRIx64 "\n", __func__, map->unmap_ip(map, addr));
if (addr < sym->start || addr > sym->end)
return 0;
}
+static int symbol__inc_addr_samples(struct symbol *sym, struct map *map,
+ int evidx, u64 addr)
+{
+ struct annotation *notes;
+
+ if (sym == NULL || use_browser != 1 || !sort__has_sym)
+ return 0;
+
+ notes = symbol__annotation(sym);
+ if (notes->src == NULL) {
+ if (symbol__alloc_hist(sym) < 0)
+ return -ENOMEM;
+ }
+
+ return __symbol__inc_addr_samples(sym, map, notes, evidx, addr);
+}
+
+int addr_map_symbol__inc_samples(struct addr_map_symbol *ams, int evidx)
+{
+ return symbol__inc_addr_samples(ams->sym, ams->map, evidx, ams->al_addr);
+}
+
+int hist_entry__inc_addr_samples(struct hist_entry *he, int evidx, u64 ip)
+{
+ return symbol__inc_addr_samples(he->ms.sym, he->ms.map, evidx, ip);
+}
+
static void disasm_line__init_ins(struct disasm_line *dl)
{
dl->ins = ins__find(dl->name);
return 0;
out_free_name:
- free(*namep);
- *namep = NULL;
+ zfree(namep);
return -1;
}
return dl;
out_free_line:
- free(dl->line);
+ zfree(&dl->line);
out_delete:
free(dl);
return NULL;
void disasm_line__free(struct disasm_line *dl)
{
- free(dl->line);
- free(dl->name);
+ zfree(&dl->line);
+ zfree(&dl->name);
if (dl->ins && dl->ins->ops->free)
dl->ins->ops->free(&dl->ops);
else
* cache, or is just a kallsyms file, well, lets hope that this
* DSO is the same as when 'perf record' ran.
*/
- filename = dso->long_name;
+ filename = (char *)dso->long_name;
snprintf(symfs_filename, sizeof(symfs_filename), "%s%s",
symbol_conf.symfs, filename);
free_filename = false;
src_line = (void *)src_line + sizeof_src_line;
}
- free(notes->src->lines);
- notes->src->lines = NULL;
+ zfree(¬es->src->lines);
}
/* Get the filename:line for the colored entries */
return 0;
}
+
+int hist_entry__annotate(struct hist_entry *he, size_t privsize)
+{
+ return symbol__annotate(he->ms.sym, he->ms.map, privsize);
+}
return &a->annotation;
}
-int symbol__inc_addr_samples(struct symbol *sym, struct map *map,
- int evidx, u64 addr);
+int addr_map_symbol__inc_samples(struct addr_map_symbol *ams, int evidx);
+
+int hist_entry__inc_addr_samples(struct hist_entry *he, int evidx, u64 addr);
+
int symbol__alloc_hist(struct symbol *sym);
void symbol__annotate_zero_histograms(struct symbol *sym);
int symbol__annotate(struct symbol *sym, struct map *map, size_t privsize);
+
+int hist_entry__annotate(struct hist_entry *he, size_t privsize);
+
int symbol__annotate_init(struct map *map __maybe_unused, struct symbol *sym);
int symbol__annotate_printf(struct symbol *sym, struct map *map,
struct perf_evsel *evsel, bool full_paths,
return raw - build_id;
}
-char *dso__build_id_filename(struct dso *dso, char *bf, size_t size)
+char *dso__build_id_filename(const struct dso *dso, char *bf, size_t size)
{
char build_id_hex[BUILD_ID_SIZE * 2 + 1];
struct dso;
int build_id__sprintf(const u8 *build_id, int len, char *bf);
-char *dso__build_id_filename(struct dso *dso, char *bf, size_t size);
+char *dso__build_id_filename(const struct dso *dso, char *bf, size_t size);
int build_id__mark_dso_hit(struct perf_tool *tool, union perf_event *event,
struct perf_sample *sample, struct perf_evsel *evsel,
#include <errno.h>
#include <math.h>
+#include "asm/bug.h"
+
#include "hist.h"
#include "util.h"
+#include "sort.h"
+#include "machine.h"
#include "callchain.h"
__thread struct callchain_cursor callchain_cursor;
/* lookup in childrens */
while (*p) {
s64 ret;
- struct callchain_list *cnode;
parent = *p;
rnode = rb_entry(parent, struct callchain_node, rb_node_in);
- cnode = list_first_entry(&rnode->val, struct callchain_list,
- list);
- /* just check first entry */
- ret = match_chain(node, cnode);
- if (ret == 0) {
- append_chain(rnode, cursor, period);
+ /* If at least first entry matches, rely to children */
+ ret = append_chain(rnode, cursor, period);
+ if (ret == 0)
goto inc_children_hit;
- }
if (ret < 0)
p = &parent->rb_left;
struct callchain_cursor *cursor,
u64 period)
{
- struct callchain_cursor_node *curr_snap = cursor->curr;
struct callchain_list *cnode;
u64 start = cursor->pos;
bool found = false;
u64 matches;
+ int cmp = 0;
/*
* Lookup in the current node
if (!node)
break;
- if (match_chain(node, cnode) != 0)
+ cmp = match_chain(node, cnode);
+ if (cmp)
break;
found = true;
/* matches not, relay no the parent */
if (!found) {
- cursor->curr = curr_snap;
- cursor->pos = start;
- return -1;
+ WARN_ONCE(!cmp, "Chain comparison error\n");
+ return cmp;
}
matches = cursor->pos - start;
return 0;
}
+
+int sample__resolve_callchain(struct perf_sample *sample, struct symbol **parent,
+ struct perf_evsel *evsel, struct addr_location *al,
+ int max_stack)
+{
+ if (sample->callchain == NULL)
+ return 0;
+
+ if (symbol_conf.use_callchain || sort__has_parent) {
+ return machine__resolve_callchain(al->machine, evsel, al->thread,
+ sample, parent, al, max_stack);
+ }
+ return 0;
+}
+
+int hist_entry__append_callchain(struct hist_entry *he, struct perf_sample *sample)
+{
+ if (!symbol_conf.use_callchain)
+ return 0;
+ return callchain_append(he->callchain, &callchain_cursor, sample->period);
+}
}
struct option;
+struct hist_entry;
-int record_parse_callchain(const char *arg, struct perf_record_opts *opts);
+int record_parse_callchain(const char *arg, struct record_opts *opts);
int record_parse_callchain_opt(const struct option *opt, const char *arg, int unset);
int record_callchain_opt(const struct option *opt, const char *arg, int unset);
+int sample__resolve_callchain(struct perf_sample *sample, struct symbol **parent,
+ struct perf_evsel *evsel, struct addr_location *al,
+ int max_stack);
+int hist_entry__append_callchain(struct hist_entry *he, struct perf_sample *sample);
+
extern const char record_callchain_help[];
#endif /* __PERF_CALLCHAIN_H */
/*
* check if cgrp is already defined, if so we reuse it
*/
- list_for_each_entry(counter, &evlist->entries, node) {
+ evlist__for_each(evlist, counter) {
cgrp = counter->cgrp;
if (!cgrp)
continue;
* if add cgroup N, then need to find event N
*/
n = 0;
- list_for_each_entry(counter, &evlist->entries, node) {
+ evlist__for_each(evlist, counter) {
if (n == nr_cgroups)
goto found;
n++;
/* XXX: not reentrant */
if (--cgrp->refcnt == 0) {
close(cgrp->fd);
- free(cgrp->name);
+ zfree(&cgrp->name);
free(cgrp);
}
}
#include <linux/kernel.h>
#include "cache.h"
#include "color.h"
+#include <math.h>
int perf_use_color_default = -1;
* entries in green - and keep the low overhead places
* normal:
*/
- if (percent >= MIN_RED)
+ if (fabs(percent) >= MIN_RED)
color = PERF_COLOR_RED;
else {
- if (percent > MIN_GREEN)
+ if (fabs(percent) > MIN_GREEN)
color = PERF_COLOR_GREEN;
}
return color;
return r;
}
+int value_color_snprintf(char *bf, size_t size, const char *fmt, double value)
+{
+ const char *color = get_percent_color(value);
+ return color_snprintf(bf, size, color, fmt, value);
+}
+
int percent_color_snprintf(char *bf, size_t size, const char *fmt, ...)
{
va_list args;
double percent;
- const char *color;
va_start(args, fmt);
percent = va_arg(args, double);
va_end(args);
- color = get_percent_color(percent);
- return color_snprintf(bf, size, color, fmt, percent);
+ return value_color_snprintf(bf, size, fmt, percent);
}
int color_snprintf(char *bf, size_t size, const char *color, const char *fmt, ...);
int color_fprintf_ln(FILE *fp, const char *color, const char *fmt, ...);
int color_fwrite_lines(FILE *fp, const char *color, size_t count, const char *buf);
+int value_color_snprintf(char *bf, size_t size, const char *fmt, double value);
int percent_color_snprintf(char *bf, size_t size, const char *fmt, ...);
int percent_color_fprintf(FILE *fp, const char *fmt, double percent);
const char *get_percent_color(double percent);
{
if (!--cs->ref) {
rb_erase(&cs->rb_node, &comm_str_root);
- free(cs->str);
+ zfree(&cs->str);
free(cs);
}
}
return comm;
}
-void comm__override(struct comm *comm, const char *str, u64 timestamp)
+int comm__override(struct comm *comm, const char *str, u64 timestamp)
{
- struct comm_str *old = comm->comm_str;
+ struct comm_str *new, *old = comm->comm_str;
- comm->comm_str = comm_str__findnew(str, &comm_str_root);
- if (!comm->comm_str) {
- comm->comm_str = old;
- return;
- }
+ new = comm_str__findnew(str, &comm_str_root);
+ if (!new)
+ return -ENOMEM;
- comm->start = timestamp;
- comm_str__get(comm->comm_str);
+ comm_str__get(new);
comm_str__put(old);
+ comm->comm_str = new;
+ comm->start = timestamp;
+
+ return 0;
}
void comm__free(struct comm *comm)
void comm__free(struct comm *comm);
struct comm *comm__new(const char *str, u64 timestamp);
const char *comm__str(const struct comm *comm);
-void comm__override(struct comm *comm, const char *str, u64 timestamp);
+int comm__override(struct comm *comm, const char *str, u64 timestamp);
#endif /* __PERF_COMM_H */
{
close(file->fd);
}
+
+ssize_t perf_data_file__write(struct perf_data_file *file,
+ void *buf, size_t size)
+{
+ return writen(file->fd, buf, size);
+}
};
struct perf_data_file {
- const char *path;
- int fd;
- bool is_pipe;
- bool force;
- unsigned long size;
- enum perf_data_mode mode;
+ const char *path;
+ int fd;
+ bool is_pipe;
+ bool force;
+ unsigned long size;
+ enum perf_data_mode mode;
};
static inline bool perf_data_file__is_read(struct perf_data_file *file)
int perf_data_file__open(struct perf_data_file *file);
void perf_data_file__close(struct perf_data_file *file);
+ssize_t perf_data_file__write(struct perf_data_file *file,
+ void *buf, size_t size);
#endif /* __PERF_DATA_H */
int verbose;
bool dump_trace = false, quiet = false;
-int eprintf(int level, const char *fmt, ...)
+static int _eprintf(int level, const char *fmt, va_list args)
{
- va_list args;
int ret = 0;
if (verbose >= level) {
- va_start(args, fmt);
if (use_browser >= 1)
ui_helpline__vshow(fmt, args);
else
ret = vfprintf(stderr, fmt, args);
- va_end(args);
}
return ret;
}
+int eprintf(int level, const char *fmt, ...)
+{
+ va_list args;
+ int ret;
+
+ va_start(args, fmt);
+ ret = _eprintf(level, fmt, args);
+ va_end(args);
+
+ return ret;
+}
+
+/*
+ * Overloading libtraceevent standard info print
+ * function, display with -v in perf.
+ */
+void pr_stat(const char *fmt, ...)
+{
+ va_list args;
+
+ va_start(args, fmt);
+ _eprintf(1, fmt, args);
+ va_end(args);
+ eprintf(1, "\n");
+}
+
int dump_printf(const char *fmt, ...)
{
va_list args;
int ui__error(const char *format, ...) __attribute__((format(printf, 1, 2)));
int ui__warning(const char *format, ...) __attribute__((format(printf, 1, 2)));
+void pr_stat(const char *fmt, ...);
+
#endif /* __PERF_DEBUG_H */
return origin[dso->symtab_type];
}
-int dso__binary_type_file(struct dso *dso, enum dso_binary_type type,
- char *root_dir, char *file, size_t size)
+int dso__read_binary_type_filename(const struct dso *dso,
+ enum dso_binary_type type,
+ char *root_dir, char *filename, size_t size)
{
char build_id_hex[BUILD_ID_SIZE * 2 + 1];
int ret = 0;
case DSO_BINARY_TYPE__DEBUGLINK: {
char *debuglink;
- strncpy(file, dso->long_name, size);
- debuglink = file + dso->long_name_len;
- while (debuglink != file && *debuglink != '/')
+ strncpy(filename, dso->long_name, size);
+ debuglink = filename + dso->long_name_len;
+ while (debuglink != filename && *debuglink != '/')
debuglink--;
if (*debuglink == '/')
debuglink++;
filename__read_debuglink(dso->long_name, debuglink,
- size - (debuglink - file));
+ size - (debuglink - filename));
}
break;
case DSO_BINARY_TYPE__BUILD_ID_CACHE:
/* skip the locally configured cache if a symfs is given */
if (symbol_conf.symfs[0] ||
- (dso__build_id_filename(dso, file, size) == NULL))
+ (dso__build_id_filename(dso, filename, size) == NULL))
ret = -1;
break;
case DSO_BINARY_TYPE__FEDORA_DEBUGINFO:
- snprintf(file, size, "%s/usr/lib/debug%s.debug",
+ snprintf(filename, size, "%s/usr/lib/debug%s.debug",
symbol_conf.symfs, dso->long_name);
break;
case DSO_BINARY_TYPE__UBUNTU_DEBUGINFO:
- snprintf(file, size, "%s/usr/lib/debug%s",
+ snprintf(filename, size, "%s/usr/lib/debug%s",
symbol_conf.symfs, dso->long_name);
break;
case DSO_BINARY_TYPE__OPENEMBEDDED_DEBUGINFO:
{
- char *last_slash;
+ const char *last_slash;
size_t len;
size_t dir_size;
while (last_slash != dso->long_name && *last_slash != '/')
last_slash--;
- len = scnprintf(file, size, "%s", symbol_conf.symfs);
+ len = scnprintf(filename, size, "%s", symbol_conf.symfs);
dir_size = last_slash - dso->long_name + 2;
if (dir_size > (size - len)) {
ret = -1;
break;
}
- len += scnprintf(file + len, dir_size, "%s", dso->long_name);
- len += scnprintf(file + len , size - len, ".debug%s",
+ len += scnprintf(filename + len, dir_size, "%s", dso->long_name);
+ len += scnprintf(filename + len , size - len, ".debug%s",
last_slash);
break;
}
build_id__sprintf(dso->build_id,
sizeof(dso->build_id),
build_id_hex);
- snprintf(file, size,
+ snprintf(filename, size,
"%s/usr/lib/debug/.build-id/%.2s/%s.debug",
symbol_conf.symfs, build_id_hex, build_id_hex + 2);
break;
case DSO_BINARY_TYPE__VMLINUX:
case DSO_BINARY_TYPE__GUEST_VMLINUX:
case DSO_BINARY_TYPE__SYSTEM_PATH_DSO:
- snprintf(file, size, "%s%s",
+ snprintf(filename, size, "%s%s",
symbol_conf.symfs, dso->long_name);
break;
case DSO_BINARY_TYPE__GUEST_KMODULE:
- snprintf(file, size, "%s%s%s", symbol_conf.symfs,
+ snprintf(filename, size, "%s%s%s", symbol_conf.symfs,
root_dir, dso->long_name);
break;
case DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE:
- snprintf(file, size, "%s%s", symbol_conf.symfs,
+ snprintf(filename, size, "%s%s", symbol_conf.symfs,
dso->long_name);
break;
case DSO_BINARY_TYPE__KCORE:
case DSO_BINARY_TYPE__GUEST_KCORE:
- snprintf(file, size, "%s", dso->long_name);
+ snprintf(filename, size, "%s", dso->long_name);
break;
default:
static int open_dso(struct dso *dso, struct machine *machine)
{
- char *root_dir = (char *) "";
- char *name;
int fd;
+ char *root_dir = (char *)"";
+ char *name = malloc(PATH_MAX);
- name = malloc(PATH_MAX);
if (!name)
return -ENOMEM;
if (machine)
root_dir = machine->root_dir;
- if (dso__binary_type_file(dso, dso->data_type,
- root_dir, name, PATH_MAX)) {
+ if (dso__read_binary_type_filename(dso, dso->binary_type,
+ root_dir, name, PATH_MAX)) {
free(name);
return -EINVAL;
}
int dso__data_fd(struct dso *dso, struct machine *machine)
{
- static enum dso_binary_type binary_type_data[] = {
+ enum dso_binary_type binary_type_data[] = {
DSO_BINARY_TYPE__BUILD_ID_CACHE,
DSO_BINARY_TYPE__SYSTEM_PATH_DSO,
DSO_BINARY_TYPE__NOT_FOUND,
};
int i = 0;
- if (dso->data_type != DSO_BINARY_TYPE__NOT_FOUND)
+ if (dso->binary_type != DSO_BINARY_TYPE__NOT_FOUND)
return open_dso(dso, machine);
do {
int fd;
- dso->data_type = binary_type_data[i++];
+ dso->binary_type = binary_type_data[i++];
fd = open_dso(dso, machine);
if (fd >= 0)
return fd;
- } while (dso->data_type != DSO_BINARY_TYPE__NOT_FOUND);
+ } while (dso->binary_type != DSO_BINARY_TYPE__NOT_FOUND);
return -EINVAL;
}
}
}
-static struct dso_cache*
-dso_cache__find(struct rb_root *root, u64 offset)
+static struct dso_cache *dso_cache__find(const struct rb_root *root, u64 offset)
{
- struct rb_node **p = &root->rb_node;
- struct rb_node *parent = NULL;
+ struct rb_node * const *p = &root->rb_node;
+ const struct rb_node *parent = NULL;
struct dso_cache *cache;
while (*p != NULL) {
* processing we had no idea this was the kernel dso.
*/
if (dso != NULL) {
- dso__set_short_name(dso, short_name);
+ dso__set_short_name(dso, short_name, false);
dso->kernel = dso_type;
}
return dso;
}
-void dso__set_long_name(struct dso *dso, char *name)
+void dso__set_long_name(struct dso *dso, const char *name, bool name_allocated)
{
if (name == NULL)
return;
- dso->long_name = name;
- dso->long_name_len = strlen(name);
+
+ if (dso->long_name_allocated)
+ free((char *)dso->long_name);
+
+ dso->long_name = name;
+ dso->long_name_len = strlen(name);
+ dso->long_name_allocated = name_allocated;
}
-void dso__set_short_name(struct dso *dso, const char *name)
+void dso__set_short_name(struct dso *dso, const char *name, bool name_allocated)
{
if (name == NULL)
return;
- dso->short_name = name;
- dso->short_name_len = strlen(name);
+
+ if (dso->short_name_allocated)
+ free((char *)dso->short_name);
+
+ dso->short_name = name;
+ dso->short_name_len = strlen(name);
+ dso->short_name_allocated = name_allocated;
}
static void dso__set_basename(struct dso *dso)
{
- dso__set_short_name(dso, basename(dso->long_name));
+ /*
+ * basename() may modify path buffer, so we must pass
+ * a copy.
+ */
+ char *base, *lname = strdup(dso->long_name);
+
+ if (!lname)
+ return;
+
+ /*
+ * basename() may return a pointer to internal
+ * storage which is reused in subsequent calls
+ * so copy the result.
+ */
+ base = strdup(basename(lname));
+
+ free(lname);
+
+ if (!base)
+ return;
+
+ dso__set_short_name(dso, base, true);
}
int dso__name_len(const struct dso *dso)
if (dso != NULL) {
int i;
strcpy(dso->name, name);
- dso__set_long_name(dso, dso->name);
- dso__set_short_name(dso, dso->name);
+ dso__set_long_name(dso, dso->name, false);
+ dso__set_short_name(dso, dso->name, false);
for (i = 0; i < MAP__NR_TYPES; ++i)
dso->symbols[i] = dso->symbol_names[i] = RB_ROOT;
dso->cache = RB_ROOT;
dso->symtab_type = DSO_BINARY_TYPE__NOT_FOUND;
- dso->data_type = DSO_BINARY_TYPE__NOT_FOUND;
+ dso->binary_type = DSO_BINARY_TYPE__NOT_FOUND;
dso->loaded = 0;
dso->rel = 0;
dso->sorted_by_name = 0;
dso->has_build_id = 0;
dso->has_srcline = 1;
+ dso->a2l_fails = 1;
dso->kernel = DSO_TYPE_USER;
dso->needs_swap = DSO_SWAP__UNSET;
INIT_LIST_HEAD(&dso->node);
int i;
for (i = 0; i < MAP__NR_TYPES; ++i)
symbols__delete(&dso->symbols[i]);
- if (dso->sname_alloc)
- free((char *)dso->short_name);
- if (dso->lname_alloc)
- free(dso->long_name);
+
+ if (dso->short_name_allocated) {
+ zfree((char **)&dso->short_name);
+ dso->short_name_allocated = false;
+ }
+
+ if (dso->long_name_allocated) {
+ zfree((char **)&dso->long_name);
+ dso->long_name_allocated = false;
+ }
+
dso_cache__free(&dso->cache);
+ dso__free_a2l(dso);
+ zfree(&dso->symsrc_filename);
free(dso);
}
list_add_tail(&dso->node, head);
}
-struct dso *dsos__find(struct list_head *head, const char *name, bool cmp_short)
+struct dso *dsos__find(const struct list_head *head, const char *name, bool cmp_short)
{
struct dso *pos;
struct rb_root symbols[MAP__NR_TYPES];
struct rb_root symbol_names[MAP__NR_TYPES];
struct rb_root cache;
+ void *a2l;
+ char *symsrc_filename;
+ unsigned int a2l_fails;
enum dso_kernel_type kernel;
enum dso_swap_type needs_swap;
enum dso_binary_type symtab_type;
- enum dso_binary_type data_type;
+ enum dso_binary_type binary_type;
u8 adjust_symbols:1;
u8 has_build_id:1;
u8 has_srcline:1;
u8 hit:1;
u8 annotate_warned:1;
- u8 sname_alloc:1;
- u8 lname_alloc:1;
+ u8 short_name_allocated:1;
+ u8 long_name_allocated:1;
u8 sorted_by_name;
u8 loaded;
u8 rel;
u8 build_id[BUILD_ID_SIZE];
const char *short_name;
- char *long_name;
+ const char *long_name;
u16 long_name_len;
u16 short_name_len;
char name[0];
struct dso *dso__new(const char *name);
void dso__delete(struct dso *dso);
-void dso__set_short_name(struct dso *dso, const char *name);
-void dso__set_long_name(struct dso *dso, char *name);
+void dso__set_short_name(struct dso *dso, const char *name, bool name_allocated);
+void dso__set_long_name(struct dso *dso, const char *name, bool name_allocated);
int dso__name_len(const struct dso *dso);
int dso__kernel_module_get_build_id(struct dso *dso, const char *root_dir);
char dso__symtab_origin(const struct dso *dso);
-int dso__binary_type_file(struct dso *dso, enum dso_binary_type type,
- char *root_dir, char *file, size_t size);
+int dso__read_binary_type_filename(const struct dso *dso, enum dso_binary_type type,
+ char *root_dir, char *filename, size_t size);
int dso__data_fd(struct dso *dso, struct machine *machine);
ssize_t dso__data_read_offset(struct dso *dso, struct machine *machine,
const char *short_name, int dso_type);
void dsos__add(struct list_head *head, struct dso *dso);
-struct dso *dsos__find(struct list_head *head, const char *name,
+struct dso *dsos__find(const struct list_head *head, const char *name,
bool cmp_short);
struct dso *__dsos__findnew(struct list_head *head, const char *name);
bool __dsos__read_build_ids(struct list_head *head, bool with_hits);
static inline bool dso__is_vmlinux(struct dso *dso)
{
- return dso->data_type == DSO_BINARY_TYPE__VMLINUX ||
- dso->data_type == DSO_BINARY_TYPE__GUEST_VMLINUX;
+ return dso->binary_type == DSO_BINARY_TYPE__VMLINUX ||
+ dso->binary_type == DSO_BINARY_TYPE__GUEST_VMLINUX;
}
static inline bool dso__is_kcore(struct dso *dso)
{
- return dso->data_type == DSO_BINARY_TYPE__KCORE ||
- dso->data_type == DSO_BINARY_TYPE__GUEST_KCORE;
+ return dso->binary_type == DSO_BINARY_TYPE__KCORE ||
+ dso->binary_type == DSO_BINARY_TYPE__GUEST_KCORE;
}
+void dso__free_a2l(struct dso *dso);
+
#endif /* __PERF_DSO */
#include "strlist.h"
#include "thread.h"
#include "thread_map.h"
+#include "symbol/kallsyms.h"
static const char *perf_event__names[] = {
[0] = "TOTAL",
memset(&event->comm, 0, sizeof(event->comm));
- tgid = perf_event__get_comm_tgid(pid, event->comm.comm,
- sizeof(event->comm.comm));
+ if (machine__is_host(machine))
+ tgid = perf_event__get_comm_tgid(pid, event->comm.comm,
+ sizeof(event->comm.comm));
+ else
+ tgid = machine->pid;
+
if (tgid < 0)
goto out;
goto out;
}
- snprintf(filename, sizeof(filename), "/proc/%d/task", pid);
+ if (machine__is_default_guest(machine))
+ return 0;
+
+ snprintf(filename, sizeof(filename), "%s/proc/%d/task",
+ machine->root_dir, pid);
tasks = opendir(filename);
if (tasks == NULL) {
return tgid;
}
-static int perf_event__synthesize_mmap_events(struct perf_tool *tool,
- union perf_event *event,
- pid_t pid, pid_t tgid,
- perf_event__handler_t process,
- struct machine *machine,
- bool mmap_data)
+int perf_event__synthesize_mmap_events(struct perf_tool *tool,
+ union perf_event *event,
+ pid_t pid, pid_t tgid,
+ perf_event__handler_t process,
+ struct machine *machine,
+ bool mmap_data)
{
char filename[PATH_MAX];
FILE *fp;
int rc = 0;
- snprintf(filename, sizeof(filename), "/proc/%d/maps", pid);
+ if (machine__is_default_guest(machine))
+ return 0;
+
+ snprintf(filename, sizeof(filename), "%s/proc/%d/maps",
+ machine->root_dir, pid);
fp = fopen(filename, "r");
if (fp == NULL) {
/*
* Just like the kernel, see __perf_event_mmap in kernel/perf_event.c
*/
- event->header.misc = PERF_RECORD_MISC_USER;
+ if (machine__is_host(machine))
+ event->header.misc = PERF_RECORD_MISC_USER;
+ else
+ event->header.misc = PERF_RECORD_MISC_GUEST_USER;
if (prot[2] != 'x') {
if (!mmap_data || prot[0] != 'r')
struct machine *machine, bool mmap_data)
{
DIR *proc;
+ char proc_path[PATH_MAX];
struct dirent dirent, *next;
union perf_event *comm_event, *mmap_event;
int err = -1;
if (mmap_event == NULL)
goto out_free_comm;
- proc = opendir("/proc");
+ if (machine__is_default_guest(machine))
+ return 0;
+
+ snprintf(proc_path, sizeof(proc_path), "%s/proc", machine->root_dir);
+ proc = opendir(proc_path);
+
if (proc == NULL)
goto out_free_mmap;
struct map_groups *mg = &thread->mg;
bool load_map = false;
+ al->machine = machine;
al->thread = thread;
al->addr = addr;
al->cpumode = cpumode;
al->level = 'g';
mg = &machine->kmaps;
load_map = true;
+ } else if (cpumode == PERF_RECORD_MISC_GUEST_USER && perf_guest) {
+ al->level = 'u';
} else {
- /*
- * 'u' means guest os user space.
- * TODO: We don't support guest user space. Might support late.
- */
- if (cpumode == PERF_RECORD_MISC_GUEST_USER && perf_guest)
- al->level = 'u';
- else
- al->level = 'H';
+ al->level = 'H';
al->map = NULL;
if ((cpumode == PERF_RECORD_MISC_GUEST_USER ||
if (thread == NULL)
return -1;
- if (symbol_conf.comm_list &&
- !strlist__has_entry(symbol_conf.comm_list, thread__comm_str(thread)))
+ if (thread__is_filtered(thread))
goto out_filtered;
dump_printf(" ... thread: %s:%d\n", thread__comm_str(thread), thread->tid);
const struct perf_sample *sample,
bool swapped);
+int perf_event__synthesize_mmap_events(struct perf_tool *tool,
+ union perf_event *event,
+ pid_t pid, pid_t tgid,
+ perf_event__handler_t process,
+ struct machine *machine,
+ bool mmap_data);
+
size_t perf_event__fprintf_comm(union perf_event *event, FILE *fp);
size_t perf_event__fprintf_mmap(union perf_event *event, FILE *fp);
size_t perf_event__fprintf_mmap2(union perf_event *event, FILE *fp);
* Released under the GPL v2. (and only v2, not any later version)
*/
#include "util.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include <poll.h>
#include "cpumap.h"
#include "thread_map.h"
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node)
+ evlist__for_each(evlist, evsel)
perf_evsel__calc_id_pos(evsel);
perf_evlist__set_id_pos(evlist);
{
struct perf_evsel *pos, *n;
- list_for_each_entry_safe(pos, n, &evlist->entries, node) {
+ evlist__for_each_safe(evlist, n, pos) {
list_del_init(&pos->node);
perf_evsel__delete(pos);
}
void perf_evlist__exit(struct perf_evlist *evlist)
{
- free(evlist->mmap);
- free(evlist->pollfd);
- evlist->mmap = NULL;
- evlist->pollfd = NULL;
+ zfree(&evlist->mmap);
+ zfree(&evlist->pollfd);
}
void perf_evlist__delete(struct perf_evlist *evlist)
{
+ perf_evlist__munmap(evlist);
+ perf_evlist__close(evlist);
+ cpu_map__delete(evlist->cpus);
+ thread_map__delete(evlist->threads);
+ evlist->cpus = NULL;
+ evlist->threads = NULL;
perf_evlist__purge(evlist);
perf_evlist__exit(evlist);
free(evlist);
leader->nr_members = evsel->idx - leader->idx + 1;
- list_for_each_entry(evsel, list, node) {
+ __evlist__for_each(list, evsel) {
evsel->leader = leader;
}
}
return 0;
out_delete_partial_list:
- list_for_each_entry_safe(evsel, n, &head, node)
+ __evlist__for_each_safe(&head, n, evsel)
perf_evsel__delete(evsel);
return -1;
}
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (evsel->attr.type == PERF_TYPE_TRACEPOINT &&
(int)evsel->attr.config == id)
return evsel;
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if ((evsel->attr.type == PERF_TYPE_TRACEPOINT) &&
(strcmp(evsel->name, name) == 0))
return evsel;
int nr_threads = thread_map__nr(evlist->threads);
for (cpu = 0; cpu < nr_cpus; cpu++) {
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
if (!perf_evsel__is_group_leader(pos) || !pos->fd)
continue;
for (thread = 0; thread < nr_threads; thread++)
int nr_threads = thread_map__nr(evlist->threads);
for (cpu = 0; cpu < nr_cpus; cpu++) {
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
if (!perf_evsel__is_group_leader(pos) || !pos->fd)
continue;
for (thread = 0; thread < nr_threads; thread++)
{
int i;
+ if (evlist->mmap == NULL)
+ return;
+
for (i = 0; i < evlist->nr_mmaps; i++)
__perf_evlist__munmap(evlist, i);
- free(evlist->mmap);
- evlist->mmap = NULL;
+ zfree(&evlist->mmap);
}
static int perf_evlist__alloc_mmap(struct perf_evlist *evlist)
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
int fd = FD(evsel, cpu, thread);
if (*output == -1) {
return -EINVAL;
}
- if ((pages == 0) && (min == 0)) {
+ if (pages == 0 && min == 0) {
/* leave number of pages at 0 */
- } else if (pages < (1UL << 31) && !is_power_of_2(pages)) {
+ } else if (!is_power_of_2(pages)) {
/* round pages up to next power of 2 */
- pages = next_pow2(pages);
+ pages = next_pow2_l(pages);
+ if (!pages)
+ return -EINVAL;
pr_info("rounding mmap pages size to %lu bytes (%lu pages)\n",
pages * page_size, pages);
}
unsigned long max = UINT_MAX;
long pages;
- if (max < SIZE_MAX / page_size)
+ if (max > SIZE_MAX / page_size)
max = SIZE_MAX / page_size;
pages = parse_pages_arg(str, 1, max);
pr_debug("mmap size %zuB\n", evlist->mmap_len);
mask = evlist->mmap_len - page_size - 1;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if ((evsel->attr.read_format & PERF_FORMAT_ID) &&
evsel->sample_id == NULL &&
perf_evsel__alloc_id(evsel, cpu_map__nr(cpus), threads->nr) < 0)
if (evlist->threads == NULL)
return -1;
- if (target->force_per_cpu)
- evlist->cpus = cpu_map__new(target->cpu_list);
- else if (target__has_task(target))
- evlist->cpus = cpu_map__dummy_new();
- else if (!target__has_cpu(target) && !target->uses_mmap)
+ if (target__uses_dummy_map(target))
evlist->cpus = cpu_map__dummy_new();
else
evlist->cpus = cpu_map__new(target->cpu_list);
return -1;
}
-void perf_evlist__delete_maps(struct perf_evlist *evlist)
-{
- cpu_map__delete(evlist->cpus);
- thread_map__delete(evlist->threads);
- evlist->cpus = NULL;
- evlist->threads = NULL;
-}
-
int perf_evlist__apply_filters(struct perf_evlist *evlist)
{
struct perf_evsel *evsel;
const int ncpus = cpu_map__nr(evlist->cpus),
nthreads = thread_map__nr(evlist->threads);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (evsel->filter == NULL)
continue;
const int ncpus = cpu_map__nr(evlist->cpus),
nthreads = thread_map__nr(evlist->threads);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
err = perf_evsel__set_filter(evsel, ncpus, nthreads, filter);
if (err)
break;
if (evlist->id_pos < 0 || evlist->is_pos < 0)
return false;
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
if (pos->id_pos != evlist->id_pos ||
pos->is_pos != evlist->is_pos)
return false;
if (evlist->combined_sample_type)
return evlist->combined_sample_type;
- list_for_each_entry(evsel, &evlist->entries, node)
+ evlist__for_each(evlist, evsel)
evlist->combined_sample_type |= evsel->attr.sample_type;
return evlist->combined_sample_type;
u64 read_format = first->attr.read_format;
u64 sample_type = first->attr.sample_type;
- list_for_each_entry_continue(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
if (read_format != pos->attr.read_format)
return false;
}
{
struct perf_evsel *first = perf_evlist__first(evlist), *pos = first;
- list_for_each_entry_continue(pos, &evlist->entries, node) {
+ evlist__for_each_continue(evlist, pos) {
if (first->attr.sample_id_all != pos->attr.sample_id_all)
return false;
}
int ncpus = cpu_map__nr(evlist->cpus);
int nthreads = thread_map__nr(evlist->threads);
- list_for_each_entry_reverse(evsel, &evlist->entries, node)
+ evlist__for_each_reverse(evlist, evsel)
perf_evsel__close(evsel, ncpus, nthreads);
}
perf_evlist__update_id_pos(evlist);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
err = perf_evsel__open(evsel, evlist->cpus, evlist->threads);
if (err < 0)
goto out_err;
int perf_evlist__prepare_workload(struct perf_evlist *evlist, struct target *target,
const char *argv[], bool pipe_output,
- bool want_signal)
+ void (*exec_error)(int signo, siginfo_t *info, void *ucontext))
{
int child_ready_pipe[2], go_pipe[2];
char bf;
execvp(argv[0], (char **)argv);
- perror(argv[0]);
- if (want_signal)
- kill(getppid(), SIGUSR1);
+ if (exec_error) {
+ union sigval val;
+
+ val.sival_int = errno;
+ if (sigqueue(getppid(), SIGUSR1, val))
+ perror(argv[0]);
+ } else
+ perror(argv[0]);
exit(-1);
}
+ if (exec_error) {
+ struct sigaction act = {
+ .sa_flags = SA_SIGINFO,
+ .sa_sigaction = exec_error,
+ };
+ sigaction(SIGUSR1, &act, NULL);
+ }
+
if (target__none(target))
evlist->threads->map[0] = evlist->workload.pid;
struct perf_evsel *evsel;
size_t printed = 0;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
printed += fprintf(fp, "%s%s", evsel->idx ? ", " : "",
perf_evsel__name(evsel));
}
"Error:\t%s.\n"
"Hint:\tCheck /proc/sys/kernel/perf_event_paranoid setting.", emsg);
- if (filename__read_int("/proc/sys/kernel/perf_event_paranoid", &value))
- break;
+ value = perf_event_paranoid();
printed += scnprintf(buf + printed, size - printed, "\nHint:\t");
return 0;
}
+
+void perf_evlist__to_front(struct perf_evlist *evlist,
+ struct perf_evsel *move_evsel)
+{
+ struct perf_evsel *evsel, *n;
+ LIST_HEAD(move);
+
+ if (move_evsel == perf_evlist__first(evlist))
+ return;
+
+ evlist__for_each_safe(evlist, n, evsel) {
+ if (evsel->leader == move_evsel->leader)
+ list_move_tail(&evsel->node, &move);
+ }
+
+ list_splice(&move, &evlist->entries);
+}
struct pollfd;
struct thread_map;
struct cpu_map;
-struct perf_record_opts;
+struct record_opts;
#define PERF_EVLIST__HLIST_BITS 8
#define PERF_EVLIST__HLIST_SIZE (1 << PERF_EVLIST__HLIST_BITS)
void perf_evlist__set_id_pos(struct perf_evlist *evlist);
bool perf_can_sample_identifier(void);
-void perf_evlist__config(struct perf_evlist *evlist,
- struct perf_record_opts *opts);
-int perf_record_opts__config(struct perf_record_opts *opts);
+void perf_evlist__config(struct perf_evlist *evlist, struct record_opts *opts);
+int record_opts__config(struct record_opts *opts);
int perf_evlist__prepare_workload(struct perf_evlist *evlist,
struct target *target,
const char *argv[], bool pipe_output,
- bool want_signal);
+ void (*exec_error)(int signo, siginfo_t *info,
+ void *ucontext));
int perf_evlist__start_workload(struct perf_evlist *evlist);
int perf_evlist__parse_mmap_pages(const struct option *opt,
}
int perf_evlist__create_maps(struct perf_evlist *evlist, struct target *target);
-void perf_evlist__delete_maps(struct perf_evlist *evlist);
int perf_evlist__apply_filters(struct perf_evlist *evlist);
void __perf_evlist__set_leader(struct list_head *list);
pc->data_tail = tail;
}
+bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str);
+void perf_evlist__to_front(struct perf_evlist *evlist,
+ struct perf_evsel *move_evsel);
+
+/**
+ * __evlist__for_each - iterate thru all the evsels
+ * @list: list_head instance to iterate
+ * @evsel: struct evsel iterator
+ */
+#define __evlist__for_each(list, evsel) \
+ list_for_each_entry(evsel, list, node)
+
+/**
+ * evlist__for_each - iterate thru all the evsels
+ * @evlist: evlist instance to iterate
+ * @evsel: struct evsel iterator
+ */
+#define evlist__for_each(evlist, evsel) \
+ __evlist__for_each(&(evlist)->entries, evsel)
+
+/**
+ * __evlist__for_each_continue - continue iteration thru all the evsels
+ * @list: list_head instance to iterate
+ * @evsel: struct evsel iterator
+ */
+#define __evlist__for_each_continue(list, evsel) \
+ list_for_each_entry_continue(evsel, list, node)
+
+/**
+ * evlist__for_each_continue - continue iteration thru all the evsels
+ * @evlist: evlist instance to iterate
+ * @evsel: struct evsel iterator
+ */
+#define evlist__for_each_continue(evlist, evsel) \
+ __evlist__for_each_continue(&(evlist)->entries, evsel)
+
+/**
+ * __evlist__for_each_reverse - iterate thru all the evsels in reverse order
+ * @list: list_head instance to iterate
+ * @evsel: struct evsel iterator
+ */
+#define __evlist__for_each_reverse(list, evsel) \
+ list_for_each_entry_reverse(evsel, list, node)
+
+/**
+ * evlist__for_each_reverse - iterate thru all the evsels in reverse order
+ * @evlist: evlist instance to iterate
+ * @evsel: struct evsel iterator
+ */
+#define evlist__for_each_reverse(evlist, evsel) \
+ __evlist__for_each_reverse(&(evlist)->entries, evsel)
+
+/**
+ * __evlist__for_each_safe - safely iterate thru all the evsels
+ * @list: list_head instance to iterate
+ * @tmp: struct evsel temp iterator
+ * @evsel: struct evsel iterator
+ */
+#define __evlist__for_each_safe(list, tmp, evsel) \
+ list_for_each_entry_safe(evsel, tmp, list, node)
+
+/**
+ * evlist__for_each_safe - safely iterate thru all the evsels
+ * @evlist: evlist instance to iterate
+ * @evsel: struct evsel iterator
+ * @tmp: struct evsel temp iterator
+ */
+#define evlist__for_each_safe(evlist, tmp, evsel) \
+ __evlist__for_each_safe(&(evlist)->entries, tmp, evsel)
+
#endif /* __PERF_EVLIST_H */
#include <byteswap.h>
#include <linux/bitops.h>
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include <traceevent/event-parse.h>
#include <linux/hw_breakpoint.h>
#include <linux/perf_event.h>
#include "target.h"
#include "perf_regs.h"
#include "debug.h"
+#include "trace-event.h"
static struct {
bool sample_id_all;
evsel->idx = idx;
evsel->attr = *attr;
evsel->leader = evsel;
+ evsel->unit = "";
+ evsel->scale = 1.0;
INIT_LIST_HEAD(&evsel->node);
hists__init(&evsel->hists);
evsel->sample_size = __perf_evsel__sample_size(attr->sample_type);
return evsel;
}
-struct event_format *event_format__new(const char *sys, const char *name)
-{
- int fd, n;
- char *filename;
- void *bf = NULL, *nbf;
- size_t size = 0, alloc_size = 0;
- struct event_format *format = NULL;
-
- if (asprintf(&filename, "%s/%s/%s/format", tracing_events_path, sys, name) < 0)
- goto out;
-
- fd = open(filename, O_RDONLY);
- if (fd < 0)
- goto out_free_filename;
-
- do {
- if (size == alloc_size) {
- alloc_size += BUFSIZ;
- nbf = realloc(bf, alloc_size);
- if (nbf == NULL)
- goto out_free_bf;
- bf = nbf;
- }
-
- n = read(fd, bf + size, alloc_size - size);
- if (n < 0)
- goto out_free_bf;
- size += n;
- } while (n > 0);
-
- pevent_parse_format(&format, bf, size, sys);
-
-out_free_bf:
- free(bf);
- close(fd);
-out_free_filename:
- free(filename);
-out:
- return format;
-}
-
struct perf_evsel *perf_evsel__newtp_idx(const char *sys, const char *name, int idx)
{
struct perf_evsel *evsel = zalloc(sizeof(*evsel));
if (asprintf(&evsel->name, "%s:%s", sys, name) < 0)
goto out_free;
- evsel->tp_format = event_format__new(sys, name);
+ evsel->tp_format = trace_event__tp_format(sys, name);
if (evsel->tp_format == NULL)
goto out_free;
return evsel;
out_free:
- free(evsel->name);
+ zfree(&evsel->name);
free(evsel);
return NULL;
}
* enable/disable events specifically, as there's no
* initial traced exec call.
*/
-void perf_evsel__config(struct perf_evsel *evsel,
- struct perf_record_opts *opts)
+void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)
{
struct perf_evsel *leader = evsel->leader;
struct perf_event_attr *attr = &evsel->attr;
int track = !evsel->idx; /* only the first counter needs these */
+ bool per_cpu = opts->target.default_per_cpu && !opts->target.per_thread;
attr->sample_id_all = perf_missing_features.sample_id_all ? 0 : 1;
attr->inherit = !opts->no_inherit;
}
}
- if (target__has_cpu(&opts->target) || opts->target.force_per_cpu)
+ if (target__has_cpu(&opts->target))
perf_evsel__set_sample_bit(evsel, CPU);
if (opts->period)
if (!perf_missing_features.sample_id_all &&
(opts->sample_time || !opts->no_inherit ||
- target__has_cpu(&opts->target) || opts->target.force_per_cpu))
+ target__has_cpu(&opts->target) || per_cpu))
perf_evsel__set_sample_bit(evsel, TIME);
if (opts->raw_samples) {
if (opts->sample_address)
perf_evsel__set_sample_bit(evsel, DATA_SRC);
- if (opts->no_delay) {
+ if (opts->no_buffering) {
attr->watermark = 0;
attr->wakeup_events = 1;
}
* Setting enable_on_exec for independent events and
* group leaders for traced executed by perf.
*/
- if (target__none(&opts->target) && perf_evsel__is_group_leader(evsel))
+ if (target__none(&opts->target) && perf_evsel__is_group_leader(evsel) &&
+ !opts->initial_delay)
attr->enable_on_exec = 1;
}
{
xyarray__delete(evsel->sample_id);
evsel->sample_id = NULL;
- free(evsel->id);
- evsel->id = NULL;
+ zfree(&evsel->id);
}
void perf_evsel__close_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
void perf_evsel__free_counts(struct perf_evsel *evsel)
{
- free(evsel->counts);
+ zfree(&evsel->counts);
}
void perf_evsel__exit(struct perf_evsel *evsel)
{
perf_evsel__exit(evsel);
close_cgroup(evsel->cgrp);
- free(evsel->group_name);
+ zfree(&evsel->group_name);
if (evsel->tp_format)
pevent_free_format(evsel->tp_format);
- free(evsel->name);
+ zfree(&evsel->name);
free(evsel);
}
evsel->attr.type = PERF_TYPE_SOFTWARE;
evsel->attr.config = PERF_COUNT_SW_CPU_CLOCK;
- free(evsel->name);
- evsel->name = NULL;
+ zfree(&evsel->name);
return true;
}
u32 ids;
struct hists hists;
char *name;
+ double scale;
+ const char *unit;
struct event_format *tp_format;
union {
void *priv;
struct cpu_map;
struct thread_map;
struct perf_evlist;
-struct perf_record_opts;
+struct record_opts;
struct perf_evsel *perf_evsel__new_idx(struct perf_event_attr *attr, int idx);
void perf_evsel__delete(struct perf_evsel *evsel);
void perf_evsel__config(struct perf_evsel *evsel,
- struct perf_record_opts *opts);
+ struct record_opts *opts);
int __perf_evsel__sample_size(u64 sample_type);
void perf_evsel__calc_id_pos(struct perf_evsel *evsel);
int __perf_evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result,
char *bf, size_t size);
const char *perf_evsel__name(struct perf_evsel *evsel);
+
const char *perf_evsel__group_name(struct perf_evsel *evsel);
int perf_evsel__group_desc(struct perf_evsel *evsel, char *buf, size_t size);
continue; \
else
-static int write_buildid(char *name, size_t name_len, u8 *build_id,
+static int write_buildid(const char *name, size_t name_len, u8 *build_id,
pid_t pid, u16 misc, int fd)
{
int err;
dsos__for_each_with_build_id(pos, head) {
int err;
- char *name;
+ const char *name;
size_t name_len;
if (!pos->hit)
{
bool is_kallsyms = dso->kernel && dso->long_name[0] != '/';
bool is_vdso = is_vdso_map(dso->short_name);
- char *name = dso->long_name;
+ const char *name = dso->long_name;
char nm[PATH_MAX];
if (dso__is_kcore(dso)) {
if (ret < 0)
return ret;
- list_for_each_entry(evsel, &evlist->entries, node) {
-
+ evlist__for_each(evlist, evsel) {
ret = do_write(fd, &evsel->attr, sz);
if (ret < 0)
return ret;
return;
for (i = 0 ; i < tp->core_sib; i++)
- free(tp->core_siblings[i]);
+ zfree(&tp->core_siblings[i]);
for (i = 0 ; i < tp->thread_sib; i++)
- free(tp->thread_siblings[i]);
+ zfree(&tp->thread_siblings[i]);
free(tp);
}
if (ret < 0)
return ret;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (perf_evsel__is_group_leader(evsel) &&
evsel->nr_members > 1) {
const char *name = evsel->group_name ?: "{anon_group}";
return;
for (evsel = events; evsel->attr.size; evsel++) {
- if (evsel->name)
- free(evsel->name);
- if (evsel->id)
- free(evsel->id);
+ zfree(&evsel->name);
+ zfree(&evsel->id);
}
free(events);
}
}
out:
- if (buf)
- free(buf);
+ free(buf);
return events;
error:
if (events)
session = container_of(ph, struct perf_session, header);
- list_for_each_entry(evsel, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, evsel) {
if (perf_evsel__is_group_leader(evsel) &&
evsel->nr_members > 1) {
fprintf(fp, "# group: %s{%s", evsel->group_name ?: "",
struct perf_header *ph, int fd,
void *data __maybe_unused)
{
- size_t ret;
+ ssize_t ret;
u32 nr;
ret = readn(fd, &nr, sizeof(nr));
void *data __maybe_unused)
{
uint64_t mem;
- size_t ret;
+ ssize_t ret;
ret = readn(fd, &mem, sizeof(mem));
if (ret != sizeof(mem))
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (evsel->idx == idx)
return evsel;
}
struct perf_header *ph, int fd,
void *data __maybe_unused)
{
- size_t ret;
+ ssize_t ret;
char *str;
u32 nr, i;
struct strbuf sb;
struct perf_header *ph, int fd,
void *data __maybe_unused)
{
- size_t ret;
+ ssize_t ret;
u32 nr, i;
char *str;
struct strbuf sb;
struct perf_header *ph, int fd,
void *data __maybe_unused)
{
- size_t ret;
+ ssize_t ret;
u32 nr, node, i;
char *str;
uint64_t mem_total, mem_free;
struct perf_header *ph, int fd,
void *data __maybe_unused)
{
- size_t ret;
+ ssize_t ret;
char *name;
u32 pmu_num;
u32 type;
session->evlist->nr_groups = nr_groups;
i = nr = 0;
- list_for_each_entry(evsel, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, evsel) {
if (evsel->idx == (int) desc[i].leader_idx) {
evsel->leader = evsel;
/* {anon_group} is a dummy name */
ret = 0;
out_free:
for (i = 0; i < nr_groups; i++)
- free(desc[i].name);
+ zfree(&desc[i].name);
free(desc);
return ret;
lseek(fd, sizeof(f_header), SEEK_SET);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(session->evlist, evsel) {
evsel->id_offset = lseek(fd, 0, SEEK_CUR);
err = do_write(fd, evsel->id, evsel->ids * sizeof(u64));
if (err < 0) {
attr_offset = lseek(fd, 0, SEEK_CUR);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
f_attr = (struct perf_file_attr){
.attr = evsel->attr,
.ids = {
}
}
- header->data_offset = lseek(fd, 0, SEEK_CUR);
+ if (!header->data_offset)
+ header->data_offset = lseek(fd, 0, SEEK_CUR);
header->feat_offset = header->data_offset + header->data_size;
if (at_exit) {
int perf_file_header__read(struct perf_file_header *header,
struct perf_header *ph, int fd)
{
- int ret;
+ ssize_t ret;
lseek(fd, 0, SEEK_SET);
struct perf_header *ph, int fd,
bool repipe)
{
- int ret;
+ ssize_t ret;
ret = readn(fd, header, sizeof(*header));
if (ret <= 0)
struct perf_event_attr *attr = &f_attr->attr;
size_t sz, left;
size_t our_sz = sizeof(f_attr->attr);
- int ret;
+ ssize_t ret;
memset(f_attr, 0, sizeof(*f_attr));
{
struct perf_evsel *pos;
- list_for_each_entry(pos, &evlist->entries, node) {
+ evlist__for_each(evlist, pos) {
if (pos->attr.type == PERF_TYPE_TRACEPOINT &&
perf_evsel__prepare_tracepoint_event(pos, pevent))
return -1;
symbol_conf.nr_events = nr_attrs;
- perf_header__process_sections(header, fd, &session->pevent,
+ perf_header__process_sections(header, fd, &session->tevent,
perf_file_section__process);
if (perf_evlist__prepare_tracepoint_events(session->evlist,
- session->pevent))
+ session->tevent.pevent))
goto out_delete_evlist;
return 0;
struct perf_evsel *evsel;
int err = 0;
- list_for_each_entry(evsel, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, evsel) {
err = perf_event__synthesize_attr(tool, &evsel->attr, evsel->ids,
evsel->id, process);
if (err) {
lseek(fd, offset + sizeof(struct tracing_data_event),
SEEK_SET);
- size_read = trace_report(fd, &session->pevent,
+ size_read = trace_report(fd, &session->tevent,
session->repipe);
padding = PERF_ALIGN(size_read, sizeof(u64)) - size_read;
}
perf_evlist__prepare_tracepoint_events(session->evlist,
- session->pevent);
+ session->tevent.pevent);
return size_read + padding;
}
unsigned long long total_mem;
int nr_cmdline;
- char *cmdline;
int nr_sibling_cores;
- char *sibling_cores;
int nr_sibling_threads;
- char *sibling_threads;
int nr_numa_nodes;
- char *numa_nodes;
int nr_pmu_mappings;
- char *pmu_mappings;
int nr_groups;
+ char *cmdline;
+ char *sibling_cores;
+ char *sibling_threads;
+ char *numa_nodes;
+ char *pmu_mappings;
};
struct perf_header {
unsigned int i;
for (i = 0; i < cmds->cnt; ++i)
- free(cmds->names[i]);
- free(cmds->names);
+ zfree(&cmds->names[i]);
+ zfree(&cmds->names);
cmds->cnt = 0;
cmds->alloc = 0;
}
for (i = 0; i < old->cnt; i++)
cmds->names[cmds->cnt++] = old->names[i];
- free(old->names);
+ zfree(&old->names);
old->cnt = 0;
- old->names = NULL;
}
const char *help_unknown_cmd(const char *cmd)
-#include "annotate.h"
#include "util.h"
#include "build-id.h"
#include "hist.h"
}
}
-static void hist_entry__add_cpumode_period(struct hist_entry *he,
- unsigned int cpumode, u64 period)
+static void he_stat__add_cpumode_period(struct he_stat *he_stat,
+ unsigned int cpumode, u64 period)
{
switch (cpumode) {
case PERF_RECORD_MISC_KERNEL:
- he->stat.period_sys += period;
+ he_stat->period_sys += period;
break;
case PERF_RECORD_MISC_USER:
- he->stat.period_us += period;
+ he_stat->period_us += period;
break;
case PERF_RECORD_MISC_GUEST_KERNEL:
- he->stat.period_guest_sys += period;
+ he_stat->period_guest_sys += period;
break;
case PERF_RECORD_MISC_GUEST_USER:
- he->stat.period_guest_us += period;
+ he_stat->period_guest_us += period;
break;
default:
break;
dest->weight += src->weight;
}
-static void hist_entry__decay(struct hist_entry *he)
+static void he_stat__decay(struct he_stat *he_stat)
{
- he->stat.period = (he->stat.period * 7) / 8;
- he->stat.nr_events = (he->stat.nr_events * 7) / 8;
+ he_stat->period = (he_stat->period * 7) / 8;
+ he_stat->nr_events = (he_stat->nr_events * 7) / 8;
/* XXX need decay for weight too? */
}
if (prev_period == 0)
return true;
- hist_entry__decay(he);
+ he_stat__decay(&he->stat);
if (!he->filtered)
hists->stats.total_period -= prev_period - he->stat.period;
}
static struct hist_entry *add_hist_entry(struct hists *hists,
- struct hist_entry *entry,
- struct addr_location *al,
- u64 period,
- u64 weight)
+ struct hist_entry *entry,
+ struct addr_location *al)
{
struct rb_node **p;
struct rb_node *parent = NULL;
struct hist_entry *he;
int64_t cmp;
+ u64 period = entry->stat.period;
+ u64 weight = entry->stat.weight;
p = &hists->entries_in->rb_node;
* This mem info was allocated from machine__resolve_mem
* and will not be used anymore.
*/
- free(entry->mem_info);
+ zfree(&entry->mem_info);
/* If the map of an existing hist_entry has
* become out-of-date due to an exec() or
rb_link_node(&he->rb_node_in, parent, p);
rb_insert_color(&he->rb_node_in, hists->entries_in);
out:
- hist_entry__add_cpumode_period(he, al->cpumode, period);
+ he_stat__add_cpumode_period(&he->stat, al->cpumode, period);
return he;
}
.transaction = transaction,
};
- return add_hist_entry(hists, &entry, al, period, weight);
+ return add_hist_entry(hists, &entry, al);
}
int64_t
void hist_entry__free(struct hist_entry *he)
{
- free(he->branch_info);
- free(he->mem_info);
+ zfree(&he->branch_info);
+ zfree(&he->mem_info);
free_srcline(he->srcline);
free(he);
}
}
}
-int hist_entry__inc_addr_samples(struct hist_entry *he, int evidx, u64 ip)
-{
- return symbol__inc_addr_samples(he->ms.sym, he->ms.map, evidx, ip);
-}
-
-int hist_entry__annotate(struct hist_entry *he, size_t privsize)
-{
- return symbol__annotate(he->ms.sym, he->ms.map, privsize);
-}
-
void events_stats__inc(struct events_stats *stats, u32 type)
{
++stats->nr_events[0];
size_t hists__fprintf(struct hists *hists, bool show_header, int max_rows,
int max_cols, float min_pcnt, FILE *fp);
-int hist_entry__inc_addr_samples(struct hist_entry *he, int evidx, u64 addr);
-int hist_entry__annotate(struct hist_entry *he, size_t privsize);
-
void hists__filter_by_dso(struct hists *hists);
void hists__filter_by_thread(struct hists *hists);
void hists__filter_by_symbol(struct hists *hists);
+++ /dev/null
-#ifndef _PERF_ASM_GENERIC_BUG_H
-#define _PERF_ASM_GENERIC_BUG_H
-
-#define __WARN_printf(arg...) do { fprintf(stderr, arg); } while (0)
-
-#define WARN(condition, format...) ({ \
- int __ret_warn_on = !!(condition); \
- if (unlikely(__ret_warn_on)) \
- __WARN_printf(format); \
- unlikely(__ret_warn_on); \
-})
-
-#define WARN_ONCE(condition, format...) ({ \
- static int __warned; \
- int __ret_warn_once = !!(condition); \
- \
- if (unlikely(__ret_warn_once)) \
- if (WARN(!__warned, format)) \
- __warned = 1; \
- unlikely(__ret_warn_once); \
-})
-#endif
+++ /dev/null
-#ifndef _PERF_LINUX_COMPILER_H_
-#define _PERF_LINUX_COMPILER_H_
-
-#ifndef __always_inline
-# define __always_inline inline __attribute__((always_inline))
-#endif
-
-#define __user
-
-#ifndef __attribute_const__
-# define __attribute_const__
-#endif
-
-#ifndef __maybe_unused
-# define __maybe_unused __attribute__((unused))
-#endif
-
-#ifndef __packed
-# define __packed __attribute__((__packed__))
-#endif
-
-#ifndef __force
-# define __force
-#endif
-
-#ifndef __weak
-# define __weak __attribute__((weak))
-#endif
-
-#endif
#include "strlist.h"
#include "thread.h"
#include <stdbool.h>
+#include <symbol/kallsyms.h>
#include "unwind.h"
int machine__init(struct machine *machine, const char *root_dir, pid_t pid)
machine->pid = pid;
machine->symbol_filter = NULL;
+ machine->id_hdr_size = 0;
machine->root_dir = strdup(root_dir);
if (machine->root_dir == NULL)
map_groups__exit(&machine->kmaps);
dsos__delete(&machine->user_dsos);
dsos__delete(&machine->kernel_dsos);
- free(machine->root_dir);
- machine->root_dir = NULL;
+ zfree(&machine->root_dir);
}
void machine__delete(struct machine *machine)
char path[PATH_MAX];
struct process_args args;
- if (machine__is_host(machine)) {
- filename = "/proc/kallsyms";
- } else {
- if (machine__is_default_guest(machine))
- filename = (char *)symbol_conf.default_guest_kallsyms;
- else {
- sprintf(path, "%s/proc/kallsyms", machine->root_dir);
- filename = path;
- }
+ if (machine__is_default_guest(machine))
+ filename = (char *)symbol_conf.default_guest_kallsyms;
+ else {
+ sprintf(path, "%s/proc/kallsyms", machine->root_dir);
+ filename = path;
}
if (symbol__restricted_filename(filename, "/proc/kallsyms"))
* on one of them.
*/
if (type == MAP__FUNCTION) {
- free((char *)kmap->ref_reloc_sym->name);
- kmap->ref_reloc_sym->name = NULL;
- free(kmap->ref_reloc_sym);
- }
- kmap->ref_reloc_sym = NULL;
+ zfree((char **)&kmap->ref_reloc_sym->name);
+ zfree(&kmap->ref_reloc_sym);
+ } else
+ kmap->ref_reloc_sym = NULL;
}
map__delete(machine->vmlinux_maps[type]);
ret = -1;
goto out;
}
- dso__set_long_name(map->dso, long_name);
- map->dso->lname_alloc = 1;
+ dso__set_long_name(map->dso, long_name, true);
dso__kernel_module_get_build_id(map->dso, "");
}
}
if (name == NULL)
goto out_problem;
- map->dso->short_name = name;
- map->dso->sname_alloc = 1;
+ dso__set_short_name(map->dso, name, true);
map->end = map->start + event->mmap.len;
} else if (is_kernel_mmap) {
const char *symbol_name = (event->mmap.filename +
*root_al = al;
callchain_cursor_reset(&callchain_cursor);
}
- if (!symbol_conf.use_callchain)
- break;
}
err = callchain_cursor_append(&callchain_cursor,
#include "strlist.h"
#include "vdso.h"
#include "build-id.h"
+#include "util.h"
#include <linux/string.h>
const char *map_type__name[MAP__NR_TYPES] = {
return fprintf(fp, "%s", dsoname);
}
+int map__fprintf_srcline(struct map *map, u64 addr, const char *prefix,
+ FILE *fp)
+{
+ char *srcline;
+ int ret = 0;
+
+ if (map && map->dso) {
+ srcline = get_srcline(map->dso,
+ map__rip_2objdump(map, addr));
+ if (srcline != SRCLINE_UNKNOWN)
+ ret = fprintf(fp, "%s%s", prefix, srcline);
+ free_srcline(srcline);
+ }
+ return ret;
+}
+
/**
* map__rip_2objdump - convert symbol start address to objdump address.
* @map: memory map
int map__overlap(struct map *l, struct map *r);
size_t map__fprintf(struct map *map, FILE *fp);
size_t map__fprintf_dsoname(struct map *map, FILE *fp);
+int map__fprintf_srcline(struct map *map, u64 addr, const char *prefix,
+ FILE *fp);
int map__load(struct map *map, symbol_filter_t filter);
struct symbol *map__find_symbol(struct map *map,
#include "symbol.h"
#include "cache.h"
#include "header.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include "parse-events-bison.h"
#define YY_EXTRA_TYPE int
#include "parse-events-flex.h"
}
path->name = malloc(MAX_EVENT_LENGTH);
if (!path->name) {
- free(path->system);
+ zfree(&path->system);
free(path);
return NULL;
}
path->name = strdup(str+1);
if (path->system == NULL || path->name == NULL) {
- free(path->system);
- free(path->name);
+ zfree(&path->system);
+ zfree(&path->name);
free(path);
path = NULL;
}
-static int __add_event(struct list_head *list, int *idx,
- struct perf_event_attr *attr,
- char *name, struct cpu_map *cpus)
+static struct perf_evsel *
+__add_event(struct list_head *list, int *idx,
+ struct perf_event_attr *attr,
+ char *name, struct cpu_map *cpus)
{
struct perf_evsel *evsel;
evsel = perf_evsel__new_idx(attr, (*idx)++);
if (!evsel)
- return -ENOMEM;
+ return NULL;
evsel->cpus = cpus;
if (name)
evsel->name = strdup(name);
list_add_tail(&evsel->node, list);
- return 0;
+ return evsel;
}
static int add_event(struct list_head *list, int *idx,
struct perf_event_attr *attr, char *name)
{
- return __add_event(list, idx, attr, name, NULL);
+ return __add_event(list, idx, attr, name, NULL) ? 0 : -ENOMEM;
}
static int parse_aliases(char *str, const char *names[][PERF_EVSEL__MAX_ALIASES], int size)
{
struct perf_event_attr attr;
struct perf_pmu *pmu;
+ struct perf_evsel *evsel;
+ char *unit;
+ double scale;
pmu = perf_pmu__find(name);
if (!pmu)
memset(&attr, 0, sizeof(attr));
- if (perf_pmu__check_alias(pmu, head_config))
+ if (perf_pmu__check_alias(pmu, head_config, &unit, &scale))
return -EINVAL;
/*
if (perf_pmu__config(pmu, &attr, head_config))
return -EINVAL;
- return __add_event(list, idx, &attr, pmu_event_name(head_config),
- pmu->cpus);
+ evsel = __add_event(list, idx, &attr, pmu_event_name(head_config),
+ pmu->cpus);
+ if (evsel) {
+ evsel->unit = unit;
+ evsel->scale = scale;
+ }
+
+ return evsel ? 0 : -ENOMEM;
}
int parse_events__modifier_group(struct list_head *list,
if (!add && get_event_modifier(&mod, str, NULL))
return -EINVAL;
- list_for_each_entry(evsel, list, node) {
-
+ __evlist__for_each(list, evsel) {
if (add && get_event_modifier(&mod, str, evsel))
return -EINVAL;
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, list, node) {
+ __evlist__for_each(list, evsel) {
if (!evsel->name)
evsel->name = strdup(name);
}
ret = parse_events__scanner(str, &data, PE_START_TERMS);
if (!ret) {
list_splice(data.terms, terms);
- free(data.terms);
+ zfree(&data.terms);
return 0;
}
case OPTION_BOOLEAN:
*(bool *)opt->value = unset ? false : true;
+ if (opt->set)
+ *(bool *)opt->set = true;
return 0;
case OPTION_INCR:
return 0;
}
if (!rest) {
+ if (!prefixcmp(options->long_name, "no-")) {
+ /*
+ * The long name itself starts with "no-", so
+ * accept the option without "no-" so that users
+ * do not have to enter "no-no-" to get the
+ * negation.
+ */
+ rest = skip_prefix(arg, options->long_name + 3);
+ if (rest) {
+ flags |= OPT_UNSET;
+ goto match;
+ }
+ /* Abbreviated case */
+ if (!prefixcmp(options->long_name + 3, arg)) {
+ flags |= OPT_UNSET;
+ goto is_abbreviated;
+ }
+ }
/* abbreviated? */
if (!strncmp(options->long_name, arg, arg_end - arg)) {
is_abbreviated:
if (!rest)
continue;
}
+match:
if (*rest) {
if (*rest != '=')
continue;
* OPTION_{BIT,SET_UINT,SET_PTR} store the {mask,integer,pointer} to put in
* the value when met.
* CALLBACKS can use it like they want.
+ *
+ * `set`::
+ * whether an option was set by the user
*/
struct option {
enum parse_opt_type type;
int flags;
parse_opt_cb *callback;
intptr_t defval;
+ bool *set;
};
#define check_vtype(v, type) ( BUILD_BUG_ON_ZERO(!__builtin_types_compatible_p(typeof(v), type)) + v )
#define OPT_GROUP(h) { .type = OPTION_GROUP, .help = (h) }
#define OPT_BIT(s, l, v, h, b) { .type = OPTION_BIT, .short_name = (s), .long_name = (l), .value = check_vtype(v, int *), .help = (h), .defval = (b) }
#define OPT_BOOLEAN(s, l, v, h) { .type = OPTION_BOOLEAN, .short_name = (s), .long_name = (l), .value = check_vtype(v, bool *), .help = (h) }
+#define OPT_BOOLEAN_SET(s, l, v, os, h) \
+ { .type = OPTION_BOOLEAN, .short_name = (s), .long_name = (l), \
+ .value = check_vtype(v, bool *), .help = (h), \
+ .set = check_vtype(os, bool *)}
#define OPT_INCR(s, l, v, h) { .type = OPTION_INCR, .short_name = (s), .long_name = (l), .value = check_vtype(v, int *), .help = (h) }
#define OPT_SET_UINT(s, l, v, h, i) { .type = OPTION_SET_UINT, .short_name = (s), .long_name = (l), .value = check_vtype(v, unsigned int *), .help = (h), .defval = (i) }
#define OPT_SET_PTR(s, l, v, h, p) { .type = OPTION_SET_PTR, .short_name = (s), .long_name = (l), .value = (v), .help = (h), .defval = (p) }
#include <linux/list.h>
#include <sys/types.h>
-#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
#include <dirent.h>
#include "fs.h"
+#include <locale.h>
#include "util.h"
#include "pmu.h"
#include "parse-events.h"
#include "cpumap.h"
+#define UNIT_MAX_LEN 31 /* max length for event unit name */
+
struct perf_pmu_alias {
char *name;
struct list_head terms;
struct list_head list;
+ char unit[UNIT_MAX_LEN+1];
+ double scale;
};
struct perf_pmu_format {
return 0;
}
-static int perf_pmu__new_alias(struct list_head *list, char *name, FILE *file)
+static int perf_pmu__parse_scale(struct perf_pmu_alias *alias, char *dir, char *name)
+{
+ struct stat st;
+ ssize_t sret;
+ char scale[128];
+ int fd, ret = -1;
+ char path[PATH_MAX];
+ char *lc;
+
+ snprintf(path, PATH_MAX, "%s/%s.scale", dir, name);
+
+ fd = open(path, O_RDONLY);
+ if (fd == -1)
+ return -1;
+
+ if (fstat(fd, &st) < 0)
+ goto error;
+
+ sret = read(fd, scale, sizeof(scale)-1);
+ if (sret < 0)
+ goto error;
+
+ scale[sret] = '\0';
+ /*
+ * save current locale
+ */
+ lc = setlocale(LC_NUMERIC, NULL);
+
+ /*
+ * force to C locale to ensure kernel
+ * scale string is converted correctly.
+ * kernel uses default C locale.
+ */
+ setlocale(LC_NUMERIC, "C");
+
+ alias->scale = strtod(scale, NULL);
+
+ /* restore locale */
+ setlocale(LC_NUMERIC, lc);
+
+ ret = 0;
+error:
+ close(fd);
+ return ret;
+}
+
+static int perf_pmu__parse_unit(struct perf_pmu_alias *alias, char *dir, char *name)
+{
+ char path[PATH_MAX];
+ ssize_t sret;
+ int fd;
+
+ snprintf(path, PATH_MAX, "%s/%s.unit", dir, name);
+
+ fd = open(path, O_RDONLY);
+ if (fd == -1)
+ return -1;
+
+ sret = read(fd, alias->unit, UNIT_MAX_LEN);
+ if (sret < 0)
+ goto error;
+
+ close(fd);
+
+ alias->unit[sret] = '\0';
+
+ return 0;
+error:
+ close(fd);
+ alias->unit[0] = '\0';
+ return -1;
+}
+
+static int perf_pmu__new_alias(struct list_head *list, char *dir, char *name, FILE *file)
{
struct perf_pmu_alias *alias;
char buf[256];
return -ENOMEM;
INIT_LIST_HEAD(&alias->terms);
+ alias->scale = 1.0;
+ alias->unit[0] = '\0';
+
ret = parse_events_terms(&alias->terms, buf);
if (ret) {
free(alias);
}
alias->name = strdup(name);
+ /*
+ * load unit name and scale if available
+ */
+ perf_pmu__parse_unit(alias, dir, name);
+ perf_pmu__parse_scale(alias, dir, name);
+
list_add_tail(&alias->list, list);
+
return 0;
}
{
struct dirent *evt_ent;
DIR *event_dir;
+ size_t len;
int ret = 0;
event_dir = opendir(dir);
if (!strcmp(name, ".") || !strcmp(name, ".."))
continue;
+ /*
+ * skip .unit and .scale info files
+ * parsed in perf_pmu__new_alias()
+ */
+ len = strlen(name);
+ if (len > 5 && !strcmp(name + len - 5, ".unit"))
+ continue;
+ if (len > 6 && !strcmp(name + len - 6, ".scale"))
+ continue;
+
snprintf(path, PATH_MAX, "%s/%s", dir, name);
ret = -EINVAL;
file = fopen(path, "r");
if (!file)
break;
- ret = perf_pmu__new_alias(head, name, file);
+
+ ret = perf_pmu__new_alias(head, dir, name, file);
fclose(file);
}
/*
* Setup one of config[12] attr members based on the
- * user input data - temr parameter.
+ * user input data - term parameter.
*/
static int pmu_config_term(struct list_head *formats,
struct perf_event_attr *attr,
return NULL;
}
+
+static int check_unit_scale(struct perf_pmu_alias *alias,
+ char **unit, double *scale)
+{
+ /*
+ * Only one term in event definition can
+ * define unit and scale, fail if there's
+ * more than one.
+ */
+ if ((*unit && alias->unit) ||
+ (*scale && alias->scale))
+ return -EINVAL;
+
+ if (alias->unit)
+ *unit = alias->unit;
+
+ if (alias->scale)
+ *scale = alias->scale;
+
+ return 0;
+}
+
/*
* Find alias in the terms list and replace it with the terms
* defined for the alias
*/
-int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms)
+int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms,
+ char **unit, double *scale)
{
struct parse_events_term *term, *h;
struct perf_pmu_alias *alias;
int ret;
+ *unit = NULL;
+ *scale = 0;
+
list_for_each_entry_safe(term, h, head_terms, list) {
alias = pmu_find_alias(pmu, term);
if (!alias)
ret = pmu_alias_terms(alias, &term->list);
if (ret)
return ret;
+
+ ret = check_unit_scale(alias, unit, scale);
+ if (ret)
+ return ret;
+
list_del(&term->list);
free(term);
}
continue;
}
printf(" %-50s [Kernel PMU event]\n", aliases[j]);
- free(aliases[j]);
+ zfree(&aliases[j]);
printed++;
}
if (printed)
int perf_pmu__config_terms(struct list_head *formats,
struct perf_event_attr *attr,
struct list_head *head_terms);
-int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms);
+int perf_pmu__check_alias(struct perf_pmu *pmu, struct list_head *head_terms,
+ char **unit, double *scale);
struct list_head *perf_pmu__alias(struct perf_pmu *pmu,
struct list_head *head_terms);
int perf_pmu_wrap(void);
#include "color.h"
#include "symbol.h"
#include "thread.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include "trace-event.h" /* For __maybe_unused */
#include "probe-event.h"
#include "probe-finder.h"
static char *synthesize_perf_probe_point(struct perf_probe_point *pp);
static int convert_name_to_addr(struct perf_probe_event *pev,
const char *exec);
+static void clear_probe_trace_event(struct probe_trace_event *tev);
static struct machine machine;
/* Initialize symbol maps and path of vmlinux/modules */
vmlinux_name = symbol_conf.vmlinux_name;
if (vmlinux_name) {
- if (dso__load_vmlinux(dso, map, vmlinux_name, NULL) <= 0)
+ if (dso__load_vmlinux(dso, map, vmlinux_name, false, NULL) <= 0)
return NULL;
} else {
if (dso__load_vmlinux_path(dso, map, NULL) <= 0) {
return ret;
}
+static int convert_exec_to_group(const char *exec, char **result)
+{
+ char *ptr1, *ptr2, *exec_copy;
+ char buf[64];
+ int ret;
+
+ exec_copy = strdup(exec);
+ if (!exec_copy)
+ return -ENOMEM;
+
+ ptr1 = basename(exec_copy);
+ if (!ptr1) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ptr2 = strpbrk(ptr1, "-._");
+ if (ptr2)
+ *ptr2 = '\0';
+ ret = e_snprintf(buf, 64, "%s_%s", PERFPROBE_GROUP, ptr1);
+ if (ret < 0)
+ goto out;
+
+ *result = strdup(buf);
+ ret = *result ? 0 : -ENOMEM;
+
+out:
+ free(exec_copy);
+ return ret;
+}
+
static int convert_to_perf_probe_point(struct probe_trace_point *tp,
struct perf_probe_point *pp)
{
return 0;
}
+static int get_text_start_address(const char *exec, unsigned long *address)
+{
+ Elf *elf;
+ GElf_Ehdr ehdr;
+ GElf_Shdr shdr;
+ int fd, ret = -ENOENT;
+
+ fd = open(exec, O_RDONLY);
+ if (fd < 0)
+ return -errno;
+
+ elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
+ if (elf == NULL)
+ return -EINVAL;
+
+ if (gelf_getehdr(elf, &ehdr) == NULL)
+ goto out;
+
+ if (!elf_section_by_name(elf, &ehdr, &shdr, ".text", NULL))
+ goto out;
+
+ *address = shdr.sh_addr - shdr.sh_offset;
+ ret = 0;
+out:
+ elf_end(elf);
+ return ret;
+}
+
+static int add_exec_to_probe_trace_events(struct probe_trace_event *tevs,
+ int ntevs, const char *exec)
+{
+ int i, ret = 0;
+ unsigned long offset, stext = 0;
+ char buf[32];
+
+ if (!exec)
+ return 0;
+
+ ret = get_text_start_address(exec, &stext);
+ if (ret < 0)
+ return ret;
+
+ for (i = 0; i < ntevs && ret >= 0; i++) {
+ offset = tevs[i].point.address - stext;
+ offset += tevs[i].point.offset;
+ tevs[i].point.offset = 0;
+ zfree(&tevs[i].point.symbol);
+ ret = e_snprintf(buf, 32, "0x%lx", offset);
+ if (ret < 0)
+ break;
+ tevs[i].point.module = strdup(exec);
+ tevs[i].point.symbol = strdup(buf);
+ if (!tevs[i].point.symbol || !tevs[i].point.module) {
+ ret = -ENOMEM;
+ break;
+ }
+ tevs[i].uprobes = true;
+ }
+
+ return ret;
+}
+
static int add_module_to_probe_trace_events(struct probe_trace_event *tevs,
int ntevs, const char *module)
{
}
}
- if (tmp)
- free(tmp);
-
+ free(tmp);
return ret;
}
+static void clear_probe_trace_events(struct probe_trace_event *tevs, int ntevs)
+{
+ int i;
+
+ for (i = 0; i < ntevs; i++)
+ clear_probe_trace_event(tevs + i);
+}
+
/* Try to find perf_probe_event with debuginfo */
static int try_to_find_probe_trace_events(struct perf_probe_event *pev,
struct probe_trace_event **tevs,
struct debuginfo *dinfo;
int ntevs, ret = 0;
- if (pev->uprobes) {
- if (need_dwarf) {
- pr_warning("Debuginfo-analysis is not yet supported"
- " with -x/--exec option.\n");
- return -ENOSYS;
- }
- return convert_name_to_addr(pev, target);
- }
-
dinfo = open_debuginfo(target);
if (!dinfo) {
if (ntevs > 0) { /* Succeeded to find trace events */
pr_debug("find %d probe_trace_events.\n", ntevs);
- if (target)
- ret = add_module_to_probe_trace_events(*tevs, ntevs,
- target);
+ if (target) {
+ if (pev->uprobes)
+ ret = add_exec_to_probe_trace_events(*tevs,
+ ntevs, target);
+ else
+ ret = add_module_to_probe_trace_events(*tevs,
+ ntevs, target);
+ }
+ if (ret < 0) {
+ clear_probe_trace_events(*tevs, ntevs);
+ zfree(tevs);
+ }
return ret < 0 ? ret : ntevs;
}
case EFAULT:
raw_path = strchr(++raw_path, '/');
if (!raw_path) {
- free(*new_path);
- *new_path = NULL;
+ zfree(new_path);
return -ENOENT;
}
continue;
default:
- free(*new_path);
- *new_path = NULL;
+ zfree(new_path);
return -errno;
}
}
*/
fprintf(stdout, "\t@<%s+%lu>\n", vl->point.symbol,
vl->point.offset);
- free(vl->point.symbol);
+ zfree(&vl->point.symbol);
nvars = 0;
if (vl->vars) {
strlist__for_each(node, vl->vars) {
static int try_to_find_probe_trace_events(struct perf_probe_event *pev,
struct probe_trace_event **tevs __maybe_unused,
- int max_tevs __maybe_unused, const char *target)
+ int max_tevs __maybe_unused,
+ const char *target __maybe_unused)
{
if (perf_probe_event_need_dwarf(pev)) {
pr_warning("Debuginfo-analysis is not supported.\n");
return -ENOSYS;
}
- if (pev->uprobes)
- return convert_name_to_addr(pev, target);
-
return 0;
}
}
#endif
+void line_range__clear(struct line_range *lr)
+{
+ struct line_node *ln;
+
+ free(lr->function);
+ free(lr->file);
+ free(lr->path);
+ free(lr->comp_dir);
+ while (!list_empty(&lr->line_list)) {
+ ln = list_first_entry(&lr->line_list, struct line_node, list);
+ list_del(&ln->list);
+ free(ln);
+ }
+ memset(lr, 0, sizeof(*lr));
+}
+
+void line_range__init(struct line_range *lr)
+{
+ memset(lr, 0, sizeof(*lr));
+ INIT_LIST_HEAD(&lr->line_list);
+}
+
static int parse_line_num(char **ptr, int *val, const char *what)
{
const char *start = *ptr;
error:
pr_debug("Failed to synthesize perf probe point: %s\n",
strerror(-ret));
- if (buf)
- free(buf);
+ free(buf);
return NULL;
}
struct perf_probe_arg_field *field, *next;
int i;
- if (pev->event)
- free(pev->event);
- if (pev->group)
- free(pev->group);
- if (pp->file)
- free(pp->file);
- if (pp->function)
- free(pp->function);
- if (pp->lazy_line)
- free(pp->lazy_line);
+ free(pev->event);
+ free(pev->group);
+ free(pp->file);
+ free(pp->function);
+ free(pp->lazy_line);
+
for (i = 0; i < pev->nargs; i++) {
- if (pev->args[i].name)
- free(pev->args[i].name);
- if (pev->args[i].var)
- free(pev->args[i].var);
- if (pev->args[i].type)
- free(pev->args[i].type);
+ free(pev->args[i].name);
+ free(pev->args[i].var);
+ free(pev->args[i].type);
field = pev->args[i].field;
while (field) {
next = field->next;
- if (field->name)
- free(field->name);
+ zfree(&field->name);
free(field);
field = next;
}
}
- if (pev->args)
- free(pev->args);
+ free(pev->args);
memset(pev, 0, sizeof(*pev));
}
struct probe_trace_arg_ref *ref, *next;
int i;
- if (tev->event)
- free(tev->event);
- if (tev->group)
- free(tev->group);
- if (tev->point.symbol)
- free(tev->point.symbol);
- if (tev->point.module)
- free(tev->point.module);
+ free(tev->event);
+ free(tev->group);
+ free(tev->point.symbol);
+ free(tev->point.module);
for (i = 0; i < tev->nargs; i++) {
- if (tev->args[i].name)
- free(tev->args[i].name);
- if (tev->args[i].value)
- free(tev->args[i].value);
- if (tev->args[i].type)
- free(tev->args[i].type);
+ free(tev->args[i].name);
+ free(tev->args[i].value);
+ free(tev->args[i].type);
ref = tev->args[i].ref;
while (ref) {
next = ref->next;
ref = next;
}
}
- if (tev->args)
- free(tev->args);
+ free(tev->args);
memset(tev, 0, sizeof(*tev));
}
int max_tevs, const char *target)
{
struct symbol *sym;
- int ret = 0, i;
+ int ret, i;
struct probe_trace_event *tev;
+ if (pev->uprobes && !pev->group) {
+ /* Replace group name if not given */
+ ret = convert_exec_to_group(target, &pev->group);
+ if (ret != 0) {
+ pr_warning("Failed to make a group name.\n");
+ return ret;
+ }
+ }
+
/* Convert perf_probe_event with debuginfo */
ret = try_to_find_probe_trace_events(pev, tevs, max_tevs, target);
if (ret != 0)
return ret; /* Found in debuginfo or got an error */
+ if (pev->uprobes) {
+ ret = convert_name_to_addr(pev, target);
+ if (ret < 0)
+ return ret;
+ }
+
/* Allocate trace event buffer */
tev = *tevs = zalloc(sizeof(struct probe_trace_event));
if (tev == NULL)
for (i = 0; i < npevs; i++) {
for (j = 0; j < pkgs[i].ntevs; j++)
clear_probe_trace_event(&pkgs[i].tevs[j]);
- free(pkgs[i].tevs);
+ zfree(&pkgs[i].tevs);
}
free(pkgs);
struct perf_probe_point *pp = &pev->point;
struct symbol *sym;
struct map *map = NULL;
- char *function = NULL, *name = NULL;
+ char *function = NULL;
int ret = -EINVAL;
unsigned long long vaddr = 0;
goto out;
}
- name = realpath(exec, NULL);
- if (!name) {
- pr_warning("Cannot find realpath for %s.\n", exec);
- goto out;
- }
- map = dso__new_map(name);
+ map = dso__new_map(exec);
if (!map) {
pr_warning("Cannot find appropriate DSO for %s.\n", exec);
goto out;
}
if (function)
free(function);
- if (name)
- free(name);
return ret;
}
char *symbol; /* Base symbol */
char *module; /* Module name */
unsigned long offset; /* Offset from symbol */
+ unsigned long address; /* Actual address of the trace point */
bool retprobe; /* Return probe flag */
};
/* Command string to line-range */
extern int parse_line_range_desc(const char *cmd, struct line_range *lr);
+/* Release line range members */
+extern void line_range__clear(struct line_range *lr);
+
+/* Initialize line range */
+extern void line_range__init(struct line_range *lr);
+
/* Internal use: Return kernel/module path */
extern const char *kernel_get_module_path(const char *module);
if (!dbg)
return NULL;
- if (debuginfo__init_offline_dwarf(dbg, path) < 0) {
- free(dbg);
- dbg = NULL;
- }
+ if (debuginfo__init_offline_dwarf(dbg, path) < 0)
+ zfree(&dbg);
return dbg;
}
if (!dbg)
return NULL;
- if (debuginfo__init_online_kernel_dwarf(dbg, (Dwarf_Addr)addr) < 0) {
- free(dbg);
- dbg = NULL;
- }
+ if (debuginfo__init_online_kernel_dwarf(dbg, (Dwarf_Addr)addr) < 0)
+ zfree(&dbg);
return dbg;
}
return -ENOENT;
}
tp->offset = (unsigned long)(paddr - sym.st_value);
+ tp->address = (unsigned long)paddr;
tp->symbol = strdup(symbol);
if (!tp->symbol)
return -ENOMEM;
ret = debuginfo__find_probes(dbg, &tf.pf);
if (ret < 0) {
- free(*tevs);
- *tevs = NULL;
+ zfree(tevs);
return ret;
}
if (ret < 0) {
/* Free vlist for error */
while (af.nvls--) {
- if (af.vls[af.nvls].point.symbol)
- free(af.vls[af.nvls].point.symbol);
- if (af.vls[af.nvls].vars)
- strlist__delete(af.vls[af.nvls].vars);
+ zfree(&af.vls[af.nvls].point.symbol);
+ strlist__delete(af.vls[af.nvls].vars);
}
- free(af.vls);
- *vls = NULL;
+ zfree(vls);
return ret;
}
if (fname) {
ppt->file = strdup(fname);
if (ppt->file == NULL) {
- if (ppt->function) {
- free(ppt->function);
- ppt->function = NULL;
- }
+ zfree(&ppt->function);
ret = -ENOMEM;
goto end;
}
else
ret = 0; /* Lines are not found */
else {
- free(lf->lr->path);
- lf->lr->path = NULL;
+ zfree(&lf->lr->path);
}
return ret;
}
util/rblist.c
util/strlist.c
util/fs.c
+util/trace-event.c
../../lib/rbtree.c
if (i >= pevlist->evlist.nr_entries)
return NULL;
- list_for_each_entry(pos, &pevlist->evlist.entries, node)
+ evlist__for_each(&pevlist->evlist, pos) {
if (i-- == 0)
break;
+ }
return Py_BuildValue("O", container_of(pos, struct pyrf_evsel, evsel));
}
return perf_probe_api(perf_probe_sample_identifier);
}
-void perf_evlist__config(struct perf_evlist *evlist,
- struct perf_record_opts *opts)
+void perf_evlist__config(struct perf_evlist *evlist, struct record_opts *opts)
{
struct perf_evsel *evsel;
bool use_sample_identifier = false;
if (evlist->cpus->map[0] < 0)
opts->no_inherit = true;
- list_for_each_entry(evsel, &evlist->entries, node)
+ evlist__for_each(evlist, evsel)
perf_evsel__config(evsel, opts);
if (evlist->nr_entries > 1) {
struct perf_evsel *first = perf_evlist__first(evlist);
- list_for_each_entry(evsel, &evlist->entries, node) {
+ evlist__for_each(evlist, evsel) {
if (evsel->attr.sample_type == first->attr.sample_type)
continue;
use_sample_identifier = perf_can_sample_identifier();
break;
}
- list_for_each_entry(evsel, &evlist->entries, node)
+ evlist__for_each(evlist, evsel)
perf_evsel__set_sample_id(evsel, use_sample_identifier);
}
return filename__read_int(path, (int *) rate);
}
-static int perf_record_opts__config_freq(struct perf_record_opts *opts)
+static int record_opts__config_freq(struct record_opts *opts)
{
bool user_freq = opts->user_freq != UINT_MAX;
unsigned int max_rate;
return 0;
}
-int perf_record_opts__config(struct perf_record_opts *opts)
+int record_opts__config(struct record_opts *opts)
{
- return perf_record_opts__config_freq(opts);
+ return record_opts__config_freq(opts);
+}
+
+bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str)
+{
+ struct perf_evlist *temp_evlist;
+ struct perf_evsel *evsel;
+ int err, fd, cpu;
+ bool ret = false;
+
+ temp_evlist = perf_evlist__new();
+ if (!temp_evlist)
+ return false;
+
+ err = parse_events(temp_evlist, str);
+ if (err)
+ goto out_delete;
+
+ evsel = perf_evlist__last(temp_evlist);
+
+ if (!evlist || cpu_map__empty(evlist->cpus)) {
+ struct cpu_map *cpus = cpu_map__new(NULL);
+
+ cpu = cpus ? cpus->map[0] : 0;
+ cpu_map__delete(cpus);
+ } else {
+ cpu = evlist->cpus->map[0];
+ }
+
+ fd = sys_perf_event_open(&evsel->attr, -1, cpu, -1, 0);
+ if (fd >= 0) {
+ close(fd);
+ ret = true;
+ }
+
+out_delete:
+ perf_evlist__delete(temp_evlist);
+ return ret;
}
zero_flag_atom = 0;
break;
case PRINT_FIELD:
- if (cur_field_name)
- free(cur_field_name);
+ free(cur_field_name);
cur_field_name = strdup(args->field.name);
break;
case PRINT_FLAGS:
return event;
}
-static void perl_process_tracepoint(union perf_event *perf_event __maybe_unused,
- struct perf_sample *sample,
+static void perl_process_tracepoint(struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine __maybe_unused,
- struct thread *thread,
- struct addr_location *al)
+ struct thread *thread)
{
struct format_field *field;
static char handler[256];
static void perl_process_event_generic(union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel,
- struct machine *machine __maybe_unused,
- struct thread *thread __maybe_unused,
- struct addr_location *al __maybe_unused)
+ struct perf_evsel *evsel)
{
dSP;
static void perl_process_event(union perf_event *event,
struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine,
struct thread *thread,
- struct addr_location *al)
+ struct addr_location *al __maybe_unused)
{
- perl_process_tracepoint(event, sample, evsel, machine, thread, al);
- perl_process_event_generic(event, sample, evsel, machine, thread, al);
+ perl_process_tracepoint(sample, evsel, thread);
+ perl_process_event_generic(event, sample, evsel);
}
static void run_start_sub(void)
zero_flag_atom = 0;
break;
case PRINT_FIELD:
- if (cur_field_name)
- free(cur_field_name);
+ free(cur_field_name);
cur_field_name = strdup(args->field.name);
break;
case PRINT_FLAGS:
return event;
}
-static void python_process_tracepoint(union perf_event *perf_event
- __maybe_unused,
- struct perf_sample *sample,
- struct perf_evsel *evsel,
- struct machine *machine __maybe_unused,
- struct thread *thread,
- struct addr_location *al)
+static void python_process_tracepoint(struct perf_sample *sample,
+ struct perf_evsel *evsel,
+ struct thread *thread,
+ struct addr_location *al)
{
PyObject *handler, *retval, *context, *t, *obj, *dict = NULL;
static char handler_name[256];
Py_DECREF(t);
}
-static void python_process_general_event(union perf_event *perf_event
- __maybe_unused,
- struct perf_sample *sample,
+static void python_process_general_event(struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine __maybe_unused,
struct thread *thread,
struct addr_location *al)
{
Py_DECREF(t);
}
-static void python_process_event(union perf_event *perf_event,
+static void python_process_event(union perf_event *event __maybe_unused,
struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine,
struct thread *thread,
struct addr_location *al)
{
switch (evsel->attr.type) {
case PERF_TYPE_TRACEPOINT:
- python_process_tracepoint(perf_event, sample, evsel,
- machine, thread, al);
+ python_process_tracepoint(sample, evsel, thread, al);
break;
/* Reserve for future process_hw/sw/raw APIs */
default:
- python_process_general_event(perf_event, sample, evsel,
- machine, thread, al);
+ python_process_general_event(sample, evsel, thread, al);
}
}
static void perf_session_env__delete(struct perf_session_env *env)
{
- free(env->hostname);
- free(env->os_release);
- free(env->version);
- free(env->arch);
- free(env->cpu_desc);
- free(env->cpuid);
+ zfree(&env->hostname);
+ zfree(&env->os_release);
+ zfree(&env->version);
+ zfree(&env->arch);
+ zfree(&env->cpu_desc);
+ zfree(&env->cpuid);
- free(env->cmdline);
- free(env->sibling_cores);
- free(env->sibling_threads);
- free(env->numa_nodes);
- free(env->pmu_mappings);
+ zfree(&env->cmdline);
+ zfree(&env->sibling_cores);
+ zfree(&env->sibling_threads);
+ zfree(&env->numa_nodes);
+ zfree(&env->pmu_mappings);
}
void perf_session__delete(struct perf_session *session)
}
}
-void mem_bswap_32(void *src, int byte_size)
-{
- u32 *m = src;
- while (byte_size > 0) {
- *m = bswap_32(*m);
- byte_size -= sizeof(u32);
- ++m;
- }
-}
-
-void mem_bswap_64(void *src, int byte_size)
-{
- u64 *m = src;
-
- while (byte_size > 0) {
- *m = bswap_64(*m);
- byte_size -= sizeof(u64);
- ++m;
- }
-}
-
static void swap_sample_id_all(union perf_event *event, void *data)
{
void *end = (void *) event + event->header.size;
struct perf_sample *sample)
{
const u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
+ struct machine *machine;
if (perf_guest &&
((cpumode == PERF_RECORD_MISC_GUEST_KERNEL) ||
else
pid = sample->pid;
- return perf_session__findnew_machine(session, pid);
+ machine = perf_session__find_machine(session, pid);
+ if (!machine)
+ machine = perf_session__findnew_machine(session,
+ DEFAULT_GUEST_KERNEL_ID);
+ return machine;
}
return &session->machines.host;
void *buf = NULL;
int skip = 0;
u64 head;
- int err;
+ ssize_t err;
void *p;
perf_tool__fill_defaults(tool);
{
struct perf_evsel *evsel;
- list_for_each_entry(evsel, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, evsel) {
if (evsel->attr.type == PERF_TYPE_TRACEPOINT)
return true;
}
ret += events_stats__fprintf(&session->stats, fp);
- list_for_each_entry(pos, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, pos) {
ret += fprintf(fp, "%s stats:\n", perf_evsel__name(pos));
ret += events_stats__fprintf(&pos->hists.stats, fp);
}
{
struct perf_evsel *pos;
- list_for_each_entry(pos, &session->evlist->entries, node) {
+ evlist__for_each(session->evlist, pos) {
if (pos->attr.type == type)
return pos;
}
return NULL;
}
-void perf_evsel__print_ip(struct perf_evsel *evsel, union perf_event *event,
- struct perf_sample *sample, struct machine *machine,
+void perf_evsel__print_ip(struct perf_evsel *evsel, struct perf_sample *sample,
+ struct addr_location *al,
unsigned int print_opts, unsigned int stack_depth)
{
- struct addr_location al;
struct callchain_cursor_node *node;
int print_ip = print_opts & PRINT_IP_OPT_IP;
int print_sym = print_opts & PRINT_IP_OPT_SYM;
int print_dso = print_opts & PRINT_IP_OPT_DSO;
int print_symoffset = print_opts & PRINT_IP_OPT_SYMOFFSET;
int print_oneline = print_opts & PRINT_IP_OPT_ONELINE;
+ int print_srcline = print_opts & PRINT_IP_OPT_SRCLINE;
char s = print_oneline ? ' ' : '\t';
- if (perf_event__preprocess_sample(event, machine, &al, sample) < 0) {
- error("problem processing %d event, skipping it.\n",
- event->header.type);
- return;
- }
-
if (symbol_conf.use_callchain && sample->callchain) {
+ struct addr_location node_al;
- if (machine__resolve_callchain(machine, evsel, al.thread,
+ if (machine__resolve_callchain(al->machine, evsel, al->thread,
sample, NULL, NULL,
PERF_MAX_STACK_DEPTH) != 0) {
if (verbose)
}
callchain_cursor_commit(&callchain_cursor);
+ if (print_symoffset)
+ node_al = *al;
+
while (stack_depth) {
+ u64 addr = 0;
+
node = callchain_cursor_current(&callchain_cursor);
if (!node)
break;
+ if (node->sym && node->sym->ignore)
+ goto next;
+
if (print_ip)
printf("%c%16" PRIx64, s, node->ip);
+ if (node->map)
+ addr = node->map->map_ip(node->map, node->ip);
+
if (print_sym) {
printf(" ");
if (print_symoffset) {
- al.addr = node->ip;
- al.map = node->map;
- symbol__fprintf_symname_offs(node->sym, &al, stdout);
+ node_al.addr = addr;
+ node_al.map = node->map;
+ symbol__fprintf_symname_offs(node->sym, &node_al, stdout);
} else
symbol__fprintf_symname(node->sym, stdout);
}
printf(")");
}
+ if (print_srcline)
+ map__fprintf_srcline(node->map, addr, "\n ",
+ stdout);
+
if (!print_oneline)
printf("\n");
- callchain_cursor_advance(&callchain_cursor);
-
stack_depth--;
+next:
+ callchain_cursor_advance(&callchain_cursor);
}
} else {
+ if (al->sym && al->sym->ignore)
+ return;
+
if (print_ip)
printf("%16" PRIx64, sample->ip);
if (print_sym) {
printf(" ");
if (print_symoffset)
- symbol__fprintf_symname_offs(al.sym, &al,
+ symbol__fprintf_symname_offs(al->sym, al,
stdout);
else
- symbol__fprintf_symname(al.sym, stdout);
+ symbol__fprintf_symname(al->sym, stdout);
}
if (print_dso) {
printf(" (");
- map__fprintf_dsoname(al.map, stdout);
+ map__fprintf_dsoname(al->map, stdout);
printf(")");
}
+
+ if (print_srcline)
+ map__fprintf_srcline(al->map, al->addr, "\n ", stdout);
}
}
#ifndef __PERF_SESSION_H
#define __PERF_SESSION_H
+#include "trace-event.h"
#include "hist.h"
#include "event.h"
#include "header.h"
struct perf_header header;
struct machines machines;
struct perf_evlist *evlist;
- struct pevent *pevent;
+ struct trace_event tevent;
struct events_stats stats;
bool repipe;
struct ordered_samples ordered_samples;
#define PRINT_IP_OPT_DSO (1<<2)
#define PRINT_IP_OPT_SYMOFFSET (1<<3)
#define PRINT_IP_OPT_ONELINE (1<<4)
+#define PRINT_IP_OPT_SRCLINE (1<<5)
struct perf_tool;
bool perf_session__has_traces(struct perf_session *session, const char *msg);
-void mem_bswap_64(void *src, int byte_size);
-void mem_bswap_32(void *src, int byte_size);
void perf_event__attr_swap(struct perf_event_attr *attr);
int perf_session__create_kernel_maps(struct perf_session *session);
struct perf_evsel *perf_session__find_first_evtype(struct perf_session *session,
unsigned int type);
-void perf_evsel__print_ip(struct perf_evsel *evsel, union perf_event *event,
- struct perf_sample *sample, struct machine *machine,
+void perf_evsel__print_ip(struct perf_evsel *evsel, struct perf_sample *sample,
+ struct addr_location *al,
unsigned int print_opts, unsigned int stack_depth);
int perf_session__cpu_bitmap(struct perf_session *session,
build_lib = getenv('PYTHON_EXTBUILD_LIB')
build_tmp = getenv('PYTHON_EXTBUILD_TMP')
libtraceevent = getenv('LIBTRACEEVENT')
-liblk = getenv('LIBLK')
+libapikfs = getenv('LIBAPIKFS')
ext_sources = [f.strip() for f in file('util/python-ext-sources')
if len(f.strip()) > 0 and f[0] != '#']
sources = ext_sources,
include_dirs = ['util/include'],
extra_compile_args = cflags,
- extra_objects = [libtraceevent, liblk],
+ extra_objects = [libtraceevent, libapikfs],
)
setup(name='perf',
int sort__need_collapse = 0;
int sort__has_parent = 0;
int sort__has_sym = 0;
+int sort__has_dso = 0;
enum sort_mode sort__mode = SORT_MODE__NORMAL;
enum sort_type sort__first_dimension;
/* --sort symbol */
+static int64_t _sort__addr_cmp(u64 left_ip, u64 right_ip)
+{
+ return (int64_t)(right_ip - left_ip);
+}
+
static int64_t _sort__sym_cmp(struct symbol *sym_l, struct symbol *sym_r)
{
u64 ip_l, ip_r;
int64_t ret;
if (!left->ms.sym && !right->ms.sym)
- return right->level - left->level;
+ return _sort__addr_cmp(left->ip, right->ip);
/*
* comparing symbol address alone is not enough since it's a
* relative address within a dso.
*/
- ret = sort__dso_cmp(left, right);
- if (ret != 0)
- return ret;
+ if (!sort__has_dso) {
+ ret = sort__dso_cmp(left, right);
+ if (ret != 0)
+ return ret;
+ }
return _sort__sym_cmp(left->ms.sym, right->ms.sym);
}
struct addr_map_symbol *from_r = &right->branch_info->from;
if (!from_l->sym && !from_r->sym)
- return right->level - left->level;
+ return _sort__addr_cmp(from_l->addr, from_r->addr);
return _sort__sym_cmp(from_l->sym, from_r->sym);
}
struct addr_map_symbol *to_r = &right->branch_info->to;
if (!to_l->sym && !to_r->sym)
- return right->level - left->level;
+ return _sort__addr_cmp(to_l->addr, to_r->addr);
return _sort__sym_cmp(to_l->sym, to_r->sym);
}
sort__has_parent = 1;
} else if (sd->entry == &sort_sym) {
sort__has_sym = 1;
+ } else if (sd->entry == &sort_dso) {
+ sort__has_dso = 1;
}
__sort_dimension__add(sd, i);
out:
if (a2l) {
- free((void *)a2l->input);
+ zfree((char **)&a2l->input);
free(a2l);
}
bfd_close(abfd);
{
if (a2l->abfd)
bfd_close(a2l->abfd);
- free((void *)a2l->input);
- free(a2l->syms);
+ zfree((char **)&a2l->input);
+ zfree(&a2l->syms);
free(a2l);
}
static int addr2line(const char *dso_name, unsigned long addr,
- char **file, unsigned int *line)
+ char **file, unsigned int *line, struct dso *dso)
{
int ret = 0;
- struct a2l_data *a2l;
+ struct a2l_data *a2l = dso->a2l;
+
+ if (!a2l) {
+ dso->a2l = addr2line_init(dso_name);
+ a2l = dso->a2l;
+ }
- a2l = addr2line_init(dso_name);
if (a2l == NULL) {
pr_warning("addr2line_init failed for %s\n", dso_name);
return 0;
}
a2l->addr = addr;
+ a2l->found = false;
+
bfd_map_over_sections(a2l->abfd, find_address_in_section, a2l);
if (a2l->found && a2l->filename) {
ret = 1;
}
- addr2line_cleanup(a2l);
return ret;
}
+void dso__free_a2l(struct dso *dso)
+{
+ struct a2l_data *a2l = dso->a2l;
+
+ if (!a2l)
+ return;
+
+ addr2line_cleanup(a2l);
+
+ dso->a2l = NULL;
+}
+
#else /* HAVE_LIBBFD_SUPPORT */
static int addr2line(const char *dso_name, unsigned long addr,
- char **file, unsigned int *line_nr)
+ char **file, unsigned int *line_nr,
+ struct dso *dso __maybe_unused)
{
FILE *fp;
char cmd[PATH_MAX];
pclose(fp);
return ret;
}
+
+void dso__free_a2l(struct dso *dso __maybe_unused)
+{
+}
+
#endif /* HAVE_LIBBFD_SUPPORT */
+/*
+ * Number of addr2line failures (without success) before disabling it for that
+ * dso.
+ */
+#define A2L_FAIL_LIMIT 123
+
char *get_srcline(struct dso *dso, unsigned long addr)
{
char *file = NULL;
unsigned line = 0;
char *srcline;
- char *dso_name = dso->long_name;
- size_t size;
+ const char *dso_name;
if (!dso->has_srcline)
return SRCLINE_UNKNOWN;
+ if (dso->symsrc_filename)
+ dso_name = dso->symsrc_filename;
+ else
+ dso_name = dso->long_name;
+
if (dso_name[0] == '[')
goto out;
if (!strncmp(dso_name, "/tmp/perf-", 10))
goto out;
- if (!addr2line(dso_name, addr, &file, &line))
+ if (!addr2line(dso_name, addr, &file, &line, dso))
goto out;
- /* just calculate actual length */
- size = snprintf(NULL, 0, "%s:%u", file, line) + 1;
+ if (asprintf(&srcline, "%s:%u", file, line) < 0) {
+ free(file);
+ goto out;
+ }
- srcline = malloc(size);
- if (srcline)
- snprintf(srcline, size, "%s:%u", file, line);
- else
- srcline = SRCLINE_UNKNOWN;
+ dso->a2l_fails = 0;
free(file);
return srcline;
out:
- dso->has_srcline = 0;
+ if (dso->a2l_fails && ++dso->a2l_fails > A2L_FAIL_LIMIT) {
+ dso->has_srcline = 0;
+ dso__free_a2l(dso);
+ }
return SRCLINE_UNKNOWN;
}
void strbuf_release(struct strbuf *sb)
{
if (sb->alloc) {
- free(sb->buf);
+ zfree(&sb->buf);
strbuf_init(sb, 0);
}
}
{
if (node) {
if (node->p && !is_operator(*node->p))
- free((char *)node->p);
+ zfree((char **)&node->p);
strfilter_node__delete(node->l);
strfilter_node__delete(node->r);
free(node);
{
char **p;
for (p = argv; *p; p++)
- free(*p);
+ zfree(p);
free(argv);
}
*/
#include "strlist.h"
+#include "util.h"
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
static void str_node__delete(struct str_node *snode, bool dupstr)
{
if (dupstr)
- free((void *)snode->s);
+ zfree((char **)&snode->s);
free(snode);
}
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
+#include <linux/bitops.h>
+#include "perf.h"
#include "svghelper.h"
+#include "util.h"
+#include "cpumap.h"
static u64 first_time, last_time;
static u64 turbo_frequency, max_freq;
#define SLOT_HEIGHT 25.0
int svg_page_width = 1000;
+u64 svg_highlight;
+const char *svg_highlight_name;
#define MIN_TEXT_SIZE 0.01
return 2 * cpu + 1;
}
+static int *topology_map;
+
static double cpu2y(int cpu)
{
- return cpu2slot(cpu) * SLOT_MULT;
+ if (topology_map)
+ return cpu2slot(topology_map[cpu]) * SLOT_MULT;
+ else
+ return cpu2slot(cpu) * SLOT_MULT;
}
static double time2pixels(u64 __time)
total_height = (1 + rows + cpu2slot(cpus)) * SLOT_MULT;
fprintf(svgfile, "<?xml version=\"1.0\" standalone=\"no\"?> \n");
+ fprintf(svgfile, "<!DOCTYPE svg SYSTEM \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n");
fprintf(svgfile, "<svg width=\"%i\" height=\"%" PRIu64 "\" version=\"1.1\" xmlns=\"http://www.w3.org/2000/svg\">\n", svg_page_width, total_height);
fprintf(svgfile, "<defs>\n <style type=\"text/css\">\n <![CDATA[\n");
fprintf(svgfile, " rect.process { fill:rgb(180,180,180); fill-opacity:0.9; stroke-width:1; stroke:rgb( 0, 0, 0); } \n");
fprintf(svgfile, " rect.process2 { fill:rgb(180,180,180); fill-opacity:0.9; stroke-width:0; stroke:rgb( 0, 0, 0); } \n");
fprintf(svgfile, " rect.sample { fill:rgb( 0, 0,255); fill-opacity:0.8; stroke-width:0; stroke:rgb( 0, 0, 0); } \n");
+ fprintf(svgfile, " rect.sample_hi{ fill:rgb(255,128, 0); fill-opacity:0.8; stroke-width:0; stroke:rgb( 0, 0, 0); } \n");
fprintf(svgfile, " rect.blocked { fill:rgb(255, 0, 0); fill-opacity:0.5; stroke-width:0; stroke:rgb( 0, 0, 0); } \n");
fprintf(svgfile, " rect.waiting { fill:rgb(224,214, 0); fill-opacity:0.8; stroke-width:0; stroke:rgb( 0, 0, 0); } \n");
fprintf(svgfile, " rect.WAITING { fill:rgb(255,214, 48); fill-opacity:0.6; stroke-width:0; stroke:rgb( 0, 0, 0); } \n");
time2pixels(start), time2pixels(end)-time2pixels(start), Yslot * SLOT_MULT, SLOT_HEIGHT, type);
}
-void svg_sample(int Yslot, int cpu, u64 start, u64 end)
+static char *time_to_string(u64 duration);
+void svg_blocked(int Yslot, int cpu, u64 start, u64 end, const char *backtrace)
+{
+ if (!svgfile)
+ return;
+
+ fprintf(svgfile, "<g>\n");
+ fprintf(svgfile, "<title>#%d blocked %s</title>\n", cpu,
+ time_to_string(end - start));
+ if (backtrace)
+ fprintf(svgfile, "<desc>Blocked on:\n%s</desc>\n", backtrace);
+ svg_box(Yslot, start, end, "blocked");
+ fprintf(svgfile, "</g>\n");
+}
+
+void svg_running(int Yslot, int cpu, u64 start, u64 end, const char *backtrace)
{
double text_size;
+ const char *type;
+
if (!svgfile)
return;
- fprintf(svgfile, "<rect x=\"%4.8f\" width=\"%4.8f\" y=\"%4.1f\" height=\"%4.1f\" class=\"sample\"/>\n",
- time2pixels(start), time2pixels(end)-time2pixels(start), Yslot * SLOT_MULT, SLOT_HEIGHT);
+ if (svg_highlight && end - start > svg_highlight)
+ type = "sample_hi";
+ else
+ type = "sample";
+ fprintf(svgfile, "<g>\n");
+
+ fprintf(svgfile, "<title>#%d running %s</title>\n",
+ cpu, time_to_string(end - start));
+ if (backtrace)
+ fprintf(svgfile, "<desc>Switched because:\n%s</desc>\n", backtrace);
+ fprintf(svgfile, "<rect x=\"%4.8f\" width=\"%4.8f\" y=\"%4.1f\" height=\"%4.1f\" class=\"%s\"/>\n",
+ time2pixels(start), time2pixels(end)-time2pixels(start), Yslot * SLOT_MULT, SLOT_HEIGHT,
+ type);
text_size = (time2pixels(end)-time2pixels(start));
if (cpu > 9)
fprintf(svgfile, "<text x=\"%1.8f\" y=\"%1.8f\" font-size=\"%1.8fpt\">%i</text>\n",
time2pixels(start), Yslot * SLOT_MULT + SLOT_HEIGHT - 1, text_size, cpu + 1);
+ fprintf(svgfile, "</g>\n");
}
static char *time_to_string(u64 duration)
return text;
}
-void svg_waiting(int Yslot, u64 start, u64 end)
+void svg_waiting(int Yslot, int cpu, u64 start, u64 end, const char *backtrace)
{
char *text;
const char *style;
font_size = round_text_size(font_size);
fprintf(svgfile, "<g transform=\"translate(%4.8f,%4.8f)\">\n", time2pixels(start), Yslot * SLOT_MULT);
+ fprintf(svgfile, "<title>#%d waiting %s</title>\n", cpu, time_to_string(end - start));
+ if (backtrace)
+ fprintf(svgfile, "<desc>Waiting on:\n%s</desc>\n", backtrace);
fprintf(svgfile, "<rect x=\"0\" width=\"%4.8f\" y=\"0\" height=\"%4.1f\" class=\"%s\"/>\n",
time2pixels(end)-time2pixels(start), SLOT_HEIGHT, style);
if (font_size > MIN_TEXT_SIZE)
max_freq = __max_freq;
turbo_frequency = __turbo_freq;
+ fprintf(svgfile, "<g>\n");
+
fprintf(svgfile, "<rect x=\"%4.8f\" width=\"%4.8f\" y=\"%4.1f\" height=\"%4.1f\" class=\"cpu\"/>\n",
time2pixels(first_time),
time2pixels(last_time)-time2pixels(first_time),
cpu2y(cpu), SLOT_MULT+SLOT_HEIGHT);
- sprintf(cpu_string, "CPU %i", (int)cpu+1);
+ sprintf(cpu_string, "CPU %i", (int)cpu);
fprintf(svgfile, "<text x=\"%4.8f\" y=\"%4.8f\">%s</text>\n",
10+time2pixels(first_time), cpu2y(cpu) + SLOT_HEIGHT/2, cpu_string);
fprintf(svgfile, "<text transform=\"translate(%4.8f,%4.8f)\" font-size=\"1.25pt\">%s</text>\n",
10+time2pixels(first_time), cpu2y(cpu) + SLOT_MULT + SLOT_HEIGHT - 4, cpu_model());
+
+ fprintf(svgfile, "</g>\n");
}
-void svg_process(int cpu, u64 start, u64 end, const char *type, const char *name)
+void svg_process(int cpu, u64 start, u64 end, int pid, const char *name, const char *backtrace)
{
double width;
+ const char *type;
if (!svgfile)
return;
+ if (svg_highlight && end - start >= svg_highlight)
+ type = "sample_hi";
+ else if (svg_highlight_name && strstr(name, svg_highlight_name))
+ type = "sample_hi";
+ else
+ type = "sample";
fprintf(svgfile, "<g transform=\"translate(%4.8f,%4.8f)\">\n", time2pixels(start), cpu2y(cpu));
+ fprintf(svgfile, "<title>%d %s running %s</title>\n", pid, name, time_to_string(end - start));
+ if (backtrace)
+ fprintf(svgfile, "<desc>Switched because:\n%s</desc>\n", backtrace);
fprintf(svgfile, "<rect x=\"0\" width=\"%4.8f\" y=\"0\" height=\"%4.1f\" class=\"%s\"/>\n",
time2pixels(end)-time2pixels(start), SLOT_MULT+SLOT_HEIGHT, type);
width = time2pixels(end)-time2pixels(start);
return;
+ fprintf(svgfile, "<g>\n");
+
if (type > 6)
type = 6;
sprintf(style, "c%i", type);
if (width > MIN_TEXT_SIZE)
fprintf(svgfile, "<text x=\"%4.8f\" y=\"%4.8f\" font-size=\"%3.8fpt\">C%i</text>\n",
time2pixels(start), cpu2y(cpu)+width, width, type);
+
+ fprintf(svgfile, "</g>\n");
}
static char *HzToHuman(unsigned long hz)
if (!svgfile)
return;
+ fprintf(svgfile, "<g>\n");
+
if (max_freq)
height = freq * 1.0 / max_freq * (SLOT_HEIGHT + SLOT_MULT);
height = 1 + cpu2y(cpu) + SLOT_MULT + SLOT_HEIGHT - height;
fprintf(svgfile, "<text x=\"%4.8f\" y=\"%4.8f\" font-size=\"0.25pt\">%s</text>\n",
time2pixels(start), height+0.9, HzToHuman(freq));
+ fprintf(svgfile, "</g>\n");
}
-void svg_partial_wakeline(u64 start, int row1, char *desc1, int row2, char *desc2)
+void svg_partial_wakeline(u64 start, int row1, char *desc1, int row2, char *desc2, const char *backtrace)
{
double height;
return;
+ fprintf(svgfile, "<g>\n");
+
+ fprintf(svgfile, "<title>%s wakes up %s</title>\n",
+ desc1 ? desc1 : "?",
+ desc2 ? desc2 : "?");
+
+ if (backtrace)
+ fprintf(svgfile, "<desc>%s</desc>\n", backtrace);
+
if (row1 < row2) {
if (row1) {
fprintf(svgfile, "<line x1=\"%4.8f\" y1=\"%4.2f\" x2=\"%4.8f\" y2=\"%4.2f\" style=\"stroke:rgb(32,255,32);stroke-width:0.009\"/>\n",
if (row1)
fprintf(svgfile, "<circle cx=\"%4.8f\" cy=\"%4.2f\" r = \"0.01\" style=\"fill:rgb(32,255,32)\"/>\n",
time2pixels(start), height);
+
+ fprintf(svgfile, "</g>\n");
}
-void svg_wakeline(u64 start, int row1, int row2)
+void svg_wakeline(u64 start, int row1, int row2, const char *backtrace)
{
double height;
return;
+ fprintf(svgfile, "<g>\n");
+
+ if (backtrace)
+ fprintf(svgfile, "<desc>%s</desc>\n", backtrace);
+
if (row1 < row2)
fprintf(svgfile, "<line x1=\"%4.8f\" y1=\"%4.2f\" x2=\"%4.8f\" y2=\"%4.2f\" style=\"stroke:rgb(32,255,32);stroke-width:0.009\"/>\n",
time2pixels(start), row1 * SLOT_MULT + SLOT_HEIGHT, time2pixels(start), row2 * SLOT_MULT);
height += SLOT_HEIGHT;
fprintf(svgfile, "<circle cx=\"%4.8f\" cy=\"%4.2f\" r = \"0.01\" style=\"fill:rgb(32,255,32)\"/>\n",
time2pixels(start), height);
+
+ fprintf(svgfile, "</g>\n");
}
-void svg_interrupt(u64 start, int row)
+void svg_interrupt(u64 start, int row, const char *backtrace)
{
if (!svgfile)
return;
+ fprintf(svgfile, "<g>\n");
+
+ fprintf(svgfile, "<title>Wakeup from interrupt</title>\n");
+
+ if (backtrace)
+ fprintf(svgfile, "<desc>%s</desc>\n", backtrace);
+
fprintf(svgfile, "<circle cx=\"%4.8f\" cy=\"%4.2f\" r = \"0.01\" style=\"fill:rgb(255,128,128)\"/>\n",
time2pixels(start), row * SLOT_MULT);
fprintf(svgfile, "<circle cx=\"%4.8f\" cy=\"%4.2f\" r = \"0.01\" style=\"fill:rgb(255,128,128)\"/>\n",
time2pixels(start), row * SLOT_MULT + SLOT_HEIGHT);
+
+ fprintf(svgfile, "</g>\n");
}
void svg_text(int Yslot, u64 start, const char *text)
if (!svgfile)
return;
+ fprintf(svgfile, "<g>\n");
svg_legenda_box(0, "Running", "sample");
svg_legenda_box(100, "Idle","c1");
svg_legenda_box(200, "Deeper Idle", "c3");
svg_legenda_box(550, "Sleeping", "process2");
svg_legenda_box(650, "Waiting for cpu", "waiting");
svg_legenda_box(800, "Blocked on IO", "blocked");
+ fprintf(svgfile, "</g>\n");
}
void svg_time_grid(void)
svgfile = NULL;
}
}
+
+#define cpumask_bits(maskp) ((maskp)->bits)
+typedef struct { DECLARE_BITMAP(bits, MAX_NR_CPUS); } cpumask_t;
+
+struct topology {
+ cpumask_t *sib_core;
+ int sib_core_nr;
+ cpumask_t *sib_thr;
+ int sib_thr_nr;
+};
+
+static void scan_thread_topology(int *map, struct topology *t, int cpu, int *pos)
+{
+ int i;
+ int thr;
+
+ for (i = 0; i < t->sib_thr_nr; i++) {
+ if (!test_bit(cpu, cpumask_bits(&t->sib_thr[i])))
+ continue;
+
+ for_each_set_bit(thr,
+ cpumask_bits(&t->sib_thr[i]),
+ MAX_NR_CPUS)
+ if (map[thr] == -1)
+ map[thr] = (*pos)++;
+ }
+}
+
+static void scan_core_topology(int *map, struct topology *t)
+{
+ int pos = 0;
+ int i;
+ int cpu;
+
+ for (i = 0; i < t->sib_core_nr; i++)
+ for_each_set_bit(cpu,
+ cpumask_bits(&t->sib_core[i]),
+ MAX_NR_CPUS)
+ scan_thread_topology(map, t, cpu, &pos);
+}
+
+static int str_to_bitmap(char *s, cpumask_t *b)
+{
+ int i;
+ int ret = 0;
+ struct cpu_map *m;
+ int c;
+
+ m = cpu_map__new(s);
+ if (!m)
+ return -1;
+
+ for (i = 0; i < m->nr; i++) {
+ c = m->map[i];
+ if (c >= MAX_NR_CPUS) {
+ ret = -1;
+ break;
+ }
+
+ set_bit(c, cpumask_bits(b));
+ }
+
+ cpu_map__delete(m);
+
+ return ret;
+}
+
+int svg_build_topology_map(char *sib_core, int sib_core_nr,
+ char *sib_thr, int sib_thr_nr)
+{
+ int i;
+ struct topology t;
+
+ t.sib_core_nr = sib_core_nr;
+ t.sib_thr_nr = sib_thr_nr;
+ t.sib_core = calloc(sib_core_nr, sizeof(cpumask_t));
+ t.sib_thr = calloc(sib_thr_nr, sizeof(cpumask_t));
+
+ if (!t.sib_core || !t.sib_thr) {
+ fprintf(stderr, "topology: no memory\n");
+ goto exit;
+ }
+
+ for (i = 0; i < sib_core_nr; i++) {
+ if (str_to_bitmap(sib_core, &t.sib_core[i])) {
+ fprintf(stderr, "topology: can't parse siblings map\n");
+ goto exit;
+ }
+
+ sib_core += strlen(sib_core) + 1;
+ }
+
+ for (i = 0; i < sib_thr_nr; i++) {
+ if (str_to_bitmap(sib_thr, &t.sib_thr[i])) {
+ fprintf(stderr, "topology: can't parse siblings map\n");
+ goto exit;
+ }
+
+ sib_thr += strlen(sib_thr) + 1;
+ }
+
+ topology_map = malloc(sizeof(int) * MAX_NR_CPUS);
+ if (!topology_map) {
+ fprintf(stderr, "topology: no memory\n");
+ goto exit;
+ }
+
+ for (i = 0; i < MAX_NR_CPUS; i++)
+ topology_map[i] = -1;
+
+ scan_core_topology(topology_map, &t);
+
+ return 0;
+
+exit:
+ zfree(&t.sib_core);
+ zfree(&t.sib_thr);
+
+ return -1;
+}
extern void open_svg(const char *filename, int cpus, int rows, u64 start, u64 end);
extern void svg_box(int Yslot, u64 start, u64 end, const char *type);
-extern void svg_sample(int Yslot, int cpu, u64 start, u64 end);
-extern void svg_waiting(int Yslot, u64 start, u64 end);
+extern void svg_blocked(int Yslot, int cpu, u64 start, u64 end, const char *backtrace);
+extern void svg_running(int Yslot, int cpu, u64 start, u64 end, const char *backtrace);
+extern void svg_waiting(int Yslot, int cpu, u64 start, u64 end, const char *backtrace);
extern void svg_cpu_box(int cpu, u64 max_frequency, u64 turbo_frequency);
-extern void svg_process(int cpu, u64 start, u64 end, const char *type, const char *name);
+extern void svg_process(int cpu, u64 start, u64 end, int pid, const char *name, const char *backtrace);
extern void svg_cstate(int cpu, u64 start, u64 end, int type);
extern void svg_pstate(int cpu, u64 start, u64 end, u64 freq);
extern void svg_time_grid(void);
extern void svg_legenda(void);
-extern void svg_wakeline(u64 start, int row1, int row2);
-extern void svg_partial_wakeline(u64 start, int row1, char *desc1, int row2, char *desc2);
-extern void svg_interrupt(u64 start, int row);
+extern void svg_wakeline(u64 start, int row1, int row2, const char *backtrace);
+extern void svg_partial_wakeline(u64 start, int row1, char *desc1, int row2, char *desc2, const char *backtrace);
+extern void svg_interrupt(u64 start, int row, const char *backtrace);
extern void svg_text(int Yslot, u64 start, const char *text);
extern void svg_close(void);
+extern int svg_build_topology_map(char *sib_core, int sib_core_nr,
+ char *sib_thr, int sib_thr_nr);
extern int svg_page_width;
+extern u64 svg_highlight;
+extern const char *svg_highlight_name;
#endif /* __PERF_SVGHELPER_H */
#include <inttypes.h>
#include "symbol.h"
+#include <symbol/kallsyms.h>
#include "debug.h"
#ifndef HAVE_ELF_GETPHDRNUM_SUPPORT
return -1;
}
-static Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
- GElf_Shdr *shp, const char *name,
- size_t *idx)
+Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
+ GElf_Shdr *shp, const char *name, size_t *idx)
{
Elf_Scn *sec = NULL;
size_t cnt = 1;
void symsrc__destroy(struct symsrc *ss)
{
- free(ss->name);
+ zfree(&ss->name);
elf_end(ss->elf);
close(ss->fd);
}
#include "symbol.h"
+#include "util.h"
#include <stdio.h>
#include <fcntl.h>
if (!ss->name)
goto out_close;
+ ss->fd = fd;
ss->type = type;
return 0;
void symsrc__destroy(struct symsrc *ss)
{
- free(ss->name);
+ zfree(&ss->name);
close(ss->fd);
}
#include <elf.h>
#include <limits.h>
+#include <symbol/kallsyms.h>
#include <sys/utsname.h>
-#ifndef KSYM_NAME_LEN
-#define KSYM_NAME_LEN 256
-#endif
-
static int dso__load_kernel_sym(struct dso *dso, struct map *map,
symbol_filter_t filter);
static int dso__load_guest_kernel_sym(struct dso *dso, struct map *map,
return ret;
}
-int kallsyms__parse(const char *filename, void *arg,
- int (*process_symbol)(void *arg, const char *name,
- char type, u64 start))
-{
- char *line = NULL;
- size_t n;
- int err = -1;
- FILE *file = fopen(filename, "r");
-
- if (file == NULL)
- goto out_failure;
-
- err = 0;
-
- while (!feof(file)) {
- u64 start;
- int line_len, len;
- char symbol_type;
- char *symbol_name;
-
- line_len = getline(&line, &n, file);
- if (line_len < 0 || !line)
- break;
-
- line[--line_len] = '\0'; /* \n */
-
- len = hex2u64(line, &start);
-
- len++;
- if (len + 2 >= line_len)
- continue;
-
- symbol_type = line[len];
- len += 2;
- symbol_name = line + len;
- len = line_len - len;
-
- if (len >= KSYM_NAME_LEN) {
- err = -1;
- break;
- }
-
- err = process_symbol(arg, symbol_name,
- symbol_type, start);
- if (err)
- break;
- }
-
- free(line);
- fclose(file);
- return err;
-
-out_failure:
- return -1;
-}
-
int modules__parse(const char *filename, void *arg,
int (*process_module)(void *arg, const char *name,
u64 start))
struct dso *dso;
};
-static u8 kallsyms2elf_type(char type)
+bool symbol__is_idle(struct symbol *sym)
{
- if (type == 'W')
- return STB_WEAK;
+ const char * const idle_symbols[] = {
+ "cpu_idle",
+ "intel_idle",
+ "default_idle",
+ "native_safe_halt",
+ "enter_idle",
+ "exit_idle",
+ "mwait_idle",
+ "mwait_idle_with_hints",
+ "poll_idle",
+ "ppc64_runlatch_off",
+ "pseries_dedicated_idle_sleep",
+ NULL
+ };
+
+ int i;
+
+ if (!sym)
+ return false;
+
+ for (i = 0; idle_symbols[i]; i++) {
+ if (!strcmp(idle_symbols[i], sym->name))
+ return true;
+ }
- return isupper(type) ? STB_GLOBAL : STB_LOCAL;
+ return false;
}
static int map__process_kallsym_symbol(void *arg, const char *name,
mi = rb_entry(next, struct module_info, rb_node);
next = rb_next(&mi->rb_node);
rb_erase(&mi->rb_node, modules);
- free(mi->name);
+ zfree(&mi->name);
free(mi);
}
}
* dso__data_read_addr().
*/
if (dso->kernel == DSO_TYPE_GUEST_KERNEL)
- dso->data_type = DSO_BINARY_TYPE__GUEST_KCORE;
+ dso->binary_type = DSO_BINARY_TYPE__GUEST_KCORE;
else
- dso->data_type = DSO_BINARY_TYPE__KCORE;
- dso__set_long_name(dso, strdup(kcore_filename));
+ dso->binary_type = DSO_BINARY_TYPE__KCORE;
+ dso__set_long_name(dso, strdup(kcore_filename), true);
close(fd);
enum dso_binary_type symtab_type = binary_type_symtab[i];
- if (dso__binary_type_file(dso, symtab_type,
- root_dir, name, PATH_MAX))
+ if (dso__read_binary_type_filename(dso, symtab_type,
+ root_dir, name, PATH_MAX))
continue;
/* Name is now the name of the next image to try */
if (!syms_ss && symsrc__has_symtab(ss)) {
syms_ss = ss;
next_slot = true;
+ if (!dso->symsrc_filename)
+ dso->symsrc_filename = strdup(name);
}
if (!runtime_ss && symsrc__possibly_runtime(ss)) {
}
int dso__load_vmlinux(struct dso *dso, struct map *map,
- const char *vmlinux, symbol_filter_t filter)
+ const char *vmlinux, bool vmlinux_allocated,
+ symbol_filter_t filter)
{
int err = -1;
struct symsrc ss;
if (err > 0) {
if (dso->kernel == DSO_TYPE_GUEST_KERNEL)
- dso->data_type = DSO_BINARY_TYPE__GUEST_VMLINUX;
+ dso->binary_type = DSO_BINARY_TYPE__GUEST_VMLINUX;
else
- dso->data_type = DSO_BINARY_TYPE__VMLINUX;
- dso__set_long_name(dso, (char *)vmlinux);
+ dso->binary_type = DSO_BINARY_TYPE__VMLINUX;
+ dso__set_long_name(dso, vmlinux, vmlinux_allocated);
dso__set_loaded(dso, map->type);
pr_debug("Using %s for symbols\n", symfs_vmlinux);
}
filename = dso__build_id_filename(dso, NULL, 0);
if (filename != NULL) {
- err = dso__load_vmlinux(dso, map, filename, filter);
- if (err > 0) {
- dso->lname_alloc = 1;
+ err = dso__load_vmlinux(dso, map, filename, true, filter);
+ if (err > 0)
goto out;
- }
free(filename);
}
for (i = 0; i < vmlinux_path__nr_entries; ++i) {
- err = dso__load_vmlinux(dso, map, vmlinux_path[i], filter);
- if (err > 0) {
- dso__set_long_name(dso, strdup(vmlinux_path[i]));
- dso->lname_alloc = 1;
+ err = dso__load_vmlinux(dso, map, vmlinux_path[i], false, filter);
+ if (err > 0)
break;
- }
}
out:
return err;
build_id__sprintf(dso->build_id, sizeof(dso->build_id), sbuild_id);
+ scnprintf(path, sizeof(path), "%s/[kernel.kcore]/%s", buildid_dir,
+ sbuild_id);
+
/* Use /proc/kallsyms if possible */
if (is_host) {
DIR *d;
int fd;
/* If no cached kcore go with /proc/kallsyms */
- scnprintf(path, sizeof(path), "%s/[kernel.kcore]/%s",
- buildid_dir, sbuild_id);
d = opendir(path);
if (!d)
goto proc_kallsyms;
goto proc_kallsyms;
}
+ /* Find kallsyms in build-id cache with kcore */
+ if (!find_matching_kcore(map, path, sizeof(path)))
+ return strdup(path);
+
scnprintf(path, sizeof(path), "%s/[kernel.kallsyms]/%s",
buildid_dir, sbuild_id);
}
if (!symbol_conf.ignore_vmlinux && symbol_conf.vmlinux_name != NULL) {
- err = dso__load_vmlinux(dso, map,
- symbol_conf.vmlinux_name, filter);
- if (err > 0) {
- dso__set_long_name(dso,
- strdup(symbol_conf.vmlinux_name));
- dso->lname_alloc = 1;
- return err;
- }
- return err;
+ return dso__load_vmlinux(dso, map, symbol_conf.vmlinux_name,
+ false, filter);
}
if (!symbol_conf.ignore_vmlinux && vmlinux_path != NULL) {
free(kallsyms_allocated_filename);
if (err > 0 && !dso__is_kcore(dso)) {
- dso__set_long_name(dso, strdup("[kernel.kallsyms]"));
+ dso__set_long_name(dso, "[kernel.kallsyms]", false);
map__fixup_start(map);
map__fixup_end(map);
}
*/
if (symbol_conf.default_guest_vmlinux_name != NULL) {
err = dso__load_vmlinux(dso, map,
- symbol_conf.default_guest_vmlinux_name, filter);
+ symbol_conf.default_guest_vmlinux_name,
+ false, filter);
return err;
}
pr_debug("Using %s for symbols\n", kallsyms_filename);
if (err > 0 && !dso__is_kcore(dso)) {
machine__mmap_name(machine, path, sizeof(path));
- dso__set_long_name(dso, strdup(path));
+ dso__set_long_name(dso, strdup(path), true);
map__fixup_start(map);
map__fixup_end(map);
}
static void vmlinux_path__exit(void)
{
- while (--vmlinux_path__nr_entries >= 0) {
- free(vmlinux_path[vmlinux_path__nr_entries]);
- vmlinux_path[vmlinux_path__nr_entries] = NULL;
- }
+ while (--vmlinux_path__nr_entries >= 0)
+ zfree(&vmlinux_path[vmlinux_path__nr_entries]);
- free(vmlinux_path);
- vmlinux_path = NULL;
+ zfree(&vmlinux_path);
}
static int vmlinux_path__init(void)
return -1;
}
-static int setup_list(struct strlist **list, const char *list_str,
+int setup_list(struct strlist **list, const char *list_str,
const char *list_name)
{
if (list_str == NULL)
# define PERF_ELF_C_READ_MMAP ELF_C_READ
#endif
+#ifdef HAVE_LIBELF_SUPPORT
+extern Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
+ GElf_Shdr *shp, const char *name, size_t *idx);
+#endif
+
#ifndef DMGL_PARAMS
#define DMGL_PARAMS (1 << 0) /* Include function args */
#define DMGL_ANSI (1 << 1) /* Include const, volatile, etc */
};
struct addr_location {
+ struct machine *machine;
struct thread *thread;
struct map *map;
struct symbol *sym;
int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter);
int dso__load_vmlinux(struct dso *dso, struct map *map,
- const char *vmlinux, symbol_filter_t filter);
+ const char *vmlinux, bool vmlinux_allocated,
+ symbol_filter_t filter);
int dso__load_vmlinux_path(struct dso *dso, struct map *map,
symbol_filter_t filter);
int dso__load_kallsyms(struct dso *dso, const char *filename, struct map *map,
int filename__read_build_id(const char *filename, void *bf, size_t size);
int sysfs__read_build_id(const char *filename, void *bf, size_t size);
-int kallsyms__parse(const char *filename, void *arg,
- int (*process_symbol)(void *arg, const char *name,
- char type, u64 start));
int modules__parse(const char *filename, void *arg,
int (*process_module)(void *arg, const char *name,
u64 start));
bool symbol_type__is_a(char symbol_type, enum map_type map_type);
bool symbol__restricted_filename(const char *filename,
const char *restricted_filename);
+bool symbol__is_idle(struct symbol *sym);
int dso__load_sym(struct dso *dso, struct map *map, struct symsrc *syms_ss,
struct symsrc *runtime_ss, symbol_filter_t filter,
int kcore_copy(const char *from_dir, const char *to_dir);
int compare_proc_modules(const char *from, const char *to);
+int setup_list(struct strlist **list, const char *list_str,
+ const char *list_name);
+
#endif /* __PERF_SYMBOL */
ret = TARGET_ERRNO__UID_OVERRIDE_SYSTEM;
}
+ /* THREAD and SYSTEM/CPU are mutually exclusive */
+ if (target->per_thread && (target->system_wide || target->cpu_list)) {
+ target->per_thread = false;
+ if (ret == TARGET_ERRNO__SUCCESS)
+ ret = TARGET_ERRNO__SYSTEM_OVERRIDE_THREAD;
+ }
+
return ret;
}
"UID switch overriding CPU",
"PID/TID switch overriding SYSTEM",
"UID switch overriding SYSTEM",
+ "SYSTEM/CPU switch overriding PER-THREAD",
"Invalid User: %s",
"Problems obtaining information for user %s",
};
msg = target__error_str[idx];
switch (errnum) {
- case TARGET_ERRNO__PID_OVERRIDE_CPU ... TARGET_ERRNO__UID_OVERRIDE_SYSTEM:
+ case TARGET_ERRNO__PID_OVERRIDE_CPU ...
+ TARGET_ERRNO__SYSTEM_OVERRIDE_THREAD:
snprintf(buf, buflen, "%s", msg);
break;
uid_t uid;
bool system_wide;
bool uses_mmap;
- bool force_per_cpu;
+ bool default_per_cpu;
+ bool per_thread;
};
enum target_errno {
TARGET_ERRNO__UID_OVERRIDE_CPU,
TARGET_ERRNO__PID_OVERRIDE_SYSTEM,
TARGET_ERRNO__UID_OVERRIDE_SYSTEM,
+ TARGET_ERRNO__SYSTEM_OVERRIDE_THREAD,
/* for target__parse_uid() */
TARGET_ERRNO__INVALID_UID,
return !target__has_task(target) && !target__has_cpu(target);
}
+static inline bool target__uses_dummy_map(struct target *target)
+{
+ bool use_dummy = false;
+
+ if (target->default_per_cpu)
+ use_dummy = target->per_thread ? true : false;
+ else if (target__has_task(target) ||
+ (!target__has_cpu(target) && !target->uses_mmap))
+ use_dummy = true;
+
+ return use_dummy;
+}
+
#endif /* _PERF_TARGET_H */
int thread__set_comm(struct thread *thread, const char *str, u64 timestamp)
{
struct comm *new, *curr = thread__comm(thread);
+ int err;
/* Override latest entry if it had no specific time coverage */
if (!curr->start) {
- comm__override(curr, str, timestamp);
+ err = comm__override(curr, str, timestamp);
+ if (err)
+ return err;
} else {
new = comm__new(str, timestamp);
if (!new)
if (!comm)
return -ENOMEM;
err = thread__set_comm(thread, comm, timestamp);
- if (!err)
+ if (err)
return err;
thread->comm_set = true;
}
#include <unistd.h>
#include <sys/types.h>
#include "symbol.h"
+#include <strlist.h>
struct thread {
union {
{
thread->priv = p;
}
+
+static inline bool thread__is_filtered(struct thread *thread)
+{
+ if (symbol_conf.comm_list &&
+ !strlist__has_entry(symbol_conf.comm_list, thread__comm_str(thread))) {
+ return true;
+ }
+
+ return false;
+}
+
#endif /* __PERF_THREAD_H */
#include "strlist.h"
#include <string.h>
#include "thread_map.h"
+#include "util.h"
/* Skip "." and ".." directories */
static int filter(const struct dirent *dir)
}
for (i=0; i<items; i++)
- free(namelist[i]);
+ zfree(&namelist[i]);
free(namelist);
return threads;
threads->map[threads->nr + i] = atoi(namelist[i]->d_name);
for (i = 0; i < items; i++)
- free(namelist[i]);
+ zfree(&namelist[i]);
free(namelist);
threads->nr += items;
out_free_namelist:
for (i = 0; i < items; i++)
- free(namelist[i]);
+ zfree(&namelist[i]);
free(namelist);
out_free_closedir:
- free(threads);
- threads = NULL;
+ zfree(&threads);
goto out_closedir;
}
for (i = 0; i < items; i++) {
threads->map[j++] = atoi(namelist[i]->d_name);
- free(namelist[i]);
+ zfree(&namelist[i]);
}
threads->nr = total_tasks;
free(namelist);
out_free_namelist:
for (i = 0; i < items; i++)
- free(namelist[i]);
+ zfree(&namelist[i]);
free(namelist);
out_free_threads:
- free(threads);
- threads = NULL;
+ zfree(&threads);
goto out;
}
return threads;
out_free_threads:
- free(threads);
- threads = NULL;
+ zfree(&threads);
goto out;
}
float samples_per_sec;
float ksamples_per_sec;
float esamples_percent;
- struct perf_record_opts *opts = &top->record_opts;
+ struct record_opts *opts = &top->record_opts;
struct target *target = &opts->target;
size_t ret = 0;
struct perf_top {
struct perf_tool tool;
struct perf_evlist *evlist;
- struct perf_record_opts record_opts;
+ struct record_opts record_opts;
/*
* Symbols will be added here in perf_event__process_sample and will
* get out after decayed.
#include "../perf.h"
#include "trace-event.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include "evsel.h"
#define VERSION "0.5"
struct tracepoint_path *t = tps;
tps = tps->next;
- free(t->name);
- free(t->system);
+ zfree(&t->name);
+ zfree(&t->system);
free(t);
}
}
output_fd = fd;
}
- if (err) {
- free(tdata);
- tdata = NULL;
- }
+ if (err)
+ zfree(&tdata);
put_tracepoints_path(tps);
return tdata;
#include "util.h"
#include "trace-event.h"
-struct pevent *read_trace_init(int file_bigendian, int host_bigendian)
-{
- struct pevent *pevent = pevent_alloc();
-
- if (pevent != NULL) {
- pevent_set_flag(pevent, PEVENT_NSEC_OUTPUT);
- pevent_set_file_bigendian(pevent, file_bigendian);
- pevent_set_host_bigendian(pevent, host_bigendian);
- }
-
- return pevent;
-}
-
static int get_common_field(struct scripting_context *context,
int *offset, int *size, const char *type)
{
return 0;
}
-ssize_t trace_report(int fd, struct pevent **ppevent, bool __repipe)
+ssize_t trace_report(int fd, struct trace_event *tevent, bool __repipe)
{
char buf[BUFSIZ];
char test[] = { 23, 8, 68 };
int host_bigendian;
int file_long_size;
int file_page_size;
- struct pevent *pevent;
+ struct pevent *pevent = NULL;
int err;
- *ppevent = NULL;
-
repipe = __repipe;
input_fd = fd;
file_bigendian = buf[0];
host_bigendian = bigendian();
- pevent = read_trace_init(file_bigendian, host_bigendian);
- if (pevent == NULL) {
- pr_debug("read_trace_init failed");
+ if (trace_event__init(tevent)) {
+ pr_debug("trace_event__init failed");
goto out;
}
+ pevent = tevent->pevent;
+
+ pevent_set_flag(pevent, PEVENT_NSEC_OUTPUT);
+ pevent_set_file_bigendian(pevent, file_bigendian);
+ pevent_set_host_bigendian(pevent, host_bigendian);
+
if (do_read(buf, 1) < 0)
goto out;
file_long_size = buf[0];
pevent_print_printk(pevent);
}
- *ppevent = pevent;
pevent = NULL;
out:
if (pevent)
- pevent_free(pevent);
+ trace_event__cleanup(tevent);
return size;
}
static void process_event_unsupported(union perf_event *event __maybe_unused,
struct perf_sample *sample __maybe_unused,
struct perf_evsel *evsel __maybe_unused,
- struct machine *machine __maybe_unused,
struct thread *thread __maybe_unused,
- struct addr_location *al __maybe_unused)
+ struct addr_location *al __maybe_unused)
{
}
--- /dev/null
+
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <linux/kernel.h>
+#include <traceevent/event-parse.h>
+#include "trace-event.h"
+#include "util.h"
+
+/*
+ * global trace_event object used by trace_event__tp_format
+ *
+ * TODO There's no cleanup call for this. Add some sort of
+ * __exit function support and call trace_event__cleanup
+ * there.
+ */
+static struct trace_event tevent;
+
+int trace_event__init(struct trace_event *t)
+{
+ struct pevent *pevent = pevent_alloc();
+
+ if (pevent) {
+ t->plugin_list = traceevent_load_plugins(pevent);
+ t->pevent = pevent;
+ }
+
+ return pevent ? 0 : -1;
+}
+
+void trace_event__cleanup(struct trace_event *t)
+{
+ traceevent_unload_plugins(t->plugin_list, t->pevent);
+ pevent_free(t->pevent);
+}
+
+static struct event_format*
+tp_format(const char *sys, const char *name)
+{
+ struct pevent *pevent = tevent.pevent;
+ struct event_format *event = NULL;
+ char path[PATH_MAX];
+ size_t size;
+ char *data;
+
+ scnprintf(path, PATH_MAX, "%s/%s/%s/format",
+ tracing_events_path, sys, name);
+
+ if (filename__read_str(path, &data, &size))
+ return NULL;
+
+ pevent_parse_format(pevent, &event, data, size, sys);
+
+ free(data);
+ return event;
+}
+
+struct event_format*
+trace_event__tp_format(const char *sys, const char *name)
+{
+ static bool initialized;
+
+ if (!initialized) {
+ int be = traceevent_host_bigendian();
+ struct pevent *pevent;
+
+ if (trace_event__init(&tevent))
+ return NULL;
+
+ pevent = tevent.pevent;
+ pevent_set_flag(pevent, PEVENT_NSEC_OUTPUT);
+ pevent_set_file_bigendian(pevent, be);
+ pevent_set_host_bigendian(pevent, be);
+ initialized = true;
+ }
+
+ return tp_format(sys, name);
+}
#include <traceevent/event-parse.h>
#include "parse-events.h"
-#include "session.h"
struct machine;
struct perf_sample;
union perf_event;
struct perf_tool;
struct thread;
+struct plugin_list;
+
+struct trace_event {
+ struct pevent *pevent;
+ struct plugin_list *plugin_list;
+};
+
+int trace_event__init(struct trace_event *t);
+void trace_event__cleanup(struct trace_event *t);
+struct event_format*
+trace_event__tp_format(const char *sys, const char *name);
int bigendian(void);
-struct pevent *read_trace_init(int file_bigendian, int host_bigendian);
void event_format__print(struct event_format *event,
int cpu, void *data, int size);
void parse_proc_kallsyms(struct pevent *pevent, char *file, unsigned int size);
void parse_ftrace_printk(struct pevent *pevent, char *file, unsigned int size);
-ssize_t trace_report(int fd, struct pevent **pevent, bool repipe);
+ssize_t trace_report(int fd, struct trace_event *tevent, bool repipe);
struct event_format *trace_find_next_event(struct pevent *pevent,
struct event_format *event);
void (*process_event) (union perf_event *event,
struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine,
struct thread *thread,
struct addr_location *al);
int (*generate_script) (struct pevent *pevent, const char *outfile);
#include "session.h"
#include "perf_regs.h"
#include "unwind.h"
+#include "symbol.h"
#include "util.h"
extern int
__v; \
})
-static Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
- GElf_Shdr *shp, const char *name)
-{
- Elf_Scn *sec = NULL;
-
- while ((sec = elf_nextscn(elf, sec)) != NULL) {
- char *str;
-
- gelf_getshdr(sec, shp);
- str = elf_strptr(elf, ep->e_shstrndx, shp->sh_name);
- if (!strcmp(name, str))
- break;
- }
-
- return sec;
-}
-
static u64 elf_section_offset(int fd, const char *name)
{
Elf *elf;
if (gelf_getehdr(elf, &ehdr) == NULL)
break;
- if (!elf_section_by_name(elf, &ehdr, &shdr, name))
+ if (!elf_section_by_name(elf, &ehdr, &shdr, name, NULL))
break;
offset = shdr.sh_offset;
/* Check the .debug_frame section for unwinding info */
if (!read_unwind_spec_debug_frame(map->dso, ui->machine, &segbase)) {
memset(&di, 0, sizeof(di));
- dwarf_find_debug_frame(0, &di, ip, 0, map->dso->name,
- map->start, map->end);
- return dwarf_search_unwind_table(as, ip, &di, pi,
- need_unwind_info, arg);
+ if (dwarf_find_debug_frame(0, &di, ip, 0, map->dso->name,
+ map->start, map->end))
+ return dwarf_search_unwind_table(as, ip, &di, pi,
+ need_unwind_info, arg);
}
#endif
#include "../perf.h"
#include "util.h"
+#include "fs.h"
#include <sys/mman.h>
#ifdef HAVE_BACKTRACE_SUPPORT
#include <execinfo.h>
#endif
#include <stdio.h>
#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <limits.h>
+#include <byteswap.h>
+#include <linux/kernel.h>
/*
* XXX We need to find a better place for these things...
return value;
}
-int readn(int fd, void *buf, size_t n)
+static ssize_t ion(bool is_read, int fd, void *buf, size_t n)
{
void *buf_start = buf;
+ size_t left = n;
- while (n) {
- int ret = read(fd, buf, n);
+ while (left) {
+ ssize_t ret = is_read ? read(fd, buf, left) :
+ write(fd, buf, left);
if (ret <= 0)
return ret;
- n -= ret;
- buf += ret;
+ left -= ret;
+ buf += ret;
}
- return buf - buf_start;
+ BUG_ON((size_t)(buf - buf_start) != n);
+ return n;
+}
+
+/*
+ * Read exactly 'n' bytes or return an error.
+ */
+ssize_t readn(int fd, void *buf, size_t n)
+{
+ return ion(true, fd, buf, n);
+}
+
+/*
+ * Write exactly 'n' bytes or return an error.
+ */
+ssize_t writen(int fd, void *buf, size_t n)
+{
+ return ion(false, fd, buf, n);
}
size_t hex_width(u64 v)
close(fd);
return err;
}
+
+int filename__read_str(const char *filename, char **buf, size_t *sizep)
+{
+ size_t size = 0, alloc_size = 0;
+ void *bf = NULL, *nbf;
+ int fd, n, err = 0;
+
+ fd = open(filename, O_RDONLY);
+ if (fd < 0)
+ return -errno;
+
+ do {
+ if (size == alloc_size) {
+ alloc_size += BUFSIZ;
+ nbf = realloc(bf, alloc_size);
+ if (!nbf) {
+ err = -ENOMEM;
+ break;
+ }
+
+ bf = nbf;
+ }
+
+ n = read(fd, bf + size, alloc_size - size);
+ if (n < 0) {
+ if (size) {
+ pr_warning("read failed %d: %s\n",
+ errno, strerror(errno));
+ err = 0;
+ } else
+ err = -errno;
+
+ break;
+ }
+
+ size += n;
+ } while (n > 0);
+
+ if (!err) {
+ *sizep = size;
+ *buf = bf;
+ } else
+ free(bf);
+
+ close(fd);
+ return err;
+}
+
+const char *get_filename_for_perf_kvm(void)
+{
+ const char *filename;
+
+ if (perf_host && !perf_guest)
+ filename = strdup("perf.data.host");
+ else if (!perf_host && perf_guest)
+ filename = strdup("perf.data.guest");
+ else
+ filename = strdup("perf.data.kvm");
+
+ return filename;
+}
+
+int perf_event_paranoid(void)
+{
+ char path[PATH_MAX];
+ const char *procfs = procfs__mountpoint();
+ int value;
+
+ if (!procfs)
+ return INT_MAX;
+
+ scnprintf(path, PATH_MAX, "%s/sys/kernel/perf_event_paranoid", procfs);
+
+ if (filename__read_int(path, &value))
+ return INT_MAX;
+
+ return value;
+}
+
+void mem_bswap_32(void *src, int byte_size)
+{
+ u32 *m = src;
+ while (byte_size > 0) {
+ *m = bswap_32(*m);
+ byte_size -= sizeof(u32);
+ ++m;
+ }
+}
+
+void mem_bswap_64(void *src, int byte_size)
+{
+ u64 *m = src;
+
+ while (byte_size > 0) {
+ *m = bswap_64(*m);
+ byte_size -= sizeof(u64);
+ ++m;
+ }
+}
#include <linux/magic.h>
#include "types.h"
#include <sys/ttydefaults.h>
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#include <termios.h>
+#include <linux/bitops.h>
extern const char *graph_line;
extern const char *graph_dotted_line;
return calloc(1, size);
}
+#define zfree(ptr) ({ free(*ptr); *ptr = NULL; })
+
static inline int has_extension(const char *filename, const char *ext)
{
size_t len = strlen(filename);
int strtailcmp(const char *s1, const char *s2);
char *strxfrchar(char *s, char from, char to);
unsigned long convert_unit(unsigned long value, char *unit);
-int readn(int fd, void *buf, size_t size);
+ssize_t readn(int fd, void *buf, size_t n);
+ssize_t writen(int fd, void *buf, size_t n);
struct perf_event_attr;
return 1ULL << (32 - __builtin_clz(x - 1));
}
+static inline unsigned long next_pow2_l(unsigned long x)
+{
+#if BITS_PER_LONG == 64
+ if (x <= (1UL << 31))
+ return next_pow2(x);
+ return (unsigned long)next_pow2(x >> 32) << 32;
+#else
+ return next_pow2(x);
+#endif
+}
+
size_t hex_width(u64 v);
int hex2u64(const char *ptr, u64 *val);
void free_srcline(char *srcline);
int filename__read_int(const char *filename, int *value);
+int filename__read_str(const char *filename, char **buf, size_t *sizep);
+int perf_event_paranoid(void);
+
+void mem_bswap_64(void *src, int byte_size);
+void mem_bswap_32(void *src, int byte_size);
+
+const char *get_filename_for_perf_kvm(void);
#endif /* GIT_COMPAT_UTIL_H */
return;
for (i = 0; i < values->threads; i++)
- free(values->value[i]);
- free(values->value);
- free(values->pid);
- free(values->tid);
- free(values->counterrawid);
+ zfree(&values->value[i]);
+ zfree(&values->value);
+ zfree(&values->pid);
+ zfree(&values->tid);
+ zfree(&values->counterrawid);
for (i = 0; i < values->counters; i++)
- free(values->countername[i]);
- free(values->countername);
+ zfree(&values->countername[i]);
+ zfree(&values->countername);
}
static void perf_read_values__enlarge_threads(struct perf_read_values *values)
dso = dso__new(VDSO__MAP_NAME);
if (dso != NULL) {
dsos__add(head, dso);
- dso__set_long_name(dso, file);
+ dso__set_long_name(dso, file, false);
}
}
ifneq ($(findstring $(MAKEFLAGS),s),s)
ifneq ($(V),1)
QUIET_CC = @echo ' CC '$@;
+ QUIET_CC_FPIC = @echo ' CC FPIC '$@;
QUIET_AR = @echo ' AR '$@;
QUIET_LINK = @echo ' LINK '$@;
QUIET_MKDIR = @echo ' MKDIR '$@;
+@echo ' DESCEND '$(1); \
mkdir -p $(OUTPUT)$(1) && \
$(MAKE) $(COMMAND_O) subdir=$(if $(subdir),$(subdir)/$(1),$(1)) $(PRINT_DIR) -C $(1) $(2)
+
+ QUIET_CLEAN = @printf ' CLEAN %s\n' $1;
+ QUIET_INSTALL = @printf ' INSTALL %s\n' $1;
endif
endif
my %opt;
my %repeat_tests;
my %repeats;
+my %evals;
#default opts
my %default = (
"TEST_TYPE" => "build",
"BUILD_TYPE" => "randconfig",
"MAKE_CMD" => "make",
+ "CLOSE_CONSOLE_SIGNAL" => "INT",
"TIMEOUT" => 120,
"TMP_DIR" => "/tmp/ktest/\${MACHINE}",
"SLEEP_TIME" => 60, # sleep time between tests
"CLEAR_LOG" => 0,
"BISECT_MANUAL" => 0,
"BISECT_SKIP" => 1,
+ "BISECT_TRIES" => 1,
"MIN_CONFIG_TYPE" => "boot",
"SUCCESS_LINE" => "login:",
"DETECT_TRIPLE_FAULT" => 1,
my $reverse_bisect;
my $bisect_manual;
my $bisect_skip;
+my $bisect_tries;
my $config_bisect_good;
my $bisect_ret_good;
my $bisect_ret_bad;
my $booted_timeout;
my $detect_triplefault;
my $console;
+my $close_console_signal;
my $reboot_success_line;
my $success_line;
my $stop_after_success;
"IGNORE_ERRORS" => \$ignore_errors,
"BISECT_MANUAL" => \$bisect_manual,
"BISECT_SKIP" => \$bisect_skip,
+ "BISECT_TRIES" => \$bisect_tries,
"CONFIG_BISECT_GOOD" => \$config_bisect_good,
"BISECT_RET_GOOD" => \$bisect_ret_good,
"BISECT_RET_BAD" => \$bisect_ret_bad,
"TIMEOUT" => \$timeout,
"BOOTED_TIMEOUT" => \$booted_timeout,
"CONSOLE" => \$console,
+ "CLOSE_CONSOLE_SIGNAL" => \$close_console_signal,
"DETECT_TRIPLE_FAULT" => \$detect_triplefault,
"SUCCESS_LINE" => \$success_line,
"REBOOT_SUCCESS_LINE" => \$reboot_success_line,
EOF
;
+sub _logit {
+ if (defined($opt{"LOG_FILE"})) {
+ open(OUT, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
+ print OUT @_;
+ close(OUT);
+ }
+}
+
+sub logit {
+ if (defined($opt{"LOG_FILE"})) {
+ _logit @_;
+ } else {
+ print @_;
+ }
+}
+
+sub doprint {
+ print @_;
+ _logit @_;
+}
+
sub read_prompt {
my ($cancel, $prompt) = @_;
}
}
+sub set_eval {
+ my ($lvalue, $rvalue, $name) = @_;
+
+ my $prvalue = process_variables($rvalue);
+ my $arr;
+
+ if (defined($evals{$lvalue})) {
+ $arr = $evals{$lvalue};
+ } else {
+ $arr = [];
+ $evals{$lvalue} = $arr;
+ }
+
+ push @{$arr}, $rvalue;
+}
+
sub set_variable {
my ($lvalue, $rvalue) = @_;
$test_case = 1;
}
+ } elsif (/^\s*([A-Z_\[\]\d]+)\s*=~\s*(.*?)\s*$/) {
+
+ next if ($skip);
+
+ my $lvalue = $1;
+ my $rvalue = $2;
+
+ if ($default || $lvalue =~ /\[\d+\]$/) {
+ set_eval($lvalue, $rvalue, $name);
+ } else {
+ my $val = "$lvalue\[$test_num\]";
+ set_eval($val, $rvalue, $name);
+ }
+
} elsif (/^\s*([A-Z_\[\]\d]+)\s*=\s*(.*?)\s*$/) {
next if ($skip);
} elsif (defined($opt{$var})) {
$o = $opt{$var};
$retval = "$retval$o";
+ } elsif ($var eq "KERNEL_VERSION" && defined($make)) {
+ # special option KERNEL_VERSION uses kernel version
+ get_version();
+ $retval = "$retval$version";
} else {
$retval = "$retval\$\{$var\}";
}
return $retval;
}
+sub process_evals {
+ my ($name, $option, $i) = @_;
+
+ my $option_name = "$name\[$i\]";
+ my $ev;
+
+ my $old_option = $option;
+
+ if (defined($evals{$option_name})) {
+ $ev = $evals{$option_name};
+ } elsif (defined($evals{$name})) {
+ $ev = $evals{$name};
+ } else {
+ return $option;
+ }
+
+ for my $e (@{$ev}) {
+ eval "\$option =~ $e";
+ }
+
+ if ($option ne $old_option) {
+ doprint("$name changed from '$old_option' to '$option'\n");
+ }
+
+ return $option;
+}
+
sub eval_option {
my ($name, $option, $i) = @_;
$option = __eval_option($name, $option, $i);
}
- return $option;
-}
+ $option = process_evals($name, $option, $i);
-sub _logit {
- if (defined($opt{"LOG_FILE"})) {
- open(OUT, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
- print OUT @_;
- close(OUT);
- }
-}
-
-sub logit {
- if (defined($opt{"LOG_FILE"})) {
- _logit @_;
- } else {
- print @_;
- }
-}
-
-sub doprint {
- print @_;
- _logit @_;
+ return $option;
}
sub run_command;
my ($fp, $pid) = @_;
doprint "kill child process $pid\n";
- kill 2, $pid;
+ kill $close_console_signal, $pid;
print "closing!\n";
close($fp);
$buildtype = "useconfig:$minconfig";
}
- my $ret = run_bisect_test $type, $buildtype;
+ # If the user sets bisect_tries to less than 1, then no tries
+ # is a success.
+ my $ret = 1;
- if ($bisect_manual) {
+ # Still let the user manually decide that though.
+ if ($bisect_tries < 1 && $bisect_manual) {
$ret = answer_bisect;
}
+ for (my $i = 0; $i < $bisect_tries; $i++) {
+ if ($bisect_tries > 1) {
+ my $t = $i + 1;
+ doprint("Running bisect trial $t of $bisect_tries:\n");
+ }
+ $ret = run_bisect_test $type, $buildtype;
+
+ if ($bisect_manual) {
+ $ret = answer_bisect;
+ }
+
+ last if (!$ret);
+ }
+
# Are we looking for where it worked, not failed?
if ($reverse_bisect && $ret >= 0) {
$ret = !$ret;
my $makecmd = set_test_option("MAKE_CMD", $i);
+ $outputdir = set_test_option("OUTPUT_DIR", $i);
+ $builddir = set_test_option("BUILD_DIR", $i);
+
+ chdir $builddir || die "can't change directory to $builddir";
+
+ if (!-d $outputdir) {
+ mkpath($outputdir) or
+ die "can't create $outputdir";
+ }
+
+ $make = "$makecmd O=$outputdir";
+
# Load all the options into their mapped variable names
foreach my $opt (keys %option_map) {
${$option_map{$opt}} = set_test_option($opt, $i);
$start_minconfig = $minconfig;
}
- chdir $builddir || die "can't change directory to $builddir";
-
- foreach my $dir ($tmpdir, $outputdir) {
- if (!-d $dir) {
- mkpath($dir) or
- die "can't create $dir";
- }
+ if (!-d $tmpdir) {
+ mkpath($tmpdir) or
+ die "can't create $tmpdir";
}
$ENV{"SSH_USER"} = $ssh_user;
$buildlog = "$tmpdir/buildlog-$machine";
$testlog = "$tmpdir/testlog-$machine";
$dmesg = "$tmpdir/dmesg-$machine";
- $make = "$makecmd O=$outputdir";
$output_config = "$outputdir/.config";
if (!$buildonly) {
# For a virtual machine with guest name "Guest".
#CONSOLE = virsh console Guest
+# Signal to send to kill console.
+# ktest.pl will create a child process to monitor the console.
+# When the console is finished, ktest will kill the child process
+# with this signal.
+# (default INT)
+#CLOSE_CONSOLE_SIGNAL = HUP
+
# Required version ending to differentiate the test
# from other linux builds on the system.
#LOCALVERSION = -test
# BISECT_BAD with BISECT_CHECK = good or
# BISECT_CHECK = bad, respectively.
#
+# BISECT_TRIES = 5 (optional, default 1)
+#
+# For those cases that it takes several tries to hit a bug,
+# the BISECT_TRIES is useful. It is the number of times the
+# test is ran before it says the kernel is good. The first failure
+# will stop trying and mark the current SHA1 as bad.
+#
+# Note, as with all race bugs, there's no guarantee that if
+# it succeeds, it is really a good bisect. But it helps in case
+# the bug is some what reliable.
+#
+# You can set BISECT_TRIES to zero, and all tests will be considered
+# good, unless you also set BISECT_MANUAL.
+#
# BISECT_RET_GOOD = 0 (optional, default undefined)
#
# In case the specificed test returns something other than just
--- /dev/null
+initrd
+linux-2.6
+b[0-9]*
+rcu-test-image
+res
+*.swp
--- /dev/null
+#!/bin/sh
+# Usage: sh config2frag.sh < .config > configfrag
+#
+# Converts the "# CONFIG_XXX is not set" to "CONFIG_XXX=n" so that the
+# resulting file becomes a legitimate Kconfig fragment.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+LANG=C sed -e 's/^# CONFIG_\([a-zA-Z0-9_]*\) is not set$/CONFIG_\1=n/'
--- /dev/null
+#!/bin/bash
+#
+# Extract the number of CPUs expected from the specified Kconfig-file
+# fragment by checking CONFIG_SMP and CONFIG_NR_CPUS. If the specified
+# file gives no clue, base the number on the number of idle CPUs on
+# the system.
+#
+# Usage: configNR_CPUS.sh config-frag
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+cf=$1
+if test ! -r $cf
+then
+ echo Unreadable config fragment $cf 1>&2
+ exit -1
+fi
+if grep -q '^CONFIG_SMP=n$' $cf
+then
+ echo 1
+ exit 0
+fi
+if grep -q '^CONFIG_NR_CPUS=' $cf
+then
+ grep '^CONFIG_NR_CPUS=' $cf |
+ sed -e 's/^CONFIG_NR_CPUS=\([0-9]*\).*$/\1/'
+ exit 0
+fi
+cpus2use.sh
--- /dev/null
+#!/bin/sh
+# Usage: sh configcheck.sh .config .config-template
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+T=/tmp/abat-chk-config.sh.$$
+trap 'rm -rf $T' 0
+mkdir $T
+
+cat $1 > $T/.config
+
+cat $2 | sed -e 's/\(.*\)=n/# \1 is not set/' -e 's/^#CHECK#//' |
+awk '
+BEGIN {
+ print "if grep -q \"" $0 "\" < '"$T/.config"'";
+ print "then";
+ print "\t:";
+ print "else";
+ if ($1 == "#") {
+ print "\tif grep -q \"" $2 "\" < '"$T/.config"'";
+ print "\tthen";
+ print "\t\tif test \"$firsttime\" = \"\""
+ print "\t\tthen"
+ print "\t\t\tfirsttime=1"
+ print "\t\tfi"
+ print "\t\techo \":" $2 ": improperly set\"";
+ print "\telse";
+ print "\t\t:";
+ print "\tfi";
+ } else {
+ print "\tif test \"$firsttime\" = \"\""
+ print "\tthen"
+ print "\t\tfirsttime=1"
+ print "\tfi"
+ print "\techo \":" $0 ": improperly set\"";
+ }
+ print "fi";
+ }' | sh
--- /dev/null
+#!/bin/sh
+#
+# sh configinit.sh config-spec-file [ build output dir ]
+#
+# Create a .config file from the spec file. Run from the kernel source tree.
+# Exits with 0 if all went well, with 1 if all went well but the config
+# did not match, and some other number for other failures.
+#
+# The first argument is the .config specification file, which contains
+# desired settings, for example, "CONFIG_NO_HZ=y". For best results,
+# this should be a full pathname.
+#
+# The second argument is a optional path to a build output directory,
+# for example, "O=/tmp/foo". If this argument is omitted, the .config
+# file will be generated directly in the current directory.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+T=/tmp/configinit.sh.$$
+trap 'rm -rf $T' 0
+mkdir $T
+
+# Capture config spec file.
+
+c=$1
+buildloc=$2
+builddir=
+if test -n $buildloc
+then
+ if echo $buildloc | grep -q '^O='
+ then
+ builddir=`echo $buildloc | sed -e 's/^O=//'`
+ if test ! -d $builddir
+ then
+ mkdir $builddir
+ fi
+ else
+ echo Bad build directory: \"$builddir\"
+ exit 2
+ fi
+fi
+
+sed -e 's/^\(CONFIG[0-9A-Z_]*\)=.*$/grep -v "^# \1" |/' < $c > $T/u.sh
+sed -e 's/^\(CONFIG[0-9A-Z_]*=\).*$/grep -v \1 |/' < $c >> $T/u.sh
+grep '^grep' < $T/u.sh > $T/upd.sh
+echo "cat - $c" >> $T/upd.sh
+make mrproper
+make $buildloc distclean > $builddir/Make.distclean 2>&1
+make $buildloc defconfig > $builddir/Make.defconfig.out 2>&1
+mv $builddir/.config $builddir/.config.sav
+sh $T/upd.sh < $builddir/.config.sav > $builddir/.config
+cp $builddir/.config $builddir/.config.new
+yes '' | make $buildloc oldconfig > $builddir/Make.modconfig.out 2>&1
+
+# verify new config matches specification.
+configcheck.sh $builddir/.config $c
+
+exit 0
--- /dev/null
+#!/bin/bash
+#
+# Get an estimate of how CPU-hoggy to be.
+#
+# Usage: cpus2use.sh
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+ncpus=`grep '^processor' /proc/cpuinfo | wc -l`
+idlecpus=`mpstat | tail -1 | \
+ awk -v ncpus=$ncpus '{ print ncpus * ($7 + $12) / 100 }'`
+awk -v ncpus=$ncpus -v idlecpus=$idlecpus < /dev/null '
+BEGIN {
+ cpus2use = idlecpus;
+ if (cpus2use < 1)
+ cpus2use = 1;
+ if (cpus2use < ncpus / 10)
+ cpus2use = ncpus / 10;
+ if (cpus2use == int(cpus2use))
+ cpus2use = int(cpus2use)
+ else
+ cpus2use = int(cpus2use) + 1
+ print cpus2use;
+}'
+
--- /dev/null
+#!/bin/bash
+#
+# Shell functions for the rest of the scripts.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+# bootparam_hotplug_cpu bootparam-string
+#
+# Returns 1 if the specified boot-parameter string tells rcutorture to
+# test CPU-hotplug operations.
+bootparam_hotplug_cpu () {
+ echo "$1" | grep -q "rcutorture\.onoff_"
+}
+
+# checkarg --argname argtype $# arg mustmatch cannotmatch
+#
+# Checks the specified argument "arg" against the mustmatch and cannotmatch
+# patterns.
+checkarg () {
+ if test $3 -le 1
+ then
+ echo $1 needs argument $2 matching \"$5\"
+ usage
+ fi
+ if echo "$4" | grep -q -e "$5"
+ then
+ :
+ else
+ echo $1 $2 \"$4\" must match \"$5\"
+ usage
+ fi
+ if echo "$4" | grep -q -e "$6"
+ then
+ echo $1 $2 \"$4\" must not match \"$6\"
+ usage
+ fi
+}
+
+# configfrag_boot_params bootparam-string config-fragment-file
+#
+# Adds boot parameters from the .boot file, if any.
+configfrag_boot_params () {
+ if test -r "$2.boot"
+ then
+ echo $1 `grep -v '^#' "$2.boot" | tr '\012' ' '`
+ else
+ echo $1
+ fi
+}
+
+# configfrag_hotplug_cpu config-fragment-file
+#
+# Returns 1 if the config fragment specifies hotplug CPU.
+configfrag_hotplug_cpu () {
+ if test ! -r "$1"
+ then
+ echo Unreadable config fragment "$1" 1>&2
+ exit -1
+ fi
+ grep -q '^CONFIG_HOTPLUG_CPU=y$' "$1"
+}
+
+# identify_qemu builddir
+#
+# Returns our best guess as to which qemu command is appropriate for
+# the kernel at hand. Override with the RCU_QEMU_CMD environment variable.
+identify_qemu () {
+ local u="`file "$1"`"
+ if test -n "$RCU_QEMU_CMD"
+ then
+ echo $RCU_QEMU_CMD
+ elif echo $u | grep -q x86-64
+ then
+ echo qemu-system-x86_64
+ elif echo $u | grep -q "Intel 80386"
+ then
+ echo qemu-system-i386
+ elif uname -a | grep -q ppc64
+ then
+ echo qemu-system-ppc64
+ else
+ echo Cannot figure out what qemu command to use! 1>&2
+ # Usually this will be one of /usr/bin/qemu-system-*
+ # Use RCU_QEMU_CMD environment variable or appropriate
+ # argument to top-level script.
+ exit 1
+ fi
+}
+
+# identify_qemu_append qemu-cmd
+#
+# Output arguments for the qemu "-append" string based on CPU type
+# and the RCU_QEMU_INTERACTIVE environment variable.
+identify_qemu_append () {
+ case "$1" in
+ qemu-system-x86_64|qemu-system-i386)
+ echo noapic selinux=0 initcall_debug debug
+ ;;
+ esac
+ if test -n "$RCU_QEMU_INTERACTIVE"
+ then
+ echo root=/dev/sda
+ else
+ echo console=ttyS0
+ fi
+}
+
+# identify_qemu_args qemu-cmd serial-file
+#
+# Output arguments for qemu arguments based on the RCU_QEMU_MAC
+# and RCU_QEMU_INTERACTIVE environment variables.
+identify_qemu_args () {
+ case "$1" in
+ qemu-system-x86_64|qemu-system-i386)
+ ;;
+ qemu-system-ppc64)
+ echo -enable-kvm -M pseries -cpu POWER7 -nodefaults
+ echo -device spapr-vscsi
+ if test -n "$RCU_QEMU_INTERACTIVE" -a -n "$RCU_QEMU_MAC"
+ then
+ echo -device spapr-vlan,netdev=net0,mac=$RCU_QEMU_MAC
+ echo -netdev bridge,br=br0,id=net0
+ elif test -n "$RCU_QEMU_INTERACTIVE"
+ then
+ echo -net nic -net user
+ fi
+ ;;
+ esac
+ if test -n "$RCU_QEMU_INTERACTIVE"
+ then
+ echo -monitor stdio -serial pty -S
+ else
+ echo -serial file:$2
+ fi
+}
+
+# identify_qemu_vcpus
+#
+# Returns the number of virtual CPUs available to the aggregate of the
+# guest OSes.
+identify_qemu_vcpus () {
+ lscpu | grep '^CPU(s):' | sed -e 's/CPU(s)://'
+}
+
+# print_bug
+#
+# Prints "BUG: " in red followed by remaining arguments
+print_bug () {
+ printf '\033[031mBUG: \033[m'
+ echo $*
+}
+
+# print_warning
+#
+# Prints "WARNING: " in yellow followed by remaining arguments
+print_warning () {
+ printf '\033[033mWARNING: \033[m'
+ echo $*
+}
+
+# specify_qemu_cpus qemu-cmd qemu-args #cpus
+#
+# Appends a string containing "-smp XXX" to qemu-args, unless the incoming
+# qemu-args already contains "-smp".
+specify_qemu_cpus () {
+ local nt;
+
+ if echo $2 | grep -q -e -smp
+ then
+ echo $2
+ else
+ case "$1" in
+ qemu-system-x86_64|qemu-system-i386)
+ echo $2 -smp $3
+ ;;
+ qemu-system-ppc64)
+ nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
+ echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
+ ;;
+ esac
+ fi
+}
--- /dev/null
+#!/bin/bash
+#
+# Build a kvm-ready Linux kernel from the tree in the current directory.
+#
+# Usage: sh kvm-build.sh config-template build-dir more-configs
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+config_template=${1}
+if test -z "$config_template" -o ! -f "$config_template" -o ! -r "$config_template"
+then
+ echo "kvm-build.sh :$config_template: Not a readable file"
+ exit 1
+fi
+builddir=${2}
+if test -z "$builddir" -o ! -d "$builddir" -o ! -w "$builddir"
+then
+ echo "kvm-build.sh :$builddir: Not a writable directory, cannot build into it"
+ exit 1
+fi
+moreconfigs=${3}
+if test -z "$moreconfigs" -o ! -r "$moreconfigs"
+then
+ echo "kvm-build.sh :$moreconfigs: Not a readable file"
+ exit 1
+fi
+
+T=/tmp/test-linux.sh.$$
+trap 'rm -rf $T' 0
+mkdir $T
+
+cat ${config_template} | grep -v CONFIG_RCU_TORTURE_TEST > $T/config
+cat << ___EOF___ >> $T/config
+CONFIG_INITRAMFS_SOURCE="$RCU_INITRD"
+CONFIG_VIRTIO_PCI=y
+CONFIG_VIRTIO_CONSOLE=y
+___EOF___
+cat $moreconfigs >> $T/config
+
+configinit.sh $T/config O=$builddir
+retval=$?
+if test $retval -gt 1
+then
+ exit 2
+fi
+ncpus=`cpus2use.sh`
+make O=$builddir -j$ncpus $RCU_KMAKE_ARG > $builddir/Make.out 2>&1
+retval=$?
+if test $retval -ne 0 || grep "rcu[^/]*": < $builddir/Make.out | egrep -q "Stop|Error|error:|warning:" || egrep -q "Stop|Error|error:" < $builddir/Make.out
+then
+ echo Kernel build error
+ egrep "Stop|Error|error:|warning:" < $builddir/Make.out
+ echo Run aborted.
+ exit 3
+fi
--- /dev/null
+#!/bin/bash
+#
+# Given the results directories for previous KVM runs of rcutorture,
+# check the build and console output for errors. Given a directory
+# containing results directories, this recursively checks them all.
+#
+# Usage: sh kvm-recheck.sh resdir ...
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+PATH=`pwd`/tools/testing/selftests/rcutorture/bin:$PATH; export PATH
+for rd in "$@"
+do
+ dirs=`find $rd -name Make.defconfig.out -print | sort | sed -e 's,/[^/]*$,,' | sort -u`
+ for i in $dirs
+ do
+ configfile=`echo $i | sed -e 's/^.*\///'`
+ echo $configfile
+ configcheck.sh $i/.config $i/ConfigFragment
+ parse-build.sh $i/Make.out $configfile
+ parse-rcutorture.sh $i/console.log $configfile
+ parse-console.sh $i/console.log $configfile
+ if test -r $i/Warnings
+ then
+ cat $i/Warnings
+ fi
+ done
+done
--- /dev/null
+#!/bin/bash
+#
+# Run a kvm-based test of the specified tree on the specified configs.
+# Fully automated run and error checking, no graphics console.
+#
+# Execute this in the source tree. Do not run it as a background task
+# because qemu does not seem to like that much.
+#
+# Usage: sh kvm-test-1-rcu.sh config builddir resdir minutes qemu-args bootargs
+#
+# qemu-args defaults to "" -- you will want "-nographic" if running headless.
+# bootargs defaults to "root=/dev/sda noapic selinux=0 console=ttyS0"
+# "initcall_debug debug rcutorture.stat_interval=15"
+# "rcutorture.shutdown_secs=$((minutes * 60))"
+# "rcutorture.rcutorture_runnable=1"
+#
+# Anything you specify for either qemu-args or bootargs is appended to
+# the default values. The "-smp" value is deduced from the contents of
+# the config fragment.
+#
+# More sophisticated argument parsing is clearly needed.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+grace=120
+
+T=/tmp/kvm-test-1-rcu.sh.$$
+trap 'rm -rf $T' 0
+
+. $KVM/bin/functions.sh
+. $KVPATH/ver_functions.sh
+
+config_template=${1}
+title=`echo $config_template | sed -e 's/^.*\///'`
+builddir=${2}
+if test -z "$builddir" -o ! -d "$builddir" -o ! -w "$builddir"
+then
+ echo "kvm-test-1-rcu.sh :$builddir: Not a writable directory, cannot build into it"
+ exit 1
+fi
+resdir=${3}
+if test -z "$resdir" -o ! -d "$resdir" -o ! -w "$resdir"
+then
+ echo "kvm-test-1-rcu.sh :$resdir: Not a writable directory, cannot build into it"
+ exit 1
+fi
+cp $config_template $resdir/ConfigFragment
+echo ' ---' `date`: Starting build
+echo ' ---' Kconfig fragment at: $config_template >> $resdir/log
+cat << '___EOF___' >> $T
+CONFIG_RCU_TORTURE_TEST=y
+___EOF___
+# Optimizations below this point
+# CONFIG_USB=n
+# CONFIG_SECURITY=n
+# CONFIG_NFS_FS=n
+# CONFIG_SOUND=n
+# CONFIG_INPUT_JOYSTICK=n
+# CONFIG_INPUT_TABLET=n
+# CONFIG_INPUT_TOUCHSCREEN=n
+# CONFIG_INPUT_MISC=n
+# CONFIG_INPUT_MOUSE=n
+# # CONFIG_NET=n # disables console access, so accept the slower build.
+# CONFIG_SCSI=n
+# CONFIG_ATA=n
+# CONFIG_FAT_FS=n
+# CONFIG_MSDOS_FS=n
+# CONFIG_VFAT_FS=n
+# CONFIG_ISO9660_FS=n
+# CONFIG_QUOTA=n
+# CONFIG_HID=n
+# CONFIG_CRYPTO=n
+# CONFIG_PCCARD=n
+# CONFIG_PCMCIA=n
+# CONFIG_CARDBUS=n
+# CONFIG_YENTA=n
+if kvm-build.sh $config_template $builddir $T
+then
+ cp $builddir/Make*.out $resdir
+ cp $builddir/.config $resdir
+ cp $builddir/arch/x86/boot/bzImage $resdir
+ parse-build.sh $resdir/Make.out $title
+else
+ cp $builddir/Make*.out $resdir
+ echo Build failed, not running KVM, see $resdir.
+ exit 1
+fi
+minutes=$4
+seconds=$(($minutes * 60))
+qemu_args=$5
+boot_args=$6
+
+cd $KVM
+kstarttime=`awk 'BEGIN { print systime() }' < /dev/null`
+echo ' ---' `date`: Starting kernel
+
+# Determine the appropriate flavor of qemu command.
+QEMU="`identify_qemu $builddir/vmlinux.o`"
+
+# Generate -smp qemu argument.
+cpu_count=`configNR_CPUS.sh $config_template`
+vcpus=`identify_qemu_vcpus`
+if test $cpu_count -gt $vcpus
+then
+ echo CPU count limited from $cpu_count to $vcpus
+ touch $resdir/Warnings
+ echo CPU count limited from $cpu_count to $vcpus >> $resdir/Warnings
+ cpu_count=$vcpus
+fi
+qemu_args="`specify_qemu_cpus "$QEMU" "$qemu_args" "$cpu_count"`"
+
+# Generate architecture-specific and interaction-specific qemu arguments
+qemu_args="$qemu_args `identify_qemu_args "$QEMU" "$builddir/console.log"`"
+
+# Generate qemu -append arguments
+qemu_append="`identify_qemu_append "$QEMU"`"
+
+# Pull in Kconfig-fragment boot parameters
+boot_args="`configfrag_boot_params "$boot_args" "$config_template"`"
+# Generate CPU-hotplug boot parameters
+boot_args="`rcutorture_param_onoff "$boot_args" $builddir/.config`"
+# Generate rcu_barrier() boot parameter
+boot_args="`rcutorture_param_n_barrier_cbs "$boot_args"`"
+# Pull in standard rcutorture boot arguments
+boot_args="$boot_args rcutorture.stat_interval=15 rcutorture.shutdown_secs=$seconds rcutorture.rcutorture_runnable=1"
+
+echo $QEMU $qemu_args -m 512 -kernel $builddir/arch/x86/boot/bzImage -append \"$qemu_append $boot_args\" > $resdir/qemu-cmd
+if test -n "$RCU_BUILDONLY"
+then
+ echo Build-only run specified, boot/test omitted.
+ exit 0
+fi
+$QEMU $qemu_args -m 512 -kernel $builddir/arch/x86/boot/bzImage -append "$qemu_append $boot_args" &
+qemu_pid=$!
+commandcompleted=0
+echo Monitoring qemu job at pid $qemu_pid
+for ((i=0;i<$seconds;i++))
+do
+ if kill -0 $qemu_pid > /dev/null 2>&1
+ then
+ sleep 1
+ else
+ commandcompleted=1
+ kruntime=`awk 'BEGIN { print systime() - '"$kstarttime"' }' < /dev/null`
+ if test $kruntime -lt $seconds
+ then
+ echo Completed in $kruntime vs. $seconds >> $resdir/Warnings 2>&1
+ else
+ echo ' ---' `date`: Kernel done
+ fi
+ break
+ fi
+done
+if test $commandcompleted -eq 0
+then
+ echo Grace period for qemu job at pid $qemu_pid
+ for ((i=0;i<=$grace;i++))
+ do
+ if kill -0 $qemu_pid > /dev/null 2>&1
+ then
+ sleep 1
+ else
+ break
+ fi
+ if test $i -eq $grace
+ then
+ kruntime=`awk 'BEGIN { print systime() - '"$kstarttime"' }'`
+ echo "!!! Hang at $kruntime vs. $seconds seconds" >> $resdir/Warnings 2>&1
+ kill -KILL $qemu_pid
+ fi
+ done
+fi
+
+cp $builddir/console.log $resdir
+parse-rcutorture.sh $resdir/console.log $title
+parse-console.sh $resdir/console.log $title
--- /dev/null
+#!/bin/bash
+#
+# Run a series of 14 tests under KVM. These are not particularly
+# well-selected or well-tuned, but are the current set. Run from the
+# top level of the source tree.
+#
+# Edit the definitions below to set the locations of the various directories,
+# as well as the test duration.
+#
+# Usage: sh kvm.sh [ options ]
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+scriptname=$0
+args="$*"
+
+dur=30
+KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
+PATH=${KVM}/bin:$PATH; export PATH
+builddir="${KVM}/b1"
+RCU_INITRD="$KVM/initrd"; export RCU_INITRD
+RCU_KMAKE_ARG=""; export RCU_KMAKE_ARG
+resdir=""
+configs=""
+ds=`date +%Y.%m.%d-%H:%M:%S`
+kversion=""
+
+. functions.sh
+
+usage () {
+ echo "Usage: $scriptname optional arguments:"
+ echo " --bootargs kernel-boot-arguments"
+ echo " --builddir absolute-pathname"
+ echo " --buildonly"
+ echo " --configs \"config-file list\""
+ echo " --datestamp string"
+ echo " --duration minutes"
+ echo " --interactive"
+ echo " --kmake-arg kernel-make-arguments"
+ echo " --kversion vN.NN"
+ echo " --mac nn:nn:nn:nn:nn:nn"
+ echo " --no-initrd"
+ echo " --qemu-args qemu-system-..."
+ echo " --qemu-cmd qemu-system-..."
+ echo " --results absolute-pathname"
+ echo " --relbuilddir relative-pathname"
+ exit 1
+}
+
+while test $# -gt 0
+do
+ case "$1" in
+ --bootargs)
+ checkarg --bootargs "(list of kernel boot arguments)" "$#" "$2" '.*' '^--'
+ RCU_BOOTARGS="$2"
+ shift
+ ;;
+ --builddir)
+ checkarg --builddir "(absolute pathname)" "$#" "$2" '^/' '^error'
+ builddir=$2
+ gotbuilddir=1
+ shift
+ ;;
+ --buildonly)
+ RCU_BUILDONLY=1; export RCU_BUILDONLY
+ ;;
+ --configs)
+ checkarg --configs "(list of config files)" "$#" "$2" '^[^/]*$' '^--'
+ configs="$2"
+ shift
+ ;;
+ --datestamp)
+ checkarg --datestamp "(relative pathname)" "$#" "$2" '^[^/]*$' '^--'
+ ds=$2
+ shift
+ ;;
+ --duration)
+ checkarg --duration "(minutes)" $# "$2" '^[0-9]*$' '^error'
+ dur=$2
+ shift
+ ;;
+ --interactive)
+ RCU_QEMU_INTERACTIVE=1; export RCU_QEMU_INTERACTIVE
+ ;;
+ --kmake-arg)
+ checkarg --kmake-arg "(kernel make arguments)" $# "$2" '.*' '^error$'
+ RCU_KMAKE_ARG="$2"; export RCU_KMAKE_ARG
+ shift
+ ;;
+ --kversion)
+ checkarg --kversion "(kernel version)" $# "$2" '^v[0-9.]*$' '^error'
+ kversion=$2
+ shift
+ ;;
+ --mac)
+ checkarg --mac "(MAC address)" $# "$2" '^\([0-9a-fA-F]\{2\}:\)\{5\}[0-9a-fA-F]\{2\}$' error
+ RCU_QEMU_MAC=$2; export RCU_QEMU_MAC
+ shift
+ ;;
+ --no-initrd)
+ RCU_INITRD=""; export RCU_INITRD
+ ;;
+ --qemu-args)
+ checkarg --qemu-args "-qemu args" $# "$2" '^-' '^error'
+ RCU_QEMU_ARG="$2"
+ shift
+ ;;
+ --qemu-cmd)
+ checkarg --qemu-cmd "(qemu-system-...)" $# "$2" 'qemu-system-' '^--'
+ RCU_QEMU_CMD="$2"; export RCU_QEMU_CMD
+ shift
+ ;;
+ --relbuilddir)
+ checkarg --relbuilddir "(relative pathname)" "$#" "$2" '^[^/]*$' '^--'
+ relbuilddir=$2
+ gotrelbuilddir=1
+ builddir=${KVM}/${relbuilddir}
+ shift
+ ;;
+ --results)
+ checkarg --results "(absolute pathname)" "$#" "$2" '^/' '^error'
+ resdir=$2
+ shift
+ ;;
+ *)
+ echo Unknown argument $1
+ usage
+ ;;
+ esac
+ shift
+done
+
+CONFIGFRAG=${KVM}/configs; export CONFIGFRAG
+KVPATH=${CONFIGFRAG}/$kversion; export KVPATH
+
+if test -z "$configs"
+then
+ configs="`cat $CONFIGFRAG/$kversion/CFLIST`"
+fi
+
+if test -z "$resdir"
+then
+ resdir=$KVM/res
+ if ! test -e $resdir
+ then
+ mkdir $resdir || :
+ fi
+else
+ if ! test -e $resdir
+ then
+ mkdir -p "$resdir" || :
+ fi
+fi
+mkdir $resdir/$ds
+touch $resdir/$ds/log
+echo $scriptname $args >> $resdir/$ds/log
+
+pwd > $resdir/$ds/testid.txt
+if test -d .git
+then
+ git status >> $resdir/$ds/testid.txt
+ git rev-parse HEAD >> $resdir/$ds/testid.txt
+fi
+builddir=$KVM/b1
+if ! test -e $builddir
+then
+ mkdir $builddir || :
+fi
+
+for CF in $configs
+do
+ # Running TREE01 multiple times creates TREE01, TREE01.2, TREE01.3, ...
+ rd=$resdir/$ds/$CF
+ if test -d "${rd}"
+ then
+ n="`ls -d "${rd}"* | grep '\.[0-9]\+$' |
+ sed -e 's/^.*\.\([0-9]\+\)/\1/' |
+ sort -k1n | tail -1`"
+ if test -z "$n"
+ then
+ rd="${rd}.2"
+ else
+ n="`expr $n + 1`"
+ rd="${rd}.${n}"
+ fi
+ fi
+ mkdir "${rd}"
+ echo Results directory: $rd
+ kvm-test-1-rcu.sh $CONFIGFRAG/$kversion/$CF $builddir $rd $dur "-nographic $RCU_QEMU_ARG" "rcutorture.test_no_idle_hz=1 rcutorture.verbose=1 $RCU_BOOTARGS"
+done
+# Tracing: trace_event=rcu:rcu_grace_period,rcu:rcu_future_grace_period,rcu:rcu_grace_period_init,rcu:rcu_nocb_wake,rcu:rcu_preempt_task,rcu:rcu_unlock_preempted_task,rcu:rcu_quiescent_state_report,rcu:rcu_fqs,rcu:rcu_callback,rcu:rcu_kfree_callback,rcu:rcu_batch_start,rcu:rcu_invoke_callback,rcu:rcu_invoke_kfree_callback,rcu:rcu_batch_end,rcu:rcu_torture_read,rcu:rcu_barrier
+
+echo " --- `date` Test summary:"
+kvm-recheck.sh $resdir/$ds
--- /dev/null
+#!/bin/sh
+#
+# Check the build output from an rcutorture run for goodness.
+# The "file" is a pathname on the local system, and "title" is
+# a text string for error-message purposes.
+#
+# The file must contain kernel build output.
+#
+# Usage:
+# sh parse-build.sh file title
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+T=$1
+title=$2
+
+. functions.sh
+
+if grep -q CC < $T
+then
+ :
+else
+ print_bug $title no build
+ exit 1
+fi
+
+if grep -q "error:" < $T
+then
+ print_bug $title build errors:
+ grep "error:" < $T
+ exit 2
+fi
+exit 0
+
+if egrep -q "rcu[^/]*\.c.*warning:|rcu.*\.h.*warning:" < $T
+then
+ print_warning $title build errors:
+ egrep "rcu[^/]*\.c.*warning:|rcu.*\.h.*warning:" < $T
+ exit 2
+fi
+exit 0
--- /dev/null
+#!/bin/sh
+#
+# Check the console output from an rcutorture run for oopses.
+# The "file" is a pathname on the local system, and "title" is
+# a text string for error-message purposes.
+#
+# Usage:
+# sh parse-console.sh file title
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+T=/tmp/abat-chk-badness.sh.$$
+trap 'rm -f $T' 0
+
+file="$1"
+title="$2"
+
+. functions.sh
+
+egrep 'Badness|WARNING:|Warn|BUG|===========|Call Trace:|Oops:' < $file | grep -v 'ODEBUG: ' | grep -v 'Warning: unable to open an initial console' > $T
+if test -s $T
+then
+ print_warning Assertion failure in $file $title
+ cat $T
+fi
--- /dev/null
+#!/bin/sh
+#
+# Check the console output from an rcutorture run for goodness.
+# The "file" is a pathname on the local system, and "title" is
+# a text string for error-message purposes.
+#
+# The file must contain rcutorture output, but can be interspersed
+# with other dmesg text.
+#
+# Usage:
+# sh parse-rcutorture.sh file title
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2011
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+T=/tmp/parse-rcutorture.sh.$$
+file="$1"
+title="$2"
+
+trap 'rm -f $T.seq' 0
+
+. functions.sh
+
+# check for presence of rcutorture.txt file
+
+if test -f "$file" -a -r "$file"
+then
+ :
+else
+ echo $title unreadable rcutorture.txt file: $file
+ exit 1
+fi
+
+# check for abject failure
+
+if grep -q FAILURE $file || grep -q -e '-torture.*!!!' $file
+then
+ nerrs=`grep --binary-files=text '!!!' $file | tail -1 | awk '{for (i=NF-8;i<=NF;i++) sum+=$i; } END {print sum}'`
+ print_bug $title FAILURE, $nerrs instances
+ echo " " $url
+ exit
+fi
+
+grep --binary-files=text 'torture:.*ver:' $file | grep --binary-files=text -v '(null)' | sed -e 's/^(initramfs)[^]]*] //' -e 's/^\[[^]]*] //' |
+awk '
+BEGIN {
+ ver = 0;
+ badseq = 0;
+ }
+
+ {
+ if (!badseq && ($5 + 0 != $5 || $5 <= ver)) {
+ badseqno1 = ver;
+ badseqno2 = $5;
+ badseqnr = NR;
+ badseq = 1;
+ }
+ ver = $5
+ }
+
+END {
+ if (badseq) {
+ if (badseqno1 == badseqno2 && badseqno2 == ver)
+ print "RCU GP HANG at " ver " rcutorture stat " badseqnr;
+ else
+ print "BAD SEQ " badseqno1 ":" badseqno2 " last:" ver " RCU version " badseqnr;
+ }
+ }' > $T.seq
+
+if grep -q SUCCESS $file
+then
+ if test -s $T.seq
+ then
+ print_warning $title $title `cat $T.seq`
+ echo " " $file
+ exit 2
+ fi
+else
+ if grep -q RCU_HOTPLUG $file
+ then
+ print_warning HOTPLUG FAILURES $title `cat $T.seq`
+ echo " " $file
+ exit 3
+ fi
+ echo $title no success message, `grep --binary-files=text 'ver:' $file | wc -l` successful RCU version messages
+ if test -s $T.seq
+ then
+ print_warning $title `cat $T.seq`
+ fi
+ exit 2
+fi
--- /dev/null
+TREE01
+TREE02
+TREE03
+TREE04
+TREE05
+TREE06
+TREE07
+TREE08
+TREE09
+SRCU-N
+SRCU-P
+TINY01
+TINY02
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+rcutorture.torture_type=srcu
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_PRINTK_TIME=y
--- /dev/null
+rcutorture.torture_type=srcu
--- /dev/null
+CONFIG_SMP=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_TRACE=n
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PREEMPT_COUNT=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_HZ_PERIODIC=y
+CONFIG_NO_HZ_IDLE=n
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_TRACE=y
+CONFIG_DEBUG_LOCK_ALLOC=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PREEMPT_COUNT=y
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_RCU_TRACE=y
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_FANOUT=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_ZERO=y
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_RCU_BOOST=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+rcutorture.torture_type=rcu_bh
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_FAST_NO_HZ=n
+CONFIG_RCU_TRACE=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_RCU_FANOUT=3
+CONFIG_RCU_FANOUT_LEAF=3
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_RCU_NOCB_CPU=n
+CONFIG_DEBUG_LOCK_ALLOC=y
+CONFIG_PROVE_LOCKING=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=y
+CONFIG_RCU_BOOST=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_HZ_PERIODIC=y
+CONFIG_NO_HZ_IDLE=n
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_TRACE=y
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_FANOUT=4
+CONFIG_RCU_FANOUT_LEAF=4
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_RCU_NOCB_CPU=n
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=n
+CONFIG_NO_HZ_FULL=y
+CONFIG_NO_HZ_FULL_ALL=y
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_RCU_TRACE=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_RCU_FANOUT=2
+CONFIG_RCU_FANOUT_LEAF=2
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_RCU_NOCB_CPU=n
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_RCU_CPU_STALL_VERBOSE=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+rcutorture.torture_type=rcu_bh
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_FAST_NO_HZ=n
+CONFIG_RCU_TRACE=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_FANOUT=6
+CONFIG_RCU_FANOUT_LEAF=6
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_NONE=y
+CONFIG_DEBUG_LOCK_ALLOC=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_PROVE_RCU_DELAY=y
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+rcutorture.torture_type=sched
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=8
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_FAST_NO_HZ=n
+CONFIG_RCU_TRACE=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_RCU_FANOUT=6
+CONFIG_RCU_FANOUT_LEAF=6
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_RCU_NOCB_CPU=n
+CONFIG_DEBUG_LOCK_ALLOC=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=16
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=n
+CONFIG_NO_HZ_FULL=y
+CONFIG_NO_HZ_FULL_ALL=y
+CONFIG_NO_HZ_FULL_SYSIDLE=y
+CONFIG_RCU_FAST_NO_HZ=n
+CONFIG_RCU_TRACE=y
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_FANOUT=2
+CONFIG_RCU_FANOUT_LEAF=2
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_RCU_NOCB_CPU=n
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=16
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_FAST_NO_HZ=n
+CONFIG_RCU_TRACE=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_RCU_FANOUT=3
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_RCU_FANOUT_LEAF=2
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_ALL=y
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_RCU_BOOST=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=y
+CONFIG_NR_CPUS=16
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_FAST_NO_HZ=n
+CONFIG_RCU_TRACE=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_RCU_FANOUT=3
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_RCU_FANOUT_LEAF=2
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_ALL=y
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_RCU_BOOST=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+CONFIG_SMP=n
+CONFIG_NR_CPUS=1
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_HZ_PERIODIC=n
+CONFIG_NO_HZ_IDLE=y
+CONFIG_NO_HZ_FULL=n
+CONFIG_RCU_TRACE=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_RCU_NOCB_CPU=n
+CONFIG_DEBUG_LOCK_ALLOC=n
+CONFIG_PROVE_RCU_DELAY=n
+CONFIG_RCU_CPU_STALL_INFO=n
+CONFIG_RCU_CPU_STALL_VERBOSE=n
+CONFIG_RCU_BOOST=n
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
+CONFIG_PRINTK_TIME=y
--- /dev/null
+P1-S-T-NH-SD-SMP-HP
+P2-2-t-nh-sd-SMP-hp
+P3-3-T-nh-SD-SMP-hp
+P4-A-t-NH-sd-SMP-HP
+P5-U-T-NH-sd-SMP-hp
+N1-S-T-NH-SD-SMP-HP
+N2-2-t-nh-sd-SMP-hp
+N3-3-T-nh-SD-SMP-hp
+N4-A-t-NH-sd-SMP-HP
+N5-U-T-NH-sd-SMP-hp
+PT1-nh
+PT2-NH
+NT1-nh
+NT3-NH
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_PROVE_RCU_DELAY=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#!/bin/bash
+#
+# Kernel-version-dependent shell functions for the rest of the scripts.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+# rcutorture_param_n_barrier_cbs bootparam-string
+#
+# Adds n_barrier_cbs rcutorture module parameter to kernels having it.
+rcutorture_param_n_barrier_cbs () {
+ echo $1
+}
+
+# rcutorture_param_onoff bootparam-string config-file
+#
+# Adds onoff rcutorture module parameters to kernels having it.
+rcutorture_param_onoff () {
+ echo $1
+}
--- /dev/null
+sysidleY.2013.06.19a
+sysidleN.2013.06.19a
+P1-S-T-NH-SD-SMP-HP
+P2-2-t-nh-sd-SMP-hp
+P3-3-T-nh-SD-SMP-hp
+P4-A-t-NH-sd-SMP-HP
+P5-U-T-NH-sd-SMP-hp
+P6---t-nh-SD-smp-hp
+N1-S-T-NH-SD-SMP-HP
+N2-2-t-nh-sd-SMP-hp
+N3-3-T-nh-SD-SMP-hp
+N4-A-t-NH-sd-SMP-HP
+N5-U-T-NH-sd-SMP-hp
+PT1-nh
+PT2-NH
+NT1-nh
+NT3-NH
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_NR_CPUS=1
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=16
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_NONE=y
+CONFIG_RCU_NOCB_CPU_ZERO=n
+CONFIG_RCU_NOCB_CPU_ALL=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=14
+CONFIG_NR_CPUS=16
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_PROVE_RCU_DELAY=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=n
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=16
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_NONE=n
+CONFIG_RCU_NOCB_CPU_ZERO=n
+CONFIG_RCU_NOCB_CPU_ALL=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_SLUB=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=16
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_NONE=y
+CONFIG_RCU_NOCB_CPU_ZERO=n
+CONFIG_RCU_NOCB_CPU_ALL=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_SLUB=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=16
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_NONE=y
+CONFIG_RCU_NOCB_CPU_ZERO=n
+CONFIG_RCU_NOCB_CPU_ALL=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_SLUB=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=16
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_NONE=n
+CONFIG_RCU_NOCB_CPU_ZERO=y
+CONFIG_RCU_NOCB_CPU_ALL=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_SLUB=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+P1-S-T-NH-SD-SMP-HP
+P2-2-t-nh-sd-SMP-hp
+P3-3-T-nh-SD-SMP-hp
+P4-A-t-NH-sd-SMP-HP
+P5-U-T-NH-sd-SMP-hp
+N1-S-T-NH-SD-SMP-HP
+N2-2-t-nh-sd-SMP-hp
+N3-3-T-nh-SD-SMP-hp
+N4-A-t-NH-sd-SMP-HP
+N5-U-T-NH-sd-SMP-hp
+PT1-nh
+PT2-NH
+NT1-nh
+NT3-NH
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_PROVE_RCU_DELAY=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#!/bin/bash
+#
+# Kernel-version-dependent shell functions for the rest of the scripts.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+# rcutorture_param_n_barrier_cbs bootparam-string
+#
+# Adds n_barrier_cbs rcutorture module parameter to kernels having it.
+rcutorture_param_n_barrier_cbs () {
+ echo $1
+}
+
+# rcutorture_param_onoff bootparam-string config-file
+#
+# Adds onoff rcutorture module parameters to kernels having it.
+rcutorture_param_onoff () {
+ if ! bootparam_hotplug_cpu "$1" && configfrag_hotplug_cpu "$2"
+ then
+ echo CPU-hotplug kernel, adding rcutorture onoff.
+ echo $1 rcutorture.onoff_interval=3 rcutorture.onoff_holdoff=30
+ else
+ echo $1
+ fi
+}
--- /dev/null
+P1-S-T-NH-SD-SMP-HP
+P2-2-t-nh-sd-SMP-hp
+P3-3-T-nh-SD-SMP-hp
+P4-A-t-NH-sd-SMP-HP
+P5-U-T-NH-sd-SMP-hp
+N1-S-T-NH-SD-SMP-HP
+N2-2-t-nh-sd-SMP-hp
+N3-3-T-nh-SD-SMP-hp
+N4-A-t-NH-sd-SMP-HP
+N5-U-T-NH-sd-SMP-hp
+PT1-nh
+PT2-NH
+NT1-nh
+NT3-NH
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+#CHECK#CONFIG_TREE_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#CHECK#CONFIG_TINY_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=y
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=n
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_RCU_FAST_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=8
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=4
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=n
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=2
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=n
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=n
+CONFIG_HOTPLUG_CPU=y
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_CPU_STALL_INFO=y
+CONFIG_NO_HZ=y
+CONFIG_SMP=y
+CONFIG_RCU_FANOUT=6
+CONFIG_NR_CPUS=8
+CONFIG_RCU_FANOUT_EXACT=y
+CONFIG_HOTPLUG_CPU=n
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+#CHECK#CONFIG_TREE_PREEMPT_RCU=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_PROVE_RCU_DELAY=y
+CONFIG_DEBUG_OBJECTS=y
+CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+CONFIG_RT_MUTEXES=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_BOOST=y
+CONFIG_RCU_BOOST_PRIO=2
+CONFIG_RCU_TRACE=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=n
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+CONFIG_TINY_PREEMPT_RCU=y
+CONFIG_RCU_TORTURE_TEST=m
+CONFIG_MODULE_UNLOAD=y
+CONFIG_SUSPEND=n
+CONFIG_HIBERNATION=n
+#
+CONFIG_SMP=n
+#
+CONFIG_HOTPLUG_CPU=n
+#
+CONFIG_NO_HZ=y
+#
+CONFIG_PREEMPT_NONE=n
+CONFIG_PREEMPT_VOLUNTARY=n
+CONFIG_PREEMPT=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_PROVE_RCU=y
+CONFIG_SYSFS_DEPRECATED_V2=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_PRINTK_TIME=y
+
--- /dev/null
+#!/bin/bash
+#
+# Kernel-version-dependent shell functions for the rest of the scripts.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+# rcutorture_param_n_barrier_cbs bootparam-string
+#
+# Adds n_barrier_cbs rcutorture module parameter to kernels having it.
+rcutorture_param_n_barrier_cbs () {
+ if echo $1 | grep -q "rcutorture\.n_barrier_cbs"
+ then
+ echo $1
+ else
+ echo $1 rcutorture.n_barrier_cbs=4
+ fi
+}
+
+# rcutorture_param_onoff bootparam-string config-file
+#
+# Adds onoff rcutorture module parameters to kernels having it.
+rcutorture_param_onoff () {
+ if ! bootparam_hotplug_cpu "$1" && configfrag_hotplug_cpu "$2"
+ then
+ echo CPU-hotplug kernel, adding rcutorture onoff.
+ echo $1 rcutorture.onoff_interval=3 rcutorture.onoff_holdoff=30
+ else
+ echo $1
+ fi
+}
--- /dev/null
+#!/bin/bash
+#
+# Kernel-version-dependent shell functions for the rest of the scripts.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# Copyright (C) IBM Corporation, 2013
+#
+# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+# rcutorture_param_n_barrier_cbs bootparam-string
+#
+# Adds n_barrier_cbs rcutorture module parameter to kernels having it.
+rcutorture_param_n_barrier_cbs () {
+ if echo $1 | grep -q "rcutorture\.n_barrier_cbs"
+ then
+ echo $1
+ else
+ echo $1 rcutorture.n_barrier_cbs=4
+ fi
+}
+
+# rcutorture_param_onoff bootparam-string config-file
+#
+# Adds onoff rcutorture module parameters to kernels having it.
+rcutorture_param_onoff () {
+ if ! bootparam_hotplug_cpu "$1" && configfrag_hotplug_cpu "$2"
+ then
+ echo CPU-hotplug kernel, adding rcutorture onoff. 1>&2
+ echo $1 rcutorture.onoff_interval=3 rcutorture.onoff_holdoff=30
+ else
+ echo $1
+ fi
+}
--- /dev/null
+This document gives a brief rationale for the TINY_RCU test cases.
+
+
+Kconfig Parameters:
+
+CONFIG_DEBUG_LOCK_ALLOC -- Do all three and none of the three.
+CONFIG_PREEMPT_COUNT
+CONFIG_RCU_TRACE
+
+The theory here is that randconfig testing will hit the other six possible
+combinations of these parameters.
+
+
+Kconfig Parameters Ignored:
+
+CONFIG_DEBUG_OBJECTS_RCU_HEAD
+CONFIG_PROVE_RCU
+
+ In common code tested by TREE_RCU test cases.
+
+CONFIG_NO_HZ_FULL_SYSIDLE
+CONFIG_RCU_NOCB_CPU
+CONFIG_RCU_USER_QS
+
+ Meaningless for TINY_RCU.
+
+CONFIG_RCU_STALL_COMMON
+CONFIG_RCU_TORTURE_TEST
+
+ Redundant with CONFIG_RCU_TRACE.
+
+CONFIG_HOTPLUG_CPU
+CONFIG_PREEMPT
+CONFIG_PREEMPT_RCU
+CONFIG_SMP
+CONFIG_TINY_RCU
+CONFIG_TREE_PREEMPT_RCU
+CONFIG_TREE_RCU
+
+ All forced by CONFIG_TINY_RCU.
--- /dev/null
+This document gives a brief rationale for the TREE_RCU-related test
+cases, a group that includes TREE_PREEMPT_RCU.
+
+
+Kconfig Parameters:
+
+CONFIG_DEBUG_LOCK_ALLOC -- Do three, covering CONFIG_PROVE_LOCKING & not.
+CONFIG_DEBUG_OBJECTS_RCU_HEAD -- Do one.
+CONFIG_HOTPLUG_CPU -- Do half. (Every second.)
+CONFIG_HZ_PERIODIC -- Do one.
+CONFIG_NO_HZ_IDLE -- Do those not otherwise specified. (Groups of two.)
+CONFIG_NO_HZ_FULL -- Do two, one with CONFIG_NO_HZ_FULL_SYSIDLE.
+CONFIG_NO_HZ_FULL_SYSIDLE -- Do one.
+CONFIG_PREEMPT -- Do half. (First three and #8.)
+CONFIG_PROVE_LOCKING -- Do all but two, covering CONFIG_PROVE_RCU and not.
+CONFIG_PROVE_RCU -- Do all but one under CONFIG_PROVE_LOCKING.
+CONFIG_PROVE_RCU_DELAY -- Do one.
+CONFIG_RCU_BOOST -- one of TREE_PREEMPT_RCU.
+CONFIG_RCU_BOOST_PRIO -- set to 2 for _BOOST testing.
+CONFIG_RCU_CPU_STALL_INFO -- do one with and without _VERBOSE.
+CONFIG_RCU_CPU_STALL_VERBOSE -- do one with and without _INFO.
+CONFIG_RCU_FANOUT -- Cover hierarchy as currently, but overlap with others.
+CONFIG_RCU_FANOUT_EXACT -- Do one.
+CONFIG_RCU_FANOUT_LEAF -- Do one non-default.
+CONFIG_RCU_FAST_NO_HZ -- Do one, but not with CONFIG_RCU_NOCB_CPU_ALL.
+CONFIG_RCU_NOCB_CPU -- Do three, see below.
+CONFIG_RCU_NOCB_CPU_ALL -- Do one.
+CONFIG_RCU_NOCB_CPU_NONE -- Do one.
+CONFIG_RCU_NOCB_CPU_ZERO -- Do one.
+CONFIG_RCU_TRACE -- Do half.
+CONFIG_SMP -- Need one !SMP for TREE_PREEMPT_RCU.
+RCU-bh: Do one with PREEMPT and one with !PREEMPT.
+RCU-sched: Do one with PREEMPT but not BOOST.
+
+
+Hierarchy:
+
+TREE01. CONFIG_NR_CPUS=8, CONFIG_RCU_FANOUT=8, CONFIG_RCU_FANOUT_EXACT=n.
+TREE02. CONFIG_NR_CPUS=8, CONFIG_RCU_FANOUT=3, CONFIG_RCU_FANOUT_EXACT=n,
+ CONFIG_RCU_FANOUT_LEAF=3.
+TREE03. CONFIG_NR_CPUS=8, CONFIG_RCU_FANOUT=4, CONFIG_RCU_FANOUT_EXACT=n,
+ CONFIG_RCU_FANOUT_LEAF=4.
+TREE04. CONFIG_NR_CPUS=8, CONFIG_RCU_FANOUT=2, CONFIG_RCU_FANOUT_EXACT=n,
+ CONFIG_RCU_FANOUT_LEAF=2.
+TREE05. CONFIG_NR_CPUS=8, CONFIG_RCU_FANOUT=6, CONFIG_RCU_FANOUT_EXACT=n
+ CONFIG_RCU_FANOUT_LEAF=6.
+TREE06. CONFIG_NR_CPUS=8, CONFIG_RCU_FANOUT=6, CONFIG_RCU_FANOUT_EXACT=y
+ CONFIG_RCU_FANOUT_LEAF=6.
+TREE07. CONFIG_NR_CPUS=16, CONFIG_RCU_FANOUT=2, CONFIG_RCU_FANOUT_EXACT=n,
+ CONFIG_RCU_FANOUT_LEAF=2.
+TREE08. CONFIG_NR_CPUS=16, CONFIG_RCU_FANOUT=3, CONFIG_RCU_FANOUT_EXACT=y,
+ CONFIG_RCU_FANOUT_LEAF=2.
+TREE09. CONFIG_NR_CPUS=1.
+
+
+Kconfig Parameters Ignored:
+
+CONFIG_64BIT
+
+ Used only to check CONFIG_RCU_FANOUT value, inspection suffices.
+
+CONFIG_NO_HZ_FULL_SYSIDLE_SMALL
+
+ Defer until Frederic uses this.
+
+CONFIG_PREEMPT_COUNT
+CONFIG_PREEMPT_RCU
+
+ Redundant with CONFIG_PREEMPT, ignore.
+
+CONFIG_RCU_BOOST_DELAY
+
+ Inspection suffices, ignore.
+
+CONFIG_RCU_CPU_STALL_TIMEOUT
+
+ Inspection suffices, ignore.
+
+CONFIG_RCU_STALL_COMMON
+
+ Implied by TREE_RCU and TREE_PREEMPT_RCU.
+
+CONFIG_RCU_TORTURE_TEST
+CONFIG_RCU_TORTURE_TEST_RUNNABLE
+
+ Always used in KVM testing.
+
+CONFIG_RCU_USER_QS
+
+ Redundant with CONFIG_NO_HZ_FULL.
+
+CONFIG_TREE_PREEMPT_RCU
+CONFIG_TREE_RCU
+
+ These are controlled by CONFIG_PREEMPT.
--- /dev/null
+This document describes one way to create the initrd directory hierarchy
+in order to allow an initrd to be built into your kernel. The trick
+here is to steal the initrd file used on your Linux laptop, Ubuntu in
+this case. There are probably much better ways of doing this.
+
+That said, here are the commands:
+
+------------------------------------------------------------------------
+zcat /initrd.img > /tmp/initrd.img.zcat
+mkdir initrd
+cd initrd
+cpio -id < /tmp/initrd.img.zcat
+------------------------------------------------------------------------
+
+Interestingly enough, if you are running rcutorture, you don't really
+need userspace in many cases. Running without userspace has the
+advantage of allowing you to test your kernel independently of the
+distro in place, the root-filesystem layout, and so on. To make this
+happen, put the following script in the initrd's tree's "/init" file,
+with 0755 mode.
+
+------------------------------------------------------------------------
+#!/bin/sh
+
+[ -d /dev ] || mkdir -m 0755 /dev
+[ -d /root ] || mkdir -m 0700 /root
+[ -d /sys ] || mkdir /sys
+[ -d /proc ] || mkdir /proc
+[ -d /tmp ] || mkdir /tmp
+mkdir -p /var/lock
+mount -t sysfs -o nodev,noexec,nosuid sysfs /sys
+mount -t proc -o nodev,noexec,nosuid proc /proc
+# Some things don't work properly without /etc/mtab.
+ln -sf /proc/mounts /etc/mtab
+
+# Note that this only becomes /dev on the real filesystem if udev's scripts
+# are used; which they will be, but it's worth pointing out
+if ! mount -t devtmpfs -o mode=0755 udev /dev; then
+ echo "W: devtmpfs not available, falling back to tmpfs for /dev"
+ mount -t tmpfs -o mode=0755 udev /dev
+ [ -e /dev/console ] || mknod --mode=600 /dev/console c 5 1
+ [ -e /dev/kmsg ] || mknod --mode=644 /dev/kmsg c 1 11
+ [ -e /dev/null ] || mknod --mode=666 /dev/null c 1 3
+fi
+
+mkdir /dev/pts
+mount -t devpts -o noexec,nosuid,gid=5,mode=0620 devpts /dev/pts || true
+mount -t tmpfs -o "nosuid,size=20%,mode=0755" tmpfs /run
+mkdir /run/initramfs
+# compatibility symlink for the pre-oneiric locations
+ln -s /run/initramfs /dev/.initramfs
+
+# Export relevant variables
+export ROOT=
+export ROOTDELAY=
+export ROOTFLAGS=
+export ROOTFSTYPE=
+export IP=
+export BOOT=
+export BOOTIF=
+export UBIMTD=
+export break=
+export init=/sbin/init
+export quiet=n
+export readonly=y
+export rootmnt=/root
+export debug=
+export panic=
+export blacklist=
+export resume=
+export resume_offset=
+export recovery=
+
+for i in /sys/devices/system/cpu/cpu*/online
+do
+ case $i in
+ '/sys/devices/system/cpu/cpu0/online')
+ ;;
+ '/sys/devices/system/cpu/cpu*/online')
+ ;;
+ *)
+ echo 1 > $i
+ ;;
+ esac
+done
+
+while :
+do
+ sleep 10
+done
--- /dev/null
+This document describes one way to created the rcu-test-image file
+that contains the filesystem used by the guest-OS kernel. There are
+probably much better ways of doing this, and this filesystem could no
+doubt be smaller. It is probably also possible to simply download
+an appropriate image from any number of places.
+
+That said, here are the commands:
+
+------------------------------------------------------------------------
+dd if=/dev/zero of=rcu-test-image bs=400M count=1
+mkfs.ext3 ./rcu-test-image
+sudo mount -o loop ./rcu-test-image /mnt
+
+# Replace "precise" below with your favorite Ubuntu release.
+# Empirical evidence says this image will work for 64-bit, but...
+# Note that debootstrap does take a few minutes to run. Or longer.
+sudo debootstrap --verbose --arch i386 precise /mnt http://archive.ubuntu.com/ubuntu
+cat << '___EOF___' | sudo dd of=/mnt/etc/fstab
+# UNCONFIGURED FSTAB FOR BASE SYSTEM
+#
+/dev/vda / ext3 defaults 1 1
+dev /dev tmpfs rw 0 0
+tmpfs /dev/shm tmpfs defaults 0 0
+devpts /dev/pts devpts gid=5,mode=620 0 0
+sysfs /sys sysfs defaults 0 0
+proc /proc proc defaults 0 0
+___EOF___
+sudo umount /mnt
+------------------------------------------------------------------------
+
+
+References:
+
+ http://sripathikodi.blogspot.com/2010/02/creating-kvm-bootable-fedora-system.html
+ https://help.ubuntu.com/community/KVM/CreateGuests
+ https://help.ubuntu.com/community/JeOSVMBuilder
+ http://wiki.libvirt.org/page/UbuntuKVMWalkthrough
+ http://www.moe.co.uk/2011/01/07/pci_add_option_rom-failed-to-find-romfile-pxe-rtl8139-bin/ -- "apt-get install kvm-pxe"
+ http://www.landley.net/writing/rootfs-howto.html
+ http://en.wikipedia.org/wiki/Initrd
+ http://en.wikipedia.org/wiki/Cpio
+ http://wiki.libvirt.org/page/UbuntuKVMWalkthrough
#
TARGETS=page-types slabinfo
-LK_DIR = ../lib/lk
-LIBLK = $(LK_DIR)/liblk.a
+LIB_DIR = ../lib/api
+LIBS = $(LIB_DIR)/libapikfs.a
CC = $(CROSS_COMPILE)gcc
CFLAGS = -Wall -Wextra -I../lib/
-LDFLAGS = $(LIBLK)
+LDFLAGS = $(LIBS)
-$(TARGETS): liblk
+$(TARGETS): $(LIBS)
-liblk:
- make -C $(LK_DIR)
+$(LIBS):
+ make -C $(LIB_DIR)
%: %.c
$(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
clean:
$(RM) page-types slabinfo
- make -C ../lib/lk clean
+ make -C $(LIB_DIR) clean
#include <sys/statfs.h>
#include "../../include/uapi/linux/magic.h"
#include "../../include/uapi/linux/kernel-page-flags.h"
-#include <lk/debugfs.h>
+#include <api/fs/debugfs.h>
#ifndef MAX_PATH
# define MAX_PATH 256
projects / linux-2.6-microblaze.git / commitdiff