--- /dev/null
+What: /sys/class/devfreq-event/event(x)/
+Date: January 2017
+Contact: Chanwoo Choi <cw00.choi@samsung.com>
+Description:
+ Provide a place in sysfs for the devfreq-event objects.
+ This allows accessing various devfreq-event specific variables.
+ The name of devfreq-event object denoted as 'event(x)' which
+ includes the unique number of 'x' for each devfreq-event object.
+
+What: /sys/class/devfreq-event/event(x)/name
+Date: January 2017
+Contact: Chanwoo Choi <cw00.choi@samsung.com>
+Description:
+ The /sys/class/devfreq-event/event(x)/name attribute contains
+ the name of the devfreq-event object. This attribute is
+ read-only.
+
+What: /sys/class/devfreq-event/event(x)/enable_count
+Date: January 2017
+Contact: Chanwoo Choi <cw00.choi@samsung.com>
+Description:
+ The /sys/class/devfreq-event/event(x)/enable_count attribute
+ contains the reference count to enable the devfreq-event
+ object. If the device is enabled, the value of attribute is
+ greater than zero.
Dominik Brodowski <linux@brodo.de>
David Kimdon <dwhedon@debian.org>
+ Rafael J. Wysocki <rafael.j.wysocki@intel.com>
+ Viresh Kumar <viresh.kumar@linaro.org>
kernel "constant" loops_per_jiffy is updated on frequency changes
here.
-Reference counting is done by cpufreq_get_cpu and cpufreq_put_cpu,
-which make sure that the cpufreq processor driver is correctly
-registered with the core, and will not be unloaded until
-cpufreq_put_cpu is called.
+Reference counting of the cpufreq policies is done by cpufreq_cpu_get
+and cpufreq_cpu_put, which make sure that the cpufreq driver is
+correctly registered with the core, and will not be unloaded until
+cpufreq_put_cpu is called. That also ensures that the respective cpufreq
+policy doesn't get freed while being used.
2. CPUFreq notifiers
====================
The phase is specified in the second argument to the notifier.
The third argument, a void *pointer, points to a struct cpufreq_policy
-consisting of five values: cpu, min, max, policy and max_cpu_freq. min
-and max are the lower and upper frequencies (in kHz) of the new
-policy, policy the new policy, cpu the number of the affected CPU; and
-max_cpu_freq the maximum supported CPU frequency. This value is given
-for informational purposes only.
+consisting of several values, including min, max (the lower and upper
+frequencies (in kHz) of the new policy).
2.2 CPUFreq transition notifiers
--------------------------------
-These are notified twice when the CPUfreq driver switches the CPU core
-frequency and this change has any external implications.
+These are notified twice for each online CPU in the policy, when the
+CPUfreq driver switches the CPU core frequency and this change has no
+any external implications.
The second argument specifies the phase - CPUFREQ_PRECHANGE or
CPUFREQ_POSTCHANGE.
cpu - number of the affected CPU
old - old frequency
new - new frequency
+flags - flags of the cpufreq driver
3. CPUFreq Table Generation with Operating Performance Point (OPP)
==================================================================
Dominik Brodowski <linux@brodo.de>
+ Rafael J. Wysocki <rafael.j.wysocki@intel.com>
+ Viresh Kumar <viresh.kumar@linaro.org>
What shall this struct cpufreq_driver contain?
-cpufreq_driver.name - The name of this driver.
+ .name - The name of this driver.
-cpufreq_driver.init - A pointer to the per-CPU initialization
- function.
+ .init - A pointer to the per-policy initialization function.
-cpufreq_driver.verify - A pointer to a "verification" function.
+ .verify - A pointer to a "verification" function.
-cpufreq_driver.setpolicy _or_
-cpufreq_driver.target/
-target_index - See below on the differences.
+ .setpolicy _or_ .fast_switch _or_ .target _or_ .target_index - See
+ below on the differences.
And optionally
-cpufreq_driver.exit - A pointer to a per-CPU cleanup
- function called during CPU_POST_DEAD
- phase of cpu hotplug process.
+ .flags - Hints for the cpufreq core.
-cpufreq_driver.stop_cpu - A pointer to a per-CPU stop function
- called during CPU_DOWN_PREPARE phase of
- cpu hotplug process.
+ .driver_data - cpufreq driver specific data.
-cpufreq_driver.resume - A pointer to a per-CPU resume function
- which is called with interrupts disabled
- and _before_ the pre-suspend frequency
- and/or policy is restored by a call to
- ->target/target_index or ->setpolicy.
+ .resolve_freq - Returns the most appropriate frequency for a target
+ frequency. Doesn't change the frequency though.
-cpufreq_driver.attr - A pointer to a NULL-terminated list of
- "struct freq_attr" which allow to
- export values to sysfs.
+ .get_intermediate and target_intermediate - Used to switch to stable
+ frequency while changing CPU frequency.
-cpufreq_driver.get_intermediate
-and target_intermediate Used to switch to stable frequency while
- changing CPU frequency.
+ .get - Returns current frequency of the CPU.
+
+ .bios_limit - Returns HW/BIOS max frequency limitations for the CPU.
+
+ .exit - A pointer to a per-policy cleanup function called during
+ CPU_POST_DEAD phase of cpu hotplug process.
+
+ .stop_cpu - A pointer to a per-policy stop function called during
+ CPU_DOWN_PREPARE phase of cpu hotplug process.
+
+ .suspend - A pointer to a per-policy suspend function which is called
+ with interrupts disabled and _after_ the governor is stopped for the
+ policy.
+
+ .resume - A pointer to a per-policy resume function which is called
+ with interrupts disabled and _before_ the governor is started again.
+
+ .ready - A pointer to a per-policy ready function which is called after
+ the policy is fully initialized.
+
+ .attr - A pointer to a NULL-terminated list of "struct freq_attr" which
+ allow to export values to sysfs.
+
+ .boost_enabled - If set, boost frequencies are enabled.
+
+ .set_boost - A pointer to a per-policy function to enable/disable boost
+ frequencies.
1.2 Per-CPU Initialization
--------------------------
Whenever a new CPU is registered with the device model, or after the
-cpufreq driver registers itself, the per-CPU initialization function
-cpufreq_driver.init is called. It takes a struct cpufreq_policy
-*policy as argument. What to do now?
+cpufreq driver registers itself, the per-policy initialization function
+cpufreq_driver.init is called if no cpufreq policy existed for the CPU.
+Note that the .init() and .exit() routines are called only once for the
+policy and not for each CPU managed by the policy. It takes a struct
+cpufreq_policy *policy as argument. What to do now?
If necessary, activate the CPUfreq support on your CPU.
cpufreq_driver.setpolicy or
cpufreq_driver.target/target_index is called
with these values.
+policy->cpus Update this with the masks of the
+ (online + offline) CPUs that do DVFS
+ along with this CPU (i.e. that share
+ clock/voltage rails with it).
For setting some of these values (cpuinfo.min[max]_freq, policy->min[max]), the
frequency table helpers might be helpful. See the section 2 for more information
on them.
-SMP systems normally have same clock source for a group of cpus. For these the
-.init() would be called only once for the first online cpu. Here the .init()
-routine must initialize policy->cpus with mask of all possible cpus (Online +
-Offline) that share the clock. Then the core would copy this mask onto
-policy->related_cpus and will reset policy->cpus to carry only online cpus.
-
1.3 verify
-------------
+----------
When the user decides a new policy (consisting of
"policy,governor,min,max") shall be set, this policy must be validated
so that incompatible values can be corrected. For verifying these
-values, a frequency table helper and/or the
-cpufreq_verify_within_limits(struct cpufreq_policy *policy, unsigned
-int min_freq, unsigned int max_freq) function might be helpful. See
-section 2 for details on frequency table helpers.
+values cpufreq_verify_within_limits(struct cpufreq_policy *policy,
+unsigned int min_freq, unsigned int max_freq) function might be helpful.
+See section 2 for details on frequency table helpers.
You need to make sure that at least one valid frequency (or operating
range) is within policy->min and policy->max. If necessary, increase
policy->max first, and only if this is no solution, decrease policy->min.
-1.4 target/target_index or setpolicy?
-----------------------------
+1.4 target or target_index or setpolicy or fast_switch?
+-------------------------------------------------------
Most cpufreq drivers or even most cpu frequency scaling algorithms
-only allow the CPU to be set to one frequency. For these, you use the
-->target/target_index call.
+only allow the CPU frequency to be set to predefined fixed values. For
+these, you use the ->target(), ->target_index() or ->fast_switch()
+callbacks.
-Some cpufreq-capable processors switch the frequency between certain
-limits on their own. These shall use the ->setpolicy call
+Some cpufreq capable processors switch the frequency between certain
+limits on their own. These shall use the ->setpolicy() callback.
1.5. target/target_index
--------------
+------------------------
The target_index call has two arguments: struct cpufreq_policy *policy,
and unsigned int index (into the exposed frequency table).
Here again the frequency table helper might assist you - see section 2
for details.
+1.6. fast_switch
+----------------
-1.6 setpolicy
----------------
+This function is used for frequency switching from scheduler's context.
+Not all drivers are expected to implement it, as sleeping from within
+this callback isn't allowed. This callback must be highly optimized to
+do switching as fast as possible.
+
+This function has two arguments: struct cpufreq_policy *policy and
+unsigned int target_frequency.
+
+
+1.7 setpolicy
+-------------
The setpolicy call only takes a struct cpufreq_policy *policy as
argument. You need to set the lower limit of the in-processor or
powersaving-oriented setting when CPUFREQ_POLICY_POWERSAVE. Also check
the reference implementation in drivers/cpufreq/longrun.c
-1.7 get_intermediate and target_intermediate
+1.8 get_intermediate and target_intermediate
--------------------------------------------
Only for drivers with target_index() and CPUFREQ_ASYNC_NOTIFICATION unset.
As most cpufreq processors only allow for being set to a few specific
frequencies, a "frequency table" with some functions might assist in
-some work of the processor driver. Such a "frequency table" consists
-of an array of struct cpufreq_frequency_table entries, with any value in
-"driver_data" you want to use, and the corresponding frequency in
-"frequency". At the end of the table, you need to add a
-cpufreq_frequency_table entry with frequency set to CPUFREQ_TABLE_END. And
-if you want to skip one entry in the table, set the frequency to
-CPUFREQ_ENTRY_INVALID. The entries don't need to be in ascending
-order.
-
-By calling cpufreq_table_validate_and_show(struct cpufreq_policy *policy,
- struct cpufreq_frequency_table *table);
-the cpuinfo.min_freq and cpuinfo.max_freq values are detected, and
-policy->min and policy->max are set to the same values. This is
-helpful for the per-CPU initialization stage.
-
-int cpufreq_frequency_table_verify(struct cpufreq_policy *policy,
- struct cpufreq_frequency_table *table);
-assures that at least one valid frequency is within policy->min and
-policy->max, and all other criteria are met. This is helpful for the
-->verify call.
-
-int cpufreq_frequency_table_target(struct cpufreq_policy *policy,
- unsigned int target_freq,
- unsigned int relation);
-
-is the corresponding frequency table helper for the ->target
-stage. Just pass the values to this function, and this function
-returns the number of the frequency table entry which contains
-the frequency the CPU shall be set to.
+some work of the processor driver. Such a "frequency table" consists of
+an array of struct cpufreq_frequency_table entries, with driver specific
+values in "driver_data", the corresponding frequency in "frequency" and
+flags set. At the end of the table, you need to add a
+cpufreq_frequency_table entry with frequency set to CPUFREQ_TABLE_END.
+And if you want to skip one entry in the table, set the frequency to
+CPUFREQ_ENTRY_INVALID. The entries don't need to be in sorted in any
+particular order, but if they are cpufreq core will do DVFS a bit
+quickly for them as search for best match is faster.
+
+By calling cpufreq_table_validate_and_show(), the cpuinfo.min_freq and
+cpuinfo.max_freq values are detected, and policy->min and policy->max
+are set to the same values. This is helpful for the per-CPU
+initialization stage.
+
+cpufreq_frequency_table_verify() assures that at least one valid
+frequency is within policy->min and policy->max, and all other criteria
+are met. This is helpful for the ->verify call.
+
+cpufreq_frequency_table_target() is the corresponding frequency table
+helper for the ->target stage. Just pass the values to this function,
+and this function returns the of the frequency table entry which
+contains the frequency the CPU shall be set to.
The following macros can be used as iterators over cpufreq_frequency_table:
cpufreq_for_each_entry(pos, table) - iterates over all entries of frequency
table.
-cpufreq-for_each_valid_entry(pos, table) - iterates over all entries,
+cpufreq_for_each_valid_entry(pos, table) - iterates over all entries,
excluding CPUFREQ_ENTRY_INVALID frequencies.
Use arguments "pos" - a cpufreq_frequency_table * as a loop cursor and
"table" - the cpufreq_frequency_table * you want to iterate over.
- total_trans
- trans_table
-All the statistics will be from the time the stats driver has been inserted
-to the time when a read of a particular statistic is done. Obviously, stats
-driver will not have any information about the frequency transitions before
-the stats driver insertion.
+All the statistics will be from the time the stats driver has been inserted
+(or the time the stats were reset) to the time when a read of a particular
+statistic is done. Obviously, stats driver will not have any information
+about the frequency transitions before the stats driver insertion.
--------------------------------------------------------------------------------
<mysystem>:/sys/devices/system/cpu/cpu0/cpufreq/stats # ls -l
CPU Frequency scaling --->
[*] CPU Frequency scaling
[*] CPU frequency translation statistics
- [*] CPU frequency translation statistics details
"CPU Frequency scaling" (CONFIG_CPU_FREQ) should be enabled to configure
cpufreq-stats.
"CPU frequency translation statistics" (CONFIG_CPU_FREQ_STAT) provides the
-basic statistics which includes time_in_state and total_trans.
+statistics which includes time_in_state, total_trans and trans_table.
-"CPU frequency translation statistics details" (CONFIG_CPU_FREQ_STAT_DETAILS)
-provides fine grained cpufreq stats by trans_table. The reason for having a
-separate config option for trans_table is:
-- trans_table goes against the traditional /sysfs rule of one value per
- interface. It provides a whole bunch of value in a 2 dimensional matrix
- form.
-
-Once these two options are enabled and your CPU supports cpufrequency, you
+Once this option is enabled and your CPU supports cpufrequency, you
will be able to see the CPU frequency statistics in /sysfs.
-
-
-
-
Dominik Brodowski <linux@brodo.de>
some additions and corrections by Nico Golde <nico@ngolde.de>
+ Rafael J. Wysocki <rafael.j.wysocki@intel.com>
+ Viresh Kumar <viresh.kumar@linaro.org>
2.3 Userspace
2.4 Ondemand
2.5 Conservative
+2.6 Schedutil
3. The Governor Interface in the CPUfreq Core
+4. References
1. What Is A CPUFreq Governor?
==============================
Most cpufreq drivers (except the intel_pstate and longrun) or even most
-cpu frequency scaling algorithms only offer the CPU to be set to one
-frequency. In order to offer dynamic frequency scaling, the cpufreq
-core must be able to tell these drivers of a "target frequency". So
-these specific drivers will be transformed to offer a "->target/target_index"
-call instead of the existing "->setpolicy" call. For "longrun", all
-stays the same, though.
+cpu frequency scaling algorithms only allow the CPU frequency to be set
+to predefined fixed values. In order to offer dynamic frequency
+scaling, the cpufreq core must be able to tell these drivers of a
+"target frequency". So these specific drivers will be transformed to
+offer a "->target/target_index/fast_switch()" call instead of the
+"->setpolicy()" call. For set_policy drivers, all stays the same,
+though.
How to decide what frequency within the CPUfreq policy should be used?
-That's done using "cpufreq governors". Two are already in this patch
--- they're the already existing "powersave" and "performance" which
-set the frequency statically to the lowest or highest frequency,
-respectively. At least two more such governors will be ready for
-addition in the near future, but likely many more as there are various
-different theories and models about dynamic frequency scaling
-around. Using such a generic interface as cpufreq offers to scaling
-governors, these can be tested extensively, and the best one can be
-selected for each specific use.
+That's done using "cpufreq governors".
Basically, it's the following flow graph:
/ the limits of policy->{min,max}
/ \
/ \
- Using the ->setpolicy call, Using the ->target/target_index call,
+ Using the ->setpolicy call, Using the ->target/target_index/fast_switch call,
the limits and the the frequency closest
"policy" is set. to target_freq is set.
It is assured that it
2.4 Ondemand
------------
-The CPUfreq governor "ondemand" sets the CPU depending on the
-current usage. To do this the CPU must have the capability to
-switch the frequency very quickly. There are a number of sysfs file
-accessible parameters:
-
-sampling_rate: measured in uS (10^-6 seconds), this is how often you
-want the kernel to look at the CPU usage and to make decisions on
-what to do about the frequency. Typically this is set to values of
-around '10000' or more. It's default value is (cmp. with users-guide.txt):
-transition_latency * 1000
-Be aware that transition latency is in ns and sampling_rate is in us, so you
-get the same sysfs value by default.
-Sampling rate should always get adjusted considering the transition latency
-To set the sampling rate 750 times as high as the transition latency
-in the bash (as said, 1000 is default), do:
-echo `$(($(cat cpuinfo_transition_latency) * 750 / 1000)) \
- >ondemand/sampling_rate
-
-sampling_rate_min:
-The sampling rate is limited by the HW transition latency:
-transition_latency * 100
-Or by kernel restrictions:
-If CONFIG_NO_HZ_COMMON is set, the limit is 10ms fixed.
-If CONFIG_NO_HZ_COMMON is not set or nohz=off boot parameter is used, the
-limits depend on the CONFIG_HZ option:
-HZ=1000: min=20000us (20ms)
-HZ=250: min=80000us (80ms)
-HZ=100: min=200000us (200ms)
-The highest value of kernel and HW latency restrictions is shown and
-used as the minimum sampling rate.
-
-up_threshold: defines what the average CPU usage between the samplings
-of 'sampling_rate' needs to be for the kernel to make a decision on
-whether it should increase the frequency. For example when it is set
-to its default value of '95' it means that between the checking
-intervals the CPU needs to be on average more than 95% in use to then
-decide that the CPU frequency needs to be increased.
-
-ignore_nice_load: this parameter takes a value of '0' or '1'. When
-set to '0' (its default), all processes are counted towards the
-'cpu utilisation' value. When set to '1', the processes that are
-run with a 'nice' value will not count (and thus be ignored) in the
-overall usage calculation. This is useful if you are running a CPU
-intensive calculation on your laptop that you do not care how long it
-takes to complete as you can 'nice' it and prevent it from taking part
-in the deciding process of whether to increase your CPU frequency.
-
-sampling_down_factor: this parameter controls the rate at which the
-kernel makes a decision on when to decrease the frequency while running
-at top speed. When set to 1 (the default) decisions to reevaluate load
-are made at the same interval regardless of current clock speed. But
-when set to greater than 1 (e.g. 100) it acts as a multiplier for the
-scheduling interval for reevaluating load when the CPU is at its top
-speed due to high load. This improves performance by reducing the overhead
-of load evaluation and helping the CPU stay at its top speed when truly
-busy, rather than shifting back and forth in speed. This tunable has no
-effect on behavior at lower speeds/lower CPU loads.
-
-powersave_bias: this parameter takes a value between 0 to 1000. It
-defines the percentage (times 10) value of the target frequency that
-will be shaved off of the target. For example, when set to 100 -- 10%,
-when ondemand governor would have targeted 1000 MHz, it will target
-1000 MHz - (10% of 1000 MHz) = 900 MHz instead. This is set to 0
-(disabled) by default.
-When AMD frequency sensitivity powersave bias driver --
-drivers/cpufreq/amd_freq_sensitivity.c is loaded, this parameter
-defines the workload frequency sensitivity threshold in which a lower
-frequency is chosen instead of ondemand governor's original target.
-The frequency sensitivity is a hardware reported (on AMD Family 16h
-Processors and above) value between 0 to 100% that tells software how
-the performance of the workload running on a CPU will change when
-frequency changes. A workload with sensitivity of 0% (memory/IO-bound)
-will not perform any better on higher core frequency, whereas a
-workload with sensitivity of 100% (CPU-bound) will perform better
-higher the frequency. When the driver is loaded, this is set to 400
-by default -- for CPUs running workloads with sensitivity value below
-40%, a lower frequency is chosen. Unloading the driver or writing 0
-will disable this feature.
+The CPUfreq governor "ondemand" sets the CPU frequency depending on the
+current system load. Load estimation is triggered by the scheduler
+through the update_util_data->func hook; when triggered, cpufreq checks
+the CPU-usage statistics over the last period and the governor sets the
+CPU accordingly. The CPU must have the capability to switch the
+frequency very quickly.
+
+Sysfs files:
+
+* sampling_rate:
+
+ Measured in uS (10^-6 seconds), this is how often you want the kernel
+ to look at the CPU usage and to make decisions on what to do about the
+ frequency. Typically this is set to values of around '10000' or more.
+ It's default value is (cmp. with users-guide.txt): transition_latency
+ * 1000. Be aware that transition latency is in ns and sampling_rate
+ is in us, so you get the same sysfs value by default. Sampling rate
+ should always get adjusted considering the transition latency to set
+ the sampling rate 750 times as high as the transition latency in the
+ bash (as said, 1000 is default), do:
+
+ $ echo `$(($(cat cpuinfo_transition_latency) * 750 / 1000)) > ondemand/sampling_rate
+
+* sampling_rate_min:
+
+ The sampling rate is limited by the HW transition latency:
+ transition_latency * 100
+
+ Or by kernel restrictions:
+ - If CONFIG_NO_HZ_COMMON is set, the limit is 10ms fixed.
+ - If CONFIG_NO_HZ_COMMON is not set or nohz=off boot parameter is
+ used, the limits depend on the CONFIG_HZ option:
+ HZ=1000: min=20000us (20ms)
+ HZ=250: min=80000us (80ms)
+ HZ=100: min=200000us (200ms)
+
+ The highest value of kernel and HW latency restrictions is shown and
+ used as the minimum sampling rate.
+
+* up_threshold:
+
+ This defines what the average CPU usage between the samplings of
+ 'sampling_rate' needs to be for the kernel to make a decision on
+ whether it should increase the frequency. For example when it is set
+ to its default value of '95' it means that between the checking
+ intervals the CPU needs to be on average more than 95% in use to then
+ decide that the CPU frequency needs to be increased.
+
+* ignore_nice_load:
+
+ This parameter takes a value of '0' or '1'. When set to '0' (its
+ default), all processes are counted towards the 'cpu utilisation'
+ value. When set to '1', the processes that are run with a 'nice'
+ value will not count (and thus be ignored) in the overall usage
+ calculation. This is useful if you are running a CPU intensive
+ calculation on your laptop that you do not care how long it takes to
+ complete as you can 'nice' it and prevent it from taking part in the
+ deciding process of whether to increase your CPU frequency.
+
+* sampling_down_factor:
+
+ This parameter controls the rate at which the kernel makes a decision
+ on when to decrease the frequency while running at top speed. When set
+ to 1 (the default) decisions to reevaluate load are made at the same
+ interval regardless of current clock speed. But when set to greater
+ than 1 (e.g. 100) it acts as a multiplier for the scheduling interval
+ for reevaluating load when the CPU is at its top speed due to high
+ load. This improves performance by reducing the overhead of load
+ evaluation and helping the CPU stay at its top speed when truly busy,
+ rather than shifting back and forth in speed. This tunable has no
+ effect on behavior at lower speeds/lower CPU loads.
+
+* powersave_bias:
+
+ This parameter takes a value between 0 to 1000. It defines the
+ percentage (times 10) value of the target frequency that will be
+ shaved off of the target. For example, when set to 100 -- 10%, when
+ ondemand governor would have targeted 1000 MHz, it will target
+ 1000 MHz - (10% of 1000 MHz) = 900 MHz instead. This is set to 0
+ (disabled) by default.
+
+ When AMD frequency sensitivity powersave bias driver --
+ drivers/cpufreq/amd_freq_sensitivity.c is loaded, this parameter
+ defines the workload frequency sensitivity threshold in which a lower
+ frequency is chosen instead of ondemand governor's original target.
+ The frequency sensitivity is a hardware reported (on AMD Family 16h
+ Processors and above) value between 0 to 100% that tells software how
+ the performance of the workload running on a CPU will change when
+ frequency changes. A workload with sensitivity of 0% (memory/IO-bound)
+ will not perform any better on higher core frequency, whereas a
+ workload with sensitivity of 100% (CPU-bound) will perform better
+ higher the frequency. When the driver is loaded, this is set to 400 by
+ default -- for CPUs running workloads with sensitivity value below
+ 40%, a lower frequency is chosen. Unloading the driver or writing 0
+ will disable this feature.
2.5 Conservative
----------------
The CPUfreq governor "conservative", much like the "ondemand"
-governor, sets the CPU depending on the current usage. It differs in
-behaviour in that it gracefully increases and decreases the CPU speed
-rather than jumping to max speed the moment there is any load on the
-CPU. This behaviour more suitable in a battery powered environment.
-The governor is tweaked in the same manner as the "ondemand" governor
-through sysfs with the addition of:
-
-freq_step: this describes what percentage steps the cpu freq should be
-increased and decreased smoothly by. By default the cpu frequency will
-increase in 5% chunks of your maximum cpu frequency. You can change this
-value to anywhere between 0 and 100 where '0' will effectively lock your
-CPU at a speed regardless of its load whilst '100' will, in theory, make
-it behave identically to the "ondemand" governor.
-
-down_threshold: same as the 'up_threshold' found for the "ondemand"
-governor but for the opposite direction. For example when set to its
-default value of '20' it means that if the CPU usage needs to be below
-20% between samples to have the frequency decreased.
-
-sampling_down_factor: similar functionality as in "ondemand" governor.
-But in "conservative", it controls the rate at which the kernel makes
-a decision on when to decrease the frequency while running in any
-speed. Load for frequency increase is still evaluated every
-sampling rate.
+governor, sets the CPU frequency depending on the current usage. It
+differs in behaviour in that it gracefully increases and decreases the
+CPU speed rather than jumping to max speed the moment there is any load
+on the CPU. This behaviour is more suitable in a battery powered
+environment. The governor is tweaked in the same manner as the
+"ondemand" governor through sysfs with the addition of:
+
+* freq_step:
+
+ This describes what percentage steps the cpu freq should be increased
+ and decreased smoothly by. By default the cpu frequency will increase
+ in 5% chunks of your maximum cpu frequency. You can change this value
+ to anywhere between 0 and 100 where '0' will effectively lock your CPU
+ at a speed regardless of its load whilst '100' will, in theory, make
+ it behave identically to the "ondemand" governor.
+
+* down_threshold:
+
+ Same as the 'up_threshold' found for the "ondemand" governor but for
+ the opposite direction. For example when set to its default value of
+ '20' it means that if the CPU usage needs to be below 20% between
+ samples to have the frequency decreased.
+
+* sampling_down_factor:
+
+ Similar functionality as in "ondemand" governor. But in
+ "conservative", it controls the rate at which the kernel makes a
+ decision on when to decrease the frequency while running in any speed.
+ Load for frequency increase is still evaluated every sampling rate.
+
+
+2.6 Schedutil
+-------------
+
+The "schedutil" governor aims at better integration with the Linux
+kernel scheduler. Load estimation is achieved through the scheduler's
+Per-Entity Load Tracking (PELT) mechanism, which also provides
+information about the recent load [1]. This governor currently does
+load based DVFS only for tasks managed by CFS. RT and DL scheduler tasks
+are always run at the highest frequency. Unlike all the other
+governors, the code is located under the kernel/sched/ directory.
+
+Sysfs files:
+
+* rate_limit_us:
+
+ This contains a value in microseconds. The governor waits for
+ rate_limit_us time before reevaluating the load again, after it has
+ evaluated the load once.
+
+For an in-depth comparison with the other governors refer to [2].
+
3. The Governor Interface in the CPUfreq Core
=============================================
"cpufreq_register_governor". The struct cpufreq_governor, which has to
be passed to that function, must contain the following values:
-governor->name - A unique name for this governor
-governor->governor - The governor callback function
-governor->owner - .THIS_MODULE for the governor module (if
- appropriate)
-
-The governor->governor callback is called with the current (or to-be-set)
-cpufreq_policy struct for that CPU, and an unsigned int event. The
-following events are currently defined:
-
-CPUFREQ_GOV_START: This governor shall start its duty for the CPU
- policy->cpu
-CPUFREQ_GOV_STOP: This governor shall end its duty for the CPU
- policy->cpu
-CPUFREQ_GOV_LIMITS: The limits for CPU policy->cpu have changed to
- policy->min and policy->max.
-
-If you need other "events" externally of your driver, _only_ use the
-cpufreq_governor_l(unsigned int cpu, unsigned int event) call to the
-CPUfreq core to ensure proper locking.
+governor->name - A unique name for this governor.
+governor->owner - .THIS_MODULE for the governor module (if appropriate).
+plus a set of hooks to the functions implementing the governor's logic.
The CPUfreq governor may call the CPU processor driver using one of
these two functions:
unsigned int relation);
target_freq must be within policy->min and policy->max, of course.
-What's the difference between these two functions? When your governor
-still is in a direct code path of a call to governor->governor, the
-per-CPU cpufreq lock is still held in the cpufreq core, and there's
-no need to lock it again (in fact, this would cause a deadlock). So
-use __cpufreq_driver_target only in these cases. In all other cases
-(for example, when there's a "daemonized" function that wakes up
-every second), use cpufreq_driver_target to lock the cpufreq per-CPU
-lock before the command is passed to the cpufreq processor driver.
+What's the difference between these two functions? When your governor is
+in a direct code path of a call to governor callbacks, like
+governor->start(), the policy->rwsem is still held in the cpufreq core,
+and there's no need to lock it again (in fact, this would cause a
+deadlock). So use __cpufreq_driver_target only in these cases. In all
+other cases (for example, when there's a "daemonized" function that
+wakes up every second), use cpufreq_driver_target to take policy->rwsem
+before the command is passed to the cpufreq driver.
+
+4. References
+=============
+
+[1] Per-entity load tracking: https://lwn.net/Articles/531853/
+[2] Improvements in CPU frequency management: https://lwn.net/Articles/682391/
Documents in this directory:
----------------------------
+
+amd-powernow.txt - AMD powernow driver specific file.
+
+boost.txt - Frequency boosting support.
+
core.txt - General description of the CPUFreq core and
- of CPUFreq notifiers
+ of CPUFreq notifiers.
+
+cpu-drivers.txt - How to implement a new cpufreq processor driver.
-cpu-drivers.txt - How to implement a new cpufreq processor driver
+cpufreq-nforce2.txt - nVidia nForce2 platform specific file.
+
+cpufreq-stats.txt - General description of sysfs cpufreq stats.
governors.txt - What are cpufreq governors and how to
implement them?
index.txt - File index, Mailing list and Links (this document)
+intel-pstate.txt - Intel pstate cpufreq driver specific file.
+
+pcc-cpufreq.txt - PCC cpufreq driver specific file.
+
user-guide.txt - User Guide to CPUFreq
------------
There is a CPU frequency changing CVS commit and general list where
you can report bugs, problems or submit patches. To post a message,
-send an email to linux-pm@vger.kernel.org, to subscribe go to
-http://vger.kernel.org/vger-lists.html#linux-pm and follow the
-instructions there.
+send an email to linux-pm@vger.kernel.org.
Links
-----
* http://cvs.arm.linux.org.uk/
the CPUFreq Mailing list:
-* http://vger.kernel.org/vger-lists.html#cpufreq
+* http://vger.kernel.org/vger-lists.html#linux-pm
Clock and voltage scaling for the SA-1100:
* http://www.lartmaker.nl/projects/scaling
Refer to "Intel® 64 and IA-32 Architectures Software Developer’s Manual
Volume 3: System Programming Guide" to understand ratios.
+There is one more sysfs attribute in /sys/devices/system/cpu/intel_pstate/
+that can be used for controlling the operation mode of the driver:
+
+ status: Three settings are possible:
+ "off" - The driver is not in use at this time.
+ "active" - The driver works as a P-state governor (default).
+ "passive" - The driver works as a regular cpufreq one and collaborates
+ with the generic cpufreq governors (it sets P-states as
+ requested by those governors).
+ The current setting is returned by reads from this attribute. Writing one
+ of the above strings to it changes the operation mode as indicated by that
+ string, if possible. If HW-managed P-states (HWP) are enabled, it is not
+ possible to change the driver's operation mode and attempts to write to
+ this attribute will fail.
+
cpufreq sysfs for Intel P-State
Since this driver registers with cpufreq, cpufreq sysfs is also presented.
Contents:
---------
1. Supported Architectures and Processors
-1.1 ARM
+1.1 ARM and ARM64
1.2 x86
1.3 sparc64
1.4 ppc
1. Supported Architectures and Processors
=========================================
-1.1 ARM
--------
-
-The following ARM processors are supported by cpufreq:
-
-ARM Integrator
-ARM-SA1100
-ARM-SA1110
-Intel PXA
+1.1 ARM and ARM64
+-----------------
+Almost all ARM and ARM64 platforms support CPU frequency scaling.
1.2 x86
-------
Transmeta Efficeon
VIA Cyrix 3 / C3
various processors on some ACPI 2.0-compatible systems [*]
+And many more
[*] Only if "ACPI Processor Performance States" are available
to the ACPI<->BIOS interface.
"cpufreq" within the cpu-device directory
(e.g. /sys/devices/system/cpu/cpu0/cpufreq/ for the first CPU).
+affected_cpus : List of Online CPUs that require software
+ coordination of frequency.
+
+cpuinfo_cur_freq : Current frequency of the CPU as obtained from
+ the hardware, in KHz. This is the frequency
+ the CPU actually runs at.
+
cpuinfo_min_freq : this file shows the minimum operating
frequency the processor can run at(in kHz)
+
cpuinfo_max_freq : this file shows the maximum operating
frequency the processor can run at(in kHz)
+
cpuinfo_transition_latency The time it takes on this CPU to
switch between two frequencies in nano
seconds. If unknown or known to be
userspace daemon. Make sure to not
switch the frequency too often
resulting in performance loss.
-scaling_driver : this file shows what cpufreq driver is
- used to set the frequency on this CPU
+
+related_cpus : List of Online + Offline CPUs that need software
+ coordination of frequency.
+
+scaling_available_frequencies : List of available frequencies, in KHz.
scaling_available_governors : this file shows the CPUfreq governors
available in this kernel. You can see the
currently activated governor in
+scaling_cur_freq : Current frequency of the CPU as determined by
+ the governor and cpufreq core, in KHz. This is
+ the frequency the kernel thinks the CPU runs
+ at.
+
+scaling_driver : this file shows what cpufreq driver is
+ used to set the frequency on this CPU
+
scaling_governor, and by "echoing" the name of another
governor you can change it. Please note
that some governors won't load - they only
work on some specific architectures or
processors.
-cpuinfo_cur_freq : Current frequency of the CPU as obtained from
- the hardware, in KHz. This is the frequency
- the CPU actually runs at.
-
-scaling_available_frequencies : List of available frequencies, in KHz.
-
scaling_min_freq and
scaling_max_freq show the current "policy limits" (in
kHz). By echoing new values into these
first set scaling_max_freq, then
scaling_min_freq.
-affected_cpus : List of Online CPUs that require software
- coordination of frequency.
-
-related_cpus : List of Online + Offline CPUs that need software
- coordination of frequency.
-
-scaling_cur_freq : Current frequency of the CPU as determined by
- the governor and cpufreq core, in KHz. This is
- the frequency the kernel thinks the CPU runs
- at.
+scaling_setspeed This can be read to get the currently programmed
+ value by the governor. This can be written to
+ change the current frequency for a group of
+ CPUs, represented by a policy. This is supported
+ currently only by the userspace governor.
bios_limit : If the BIOS tells the OS to limit a CPU to
lower frequencies, the user can read out the
--- /dev/null
+TI CPUFreq and OPP bindings
+================================
+
+Certain TI SoCs, like those in the am335x, am437x, am57xx, and dra7xx
+families support different OPPs depending on the silicon variant in use.
+The ti-cpufreq driver can use revision and an efuse value from the SoC to
+provide the OPP framework with supported hardware information. This is
+used to determine which OPPs from the operating-points-v2 table get enabled
+when it is parsed by the OPP framework.
+
+Required properties:
+--------------------
+In 'cpus' nodes:
+- operating-points-v2: Phandle to the operating-points-v2 table to use.
+
+In 'operating-points-v2' table:
+- compatible: Should be
+ - 'operating-points-v2-ti-cpu' for am335x, am43xx, and dra7xx/am57xx SoCs
+- syscon: A phandle pointing to a syscon node representing the control module
+ register space of the SoC.
+
+Optional properties:
+--------------------
+For each opp entry in 'operating-points-v2' table:
+- opp-supported-hw: Two bitfields indicating:
+ 1. Which revision of the SoC the OPP is supported by
+ 2. Which eFuse bits indicate this OPP is available
+
+ A bitwise AND is performed against these values and if any bit
+ matches, the OPP gets enabled.
+
+Example:
+--------
+
+/* From arch/arm/boot/dts/am33xx.dtsi */
+cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ cpu@0 {
+ compatible = "arm,cortex-a8";
+ device_type = "cpu";
+ reg = <0>;
+
+ operating-points-v2 = <&cpu0_opp_table>;
+
+ clocks = <&dpll_mpu_ck>;
+ clock-names = "cpu";
+
+ clock-latency = <300000>; /* From omap-cpufreq driver */
+ };
+};
+
+/*
+ * cpu0 has different OPPs depending on SoC revision and some on revisions
+ * 0x2 and 0x4 have eFuse bits that indicate if they are available or not
+ */
+cpu0_opp_table: opp-table {
+ compatible = "operating-points-v2-ti-cpu";
+ syscon = <&scm_conf>;
+
+ /*
+ * The three following nodes are marked with opp-suspend
+ * because they can not be enabled simultaneously on a
+ * single SoC.
+ */
+ opp50@300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ opp-microvolt = <950000 931000 969000>;
+ opp-supported-hw = <0x06 0x0010>;
+ opp-suspend;
+ };
+
+ opp100@275000000 {
+ opp-hz = /bits/ 64 <275000000>;
+ opp-microvolt = <1100000 1078000 1122000>;
+ opp-supported-hw = <0x01 0x00FF>;
+ opp-suspend;
+ };
+
+ opp100@300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ opp-microvolt = <1100000 1078000 1122000>;
+ opp-supported-hw = <0x06 0x0020>;
+ opp-suspend;
+ };
+
+ opp100@500000000 {
+ opp-hz = /bits/ 64 <500000000>;
+ opp-microvolt = <1100000 1078000 1122000>;
+ opp-supported-hw = <0x01 0xFFFF>;
+ };
+
+ opp100@600000000 {
+ opp-hz = /bits/ 64 <600000000>;
+ opp-microvolt = <1100000 1078000 1122000>;
+ opp-supported-hw = <0x06 0x0040>;
+ };
+
+ opp120@600000000 {
+ opp-hz = /bits/ 64 <600000000>;
+ opp-microvolt = <1200000 1176000 1224000>;
+ opp-supported-hw = <0x01 0xFFFF>;
+ };
+
+ opp120@720000000 {
+ opp-hz = /bits/ 64 <720000000>;
+ opp-microvolt = <1200000 1176000 1224000>;
+ opp-supported-hw = <0x06 0x0080>;
+ };
+
+ oppturbo@720000000 {
+ opp-hz = /bits/ 64 <720000000>;
+ opp-microvolt = <1260000 1234800 1285200>;
+ opp-supported-hw = <0x01 0xFFFF>;
+ };
+
+ oppturbo@800000000 {
+ opp-hz = /bits/ 64 <800000000>;
+ opp-microvolt = <1260000 1234800 1285200>;
+ opp-supported-hw = <0x06 0x0100>;
+ };
+
+ oppnitro@1000000000 {
+ opp-hz = /bits/ 64 <1000000000>;
+ opp-microvolt = <1325000 1298500 1351500>;
+ opp-supported-hw = <0x04 0x0200>;
+ };
+};
|--- FSYS
|--- FSYS2
+- In case of Exynos5433, there is VDD_INT power line as following:
+ VDD_INT |--- G2D (parent device)
+ |--- MSCL
+ |--- GSCL
+ |--- JPEG
+ |--- MFC
+ |--- HEVC
+ |--- BUS0
+ |--- BUS1
+ |--- BUS2
+ |--- PERIS (Fixed clock rate)
+ |--- PERIC (Fixed clock rate)
+ |--- FSYS (Fixed clock rate)
+
Example1:
Show the AXI buses of Exynos3250 SoC. Exynos3250 divides the buses to
power line (regulator). The MIF (Memory Interface) AXI bus is used to
Second cell specifies the irq distribution mode to cores
0=Round Robin; 1=cpu0, 2=cpu1, 4=cpu2, 8=cpu3
+ The second cell in interrupts property is deprecated and may be ignored by
+ the kernel.
+
intc accessed via the special ARC AUX register interface, hence "reg" property
is not specified.
* Ethernet controller node
Required properties:
-- compatible: Should be "mediatek,mt7623-eth"
+- compatible: Should be "mediatek,mt2701-eth"
- reg: Address and length of the register set for the device
- interrupts: Should contain the three frame engines interrupts in numeric
order. These are fe_int0, fe_int1 and fe_int2.
specifications. If neither of these are specified, the default is to
assume clause 22.
- If the phy's identifier is known then the list may contain an entry
- of the form: "ethernet-phy-idAAAA.BBBB" where
+ If the PHY reports an incorrect ID (or none at all) then the
+ "compatible" list may contain an entry with the correct PHY ID in the
+ form: "ethernet-phy-idAAAA.BBBB" where
AAAA - The value of the 16 bit Phy Identifier 1 register as
4 hex digits. This is the chip vendor OUI bits 3:18
BBBB - The value of the 16 bit Phy Identifier 2 register as
snapshot of a moment, you can see /proc/<pid>/smaps file and scan page table.
It's slow but very precise.
-Table 1-2: Contents of the status files (as of 4.1)
+Table 1-2: Contents of the status files (as of 4.8)
..............................................................................
Field Content
Name filename of the executable
+ Umask file mode creation mask
State state (R is running, S is sleeping, D is sleeping
in an uninterruptible wait, Z is zombie,
T is traced or stopped)
TracerPid PID of process tracing this process (0 if not)
Uid Real, effective, saved set, and file system UIDs
Gid Real, effective, saved set, and file system GIDs
- Umask file mode creation mask
FDSize number of file descriptor slots currently allocated
Groups supplementary group list
NStgid descendant namespace thread group ID hierarchy
VmPeak peak virtual memory size
VmSize total program size
VmLck locked memory size
+ VmPin pinned memory size
VmHWM peak resident set size ("high water mark")
VmRSS size of memory portions. It contains the three
following parts (VmRSS = RssAnon + RssFile + RssShmem)
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
Closes the cec device. Resources associated with the file descriptor are
freed. The device configuration remain unchanged.
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
The :c:func:`ioctl()` function manipulates cec device parameters. The
argument ``fd`` must be an open file descriptor.
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
To open a cec device applications call :c:func:`open()` with the
desired device name. The function has no side effects; the device
configuration remain unchanged.
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
With the :c:func:`poll()` function applications can wait for CEC
events.
Introduction
============
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
HDMI connectors provide a single pin for use by the Consumer Electronics
Control protocol. This protocol allows different devices connected by an
HDMI cable to communicate. The protocol for CEC version 1.4 is defined
Drivers that support CEC will create a CEC device node (/dev/cecX) to
give userspace access to the CEC adapter. The
:ref:`CEC_ADAP_G_CAPS` ioctl will tell userspace what it is allowed to do.
+
+In order to check the support and test it, it is suggested to download
+the `v4l-utils <https://git.linuxtv.org/v4l-utils.git/>`_ package. It
+provides three tools to handle CEC:
+
+- cec-ctl: the Swiss army knife of CEC. Allows you to configure, transmit
+ and monitor CEC messages.
+
+- cec-compliance: does a CEC compliance test of a remote CEC device to
+ determine how compliant the CEC implementation is.
+
+- cec-follower: emulates a CEC follower.
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
All cec devices must support :ref:`ioctl CEC_ADAP_G_CAPS <CEC_ADAP_G_CAPS>`. To query
device information, applications call the ioctl with a pointer to a
struct :c:type:`cec_caps`. The driver fills the structure and
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
To query the current CEC logical addresses, applications call
:ref:`ioctl CEC_ADAP_G_LOG_ADDRS <CEC_ADAP_G_LOG_ADDRS>` with a pointer to a
struct :c:type:`cec_log_addrs` where the driver stores the logical addresses.
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
To query the current physical address applications call
:ref:`ioctl CEC_ADAP_G_PHYS_ADDR <CEC_ADAP_G_PHYS_ADDR>` with a pointer to a __u16 where the
driver stores the physical address.
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
CEC devices can send asynchronous events. These can be retrieved by
calling :c:func:`CEC_DQEVENT`. If the file descriptor is in
non-blocking mode and no event is pending, then it will return -1 and
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
By default any filehandle can use :ref:`CEC_TRANSMIT`, but in order to prevent
applications from stepping on each others toes it must be possible to
obtain exclusive access to the CEC adapter. This ioctl sets the
Description
===========
-.. note::
-
- This documents the proposed CEC API. This API is not yet finalized
- and is currently only available as a staging kernel module.
-
To receive a CEC message the application has to fill in the
``timeout`` field of struct :c:type:`cec_msg` and pass it to
:ref:`ioctl CEC_RECEIVE <CEC_RECEIVE>`.
specific framework which uses the OPP library. Similar care needs to be taken
care to refresh the cpufreq table in cases of these operations.
-WARNING on OPP List locking mechanism:
--------------------------------------------------
-OPP library uses RCU for exclusivity. RCU allows the query functions to operate
-in multiple contexts and this synchronization mechanism is optimal for a read
-intensive operations on data structure as the OPP library caters to.
-
-To ensure that the data retrieved are sane, the users such as SoC framework
-should ensure that the section of code operating on OPP queries are locked
-using RCU read locks. The opp_find_freq_{exact,ceil,floor},
-opp_get_{voltage, freq, opp_count} fall into this category.
-
-opp_{add,enable,disable} are updaters which use mutex and implement it's own
-RCU locking mechanisms. These functions should *NOT* be called under RCU locks
-and other contexts that prevent blocking functions in RCU or mutex operations
-from working.
-
2. Initial OPP List Registration
================================
The SoC implementation calls dev_pm_opp_add function iteratively to add OPPs per
found, else returns error. These errors are expected to be handled by standard
error checks such as IS_ERR() and appropriate actions taken by the caller.
+Callers of these functions shall call dev_pm_opp_put() after they have used the
+OPP. Otherwise the memory for the OPP will never get freed and result in
+memleak.
+
dev_pm_opp_find_freq_exact - Search for an OPP based on an *exact* frequency and
availability. This function is especially useful to enable an OPP which
is not available by default.
Example: In a case when SoC framework detects a situation where a
higher frequency could be made available, it can use this function to
find the OPP prior to call the dev_pm_opp_enable to actually make it available.
- rcu_read_lock();
opp = dev_pm_opp_find_freq_exact(dev, 1000000000, false);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
/* dont operate on the pointer.. just do a sanity check.. */
if (IS_ERR(opp)) {
pr_err("frequency not disabled!\n");
frequency.
Example: To find the highest opp for a device:
freq = ULONG_MAX;
- rcu_read_lock();
- dev_pm_opp_find_freq_floor(dev, &freq);
- rcu_read_unlock();
+ opp = dev_pm_opp_find_freq_floor(dev, &freq);
+ dev_pm_opp_put(opp);
dev_pm_opp_find_freq_ceil - Search for an available OPP which is *at least* the
provided frequency. This function is useful while searching for a
frequency.
Example 1: To find the lowest opp for a device:
freq = 0;
- rcu_read_lock();
- dev_pm_opp_find_freq_ceil(dev, &freq);
- rcu_read_unlock();
+ opp = dev_pm_opp_find_freq_ceil(dev, &freq);
+ dev_pm_opp_put(opp);
Example 2: A simplified implementation of a SoC cpufreq_driver->target:
soc_cpufreq_target(..)
{
/* Do stuff like policy checks etc. */
/* Find the best frequency match for the req */
- rcu_read_lock();
opp = dev_pm_opp_find_freq_ceil(dev, &freq);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
if (!IS_ERR(opp))
soc_switch_to_freq_voltage(freq);
else
implementation might choose to do something as follows:
if (cur_temp < temp_low_thresh) {
/* Enable 1GHz if it was disabled */
- rcu_read_lock();
opp = dev_pm_opp_find_freq_exact(dev, 1000000000, false);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
/* just error check */
if (!IS_ERR(opp))
ret = dev_pm_opp_enable(dev, 1000000000);
choose to do something as follows:
if (cur_temp > temp_high_thresh) {
/* Disable 1GHz if it was enabled */
- rcu_read_lock();
opp = dev_pm_opp_find_freq_exact(dev, 1000000000, true);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
/* just error check */
if (!IS_ERR(opp))
ret = dev_pm_opp_disable(dev, 1000000000);
soc_switch_to_freq_voltage(freq)
{
/* do things */
- rcu_read_lock();
opp = dev_pm_opp_find_freq_ceil(dev, &freq);
v = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
if (v)
regulator_set_voltage(.., v);
/* do other things */
{
/* do things.. */
max_freq = ULONG_MAX;
- rcu_read_lock();
max_opp = dev_pm_opp_find_freq_floor(dev,&max_freq);
requested_opp = dev_pm_opp_find_freq_ceil(dev,&freq);
if (!IS_ERR(max_opp) && !IS_ERR(requested_opp))
r = soc_test_validity(max_opp, requested_opp);
- rcu_read_unlock();
+ dev_pm_opp_put(max_opp);
+ dev_pm_opp_put(requested_opp);
/* do other things */
}
soc_test_validity(..)
soc_notify_coproc_available_frequencies()
{
/* Do things */
- rcu_read_lock();
num_available = dev_pm_opp_get_opp_count(dev);
speeds = kzalloc(sizeof(u32) * num_available, GFP_KERNEL);
/* populate the table in increasing order */
speeds[i] = freq;
freq++;
i++;
+ dev_pm_opp_put(opp);
}
- rcu_read_unlock();
soc_notify_coproc(AVAILABLE_FREQs, speeds, num_available);
/* Do other things */
The default suspend mode (ie. the one to be used without writing anything into
/sys/power/mem_sleep) is either "deep" (if Suspend-To-RAM is supported) or
"s2idle", but it can be overridden by the value of the "mem_sleep_default"
-parameter in the kernel command line. On some ACPI-based systems, depending on
-the information in the FADT, the default may be "s2idle" even if Suspend-To-RAM
-is supported.
+parameter in the kernel command line.
The properties of all of the sleep states are described below.
F: drivers/*/*aspeed*
ARM/ATMEL AT91RM9200, AT91SAM9 AND SAMA5 SOC SUPPORT
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
M: Alexandre Belloni <alexandre.belloni@free-electrons.com>
M: Jean-Christophe Plagniol-Villard <plagnioj@jcrosoft.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
F: include/linux/soc/renesas/
ARM/SOCFPGA ARCHITECTURE
-M: Dinh Nguyen <dinguyen@opensource.altera.com>
+M: Dinh Nguyen <dinguyen@kernel.org>
S: Maintained
F: arch/arm/mach-socfpga/
F: arch/arm/boot/dts/socfpga*
T: git git://git.kernel.org/pub/scm/linux/kernel/git/dinguyen/linux.git
ARM/SOCFPGA CLOCK FRAMEWORK SUPPORT
-M: Dinh Nguyen <dinguyen@opensource.altera.com>
+M: Dinh Nguyen <dinguyen@kernel.org>
S: Maintained
F: drivers/clk/socfpga/
F: include/uapi/linux/atm*
ATMEL AT91 / AT32 MCI DRIVER
-M: Ludovic Desroches <ludovic.desroches@atmel.com>
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
S: Maintained
F: drivers/mmc/host/atmel-mci.c
ATMEL AT91 SAMA5D2-Compatible Shutdown Controller
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
S: Supported
F: drivers/power/reset/at91-sama5d2_shdwc.c
ATMEL SAMA5D2 ADC DRIVER
-M: Ludovic Desroches <ludovic.desroches@atmel.com>
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
L: linux-iio@vger.kernel.org
S: Supported
F: drivers/iio/adc/at91-sama5d2_adc.c
ATMEL Audio ALSA driver
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
L: alsa-devel@alsa-project.org (moderated for non-subscribers)
S: Supported
F: sound/soc/atmel
ATMEL XDMA DRIVER
-M: Ludovic Desroches <ludovic.desroches@atmel.com>
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
L: linux-arm-kernel@lists.infradead.org
L: dmaengine@vger.kernel.org
S: Supported
F: drivers/dma/at_xdmac.c
ATMEL I2C DRIVER
-M: Ludovic Desroches <ludovic.desroches@atmel.com>
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
L: linux-i2c@vger.kernel.org
S: Supported
F: drivers/i2c/busses/i2c-at91.c
ATMEL ISI DRIVER
-M: Ludovic Desroches <ludovic.desroches@atmel.com>
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
L: linux-media@vger.kernel.org
S: Supported
F: drivers/media/platform/soc_camera/atmel-isi.c
F: include/media/atmel-isi.h
ATMEL LCDFB DRIVER
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
L: linux-fbdev@vger.kernel.org
S: Maintained
F: drivers/video/fbdev/atmel_lcdfb.c
F: include/video/atmel_lcdc.h
ATMEL MACB ETHERNET DRIVER
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
S: Supported
F: drivers/net/ethernet/cadence/
F: drivers/mtd/nand/atmel_nand*
ATMEL SDMMC DRIVER
-M: Ludovic Desroches <ludovic.desroches@atmel.com>
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
L: linux-mmc@vger.kernel.org
S: Supported
F: drivers/mmc/host/sdhci-of-at91.c
ATMEL SPI DRIVER
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
S: Supported
F: drivers/spi/spi-atmel.*
ATMEL SSC DRIVER
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Supported
F: drivers/misc/atmel-ssc.c
F: include/linux/atmel-ssc.h
ATMEL Timer Counter (TC) AND CLOCKSOURCE DRIVERS
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Supported
F: drivers/misc/atmel_tclib.c
F: drivers/clocksource/tcb_clksrc.c
ATMEL USBA UDC DRIVER
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
+M: Nicolas Ferre <nicolas.ferre@microchip.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Supported
F: drivers/usb/gadget/udc/atmel_usba_udc.*
F: include/linux/bcm963xx_nvram.h
F: include/linux/bcm963xx_tag.h
+BROADCOM BMIPS CPUFREQ DRIVER
+M: Markus Mayer <mmayer@broadcom.com>
+M: bcm-kernel-feedback-list@broadcom.com
+L: linux-pm@vger.kernel.org
+S: Maintained
+F: drivers/cpufreq/bmips-cpufreq.c
+
BROADCOM TG3 GIGABIT ETHERNET DRIVER
M: Siva Reddy Kallam <siva.kallam@broadcom.com>
M: Prashant Sreedharan <prashant@broadcom.com>
F: include/uapi/rdma/cxgb3-abi.h
CXGB4 ETHERNET DRIVER (CXGB4)
-M: Hariprasad S <hariprasad@chelsio.com>
+M: Ganesh Goudar <ganeshgr@chelsio.com>
L: netdev@vger.kernel.org
W: http://www.chelsio.com
S: Supported
DRM DRIVER FOR BOCHS VIRTUAL GPU
M: Gerd Hoffmann <kraxel@redhat.com>
-S: Odd Fixes
+L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
+S: Maintained
F: drivers/gpu/drm/bochs/
DRM DRIVER FOR QEMU'S CIRRUS DEVICE
M: Dave Airlie <airlied@redhat.com>
-S: Odd Fixes
+M: Gerd Hoffmann <kraxel@redhat.com>
+L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
+S: Obsolete
+W: https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/
F: drivers/gpu/drm/cirrus/
RADEON and AMDGPU DRM DRIVERS
INTEL GVT-g DRIVERS (Intel GPU Virtualization)
M: Zhenyu Wang <zhenyuw@linux.intel.com>
M: Zhi Wang <zhi.a.wang@intel.com>
-L: igvt-g-dev@lists.01.org
+L: intel-gvt-dev@lists.freedesktop.org
L: intel-gfx@lists.freedesktop.org
W: https://01.org/igvt-g
T: git https://github.com/01org/gvt-linux.git
DRM DRIVER FOR QXL VIRTUAL GPU
M: Dave Airlie <airlied@redhat.com>
-S: Odd Fixes
+M: Gerd Hoffmann <kraxel@redhat.com>
+L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
+S: Maintained
F: drivers/gpu/drm/qxl/
F: include/uapi/drm/qxl_drm.h
F: drivers/pinctrl/pinctrl-at91.*
PIN CONTROLLER - ATMEL AT91 PIO4
-M: Ludovic Desroches <ludovic.desroches@atmel.com>
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
L: linux-gpio@vger.kernel.org
S: Supported
QUALCOMM ATHEROS ATH9K WIRELESS DRIVER
M: QCA ath9k Development <ath9k-devel@qca.qualcomm.com>
L: linux-wireless@vger.kernel.org
-L: ath9k-devel@lists.ath9k.org
W: http://wireless.kernel.org/en/users/Drivers/ath9k
S: Supported
F: drivers/net/wireless/ath/ath9k/
F: include/uapi/linux/userio.h
VIRTIO CONSOLE DRIVER
-M: Amit Shah <amit.shah@redhat.com>
+M: Amit Shah <amit@kernel.org>
L: virtualization@lists.linux-foundation.org
S: Maintained
F: drivers/char/virtio_console.c
M: Gerd Hoffmann <kraxel@redhat.com>
L: dri-devel@lists.freedesktop.org
L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
S: Maintained
F: drivers/gpu/drm/virtio/
F: include/uapi/linux/virtio_gpu.h
X86 PLATFORM DRIVERS
M: Darren Hart <dvhart@infradead.org>
+M: Andy Shevchenko <andy@infradead.org>
L: platform-driver-x86@vger.kernel.org
T: git git://git.infradead.org/users/dvhart/linux-platform-drivers-x86.git
S: Maintained
ZBUD COMPRESSED PAGE ALLOCATOR
M: Seth Jennings <sjenning@redhat.com>
+M: Dan Streetman <ddstreet@ieee.org>
L: linux-mm@kvack.org
S: Maintained
F: mm/zbud.c
ZSWAP COMPRESSED SWAP CACHING
M: Seth Jennings <sjenning@redhat.com>
+M: Dan Streetman <ddstreet@ieee.org>
L: linux-mm@kvack.org
S: Maintained
F: mm/zswap.c
VERSION = 4
PATCHLEVEL = 10
SUBLEVEL = 0
-EXTRAVERSION = -rc5
-NAME = Anniversary Edition
+EXTRAVERSION = -rc8
+NAME = Fearless Coyote
# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
KBUILD_ARFLAGS := $(call ar-option,D)
# check for 'asm goto'
-ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC)), y)
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC) $(KBUILD_CFLAGS)), y)
KBUILD_CFLAGS += -DCC_HAVE_ASM_GOTO
KBUILD_AFLAGS += -DCC_HAVE_ASM_GOTO
endif
" lp 1f \n"
" nop \n"
"1: \n"
- : : "r"(loops));
+ :
+ : "r"(loops)
+ : "lp_count");
}
extern void __bad_udelay(void);
GET_CPU_ID r5
cmp r5, 0
mov.nz r0, r5
-#ifdef CONFIG_ARC_SMP_HALT_ON_RESET
- ; Non-Master can proceed as system would be booted sufficiently
- jnz first_lines_of_secondary
-#else
+ bz .Lmaster_proceed
+
; Non-Masters wait for Master to boot enough and bring them up
- jnz arc_platform_smp_wait_to_boot
-#endif
- ; Master falls thru
+ ; when they resume, tail-call to entry point
+ mov blink, @first_lines_of_secondary
+ j arc_platform_smp_wait_to_boot
+
+.Lmaster_proceed:
#endif
; Clear BSS before updating any globals
READ_BCR(ARC_REG_MCIP_BCR, mp);
sprintf(smp_cpuinfo_buf,
- "Extn [SMP]\t: ARConnect (v%d): %d cores with %s%s%s%s%s\n",
+ "Extn [SMP]\t: ARConnect (v%d): %d cores with %s%s%s%s\n",
mp.ver, mp.num_cores,
IS_AVAIL1(mp.ipi, "IPI "),
IS_AVAIL1(mp.idu, "IDU "),
- IS_AVAIL1(mp.llm, "LLM "),
IS_AVAIL1(mp.dbg, "DEBUG "),
IS_AVAIL1(mp.gfrc, "GFRC"));
raw_spin_unlock_irqrestore(&mcip_lock, flags);
}
-#ifdef CONFIG_SMP
static int
idu_irq_set_affinity(struct irq_data *data, const struct cpumask *cpumask,
bool force)
return IRQ_SET_MASK_OK;
}
-#endif
+
+static void idu_irq_enable(struct irq_data *data)
+{
+ /*
+ * By default send all common interrupts to all available online CPUs.
+ * The affinity of common interrupts in IDU must be set manually since
+ * in some cases the kernel will not call irq_set_affinity() by itself:
+ * 1. When the kernel is not configured with support of SMP.
+ * 2. When the kernel is configured with support of SMP but upper
+ * interrupt controllers does not support setting of the affinity
+ * and cannot propagate it to IDU.
+ */
+ idu_irq_set_affinity(data, cpu_online_mask, false);
+ idu_irq_unmask(data);
+}
static struct irq_chip idu_irq_chip = {
.name = "MCIP IDU Intc",
.irq_mask = idu_irq_mask,
.irq_unmask = idu_irq_unmask,
+ .irq_enable = idu_irq_enable,
#ifdef CONFIG_SMP
.irq_set_affinity = idu_irq_set_affinity,
#endif
const u32 *intspec, unsigned int intsize,
irq_hw_number_t *out_hwirq, unsigned int *out_type)
{
- irq_hw_number_t hwirq = *out_hwirq = intspec[0];
- int distri = intspec[1];
- unsigned long flags;
-
+ /*
+ * Ignore value of interrupt distribution mode for common interrupts in
+ * IDU which resides in intspec[1] since setting an affinity using value
+ * from Device Tree is deprecated in ARC.
+ */
+ *out_hwirq = intspec[0];
*out_type = IRQ_TYPE_NONE;
- /* XXX: validate distribution scheme again online cpu mask */
- if (distri == 0) {
- /* 0 - Round Robin to all cpus, otherwise 1 bit per core */
- raw_spin_lock_irqsave(&mcip_lock, flags);
- idu_set_dest(hwirq, BIT(num_online_cpus()) - 1);
- idu_set_mode(hwirq, IDU_M_TRIG_LEVEL, IDU_M_DISTRI_RR);
- raw_spin_unlock_irqrestore(&mcip_lock, flags);
- } else {
- /*
- * DEST based distribution for Level Triggered intr can only
- * have 1 CPU, so generalize it to always contain 1 cpu
- */
- int cpu = ffs(distri);
-
- if (cpu != fls(distri))
- pr_warn("IDU irq %lx distri mode set to cpu %x\n",
- hwirq, cpu);
-
- raw_spin_lock_irqsave(&mcip_lock, flags);
- idu_set_dest(hwirq, cpu);
- idu_set_mode(hwirq, IDU_M_TRIG_LEVEL, IDU_M_DISTRI_DEST);
- raw_spin_unlock_irqrestore(&mcip_lock, flags);
- }
-
return 0;
}
*/
static volatile int wake_flag;
+#ifdef CONFIG_ISA_ARCOMPACT
+
+#define __boot_read(f) f
+#define __boot_write(f, v) f = v
+
+#else
+
+#define __boot_read(f) arc_read_uncached_32(&f)
+#define __boot_write(f, v) arc_write_uncached_32(&f, v)
+
+#endif
+
static void arc_default_smp_cpu_kick(int cpu, unsigned long pc)
{
BUG_ON(cpu == 0);
- wake_flag = cpu;
+
+ __boot_write(wake_flag, cpu);
}
void arc_platform_smp_wait_to_boot(int cpu)
{
- while (wake_flag != cpu)
+ /* for halt-on-reset, we've waited already */
+ if (IS_ENABLED(CONFIG_ARC_SMP_HALT_ON_RESET))
+ return;
+
+ while (__boot_read(wake_flag) != cpu)
;
- wake_flag = 0;
- __asm__ __volatile__("j @first_lines_of_secondary \n");
+ __boot_write(wake_flag, 0);
}
-
const char *arc_platform_smp_cpuinfo(void)
{
return plat_smp_ops.info ? : "";
if (state.fault)
goto fault;
+ /* clear any remanants of delay slot */
if (delay_mode(regs)) {
- regs->ret = regs->bta;
+ regs->ret = regs->bta & ~1U;
regs->status32 &= ~STATUS_DE_MASK;
} else {
regs->ret += state.instr_len;
orion5x-lacie-ethernet-disk-mini-v2.dtb \
orion5x-linkstation-lsgl.dtb \
orion5x-linkstation-lswtgl.dtb \
- orion5x-lschl.dtb \
+ orion5x-linkstation-lschl.dtb \
orion5x-lswsgl.dtb \
orion5x-maxtor-shared-storage-2.dtb \
orion5x-netgear-wnr854t.dtb \
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
gpio0 = &gpio1;
#size-cells = <1>;
interrupt-parent = <&icoll>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
gpio0 = &gpio0;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
#size-cells = <1>;
interrupt-parent = <&icoll>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &mac0;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
serial0 = &uart1;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
&gpio4 {
gpio-ranges = <&iomuxc 5 136 1>, <&iomuxc 6 145 1>, <&iomuxc 7 150 1>,
<&iomuxc 8 146 1>, <&iomuxc 9 151 1>, <&iomuxc 10 147 1>,
- <&iomuxc 11 151 1>, <&iomuxc 12 148 1>, <&iomuxc 13 153 1>,
+ <&iomuxc 11 152 1>, <&iomuxc 12 148 1>, <&iomuxc 13 153 1>,
<&iomuxc 14 149 1>, <&iomuxc 15 154 1>, <&iomuxc 16 39 7>,
<&iomuxc 23 56 1>, <&iomuxc 24 61 7>, <&iomuxc 31 46 1>;
};
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
can0 = &flexcan1;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
ethernet0 = &fec1;
/ {
#address-cells = <1>;
#size-cells = <1>;
+ /*
+ * The decompressor and also some bootloaders rely on a
+ * pre-existing /chosen node to be available to insert the
+ * command line and merge other ATAGS info.
+ * Also for U-Boot there must be a pre-existing /memory node.
+ */
+ chosen {};
+ memory { device_type = "memory"; reg = <0 0>; };
aliases {
gpio0 = &gpio1;
--- /dev/null
+/*
+ * Device Tree file for Buffalo Linkstation LS-CHLv3
+ *
+ * Copyright (C) 2016 Ash Hughes <ashley.hughes@blueyonder.co.uk>
+ * Copyright (C) 2015-2017
+ * Roger Shimizu <rogershimizu@gmail.com>
+ *
+ * This file is dual-licensed: you can use it either under the terms
+ * of the GPL or the X11 license, at your option. Note that this dual
+ * licensing only applies to this file, and not this project as a
+ * whole.
+ *
+ * a) This file is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This file is distributed in the hope that it will be useful
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Or, alternatively
+ *
+ * b) Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use
+ * copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following
+ * conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED , WITHOUT WARRANTY OF ANY KIND
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+ * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/dts-v1/;
+
+#include "orion5x-linkstation.dtsi"
+#include "mvebu-linkstation-gpio-simple.dtsi"
+#include "mvebu-linkstation-fan.dtsi"
+#include <dt-bindings/gpio/gpio.h>
+
+/ {
+ model = "Buffalo Linkstation LiveV3 (LS-CHL)";
+ compatible = "buffalo,lschl", "marvell,orion5x-88f5182", "marvell,orion5x";
+
+ memory { /* 128 MB */
+ device_type = "memory";
+ reg = <0x00000000 0x8000000>;
+ };
+
+ gpio_keys {
+ func {
+ label = "Function Button";
+ linux,code = <KEY_OPTION>;
+ gpios = <&gpio0 15 GPIO_ACTIVE_LOW>;
+ };
+
+ power-on-switch {
+ gpios = <&gpio0 8 GPIO_ACTIVE_LOW>;
+ };
+
+ power-auto-switch {
+ gpios = <&gpio0 10 GPIO_ACTIVE_LOW>;
+ };
+ };
+
+ gpio_leds {
+ pinctrl-0 = <&pmx_led_power &pmx_led_alarm &pmx_led_info &pmx_led_func>;
+ blue-power-led {
+ gpios = <&gpio0 0 GPIO_ACTIVE_LOW>;
+ };
+
+ red-alarm-led {
+ gpios = <&gpio0 2 GPIO_ACTIVE_LOW>;
+ };
+
+ amber-info-led {
+ gpios = <&gpio0 3 GPIO_ACTIVE_LOW>;
+ };
+
+ func {
+ label = "lschl:func:blue:top";
+ gpios = <&gpio0 17 GPIO_ACTIVE_LOW>;
+ };
+ };
+
+ gpio_fan {
+ gpios = <&gpio0 14 GPIO_ACTIVE_LOW
+ &gpio0 16 GPIO_ACTIVE_LOW>;
+
+ alarm-gpios = <&gpio0 6 GPIO_ACTIVE_HIGH>;
+ };
+};
+
+&pinctrl {
+ pmx_led_power: pmx-leds {
+ marvell,pins = "mpp0";
+ marvell,function = "gpio";
+ };
+
+ pmx_power_hdd: pmx-power-hdd {
+ marvell,pins = "mpp1";
+ marvell,function = "gpio";
+ };
+
+ pmx_led_alarm: pmx-leds {
+ marvell,pins = "mpp2";
+ marvell,function = "gpio";
+ };
+
+ pmx_led_info: pmx-leds {
+ marvell,pins = "mpp3";
+ marvell,function = "gpio";
+ };
+
+ pmx_fan_lock: pmx-fan-lock {
+ marvell,pins = "mpp6";
+ marvell,function = "gpio";
+ };
+
+ pmx_power_switch: pmx-power-switch {
+ marvell,pins = "mpp8", "mpp10", "mpp15";
+ marvell,function = "gpio";
+ };
+
+ pmx_power_usb: pmx-power-usb {
+ marvell,pins = "mpp9";
+ marvell,function = "gpio";
+ };
+
+ pmx_fan_high: pmx-fan-high {
+ marvell,pins = "mpp14";
+ marvell,function = "gpio";
+ };
+
+ pmx_fan_low: pmx-fan-low {
+ marvell,pins = "mpp16";
+ marvell,function = "gpio";
+ };
+
+ pmx_led_func: pmx-leds {
+ marvell,pins = "mpp17";
+ marvell,function = "gpio";
+ };
+
+ pmx_sw_init: pmx-sw-init {
+ marvell,pins = "mpp7";
+ marvell,function = "gpio";
+ };
+};
+
+&hdd_power {
+ gpios = <&gpio0 1 GPIO_ACTIVE_HIGH>;
+};
+
+&usb_power {
+ gpios = <&gpio0 9 GPIO_ACTIVE_HIGH>;
+};
+
+++ /dev/null
-/*
- * Device Tree file for Buffalo Linkstation LS-CHLv3
- *
- * Copyright (C) 2016 Ash Hughes <ashley.hughes@blueyonder.co.uk>
- * Copyright (C) 2015, 2016
- * Roger Shimizu <rogershimizu@gmail.com>
- *
- * This file is dual-licensed: you can use it either under the terms
- * of the GPL or the X11 license, at your option. Note that this dual
- * licensing only applies to this file, and not this project as a
- * whole.
- *
- * a) This file is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
- * License, or (at your option) any later version.
- *
- * This file is distributed in the hope that it will be useful
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * Or, alternatively
- *
- * b) Permission is hereby granted, free of charge, to any person
- * obtaining a copy of this software and associated documentation
- * files (the "Software"), to deal in the Software without
- * restriction, including without limitation the rights to use
- * copy, modify, merge, publish, distribute, sublicense, and/or
- * sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following
- * conditions:
- *
- * The above copyright notice and this permission notice shall be
- * included in all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED , WITHOUT WARRANTY OF ANY KIND
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
- * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
- * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY
- * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
- */
-
-/dts-v1/;
-
-#include "orion5x-linkstation.dtsi"
-#include "mvebu-linkstation-gpio-simple.dtsi"
-#include "mvebu-linkstation-fan.dtsi"
-#include <dt-bindings/gpio/gpio.h>
-
-/ {
- model = "Buffalo Linkstation Live v3 (LS-CHL)";
- compatible = "buffalo,lschl", "marvell,orion5x-88f5182", "marvell,orion5x";
-
- memory { /* 128 MB */
- device_type = "memory";
- reg = <0x00000000 0x8000000>;
- };
-
- gpio_keys {
- func {
- label = "Function Button";
- linux,code = <KEY_OPTION>;
- gpios = <&gpio0 15 GPIO_ACTIVE_LOW>;
- };
-
- power-on-switch {
- gpios = <&gpio0 8 GPIO_ACTIVE_LOW>;
- };
-
- power-auto-switch {
- gpios = <&gpio0 10 GPIO_ACTIVE_LOW>;
- };
- };
-
- gpio_leds {
- pinctrl-0 = <&pmx_led_power &pmx_led_alarm &pmx_led_info &pmx_led_func>;
- blue-power-led {
- gpios = <&gpio0 0 GPIO_ACTIVE_LOW>;
- };
-
- red-alarm-led {
- gpios = <&gpio0 2 GPIO_ACTIVE_LOW>;
- };
-
- amber-info-led {
- gpios = <&gpio0 3 GPIO_ACTIVE_LOW>;
- };
-
- func {
- label = "lschl:func:blue:top";
- gpios = <&gpio0 17 GPIO_ACTIVE_LOW>;
- };
- };
-
- gpio_fan {
- gpios = <&gpio0 14 GPIO_ACTIVE_LOW
- &gpio0 16 GPIO_ACTIVE_LOW>;
-
- alarm-gpios = <&gpio0 6 GPIO_ACTIVE_HIGH>;
- };
-};
-
-&pinctrl {
- pmx_led_power: pmx-leds {
- marvell,pins = "mpp0";
- marvell,function = "gpio";
- };
-
- pmx_power_hdd: pmx-power-hdd {
- marvell,pins = "mpp1";
- marvell,function = "gpio";
- };
-
- pmx_led_alarm: pmx-leds {
- marvell,pins = "mpp2";
- marvell,function = "gpio";
- };
-
- pmx_led_info: pmx-leds {
- marvell,pins = "mpp3";
- marvell,function = "gpio";
- };
-
- pmx_fan_lock: pmx-fan-lock {
- marvell,pins = "mpp6";
- marvell,function = "gpio";
- };
-
- pmx_power_switch: pmx-power-switch {
- marvell,pins = "mpp8", "mpp10", "mpp15";
- marvell,function = "gpio";
- };
-
- pmx_power_usb: pmx-power-usb {
- marvell,pins = "mpp9";
- marvell,function = "gpio";
- };
-
- pmx_fan_high: pmx-fan-high {
- marvell,pins = "mpp14";
- marvell,function = "gpio";
- };
-
- pmx_fan_low: pmx-fan-low {
- marvell,pins = "mpp16";
- marvell,function = "gpio";
- };
-
- pmx_led_func: pmx-leds {
- marvell,pins = "mpp17";
- marvell,function = "gpio";
- };
-
- pmx_sw_init: pmx-sw-init {
- marvell,pins = "mpp7";
- marvell,function = "gpio";
- };
-};
-
-&hdd_power {
- gpios = <&gpio0 1 GPIO_ACTIVE_HIGH>;
-};
-
-&usb_power {
- gpios = <&gpio0 9 GPIO_ACTIVE_HIGH>;
-};
-
phy-names = "usb2-phy", "usb3-phy";
phys = <&usb2_picophy0>,
<&phy_port2 PHY_TYPE_USB3>;
+ snps,dis_u3_susphy_quirk;
};
};
CONFIG_ARM_ATAG_DTB_COMPAT=y
CONFIG_CMDLINE="root=/dev/ram0 rw ramdisk=8192 initrd=0x41000000,8M console=ttySAC1,115200 init=/linuxrc mem=256M"
CONFIG_CPU_FREQ=y
-CONFIG_CPU_FREQ_STAT_DETAILS=y
+CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_EVENTS=y
-CONFIG_NF_CT_PROTO_SCTP=m
-CONFIG_NF_CT_PROTO_UDPLITE=m
+CONFIG_NF_CT_PROTO_SCTP=y
+CONFIG_NF_CT_PROTO_UDPLITE=y
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_EVENTS=y
-CONFIG_NF_CT_PROTO_SCTP=m
-CONFIG_NF_CT_PROTO_UDPLITE=m
+CONFIG_NF_CT_PROTO_SCTP=y
+CONFIG_NF_CT_PROTO_UDPLITE=y
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_ARM_APPENDED_DTB=y
CONFIG_ARM_ATAG_DTB_COMPAT=y
CONFIG_CPU_FREQ=y
-CONFIG_CPU_FREQ_STAT_DETAILS=y
+CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_IDLE=y
CONFIG_ARM_KIRKWOOD_CPUIDLE=y
CONFIG_KEXEC=y
CONFIG_EFI=y
CONFIG_CPU_FREQ=y
-CONFIG_CPU_FREQ_STAT_DETAILS=y
+CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=m
CONFIG_ARM_APPENDED_DTB=y
CONFIG_ARM_ATAG_DTB_COMPAT=y
CONFIG_CPU_FREQ=y
-CONFIG_CPU_FREQ_STAT_DETAILS=y
+CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_IDLE=y
CONFIG_ARM_KIRKWOOD_CPUIDLE=y
CONFIG_CMDLINE="root=/dev/ram0 ro"
CONFIG_KEXEC=y
CONFIG_CPU_FREQ=y
-CONFIG_CPU_FREQ_STAT_DETAILS=y
+CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=m
CONFIG_ARM_APPENDED_DTB=y
CONFIG_KEXEC=y
CONFIG_CPU_FREQ=y
-CONFIG_CPU_FREQ_STAT_DETAILS=y
+CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
const void *kbuf, const void __user *ubuf)
{
int ret;
- struct pt_regs newregs;
+ struct pt_regs newregs = *task_pt_regs(target);
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&newregs,
#define to_mmdc_pmu(p) container_of(p, struct mmdc_pmu, pmu)
-static enum cpuhp_state cpuhp_mmdc_state;
static int ddr_type;
struct fsl_mmdc_devtype_data {
#ifdef CONFIG_PERF_EVENTS
+static enum cpuhp_state cpuhp_mmdc_state;
static DEFINE_IDA(mmdc_ida);
PMU_EVENT_ATTR_STRING(total-cycles, mmdc_pmu_total_cycles, "event=0x00")
freq = clk_get_rate(clk);
clk_put(clk);
- rcu_read_lock();
opp = dev_pm_opp_find_freq_ceil(dev, &freq);
if (IS_ERR(opp)) {
- rcu_read_unlock();
pr_err("%s: unable to find boot up OPP for vdd_%s\n",
__func__, vdd_name);
goto exit;
}
bootup_volt = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
+
if (!bootup_volt) {
pr_err("%s: unable to find voltage corresponding to the bootup OPP for vdd_%s\n",
__func__, vdd_name);
void __init early_abt_enable(void)
{
- fsr_info[22].fn = early_abort_handler;
+ fsr_info[FSR_FS_AEA].fn = early_abort_handler;
local_abt_enable();
- fsr_info[22].fn = do_bad;
+ fsr_info[FSR_FS_AEA].fn = do_bad;
}
#ifndef CONFIG_ARM_LPAE
#define FSR_FS5_0 (0x3f)
#ifdef CONFIG_ARM_LPAE
+#define FSR_FS_AEA 17
+
static inline int fsr_fs(unsigned int fsr)
{
return fsr & FSR_FS5_0;
}
#else
+#define FSR_FS_AEA 22
+
static inline int fsr_fs(unsigned int fsr)
{
return (fsr & FSR_FS3_0) | (fsr & FSR_FS4) >> 6;
#address-cells = <2>;
#size-cells = <2>;
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ /* 16 MiB reserved for Hardware ROM Firmware */
+ hwrom_reserved: hwrom@0 {
+ reg = <0x0 0x0 0x0 0x1000000>;
+ no-map;
+ };
+
+ /* 2 MiB reserved for ARM Trusted Firmware (BL31) */
+ secmon_reserved: secmon@10000000 {
+ reg = <0x0 0x10000000 0x0 0x200000>;
+ no-map;
+ };
+ };
+
cpus {
#address-cells = <0x2>;
#size-cells = <0x0>;
status = "okay";
pinctrl-0 = <ð_rgmii_pins>;
pinctrl-names = "default";
+ phy-handle = <ð_phy0>;
+
+ mdio {
+ compatible = "snps,dwmac-mdio";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ eth_phy0: ethernet-phy@0 {
+ reg = <0>;
+ eee-broken-1000t;
+ };
+ };
};
&ir {
cbz w6, .Lcbcencloop
ld1 {v0.16b}, [x5] /* get iv */
- enc_prepare w3, x2, x5
+ enc_prepare w3, x2, x6
.Lcbcencloop:
ld1 {v1.16b}, [x1], #16 /* get next pt block */
eor v0.16b, v0.16b, v1.16b /* ..and xor with iv */
- encrypt_block v0, w3, x2, x5, w6
+ encrypt_block v0, w3, x2, x6, w7
st1 {v0.16b}, [x0], #16
subs w4, w4, #1
bne .Lcbcencloop
+ st1 {v0.16b}, [x5] /* return iv */
ret
AES_ENDPROC(aes_cbc_encrypt)
cbz w6, .LcbcdecloopNx
ld1 {v7.16b}, [x5] /* get iv */
- dec_prepare w3, x2, x5
+ dec_prepare w3, x2, x6
.LcbcdecloopNx:
#if INTERLEAVE >= 2
.Lcbcdecloop:
ld1 {v1.16b}, [x1], #16 /* get next ct block */
mov v0.16b, v1.16b /* ...and copy to v0 */
- decrypt_block v0, w3, x2, x5, w6
+ decrypt_block v0, w3, x2, x6, w7
eor v0.16b, v0.16b, v7.16b /* xor with iv => pt */
mov v7.16b, v1.16b /* ct is next iv */
st1 {v0.16b}, [x0], #16
bne .Lcbcdecloop
.Lcbcdecout:
FRAME_POP
+ st1 {v7.16b}, [x5] /* return iv */
ret
AES_ENDPROC(aes_cbc_decrypt)
AES_ENTRY(aes_ctr_encrypt)
FRAME_PUSH
- cbnz w6, .Lctrfirst /* 1st time around? */
- umov x5, v4.d[1] /* keep swabbed ctr in reg */
- rev x5, x5
-#if INTERLEAVE >= 2
- cmn w5, w4 /* 32 bit overflow? */
- bcs .Lctrinc
- add x5, x5, #1 /* increment BE ctr */
- b .LctrincNx
-#else
- b .Lctrinc
-#endif
-.Lctrfirst:
+ cbz w6, .Lctrnotfirst /* 1st time around? */
enc_prepare w3, x2, x6
ld1 {v4.16b}, [x5]
- umov x5, v4.d[1] /* keep swabbed ctr in reg */
- rev x5, x5
+
+.Lctrnotfirst:
+ umov x8, v4.d[1] /* keep swabbed ctr in reg */
+ rev x8, x8
#if INTERLEAVE >= 2
- cmn w5, w4 /* 32 bit overflow? */
+ cmn w8, w4 /* 32 bit overflow? */
bcs .Lctrloop
.LctrloopNx:
subs w4, w4, #INTERLEAVE
#if INTERLEAVE == 2
mov v0.8b, v4.8b
mov v1.8b, v4.8b
- rev x7, x5
- add x5, x5, #1
+ rev x7, x8
+ add x8, x8, #1
ins v0.d[1], x7
- rev x7, x5
- add x5, x5, #1
+ rev x7, x8
+ add x8, x8, #1
ins v1.d[1], x7
ld1 {v2.16b-v3.16b}, [x1], #32 /* get 2 input blocks */
do_encrypt_block2x
st1 {v0.16b-v1.16b}, [x0], #32
#else
ldr q8, =0x30000000200000001 /* addends 1,2,3[,0] */
- dup v7.4s, w5
+ dup v7.4s, w8
mov v0.16b, v4.16b
add v7.4s, v7.4s, v8.4s
mov v1.16b, v4.16b
eor v2.16b, v7.16b, v2.16b
eor v3.16b, v5.16b, v3.16b
st1 {v0.16b-v3.16b}, [x0], #64
- add x5, x5, #INTERLEAVE
+ add x8, x8, #INTERLEAVE
#endif
- cbz w4, .LctroutNx
-.LctrincNx:
- rev x7, x5
+ rev x7, x8
ins v4.d[1], x7
+ cbz w4, .Lctrout
b .LctrloopNx
-.LctroutNx:
- sub x5, x5, #1
- rev x7, x5
- ins v4.d[1], x7
- b .Lctrout
.Lctr1x:
adds w4, w4, #INTERLEAVE
beq .Lctrout
.Lctrloop:
mov v0.16b, v4.16b
encrypt_block v0, w3, x2, x6, w7
+
+ adds x8, x8, #1 /* increment BE ctr */
+ rev x7, x8
+ ins v4.d[1], x7
+ bcs .Lctrcarry /* overflow? */
+
+.Lctrcarrydone:
subs w4, w4, #1
bmi .Lctrhalfblock /* blocks < 0 means 1/2 block */
ld1 {v3.16b}, [x1], #16
eor v3.16b, v0.16b, v3.16b
st1 {v3.16b}, [x0], #16
- beq .Lctrout
-.Lctrinc:
- adds x5, x5, #1 /* increment BE ctr */
- rev x7, x5
- ins v4.d[1], x7
- bcc .Lctrloop /* no overflow? */
- umov x7, v4.d[0] /* load upper word of ctr */
- rev x7, x7 /* ... to handle the carry */
- add x7, x7, #1
- rev x7, x7
- ins v4.d[0], x7
- b .Lctrloop
+ bne .Lctrloop
+
+.Lctrout:
+ st1 {v4.16b}, [x5] /* return next CTR value */
+ FRAME_POP
+ ret
+
.Lctrhalfblock:
ld1 {v3.8b}, [x1]
eor v3.8b, v0.8b, v3.8b
st1 {v3.8b}, [x0]
-.Lctrout:
FRAME_POP
ret
+
+.Lctrcarry:
+ umov x7, v4.d[0] /* load upper word of ctr */
+ rev x7, x7 /* ... to handle the carry */
+ add x7, x7, #1
+ rev x7, x7
+ ins v4.d[0], x7
+ b .Lctrcarrydone
AES_ENDPROC(aes_ctr_encrypt)
.ltorg
* for more details.
*/
+#include <linux/acpi.h>
#include <linux/cpu.h>
#include <linux/cpumask.h>
#include <linux/init.h>
static int __init register_cpufreq_notifier(void)
{
- if (cap_parsing_failed)
+ /*
+ * on ACPI-based systems we need to use the default cpu capacity
+ * until we have the necessary code to parse the cpu capacity, so
+ * skip registering cpufreq notifier.
+ */
+ if (!acpi_disabled || cap_parsing_failed)
return -EINVAL;
if (!alloc_cpumask_var(&cpus_to_visit, GFP_KERNEL)) {
#define atomic64_sub_and_test(i,v) (atomic64_sub_return((i), (v)) == 0)
#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0)
#define atomic64_inc_and_test(v) (atomic64_inc_return((v)) == 0)
-
+#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
#define atomic_cmpxchg(v, old, new) (cmpxchg(&(v)->counter, old, new))
#define atomic_xchg(v, new) (xchg(&(v)->counter, new))
return c;
}
+static inline int atomic64_add_unless(atomic64_t *v, long long i, long long u)
+{
+ long long c, old;
+
+ c = atomic64_read(v);
+ for (;;) {
+ if (unlikely(c == u))
+ break;
+ old = atomic64_cmpxchg(v, c, c + i);
+ if (likely(old == c))
+ break;
+ c = old;
+ }
+ return c != u;
+}
+
+static inline long long atomic64_dec_if_positive(atomic64_t *v)
+{
+ long long c, old, dec;
+
+ c = atomic64_read(v);
+ for (;;) {
+ dec = c - 1;
+ if (unlikely(dec < 0))
+ break;
+ old = atomic64_cmpxchg((v), c, dec);
+ if (likely(old == c))
+ break;
+ c = old;
+ }
+ return dec;
+}
+
#define ATOMIC_OP(op) \
static inline int atomic_fetch_##op(int i, atomic_t *v) \
{ \
select WEAK_ORDERING
select CPU_SUPPORTS_HIGHMEM
select CPU_HAS_PREFETCH
+ select CPU_SUPPORTS_CPUFREQ
+ select MIPS_EXTERNAL_TIMER
help
Support for BMIPS32/3300/4350/4380 and BMIPS5000 processors.
# CONFIG_SWAP is not set
CONFIG_NO_HZ=y
CONFIG_BLK_DEV_INITRD=y
-CONFIG_RD_GZIP=y
CONFIG_EXPERT=y
# CONFIG_VM_EVENT_COUNTERS is not set
# CONFIG_SLUB_DEBUG is not set
# CONFIG_BLK_DEV_BSG is not set
# CONFIG_IOSCHED_DEADLINE is not set
# CONFIG_IOSCHED_CFQ is not set
+CONFIG_CPU_FREQ=y
+CONFIG_CPU_FREQ_STAT=y
+CONFIG_CPU_FREQ_GOV_POWERSAVE=y
+CONFIG_CPU_FREQ_GOV_USERSPACE=y
+CONFIG_CPU_FREQ_GOV_ONDEMAND=y
+CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
+CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y
+CONFIG_BMIPS_CPUFREQ=y
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_PACKET_DIAG=y
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
-# CONFIG_INET_LRO is not set
# CONFIG_INET_DIAG is not set
CONFIG_CFG80211=y
CONFIG_NL80211_TESTMODE=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
-CONFIG_PRINTK_TIME=y
-CONFIG_BRCMSTB_GISB_ARB=y
CONFIG_MTD=y
CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_INTELEXT=y
# CONFIG_INPUT is not set
# CONFIG_SERIO is not set
# CONFIG_VT is not set
-# CONFIG_DEVKMEM is not set
CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_OF_PLATFORM=y
# CONFIG_HW_RANDOM is not set
-CONFIG_POWER_SUPPLY=y
CONFIG_POWER_RESET=y
CONFIG_POWER_RESET_BRCMSTB=y
CONFIG_POWER_RESET_SYSCON=y
+CONFIG_POWER_SUPPLY=y
# CONFIG_HWMON is not set
CONFIG_USB=y
CONFIG_USB_EHCI_HCD=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
+CONFIG_PRINTK_TIME=y
CONFIG_DEBUG_FS=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_CMDLINE_BOOL=y
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_DEBUG=y
CONFIG_CPU_FREQ_STAT=m
-CONFIG_CPU_FREQ_STAT_DETAILS=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=m
struct task_struct;
struct thread_struct;
-#if !defined(CONFIG_LAZY_SAVE_FPU)
+#if defined(CONFIG_FPU) && !defined(CONFIG_LAZY_SAVE_FPU)
struct fpu_state_struct;
extern asmlinkage void fpu_save(struct fpu_state_struct *);
#define switch_fpu(prev, next) \
#endif
#include <linux/compiler.h>
-#include <asm/types.h> /* for BITS_PER_LONG/SHIFT_PER_LONG */
+#include <asm/types.h>
#include <asm/byteorder.h>
#include <asm/barrier.h>
#include <linux/atomic.h>
* to include/asm-i386/bitops.h or kerneldoc
*/
+#if __BITS_PER_LONG == 64
+#define SHIFT_PER_LONG 6
+#else
+#define SHIFT_PER_LONG 5
+#endif
+
#define CHOP_SHIFTCOUNT(x) (((unsigned long) (x)) & (BITS_PER_LONG - 1))
#if defined(__LP64__)
#define __BITS_PER_LONG 64
-#define SHIFT_PER_LONG 6
#else
#define __BITS_PER_LONG 32
-#define SHIFT_PER_LONG 5
#endif
#include <asm-generic/bitsperlong.h>
#ifndef _PARISC_SWAB_H
#define _PARISC_SWAB_H
+#include <asm/bitsperlong.h>
#include <linux/types.h>
#include <linux/compiler.h>
}
#define __arch_swab32 __arch_swab32
-#if BITS_PER_LONG > 32
+#if __BITS_PER_LONG > 32
/*
** From "PA-RISC 2.0 Architecture", HP Professional Books.
** See Appendix I page 8 , "Endian Byte Swapping".
return x;
}
#define __arch_swab64 __arch_swab64
-#endif /* BITS_PER_LONG > 32 */
+#endif /* __BITS_PER_LONG > 32 */
#endif /* _PARISC_SWAB_H */
select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE
select HAVE_ARCH_HARDENED_USERCOPY
select HAVE_KERNEL_GZIP
- select HAVE_CC_STACKPROTECTOR
config GENERIC_CSUM
def_bool CPU_LITTLE_ENDIAN
bool "Build a relocatable kernel"
depends on (PPC64 && !COMPILE_TEST) || (FLATMEM && (44x || FSL_BOOKE))
select NONSTATIC_KERNEL
+ select MODULE_REL_CRCS if MODVERSIONS
help
This builds a kernel image that is capable of running at the
location the kernel is loaded at. For ppc32, there is no any
CONFIG_GEF_SBC610=y
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_STAT=m
-CONFIG_CPU_FREQ_STAT_DETAILS=y
CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
{
int i;
+#ifndef __clang__ /* clang can't cope with this */
BUILD_BUG_ON(!__builtin_constant_p(feature));
+#endif
#ifdef CONFIG_JUMP_LABEL_FEATURE_CHECK_DEBUG
if (!static_key_initialized) {
{
int i;
+#ifndef __clang__ /* clang can't cope with this */
BUILD_BUG_ON(!__builtin_constant_p(feature));
+#endif
#ifdef CONFIG_JUMP_LABEL_FEATURE_CHECK_DEBUG
if (!static_key_initialized) {
}
#endif
-#if defined(CONFIG_MODVERSIONS) && defined(CONFIG_PPC64)
-#define ARCH_RELOCATES_KCRCTAB
-#define reloc_start PHYSICAL_START
-#endif
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_MODULE_H */
#define SRR1_ISI_N_OR_G 0x10000000 /* ISI: Access is no-exec or G */
#define SRR1_ISI_PROT 0x08000000 /* ISI: Other protection fault */
#define SRR1_WAKEMASK 0x00380000 /* reason for wakeup */
-#define SRR1_WAKEMASK_P8 0x003c0000 /* reason for wakeup on POWER8 */
+#define SRR1_WAKEMASK_P8 0x003c0000 /* reason for wakeup on POWER8 and 9 */
#define SRR1_WAKESYSERR 0x00300000 /* System error */
#define SRR1_WAKEEE 0x00200000 /* External interrupt */
+#define SRR1_WAKEHVI 0x00240000 /* Hypervisor Virtualization Interrupt (P9) */
#define SRR1_WAKEMT 0x00280000 /* mtctrl */
#define SRR1_WAKEHMI 0x00280000 /* Hypervisor maintenance */
#define SRR1_WAKEDEC 0x00180000 /* Decrementer interrupt */
+++ /dev/null
-/*
- * GCC stack protector support.
- *
- * Stack protector works by putting predefined pattern at the start of
- * the stack frame and verifying that it hasn't been overwritten when
- * returning from the function. The pattern is called stack canary
- * and gcc expects it to be defined by a global variable called
- * "__stack_chk_guard" on PPC. This unfortunately means that on SMP
- * we cannot have a different canary value per task.
- */
-
-#ifndef _ASM_STACKPROTECTOR_H
-#define _ASM_STACKPROTECTOR_H
-
-#include <linux/random.h>
-#include <linux/version.h>
-#include <asm/reg.h>
-
-extern unsigned long __stack_chk_guard;
-
-/*
- * Initialize the stackprotector canary value.
- *
- * NOTE: this must only be called from functions that never return,
- * and it must always be inlined.
- */
-static __always_inline void boot_init_stack_canary(void)
-{
- unsigned long canary;
-
- /* Try to get a semi random initial value. */
- get_random_bytes(&canary, sizeof(canary));
- canary ^= mftb();
- canary ^= LINUX_VERSION_CODE;
-
- current->stack_canary = canary;
- __stack_chk_guard = current->stack_canary;
-}
-
-#endif /* _ASM_STACKPROTECTOR_H */
#ifdef CONFIG_PPC_POWERNV
extern int icp_opal_init(void);
+extern void icp_opal_flush_interrupt(void);
#else
static inline int icp_opal_init(void) { return -ENODEV; }
#endif
CFLAGS_btext.o += $(DISABLE_LATENT_ENTROPY_PLUGIN)
CFLAGS_prom.o += $(DISABLE_LATENT_ENTROPY_PLUGIN)
-# -fstack-protector triggers protection checks in this code,
-# but it is being used too early to link to meaningful stack_chk logic.
-CFLAGS_prom_init.o += $(call cc-option, -fno-stack-protector)
-
ifdef CONFIG_FUNCTION_TRACER
# Do not trace early boot code
CFLAGS_REMOVE_cputable.o = -mno-sched-epilog $(CC_FLAGS_FTRACE)
DEFINE(TI_livepatch_sp, offsetof(struct thread_info, livepatch_sp));
#endif
-#ifdef CONFIG_CC_STACKPROTECTOR
- DEFINE(TSK_STACK_CANARY, offsetof(struct task_struct, stack_canary));
-#endif
DEFINE(KSP, offsetof(struct thread_struct, ksp));
DEFINE(PT_REGS, offsetof(struct thread_struct, regs));
#ifdef CONFIG_BOOKE
static void *__eeh_clear_pe_frozen_state(void *data, void *flag)
{
struct eeh_pe *pe = (struct eeh_pe *)data;
- bool *clear_sw_state = flag;
+ bool clear_sw_state = *(bool *)flag;
int i, rc = 1;
for (i = 0; rc && i < 3; i++)
mtspr SPRN_SPEFSCR,r0 /* restore SPEFSCR reg */
END_FTR_SECTION_IFSET(CPU_FTR_SPE)
#endif /* CONFIG_SPE */
-#if defined(CONFIG_CC_STACKPROTECTOR) && !defined(CONFIG_SMP)
- lwz r0,TSK_STACK_CANARY(r2)
- lis r4,__stack_chk_guard@ha
- stw r0,__stack_chk_guard@l(r4)
-#endif
+
lwz r0,_CCR(r1)
mtcrf 0xFF,r0
/* r3-r12 are destroyed -- Cort */
for (end = (void *)vers + size; vers < end; vers++)
if (vers->name[0] == '.') {
memmove(vers->name, vers->name+1, strlen(vers->name));
-#ifdef ARCH_RELOCATES_KCRCTAB
- /* The TOC symbol has no CRC computed. To avoid CRC
- * check failing, we must force it to the expected
- * value (see CRC check in module.c).
- */
- if (!strcmp(vers->name, "TOC."))
- vers->crc = -(unsigned long)reloc_start;
-#endif
}
}
#include <linux/kprobes.h>
#include <linux/kdebug.h>
-#ifdef CONFIG_CC_STACKPROTECTOR
-#include <linux/stackprotector.h>
-unsigned long __stack_chk_guard __read_mostly;
-EXPORT_SYMBOL(__stack_chk_guard);
-#endif
-
/* Transactional Memory debug */
#ifdef TM_DEBUG_SW
#define TM_DEBUG(x...) printk(KERN_INFO x)
cpu_pkg = call_prom("instance-to-package", 1, 1, prom_cpu);
+ if (!PHANDLE_VALID(cpu_pkg))
+ return;
+
prom_getprop(cpu_pkg, "reg", &rval, sizeof(rval));
prom.cpu = be32_to_cpu(rval);
if (unlikely(debugger_fault_handler(regs)))
goto bail;
- /* On a kernel SLB miss we can only check for a valid exception entry */
- if (!user_mode(regs) && (address >= TASK_SIZE)) {
+ /*
+ * The kernel should never take an execute fault nor should it
+ * take a page fault to a kernel address.
+ */
+ if (!user_mode(regs) && (is_exec || (address >= TASK_SIZE))) {
rc = SIGSEGV;
goto bail;
}
#endif /* CONFIG_8xx */
if (is_exec) {
- /*
- * An execution fault + no execute ?
- *
- * On CPUs that don't have CPU_FTR_COHERENT_ICACHE we
- * deliberately create NX mappings, and use the fault to do the
- * cache flush. This is usually handled in hash_page_do_lazy_icache()
- * but we could end up here if that races with a concurrent PTE
- * update. In that case we need to fall through here to the VMA
- * check below.
- */
- if (cpu_has_feature(CPU_FTR_COHERENT_ICACHE) &&
- (regs->msr & SRR1_ISI_N_OR_G))
- goto bad_area;
-
/*
* Allow execution from readable areas if the MMU does not
* provide separate controls over reading and executing.
if (!pmdp)
return -ENOMEM;
if (map_page_size == PMD_SIZE) {
- ptep = (pte_t *)pudp;
+ ptep = pmdp_ptep(pmdp);
goto set_the_pte;
}
ptep = pte_alloc_kernel(pmdp, ea);
}
pmdp = pmd_offset(pudp, ea);
if (map_page_size == PMD_SIZE) {
- ptep = (pte_t *)pudp;
+ ptep = pmdp_ptep(pmdp);
goto set_the_pte;
}
if (!pmd_present(*pmdp)) {
for (set = 0; set < POWER9_TLB_SETS_RADIX ; set++) {
__tlbiel_pid(pid, set, ric);
}
- if (cpu_has_feature(CPU_FTR_POWER9_DD1))
- asm volatile(PPC_INVALIDATE_ERAT : : :"memory");
- return;
+ asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory");
}
static inline void _tlbie_pid(unsigned long pid, unsigned long ric)
asm volatile(PPC_TLBIEL(%0, %4, %3, %2, %1)
: : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
asm volatile("ptesync": : :"memory");
- if (cpu_has_feature(CPU_FTR_POWER9_DD1))
- asm volatile(PPC_INVALIDATE_ERAT : : :"memory");
}
static inline void _tlbie_va(unsigned long va, unsigned long pid,
wmask = SRR1_WAKEMASK_P8;
idle_states = pnv_get_supported_cpuidle_states();
+
/* We don't want to take decrementer interrupts while we are offline,
- * so clear LPCR:PECE1. We keep PECE2 enabled.
+ * so clear LPCR:PECE1. We keep PECE2 (and LPCR_PECE_HVEE on P9)
+ * enabled as to let IPIs in.
*/
mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1);
* contains 0.
*/
if (((srr1 & wmask) == SRR1_WAKEEE) ||
+ ((srr1 & wmask) == SRR1_WAKEHVI) ||
(local_paca->irq_happened & PACA_IRQ_EE)) {
- icp_native_flush_interrupt();
+ if (cpu_has_feature(CPU_FTR_ARCH_300))
+ icp_opal_flush_interrupt();
+ else
+ icp_native_flush_interrupt();
} else if ((srr1 & wmask) == SRR1_WAKEHDBELL) {
unsigned long msg = PPC_DBELL_TYPE(PPC_DBELL_SERVER);
asm volatile(PPC_MSGCLR(%0) : : "r" (msg));
if (srr1 && !generic_check_cpu_restart(cpu))
DBG("CPU%d Unexpected exit while offline !\n", cpu);
}
+
+ /* Re-enable decrementer interrupts */
mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_PECE1);
DBG("CPU%d coming online...\n", cpu);
}
{
int hw_cpu = get_hard_smp_processor_id(cpu);
+ kvmppc_set_host_ipi(cpu, 1);
opal_int_set_mfrr(hw_cpu, IPI_PRIORITY);
}
static irqreturn_t icp_opal_ipi_action(int irq, void *dev_id)
{
- int hw_cpu = hard_smp_processor_id();
+ int cpu = smp_processor_id();
- opal_int_set_mfrr(hw_cpu, 0xff);
+ kvmppc_set_host_ipi(cpu, 0);
+ opal_int_set_mfrr(get_hard_smp_processor_id(cpu), 0xff);
return smp_ipi_demux();
}
+/*
+ * Called when an interrupt is received on an off-line CPU to
+ * clear the interrupt, so that the CPU can go back to nap mode.
+ */
+void icp_opal_flush_interrupt(void)
+{
+ unsigned int xirr;
+ unsigned int vec;
+
+ do {
+ xirr = icp_opal_get_xirr();
+ vec = xirr & 0x00ffffff;
+ if (vec == XICS_IRQ_SPURIOUS)
+ break;
+ if (vec == XICS_IPI) {
+ /* Clear pending IPI */
+ int cpu = smp_processor_id();
+ kvmppc_set_host_ipi(cpu, 0);
+ opal_int_set_mfrr(get_hard_smp_processor_id(cpu), 0xff);
+ } else {
+ pr_err("XICS: hw interrupt 0x%x to offline cpu, "
+ "disabling\n", vec);
+ xics_mask_unknown_vec(vec);
+ }
+
+ /* EOI the interrupt */
+ } while (opal_int_eoi(xirr) > 0);
+}
+
#endif /* CONFIG_SMP */
static const struct icp_ops icp_opal_ops = {
if (target == current)
save_fpu_regs();
+ if (MACHINE_HAS_VX)
+ convert_vx_to_fp(fprs, target->thread.fpu.vxrs);
+ else
+ memcpy(&fprs, target->thread.fpu.fprs, sizeof(fprs));
+
/* If setting FPC, must validate it first. */
if (count > 0 && pos < offsetof(s390_fp_regs, fprs)) {
u32 ufpc[2] = { target->thread.fpu.fpc, 0 };
if (target == current)
save_fpu_regs();
+ for (i = 0; i < __NUM_VXRS_LOW; i++)
+ vxrs[i] = *((__u64 *)(target->thread.fpu.vxrs + i) + 1);
+
rc = user_regset_copyin(&pos, &count, &kbuf, &ubuf, vxrs, 0, -1);
if (rc == 0)
for (i = 0; i < __NUM_VXRS_LOW; i++)
return pgste;
}
-static inline void ptep_xchg_commit(struct mm_struct *mm,
+static inline pte_t ptep_xchg_commit(struct mm_struct *mm,
unsigned long addr, pte_t *ptep,
pgste_t pgste, pte_t old, pte_t new)
{
} else {
*ptep = new;
}
+ return old;
}
pte_t ptep_xchg_direct(struct mm_struct *mm, unsigned long addr,
preempt_disable();
pgste = ptep_xchg_start(mm, addr, ptep);
old = ptep_flush_direct(mm, addr, ptep);
- ptep_xchg_commit(mm, addr, ptep, pgste, old, new);
+ old = ptep_xchg_commit(mm, addr, ptep, pgste, old, new);
preempt_enable();
return old;
}
preempt_disable();
pgste = ptep_xchg_start(mm, addr, ptep);
old = ptep_flush_lazy(mm, addr, ptep);
- ptep_xchg_commit(mm, addr, ptep, pgste, old, new);
+ old = ptep_xchg_commit(mm, addr, ptep, pgste, old, new);
preempt_enable();
return old;
}
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_CPU_FREQ=y
-CONFIG_CPU_FREQ_STAT_DETAILS=y
+CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_SH_CPU_FREQ=y
CONFIG_HEARTBEAT=y
static inline void tsb_context_switch(struct mm_struct *mm)
{
__tsb_context_switch(__pa(mm->pgd),
- &mm->context.tsb_block[0],
+ &mm->context.tsb_block[MM_TSB_BASE],
#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
- (mm->context.tsb_block[1].tsb ?
- &mm->context.tsb_block[1] :
+ (mm->context.tsb_block[MM_TSB_HUGE].tsb ?
+ &mm->context.tsb_block[MM_TSB_HUGE] :
NULL)
#else
NULL
#endif
- , __pa(&mm->context.tsb_descr[0]));
+ , __pa(&mm->context.tsb_descr[MM_TSB_BASE]));
}
void tsb_grow(struct mm_struct *mm,
unsigned long order = get_order(size);
unsigned long p;
- p = __get_free_pages(GFP_KERNEL, order);
+ p = __get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
if (!p) {
prom_printf("SUN4V: Error, cannot allocate queue.\n");
prom_halt();
"Linux powering off";
static const char rebooting_msg[32] __attribute__((aligned(32))) =
"Linux rebooting";
-static const char panicing_msg[32] __attribute__((aligned(32))) =
- "Linux panicing";
+static const char panicking_msg[32] __attribute__((aligned(32))) =
+ "Linux panicking";
static int sstate_reboot_call(struct notifier_block *np, unsigned long type, void *_unused)
{
static int sstate_panic_event(struct notifier_block *n, unsigned long event, void *ptr)
{
- do_set_sstate(HV_SOFT_STATE_TRANSITION, panicing_msg);
+ do_set_sstate(HV_SOFT_STATE_TRANSITION, panicking_msg);
return NOTIFY_DONE;
}
atomic_inc(&sun4v_resum_oflow_cnt);
}
+/* Given a set of registers, get the virtual addressi that was being accessed
+ * by the faulting instructions at tpc.
+ */
+static unsigned long sun4v_get_vaddr(struct pt_regs *regs)
+{
+ unsigned int insn;
+
+ if (!copy_from_user(&insn, (void __user *)regs->tpc, 4)) {
+ return compute_effective_address(regs, insn,
+ (insn >> 25) & 0x1f);
+ }
+ return 0;
+}
+
+/* Attempt to handle non-resumable errors generated from userspace.
+ * Returns true if the signal was handled, false otherwise.
+ */
+bool sun4v_nonresum_error_user_handled(struct pt_regs *regs,
+ struct sun4v_error_entry *ent) {
+
+ unsigned int attrs = ent->err_attrs;
+
+ if (attrs & SUN4V_ERR_ATTRS_MEMORY) {
+ unsigned long addr = ent->err_raddr;
+ siginfo_t info;
+
+ if (addr == ~(u64)0) {
+ /* This seems highly unlikely to ever occur */
+ pr_emerg("SUN4V NON-RECOVERABLE ERROR: Memory error detected in unknown location!\n");
+ } else {
+ unsigned long page_cnt = DIV_ROUND_UP(ent->err_size,
+ PAGE_SIZE);
+
+ /* Break the unfortunate news. */
+ pr_emerg("SUN4V NON-RECOVERABLE ERROR: Memory failed at %016lX\n",
+ addr);
+ pr_emerg("SUN4V NON-RECOVERABLE ERROR: Claiming %lu ages.\n",
+ page_cnt);
+
+ while (page_cnt-- > 0) {
+ if (pfn_valid(addr >> PAGE_SHIFT))
+ get_page(pfn_to_page(addr >> PAGE_SHIFT));
+ addr += PAGE_SIZE;
+ }
+ }
+ info.si_signo = SIGKILL;
+ info.si_errno = 0;
+ info.si_trapno = 0;
+ force_sig_info(info.si_signo, &info, current);
+
+ return true;
+ }
+ if (attrs & SUN4V_ERR_ATTRS_PIO) {
+ siginfo_t info;
+
+ info.si_signo = SIGBUS;
+ info.si_code = BUS_ADRERR;
+ info.si_addr = (void __user *)sun4v_get_vaddr(regs);
+ force_sig_info(info.si_signo, &info, current);
+
+ return true;
+ }
+
+ /* Default to doing nothing */
+ return false;
+}
+
/* We run with %pil set to PIL_NORMAL_MAX and PSTATE_IE enabled in %pstate.
* Log the event, clear the first word of the entry, and die.
*/
put_cpu();
+ if (!(regs->tstate & TSTATE_PRIV) &&
+ sun4v_nonresum_error_user_handled(regs, &local_copy)) {
+ /* DON'T PANIC: This userspace error was handled. */
+ return;
+ }
+
#ifdef CONFIG_PCI
/* Check for the special PCI poke sequence. */
if (pci_poke_in_progress && pci_poke_cpu == cpu) {
const void *kbuf, const void __user *ubuf)
{
int ret;
- struct pt_regs regs;
+ struct pt_regs regs = *task_pt_regs(target);
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, ®s, 0,
sizeof(regs));
aesni_simd_skciphers[i]; i++)
simd_skcipher_free(aesni_simd_skciphers[i]);
- for (i = 0; i < ARRAY_SIZE(aesni_simd_skciphers2) &&
- aesni_simd_skciphers2[i].simd; i++)
- simd_skcipher_free(aesni_simd_skciphers2[i].simd);
+ for (i = 0; i < ARRAY_SIZE(aesni_simd_skciphers2); i++)
+ if (aesni_simd_skciphers2[i].simd)
+ simd_skcipher_free(aesni_simd_skciphers2[i].simd);
}
static int __init aesni_init(void)
simd = simd_skcipher_create_compat(algname, drvname, basename);
err = PTR_ERR(simd);
if (IS_ERR(simd))
- goto unregister_simds;
+ continue;
aesni_simd_skciphers2[i].simd = simd;
}
static inline struct rapl_pmu *cpu_to_rapl_pmu(unsigned int cpu)
{
- return rapl_pmus->pmus[topology_logical_package_id(cpu)];
+ unsigned int pkgid = topology_logical_package_id(cpu);
+
+ /*
+ * The unsigned check also catches the '-1' return value for non
+ * existent mappings in the topology map.
+ */
+ return pkgid < rapl_pmus->maxpkg ? rapl_pmus->pmus[pkgid] : NULL;
}
static inline u64 rapl_read_counter(struct perf_event *event)
/* must be done before validate_group */
pmu = cpu_to_rapl_pmu(event->cpu);
+ if (!pmu)
+ return -EINVAL;
event->cpu = pmu->cpu;
event->pmu_private = pmu;
event->hw.event_base = msr;
struct rapl_pmu *pmu = cpu_to_rapl_pmu(cpu);
int target;
+ if (!pmu) {
+ pmu = kzalloc_node(sizeof(*pmu), GFP_KERNEL, cpu_to_node(cpu));
+ if (!pmu)
+ return -ENOMEM;
+
+ raw_spin_lock_init(&pmu->lock);
+ INIT_LIST_HEAD(&pmu->active_list);
+ pmu->pmu = &rapl_pmus->pmu;
+ pmu->timer_interval = ms_to_ktime(rapl_timer_ms);
+ rapl_hrtimer_init(pmu);
+
+ rapl_pmus->pmus[topology_logical_package_id(cpu)] = pmu;
+ }
+
/*
* Check if there is an online cpu in the package which collects rapl
* events already.
return 0;
}
-static int rapl_cpu_prepare(unsigned int cpu)
-{
- struct rapl_pmu *pmu = cpu_to_rapl_pmu(cpu);
-
- if (pmu)
- return 0;
-
- pmu = kzalloc_node(sizeof(*pmu), GFP_KERNEL, cpu_to_node(cpu));
- if (!pmu)
- return -ENOMEM;
-
- raw_spin_lock_init(&pmu->lock);
- INIT_LIST_HEAD(&pmu->active_list);
- pmu->pmu = &rapl_pmus->pmu;
- pmu->timer_interval = ms_to_ktime(rapl_timer_ms);
- pmu->cpu = -1;
- rapl_hrtimer_init(pmu);
- rapl_pmus->pmus[topology_logical_package_id(cpu)] = pmu;
- return 0;
-}
-
static int rapl_check_hw_unit(bool apply_quirk)
{
u64 msr_rapl_power_unit_bits;
/*
* Install callbacks. Core will call them for each online cpu.
*/
-
- ret = cpuhp_setup_state(CPUHP_PERF_X86_RAPL_PREP, "perf/x86/rapl:prepare",
- rapl_cpu_prepare, NULL);
- if (ret)
- goto out;
-
ret = cpuhp_setup_state(CPUHP_AP_PERF_X86_RAPL_ONLINE,
"perf/x86/rapl:online",
rapl_cpu_online, rapl_cpu_offline);
if (ret)
- goto out1;
+ goto out;
ret = perf_pmu_register(&rapl_pmus->pmu, "power", -1);
if (ret)
- goto out2;
+ goto out1;
rapl_advertise();
return 0;
-out2:
- cpuhp_remove_state(CPUHP_AP_PERF_X86_RAPL_ONLINE);
out1:
- cpuhp_remove_state(CPUHP_PERF_X86_RAPL_PREP);
+ cpuhp_remove_state(CPUHP_AP_PERF_X86_RAPL_ONLINE);
out:
pr_warn("Initialization failed (%d), disabled\n", ret);
cleanup_rapl_pmus();
static void __exit intel_rapl_exit(void)
{
cpuhp_remove_state_nocalls(CPUHP_AP_PERF_X86_RAPL_ONLINE);
- cpuhp_remove_state_nocalls(CPUHP_PERF_X86_RAPL_PREP);
perf_pmu_unregister(&rapl_pmus->pmu);
cleanup_rapl_pmus();
}
struct intel_uncore_box *uncore_pmu_to_box(struct intel_uncore_pmu *pmu, int cpu)
{
- return pmu->boxes[topology_logical_package_id(cpu)];
+ unsigned int pkgid = topology_logical_package_id(cpu);
+
+ /*
+ * The unsigned check also catches the '-1' return value for non
+ * existent mappings in the topology map.
+ */
+ return pkgid < max_packages ? pmu->boxes[pkgid] : NULL;
}
u64 uncore_msr_read_counter(struct intel_uncore_box *box, struct perf_event *event)
pmu->registered = false;
}
-static void __uncore_exit_boxes(struct intel_uncore_type *type, int cpu)
-{
- struct intel_uncore_pmu *pmu = type->pmus;
- struct intel_uncore_box *box;
- int i, pkg;
-
- if (pmu) {
- pkg = topology_physical_package_id(cpu);
- for (i = 0; i < type->num_boxes; i++, pmu++) {
- box = pmu->boxes[pkg];
- if (box)
- uncore_box_exit(box);
- }
- }
-}
-
-static void uncore_exit_boxes(void *dummy)
-{
- struct intel_uncore_type **types;
-
- for (types = uncore_msr_uncores; *types; types++)
- __uncore_exit_boxes(*types++, smp_processor_id());
-}
-
static void uncore_free_boxes(struct intel_uncore_pmu *pmu)
{
int pkg;
}
}
-static int uncore_cpu_dying(unsigned int cpu)
-{
- struct intel_uncore_type *type, **types = uncore_msr_uncores;
- struct intel_uncore_pmu *pmu;
- struct intel_uncore_box *box;
- int i, pkg;
-
- pkg = topology_logical_package_id(cpu);
- for (; *types; types++) {
- type = *types;
- pmu = type->pmus;
- for (i = 0; i < type->num_boxes; i++, pmu++) {
- box = pmu->boxes[pkg];
- if (box && atomic_dec_return(&box->refcnt) == 0)
- uncore_box_exit(box);
- }
- }
- return 0;
-}
-
-static int first_init;
-
-static int uncore_cpu_starting(unsigned int cpu)
-{
- struct intel_uncore_type *type, **types = uncore_msr_uncores;
- struct intel_uncore_pmu *pmu;
- struct intel_uncore_box *box;
- int i, pkg, ncpus = 1;
-
- if (first_init) {
- /*
- * On init we get the number of online cpus in the package
- * and set refcount for all of them.
- */
- ncpus = cpumask_weight(topology_core_cpumask(cpu));
- }
-
- pkg = topology_logical_package_id(cpu);
- for (; *types; types++) {
- type = *types;
- pmu = type->pmus;
- for (i = 0; i < type->num_boxes; i++, pmu++) {
- box = pmu->boxes[pkg];
- if (!box)
- continue;
- /* The first cpu on a package activates the box */
- if (atomic_add_return(ncpus, &box->refcnt) == ncpus)
- uncore_box_init(box);
- }
- }
-
- return 0;
-}
-
-static int uncore_cpu_prepare(unsigned int cpu)
-{
- struct intel_uncore_type *type, **types = uncore_msr_uncores;
- struct intel_uncore_pmu *pmu;
- struct intel_uncore_box *box;
- int i, pkg;
-
- pkg = topology_logical_package_id(cpu);
- for (; *types; types++) {
- type = *types;
- pmu = type->pmus;
- for (i = 0; i < type->num_boxes; i++, pmu++) {
- if (pmu->boxes[pkg])
- continue;
- /* First cpu of a package allocates the box */
- box = uncore_alloc_box(type, cpu_to_node(cpu));
- if (!box)
- return -ENOMEM;
- box->pmu = pmu;
- box->pkgid = pkg;
- pmu->boxes[pkg] = box;
- }
- }
- return 0;
-}
-
static void uncore_change_type_ctx(struct intel_uncore_type *type, int old_cpu,
int new_cpu)
{
static int uncore_event_cpu_offline(unsigned int cpu)
{
- int target;
+ struct intel_uncore_type *type, **types = uncore_msr_uncores;
+ struct intel_uncore_pmu *pmu;
+ struct intel_uncore_box *box;
+ int i, pkg, target;
/* Check if exiting cpu is used for collecting uncore events */
if (!cpumask_test_and_clear_cpu(cpu, &uncore_cpu_mask))
- return 0;
-
+ goto unref;
/* Find a new cpu to collect uncore events */
target = cpumask_any_but(topology_core_cpumask(cpu), cpu);
uncore_change_context(uncore_msr_uncores, cpu, target);
uncore_change_context(uncore_pci_uncores, cpu, target);
+
+unref:
+ /* Clear the references */
+ pkg = topology_logical_package_id(cpu);
+ for (; *types; types++) {
+ type = *types;
+ pmu = type->pmus;
+ for (i = 0; i < type->num_boxes; i++, pmu++) {
+ box = pmu->boxes[pkg];
+ if (box && atomic_dec_return(&box->refcnt) == 0)
+ uncore_box_exit(box);
+ }
+ }
return 0;
}
+static int allocate_boxes(struct intel_uncore_type **types,
+ unsigned int pkg, unsigned int cpu)
+{
+ struct intel_uncore_box *box, *tmp;
+ struct intel_uncore_type *type;
+ struct intel_uncore_pmu *pmu;
+ LIST_HEAD(allocated);
+ int i;
+
+ /* Try to allocate all required boxes */
+ for (; *types; types++) {
+ type = *types;
+ pmu = type->pmus;
+ for (i = 0; i < type->num_boxes; i++, pmu++) {
+ if (pmu->boxes[pkg])
+ continue;
+ box = uncore_alloc_box(type, cpu_to_node(cpu));
+ if (!box)
+ goto cleanup;
+ box->pmu = pmu;
+ box->pkgid = pkg;
+ list_add(&box->active_list, &allocated);
+ }
+ }
+ /* Install them in the pmus */
+ list_for_each_entry_safe(box, tmp, &allocated, active_list) {
+ list_del_init(&box->active_list);
+ box->pmu->boxes[pkg] = box;
+ }
+ return 0;
+
+cleanup:
+ list_for_each_entry_safe(box, tmp, &allocated, active_list) {
+ list_del_init(&box->active_list);
+ kfree(box);
+ }
+ return -ENOMEM;
+}
+
static int uncore_event_cpu_online(unsigned int cpu)
{
- int target;
+ struct intel_uncore_type *type, **types = uncore_msr_uncores;
+ struct intel_uncore_pmu *pmu;
+ struct intel_uncore_box *box;
+ int i, ret, pkg, target;
+
+ pkg = topology_logical_package_id(cpu);
+ ret = allocate_boxes(types, pkg, cpu);
+ if (ret)
+ return ret;
+
+ for (; *types; types++) {
+ type = *types;
+ pmu = type->pmus;
+ for (i = 0; i < type->num_boxes; i++, pmu++) {
+ box = pmu->boxes[pkg];
+ if (!box && atomic_inc_return(&box->refcnt) == 1)
+ uncore_box_init(box);
+ }
+ }
/*
* Check if there is an online cpu in the package
if (cret && pret)
return -ENODEV;
- /*
- * Install callbacks. Core will call them for each online cpu.
- *
- * The first online cpu of each package allocates and takes
- * the refcounts for all other online cpus in that package.
- * If msrs are not enabled no allocation is required and
- * uncore_cpu_prepare() is not called for each online cpu.
- */
- if (!cret) {
- ret = cpuhp_setup_state(CPUHP_PERF_X86_UNCORE_PREP,
- "perf/x86/intel/uncore:prepare",
- uncore_cpu_prepare, NULL);
- if (ret)
- goto err;
- } else {
- cpuhp_setup_state_nocalls(CPUHP_PERF_X86_UNCORE_PREP,
- "perf/x86/intel/uncore:prepare",
- uncore_cpu_prepare, NULL);
- }
- first_init = 1;
- cpuhp_setup_state(CPUHP_AP_PERF_X86_UNCORE_STARTING,
- "perf/x86/uncore:starting",
- uncore_cpu_starting, uncore_cpu_dying);
- first_init = 0;
- cpuhp_setup_state(CPUHP_AP_PERF_X86_UNCORE_ONLINE,
- "perf/x86/uncore:online",
- uncore_event_cpu_online, uncore_event_cpu_offline);
+ /* Install hotplug callbacks to setup the targets for each package */
+ ret = cpuhp_setup_state(CPUHP_AP_PERF_X86_UNCORE_ONLINE,
+ "perf/x86/intel/uncore:online",
+ uncore_event_cpu_online,
+ uncore_event_cpu_offline);
+ if (ret)
+ goto err;
return 0;
err:
- /* Undo box->init_box() */
- on_each_cpu_mask(&uncore_cpu_mask, uncore_exit_boxes, NULL, 1);
uncore_types_exit(uncore_msr_uncores);
uncore_pci_exit();
return ret;
static void __exit intel_uncore_exit(void)
{
- cpuhp_remove_state_nocalls(CPUHP_AP_PERF_X86_UNCORE_ONLINE);
- cpuhp_remove_state_nocalls(CPUHP_AP_PERF_X86_UNCORE_STARTING);
- cpuhp_remove_state_nocalls(CPUHP_PERF_X86_UNCORE_PREP);
+ cpuhp_remove_state(CPUHP_AP_PERF_X86_UNCORE_ONLINE);
uncore_types_exit(uncore_msr_uncores);
uncore_pci_exit();
}
extern void load_ucode_ap(void);
void reload_early_microcode(void);
extern bool get_builtin_firmware(struct cpio_data *cd, const char *name);
+extern bool initrd_gone;
#else
static inline int __init microcode_init(void) { return 0; };
static inline void __init load_ucode_bsp(void) { }
__u8 x86_phys_bits;
/* CPUID returned core id bits: */
__u8 x86_coreid_bits;
+ __u8 cu_id;
/* Max extended CPUID function supported: */
__u32 extended_cpuid_level;
/* Maximum supported CPUID level, -1=no CPUID: */
#include <linux/sched.h>
#include <acpi/processor.h>
-#include <asm/acpi.h>
#include <asm/mwait.h>
#include <asm/special_insns.h>
retval = 0;
/* If the HW does not support any sub-states in this C-state */
if (num_cstate_subtype == 0) {
- pr_warn(FW_BUG "ACPI MWAIT C-state 0x%x not supported by HW (0x%x)\n", cx->address, edx_part);
+ pr_warn(FW_BUG "ACPI MWAIT C-state 0x%x not supported by HW (0x%x)\n",
+ cx->address, edx_part);
retval = -1;
goto out;
}
if (!mwait_supported[cstate_type]) {
mwait_supported[cstate_type] = 1;
printk(KERN_DEBUG
- "Monitor-Mwait will be used to enter C-%d "
- "state\n", cx->type);
+ "Monitor-Mwait will be used to enter C-%d state\n",
+ cx->type);
}
snprintf(cx->desc,
ACPI_CX_DESC_LEN, "ACPI FFH INTEL MWAIT 0x%x",
static int __init ffh_cstate_init(void)
{
struct cpuinfo_x86 *c = &boot_cpu_data;
+
if (c->x86_vendor != X86_VENDOR_INTEL)
return -1;
.irq_ack = irq_chip_ack_parent,
.irq_eoi = ioapic_ack_level,
.irq_set_affinity = ioapic_set_affinity,
- .irq_retrigger = irq_chip_retrigger_hierarchy,
.flags = IRQCHIP_SKIP_SET_WAKE,
};
.irq_ack = irq_chip_ack_parent,
.irq_eoi = ioapic_ir_ack_level,
.irq_set_affinity = ioapic_set_affinity,
- .irq_retrigger = irq_chip_retrigger_hierarchy,
.flags = IRQCHIP_SKIP_SET_WAKE,
};
if (idx != -1 && irq_trigger(idx))
unmask_ioapic_irq(irq_get_chip_data(0));
}
+ irq_domain_deactivate_irq(irq_data);
irq_domain_activate_irq(irq_data);
if (timer_irq_works()) {
if (disable_timer_pin_1 > 0)
* legacy devices should be connected to IO APIC #0
*/
replace_pin_at_irq_node(data, node, apic1, pin1, apic2, pin2);
+ irq_domain_deactivate_irq(irq_data);
irq_domain_activate_irq(irq_data);
legacy_pic->unmask(0);
if (timer_irq_works()) {
/* get information required for multi-node processors */
if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {
+ u32 eax, ebx, ecx, edx;
- node_id = cpuid_ecx(0x8000001e) & 7;
+ cpuid(0x8000001e, &eax, &ebx, &ecx, &edx);
+
+ node_id = ecx & 0xff;
+ smp_num_siblings = ((ebx >> 8) & 0xff) + 1;
+
+ if (c->x86 == 0x15)
+ c->cu_id = ebx & 0xff;
+
+ if (c->x86 >= 0x17) {
+ c->cpu_core_id = ebx & 0xff;
+
+ if (smp_num_siblings > 1)
+ c->x86_max_cores /= smp_num_siblings;
+ }
/*
* We may have multiple LLCs if L3 caches exist, so check if we
c->x86_model_id[0] = '\0'; /* Unset */
c->x86_max_cores = 1;
c->x86_coreid_bits = 0;
+ c->cu_id = 0xff;
#ifdef CONFIG_X86_64
c->x86_clflush_size = 64;
c->x86_phys_bits = 36;
static unsigned long (*mce_adjust_timer)(unsigned long interval) = mce_adjust_timer_default;
-static void __restart_timer(struct timer_list *t, unsigned long interval)
+static void __start_timer(struct timer_list *t, unsigned long interval)
{
unsigned long when = jiffies + interval;
unsigned long flags;
local_irq_save(flags);
- if (timer_pending(t)) {
- if (time_before(when, t->expires))
- mod_timer(t, when);
- } else {
- t->expires = round_jiffies(when);
- add_timer_on(t, smp_processor_id());
- }
+ if (!timer_pending(t) || time_before(when, t->expires))
+ mod_timer(t, round_jiffies(when));
local_irq_restore(flags);
}
done:
__this_cpu_write(mce_next_interval, iv);
- __restart_timer(t, iv);
+ __start_timer(t, iv);
}
/*
struct timer_list *t = this_cpu_ptr(&mce_timer);
unsigned long iv = __this_cpu_read(mce_next_interval);
- __restart_timer(t, interval);
+ __start_timer(t, interval);
if (interval < iv)
__this_cpu_write(mce_next_interval, interval);
}
}
-static void mce_start_timer(unsigned int cpu, struct timer_list *t)
+static void mce_start_timer(struct timer_list *t)
{
unsigned long iv = check_interval * HZ;
if (mca_cfg.ignore_ce || !iv)
return;
- per_cpu(mce_next_interval, cpu) = iv;
-
- t->expires = round_jiffies(jiffies + iv);
- add_timer_on(t, cpu);
+ this_cpu_write(mce_next_interval, iv);
+ __start_timer(t, iv);
}
static void __mcheck_cpu_setup_timer(void)
unsigned int cpu = smp_processor_id();
setup_pinned_timer(t, mce_timer_fn, cpu);
- mce_start_timer(cpu, t);
+ mce_start_timer(t);
}
/* Handle unconfigured int18 (should never happen) */
static int mce_cpu_online(unsigned int cpu)
{
- struct timer_list *t = &per_cpu(mce_timer, cpu);
+ struct timer_list *t = this_cpu_ptr(&mce_timer);
int ret;
mce_device_create(cpu);
return ret;
}
mce_reenable_cpu();
- mce_start_timer(cpu, t);
+ mce_start_timer(t);
return 0;
}
static int mce_cpu_pre_down(unsigned int cpu)
{
- struct timer_list *t = &per_cpu(mce_timer, cpu);
+ struct timer_list *t = this_cpu_ptr(&mce_timer);
mce_disable_cpu();
del_timer_sync(t);
reget:
if (!get_builtin_microcode(&cp, family)) {
#ifdef CONFIG_BLK_DEV_INITRD
- cp = find_cpio_data(ucode_path, (void *)initrd_start,
- initrd_end - initrd_start, NULL);
+ if (!initrd_gone)
+ cp = find_cpio_data(ucode_path, (void *)initrd_start,
+ initrd_end - initrd_start, NULL);
#endif
if (!(cp.data && cp.size)) {
/*
static struct microcode_ops *microcode_ops;
static bool dis_ucode_ldr = true;
+bool initrd_gone;
+
LIST_HEAD(microcode_cache);
/*
static int __init save_microcode_in_initrd(void)
{
struct cpuinfo_x86 *c = &boot_cpu_data;
+ int ret = -EINVAL;
switch (c->x86_vendor) {
case X86_VENDOR_INTEL:
if (c->x86 >= 6)
- return save_microcode_in_initrd_intel();
+ ret = save_microcode_in_initrd_intel();
break;
case X86_VENDOR_AMD:
if (c->x86 >= 0x10)
- return save_microcode_in_initrd_amd(c->x86);
+ ret = save_microcode_in_initrd_amd(c->x86);
break;
default:
break;
}
- return -EINVAL;
+ initrd_gone = true;
+
+ return ret;
}
struct cpio_data find_microcode_in_initrd(const char *path, bool use_pa)
* has the virtual address of the beginning of the initrd. It also
* possibly relocates the ramdisk. In either case, initrd_start contains
* the updated address so use that instead.
+ *
+ * initrd_gone is for the hotplug case where we've thrown out initrd
+ * already.
*/
- if (!use_pa && initrd_start)
- start = initrd_start;
+ if (!use_pa) {
+ if (initrd_gone)
+ return (struct cpio_data){ NULL, 0, "" };
+ if (initrd_start)
+ start = initrd_start;
+ }
return find_cpio_data(path, (void *)start, size, NULL);
#else /* !CONFIG_BLK_DEV_INITRD */
static const char ucode_path[] = "kernel/x86/microcode/GenuineIntel.bin";
-/* Current microcode patch used in early patching */
+/* Current microcode patch used in early patching on the APs. */
struct microcode_intel *intel_ucode_patch;
static inline bool cpu_signatures_match(unsigned int s1, unsigned int p1,
struct ucode_cpu_info uci;
struct cpio_data cp;
- /*
- * AP loading didn't find any microcode patch, no need to save anything.
- */
- if (!intel_ucode_patch || IS_ERR(intel_ucode_patch))
- return 0;
-
if (!load_builtin_intel_microcode(&cp))
cp = find_microcode_in_initrd(ucode_path, false);
return 0;
}
-
/*
* @res_patch, output: a pointer to the patch we found.
*/
#include <asm/fpu/regset.h>
#include <asm/fpu/signal.h>
#include <asm/fpu/types.h>
+#include <asm/fpu/xstate.h>
#include <asm/traps.h>
#include <linux/hardirq.h>
* it will #GP. Make sure it is replaced after the memset().
*/
if (static_cpu_has(X86_FEATURE_XSAVES))
- state->xsave.header.xcomp_bv = XCOMP_BV_COMPACTED_FORMAT;
+ state->xsave.header.xcomp_bv = XCOMP_BV_COMPACTED_FORMAT |
+ xfeatures_mask;
if (static_cpu_has(X86_FEATURE_FXSR))
fpstate_init_fxstate(&state->fxsave);
} else {
struct hpet_dev *hdev = EVT_TO_HPET_DEV(evt);
+ irq_domain_deactivate_irq(irq_get_irq_data(hdev->irq));
irq_domain_activate_irq(irq_get_irq_data(hdev->irq));
disable_irq(hdev->irq);
irq_set_affinity(hdev->irq, cpumask_of(hdev->cpu));
int cpu1 = c->cpu_index, cpu2 = o->cpu_index;
if (c->phys_proc_id == o->phys_proc_id &&
- per_cpu(cpu_llc_id, cpu1) == per_cpu(cpu_llc_id, cpu2) &&
- c->cpu_core_id == o->cpu_core_id)
- return topology_sane(c, o, "smt");
+ per_cpu(cpu_llc_id, cpu1) == per_cpu(cpu_llc_id, cpu2)) {
+ if (c->cpu_core_id == o->cpu_core_id)
+ return topology_sane(c, o, "smt");
+
+ if ((c->cu_id != 0xff) &&
+ (o->cu_id != 0xff) &&
+ (c->cu_id == o->cu_id))
+ return topology_sane(c, o, "smt");
+ }
} else if (c->phys_proc_id == o->phys_proc_id &&
c->cpu_core_id == o->cpu_core_id) {
(unsigned long)cpu_khz / 1000,
(unsigned long)cpu_khz % 1000);
+ /* Sanitize TSC ADJUST before cyc2ns gets initialized */
+ tsc_store_and_check_tsc_adjust(true);
+
/*
* Secondary CPUs do not run through tsc_init(), so set up
* all the scale factors for all CPUs, assuming the same
if (unsynchronized_tsc())
mark_tsc_unstable("TSCs unsynchronized");
- else
- tsc_store_and_check_tsc_adjust(true);
check_system_tsc_reliable();
if (unsynchronized_tsc())
return;
- if (tsc_clocksource_reliable) {
- if (cpu == (nr_cpu_ids-1) || system_state != SYSTEM_BOOTING)
- pr_info(
- "Skipped synchronization checks as TSC is reliable.\n");
- return;
- }
-
/*
* Set the maximum number of test runs to
* 1 if the CPU does not provide the TSC_ADJUST MSR
int cpus = 2;
/* Also aborts if there is no TSC. */
- if (unsynchronized_tsc() || tsc_clocksource_reliable)
+ if (unsynchronized_tsc())
return;
/*
* Store, verify and sanitize the TSC adjust register. If
* successful skip the test.
+ *
+ * The test is also skipped when the TSC is marked reliable. This
+ * is true for SoCs which have no fallback clocksource. On these
+ * SoCs the TSC is frequency synchronized, but still the TSC ADJUST
+ * register might have been wreckaged by the BIOS..
*/
- if (tsc_store_and_check_tsc_adjust(false)) {
+ if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable) {
atomic_inc(&skip_test);
return;
}
memcpy(dest, xsave, XSAVE_HDR_OFFSET);
/* Set XSTATE_BV */
+ xstate_bv &= vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE;
*(u64 *)(dest + XSAVE_HDR_OFFSET) = xstate_bv;
/*
#include <linux/debugfs.h>
#include <linux/mm.h>
#include <linux/init.h>
+#include <linux/sched.h>
#include <linux/seq_file.h>
#include <asm/pgtable.h>
} else
note_page(m, &st, __pgprot(0), 1);
+ cond_resched();
start++;
}
efi_scratch.use_pgd = true;
+ /*
+ * Certain firmware versions are way too sentimential and still believe
+ * they are exclusive and unquestionable owners of the first physical page,
+ * even though they explicitly mark it as EFI_CONVENTIONAL_MEMORY
+ * (but then write-access it later during SetVirtualAddressMap()).
+ *
+ * Create a 1:1 mapping for this page, to avoid triple faults during early
+ * boot with such firmware. We are free to hand this page to the BIOS,
+ * as trim_bios_range() will reserve the first page and isolate it away
+ * from memory allocators anyway.
+ */
+ if (kernel_map_pages_in_pgd(pgd, 0x0, 0x0, 1, _PAGE_RW)) {
+ pr_err("Failed to create 1:1 mapping for the first page!\n");
+ return 1;
+ }
+
/*
* When making calls to the firmware everything needs to be 1:1
* mapped and addressable with 32-bit pointers. Map the kernel
void cpu_reset(void)
{
-#if XCHAL_HAVE_PTP_MMU
+#if XCHAL_HAVE_PTP_MMU && IS_ENABLED(CONFIG_MMU)
local_irq_disable();
/*
* We have full MMU: all autoload ways, ways 7, 8 and 9 of DTLB must
if (ret == 0 || (ret && ret != -EOPNOTSUPP))
goto out;
- ret = __blkdev_issue_write_same(bdev, sector, nr_sects, gfp_mask,
- ZERO_PAGE(0), biop);
- if (ret == 0 || (ret && ret != -EOPNOTSUPP))
- goto out;
-
ret = 0;
while (nr_sects != 0) {
bio = next_bio(bio, min(nr_sects, (sector_t)BIO_MAX_PAGES),
return 0;
}
+ if (!blkdev_issue_write_same(bdev, sector, nr_sects, gfp_mask,
+ ZERO_PAGE(0)))
+ return 0;
+
blk_start_plug(&plug);
ret = __blkdev_issue_zeroout(bdev, sector, nr_sects, gfp_mask,
&bio, discard);
struct crypto_larval *larval;
int err;
+ alg->cra_flags &= ~CRYPTO_ALG_DEAD;
err = crypto_check_alg(alg);
if (err)
return err;
unlock:
list_for_each_entry_safe(rsgl, tmp, &ctx->list, list) {
af_alg_free_sg(&rsgl->sgl);
+ list_del(&rsgl->list);
if (rsgl != &ctx->first_rsgl)
sock_kfree_s(sk, rsgl, sizeof(*rsgl));
- list_del(&rsgl->list);
}
INIT_LIST_HEAD(&ctx->list);
aead_wmem_wakeup(sk);
ACPI_FUNCTION_TRACE(tb_install_and_load_table);
- (void)acpi_ut_acquire_mutex(ACPI_MTX_TABLES);
-
/* Install the table and load it into the namespace */
status = acpi_tb_install_standard_table(address, flags, TRUE,
override, &i);
if (ACPI_FAILURE(status)) {
- goto unlock_and_exit;
+ goto exit;
}
- (void)acpi_ut_release_mutex(ACPI_MTX_TABLES);
status = acpi_tb_load_table(i, acpi_gbl_root_node);
- (void)acpi_ut_acquire_mutex(ACPI_MTX_TABLES);
-unlock_and_exit:
+exit:
*table_index = i;
- (void)acpi_ut_release_mutex(ACPI_MTX_TABLES);
return_ACPI_STATUS(status);
}
goto release_and_exit;
}
+ /* Acquire the table lock */
+
+ (void)acpi_ut_acquire_mutex(ACPI_MTX_TABLES);
+
if (reload) {
/*
* Validate the incoming table signature.
new_table_desc.signature.integer));
status = AE_BAD_SIGNATURE;
- goto release_and_exit;
+ goto unlock_and_exit;
}
/* Check if table is already registered */
/* Table is still loaded, this is an error */
status = AE_ALREADY_EXISTS;
- goto release_and_exit;
+ goto unlock_and_exit;
} else {
/*
* Table was unloaded, allow it to be reloaded.
* indicate the re-installation.
*/
acpi_tb_uninstall_table(&new_table_desc);
+ (void)acpi_ut_release_mutex(ACPI_MTX_TABLES);
*table_index = i;
return_ACPI_STATUS(AE_OK);
}
/* Invoke table handler if present */
+ (void)acpi_ut_release_mutex(ACPI_MTX_TABLES);
if (acpi_gbl_table_handler) {
(void)acpi_gbl_table_handler(ACPI_TABLE_EVENT_INSTALL,
new_table_desc.pointer,
acpi_gbl_table_handler_context);
}
+ (void)acpi_ut_acquire_mutex(ACPI_MTX_TABLES);
+
+unlock_and_exit:
+
+ /* Release the table lock */
+
+ (void)acpi_ut_release_mutex(ACPI_MTX_TABLES);
release_and_exit:
struct acpi_nfit_desc *acpi_desc = to_acpi_nfit_desc(nd_desc);
struct device *dev = acpi_desc->dev;
struct acpi_nfit_flush_work flush;
+ int rc;
/* bounce the device lock to flush acpi_nfit_add / acpi_nfit_notify */
device_lock(dev);
INIT_WORK_ONSTACK(&flush.work, flush_probe);
COMPLETION_INITIALIZER_ONSTACK(flush.cmp);
queue_work(nfit_wq, &flush.work);
- return wait_for_completion_interruptible(&flush.cmp);
+
+ rc = wait_for_completion_interruptible(&flush.cmp);
+ cancel_work_sync(&flush.work);
+ return rc;
}
static int acpi_nfit_clear_to_send(struct nvdimm_bus_descriptor *nd_desc,
struct acpi_processor *pr;
unsigned int ppc = 0;
- if (event == CPUFREQ_START && ignore_ppc <= 0) {
+ if (ignore_ppc < 0)
ignore_ppc = 0;
- return 0;
- }
if (ignore_ppc)
return 0;
if (acpi_sleep_state_supported(i))
sleep_states[i] = 1;
- /*
- * Use suspend-to-idle by default if ACPI_FADT_LOW_POWER_S0 is set and
- * the default suspend mode was not selected from the command line.
- */
- if (acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0 &&
- mem_sleep_default > PM_SUSPEND_MEM)
- mem_sleep_default = PM_SUSPEND_FREEZE;
-
suspend_set_ops(old_suspend_ordering ?
&acpi_suspend_ops_old : &acpi_suspend_ops);
freeze_set_ops(&acpi_freeze_ops);
DMI_MATCH(DMI_PRODUCT_NAME, "Dell System XPS L702X"),
},
},
- {
- /* https://bugzilla.redhat.com/show_bug.cgi?id=1204476 */
- /* https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1416940 */
- .callback = video_detect_force_native,
- .ident = "HP Pavilion dv6",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"),
- DMI_MATCH(DMI_PRODUCT_NAME, "HP Pavilion dv6 Notebook PC"),
- },
- },
-
{ },
};
if (qc->err_mask & ~AC_ERR_OTHER)
qc->err_mask &= ~AC_ERR_OTHER;
+ } else if (qc->tf.command == ATA_CMD_REQ_SENSE_DATA) {
+ qc->result_tf.command |= ATA_SENSE;
}
/* finish up */
{ "ST380013AS", "3.20", ATA_HORKAGE_MAX_SEC_1024 },
/*
- * Device times out with higher max sects.
+ * These devices time out with higher max sects.
* https://bugzilla.kernel.org/show_bug.cgi?id=121671
*/
- { "LITEON CX1-JB256-HP", NULL, ATA_HORKAGE_MAX_SEC_1024 },
+ { "LITEON CX1-JB*-HP", NULL, ATA_HORKAGE_MAX_SEC_1024 },
/* Devices we expect to fail diagnostics */
host->iomap = NULL;
hpriv->base = devm_ioremap(&pdev->dev, res->start,
resource_size(res));
+ if (!hpriv->base)
+ return -ENOMEM;
+
hpriv->base -= SATAHC0_REG_BASE;
hpriv->clk = clk_get(&pdev->dev, NULL);
#include <linux/of.h>
#include <linux/cpufeature.h>
#include <linux/tick.h>
+#include <linux/pm_qos.h>
#include "base.h"
per_cpu(cpu_sys_devices, num) = &cpu->dev;
register_cpu_under_node(num, cpu_to_node(num));
+ dev_pm_qos_expose_latency_limit(&cpu->dev, 0);
return 0;
}
struct firmware_buf *buf = fw_priv->buf;
__fw_load_abort(buf);
-
- /* avoid user action after loading abort */
- fw_priv->buf = NULL;
}
static LIST_HEAD(pending_fw_head);
mutex_lock(&fw_lock);
fw_buf = fw_priv->buf;
- if (!fw_buf)
+ if (fw_state_is_aborted(&fw_buf->fw_st))
goto out;
switch (loading) {
{
struct memory_block *mem = to_memory_block(dev);
unsigned long start_pfn, end_pfn;
+ unsigned long valid_start, valid_end, valid_pages;
unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
- struct page *first_page;
struct zone *zone;
int zone_shift = 0;
start_pfn = section_nr_to_pfn(mem->start_section_nr);
end_pfn = start_pfn + nr_pages;
- first_page = pfn_to_page(start_pfn);
/* The block contains more than one zone can not be offlined. */
- if (!test_pages_in_a_zone(start_pfn, end_pfn))
+ if (!test_pages_in_a_zone(start_pfn, end_pfn, &valid_start, &valid_end))
return sprintf(buf, "none\n");
- zone = page_zone(first_page);
+ zone = page_zone(pfn_to_page(valid_start));
+ valid_pages = valid_end - valid_start;
/* MMOP_ONLINE_KEEP */
sprintf(buf, "%s", zone->name);
/* MMOP_ONLINE_KERNEL */
- zone_shift = zone_can_shift(start_pfn, nr_pages, ZONE_NORMAL);
+ zone_can_shift(valid_start, valid_pages, ZONE_NORMAL, &zone_shift);
if (zone_shift) {
strcat(buf, " ");
strcat(buf, (zone + zone_shift)->name);
}
/* MMOP_ONLINE_MOVABLE */
- zone_shift = zone_can_shift(start_pfn, nr_pages, ZONE_MOVABLE);
+ zone_can_shift(valid_start, valid_pages, ZONE_MOVABLE, &zone_shift);
if (zone_shift) {
strcat(buf, " ");
strcat(buf, (zone + zone_shift)->name);
ret = pm_runtime_is_irq_safe(dev) && !genpd_is_irq_safe(genpd);
- /* Warn once for each IRQ safe dev in no sleep domain */
+ /* Warn once if IRQ safe dev in no sleep domain */
if (ret)
dev_warn_once(dev, "PM domain %s will not be powered off\n",
genpd->name);
smp_mb__after_atomic();
}
-static int genpd_power_on(struct generic_pm_domain *genpd, bool timed)
+static int _genpd_power_on(struct generic_pm_domain *genpd, bool timed)
{
unsigned int state_idx = genpd->state_idx;
ktime_t time_start;
return ret;
}
-static int genpd_power_off(struct generic_pm_domain *genpd, bool timed)
+static int _genpd_power_off(struct generic_pm_domain *genpd, bool timed)
{
unsigned int state_idx = genpd->state_idx;
ktime_t time_start;
}
/**
- * genpd_queue_power_off_work - Queue up the execution of genpd_poweroff().
+ * genpd_queue_power_off_work - Queue up the execution of genpd_power_off().
* @genpd: PM domain to power off.
*
- * Queue up the execution of genpd_poweroff() unless it's already been done
+ * Queue up the execution of genpd_power_off() unless it's already been done
* before.
*/
static void genpd_queue_power_off_work(struct generic_pm_domain *genpd)
}
/**
- * genpd_poweron - Restore power to a given PM domain and its masters.
+ * genpd_power_on - Restore power to a given PM domain and its masters.
* @genpd: PM domain to power up.
* @depth: nesting count for lockdep.
*
* Restore power to @genpd and all of its masters so that it is possible to
* resume a device belonging to it.
*/
-static int genpd_poweron(struct generic_pm_domain *genpd, unsigned int depth)
+static int genpd_power_on(struct generic_pm_domain *genpd, unsigned int depth)
{
struct gpd_link *link;
int ret = 0;
genpd_sd_counter_inc(master);
genpd_lock_nested(master, depth + 1);
- ret = genpd_poweron(master, depth + 1);
+ ret = genpd_power_on(master, depth + 1);
genpd_unlock(master);
if (ret) {
}
}
- ret = genpd_power_on(genpd, true);
+ ret = _genpd_power_on(genpd, true);
if (ret)
goto err;
}
/**
- * genpd_poweroff - Remove power from a given PM domain.
+ * genpd_power_off - Remove power from a given PM domain.
* @genpd: PM domain to power down.
* @is_async: PM domain is powered down from a scheduled work
*
* If all of the @genpd's devices have been suspended and all of its subdomains
* have been powered down, remove power from @genpd.
*/
-static int genpd_poweroff(struct generic_pm_domain *genpd, bool is_async)
+static int genpd_power_off(struct generic_pm_domain *genpd, bool is_async)
{
struct pm_domain_data *pdd;
struct gpd_link *link;
/*
* If sd_count > 0 at this point, one of the subdomains hasn't
- * managed to call genpd_poweron() for the master yet after
- * incrementing it. In that case genpd_poweron() will wait
+ * managed to call genpd_power_on() for the master yet after
+ * incrementing it. In that case genpd_power_on() will wait
* for us to drop the lock, so we can call .power_off() and let
- * the genpd_poweron() restore power for us (this shouldn't
+ * the genpd_power_on() restore power for us (this shouldn't
* happen very often).
*/
- ret = genpd_power_off(genpd, true);
+ ret = _genpd_power_off(genpd, true);
if (ret)
return ret;
}
genpd = container_of(work, struct generic_pm_domain, power_off_work);
genpd_lock(genpd);
- genpd_poweroff(genpd, true);
+ genpd_power_off(genpd, true);
genpd_unlock(genpd);
}
return 0;
genpd_lock(genpd);
- genpd_poweroff(genpd, false);
+ genpd_power_off(genpd, false);
genpd_unlock(genpd);
return 0;
}
genpd_lock(genpd);
- ret = genpd_poweron(genpd, 0);
+ ret = genpd_power_on(genpd, 0);
genpd_unlock(genpd);
if (ret)
if (!pm_runtime_is_irq_safe(dev) ||
(pm_runtime_is_irq_safe(dev) && genpd_is_irq_safe(genpd))) {
genpd_lock(genpd);
- genpd_poweroff(genpd, 0);
+ genpd_power_off(genpd, 0);
genpd_unlock(genpd);
}
__setup("pd_ignore_unused", pd_ignore_unused_setup);
/**
- * genpd_poweroff_unused - Power off all PM domains with no devices in use.
+ * genpd_power_off_unused - Power off all PM domains with no devices in use.
*/
-static int __init genpd_poweroff_unused(void)
+static int __init genpd_power_off_unused(void)
{
struct generic_pm_domain *genpd;
return 0;
}
-late_initcall(genpd_poweroff_unused);
+late_initcall(genpd_power_off_unused);
#if defined(CONFIG_PM_SLEEP) || defined(CONFIG_PM_GENERIC_DOMAINS_OF)
}
/**
- * genpd_sync_poweroff - Synchronously power off a PM domain and its masters.
+ * genpd_sync_power_off - Synchronously power off a PM domain and its masters.
* @genpd: PM domain to power off, if possible.
+ * @use_lock: use the lock.
+ * @depth: nesting count for lockdep.
*
* Check if the given PM domain can be powered off (during system suspend or
* hibernation) and do that if so. Also, in that case propagate to its masters.
*
* This function is only called in "noirq" and "syscore" stages of system power
- * transitions, so it need not acquire locks (all of the "noirq" callbacks are
- * executed sequentially, so it is guaranteed that it will never run twice in
- * parallel).
+ * transitions. The "noirq" callbacks may be executed asynchronously, thus in
+ * these cases the lock must be held.
*/
-static void genpd_sync_poweroff(struct generic_pm_domain *genpd)
+static void genpd_sync_power_off(struct generic_pm_domain *genpd, bool use_lock,
+ unsigned int depth)
{
struct gpd_link *link;
/* Choose the deepest state when suspending */
genpd->state_idx = genpd->state_count - 1;
- genpd_power_off(genpd, false);
+ _genpd_power_off(genpd, false);
genpd->status = GPD_STATE_POWER_OFF;
list_for_each_entry(link, &genpd->slave_links, slave_node) {
genpd_sd_counter_dec(link->master);
- genpd_sync_poweroff(link->master);
+
+ if (use_lock)
+ genpd_lock_nested(link->master, depth + 1);
+
+ genpd_sync_power_off(link->master, use_lock, depth + 1);
+
+ if (use_lock)
+ genpd_unlock(link->master);
}
}
/**
- * genpd_sync_poweron - Synchronously power on a PM domain and its masters.
+ * genpd_sync_power_on - Synchronously power on a PM domain and its masters.
* @genpd: PM domain to power on.
+ * @use_lock: use the lock.
+ * @depth: nesting count for lockdep.
*
* This function is only called in "noirq" and "syscore" stages of system power
- * transitions, so it need not acquire locks (all of the "noirq" callbacks are
- * executed sequentially, so it is guaranteed that it will never run twice in
- * parallel).
+ * transitions. The "noirq" callbacks may be executed asynchronously, thus in
+ * these cases the lock must be held.
*/
-static void genpd_sync_poweron(struct generic_pm_domain *genpd)
+static void genpd_sync_power_on(struct generic_pm_domain *genpd, bool use_lock,
+ unsigned int depth)
{
struct gpd_link *link;
return;
list_for_each_entry(link, &genpd->slave_links, slave_node) {
- genpd_sync_poweron(link->master);
genpd_sd_counter_inc(link->master);
+
+ if (use_lock)
+ genpd_lock_nested(link->master, depth + 1);
+
+ genpd_sync_power_on(link->master, use_lock, depth + 1);
+
+ if (use_lock)
+ genpd_unlock(link->master);
}
- genpd_power_on(genpd, false);
+ _genpd_power_on(genpd, false);
genpd->status = GPD_STATE_ACTIVE;
}
return ret;
}
- /*
- * Since all of the "noirq" callbacks are executed sequentially, it is
- * guaranteed that this function will never run twice in parallel for
- * the same PM domain, so it is not necessary to use locking here.
- */
+ genpd_lock(genpd);
genpd->suspended_count++;
- genpd_sync_poweroff(genpd);
+ genpd_sync_power_off(genpd, true, 0);
+ genpd_unlock(genpd);
return 0;
}
if (dev->power.wakeup_path && genpd_dev_active_wakeup(genpd, dev))
return 0;
- /*
- * Since all of the "noirq" callbacks are executed sequentially, it is
- * guaranteed that this function will never run twice in parallel for
- * the same PM domain, so it is not necessary to use locking here.
- */
- genpd_sync_poweron(genpd);
+ genpd_lock(genpd);
+ genpd_sync_power_on(genpd, true, 0);
genpd->suspended_count--;
+ genpd_unlock(genpd);
if (genpd->dev_ops.stop && genpd->dev_ops.start)
ret = pm_runtime_force_resume(dev);
return -EINVAL;
/*
- * Since all of the "noirq" callbacks are executed sequentially, it is
- * guaranteed that this function will never run twice in parallel for
- * the same PM domain, so it is not necessary to use locking here.
- *
* At this point suspended_count == 0 means we are being run for the
* first time for the given domain in the present cycle.
*/
+ genpd_lock(genpd);
if (genpd->suspended_count++ == 0)
/*
* The boot kernel might put the domain into arbitrary state,
- * so make it appear as powered off to genpd_sync_poweron(),
+ * so make it appear as powered off to genpd_sync_power_on(),
* so that it tries to power it on in case it was really off.
*/
genpd->status = GPD_STATE_POWER_OFF;
- genpd_sync_poweron(genpd);
+ genpd_sync_power_on(genpd, true, 0);
+ genpd_unlock(genpd);
if (genpd->dev_ops.stop && genpd->dev_ops.start)
ret = pm_runtime_force_resume(dev);
if (suspend) {
genpd->suspended_count++;
- genpd_sync_poweroff(genpd);
+ genpd_sync_power_off(genpd, false, 0);
} else {
- genpd_sync_poweron(genpd);
+ genpd_sync_power_on(genpd, false, 0);
genpd->suspended_count--;
}
}
dev->pm_domain->sync = genpd_dev_pm_sync;
genpd_lock(pd);
- ret = genpd_poweron(pd, 0);
+ ret = genpd_power_on(pd, 0);
genpd_unlock(pd);
out:
return ret ? -EPROBE_DEFER : 0;
/* Lock to allow exclusive modification to the device and opp lists */
DEFINE_MUTEX(opp_table_lock);
-#define opp_rcu_lockdep_assert() \
-do { \
- RCU_LOCKDEP_WARN(!rcu_read_lock_held() && \
- !lockdep_is_held(&opp_table_lock), \
- "Missing rcu_read_lock() or " \
- "opp_table_lock protection"); \
-} while (0)
+static void dev_pm_opp_get(struct dev_pm_opp *opp);
static struct opp_device *_find_opp_dev(const struct device *dev,
struct opp_table *opp_table)
return NULL;
}
+static struct opp_table *_find_opp_table_unlocked(struct device *dev)
+{
+ struct opp_table *opp_table;
+
+ list_for_each_entry(opp_table, &opp_tables, node) {
+ if (_find_opp_dev(dev, opp_table)) {
+ _get_opp_table_kref(opp_table);
+
+ return opp_table;
+ }
+ }
+
+ return ERR_PTR(-ENODEV);
+}
+
/**
* _find_opp_table() - find opp_table struct using device pointer
* @dev: device pointer used to lookup OPP table
*
- * Search OPP table for one containing matching device. Does a RCU reader
- * operation to grab the pointer needed.
+ * Search OPP table for one containing matching device.
*
* Return: pointer to 'struct opp_table' if found, otherwise -ENODEV or
* -EINVAL based on type of error.
*
- * Locking: For readers, this function must be called under rcu_read_lock().
- * opp_table is a RCU protected pointer, which means that opp_table is valid
- * as long as we are under RCU lock.
- *
- * For Writers, this function must be called with opp_table_lock held.
+ * The callers must call dev_pm_opp_put_opp_table() after the table is used.
*/
struct opp_table *_find_opp_table(struct device *dev)
{
struct opp_table *opp_table;
- opp_rcu_lockdep_assert();
-
if (IS_ERR_OR_NULL(dev)) {
pr_err("%s: Invalid parameters\n", __func__);
return ERR_PTR(-EINVAL);
}
- list_for_each_entry_rcu(opp_table, &opp_tables, node)
- if (_find_opp_dev(dev, opp_table))
- return opp_table;
+ mutex_lock(&opp_table_lock);
+ opp_table = _find_opp_table_unlocked(dev);
+ mutex_unlock(&opp_table_lock);
- return ERR_PTR(-ENODEV);
+ return opp_table;
}
/**
* return 0
*
* This is useful only for devices with single power supply.
- *
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. This means that opp which could have been fetched by
- * opp_find_freq_{exact,ceil,floor} functions is valid as long as we are
- * under RCU lock. The pointer returned by the opp_find_freq family must be
- * used in the same section as the usage of this function with the pointer
- * prior to unlocking with rcu_read_unlock() to maintain the integrity of the
- * pointer.
*/
unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp)
{
- struct dev_pm_opp *tmp_opp;
- unsigned long v = 0;
-
- opp_rcu_lockdep_assert();
-
- tmp_opp = rcu_dereference(opp);
- if (IS_ERR_OR_NULL(tmp_opp))
+ if (IS_ERR_OR_NULL(opp)) {
pr_err("%s: Invalid parameters\n", __func__);
- else
- v = tmp_opp->supplies[0].u_volt;
+ return 0;
+ }
- return v;
+ return opp->supplies[0].u_volt;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_get_voltage);
*
* Return: frequency in hertz corresponding to the opp, else
* return 0
- *
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. This means that opp which could have been fetched by
- * opp_find_freq_{exact,ceil,floor} functions is valid as long as we are
- * under RCU lock. The pointer returned by the opp_find_freq family must be
- * used in the same section as the usage of this function with the pointer
- * prior to unlocking with rcu_read_unlock() to maintain the integrity of the
- * pointer.
*/
unsigned long dev_pm_opp_get_freq(struct dev_pm_opp *opp)
{
- struct dev_pm_opp *tmp_opp;
- unsigned long f = 0;
-
- opp_rcu_lockdep_assert();
-
- tmp_opp = rcu_dereference(opp);
- if (IS_ERR_OR_NULL(tmp_opp) || !tmp_opp->available)
+ if (IS_ERR_OR_NULL(opp) || !opp->available) {
pr_err("%s: Invalid parameters\n", __func__);
- else
- f = tmp_opp->rate;
+ return 0;
+ }
- return f;
+ return opp->rate;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_get_freq);
* quickly. Running on them for longer times may overheat the chip.
*
* Return: true if opp is turbo opp, else false.
- *
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. This means that opp which could have been fetched by
- * opp_find_freq_{exact,ceil,floor} functions is valid as long as we are
- * under RCU lock. The pointer returned by the opp_find_freq family must be
- * used in the same section as the usage of this function with the pointer
- * prior to unlocking with rcu_read_unlock() to maintain the integrity of the
- * pointer.
*/
bool dev_pm_opp_is_turbo(struct dev_pm_opp *opp)
{
- struct dev_pm_opp *tmp_opp;
-
- opp_rcu_lockdep_assert();
-
- tmp_opp = rcu_dereference(opp);
- if (IS_ERR_OR_NULL(tmp_opp) || !tmp_opp->available) {
+ if (IS_ERR_OR_NULL(opp) || !opp->available) {
pr_err("%s: Invalid parameters\n", __func__);
return false;
}
- return tmp_opp->turbo;
+ return opp->turbo;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_is_turbo);
* @dev: device for which we do this operation
*
* Return: This function returns the max clock latency in nanoseconds.
- *
- * Locking: This function takes rcu_read_lock().
*/
unsigned long dev_pm_opp_get_max_clock_latency(struct device *dev)
{
struct opp_table *opp_table;
unsigned long clock_latency_ns;
- rcu_read_lock();
-
opp_table = _find_opp_table(dev);
if (IS_ERR(opp_table))
- clock_latency_ns = 0;
- else
- clock_latency_ns = opp_table->clock_latency_ns_max;
-
- rcu_read_unlock();
- return clock_latency_ns;
-}
-EXPORT_SYMBOL_GPL(dev_pm_opp_get_max_clock_latency);
-
-static int _get_regulator_count(struct device *dev)
-{
- struct opp_table *opp_table;
- int count;
+ return 0;
- rcu_read_lock();
+ clock_latency_ns = opp_table->clock_latency_ns_max;
- opp_table = _find_opp_table(dev);
- if (!IS_ERR(opp_table))
- count = opp_table->regulator_count;
- else
- count = 0;
+ dev_pm_opp_put_opp_table(opp_table);
- rcu_read_unlock();
-
- return count;
+ return clock_latency_ns;
}
+EXPORT_SYMBOL_GPL(dev_pm_opp_get_max_clock_latency);
/**
* dev_pm_opp_get_max_volt_latency() - Get max voltage latency in nanoseconds
* @dev: device for which we do this operation
*
* Return: This function returns the max voltage latency in nanoseconds.
- *
- * Locking: This function takes rcu_read_lock().
*/
unsigned long dev_pm_opp_get_max_volt_latency(struct device *dev)
{
unsigned long max;
} *uV;
- count = _get_regulator_count(dev);
+ opp_table = _find_opp_table(dev);
+ if (IS_ERR(opp_table))
+ return 0;
+
+ count = opp_table->regulator_count;
/* Regulator may not be required for the device */
if (!count)
- return 0;
+ goto put_opp_table;
regulators = kmalloc_array(count, sizeof(*regulators), GFP_KERNEL);
if (!regulators)
- return 0;
+ goto put_opp_table;
uV = kmalloc_array(count, sizeof(*uV), GFP_KERNEL);
if (!uV)
goto free_regulators;
- rcu_read_lock();
-
- opp_table = _find_opp_table(dev);
- if (IS_ERR(opp_table)) {
- rcu_read_unlock();
- goto free_uV;
- }
-
memcpy(regulators, opp_table->regulators, count * sizeof(*regulators));
+ mutex_lock(&opp_table->lock);
+
for (i = 0; i < count; i++) {
uV[i].min = ~0;
uV[i].max = 0;
- list_for_each_entry_rcu(opp, &opp_table->opp_list, node) {
+ list_for_each_entry(opp, &opp_table->opp_list, node) {
if (!opp->available)
continue;
}
}
- rcu_read_unlock();
+ mutex_unlock(&opp_table->lock);
/*
* The caller needs to ensure that opp_table (and hence the regulator)
latency_ns += ret * 1000;
}
-free_uV:
kfree(uV);
free_regulators:
kfree(regulators);
+put_opp_table:
+ dev_pm_opp_put_opp_table(opp_table);
return latency_ns;
}
*
* Return: This function returns the max transition latency, in nanoseconds, to
* switch from one OPP to other.
- *
- * Locking: This function takes rcu_read_lock().
*/
unsigned long dev_pm_opp_get_max_transition_latency(struct device *dev)
{
EXPORT_SYMBOL_GPL(dev_pm_opp_get_max_transition_latency);
/**
- * dev_pm_opp_get_suspend_opp() - Get suspend opp
+ * dev_pm_opp_get_suspend_opp_freq() - Get frequency of suspend opp in Hz
* @dev: device for which we do this operation
*
- * Return: This function returns pointer to the suspend opp if it is
- * defined and available, otherwise it returns NULL.
- *
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. The reason for the same is that the opp pointer which is
- * returned will remain valid for use with opp_get_{voltage, freq} only while
- * under the locked area. The pointer returned must be used prior to unlocking
- * with rcu_read_unlock() to maintain the integrity of the pointer.
+ * Return: This function returns the frequency of the OPP marked as suspend_opp
+ * if one is available, else returns 0;
*/
-struct dev_pm_opp *dev_pm_opp_get_suspend_opp(struct device *dev)
+unsigned long dev_pm_opp_get_suspend_opp_freq(struct device *dev)
{
struct opp_table *opp_table;
-
- opp_rcu_lockdep_assert();
+ unsigned long freq = 0;
opp_table = _find_opp_table(dev);
- if (IS_ERR(opp_table) || !opp_table->suspend_opp ||
- !opp_table->suspend_opp->available)
- return NULL;
+ if (IS_ERR(opp_table))
+ return 0;
+
+ if (opp_table->suspend_opp && opp_table->suspend_opp->available)
+ freq = dev_pm_opp_get_freq(opp_table->suspend_opp);
- return opp_table->suspend_opp;
+ dev_pm_opp_put_opp_table(opp_table);
+
+ return freq;
}
-EXPORT_SYMBOL_GPL(dev_pm_opp_get_suspend_opp);
+EXPORT_SYMBOL_GPL(dev_pm_opp_get_suspend_opp_freq);
/**
* dev_pm_opp_get_opp_count() - Get number of opps available in the opp table
*
* Return: This function returns the number of available opps if there are any,
* else returns 0 if none or the corresponding error value.
- *
- * Locking: This function takes rcu_read_lock().
*/
int dev_pm_opp_get_opp_count(struct device *dev)
{
struct dev_pm_opp *temp_opp;
int count = 0;
- rcu_read_lock();
-
opp_table = _find_opp_table(dev);
if (IS_ERR(opp_table)) {
count = PTR_ERR(opp_table);
dev_err(dev, "%s: OPP table not found (%d)\n",
__func__, count);
- goto out_unlock;
+ return count;
}
- list_for_each_entry_rcu(temp_opp, &opp_table->opp_list, node) {
+ mutex_lock(&opp_table->lock);
+
+ list_for_each_entry(temp_opp, &opp_table->opp_list, node) {
if (temp_opp->available)
count++;
}
-out_unlock:
- rcu_read_unlock();
+ mutex_unlock(&opp_table->lock);
+ dev_pm_opp_put_opp_table(opp_table);
+
return count;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_get_opp_count);
* This provides a mechanism to enable an opp which is not available currently
* or the opposite as well.
*
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. The reason for the same is that the opp pointer which is
- * returned will remain valid for use with opp_get_{voltage, freq} only while
- * under the locked area. The pointer returned must be used prior to unlocking
- * with rcu_read_unlock() to maintain the integrity of the pointer.
+ * The callers are required to call dev_pm_opp_put() for the returned OPP after
+ * use.
*/
struct dev_pm_opp *dev_pm_opp_find_freq_exact(struct device *dev,
unsigned long freq,
struct opp_table *opp_table;
struct dev_pm_opp *temp_opp, *opp = ERR_PTR(-ERANGE);
- opp_rcu_lockdep_assert();
-
opp_table = _find_opp_table(dev);
if (IS_ERR(opp_table)) {
int r = PTR_ERR(opp_table);
return ERR_PTR(r);
}
- list_for_each_entry_rcu(temp_opp, &opp_table->opp_list, node) {
+ mutex_lock(&opp_table->lock);
+
+ list_for_each_entry(temp_opp, &opp_table->opp_list, node) {
if (temp_opp->available == available &&
temp_opp->rate == freq) {
opp = temp_opp;
+
+ /* Increment the reference count of OPP */
+ dev_pm_opp_get(opp);
break;
}
}
+ mutex_unlock(&opp_table->lock);
+ dev_pm_opp_put_opp_table(opp_table);
+
return opp;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_find_freq_exact);
{
struct dev_pm_opp *temp_opp, *opp = ERR_PTR(-ERANGE);
- list_for_each_entry_rcu(temp_opp, &opp_table->opp_list, node) {
+ mutex_lock(&opp_table->lock);
+
+ list_for_each_entry(temp_opp, &opp_table->opp_list, node) {
if (temp_opp->available && temp_opp->rate >= *freq) {
opp = temp_opp;
*freq = opp->rate;
+
+ /* Increment the reference count of OPP */
+ dev_pm_opp_get(opp);
break;
}
}
+ mutex_unlock(&opp_table->lock);
+
return opp;
}
* ERANGE: no match found for search
* ENODEV: if device not found in list of registered devices
*
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. The reason for the same is that the opp pointer which is
- * returned will remain valid for use with opp_get_{voltage, freq} only while
- * under the locked area. The pointer returned must be used prior to unlocking
- * with rcu_read_unlock() to maintain the integrity of the pointer.
+ * The callers are required to call dev_pm_opp_put() for the returned OPP after
+ * use.
*/
struct dev_pm_opp *dev_pm_opp_find_freq_ceil(struct device *dev,
unsigned long *freq)
{
struct opp_table *opp_table;
-
- opp_rcu_lockdep_assert();
+ struct dev_pm_opp *opp;
if (!dev || !freq) {
dev_err(dev, "%s: Invalid argument freq=%p\n", __func__, freq);
if (IS_ERR(opp_table))
return ERR_CAST(opp_table);
- return _find_freq_ceil(opp_table, freq);
+ opp = _find_freq_ceil(opp_table, freq);
+
+ dev_pm_opp_put_opp_table(opp_table);
+
+ return opp;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_find_freq_ceil);
* ERANGE: no match found for search
* ENODEV: if device not found in list of registered devices
*
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. The reason for the same is that the opp pointer which is
- * returned will remain valid for use with opp_get_{voltage, freq} only while
- * under the locked area. The pointer returned must be used prior to unlocking
- * with rcu_read_unlock() to maintain the integrity of the pointer.
+ * The callers are required to call dev_pm_opp_put() for the returned OPP after
+ * use.
*/
struct dev_pm_opp *dev_pm_opp_find_freq_floor(struct device *dev,
unsigned long *freq)
struct opp_table *opp_table;
struct dev_pm_opp *temp_opp, *opp = ERR_PTR(-ERANGE);
- opp_rcu_lockdep_assert();
-
if (!dev || !freq) {
dev_err(dev, "%s: Invalid argument freq=%p\n", __func__, freq);
return ERR_PTR(-EINVAL);
if (IS_ERR(opp_table))
return ERR_CAST(opp_table);
- list_for_each_entry_rcu(temp_opp, &opp_table->opp_list, node) {
+ mutex_lock(&opp_table->lock);
+
+ list_for_each_entry(temp_opp, &opp_table->opp_list, node) {
if (temp_opp->available) {
/* go to the next node, before choosing prev */
if (temp_opp->rate > *freq)
opp = temp_opp;
}
}
+
+ /* Increment the reference count of OPP */
+ if (!IS_ERR(opp))
+ dev_pm_opp_get(opp);
+ mutex_unlock(&opp_table->lock);
+ dev_pm_opp_put_opp_table(opp_table);
+
if (!IS_ERR(opp))
*freq = opp->rate;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_find_freq_floor);
-/*
- * The caller needs to ensure that opp_table (and hence the clk) isn't freed,
- * while clk returned here is used.
- */
-static struct clk *_get_opp_clk(struct device *dev)
-{
- struct opp_table *opp_table;
- struct clk *clk;
-
- rcu_read_lock();
-
- opp_table = _find_opp_table(dev);
- if (IS_ERR(opp_table)) {
- dev_err(dev, "%s: device opp doesn't exist\n", __func__);
- clk = ERR_CAST(opp_table);
- goto unlock;
- }
-
- clk = opp_table->clk;
- if (IS_ERR(clk))
- dev_err(dev, "%s: No clock available for the device\n",
- __func__);
-
-unlock:
- rcu_read_unlock();
- return clk;
-}
-
static int _set_opp_voltage(struct device *dev, struct regulator *reg,
struct dev_pm_opp_supply *supply)
{
*
* This configures the power-supplies and clock source to the levels specified
* by the OPP corresponding to the target_freq.
- *
- * Locking: This function takes rcu_read_lock().
*/
int dev_pm_opp_set_rate(struct device *dev, unsigned long target_freq)
{
return -EINVAL;
}
- clk = _get_opp_clk(dev);
- if (IS_ERR(clk))
- return PTR_ERR(clk);
+ opp_table = _find_opp_table(dev);
+ if (IS_ERR(opp_table)) {
+ dev_err(dev, "%s: device opp doesn't exist\n", __func__);
+ return PTR_ERR(opp_table);
+ }
+
+ clk = opp_table->clk;
+ if (IS_ERR(clk)) {
+ dev_err(dev, "%s: No clock available for the device\n",
+ __func__);
+ ret = PTR_ERR(clk);
+ goto put_opp_table;
+ }
freq = clk_round_rate(clk, target_freq);
if ((long)freq <= 0)
if (old_freq == freq) {
dev_dbg(dev, "%s: old/new frequencies (%lu Hz) are same, nothing to do\n",
__func__, freq);
- return 0;
- }
-
- rcu_read_lock();
-
- opp_table = _find_opp_table(dev);
- if (IS_ERR(opp_table)) {
- dev_err(dev, "%s: device opp doesn't exist\n", __func__);
- rcu_read_unlock();
- return PTR_ERR(opp_table);
+ ret = 0;
+ goto put_opp_table;
}
old_opp = _find_freq_ceil(opp_table, &old_freq);
ret = PTR_ERR(opp);
dev_err(dev, "%s: failed to find OPP for freq %lu (%d)\n",
__func__, freq, ret);
- rcu_read_unlock();
- return ret;
+ goto put_old_opp;
}
dev_dbg(dev, "%s: switching OPP: %lu Hz --> %lu Hz\n", __func__,
/* Only frequency scaling */
if (!regulators) {
- rcu_read_unlock();
- return _generic_set_opp_clk_only(dev, clk, old_freq, freq);
+ ret = _generic_set_opp_clk_only(dev, clk, old_freq, freq);
+ goto put_opps;
}
if (opp_table->set_opp)
data->new_opp.rate = freq;
memcpy(data->new_opp.supplies, opp->supplies, size);
- rcu_read_unlock();
+ ret = set_opp(data);
- return set_opp(data);
+put_opps:
+ dev_pm_opp_put(opp);
+put_old_opp:
+ if (!IS_ERR(old_opp))
+ dev_pm_opp_put(old_opp);
+put_opp_table:
+ dev_pm_opp_put_opp_table(opp_table);
+ return ret;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_set_rate);
/* OPP-dev Helpers */
-static void _kfree_opp_dev_rcu(struct rcu_head *head)
-{
- struct opp_device *opp_dev;
-
- opp_dev = container_of(head, struct opp_device, rcu_head);
- kfree_rcu(opp_dev, rcu_head);
-}
-
static void _remove_opp_dev(struct opp_device *opp_dev,
struct opp_table *opp_table)
{
opp_debug_unregister(opp_dev, opp_table);
list_del(&opp_dev->node);
- call_srcu(&opp_table->srcu_head.srcu, &opp_dev->rcu_head,
- _kfree_opp_dev_rcu);
+ kfree(opp_dev);
}
struct opp_device *_add_opp_dev(const struct device *dev,
/* Initialize opp-dev */
opp_dev->dev = dev;
- list_add_rcu(&opp_dev->node, &opp_table->dev_list);
+ list_add(&opp_dev->node, &opp_table->dev_list);
/* Create debugfs entries for the opp_table */
ret = opp_debug_register(opp_dev, opp_table);
return opp_dev;
}
-/**
- * _add_opp_table() - Find OPP table or allocate a new one
- * @dev: device for which we do this operation
- *
- * It tries to find an existing table first, if it couldn't find one, it
- * allocates a new OPP table and returns that.
- *
- * Return: valid opp_table pointer if success, else NULL.
- */
-static struct opp_table *_add_opp_table(struct device *dev)
+static struct opp_table *_allocate_opp_table(struct device *dev)
{
struct opp_table *opp_table;
struct opp_device *opp_dev;
int ret;
- /* Check for existing table for 'dev' first */
- opp_table = _find_opp_table(dev);
- if (!IS_ERR(opp_table))
- return opp_table;
-
/*
* Allocate a new OPP table. In the infrequent case where a new
* device is needed to be added, we pay this penalty.
ret);
}
- srcu_init_notifier_head(&opp_table->srcu_head);
+ BLOCKING_INIT_NOTIFIER_HEAD(&opp_table->head);
INIT_LIST_HEAD(&opp_table->opp_list);
+ mutex_init(&opp_table->lock);
+ kref_init(&opp_table->kref);
/* Secure the device table modification */
- list_add_rcu(&opp_table->node, &opp_tables);
+ list_add(&opp_table->node, &opp_tables);
return opp_table;
}
-/**
- * _kfree_device_rcu() - Free opp_table RCU handler
- * @head: RCU head
- */
-static void _kfree_device_rcu(struct rcu_head *head)
+void _get_opp_table_kref(struct opp_table *opp_table)
{
- struct opp_table *opp_table = container_of(head, struct opp_table,
- rcu_head);
-
- kfree_rcu(opp_table, rcu_head);
+ kref_get(&opp_table->kref);
}
-/**
- * _remove_opp_table() - Removes a OPP table
- * @opp_table: OPP table to be removed.
- *
- * Removes/frees OPP table if it doesn't contain any OPPs.
- */
-static void _remove_opp_table(struct opp_table *opp_table)
+struct opp_table *dev_pm_opp_get_opp_table(struct device *dev)
{
- struct opp_device *opp_dev;
+ struct opp_table *opp_table;
- if (!list_empty(&opp_table->opp_list))
- return;
+ /* Hold our table modification lock here */
+ mutex_lock(&opp_table_lock);
- if (opp_table->supported_hw)
- return;
+ opp_table = _find_opp_table_unlocked(dev);
+ if (!IS_ERR(opp_table))
+ goto unlock;
- if (opp_table->prop_name)
- return;
+ opp_table = _allocate_opp_table(dev);
- if (opp_table->regulators)
- return;
+unlock:
+ mutex_unlock(&opp_table_lock);
- if (opp_table->set_opp)
- return;
+ return opp_table;
+}
+EXPORT_SYMBOL_GPL(dev_pm_opp_get_opp_table);
+
+static void _opp_table_kref_release(struct kref *kref)
+{
+ struct opp_table *opp_table = container_of(kref, struct opp_table, kref);
+ struct opp_device *opp_dev;
/* Release clk */
if (!IS_ERR(opp_table->clk))
/* dev_list must be empty now */
WARN_ON(!list_empty(&opp_table->dev_list));
- list_del_rcu(&opp_table->node);
- call_srcu(&opp_table->srcu_head.srcu, &opp_table->rcu_head,
- _kfree_device_rcu);
+ mutex_destroy(&opp_table->lock);
+ list_del(&opp_table->node);
+ kfree(opp_table);
+
+ mutex_unlock(&opp_table_lock);
}
-/**
- * _kfree_opp_rcu() - Free OPP RCU handler
- * @head: RCU head
- */
-static void _kfree_opp_rcu(struct rcu_head *head)
+void dev_pm_opp_put_opp_table(struct opp_table *opp_table)
{
- struct dev_pm_opp *opp = container_of(head, struct dev_pm_opp, rcu_head);
+ kref_put_mutex(&opp_table->kref, _opp_table_kref_release,
+ &opp_table_lock);
+}
+EXPORT_SYMBOL_GPL(dev_pm_opp_put_opp_table);
- kfree_rcu(opp, rcu_head);
+void _opp_free(struct dev_pm_opp *opp)
+{
+ kfree(opp);
}
-/**
- * _opp_remove() - Remove an OPP from a table definition
- * @opp_table: points back to the opp_table struct this opp belongs to
- * @opp: pointer to the OPP to remove
- * @notify: OPP_EVENT_REMOVE notification should be sent or not
- *
- * This function removes an opp definition from the opp table.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * It is assumed that the caller holds required mutex for an RCU updater
- * strategy.
- */
-void _opp_remove(struct opp_table *opp_table, struct dev_pm_opp *opp,
- bool notify)
+static void _opp_kref_release(struct kref *kref)
{
+ struct dev_pm_opp *opp = container_of(kref, struct dev_pm_opp, kref);
+ struct opp_table *opp_table = opp->opp_table;
+
/*
* Notify the changes in the availability of the operable
* frequency/voltage list.
*/
- if (notify)
- srcu_notifier_call_chain(&opp_table->srcu_head,
- OPP_EVENT_REMOVE, opp);
+ blocking_notifier_call_chain(&opp_table->head, OPP_EVENT_REMOVE, opp);
opp_debug_remove_one(opp);
- list_del_rcu(&opp->node);
- call_srcu(&opp_table->srcu_head.srcu, &opp->rcu_head, _kfree_opp_rcu);
+ list_del(&opp->node);
+ kfree(opp);
- _remove_opp_table(opp_table);
+ mutex_unlock(&opp_table->lock);
+ dev_pm_opp_put_opp_table(opp_table);
+}
+
+static void dev_pm_opp_get(struct dev_pm_opp *opp)
+{
+ kref_get(&opp->kref);
}
+void dev_pm_opp_put(struct dev_pm_opp *opp)
+{
+ kref_put_mutex(&opp->kref, _opp_kref_release, &opp->opp_table->lock);
+}
+EXPORT_SYMBOL_GPL(dev_pm_opp_put);
+
/**
* dev_pm_opp_remove() - Remove an OPP from OPP table
* @dev: device for which we do this operation
* @freq: OPP to remove with matching 'freq'
*
* This function removes an opp from the opp table.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
void dev_pm_opp_remove(struct device *dev, unsigned long freq)
{
struct opp_table *opp_table;
bool found = false;
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
-
opp_table = _find_opp_table(dev);
if (IS_ERR(opp_table))
- goto unlock;
+ return;
+
+ mutex_lock(&opp_table->lock);
list_for_each_entry(opp, &opp_table->opp_list, node) {
if (opp->rate == freq) {
}
}
- if (!found) {
+ mutex_unlock(&opp_table->lock);
+
+ if (found) {
+ dev_pm_opp_put(opp);
+ } else {
dev_warn(dev, "%s: Couldn't find OPP with freq: %lu\n",
__func__, freq);
- goto unlock;
}
- _opp_remove(opp_table, opp, true);
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_remove);
-struct dev_pm_opp *_allocate_opp(struct device *dev,
- struct opp_table **opp_table)
+struct dev_pm_opp *_opp_allocate(struct opp_table *table)
{
struct dev_pm_opp *opp;
int count, supply_size;
- struct opp_table *table;
-
- table = _add_opp_table(dev);
- if (!table)
- return NULL;
/* Allocate space for at least one supply */
count = table->regulator_count ? table->regulator_count : 1;
/* allocate new OPP node and supplies structures */
opp = kzalloc(sizeof(*opp) + supply_size, GFP_KERNEL);
- if (!opp) {
- kfree(table);
+ if (!opp)
return NULL;
- }
/* Put the supplies at the end of the OPP structure as an empty array */
opp->supplies = (struct dev_pm_opp_supply *)(opp + 1);
INIT_LIST_HEAD(&opp->node);
- *opp_table = table;
-
return opp;
}
return true;
}
+/*
+ * Returns:
+ * 0: On success. And appropriate error message for duplicate OPPs.
+ * -EBUSY: For OPP with same freq/volt and is available. The callers of
+ * _opp_add() must return 0 if they receive -EBUSY from it. This is to make
+ * sure we don't print error messages unnecessarily if different parts of
+ * kernel try to initialize the OPP table.
+ * -EEXIST: For OPP with same freq but different volt or is unavailable. This
+ * should be considered an error by the callers of _opp_add().
+ */
int _opp_add(struct device *dev, struct dev_pm_opp *new_opp,
struct opp_table *opp_table)
{
struct dev_pm_opp *opp;
- struct list_head *head = &opp_table->opp_list;
+ struct list_head *head;
int ret;
/*
* loop, don't replace it with head otherwise it will become an infinite
* loop.
*/
- list_for_each_entry_rcu(opp, &opp_table->opp_list, node) {
+ mutex_lock(&opp_table->lock);
+ head = &opp_table->opp_list;
+
+ list_for_each_entry(opp, &opp_table->opp_list, node) {
if (new_opp->rate > opp->rate) {
head = &opp->node;
continue;
new_opp->supplies[0].u_volt, new_opp->available);
/* Should we compare voltages for all regulators here ? */
- return opp->available &&
- new_opp->supplies[0].u_volt == opp->supplies[0].u_volt ? 0 : -EEXIST;
+ ret = opp->available &&
+ new_opp->supplies[0].u_volt == opp->supplies[0].u_volt ? -EBUSY : -EEXIST;
+
+ mutex_unlock(&opp_table->lock);
+ return ret;
}
+ list_add(&new_opp->node, head);
+ mutex_unlock(&opp_table->lock);
+
new_opp->opp_table = opp_table;
- list_add_rcu(&new_opp->node, head);
+ kref_init(&new_opp->kref);
+
+ /* Get a reference to the OPP table */
+ _get_opp_table_kref(opp_table);
ret = opp_debug_create_one(new_opp, opp_table);
if (ret)
/**
* _opp_add_v1() - Allocate a OPP based on v1 bindings.
+ * @opp_table: OPP table
* @dev: device for which we do this operation
* @freq: Frequency in Hz for this OPP
* @u_volt: Voltage in uVolts for this OPP
* NOTE: "dynamic" parameter impacts OPPs added by the dev_pm_opp_of_add_table
* and freed by dev_pm_opp_of_remove_table.
*
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
- *
* Return:
* 0 On success OR
* Duplicate OPPs (both freq and volt are same) and opp->available
* Duplicate OPPs (both freq and volt are same) and !opp->available
* -ENOMEM Memory allocation failure
*/
-int _opp_add_v1(struct device *dev, unsigned long freq, long u_volt,
- bool dynamic)
+int _opp_add_v1(struct opp_table *opp_table, struct device *dev,
+ unsigned long freq, long u_volt, bool dynamic)
{
- struct opp_table *opp_table;
struct dev_pm_opp *new_opp;
unsigned long tol;
int ret;
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
-
- new_opp = _allocate_opp(dev, &opp_table);
- if (!new_opp) {
- ret = -ENOMEM;
- goto unlock;
- }
+ new_opp = _opp_allocate(opp_table);
+ if (!new_opp)
+ return -ENOMEM;
/* populate the opp table */
new_opp->rate = freq;
new_opp->dynamic = dynamic;
ret = _opp_add(dev, new_opp, opp_table);
- if (ret)
+ if (ret) {
+ /* Don't return error for duplicate OPPs */
+ if (ret == -EBUSY)
+ ret = 0;
goto free_opp;
-
- mutex_unlock(&opp_table_lock);
+ }
/*
* Notify the changes in the availability of the operable
* frequency/voltage list.
*/
- srcu_notifier_call_chain(&opp_table->srcu_head, OPP_EVENT_ADD, new_opp);
+ blocking_notifier_call_chain(&opp_table->head, OPP_EVENT_ADD, new_opp);
return 0;
free_opp:
- _opp_remove(opp_table, new_opp, false);
-unlock:
- mutex_unlock(&opp_table_lock);
+ _opp_free(new_opp);
+
return ret;
}
* specify the hierarchy of versions it supports. OPP layer will then enable
* OPPs, which are available for those versions, based on its 'opp-supported-hw'
* property.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
-int dev_pm_opp_set_supported_hw(struct device *dev, const u32 *versions,
- unsigned int count)
+struct opp_table *dev_pm_opp_set_supported_hw(struct device *dev,
+ const u32 *versions, unsigned int count)
{
struct opp_table *opp_table;
- int ret = 0;
-
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
+ int ret;
- opp_table = _add_opp_table(dev);
- if (!opp_table) {
- ret = -ENOMEM;
- goto unlock;
- }
+ opp_table = dev_pm_opp_get_opp_table(dev);
+ if (!opp_table)
+ return ERR_PTR(-ENOMEM);
/* Make sure there are no concurrent readers while updating opp_table */
WARN_ON(!list_empty(&opp_table->opp_list));
}
opp_table->supported_hw_count = count;
- mutex_unlock(&opp_table_lock);
- return 0;
+
+ return opp_table;
err:
- _remove_opp_table(opp_table);
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
- return ret;
+ return ERR_PTR(ret);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_set_supported_hw);
/**
* dev_pm_opp_put_supported_hw() - Releases resources blocked for supported hw
- * @dev: Device for which supported-hw has to be put.
+ * @opp_table: OPP table returned by dev_pm_opp_set_supported_hw().
*
* This is required only for the V2 bindings, and is called for a matching
* dev_pm_opp_set_supported_hw(). Until this is called, the opp_table structure
* will not be freed.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
-void dev_pm_opp_put_supported_hw(struct device *dev)
+void dev_pm_opp_put_supported_hw(struct opp_table *opp_table)
{
- struct opp_table *opp_table;
-
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
-
- /* Check for existing table for 'dev' first */
- opp_table = _find_opp_table(dev);
- if (IS_ERR(opp_table)) {
- dev_err(dev, "Failed to find opp_table: %ld\n",
- PTR_ERR(opp_table));
- goto unlock;
- }
-
/* Make sure there are no concurrent readers while updating opp_table */
WARN_ON(!list_empty(&opp_table->opp_list));
if (!opp_table->supported_hw) {
- dev_err(dev, "%s: Doesn't have supported hardware list\n",
- __func__);
- goto unlock;
+ pr_err("%s: Doesn't have supported hardware list\n",
+ __func__);
+ return;
}
kfree(opp_table->supported_hw);
opp_table->supported_hw = NULL;
opp_table->supported_hw_count = 0;
- /* Try freeing opp_table if this was the last blocking resource */
- _remove_opp_table(opp_table);
-
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_put_supported_hw);
* specify the extn to be used for certain property names. The properties to
* which the extension will apply are opp-microvolt and opp-microamp. OPP core
* should postfix the property name with -<name> while looking for them.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
-int dev_pm_opp_set_prop_name(struct device *dev, const char *name)
+struct opp_table *dev_pm_opp_set_prop_name(struct device *dev, const char *name)
{
struct opp_table *opp_table;
- int ret = 0;
-
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
+ int ret;
- opp_table = _add_opp_table(dev);
- if (!opp_table) {
- ret = -ENOMEM;
- goto unlock;
- }
+ opp_table = dev_pm_opp_get_opp_table(dev);
+ if (!opp_table)
+ return ERR_PTR(-ENOMEM);
/* Make sure there are no concurrent readers while updating opp_table */
WARN_ON(!list_empty(&opp_table->opp_list));
goto err;
}
- mutex_unlock(&opp_table_lock);
- return 0;
+ return opp_table;
err:
- _remove_opp_table(opp_table);
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
- return ret;
+ return ERR_PTR(ret);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_set_prop_name);
/**
* dev_pm_opp_put_prop_name() - Releases resources blocked for prop-name
- * @dev: Device for which the prop-name has to be put.
+ * @opp_table: OPP table returned by dev_pm_opp_set_prop_name().
*
* This is required only for the V2 bindings, and is called for a matching
* dev_pm_opp_set_prop_name(). Until this is called, the opp_table structure
* will not be freed.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
-void dev_pm_opp_put_prop_name(struct device *dev)
+void dev_pm_opp_put_prop_name(struct opp_table *opp_table)
{
- struct opp_table *opp_table;
-
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
-
- /* Check for existing table for 'dev' first */
- opp_table = _find_opp_table(dev);
- if (IS_ERR(opp_table)) {
- dev_err(dev, "Failed to find opp_table: %ld\n",
- PTR_ERR(opp_table));
- goto unlock;
- }
-
/* Make sure there are no concurrent readers while updating opp_table */
WARN_ON(!list_empty(&opp_table->opp_list));
if (!opp_table->prop_name) {
- dev_err(dev, "%s: Doesn't have a prop-name\n", __func__);
- goto unlock;
+ pr_err("%s: Doesn't have a prop-name\n", __func__);
+ return;
}
kfree(opp_table->prop_name);
opp_table->prop_name = NULL;
- /* Try freeing opp_table if this was the last blocking resource */
- _remove_opp_table(opp_table);
-
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_put_prop_name);
* well.
*
* This must be called before any OPPs are initialized for the device.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
struct opp_table *dev_pm_opp_set_regulators(struct device *dev,
const char * const names[],
struct regulator *reg;
int ret, i;
- mutex_lock(&opp_table_lock);
-
- opp_table = _add_opp_table(dev);
- if (!opp_table) {
- ret = -ENOMEM;
- goto unlock;
- }
+ opp_table = dev_pm_opp_get_opp_table(dev);
+ if (!opp_table)
+ return ERR_PTR(-ENOMEM);
/* This should be called before OPPs are initialized */
if (WARN_ON(!list_empty(&opp_table->opp_list))) {
if (ret)
goto free_regulators;
- mutex_unlock(&opp_table_lock);
return opp_table;
free_regulators:
opp_table->regulators = NULL;
opp_table->regulator_count = 0;
err:
- _remove_opp_table(opp_table);
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
return ERR_PTR(ret);
}
/**
* dev_pm_opp_put_regulators() - Releases resources blocked for regulator
* @opp_table: OPP table returned from dev_pm_opp_set_regulators().
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
void dev_pm_opp_put_regulators(struct opp_table *opp_table)
{
int i;
- mutex_lock(&opp_table_lock);
-
if (!opp_table->regulators) {
pr_err("%s: Doesn't have regulators set\n", __func__);
- goto unlock;
+ return;
}
/* Make sure there are no concurrent readers while updating opp_table */
opp_table->regulators = NULL;
opp_table->regulator_count = 0;
- /* Try freeing opp_table if this was the last blocking resource */
- _remove_opp_table(opp_table);
-
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_put_regulators);
* regulators per device), instead of the generic OPP set rate helper.
*
* This must be called before any OPPs are initialized for the device.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
-int dev_pm_opp_register_set_opp_helper(struct device *dev,
+struct opp_table *dev_pm_opp_register_set_opp_helper(struct device *dev,
int (*set_opp)(struct dev_pm_set_opp_data *data))
{
struct opp_table *opp_table;
int ret;
if (!set_opp)
- return -EINVAL;
-
- mutex_lock(&opp_table_lock);
+ return ERR_PTR(-EINVAL);
- opp_table = _add_opp_table(dev);
- if (!opp_table) {
- ret = -ENOMEM;
- goto unlock;
- }
+ opp_table = dev_pm_opp_get_opp_table(dev);
+ if (!opp_table)
+ return ERR_PTR(-ENOMEM);
/* This should be called before OPPs are initialized */
if (WARN_ON(!list_empty(&opp_table->opp_list))) {
opp_table->set_opp = set_opp;
- mutex_unlock(&opp_table_lock);
- return 0;
+ return opp_table;
err:
- _remove_opp_table(opp_table);
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
- return ret;
+ return ERR_PTR(ret);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_register_set_opp_helper);
/**
* dev_pm_opp_register_put_opp_helper() - Releases resources blocked for
* set_opp helper
- * @dev: Device for which custom set_opp helper has to be cleared.
+ * @opp_table: OPP table returned from dev_pm_opp_register_set_opp_helper().
*
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
+ * Release resources blocked for platform specific set_opp helper.
*/
-void dev_pm_opp_register_put_opp_helper(struct device *dev)
+void dev_pm_opp_register_put_opp_helper(struct opp_table *opp_table)
{
- struct opp_table *opp_table;
-
- mutex_lock(&opp_table_lock);
-
- /* Check for existing table for 'dev' first */
- opp_table = _find_opp_table(dev);
- if (IS_ERR(opp_table)) {
- dev_err(dev, "Failed to find opp_table: %ld\n",
- PTR_ERR(opp_table));
- goto unlock;
- }
-
if (!opp_table->set_opp) {
- dev_err(dev, "%s: Doesn't have custom set_opp helper set\n",
- __func__);
- goto unlock;
+ pr_err("%s: Doesn't have custom set_opp helper set\n",
+ __func__);
+ return;
}
/* Make sure there are no concurrent readers while updating opp_table */
opp_table->set_opp = NULL;
- /* Try freeing opp_table if this was the last blocking resource */
- _remove_opp_table(opp_table);
-
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_register_put_opp_helper);
* The opp is made available by default and it can be controlled using
* dev_pm_opp_enable/disable functions.
*
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
- *
* Return:
* 0 On success OR
* Duplicate OPPs (both freq and volt are same) and opp->available
*/
int dev_pm_opp_add(struct device *dev, unsigned long freq, unsigned long u_volt)
{
- return _opp_add_v1(dev, freq, u_volt, true);
+ struct opp_table *opp_table;
+ int ret;
+
+ opp_table = dev_pm_opp_get_opp_table(dev);
+ if (!opp_table)
+ return -ENOMEM;
+
+ ret = _opp_add_v1(opp_table, dev, freq, u_volt, true);
+
+ dev_pm_opp_put_opp_table(opp_table);
+ return ret;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_add);
* @freq: OPP frequency to modify availability
* @availability_req: availability status requested for this opp
*
- * Set the availability of an OPP with an RCU operation, opp_{enable,disable}
- * share a common logic which is isolated here.
+ * Set the availability of an OPP, opp_{enable,disable} share a common logic
+ * which is isolated here.
*
* Return: -EINVAL for bad pointers, -ENOMEM if no memory available for the
* copy operation, returns 0 if no modification was done OR modification was
* successful.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks to
- * keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex locking or synchronize_rcu() blocking calls cannot be used.
*/
static int _opp_set_availability(struct device *dev, unsigned long freq,
bool availability_req)
{
struct opp_table *opp_table;
- struct dev_pm_opp *new_opp, *tmp_opp, *opp = ERR_PTR(-ENODEV);
+ struct dev_pm_opp *tmp_opp, *opp = ERR_PTR(-ENODEV);
int r = 0;
- /* keep the node allocated */
- new_opp = kmalloc(sizeof(*new_opp), GFP_KERNEL);
- if (!new_opp)
- return -ENOMEM;
-
- mutex_lock(&opp_table_lock);
-
/* Find the opp_table */
opp_table = _find_opp_table(dev);
if (IS_ERR(opp_table)) {
r = PTR_ERR(opp_table);
dev_warn(dev, "%s: Device OPP not found (%d)\n", __func__, r);
- goto unlock;
+ return r;
}
+ mutex_lock(&opp_table->lock);
+
/* Do we have the frequency? */
list_for_each_entry(tmp_opp, &opp_table->opp_list, node) {
if (tmp_opp->rate == freq) {
break;
}
}
+
if (IS_ERR(opp)) {
r = PTR_ERR(opp);
goto unlock;
/* Is update really needed? */
if (opp->available == availability_req)
goto unlock;
- /* copy the old data over */
- *new_opp = *opp;
- /* plug in new node */
- new_opp->available = availability_req;
-
- list_replace_rcu(&opp->node, &new_opp->node);
- mutex_unlock(&opp_table_lock);
- call_srcu(&opp_table->srcu_head.srcu, &opp->rcu_head, _kfree_opp_rcu);
+ opp->available = availability_req;
/* Notify the change of the OPP availability */
if (availability_req)
- srcu_notifier_call_chain(&opp_table->srcu_head,
- OPP_EVENT_ENABLE, new_opp);
+ blocking_notifier_call_chain(&opp_table->head, OPP_EVENT_ENABLE,
+ opp);
else
- srcu_notifier_call_chain(&opp_table->srcu_head,
- OPP_EVENT_DISABLE, new_opp);
-
- return 0;
+ blocking_notifier_call_chain(&opp_table->head,
+ OPP_EVENT_DISABLE, opp);
unlock:
- mutex_unlock(&opp_table_lock);
- kfree(new_opp);
+ mutex_unlock(&opp_table->lock);
+ dev_pm_opp_put_opp_table(opp_table);
return r;
}
* corresponding error value. It is meant to be used for users an OPP available
* after being temporarily made unavailable with dev_pm_opp_disable.
*
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function indirectly uses RCU and mutex locks to keep the
- * integrity of the internal data structures. Callers should ensure that
- * this function is *NOT* called under RCU protection or in contexts where
- * mutex locking or synchronize_rcu() blocking calls cannot be used.
- *
* Return: -EINVAL for bad pointers, -ENOMEM if no memory available for the
* copy operation, returns 0 if no modification was done OR modification was
* successful.
* control by users to make this OPP not available until the circumstances are
* right to make it available again (with a call to dev_pm_opp_enable).
*
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function indirectly uses RCU and mutex locks to keep the
- * integrity of the internal data structures. Callers should ensure that
- * this function is *NOT* called under RCU protection or in contexts where
- * mutex locking or synchronize_rcu() blocking calls cannot be used.
- *
* Return: -EINVAL for bad pointers, -ENOMEM if no memory available for the
* copy operation, returns 0 if no modification was done OR modification was
* successful.
EXPORT_SYMBOL_GPL(dev_pm_opp_disable);
/**
- * dev_pm_opp_get_notifier() - find notifier_head of the device with opp
- * @dev: device pointer used to lookup OPP table.
+ * dev_pm_opp_register_notifier() - Register OPP notifier for the device
+ * @dev: Device for which notifier needs to be registered
+ * @nb: Notifier block to be registered
*
- * Return: pointer to notifier head if found, otherwise -ENODEV or
- * -EINVAL based on type of error casted as pointer. value must be checked
- * with IS_ERR to determine valid pointer or error result.
+ * Return: 0 on success or a negative error value.
+ */
+int dev_pm_opp_register_notifier(struct device *dev, struct notifier_block *nb)
+{
+ struct opp_table *opp_table;
+ int ret;
+
+ opp_table = _find_opp_table(dev);
+ if (IS_ERR(opp_table))
+ return PTR_ERR(opp_table);
+
+ ret = blocking_notifier_chain_register(&opp_table->head, nb);
+
+ dev_pm_opp_put_opp_table(opp_table);
+
+ return ret;
+}
+EXPORT_SYMBOL(dev_pm_opp_register_notifier);
+
+/**
+ * dev_pm_opp_unregister_notifier() - Unregister OPP notifier for the device
+ * @dev: Device for which notifier needs to be unregistered
+ * @nb: Notifier block to be unregistered
*
- * Locking: This function must be called under rcu_read_lock(). opp_table is a
- * RCU protected pointer. The reason for the same is that the opp pointer which
- * is returned will remain valid for use with opp_get_{voltage, freq} only while
- * under the locked area. The pointer returned must be used prior to unlocking
- * with rcu_read_unlock() to maintain the integrity of the pointer.
+ * Return: 0 on success or a negative error value.
*/
-struct srcu_notifier_head *dev_pm_opp_get_notifier(struct device *dev)
+int dev_pm_opp_unregister_notifier(struct device *dev,
+ struct notifier_block *nb)
{
- struct opp_table *opp_table = _find_opp_table(dev);
+ struct opp_table *opp_table;
+ int ret;
+ opp_table = _find_opp_table(dev);
if (IS_ERR(opp_table))
- return ERR_CAST(opp_table); /* matching type */
+ return PTR_ERR(opp_table);
+
+ ret = blocking_notifier_chain_unregister(&opp_table->head, nb);
- return &opp_table->srcu_head;
+ dev_pm_opp_put_opp_table(opp_table);
+
+ return ret;
}
-EXPORT_SYMBOL_GPL(dev_pm_opp_get_notifier);
+EXPORT_SYMBOL(dev_pm_opp_unregister_notifier);
/*
* Free OPPs either created using static entries present in DT or even the
* dynamically added entries based on remove_all param.
*/
-void _dev_pm_opp_remove_table(struct device *dev, bool remove_all)
+void _dev_pm_opp_remove_table(struct opp_table *opp_table, struct device *dev,
+ bool remove_all)
{
- struct opp_table *opp_table;
struct dev_pm_opp *opp, *tmp;
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
+ /* Find if opp_table manages a single device */
+ if (list_is_singular(&opp_table->dev_list)) {
+ /* Free static OPPs */
+ list_for_each_entry_safe(opp, tmp, &opp_table->opp_list, node) {
+ if (remove_all || !opp->dynamic)
+ dev_pm_opp_put(opp);
+ }
+ } else {
+ _remove_opp_dev(_find_opp_dev(dev, opp_table), opp_table);
+ }
+}
+
+void _dev_pm_opp_find_and_remove_table(struct device *dev, bool remove_all)
+{
+ struct opp_table *opp_table;
/* Check for existing table for 'dev' */
opp_table = _find_opp_table(dev);
IS_ERR_OR_NULL(dev) ?
"Invalid device" : dev_name(dev),
error);
- goto unlock;
+ return;
}
- /* Find if opp_table manages a single device */
- if (list_is_singular(&opp_table->dev_list)) {
- /* Free static OPPs */
- list_for_each_entry_safe(opp, tmp, &opp_table->opp_list, node) {
- if (remove_all || !opp->dynamic)
- _opp_remove(opp_table, opp, true);
- }
- } else {
- _remove_opp_dev(_find_opp_dev(dev, opp_table), opp_table);
- }
+ _dev_pm_opp_remove_table(opp_table, dev, remove_all);
-unlock:
- mutex_unlock(&opp_table_lock);
+ dev_pm_opp_put_opp_table(opp_table);
}
/**
*
* Free both OPPs created using static entries present in DT and the
* dynamically added entries.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function indirectly uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
void dev_pm_opp_remove_table(struct device *dev)
{
- _dev_pm_opp_remove_table(dev, true);
+ _dev_pm_opp_find_and_remove_table(dev, true);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_remove_table);
*
* WARNING: It is important for the callers to ensure refreshing their copy of
* the table if any of the mentioned functions have been invoked in the interim.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Since we just use the regular accessor functions to access the internal data
- * structures, we use RCU read lock inside this function. As a result, users of
- * this function DONOT need to use explicit locks for invoking.
*/
int dev_pm_opp_init_cpufreq_table(struct device *dev,
struct cpufreq_frequency_table **table)
int i, max_opps, ret = 0;
unsigned long rate;
- rcu_read_lock();
-
max_opps = dev_pm_opp_get_opp_count(dev);
- if (max_opps <= 0) {
- ret = max_opps ? max_opps : -ENODATA;
- goto out;
- }
+ if (max_opps <= 0)
+ return max_opps ? max_opps : -ENODATA;
freq_table = kcalloc((max_opps + 1), sizeof(*freq_table), GFP_ATOMIC);
- if (!freq_table) {
- ret = -ENOMEM;
- goto out;
- }
+ if (!freq_table)
+ return -ENOMEM;
for (i = 0, rate = 0; i < max_opps; i++, rate++) {
/* find next rate */
/* Is Boost/turbo opp ? */
if (dev_pm_opp_is_turbo(opp))
freq_table[i].flags = CPUFREQ_BOOST_FREQ;
+
+ dev_pm_opp_put(opp);
}
freq_table[i].driver_data = i;
*table = &freq_table[0];
out:
- rcu_read_unlock();
if (ret)
kfree(freq_table);
* This removes the OPP tables for CPUs present in the @cpumask.
* This should be used to remove all the OPPs entries associated with
* the cpus in @cpumask.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
void dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask)
{
* @cpumask.
*
* Returns -ENODEV if OPP table isn't already present.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
int dev_pm_opp_set_sharing_cpus(struct device *cpu_dev,
const struct cpumask *cpumask)
struct device *dev;
int cpu, ret = 0;
- mutex_lock(&opp_table_lock);
-
opp_table = _find_opp_table(cpu_dev);
- if (IS_ERR(opp_table)) {
- ret = PTR_ERR(opp_table);
- goto unlock;
- }
+ if (IS_ERR(opp_table))
+ return PTR_ERR(opp_table);
for_each_cpu(cpu, cpumask) {
if (cpu == cpu_dev->id)
/* Mark opp-table as multiple CPUs are sharing it now */
opp_table->shared_opp = OPP_TABLE_ACCESS_SHARED;
}
-unlock:
- mutex_unlock(&opp_table_lock);
+
+ dev_pm_opp_put_opp_table(opp_table);
return ret;
}
*
* Returns -ENODEV if OPP table isn't already present and -EINVAL if the OPP
* table's status is access-unknown.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
int dev_pm_opp_get_sharing_cpus(struct device *cpu_dev, struct cpumask *cpumask)
{
struct opp_table *opp_table;
int ret = 0;
- mutex_lock(&opp_table_lock);
-
opp_table = _find_opp_table(cpu_dev);
- if (IS_ERR(opp_table)) {
- ret = PTR_ERR(opp_table);
- goto unlock;
- }
+ if (IS_ERR(opp_table))
+ return PTR_ERR(opp_table);
if (opp_table->shared_opp == OPP_TABLE_ACCESS_UNKNOWN) {
ret = -EINVAL;
- goto unlock;
+ goto put_opp_table;
}
cpumask_clear(cpumask);
cpumask_set_cpu(cpu_dev->id, cpumask);
}
-unlock:
- mutex_unlock(&opp_table_lock);
+put_opp_table:
+ dev_pm_opp_put_opp_table(opp_table);
return ret;
}
static struct opp_table *_managed_opp(const struct device_node *np)
{
- struct opp_table *opp_table;
+ struct opp_table *opp_table, *managed_table = NULL;
+
+ mutex_lock(&opp_table_lock);
- list_for_each_entry_rcu(opp_table, &opp_tables, node) {
+ list_for_each_entry(opp_table, &opp_tables, node) {
if (opp_table->np == np) {
/*
* Multiple devices can point to the same OPP table and
* But the OPPs will be considered as shared only if the
* OPP table contains a "opp-shared" property.
*/
- if (opp_table->shared_opp == OPP_TABLE_ACCESS_SHARED)
- return opp_table;
+ if (opp_table->shared_opp == OPP_TABLE_ACCESS_SHARED) {
+ _get_opp_table_kref(opp_table);
+ managed_table = opp_table;
+ }
- return NULL;
+ break;
}
}
- return NULL;
+ mutex_unlock(&opp_table_lock);
+
+ return managed_table;
}
void _of_init_opp_table(struct opp_table *opp_table, struct device *dev)
* @dev: device pointer used to lookup OPP table.
*
* Free OPPs created using static entries present in DT.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function indirectly uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
void dev_pm_opp_of_remove_table(struct device *dev)
{
- _dev_pm_opp_remove_table(dev, false);
+ _dev_pm_opp_find_and_remove_table(dev, false);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_of_remove_table);
/* Returns opp descriptor node for a device, caller must do of_node_put() */
-static struct device_node *_of_get_opp_desc_node(struct device *dev)
+struct device_node *dev_pm_opp_of_get_opp_desc_node(struct device *dev)
{
/*
- * TODO: Support for multiple OPP tables.
- *
* There should be only ONE phandle present in "operating-points-v2"
* property.
*/
return of_parse_phandle(dev->of_node, "operating-points-v2", 0);
}
+EXPORT_SYMBOL_GPL(dev_pm_opp_of_get_opp_desc_node);
/**
* _opp_add_static_v2() - Allocate static OPPs (As per 'v2' DT bindings)
+ * @opp_table: OPP table
* @dev: device for which we do this operation
* @np: device node
*
* opp can be controlled using dev_pm_opp_enable/disable functions and may be
* removed by dev_pm_opp_remove.
*
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
- *
* Return:
* 0 On success OR
* Duplicate OPPs (both freq and volt are same) and opp->available
* -ENOMEM Memory allocation failure
* -EINVAL Failed parsing the OPP node
*/
-static int _opp_add_static_v2(struct device *dev, struct device_node *np)
+static int _opp_add_static_v2(struct opp_table *opp_table, struct device *dev,
+ struct device_node *np)
{
- struct opp_table *opp_table;
struct dev_pm_opp *new_opp;
u64 rate;
u32 val;
int ret;
- /* Hold our table modification lock here */
- mutex_lock(&opp_table_lock);
-
- new_opp = _allocate_opp(dev, &opp_table);
- if (!new_opp) {
- ret = -ENOMEM;
- goto unlock;
- }
+ new_opp = _opp_allocate(opp_table);
+ if (!new_opp)
+ return -ENOMEM;
ret = of_property_read_u64(np, "opp-hz", &rate);
if (ret < 0) {
goto free_opp;
ret = _opp_add(dev, new_opp, opp_table);
- if (ret)
+ if (ret) {
+ /* Don't return error for duplicate OPPs */
+ if (ret == -EBUSY)
+ ret = 0;
goto free_opp;
+ }
/* OPP to select on device suspend */
if (of_property_read_bool(np, "opp-suspend")) {
if (new_opp->clock_latency_ns > opp_table->clock_latency_ns_max)
opp_table->clock_latency_ns_max = new_opp->clock_latency_ns;
- mutex_unlock(&opp_table_lock);
-
pr_debug("%s: turbo:%d rate:%lu uv:%lu uvmin:%lu uvmax:%lu latency:%lu\n",
__func__, new_opp->turbo, new_opp->rate,
new_opp->supplies[0].u_volt, new_opp->supplies[0].u_volt_min,
* Notify the changes in the availability of the operable
* frequency/voltage list.
*/
- srcu_notifier_call_chain(&opp_table->srcu_head, OPP_EVENT_ADD, new_opp);
+ blocking_notifier_call_chain(&opp_table->head, OPP_EVENT_ADD, new_opp);
return 0;
free_opp:
- _opp_remove(opp_table, new_opp, false);
-unlock:
- mutex_unlock(&opp_table_lock);
+ _opp_free(new_opp);
+
return ret;
}
struct opp_table *opp_table;
int ret = 0, count = 0;
- mutex_lock(&opp_table_lock);
-
opp_table = _managed_opp(opp_np);
if (opp_table) {
/* OPPs are already managed */
if (!_add_opp_dev(dev, opp_table))
ret = -ENOMEM;
- mutex_unlock(&opp_table_lock);
- return ret;
+ goto put_opp_table;
}
- mutex_unlock(&opp_table_lock);
+
+ opp_table = dev_pm_opp_get_opp_table(dev);
+ if (!opp_table)
+ return -ENOMEM;
/* We have opp-table node now, iterate over it and add OPPs */
for_each_available_child_of_node(opp_np, np) {
count++;
- ret = _opp_add_static_v2(dev, np);
+ ret = _opp_add_static_v2(opp_table, dev, np);
if (ret) {
dev_err(dev, "%s: Failed to add OPP, %d\n", __func__,
ret);
- goto free_table;
+ _dev_pm_opp_remove_table(opp_table, dev, false);
+ goto put_opp_table;
}
}
/* There should be one of more OPP defined */
- if (WARN_ON(!count))
- return -ENOENT;
-
- mutex_lock(&opp_table_lock);
-
- opp_table = _find_opp_table(dev);
- if (WARN_ON(IS_ERR(opp_table))) {
- ret = PTR_ERR(opp_table);
- mutex_unlock(&opp_table_lock);
- goto free_table;
+ if (WARN_ON(!count)) {
+ ret = -ENOENT;
+ goto put_opp_table;
}
opp_table->np = opp_np;
else
opp_table->shared_opp = OPP_TABLE_ACCESS_EXCLUSIVE;
- mutex_unlock(&opp_table_lock);
-
- return 0;
-
-free_table:
- dev_pm_opp_of_remove_table(dev);
+put_opp_table:
+ dev_pm_opp_put_opp_table(opp_table);
return ret;
}
/* Initializes OPP tables based on old-deprecated bindings */
static int _of_add_opp_table_v1(struct device *dev)
{
+ struct opp_table *opp_table;
const struct property *prop;
const __be32 *val;
- int nr;
+ int nr, ret = 0;
prop = of_find_property(dev->of_node, "operating-points", NULL);
if (!prop)
return -EINVAL;
}
+ opp_table = dev_pm_opp_get_opp_table(dev);
+ if (!opp_table)
+ return -ENOMEM;
+
val = prop->value;
while (nr) {
unsigned long freq = be32_to_cpup(val++) * 1000;
unsigned long volt = be32_to_cpup(val++);
- if (_opp_add_v1(dev, freq, volt, false))
- dev_warn(dev, "%s: Failed to add OPP %ld\n",
- __func__, freq);
+ ret = _opp_add_v1(opp_table, dev, freq, volt, false);
+ if (ret) {
+ dev_err(dev, "%s: Failed to add OPP %ld (%d)\n",
+ __func__, freq, ret);
+ _dev_pm_opp_remove_table(opp_table, dev, false);
+ break;
+ }
nr -= 2;
}
- return 0;
+ dev_pm_opp_put_opp_table(opp_table);
+ return ret;
}
/**
*
* Register the initial OPP table with the OPP library for given device.
*
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function indirectly uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
- *
* Return:
* 0 On success OR
* Duplicate OPPs (both freq and volt are same) and opp->available
* OPPs have two version of bindings now. The older one is deprecated,
* try for the new binding first.
*/
- opp_np = _of_get_opp_desc_node(dev);
+ opp_np = dev_pm_opp_of_get_opp_desc_node(dev);
if (!opp_np) {
/*
* Try old-deprecated bindings for backward compatibility with
*
* This removes the OPP tables for CPUs present in the @cpumask.
* This should be used only to remove static entries created from DT.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
void dev_pm_opp_of_cpumask_remove_table(const struct cpumask *cpumask)
{
* @cpumask: cpumask for which OPP table needs to be added.
*
* This adds the OPP tables for CPUs present in the @cpumask.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
int dev_pm_opp_of_cpumask_add_table(const struct cpumask *cpumask)
{
* This updates the @cpumask with CPUs that are sharing OPPs with @cpu_dev.
*
* Returns -ENOENT if operating-points-v2 isn't present for @cpu_dev.
- *
- * Locking: The internal opp_table and opp structures are RCU protected.
- * Hence this function internally uses RCU updater strategy with mutex locks
- * to keep the integrity of the internal data structures. Callers should ensure
- * that this function is *NOT* called under RCU protection or in contexts where
- * mutex cannot be locked.
*/
int dev_pm_opp_of_get_sharing_cpus(struct device *cpu_dev,
struct cpumask *cpumask)
int cpu, ret = 0;
/* Get OPP descriptor node */
- np = _of_get_opp_desc_node(cpu_dev);
+ np = dev_pm_opp_of_get_opp_desc_node(cpu_dev);
if (!np) {
dev_dbg(cpu_dev, "%s: Couldn't find opp node.\n", __func__);
return -ENOENT;
}
/* Get OPP descriptor node */
- tmp_np = _of_get_opp_desc_node(tcpu_dev);
+ tmp_np = dev_pm_opp_of_get_opp_desc_node(tcpu_dev);
if (!tmp_np) {
dev_err(tcpu_dev, "%s: Couldn't find opp node.\n",
__func__);
#include <linux/device.h>
#include <linux/kernel.h>
+#include <linux/kref.h>
#include <linux/list.h>
#include <linux/limits.h>
#include <linux/pm_opp.h>
-#include <linux/rculist.h>
-#include <linux/rcupdate.h>
+#include <linux/notifier.h>
struct clk;
struct regulator;
* @node: opp table node. The nodes are maintained throughout the lifetime
* of boot. It is expected only an optimal set of OPPs are
* added to the library by the SoC framework.
- * RCU usage: opp table is traversed with RCU locks. node
- * modification is possible realtime, hence the modifications
- * are protected by the opp_table_lock for integrity.
* IMPORTANT: the opp nodes should be maintained in increasing
* order.
+ * @kref: for reference count of the OPP.
* @available: true/false - marks if this OPP as available or not
* @dynamic: not-created from static DT entries.
* @turbo: true if turbo (boost) OPP
* @clock_latency_ns: Latency (in nanoseconds) of switching to this OPP's
* frequency from any other OPP's frequency.
* @opp_table: points back to the opp_table struct this opp belongs to
- * @rcu_head: RCU callback head used for deferred freeing
* @np: OPP's device node.
* @dentry: debugfs dentry pointer (per opp)
*
*/
struct dev_pm_opp {
struct list_head node;
+ struct kref kref;
bool available;
bool dynamic;
unsigned long clock_latency_ns;
struct opp_table *opp_table;
- struct rcu_head rcu_head;
struct device_node *np;
* struct opp_device - devices managed by 'struct opp_table'
* @node: list node
* @dev: device to which the struct object belongs
- * @rcu_head: RCU callback head used for deferred freeing
* @dentry: debugfs dentry pointer (per device)
*
* This is an internal data structure maintaining the devices that are managed
struct opp_device {
struct list_head node;
const struct device *dev;
- struct rcu_head rcu_head;
#ifdef CONFIG_DEBUG_FS
struct dentry *dentry;
* @node: table node - contains the devices with OPPs that
* have been registered. Nodes once added are not modified in this
* table.
- * RCU usage: nodes are not modified in the table of opp_table,
- * however addition is possible and is secured by opp_table_lock
- * @srcu_head: notifier head to notify the OPP availability changes.
- * @rcu_head: RCU callback head used for deferred freeing
+ * @head: notifier head to notify the OPP availability changes.
* @dev_list: list of devices that share these OPPs
* @opp_list: table of opps
+ * @kref: for reference count of the table.
+ * @lock: mutex protecting the opp_list.
* @np: struct device_node pointer for opp's DT node.
* @clock_latency_ns_max: Max clock latency in nanoseconds.
* @shared_opp: OPP is shared between multiple devices.
* This is an internal data structure maintaining the link to opps attached to
* a device. This structure is not meant to be shared to users as it is
* meant for book keeping and private to OPP library.
- *
- * Because the opp structures can be used from both rcu and srcu readers, we
- * need to wait for the grace period of both of them before freeing any
- * resources. And so we have used kfree_rcu() from within call_srcu() handlers.
*/
struct opp_table {
struct list_head node;
- struct srcu_notifier_head srcu_head;
- struct rcu_head rcu_head;
+ struct blocking_notifier_head head;
struct list_head dev_list;
struct list_head opp_list;
+ struct kref kref;
+ struct mutex lock;
struct device_node *np;
unsigned long clock_latency_ns_max;
};
/* Routines internal to opp core */
+void _get_opp_table_kref(struct opp_table *opp_table);
struct opp_table *_find_opp_table(struct device *dev);
struct opp_device *_add_opp_dev(const struct device *dev, struct opp_table *opp_table);
-void _dev_pm_opp_remove_table(struct device *dev, bool remove_all);
-struct dev_pm_opp *_allocate_opp(struct device *dev, struct opp_table **opp_table);
+void _dev_pm_opp_remove_table(struct opp_table *opp_table, struct device *dev, bool remove_all);
+void _dev_pm_opp_find_and_remove_table(struct device *dev, bool remove_all);
+struct dev_pm_opp *_opp_allocate(struct opp_table *opp_table);
+void _opp_free(struct dev_pm_opp *opp);
int _opp_add(struct device *dev, struct dev_pm_opp *new_opp, struct opp_table *opp_table);
-void _opp_remove(struct opp_table *opp_table, struct dev_pm_opp *opp, bool notify);
-int _opp_add_v1(struct device *dev, unsigned long freq, long u_volt, bool dynamic);
+int _opp_add_v1(struct opp_table *opp_table, struct device *dev, unsigned long freq, long u_volt, bool dynamic);
void _dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask, bool of);
+struct opp_table *_add_opp_table(struct device *dev);
#ifdef CONFIG_OF
void _of_init_opp_table(struct opp_table *opp_table, struct device *dev);
unsigned long flags;
int retval;
- might_sleep_if(!(rpmflags & RPM_ASYNC) && !dev->power.irq_safe);
-
if (rpmflags & RPM_GET_PUT) {
if (!atomic_dec_and_test(&dev->power.usage_count))
return 0;
}
+ might_sleep_if(!(rpmflags & RPM_ASYNC) && !dev->power.irq_safe);
+
spin_lock_irqsave(&dev->power.lock, flags);
retval = rpm_idle(dev, rpmflags);
spin_unlock_irqrestore(&dev->power.lock, flags);
unsigned long flags;
int retval;
- might_sleep_if(!(rpmflags & RPM_ASYNC) && !dev->power.irq_safe);
-
if (rpmflags & RPM_GET_PUT) {
if (!atomic_dec_and_test(&dev->power.usage_count))
return 0;
}
+ might_sleep_if(!(rpmflags & RPM_ASYNC) && !dev->power.irq_safe);
+
spin_lock_irqsave(&dev->power.lock, flags);
retval = rpm_suspend(dev, rpmflags);
spin_unlock_irqrestore(&dev->power.lock, flags);
unsigned long flags;
int retval;
- might_sleep_if(!(rpmflags & RPM_ASYNC) && !dev->power.irq_safe);
+ might_sleep_if(!(rpmflags & RPM_ASYNC) && !dev->power.irq_safe &&
+ dev->power.runtime_status != RPM_ACTIVE);
if (rpmflags & RPM_GET_PUT)
atomic_inc(&dev->power.usage_count);
struct wake_irq *wirq = _wirq;
int res;
+ /* Maybe abort suspend? */
+ if (irqd_is_wakeup_set(irq_get_irq_data(irq))) {
+ pm_wakeup_event(wirq->dev, 0);
+
+ return IRQ_HANDLED;
+ }
+
/* We don't want RPM_ASYNC or RPM_NOWAIT here */
res = pm_runtime_resume(wirq->dev);
if (res < 0)
wirq->irq = irq;
irq_set_status_flags(irq, IRQ_NOAUTOEN);
+ /* Prevent deferred spurious wakeirqs with disable_irq_nosync() */
+ irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
+
/*
* Consumer device may need to power up and restore state
* so we use a threaded irq.
if (!wirq)
return;
- if (device_may_wakeup(wirq->dev))
+ if (device_may_wakeup(wirq->dev)) {
+ if (wirq->status & WAKE_IRQ_DEDICATED_ALLOCATED)
+ enable_irq(wirq->irq);
+
enable_irq_wake(wirq->irq);
+ }
}
/**
if (!wirq)
return;
- if (device_may_wakeup(wirq->dev))
+ if (device_may_wakeup(wirq->dev)) {
disable_irq_wake(wirq->irq);
+
+ if (wirq->status & WAKE_IRQ_DEDICATED_ALLOCATED)
+ disable_irq_nosync(wirq->irq);
+ }
}
void bcma_core_chipcommon_early_init(struct bcma_drv_cc *cc);
void bcma_core_chipcommon_init(struct bcma_drv_cc *cc);
void bcma_chipco_bcm4331_ext_pa_lines_ctl(struct bcma_drv_cc *cc, bool enable);
+#ifdef CONFIG_BCMA_DRIVER_MIPS
+void bcma_chipco_serial_init(struct bcma_drv_cc *cc);
+#endif /* CONFIG_BCMA_DRIVER_MIPS */
/* driver_chipcommon_b.c */
int bcma_core_chipcommon_b_init(struct bcma_drv_cc_b *ccb);
#include <linux/platform_device.h>
#include <linux/bcma/bcma.h>
-static void bcma_chipco_serial_init(struct bcma_drv_cc *cc);
-
static inline u32 bcma_cc_write32_masked(struct bcma_drv_cc *cc, u16 offset,
u32 mask, u32 value)
{
if (cc->capabilities & BCMA_CC_CAP_PMU)
bcma_pmu_early_init(cc);
- if (IS_BUILTIN(CONFIG_BCM47XX) && bus->hosttype == BCMA_HOSTTYPE_SOC)
- bcma_chipco_serial_init(cc);
-
if (bus->hosttype == BCMA_HOSTTYPE_SOC)
bcma_core_chipcommon_flash_detect(cc);
return res;
}
-static void bcma_chipco_serial_init(struct bcma_drv_cc *cc)
+#ifdef CONFIG_BCMA_DRIVER_MIPS
+void bcma_chipco_serial_init(struct bcma_drv_cc *cc)
{
-#if IS_BUILTIN(CONFIG_BCM47XX)
unsigned int irq;
u32 baud_base;
u32 i;
ports[i].baud_base = baud_base;
ports[i].reg_shift = 0;
}
-#endif /* CONFIG_BCM47XX */
}
+#endif /* CONFIG_BCMA_DRIVER_MIPS */
void bcma_core_mips_early_init(struct bcma_drv_mips *mcore)
{
+ struct bcma_bus *bus = mcore->core->bus;
+
if (mcore->early_setup_done)
return;
+ bcma_chipco_serial_init(&bus->drv_cc);
bcma_core_mips_nvram_init(mcore);
mcore->early_setup_done = true;
/* Number of pages per ring buffer. */
unsigned int nr_ring_pages;
struct request_queue *rq;
- unsigned int feature_flush;
- unsigned int feature_fua;
+ unsigned int feature_flush:1;
+ unsigned int feature_fua:1;
unsigned int feature_discard:1;
unsigned int feature_secdiscard:1;
+ unsigned int feature_persistent:1;
unsigned int discard_granularity;
unsigned int discard_alignment;
- unsigned int feature_persistent:1;
/* Number of 4KB segments handled */
unsigned int max_indirect_segments;
int is_ready;
}
else
grants = info->max_indirect_segments;
- psegs = grants / GRANTS_PER_PSEG;
+ psegs = DIV_ROUND_UP(grants, GRANTS_PER_PSEG);
err = fill_grant_buffer(rinfo,
(grants + INDIRECT_GREFS(grants)) * BLK_RING_SIZE(info));
blkfront_setup_discard(info);
info->feature_persistent =
- xenbus_read_unsigned(info->xbdev->otherend,
- "feature-persistent", 0);
+ !!xenbus_read_unsigned(info->xbdev->otherend,
+ "feature-persistent", 0);
indirect_segments = xenbus_read_unsigned(info->xbdev->otherend,
"feature-max-indirect-segments", 0);
- info->max_indirect_segments = min(indirect_segments,
- xen_blkif_max_segments);
+ if (indirect_segments > xen_blkif_max_segments)
+ indirect_segments = xen_blkif_max_segments;
+ if (indirect_segments <= BLKIF_MAX_SEGMENTS_PER_REQUEST)
+ indirect_segments = 0;
+ info->max_indirect_segments = indirect_segments;
}
/*
if (!xen_domain())
return -ENODEV;
+ if (xen_blkif_max_segments < BLKIF_MAX_SEGMENTS_PER_REQUEST)
+ xen_blkif_max_segments = BLKIF_MAX_SEGMENTS_PER_REQUEST;
+
if (xen_blkif_max_ring_order > XENBUS_MAX_RING_GRANT_ORDER) {
pr_info("Invalid max_ring_order (%d), will use default max: %d.\n",
xen_blkif_max_ring_order, XENBUS_MAX_RING_GRANT_ORDER);
mutex_unlock(&reading_mutex);
if (bytes_read > 0)
add_device_randomness(rng_buffer, bytes_read);
- memset(rng_buffer, 0, size);
}
static inline void cleanup_rng(struct kref *kref)
}
}
out:
- memset(rng_buffer, 0, rng_buffer_size());
return ret ? : err;
out_unlock_reading:
/* Outside lock, sure, but y'know: randomness. */
add_hwgenerator_randomness((void *)rng_fillbuf, rc,
rc * current_quality * 8 >> 10);
- memset(rng_fillbuf, 0, rng_buffer_size());
}
hwrng_fill = NULL;
return 0;
struct dev_pm_opp *opp;
int i, uv;
- rcu_read_lock();
-
opp = dev_pm_opp_find_freq_ceil(td->soc->dev, &rate);
- if (IS_ERR(opp)) {
- rcu_read_unlock();
+ if (IS_ERR(opp))
return PTR_ERR(opp);
- }
- uv = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ uv = dev_pm_opp_get_voltage(opp);
+ dev_pm_opp_put(opp);
for (i = 0; i < td->i2c_lut_size; i++) {
if (regulator_list_voltage(td->vdd_reg, td->i2c_lut[i]) == uv)
struct dev_pm_opp *opp;
int lut;
- rcu_read_lock();
-
rate = ULONG_MAX;
opp = dev_pm_opp_find_freq_floor(td->soc->dev, &rate);
if (IS_ERR(opp)) {
goto out;
}
v_max = dev_pm_opp_get_voltage(opp);
+ dev_pm_opp_put(opp);
v = td->soc->cvb->min_millivolts * 1000;
lut = find_vdd_map_entry_exact(td, v);
if (v_opp <= td->soc->cvb->min_millivolts * 1000)
td->dvco_rate_min = dev_pm_opp_get_freq(opp);
+ dev_pm_opp_put(opp);
+
for (;;) {
v += max(1, (v_max - v) / (MAX_DFLL_VOLTAGES - j));
if (v >= v_opp)
ret = 0;
out:
- rcu_read_unlock();
-
return ret;
}
If in doubt, say N.
-config CPU_FREQ_STAT_DETAILS
- bool "CPU frequency transition statistics details"
- depends on CPU_FREQ_STAT
- help
- Show detailed CPU frequency transition table in sysfs.
-
- If in doubt, say N.
-
choice
prompt "Default CPUFreq governor"
default CPU_FREQ_DEFAULT_GOV_USERSPACE if ARM_SA1100_CPUFREQ || ARM_SA1110_CPUFREQ
endif
if MIPS
+config BMIPS_CPUFREQ
+ tristate "BMIPS CPUfreq Driver"
+ help
+ This option adds a CPUfreq driver for BMIPS processors with
+ support for configurable CPU frequency.
+
+ For now, BMIPS5 chips are supported (such as the Broadcom 7425).
+
+ If in doubt, say N.
+
config LOONGSON2_CPUFREQ
tristate "Loongson2 CPUFreq Driver"
help
config QORIQ_CPUFREQ
tristate "CPU frequency scaling driver for Freescale QorIQ SoCs"
- depends on OF && COMMON_CLK && (PPC_E500MC || ARM)
+ depends on OF && COMMON_CLK && (PPC_E500MC || ARM || ARM64)
depends on !CPU_THERMAL || THERMAL
select CLK_QORIQ
help
help
This adds the CPUFreq driver support for Tegra124 SOCs.
+config ARM_TI_CPUFREQ
+ bool "Texas Instruments CPUFreq support"
+ depends on ARCH_OMAP2PLUS
+ help
+ This driver enables valid OPPs on the running platform based on
+ values contained within the SoC in use. Enable this in order to
+ use the cpufreq-dt driver on all Texas Instruments platforms that
+ provide dt based operating-points-v2 tables with opp-supported-hw
+ data provided. Required for cpufreq support on AM335x, AM437x,
+ DRA7x, and AM57x platforms.
+
config ARM_PXA2xx_CPUFREQ
tristate "Intel PXA2xx CPUfreq driver"
depends on PXA27x || PXA25x
config ACPI_CPPC_CPUFREQ
tristate "CPUFreq driver based on the ACPI CPPC spec"
- depends on ACPI
+ depends on ACPI_PROCESSOR
select ACPI_CPPC_LIB
default n
help
obj-$(CONFIG_ARM_STI_CPUFREQ) += sti-cpufreq.o
obj-$(CONFIG_ARM_TEGRA20_CPUFREQ) += tegra20-cpufreq.o
obj-$(CONFIG_ARM_TEGRA124_CPUFREQ) += tegra124-cpufreq.o
+obj-$(CONFIG_ARM_TI_CPUFREQ) += ti-cpufreq.o
obj-$(CONFIG_ARM_VEXPRESS_SPC_CPUFREQ) += vexpress-spc-cpufreq.o
obj-$(CONFIG_ACPI_CPPC_CPUFREQ) += cppc_cpufreq.o
obj-$(CONFIG_MACH_MVEBU_V7) += mvebu-cpufreq.o
# Other platform drivers
obj-$(CONFIG_AVR32_AT32AP_CPUFREQ) += at32ap-cpufreq.o
obj-$(CONFIG_BFIN_CPU_FREQ) += blackfin-cpufreq.o
+obj-$(CONFIG_BMIPS_CPUFREQ) += bmips-cpufreq.o
obj-$(CONFIG_CRIS_MACH_ARTPEC3) += cris-artpec3-cpufreq.o
obj-$(CONFIG_ETRAXFS) += cris-etraxfs-cpufreq.o
obj-$(CONFIG_IA64_ACPI_CPUFREQ) += ia64-acpi-cpufreq.o
--- /dev/null
+/*
+ * CPU frequency scaling for Broadcom BMIPS SoCs
+ *
+ * Copyright (c) 2017 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/cpufreq.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+
+/* for mips_hpt_frequency */
+#include <asm/time.h>
+
+#define BMIPS_CPUFREQ_PREFIX "bmips"
+#define BMIPS_CPUFREQ_NAME BMIPS_CPUFREQ_PREFIX "-cpufreq"
+
+#define TRANSITION_LATENCY (25 * 1000) /* 25 us */
+
+#define BMIPS5_CLK_DIV_SET_SHIFT 0x7
+#define BMIPS5_CLK_DIV_SHIFT 0x4
+#define BMIPS5_CLK_DIV_MASK 0xf
+
+enum bmips_type {
+ BMIPS5000,
+ BMIPS5200,
+};
+
+struct cpufreq_compat {
+ const char *compatible;
+ unsigned int bmips_type;
+ unsigned int clk_mult;
+ unsigned int max_freqs;
+};
+
+#define BMIPS(c, t, m, f) { \
+ .compatible = c, \
+ .bmips_type = (t), \
+ .clk_mult = (m), \
+ .max_freqs = (f), \
+}
+
+static struct cpufreq_compat bmips_cpufreq_compat[] = {
+ BMIPS("brcm,bmips5000", BMIPS5000, 8, 4),
+ BMIPS("brcm,bmips5200", BMIPS5200, 8, 4),
+ { }
+};
+
+static struct cpufreq_compat *priv;
+
+static int htp_freq_to_cpu_freq(unsigned int clk_mult)
+{
+ return mips_hpt_frequency * clk_mult / 1000;
+}
+
+static struct cpufreq_frequency_table *
+bmips_cpufreq_get_freq_table(const struct cpufreq_policy *policy)
+{
+ struct cpufreq_frequency_table *table;
+ unsigned long cpu_freq;
+ int i;
+
+ cpu_freq = htp_freq_to_cpu_freq(priv->clk_mult);
+
+ table = kmalloc((priv->max_freqs + 1) * sizeof(*table), GFP_KERNEL);
+ if (!table)
+ return ERR_PTR(-ENOMEM);
+
+ for (i = 0; i < priv->max_freqs; i++) {
+ table[i].frequency = cpu_freq / (1 << i);
+ table[i].driver_data = i;
+ }
+ table[i].frequency = CPUFREQ_TABLE_END;
+
+ return table;
+}
+
+static unsigned int bmips_cpufreq_get(unsigned int cpu)
+{
+ unsigned int div;
+ uint32_t mode;
+
+ switch (priv->bmips_type) {
+ case BMIPS5200:
+ case BMIPS5000:
+ mode = read_c0_brcm_mode();
+ div = ((mode >> BMIPS5_CLK_DIV_SHIFT) & BMIPS5_CLK_DIV_MASK);
+ break;
+ default:
+ div = 0;
+ }
+
+ return htp_freq_to_cpu_freq(priv->clk_mult) / (1 << div);
+}
+
+static int bmips_cpufreq_target_index(struct cpufreq_policy *policy,
+ unsigned int index)
+{
+ unsigned int div = policy->freq_table[index].driver_data;
+
+ switch (priv->bmips_type) {
+ case BMIPS5200:
+ case BMIPS5000:
+ change_c0_brcm_mode(BMIPS5_CLK_DIV_MASK << BMIPS5_CLK_DIV_SHIFT,
+ (1 << BMIPS5_CLK_DIV_SET_SHIFT) |
+ (div << BMIPS5_CLK_DIV_SHIFT));
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ return 0;
+}
+
+static int bmips_cpufreq_exit(struct cpufreq_policy *policy)
+{
+ kfree(policy->freq_table);
+
+ return 0;
+}
+
+static int bmips_cpufreq_init(struct cpufreq_policy *policy)
+{
+ struct cpufreq_frequency_table *freq_table;
+ int ret;
+
+ freq_table = bmips_cpufreq_get_freq_table(policy);
+ if (IS_ERR(freq_table)) {
+ ret = PTR_ERR(freq_table);
+ pr_err("%s: couldn't determine frequency table (%d).\n",
+ BMIPS_CPUFREQ_NAME, ret);
+ return ret;
+ }
+
+ ret = cpufreq_generic_init(policy, freq_table, TRANSITION_LATENCY);
+ if (ret)
+ bmips_cpufreq_exit(policy);
+ else
+ pr_info("%s: registered\n", BMIPS_CPUFREQ_NAME);
+
+ return ret;
+}
+
+static struct cpufreq_driver bmips_cpufreq_driver = {
+ .flags = CPUFREQ_NEED_INITIAL_FREQ_CHECK,
+ .verify = cpufreq_generic_frequency_table_verify,
+ .target_index = bmips_cpufreq_target_index,
+ .get = bmips_cpufreq_get,
+ .init = bmips_cpufreq_init,
+ .exit = bmips_cpufreq_exit,
+ .attr = cpufreq_generic_attr,
+ .name = BMIPS_CPUFREQ_PREFIX,
+};
+
+static int __init bmips_cpufreq_probe(void)
+{
+ struct cpufreq_compat *cc;
+ struct device_node *np;
+
+ for (cc = bmips_cpufreq_compat; cc->compatible; cc++) {
+ np = of_find_compatible_node(NULL, "cpu", cc->compatible);
+ if (np) {
+ of_node_put(np);
+ priv = cc;
+ break;
+ }
+ }
+
+ /* We hit the guard element of the array. No compatible CPU found. */
+ if (!cc->compatible)
+ return -ENODEV;
+
+ return cpufreq_register_driver(&bmips_cpufreq_driver);
+}
+device_initcall(bmips_cpufreq_probe);
+
+MODULE_AUTHOR("Markus Mayer <mmayer@broadcom.com>");
+MODULE_DESCRIPTION("CPUfreq driver for Broadcom BMIPS SoCs");
+MODULE_LICENSE("GPL");
static int brcm_avs_suspend(struct cpufreq_policy *policy)
{
struct private_data *priv = policy->driver_data;
+ int ret;
+
+ ret = brcm_avs_get_pmap(priv, &priv->pmap);
+ if (ret)
+ return ret;
- return brcm_avs_get_pmap(priv, &priv->pmap);
+ /*
+ * We can't use the P-state returned by brcm_avs_get_pmap(), since
+ * that's the initial P-state from when the P-map was downloaded to the
+ * AVS co-processor, not necessarily the P-state we are running at now.
+ * So, we get the current P-state explicitly.
+ */
+ return brcm_avs_get_pstate(priv, &priv->pmap.state);
}
static int brcm_avs_resume(struct cpufreq_policy *policy)
iounmap(priv->avs_intr_base);
unmap_base:
iounmap(priv->base);
- platform_set_drvdata(pdev, NULL);
return ret;
}
brcm_avs_parse_p1(pmap.p1, &mdiv_p0, &pdiv, &ndiv);
brcm_avs_parse_p2(pmap.p2, &mdiv_p1, &mdiv_p2, &mdiv_p3, &mdiv_p4);
- return sprintf(buf, "0x%08x 0x%08x %u %u %u %u %u %u %u\n",
+ return sprintf(buf, "0x%08x 0x%08x %u %u %u %u %u %u %u %u %u\n",
pmap.p1, pmap.p2, ndiv, pdiv, mdiv_p0, mdiv_p1, mdiv_p2,
- mdiv_p3, mdiv_p4);
+ mdiv_p3, mdiv_p4, pmap.mode, pmap.state);
}
static ssize_t show_brcm_avs_voltage(struct cpufreq_policy *policy, char *buf)
priv = platform_get_drvdata(pdev);
iounmap(priv->base);
iounmap(priv->avs_intr_base);
- platform_set_drvdata(pdev, NULL);
return 0;
}
{ .compatible = "socionext,uniphier-ld11", },
{ .compatible = "socionext,uniphier-ld20", },
- { .compatible = "ti,am33xx", },
- { .compatible = "ti,dra7", },
{ .compatible = "ti,omap2", },
{ .compatible = "ti,omap3", },
{ .compatible = "ti,omap4", },
struct private_data *priv;
struct device *cpu_dev;
struct clk *cpu_clk;
- struct dev_pm_opp *suspend_opp;
unsigned int transition_latency;
bool fallback = false;
const char *name;
policy->driver_data = priv;
policy->clk = cpu_clk;
- rcu_read_lock();
- suspend_opp = dev_pm_opp_get_suspend_opp(cpu_dev);
- if (suspend_opp)
- policy->suspend_freq = dev_pm_opp_get_freq(suspend_opp) / 1000;
- rcu_read_unlock();
+ policy->suspend_freq = dev_pm_opp_get_suspend_opp_freq(cpu_dev) / 1000;
ret = cpufreq_table_validate_and_show(policy, freq_table);
if (ret) {
return NULL;
}
-static void cpufreq_policy_put_kobj(struct cpufreq_policy *policy, bool notify)
+static void cpufreq_policy_put_kobj(struct cpufreq_policy *policy)
{
struct kobject *kobj;
struct completion *cmp;
- if (notify)
- blocking_notifier_call_chain(&cpufreq_policy_notifier_list,
- CPUFREQ_REMOVE_POLICY, policy);
-
down_write(&policy->rwsem);
cpufreq_stats_free_table(policy);
kobj = &policy->kobj;
pr_debug("wait complete\n");
}
-static void cpufreq_policy_free(struct cpufreq_policy *policy, bool notify)
+static void cpufreq_policy_free(struct cpufreq_policy *policy)
{
unsigned long flags;
int cpu;
per_cpu(cpufreq_cpu_data, cpu) = NULL;
write_unlock_irqrestore(&cpufreq_driver_lock, flags);
- cpufreq_policy_put_kobj(policy, notify);
+ cpufreq_policy_put_kobj(policy);
free_cpumask_var(policy->real_cpus);
free_cpumask_var(policy->related_cpus);
free_cpumask_var(policy->cpus);
if (new_policy) {
/* related_cpus should at least include policy->cpus. */
cpumask_copy(policy->related_cpus, policy->cpus);
- /* Clear mask of registered CPUs */
- cpumask_clear(policy->real_cpus);
}
/*
goto out_exit_policy;
cpufreq_stats_create_table(policy);
- blocking_notifier_call_chain(&cpufreq_policy_notifier_list,
- CPUFREQ_CREATE_POLICY, policy);
write_lock_irqsave(&cpufreq_driver_lock, flags);
list_add(&policy->policy_list, &cpufreq_policy_list);
write_unlock_irqrestore(&cpufreq_driver_lock, flags);
}
- blocking_notifier_call_chain(&cpufreq_policy_notifier_list,
- CPUFREQ_START, policy);
-
ret = cpufreq_init_policy(policy);
if (ret) {
pr_err("%s: Failed to initialize policy for cpu: %d (%d)\n",
if (cpufreq_driver->exit)
cpufreq_driver->exit(policy);
out_free_policy:
- cpufreq_policy_free(policy, !new_policy);
+ cpufreq_policy_free(policy);
return ret;
}
remove_cpu_dev_symlink(policy, dev);
if (cpumask_empty(policy->real_cpus))
- cpufreq_policy_free(policy, true);
+ cpufreq_policy_free(policy);
}
/**
unsigned int last_index;
u64 *time_in_state;
unsigned int *freq_table;
-#ifdef CONFIG_CPU_FREQ_STAT_DETAILS
unsigned int *trans_table;
-#endif
};
static int cpufreq_stats_update(struct cpufreq_stats *stats)
unsigned int count = stats->max_state;
memset(stats->time_in_state, 0, count * sizeof(u64));
-#ifdef CONFIG_CPU_FREQ_STAT_DETAILS
memset(stats->trans_table, 0, count * count * sizeof(int));
-#endif
stats->last_time = get_jiffies_64();
stats->total_trans = 0;
}
return count;
}
-#ifdef CONFIG_CPU_FREQ_STAT_DETAILS
static ssize_t show_trans_table(struct cpufreq_policy *policy, char *buf)
{
struct cpufreq_stats *stats = policy->stats;
return len;
}
cpufreq_freq_attr_ro(trans_table);
-#endif
cpufreq_freq_attr_ro(total_trans);
cpufreq_freq_attr_ro(time_in_state);
&total_trans.attr,
&time_in_state.attr,
&reset.attr,
-#ifdef CONFIG_CPU_FREQ_STAT_DETAILS
&trans_table.attr,
-#endif
NULL
};
static struct attribute_group stats_attr_group = {
alloc_size = count * sizeof(int) + count * sizeof(u64);
-#ifdef CONFIG_CPU_FREQ_STAT_DETAILS
alloc_size += count * count * sizeof(int);
-#endif
/* Allocate memory for time_in_state/freq_table/trans_table in one go */
stats->time_in_state = kzalloc(alloc_size, GFP_KERNEL);
stats->freq_table = (unsigned int *)(stats->time_in_state + count);
-#ifdef CONFIG_CPU_FREQ_STAT_DETAILS
stats->trans_table = stats->freq_table + count;
-#endif
stats->max_state = count;
cpufreq_stats_update(stats);
stats->last_index = new_index;
-#ifdef CONFIG_CPU_FREQ_STAT_DETAILS
stats->trans_table[old_index * stats->max_state + new_index]++;
-#endif
stats->total_trans++;
}
unsigned int tmp, clk_div, ema_div, freq, volt_id;
struct dev_pm_opp *opp;
- rcu_read_lock();
cpufreq_for_each_entry(pos, freq_tbl) {
opp = dev_pm_opp_find_freq_exact(dvfs_info->dev,
pos->frequency * 1000, true);
if (IS_ERR(opp)) {
- rcu_read_unlock();
dev_err(dvfs_info->dev,
"failed to find valid OPP for %u KHZ\n",
pos->frequency);
/* Calculate EMA */
volt_id = dev_pm_opp_get_voltage(opp);
+
volt_id = (MAX_VOLTAGE - volt_id) / VOLTAGE_STEP;
if (volt_id < PMIC_HIGH_VOLT) {
ema_div = (CPUEMA_HIGH << P0_7_CPUEMA_SHIFT) |
__raw_writel(tmp, dvfs_info->base + XMU_PMU_P0_7 + 4 *
(pos - freq_tbl));
+ dev_pm_opp_put(opp);
}
- rcu_read_unlock();
return 0;
}
freq_hz = new_freq * 1000;
old_freq = clk_get_rate(arm_clk) / 1000;
- rcu_read_lock();
opp = dev_pm_opp_find_freq_ceil(cpu_dev, &freq_hz);
if (IS_ERR(opp)) {
- rcu_read_unlock();
dev_err(cpu_dev, "failed to find OPP for %ld\n", freq_hz);
return PTR_ERR(opp);
}
volt = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
+
volt_old = regulator_get_voltage(arm_reg);
dev_dbg(cpu_dev, "%u MHz, %ld mV --> %u MHz, %ld mV\n",
* freq_table initialised from OPP is therefore sorted in the
* same order.
*/
- rcu_read_lock();
opp = dev_pm_opp_find_freq_exact(cpu_dev,
freq_table[0].frequency * 1000, true);
min_volt = dev_pm_opp_get_voltage(opp);
+ dev_pm_opp_put(opp);
opp = dev_pm_opp_find_freq_exact(cpu_dev,
freq_table[--num].frequency * 1000, true);
max_volt = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
+
ret = regulator_set_voltage_time(arm_reg, min_volt, max_volt);
if (ret > 0)
transition_latency += ret * 1000;
static int hwp_active __read_mostly;
static bool per_cpu_limits __read_mostly;
+static bool driver_registered __read_mostly;
+
#ifdef CONFIG_ACPI
static bool acpi_ppc;
#endif
static struct perf_limits *limits = &powersave_limits;
#endif
+static DEFINE_MUTEX(intel_pstate_driver_lock);
static DEFINE_MUTEX(intel_pstate_limits_lock);
#ifdef CONFIG_ACPI
acpi_processor_unregister_performance(policy->cpu);
}
-
#else
static inline void intel_pstate_init_acpi_perf_limits(struct cpufreq_policy *policy)
{
rdmsrl_on_cpu(cpu, MSR_HWP_CAPABILITIES, &cap);
hw_min = HWP_LOWEST_PERF(cap);
- hw_max = HWP_HIGHEST_PERF(cap);
+ if (limits->no_turbo)
+ hw_max = HWP_GUARANTEED_PERF(cap);
+ else
+ hw_max = HWP_HIGHEST_PERF(cap);
range = hw_max - hw_min;
max_perf_pct = perf_limits->max_perf_pct;
adj_range = max_perf_pct * range / 100;
max = hw_min + adj_range;
- if (limits->no_turbo) {
- hw_max = HWP_GUARANTEED_PERF(cap);
- if (hw_max < max)
- max = hw_max;
- }
value &= ~HWP_MAX_PERF(~0L);
value |= HWP_MAX_PERF(max);
}
DEFINE_SIMPLE_ATTRIBUTE(fops_pid_param, pid_param_get, pid_param_set, "%llu\n");
+static struct dentry *debugfs_parent;
+
struct pid_param {
char *name;
void *value;
+ struct dentry *dentry;
};
static struct pid_param pid_files[] = {
- {"sample_rate_ms", &pid_params.sample_rate_ms},
- {"d_gain_pct", &pid_params.d_gain_pct},
- {"i_gain_pct", &pid_params.i_gain_pct},
- {"deadband", &pid_params.deadband},
- {"setpoint", &pid_params.setpoint},
- {"p_gain_pct", &pid_params.p_gain_pct},
- {NULL, NULL}
+ {"sample_rate_ms", &pid_params.sample_rate_ms, },
+ {"d_gain_pct", &pid_params.d_gain_pct, },
+ {"i_gain_pct", &pid_params.i_gain_pct, },
+ {"deadband", &pid_params.deadband, },
+ {"setpoint", &pid_params.setpoint, },
+ {"p_gain_pct", &pid_params.p_gain_pct, },
+ {NULL, NULL, }
};
-static void __init intel_pstate_debug_expose_params(void)
+static void intel_pstate_debug_expose_params(void)
{
- struct dentry *debugfs_parent;
- int i = 0;
+ int i;
debugfs_parent = debugfs_create_dir("pstate_snb", NULL);
if (IS_ERR_OR_NULL(debugfs_parent))
return;
- while (pid_files[i].name) {
- debugfs_create_file(pid_files[i].name, 0660,
- debugfs_parent, pid_files[i].value,
- &fops_pid_param);
- i++;
+
+ for (i = 0; pid_files[i].name; i++) {
+ struct dentry *dentry;
+
+ dentry = debugfs_create_file(pid_files[i].name, 0660,
+ debugfs_parent, pid_files[i].value,
+ &fops_pid_param);
+ if (!IS_ERR(dentry))
+ pid_files[i].dentry = dentry;
}
}
+static void intel_pstate_debug_hide_params(void)
+{
+ int i;
+
+ if (IS_ERR_OR_NULL(debugfs_parent))
+ return;
+
+ for (i = 0; pid_files[i].name; i++) {
+ debugfs_remove(pid_files[i].dentry);
+ pid_files[i].dentry = NULL;
+ }
+
+ debugfs_remove(debugfs_parent);
+ debugfs_parent = NULL;
+}
+
/************************** debugfs end ************************/
/************************** sysfs begin ************************/
return sprintf(buf, "%u\n", limits->object); \
}
+static ssize_t intel_pstate_show_status(char *buf);
+static int intel_pstate_update_status(const char *buf, size_t size);
+
+static ssize_t show_status(struct kobject *kobj,
+ struct attribute *attr, char *buf)
+{
+ ssize_t ret;
+
+ mutex_lock(&intel_pstate_driver_lock);
+ ret = intel_pstate_show_status(buf);
+ mutex_unlock(&intel_pstate_driver_lock);
+
+ return ret;
+}
+
+static ssize_t store_status(struct kobject *a, struct attribute *b,
+ const char *buf, size_t count)
+{
+ char *p = memchr(buf, '\n', count);
+ int ret;
+
+ mutex_lock(&intel_pstate_driver_lock);
+ ret = intel_pstate_update_status(buf, p ? p - buf : count);
+ mutex_unlock(&intel_pstate_driver_lock);
+
+ return ret < 0 ? ret : count;
+}
+
static ssize_t show_turbo_pct(struct kobject *kobj,
struct attribute *attr, char *buf)
{
int total, no_turbo, turbo_pct;
uint32_t turbo_fp;
+ mutex_lock(&intel_pstate_driver_lock);
+
+ if (!driver_registered) {
+ mutex_unlock(&intel_pstate_driver_lock);
+ return -EAGAIN;
+ }
+
cpu = all_cpu_data[0];
total = cpu->pstate.turbo_pstate - cpu->pstate.min_pstate + 1;
no_turbo = cpu->pstate.max_pstate - cpu->pstate.min_pstate + 1;
turbo_fp = div_fp(no_turbo, total);
turbo_pct = 100 - fp_toint(mul_fp(turbo_fp, int_tofp(100)));
+
+ mutex_unlock(&intel_pstate_driver_lock);
+
return sprintf(buf, "%u\n", turbo_pct);
}
struct cpudata *cpu;
int total;
+ mutex_lock(&intel_pstate_driver_lock);
+
+ if (!driver_registered) {
+ mutex_unlock(&intel_pstate_driver_lock);
+ return -EAGAIN;
+ }
+
cpu = all_cpu_data[0];
total = cpu->pstate.turbo_pstate - cpu->pstate.min_pstate + 1;
+
+ mutex_unlock(&intel_pstate_driver_lock);
+
return sprintf(buf, "%u\n", total);
}
{
ssize_t ret;
+ mutex_lock(&intel_pstate_driver_lock);
+
+ if (!driver_registered) {
+ mutex_unlock(&intel_pstate_driver_lock);
+ return -EAGAIN;
+ }
+
update_turbo_state();
if (limits->turbo_disabled)
ret = sprintf(buf, "%u\n", limits->turbo_disabled);
else
ret = sprintf(buf, "%u\n", limits->no_turbo);
+ mutex_unlock(&intel_pstate_driver_lock);
+
return ret;
}
if (ret != 1)
return -EINVAL;
+ mutex_lock(&intel_pstate_driver_lock);
+
+ if (!driver_registered) {
+ mutex_unlock(&intel_pstate_driver_lock);
+ return -EAGAIN;
+ }
+
mutex_lock(&intel_pstate_limits_lock);
update_turbo_state();
if (limits->turbo_disabled) {
pr_warn("Turbo disabled by BIOS or unavailable on processor\n");
mutex_unlock(&intel_pstate_limits_lock);
+ mutex_unlock(&intel_pstate_driver_lock);
return -EPERM;
}
intel_pstate_update_policies();
+ mutex_unlock(&intel_pstate_driver_lock);
+
return count;
}
if (ret != 1)
return -EINVAL;
+ mutex_lock(&intel_pstate_driver_lock);
+
+ if (!driver_registered) {
+ mutex_unlock(&intel_pstate_driver_lock);
+ return -EAGAIN;
+ }
+
mutex_lock(&intel_pstate_limits_lock);
limits->max_sysfs_pct = clamp_t(int, input, 0 , 100);
intel_pstate_update_policies();
+ mutex_unlock(&intel_pstate_driver_lock);
+
return count;
}
if (ret != 1)
return -EINVAL;
+ mutex_lock(&intel_pstate_driver_lock);
+
+ if (!driver_registered) {
+ mutex_unlock(&intel_pstate_driver_lock);
+ return -EAGAIN;
+ }
+
mutex_lock(&intel_pstate_limits_lock);
limits->min_sysfs_pct = clamp_t(int, input, 0 , 100);
intel_pstate_update_policies();
+ mutex_unlock(&intel_pstate_driver_lock);
+
return count;
}
show_one(max_perf_pct, max_perf_pct);
show_one(min_perf_pct, min_perf_pct);
+define_one_global_rw(status);
define_one_global_rw(no_turbo);
define_one_global_rw(max_perf_pct);
define_one_global_rw(min_perf_pct);
define_one_global_ro(num_pstates);
static struct attribute *intel_pstate_attributes[] = {
+ &status.attr,
&no_turbo.attr,
&turbo_pct.attr,
&num_pstates.attr,
cpudata->epp_default = intel_pstate_get_epp(cpudata, 0);
}
+#define MSR_IA32_POWER_CTL_BIT_EE 19
+
+/* Disable energy efficiency optimization */
+static void intel_pstate_disable_ee(int cpu)
+{
+ u64 power_ctl;
+ int ret;
+
+ ret = rdmsrl_on_cpu(cpu, MSR_IA32_POWER_CTL, &power_ctl);
+ if (ret)
+ return;
+
+ if (!(power_ctl & BIT(MSR_IA32_POWER_CTL_BIT_EE))) {
+ pr_info("Disabling energy efficiency optimization\n");
+ power_ctl |= BIT(MSR_IA32_POWER_CTL_BIT_EE);
+ wrmsrl_on_cpu(cpu, MSR_IA32_POWER_CTL, power_ctl);
+ }
+}
+
static int atom_get_min_pstate(void)
{
u64 value;
return (value >> 8) & 0xFF;
}
+static int core_get_tdp_ratio(u64 plat_info)
+{
+ /* Check how many TDP levels present */
+ if (plat_info & 0x600000000) {
+ u64 tdp_ctrl;
+ u64 tdp_ratio;
+ int tdp_msr;
+ int err;
+
+ /* Get the TDP level (0, 1, 2) to get ratios */
+ err = rdmsrl_safe(MSR_CONFIG_TDP_CONTROL, &tdp_ctrl);
+ if (err)
+ return err;
+
+ /* TDP MSR are continuous starting at 0x648 */
+ tdp_msr = MSR_CONFIG_TDP_NOMINAL + (tdp_ctrl & 0x03);
+ err = rdmsrl_safe(tdp_msr, &tdp_ratio);
+ if (err)
+ return err;
+
+ /* For level 1 and 2, bits[23:16] contain the ratio */
+ if (tdp_ctrl & 0x03)
+ tdp_ratio >>= 16;
+
+ tdp_ratio &= 0xff; /* ratios are only 8 bits long */
+ pr_debug("tdp_ratio %x\n", (int)tdp_ratio);
+
+ return (int)tdp_ratio;
+ }
+
+ return -ENXIO;
+}
+
static int core_get_max_pstate(void)
{
u64 tar;
u64 plat_info;
int max_pstate;
+ int tdp_ratio;
int err;
rdmsrl(MSR_PLATFORM_INFO, plat_info);
max_pstate = (plat_info >> 8) & 0xFF;
+ tdp_ratio = core_get_tdp_ratio(plat_info);
+ if (tdp_ratio <= 0)
+ return max_pstate;
+
+ if (hwp_active) {
+ /* Turbo activation ratio is not used on HWP platforms */
+ return tdp_ratio;
+ }
+
err = rdmsrl_safe(MSR_TURBO_ACTIVATION_RATIO, &tar);
if (!err) {
+ int tar_levels;
+
/* Do some sanity checking for safety */
- if (plat_info & 0x600000000) {
- u64 tdp_ctrl;
- u64 tdp_ratio;
- int tdp_msr;
-
- err = rdmsrl_safe(MSR_CONFIG_TDP_CONTROL, &tdp_ctrl);
- if (err)
- goto skip_tar;
-
- tdp_msr = MSR_CONFIG_TDP_NOMINAL + (tdp_ctrl & 0x3);
- err = rdmsrl_safe(tdp_msr, &tdp_ratio);
- if (err)
- goto skip_tar;
-
- /* For level 1 and 2, bits[23:16] contain the ratio */
- if (tdp_ctrl)
- tdp_ratio >>= 16;
-
- tdp_ratio &= 0xff; /* ratios are only 8 bits long */
- if (tdp_ratio - 1 == tar) {
- max_pstate = tar;
- pr_debug("max_pstate=TAC %x\n", max_pstate);
- } else {
- goto skip_tar;
- }
+ tar_levels = tar & 0xff;
+ if (tdp_ratio - 1 == tar_levels) {
+ max_pstate = tar_levels;
+ pr_debug("max_pstate=TAC %x\n", max_pstate);
}
}
-skip_tar:
return max_pstate;
}
{}
};
+static const struct x86_cpu_id intel_pstate_cpu_ee_disable_ids[] = {
+ ICPU(INTEL_FAM6_KABYLAKE_DESKTOP, core_params),
+ {}
+};
+
static int intel_pstate_init_cpu(unsigned int cpunum)
{
struct cpudata *cpu;
cpu->cpu = cpunum;
if (hwp_active) {
+ const struct x86_cpu_id *id;
+
+ id = x86_match_cpu(intel_pstate_cpu_ee_disable_ids);
+ if (id)
+ intel_pstate_disable_ee(cpunum);
+
intel_pstate_hwp_enable(cpu);
pid_params.sample_rate_ms = 50;
pid_params.sample_rate_ns = 50 * NSEC_PER_MSEC;
limits = &performance_limits;
perf_limits = limits;
}
- if (policy->max >= policy->cpuinfo.max_freq) {
+ if (policy->max >= policy->cpuinfo.max_freq &&
+ !limits->no_turbo) {
pr_debug("set performance\n");
intel_pstate_set_performance_limits(perf_limits);
goto out;
static int intel_pstate_verify_policy(struct cpufreq_policy *policy)
{
+ struct cpudata *cpu = all_cpu_data[policy->cpu];
+ struct perf_limits *perf_limits;
+
+ if (policy->policy == CPUFREQ_POLICY_PERFORMANCE)
+ perf_limits = &performance_limits;
+ else
+ perf_limits = &powersave_limits;
+
+ update_turbo_state();
+ policy->cpuinfo.max_freq = perf_limits->turbo_disabled ||
+ perf_limits->no_turbo ?
+ cpu->pstate.max_freq :
+ cpu->pstate.turbo_freq;
+
cpufreq_verify_within_cpu_limits(policy);
if (policy->policy != CPUFREQ_POLICY_POWERSAVE &&
policy->policy != CPUFREQ_POLICY_PERFORMANCE)
return -EINVAL;
+ /* When per-CPU limits are used, sysfs limits are not used */
+ if (!per_cpu_limits) {
+ unsigned int max_freq, min_freq;
+
+ max_freq = policy->cpuinfo.max_freq *
+ limits->max_sysfs_pct / 100;
+ min_freq = policy->cpuinfo.max_freq *
+ limits->min_sysfs_pct / 100;
+ cpufreq_verify_within_limits(policy, min_freq, max_freq);
+ }
+
return 0;
}
static struct cpufreq_driver *intel_pstate_driver = &intel_pstate;
+static void intel_pstate_driver_cleanup(void)
+{
+ unsigned int cpu;
+
+ get_online_cpus();
+ for_each_online_cpu(cpu) {
+ if (all_cpu_data[cpu]) {
+ if (intel_pstate_driver == &intel_pstate)
+ intel_pstate_clear_update_util_hook(cpu);
+
+ kfree(all_cpu_data[cpu]);
+ all_cpu_data[cpu] = NULL;
+ }
+ }
+ put_online_cpus();
+}
+
+static int intel_pstate_register_driver(void)
+{
+ int ret;
+
+ ret = cpufreq_register_driver(intel_pstate_driver);
+ if (ret) {
+ intel_pstate_driver_cleanup();
+ return ret;
+ }
+
+ mutex_lock(&intel_pstate_limits_lock);
+ driver_registered = true;
+ mutex_unlock(&intel_pstate_limits_lock);
+
+ if (intel_pstate_driver == &intel_pstate && !hwp_active &&
+ pstate_funcs.get_target_pstate != get_target_pstate_use_cpu_load)
+ intel_pstate_debug_expose_params();
+
+ return 0;
+}
+
+static int intel_pstate_unregister_driver(void)
+{
+ if (hwp_active)
+ return -EBUSY;
+
+ if (intel_pstate_driver == &intel_pstate && !hwp_active &&
+ pstate_funcs.get_target_pstate != get_target_pstate_use_cpu_load)
+ intel_pstate_debug_hide_params();
+
+ mutex_lock(&intel_pstate_limits_lock);
+ driver_registered = false;
+ mutex_unlock(&intel_pstate_limits_lock);
+
+ cpufreq_unregister_driver(intel_pstate_driver);
+ intel_pstate_driver_cleanup();
+
+ return 0;
+}
+
+static ssize_t intel_pstate_show_status(char *buf)
+{
+ if (!driver_registered)
+ return sprintf(buf, "off\n");
+
+ return sprintf(buf, "%s\n", intel_pstate_driver == &intel_pstate ?
+ "active" : "passive");
+}
+
+static int intel_pstate_update_status(const char *buf, size_t size)
+{
+ int ret;
+
+ if (size == 3 && !strncmp(buf, "off", size))
+ return driver_registered ?
+ intel_pstate_unregister_driver() : -EINVAL;
+
+ if (size == 6 && !strncmp(buf, "active", size)) {
+ if (driver_registered) {
+ if (intel_pstate_driver == &intel_pstate)
+ return 0;
+
+ ret = intel_pstate_unregister_driver();
+ if (ret)
+ return ret;
+ }
+
+ intel_pstate_driver = &intel_pstate;
+ return intel_pstate_register_driver();
+ }
+
+ if (size == 7 && !strncmp(buf, "passive", size)) {
+ if (driver_registered) {
+ if (intel_pstate_driver != &intel_pstate)
+ return 0;
+
+ ret = intel_pstate_unregister_driver();
+ if (ret)
+ return ret;
+ }
+
+ intel_pstate_driver = &intel_cpufreq;
+ return intel_pstate_register_driver();
+ }
+
+ return -EINVAL;
+}
+
static int no_load __initdata;
static int no_hwp __initdata;
static int hwp_only __initdata;
static int __init intel_pstate_init(void)
{
- int cpu, rc = 0;
const struct x86_cpu_id *id;
struct cpu_defaults *cpu_def;
+ int rc = 0;
if (no_load)
return -ENODEV;
if (intel_pstate_platform_pwr_mgmt_exists())
return -ENODEV;
+ if (!hwp_active && hwp_only)
+ return -ENOTSUPP;
+
pr_info("Intel P-state driver initializing\n");
all_cpu_data = vzalloc(sizeof(void *) * num_possible_cpus());
if (!all_cpu_data)
return -ENOMEM;
- if (!hwp_active && hwp_only)
- goto out;
-
intel_pstate_request_control_from_smm();
- rc = cpufreq_register_driver(intel_pstate_driver);
- if (rc)
- goto out;
-
- if (intel_pstate_driver == &intel_pstate && !hwp_active &&
- pstate_funcs.get_target_pstate != get_target_pstate_use_cpu_load)
- intel_pstate_debug_expose_params();
-
intel_pstate_sysfs_expose_params();
+ mutex_lock(&intel_pstate_driver_lock);
+ rc = intel_pstate_register_driver();
+ mutex_unlock(&intel_pstate_driver_lock);
+ if (rc)
+ return rc;
+
if (hwp_active)
pr_info("HWP enabled\n");
- return rc;
-out:
- get_online_cpus();
- for_each_online_cpu(cpu) {
- if (all_cpu_data[cpu]) {
- if (intel_pstate_driver == &intel_pstate)
- intel_pstate_clear_update_util_hook(cpu);
-
- kfree(all_cpu_data[cpu]);
- }
- }
-
- put_online_cpus();
- vfree(all_cpu_data);
- return -ENODEV;
+ return 0;
}
device_initcall(intel_pstate_init);
freq_hz = freq_table[index].frequency * 1000;
- rcu_read_lock();
opp = dev_pm_opp_find_freq_ceil(cpu_dev, &freq_hz);
if (IS_ERR(opp)) {
- rcu_read_unlock();
pr_err("cpu%d: failed to find OPP for %ld\n",
policy->cpu, freq_hz);
return PTR_ERR(opp);
}
vproc = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
/*
* If the new voltage or the intermediate voltage is higher than the
/* Search a safe voltage for intermediate frequency. */
rate = clk_get_rate(inter_clk);
- rcu_read_lock();
opp = dev_pm_opp_find_freq_ceil(cpu_dev, &rate);
if (IS_ERR(opp)) {
- rcu_read_unlock();
pr_err("failed to get intermediate opp for cpu%d\n", cpu);
ret = PTR_ERR(opp);
goto out_free_opp_table;
}
info->intermediate_voltage = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
info->cpu_dev = cpu_dev;
info->proc_reg = proc_reg;
freq = ret;
if (mpu_reg) {
- rcu_read_lock();
opp = dev_pm_opp_find_freq_ceil(mpu_dev, &freq);
if (IS_ERR(opp)) {
- rcu_read_unlock();
dev_err(mpu_dev, "%s: unable to find MPU OPP for %d\n",
__func__, new_freq);
return -EINVAL;
}
volt = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
tol = volt * OPP_TOLERANCE / 100;
volt_old = regulator_get_voltage(mpu_reg);
}
unsigned int max;
unsigned int nominal;
unsigned int nr_pstates;
+ bool wof_enabled;
} powernv_pstate_info;
/* Use following macros for conversions between pstate_id and index */
const __be32 *pstate_ids, *pstate_freqs;
u32 len_ids, len_freqs;
u32 pstate_min, pstate_max, pstate_nominal;
+ u32 pstate_turbo, pstate_ultra_turbo;
power_mgt = of_find_node_by_path("/ibm,opal/power-mgt");
if (!power_mgt) {
pr_warn("ibm,pstate-nominal not found\n");
return -ENODEV;
}
+
+ if (of_property_read_u32(power_mgt, "ibm,pstate-ultra-turbo",
+ &pstate_ultra_turbo)) {
+ powernv_pstate_info.wof_enabled = false;
+ goto next;
+ }
+
+ if (of_property_read_u32(power_mgt, "ibm,pstate-turbo",
+ &pstate_turbo)) {
+ powernv_pstate_info.wof_enabled = false;
+ goto next;
+ }
+
+ if (pstate_turbo == pstate_ultra_turbo)
+ powernv_pstate_info.wof_enabled = false;
+ else
+ powernv_pstate_info.wof_enabled = true;
+
+next:
pr_info("cpufreq pstate min %d nominal %d max %d\n", pstate_min,
pstate_nominal, pstate_max);
+ pr_info("Workload Optimized Frequency is %s in the platform\n",
+ (powernv_pstate_info.wof_enabled) ? "enabled" : "disabled");
pstate_ids = of_get_property(power_mgt, "ibm,pstate-ids", &len_ids);
if (!pstate_ids) {
powernv_pstate_info.nominal = i;
else if (id == pstate_min)
powernv_pstate_info.min = i;
+
+ if (powernv_pstate_info.wof_enabled && id == pstate_turbo) {
+ int j;
+
+ for (j = i - 1; j >= (int)powernv_pstate_info.max; j--)
+ powernv_freqs[j].flags = CPUFREQ_BOOST_FREQ;
+ }
}
/* End of list marker entry */
struct freq_attr cpufreq_freq_attr_cpuinfo_nominal_freq =
__ATTR_RO(cpuinfo_nominal_freq);
+#define SCALING_BOOST_FREQS_ATTR_INDEX 2
+
static struct freq_attr *powernv_cpu_freq_attr[] = {
&cpufreq_freq_attr_scaling_available_freqs,
&cpufreq_freq_attr_cpuinfo_nominal_freq,
+ &cpufreq_freq_attr_scaling_boost_freqs,
NULL,
};
register_reboot_notifier(&powernv_cpufreq_reboot_nb);
opal_message_notifier_register(OPAL_MSG_OCC, &powernv_cpufreq_opal_nb);
+ if (powernv_pstate_info.wof_enabled)
+ powernv_cpufreq_driver.boost_enabled = true;
+ else
+ powernv_cpu_freq_attr[SCALING_BOOST_FREQS_ATTR_INDEX] = NULL;
+
rc = cpufreq_register_driver(&powernv_cpufreq_driver);
- if (!rc)
- return 0;
+ if (rc) {
+ pr_info("Failed to register the cpufreq driver (%d)\n", rc);
+ goto cleanup_notifiers;
+ }
- pr_info("Failed to register the cpufreq driver (%d)\n", rc);
+ if (powernv_pstate_info.wof_enabled)
+ cpufreq_enable_boost_support();
+
+ return 0;
+cleanup_notifiers:
unregister_all_notifiers();
clean_chip_info();
out:
/* Should this really be called for CPUFREQ_ADJUST and CPUFREQ_NOTIFY
* policy events?)
*/
- if (event == CPUFREQ_START)
- return 0;
-
node = cbe_cpu_to_node(policy->cpu);
pr_debug("got notified, event=%lu, node=%u\n", event, node);
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include <linux/cpufreq.h>
#include <linux/cpu_cooling.h>
#include <linux/errno.h>
struct thermal_cooling_device *cdev;
};
+/*
+ * Don't use cpufreq on this SoC -- used when the SoC would have otherwise
+ * matched a more generic compatible.
+ */
+#define SOC_BLACKLIST 1
+
/**
* struct soc_data - SoC specific data
- * @freq_mask: mask the disallowed frequencies
- * @flag: unique flags
+ * @flags: SOC_xxx
*/
struct soc_data {
- u32 freq_mask[4];
- u32 flag;
-};
-
-#define FREQ_MASK 1
-/* see hardware specification for the allowed frqeuencies */
-static const struct soc_data sdata[] = {
- { /* used by p2041 and p3041 */
- .freq_mask = {0x8, 0x8, 0x2, 0x2},
- .flag = FREQ_MASK,
- },
- { /* used by p5020 */
- .freq_mask = {0x8, 0x2},
- .flag = FREQ_MASK,
- },
- { /* used by p4080, p5040 */
- .freq_mask = {0},
- .flag = 0,
- },
+ u32 flags;
};
-/*
- * the minimum allowed core frequency, in Hz
- * for chassis v1.0, >= platform frequency
- * for chassis v2.0, >= platform frequency / 2
- */
-static u32 min_cpufreq;
-static const u32 *fmask;
-
-#if defined(CONFIG_ARM)
-static int get_cpu_physical_id(int cpu)
-{
- return topology_core_id(cpu);
-}
-#else
-static int get_cpu_physical_id(int cpu)
-{
- return get_hard_smp_processor_id(cpu);
-}
-#endif
-
static u32 get_bus_freq(void)
{
struct device_node *soc;
return sysfreq;
}
-static struct device_node *cpu_to_clk_node(int cpu)
+static struct clk *cpu_to_clk(int cpu)
{
- struct device_node *np, *clk_np;
+ struct device_node *np;
+ struct clk *clk;
if (!cpu_present(cpu))
return NULL;
if (!np)
return NULL;
- clk_np = of_parse_phandle(np, "clocks", 0);
- if (!clk_np)
- return NULL;
-
+ clk = of_clk_get(np, 0);
of_node_put(np);
-
- return clk_np;
+ return clk;
}
/* traverse cpu nodes to get cpu mask of sharing clock wire */
static void set_affected_cpus(struct cpufreq_policy *policy)
{
- struct device_node *np, *clk_np;
struct cpumask *dstp = policy->cpus;
+ struct clk *clk;
int i;
- np = cpu_to_clk_node(policy->cpu);
- if (!np)
- return;
-
for_each_present_cpu(i) {
- clk_np = cpu_to_clk_node(i);
- if (!clk_np)
+ clk = cpu_to_clk(i);
+ if (IS_ERR(clk)) {
+ pr_err("%s: no clock for cpu %d\n", __func__, i);
continue;
+ }
- if (clk_np == np)
+ if (clk_is_match(policy->clk, clk))
cpumask_set_cpu(i, dstp);
-
- of_node_put(clk_np);
}
- of_node_put(np);
}
/* reduce the duplicated frequencies in frequency table */
static int qoriq_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
- struct device_node *np, *pnode;
+ struct device_node *np;
int i, count, ret;
- u32 freq, mask;
+ u32 freq;
struct clk *clk;
+ const struct clk_hw *hwclk;
struct cpufreq_frequency_table *table;
struct cpu_data *data;
unsigned int cpu = policy->cpu;
goto err_nomem2;
}
- pnode = of_parse_phandle(np, "clocks", 0);
- if (!pnode) {
- pr_err("%s: could not get clock information\n", __func__);
- goto err_nomem2;
- }
+ hwclk = __clk_get_hw(policy->clk);
+ count = clk_hw_get_num_parents(hwclk);
- count = of_property_count_strings(pnode, "clock-names");
data->pclk = kcalloc(count, sizeof(struct clk *), GFP_KERNEL);
if (!data->pclk) {
pr_err("%s: no memory\n", __func__);
- goto err_node;
+ goto err_nomem2;
}
table = kcalloc(count + 1, sizeof(*table), GFP_KERNEL);
goto err_pclk;
}
- if (fmask)
- mask = fmask[get_cpu_physical_id(cpu)];
- else
- mask = 0x0;
-
for (i = 0; i < count; i++) {
- clk = of_clk_get(pnode, i);
+ clk = clk_hw_get_parent_by_index(hwclk, i)->clk;
data->pclk[i] = clk;
freq = clk_get_rate(clk);
- /*
- * the clock is valid if its frequency is not masked
- * and large than minimum allowed frequency.
- */
- if (freq < min_cpufreq || (mask & (1 << i)))
- table[i].frequency = CPUFREQ_ENTRY_INVALID;
- else
- table[i].frequency = freq / 1000;
+ table[i].frequency = freq / 1000;
table[i].driver_data = i;
}
freq_table_redup(table, count);
policy->cpuinfo.transition_latency = u64temp + 1;
of_node_put(np);
- of_node_put(pnode);
return 0;
kfree(table);
err_pclk:
kfree(data->pclk);
-err_node:
- of_node_put(pnode);
err_nomem2:
- policy->driver_data = NULL;
kfree(data);
err_np:
of_node_put(np);
.attr = cpufreq_generic_attr,
};
+static const struct soc_data blacklist = {
+ .flags = SOC_BLACKLIST,
+};
+
static const struct of_device_id node_matches[] __initconst = {
- { .compatible = "fsl,p2041-clockgen", .data = &sdata[0], },
- { .compatible = "fsl,p3041-clockgen", .data = &sdata[0], },
- { .compatible = "fsl,p5020-clockgen", .data = &sdata[1], },
- { .compatible = "fsl,p4080-clockgen", .data = &sdata[2], },
- { .compatible = "fsl,p5040-clockgen", .data = &sdata[2], },
+ /* e6500 cannot use cpufreq due to erratum A-008083 */
+ { .compatible = "fsl,b4420-clockgen", &blacklist },
+ { .compatible = "fsl,b4860-clockgen", &blacklist },
+ { .compatible = "fsl,t2080-clockgen", &blacklist },
+ { .compatible = "fsl,t4240-clockgen", &blacklist },
+
+ { .compatible = "fsl,ls1012a-clockgen", },
+ { .compatible = "fsl,ls1021a-clockgen", },
+ { .compatible = "fsl,ls1043a-clockgen", },
+ { .compatible = "fsl,ls1046a-clockgen", },
+ { .compatible = "fsl,ls1088a-clockgen", },
+ { .compatible = "fsl,ls2080a-clockgen", },
+ { .compatible = "fsl,p4080-clockgen", },
+ { .compatible = "fsl,qoriq-clockgen-1.0", },
{ .compatible = "fsl,qoriq-clockgen-2.0", },
{}
};
match = of_match_node(node_matches, np);
data = match->data;
- if (data) {
- if (data->flag)
- fmask = data->freq_mask;
- min_cpufreq = get_bus_freq();
- } else {
- min_cpufreq = get_bus_freq() / 2;
- }
of_node_put(np);
+ if (data && data->flags & SOC_BLACKLIST)
+ return -ENODEV;
+
ret = cpufreq_register_driver(&qoriq_cpufreq_driver);
if (!ret)
pr_info("Freescale QorIQ CPU frequency scaling driver\n");
rate = clk_get_rate(s3c_freq->hclk);
if (rate < 133 * 1000 * 1000) {
pr_err("cpufreq: HCLK not at 133MHz\n");
- clk_put(s3c_freq->hclk);
ret = -EINVAL;
goto err_armclk;
}
int pcode, substrate, major, minor;
int ret;
char name[MAX_PCODE_NAME_LEN];
+ struct opp_table *opp_table;
reg_fields = sti_cpufreq_match();
if (!reg_fields) {
snprintf(name, MAX_PCODE_NAME_LEN, "pcode%d", pcode);
- ret = dev_pm_opp_set_prop_name(dev, name);
- if (ret) {
+ opp_table = dev_pm_opp_set_prop_name(dev, name);
+ if (IS_ERR(opp_table)) {
dev_err(dev, "Failed to set prop name\n");
- return ret;
+ return PTR_ERR(opp_table);
}
version[0] = BIT(major);
version[1] = BIT(minor);
version[2] = BIT(substrate);
- ret = dev_pm_opp_set_supported_hw(dev, version, VERSION_ELEMENTS);
- if (ret) {
+ opp_table = dev_pm_opp_set_supported_hw(dev, version, VERSION_ELEMENTS);
+ if (IS_ERR(opp_table)) {
dev_err(dev, "Failed to set supported hardware\n");
- return ret;
+ return PTR_ERR(opp_table);
}
dev_dbg(dev, "pcode: %d major: %d minor: %d substrate: %d\n",
--- /dev/null
+/*
+ * TI CPUFreq/OPP hw-supported driver
+ *
+ * Copyright (C) 2016-2017 Texas Instruments, Inc.
+ * Dave Gerlach <d-gerlach@ti.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/cpu.h>
+#include <linux/io.h>
+#include <linux/mfd/syscon.h>
+#include <linux/init.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/pm_opp.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+
+#define REVISION_MASK 0xF
+#define REVISION_SHIFT 28
+
+#define AM33XX_800M_ARM_MPU_MAX_FREQ 0x1E2F
+#define AM43XX_600M_ARM_MPU_MAX_FREQ 0xFFA
+
+#define DRA7_EFUSE_HAS_OD_MPU_OPP 11
+#define DRA7_EFUSE_HAS_HIGH_MPU_OPP 15
+#define DRA7_EFUSE_HAS_ALL_MPU_OPP 23
+
+#define DRA7_EFUSE_NOM_MPU_OPP BIT(0)
+#define DRA7_EFUSE_OD_MPU_OPP BIT(1)
+#define DRA7_EFUSE_HIGH_MPU_OPP BIT(2)
+
+#define VERSION_COUNT 2
+
+struct ti_cpufreq_data;
+
+struct ti_cpufreq_soc_data {
+ unsigned long (*efuse_xlate)(struct ti_cpufreq_data *opp_data,
+ unsigned long efuse);
+ unsigned long efuse_fallback;
+ unsigned long efuse_offset;
+ unsigned long efuse_mask;
+ unsigned long efuse_shift;
+ unsigned long rev_offset;
+};
+
+struct ti_cpufreq_data {
+ struct device *cpu_dev;
+ struct device_node *opp_node;
+ struct regmap *syscon;
+ const struct ti_cpufreq_soc_data *soc_data;
+};
+
+static unsigned long amx3_efuse_xlate(struct ti_cpufreq_data *opp_data,
+ unsigned long efuse)
+{
+ if (!efuse)
+ efuse = opp_data->soc_data->efuse_fallback;
+ /* AM335x and AM437x use "OPP disable" bits, so invert */
+ return ~efuse;
+}
+
+static unsigned long dra7_efuse_xlate(struct ti_cpufreq_data *opp_data,
+ unsigned long efuse)
+{
+ unsigned long calculated_efuse = DRA7_EFUSE_NOM_MPU_OPP;
+
+ /*
+ * The efuse on dra7 and am57 parts contains a specific
+ * value indicating the highest available OPP.
+ */
+
+ switch (efuse) {
+ case DRA7_EFUSE_HAS_ALL_MPU_OPP:
+ case DRA7_EFUSE_HAS_HIGH_MPU_OPP:
+ calculated_efuse |= DRA7_EFUSE_HIGH_MPU_OPP;
+ case DRA7_EFUSE_HAS_OD_MPU_OPP:
+ calculated_efuse |= DRA7_EFUSE_OD_MPU_OPP;
+ }
+
+ return calculated_efuse;
+}
+
+static struct ti_cpufreq_soc_data am3x_soc_data = {
+ .efuse_xlate = amx3_efuse_xlate,
+ .efuse_fallback = AM33XX_800M_ARM_MPU_MAX_FREQ,
+ .efuse_offset = 0x07fc,
+ .efuse_mask = 0x1fff,
+ .rev_offset = 0x600,
+};
+
+static struct ti_cpufreq_soc_data am4x_soc_data = {
+ .efuse_xlate = amx3_efuse_xlate,
+ .efuse_fallback = AM43XX_600M_ARM_MPU_MAX_FREQ,
+ .efuse_offset = 0x0610,
+ .efuse_mask = 0x3f,
+ .rev_offset = 0x600,
+};
+
+static struct ti_cpufreq_soc_data dra7_soc_data = {
+ .efuse_xlate = dra7_efuse_xlate,
+ .efuse_offset = 0x020c,
+ .efuse_mask = 0xf80000,
+ .efuse_shift = 19,
+ .rev_offset = 0x204,
+};
+
+/**
+ * ti_cpufreq_get_efuse() - Parse and return efuse value present on SoC
+ * @opp_data: pointer to ti_cpufreq_data context
+ * @efuse_value: Set to the value parsed from efuse
+ *
+ * Returns error code if efuse not read properly.
+ */
+static int ti_cpufreq_get_efuse(struct ti_cpufreq_data *opp_data,
+ u32 *efuse_value)
+{
+ struct device *dev = opp_data->cpu_dev;
+ u32 efuse;
+ int ret;
+
+ ret = regmap_read(opp_data->syscon, opp_data->soc_data->efuse_offset,
+ &efuse);
+ if (ret) {
+ dev_err(dev,
+ "Failed to read the efuse value from syscon: %d\n",
+ ret);
+ return ret;
+ }
+
+ efuse = (efuse & opp_data->soc_data->efuse_mask);
+ efuse >>= opp_data->soc_data->efuse_shift;
+
+ *efuse_value = opp_data->soc_data->efuse_xlate(opp_data, efuse);
+
+ return 0;
+}
+
+/**
+ * ti_cpufreq_get_rev() - Parse and return rev value present on SoC
+ * @opp_data: pointer to ti_cpufreq_data context
+ * @revision_value: Set to the value parsed from revision register
+ *
+ * Returns error code if revision not read properly.
+ */
+static int ti_cpufreq_get_rev(struct ti_cpufreq_data *opp_data,
+ u32 *revision_value)
+{
+ struct device *dev = opp_data->cpu_dev;
+ u32 revision;
+ int ret;
+
+ ret = regmap_read(opp_data->syscon, opp_data->soc_data->rev_offset,
+ &revision);
+ if (ret) {
+ dev_err(dev,
+ "Failed to read the revision number from syscon: %d\n",
+ ret);
+ return ret;
+ }
+
+ *revision_value = BIT((revision >> REVISION_SHIFT) & REVISION_MASK);
+
+ return 0;
+}
+
+static int ti_cpufreq_setup_syscon_register(struct ti_cpufreq_data *opp_data)
+{
+ struct device *dev = opp_data->cpu_dev;
+ struct device_node *np = opp_data->opp_node;
+
+ opp_data->syscon = syscon_regmap_lookup_by_phandle(np,
+ "syscon");
+ if (IS_ERR(opp_data->syscon)) {
+ dev_err(dev,
+ "\"syscon\" is missing, cannot use OPPv2 table.\n");
+ return PTR_ERR(opp_data->syscon);
+ }
+
+ return 0;
+}
+
+static const struct of_device_id ti_cpufreq_of_match[] = {
+ { .compatible = "ti,am33xx", .data = &am3x_soc_data, },
+ { .compatible = "ti,am4372", .data = &am4x_soc_data, },
+ { .compatible = "ti,dra7", .data = &dra7_soc_data },
+ {},
+};
+
+static int ti_cpufreq_init(void)
+{
+ u32 version[VERSION_COUNT];
+ struct device_node *np;
+ const struct of_device_id *match;
+ struct ti_cpufreq_data *opp_data;
+ int ret;
+
+ np = of_find_node_by_path("/");
+ match = of_match_node(ti_cpufreq_of_match, np);
+ if (!match)
+ return -ENODEV;
+
+ opp_data = kzalloc(sizeof(*opp_data), GFP_KERNEL);
+ if (!opp_data)
+ return -ENOMEM;
+
+ opp_data->soc_data = match->data;
+
+ opp_data->cpu_dev = get_cpu_device(0);
+ if (!opp_data->cpu_dev) {
+ pr_err("%s: Failed to get device for CPU0\n", __func__);
+ return -ENODEV;
+ }
+
+ opp_data->opp_node = dev_pm_opp_of_get_opp_desc_node(opp_data->cpu_dev);
+ if (!opp_data->opp_node) {
+ dev_info(opp_data->cpu_dev,
+ "OPP-v2 not supported, cpufreq-dt will attempt to use legacy tables.\n");
+ goto register_cpufreq_dt;
+ }
+
+ ret = ti_cpufreq_setup_syscon_register(opp_data);
+ if (ret)
+ goto fail_put_node;
+
+ /*
+ * OPPs determine whether or not they are supported based on
+ * two metrics:
+ * 0 - SoC Revision
+ * 1 - eFuse value
+ */
+ ret = ti_cpufreq_get_rev(opp_data, &version[0]);
+ if (ret)
+ goto fail_put_node;
+
+ ret = ti_cpufreq_get_efuse(opp_data, &version[1]);
+ if (ret)
+ goto fail_put_node;
+
+ of_node_put(opp_data->opp_node);
+
+ ret = PTR_ERR_OR_ZERO(dev_pm_opp_set_supported_hw(opp_data->cpu_dev,
+ version, VERSION_COUNT));
+ if (ret) {
+ dev_err(opp_data->cpu_dev,
+ "Failed to set supported hardware\n");
+ goto fail_put_node;
+ }
+
+register_cpufreq_dt:
+ platform_device_register_simple("cpufreq-dt", -1, NULL, 0);
+
+ return 0;
+
+fail_put_node:
+ of_node_put(opp_data->opp_node);
+
+ return ret;
+}
+device_initcall(ti_cpufreq_init);
#include <linux/tick.h>
#include <linux/sched.h>
#include <linux/math64.h>
+#include <linux/cpu.h>
/*
* Please note when changing the tuning values:
static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev)
{
struct menu_device *data = this_cpu_ptr(&menu_devices);
+ struct device *device = get_cpu_device(dev->cpu);
int latency_req = pm_qos_request(PM_QOS_CPU_DMA_LATENCY);
int i;
unsigned int interactivity_req;
unsigned int expected_interval;
unsigned long nr_iowaiters, cpu_load;
+ int resume_latency = dev_pm_qos_read_value(device);
if (data->needs_update) {
menu_update(drv, dev);
data->needs_update = 0;
}
+ /* resume_latency is 0 means no restriction */
+ if (resume_latency && resume_latency < latency_req)
+ latency_req = resume_latency;
+
/* Special case when user has set very strict latency requirement */
if (unlikely(latency_req == 0))
return 0;
if (s->disabled || su->disable)
continue;
if (s->target_residency > data->predicted_us)
- continue;
+ break;
if (s->exit_latency > latency_req)
- continue;
+ break;
data->last_state_idx = i;
}
static void ccp5_config(struct ccp_device *ccp)
{
/* Public side */
- iowrite32(0x00001249, ccp->io_regs + CMD5_REQID_CONFIG_OFFSET);
+ iowrite32(0x0, ccp->io_regs + CMD5_REQID_CONFIG_OFFSET);
}
static void ccp5other_config(struct ccp_device *ccp)
struct ccp_device *ccp;
spinlock_t lock;
+ struct list_head created;
struct list_head pending;
struct list_head active;
struct list_head complete;
ccp_free_desc_resources(chan->ccp, &chan->complete);
ccp_free_desc_resources(chan->ccp, &chan->active);
ccp_free_desc_resources(chan->ccp, &chan->pending);
+ ccp_free_desc_resources(chan->ccp, &chan->created);
spin_unlock_irqrestore(&chan->lock, flags);
}
spin_lock_irqsave(&chan->lock, flags);
cookie = dma_cookie_assign(tx_desc);
+ list_del(&desc->entry);
list_add_tail(&desc->entry, &chan->pending);
spin_unlock_irqrestore(&chan->lock, flags);
spin_lock_irqsave(&chan->lock, sflags);
- list_add_tail(&desc->entry, &chan->pending);
+ list_add_tail(&desc->entry, &chan->created);
spin_unlock_irqrestore(&chan->lock, sflags);
/*TODO: Purge the complete list? */
ccp_free_desc_resources(chan->ccp, &chan->active);
ccp_free_desc_resources(chan->ccp, &chan->pending);
+ ccp_free_desc_resources(chan->ccp, &chan->created);
spin_unlock_irqrestore(&chan->lock, flags);
chan->ccp = ccp;
spin_lock_init(&chan->lock);
+ INIT_LIST_HEAD(&chan->created);
INIT_LIST_HEAD(&chan->pending);
INIT_LIST_HEAD(&chan->active);
INIT_LIST_HEAD(&chan->complete);
case CRYPTO_ALG_TYPE_AEAD:
ctx_req.req.aead_req = (struct aead_request *)req;
ctx_req.ctx.reqctx = aead_request_ctx(ctx_req.req.aead_req);
- dma_unmap_sg(&u_ctx->lldi.pdev->dev, ctx_req.req.aead_req->dst,
+ dma_unmap_sg(&u_ctx->lldi.pdev->dev, ctx_req.ctx.reqctx->dst,
ctx_req.ctx.reqctx->dst_nents, DMA_FROM_DEVICE);
if (ctx_req.ctx.reqctx->skb) {
kfree_skb(ctx_req.ctx.reqctx->skb);
struct chcr_wr *chcr_req;
struct cpl_rx_phys_dsgl *phys_cpl;
struct phys_sge_parm sg_param;
- struct scatterlist *src, *dst;
- struct scatterlist src_sg[2], dst_sg[2];
+ struct scatterlist *src;
unsigned int frags = 0, transhdr_len;
unsigned int ivsize = crypto_aead_ivsize(tfm), dst_size = 0;
unsigned int kctx_len = 0;
if (sg_nents_for_len(req->src, req->assoclen + req->cryptlen) < 0)
goto err;
- src = scatterwalk_ffwd(src_sg, req->src, req->assoclen);
- dst = src;
+ src = scatterwalk_ffwd(reqctx->srcffwd, req->src, req->assoclen);
+ reqctx->dst = src;
+
if (req->src != req->dst) {
err = chcr_copy_assoc(req, aeadctx);
if (err)
return ERR_PTR(err);
- dst = scatterwalk_ffwd(dst_sg, req->dst, req->assoclen);
+ reqctx->dst = scatterwalk_ffwd(reqctx->dstffwd, req->dst,
+ req->assoclen);
}
if (get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_NULL) {
null = 1;
assoclen = 0;
}
- reqctx->dst_nents = sg_nents_for_len(dst, req->cryptlen +
+ reqctx->dst_nents = sg_nents_for_len(reqctx->dst, req->cryptlen +
(op_type ? -authsize : authsize));
if (reqctx->dst_nents <= 0) {
pr_err("AUTHENC:Invalid Destination sg entries\n");
sg_param.obsize = req->cryptlen + (op_type ? -authsize : authsize);
sg_param.qid = qid;
sg_param.align = 0;
- if (map_writesg_phys_cpl(&u_ctx->lldi.pdev->dev, phys_cpl, dst,
+ if (map_writesg_phys_cpl(&u_ctx->lldi.pdev->dev, phys_cpl, reqctx->dst,
&sg_param))
goto dstmap_fail;
struct chcr_wr *chcr_req;
struct cpl_rx_phys_dsgl *phys_cpl;
struct phys_sge_parm sg_param;
- struct scatterlist *src, *dst;
- struct scatterlist src_sg[2], dst_sg[2];
+ struct scatterlist *src;
unsigned int frags = 0, transhdr_len, ivsize = AES_BLOCK_SIZE;
unsigned int dst_size = 0, kctx_len;
unsigned int sub_type;
if (sg_nents_for_len(req->src, req->assoclen + req->cryptlen) < 0)
goto err;
sub_type = get_aead_subtype(tfm);
- src = scatterwalk_ffwd(src_sg, req->src, req->assoclen);
- dst = src;
+ src = scatterwalk_ffwd(reqctx->srcffwd, req->src, req->assoclen);
+ reqctx->dst = src;
+
if (req->src != req->dst) {
err = chcr_copy_assoc(req, aeadctx);
if (err) {
pr_err("AAD copy to destination buffer fails\n");
return ERR_PTR(err);
}
- dst = scatterwalk_ffwd(dst_sg, req->dst, req->assoclen);
+ reqctx->dst = scatterwalk_ffwd(reqctx->dstffwd, req->dst,
+ req->assoclen);
}
- reqctx->dst_nents = sg_nents_for_len(dst, req->cryptlen +
+ reqctx->dst_nents = sg_nents_for_len(reqctx->dst, req->cryptlen +
(op_type ? -authsize : authsize));
if (reqctx->dst_nents <= 0) {
pr_err("CCM:Invalid Destination sg entries\n");
sg_param.obsize = req->cryptlen + (op_type ? -authsize : authsize);
sg_param.qid = qid;
sg_param.align = 0;
- if (map_writesg_phys_cpl(&u_ctx->lldi.pdev->dev, phys_cpl, dst,
+ if (map_writesg_phys_cpl(&u_ctx->lldi.pdev->dev, phys_cpl, reqctx->dst,
&sg_param))
goto dstmap_fail;
struct chcr_wr *chcr_req;
struct cpl_rx_phys_dsgl *phys_cpl;
struct phys_sge_parm sg_param;
- struct scatterlist *src, *dst;
- struct scatterlist src_sg[2], dst_sg[2];
+ struct scatterlist *src;
unsigned int frags = 0, transhdr_len;
unsigned int ivsize = AES_BLOCK_SIZE;
unsigned int dst_size = 0, kctx_len;
if (sg_nents_for_len(req->src, req->assoclen + req->cryptlen) < 0)
goto err;
- src = scatterwalk_ffwd(src_sg, req->src, req->assoclen);
- dst = src;
+ src = scatterwalk_ffwd(reqctx->srcffwd, req->src, req->assoclen);
+ reqctx->dst = src;
if (req->src != req->dst) {
err = chcr_copy_assoc(req, aeadctx);
if (err)
return ERR_PTR(err);
- dst = scatterwalk_ffwd(dst_sg, req->dst, req->assoclen);
+ reqctx->dst = scatterwalk_ffwd(reqctx->dstffwd, req->dst,
+ req->assoclen);
}
if (!req->cryptlen)
crypt_len = AES_BLOCK_SIZE;
else
crypt_len = req->cryptlen;
- reqctx->dst_nents = sg_nents_for_len(dst, req->cryptlen +
+ reqctx->dst_nents = sg_nents_for_len(reqctx->dst, req->cryptlen +
(op_type ? -authsize : authsize));
if (reqctx->dst_nents <= 0) {
pr_err("GCM:Invalid Destination sg entries\n");
sg_param.obsize = req->cryptlen + (op_type ? -authsize : authsize);
sg_param.qid = qid;
sg_param.align = 0;
- if (map_writesg_phys_cpl(&u_ctx->lldi.pdev->dev, phys_cpl, dst,
+ if (map_writesg_phys_cpl(&u_ctx->lldi.pdev->dev, phys_cpl, reqctx->dst,
&sg_param))
goto dstmap_fail;
write_sg_to_skb(skb, &frags, src, req->cryptlen);
} else {
aes_gcm_empty_pld_pad(req->dst, authsize - 1);
- write_sg_to_skb(skb, &frags, dst, crypt_len);
+ write_sg_to_skb(skb, &frags, reqctx->dst, crypt_len);
+
}
create_wreq(ctx, chcr_req, req, skb, kctx_len, size, 1,
unsigned int ck_size;
int ret = 0, key_ctx_size = 0;
- if (get_aead_subtype(aead) ==
- CRYPTO_ALG_SUB_TYPE_AEAD_RFC4106) {
+ if (get_aead_subtype(aead) == CRYPTO_ALG_SUB_TYPE_AEAD_RFC4106 &&
+ keylen > 3) {
keylen -= 4; /* nonce/salt is present in the last 4 bytes */
memcpy(aeadctx->salt, key + keylen, 4);
}
int assign_chcr_device(struct chcr_dev **dev)
{
struct uld_ctx *u_ctx;
+ int ret = -ENXIO;
/*
* Which device to use if multiple devices are available TODO
* must go to the same device to maintain the ordering.
*/
mutex_lock(&dev_mutex); /* TODO ? */
- u_ctx = list_first_entry(&uld_ctx_list, struct uld_ctx, entry);
- if (!u_ctx) {
- mutex_unlock(&dev_mutex);
- return -ENXIO;
+ list_for_each_entry(u_ctx, &uld_ctx_list, entry)
+ if (u_ctx && u_ctx->dev) {
+ *dev = u_ctx->dev;
+ ret = 0;
+ break;
}
-
- *dev = u_ctx->dev;
mutex_unlock(&dev_mutex);
- return 0;
+ return ret;
}
static int chcr_dev_add(struct uld_ctx *u_ctx)
static int __init chcr_crypto_init(void)
{
- if (cxgb4_register_uld(CXGB4_ULD_CRYPTO, &chcr_uld_info)) {
+ if (cxgb4_register_uld(CXGB4_ULD_CRYPTO, &chcr_uld_info))
pr_err("ULD register fail: No chcr crypto support in cxgb4");
- return -1;
- }
return 0;
}
};
struct chcr_aead_reqctx {
struct sk_buff *skb;
+ struct scatterlist *dst;
+ struct scatterlist srcffwd[2];
+ struct scatterlist dstffwd[2];
short int dst_nents;
u16 verify;
u8 iv[CHCR_MAX_CRYPTO_IV_LEN];
&hw_data->accel_capabilities_mask);
/* Find and map all the device's BARS */
- i = 0;
+ i = (hw_data->fuses & ADF_DEVICE_FUSECTL_MASK) ? 1 : 0;
bar_mask = pci_select_bars(pdev, IORESOURCE_MEM);
for_each_set_bit(bar_nr, (const unsigned long *)&bar_mask,
ADF_PCI_MAX_BARS * 2) {
#define ADF_ERRSOU5 (0x3A000 + 0xD8)
#define ADF_DEVICE_FUSECTL_OFFSET 0x40
#define ADF_DEVICE_LEGFUSE_OFFSET 0x4C
+#define ADF_DEVICE_FUSECTL_MASK 0x80000000
#define ADF_PCI_MAX_BARS 3
#define ADF_DEVICE_NAME_LENGTH 32
#define ADF_ETR_MAX_RINGS_PER_BANK 16
unsigned int csr_val;
int times = 30;
- if (handle->pci_dev->device == ADF_C3XXX_PCI_DEVICE_ID)
+ if (handle->pci_dev->device != ADF_DH895XCC_PCI_DEVICE_ID)
return 0;
csr_val = ADF_CSR_RD(csr_addr, 0);
(void __iomem *)((uintptr_t)handle->hal_cap_ae_xfer_csr_addr_v +
LOCAL_TO_XFER_REG_OFFSET);
handle->pci_dev = pci_info->pci_dev;
- if (handle->pci_dev->device != ADF_C3XXX_PCI_DEVICE_ID) {
+ if (handle->pci_dev->device == ADF_DH895XCC_PCI_DEVICE_ID) {
sram_bar =
&pci_info->pci_bars[hw_data->get_sram_bar_id(hw_data)];
handle->hal_sram_addr_v = sram_bar->virt_addr;
struct devfreq_event_desc *desc)
{
struct devfreq_event_dev *edev;
- static atomic_t event_no = ATOMIC_INIT(0);
+ static atomic_t event_no = ATOMIC_INIT(-1);
int ret;
if (!dev || !desc)
edev->dev.class = devfreq_event_class;
edev->dev.release = devfreq_event_release_edev;
- dev_set_name(&edev->dev, "event.%d", atomic_inc_return(&event_no) - 1);
+ dev_set_name(&edev->dev, "event%d", atomic_inc_return(&event_no));
ret = device_register(&edev->dev);
if (ret < 0) {
put_device(&edev->dev);
return;
}
- rcu_read_lock();
for (i = 0, freq = 0; i < profile->max_state; i++, freq++) {
opp = dev_pm_opp_find_freq_ceil(devfreq->dev.parent, &freq);
if (IS_ERR(opp)) {
devm_kfree(devfreq->dev.parent, profile->freq_table);
profile->max_state = 0;
- rcu_read_unlock();
return;
}
+ dev_pm_opp_put(opp);
profile->freq_table[i] = freq;
}
- rcu_read_unlock();
}
/**
* @devfreq: the devfreq instance
* @freq: the update target frequency
*/
-static int devfreq_update_status(struct devfreq *devfreq, unsigned long freq)
+int devfreq_update_status(struct devfreq *devfreq, unsigned long freq)
{
int lev, prev_lev, ret = 0;
unsigned long cur_time;
devfreq->last_stat_updated = cur_time;
return ret;
}
+EXPORT_SYMBOL(devfreq_update_status);
/**
* find_devfreq_governor() - find devfreq governor from name
}
/**
- * _remove_devfreq() - Remove devfreq from the list and release its resources.
- * @devfreq: the devfreq struct
+ * devfreq_dev_release() - Callback for struct device to release the device.
+ * @dev: the devfreq device
+ *
+ * Remove devfreq from the list and release its resources.
*/
-static void _remove_devfreq(struct devfreq *devfreq)
+static void devfreq_dev_release(struct device *dev)
{
+ struct devfreq *devfreq = to_devfreq(dev);
+
mutex_lock(&devfreq_list_lock);
if (IS_ERR(find_device_devfreq(devfreq->dev.parent))) {
mutex_unlock(&devfreq_list_lock);
kfree(devfreq);
}
-/**
- * devfreq_dev_release() - Callback for struct device to release the device.
- * @dev: the devfreq device
- *
- * This calls _remove_devfreq() if _remove_devfreq() is not called.
- */
-static void devfreq_dev_release(struct device *dev)
-{
- struct devfreq *devfreq = to_devfreq(dev);
-
- _remove_devfreq(devfreq);
-}
-
/**
* devfreq_add_device() - Add devfreq feature to the device
* @dev: the device to add devfreq feature.
{
struct devfreq *devfreq;
struct devfreq_governor *governor;
+ static atomic_t devfreq_no = ATOMIC_INIT(-1);
int err = 0;
if (!dev || !profile || !governor_name) {
devfreq = find_device_devfreq(dev);
mutex_unlock(&devfreq_list_lock);
if (!IS_ERR(devfreq)) {
- dev_err(dev, "%s: Unable to create devfreq for the device. It already has one.\n", __func__);
+ dev_err(dev, "%s: Unable to create devfreq for the device.\n",
+ __func__);
err = -EINVAL;
goto err_out;
}
devfreq = kzalloc(sizeof(struct devfreq), GFP_KERNEL);
if (!devfreq) {
- dev_err(dev, "%s: Unable to create devfreq for the device\n",
- __func__);
err = -ENOMEM;
goto err_out;
}
mutex_lock(&devfreq->lock);
}
- dev_set_name(&devfreq->dev, "%s", dev_name(dev));
+ dev_set_name(&devfreq->dev, "devfreq%d",
+ atomic_inc_return(&devfreq_no));
err = device_register(&devfreq->dev);
if (err) {
mutex_unlock(&devfreq->lock);
goto err_out;
}
- devfreq->trans_table = devm_kzalloc(&devfreq->dev, sizeof(unsigned int) *
+ devfreq->trans_table = devm_kzalloc(&devfreq->dev,
+ sizeof(unsigned int) *
devfreq->profile->max_state *
devfreq->profile->max_state,
GFP_KERNEL);
- devfreq->time_in_state = devm_kzalloc(&devfreq->dev, sizeof(unsigned long) *
+ devfreq->time_in_state = devm_kzalloc(&devfreq->dev,
+ sizeof(unsigned long) *
devfreq->profile->max_state,
GFP_KERNEL);
devfreq->last_stat_updated = jiffies;
if (df->governor == governor) {
ret = 0;
goto out;
+ } else if (df->governor->immutable || governor->immutable) {
+ ret = -EINVAL;
+ goto out;
}
if (df->governor) {
struct device_attribute *attr,
char *buf)
{
- struct devfreq_governor *tmp_governor;
+ struct devfreq *df = to_devfreq(d);
ssize_t count = 0;
mutex_lock(&devfreq_list_lock);
- list_for_each_entry(tmp_governor, &devfreq_governor_list, node)
- count += scnprintf(&buf[count], (PAGE_SIZE - count - 2),
- "%s ", tmp_governor->name);
+
+ /*
+ * The devfreq with immutable governor (e.g., passive) shows
+ * only own governor.
+ */
+ if (df->governor->immutable) {
+ count = scnprintf(&buf[count], DEVFREQ_NAME_LEN,
+ "%s ", df->governor_name);
+ /*
+ * The devfreq device shows the registered governor except for
+ * immutable governors such as passive governor .
+ */
+ } else {
+ struct devfreq_governor *governor;
+
+ list_for_each_entry(governor, &devfreq_governor_list, node) {
+ if (governor->immutable)
+ continue;
+ count += scnprintf(&buf[count], (PAGE_SIZE - count - 2),
+ "%s ", governor->name);
+ }
+ }
+
mutex_unlock(&devfreq_list_lock);
/* Truncate the trailing space */
if (devfreq->profile->get_cur_freq &&
!devfreq->profile->get_cur_freq(devfreq->dev.parent, &freq))
- return sprintf(buf, "%lu\n", freq);
+ return sprintf(buf, "%lu\n", freq);
return sprintf(buf, "%lu\n", devfreq->previous_freq);
}
ssize_t count = 0;
unsigned long freq = 0;
- rcu_read_lock();
do {
opp = dev_pm_opp_find_freq_ceil(dev, &freq);
if (IS_ERR(opp))
break;
+ dev_pm_opp_put(opp);
count += scnprintf(&buf[count], (PAGE_SIZE - count - 2),
"%lu ", freq);
freq++;
} while (1);
- rcu_read_unlock();
/* Truncate the trailing space */
if (count)
* @freq: The frequency given to target function
* @flags: Flags handed from devfreq framework.
*
- * Locking: This function must be called under rcu_read_lock(). opp is a rcu
- * protected pointer. The reason for the same is that the opp pointer which is
- * returned will remain valid for use with opp_get_{voltage, freq} only while
- * under the locked area. The pointer returned must be used prior to unlocking
- * with rcu_read_unlock() to maintain the integrity of the pointer.
+ * The callers are required to call dev_pm_opp_put() for the returned OPP after
+ * use.
*/
struct dev_pm_opp *devfreq_recommended_opp(struct device *dev,
unsigned long *freq,
*/
int devfreq_register_opp_notifier(struct device *dev, struct devfreq *devfreq)
{
- struct srcu_notifier_head *nh;
- int ret = 0;
-
- rcu_read_lock();
- nh = dev_pm_opp_get_notifier(dev);
- if (IS_ERR(nh))
- ret = PTR_ERR(nh);
- rcu_read_unlock();
- if (!ret)
- ret = srcu_notifier_chain_register(nh, &devfreq->nb);
-
- return ret;
+ return dev_pm_opp_register_notifier(dev, &devfreq->nb);
}
EXPORT_SYMBOL(devfreq_register_opp_notifier);
*/
int devfreq_unregister_opp_notifier(struct device *dev, struct devfreq *devfreq)
{
- struct srcu_notifier_head *nh;
- int ret = 0;
-
- rcu_read_lock();
- nh = dev_pm_opp_get_notifier(dev);
- if (IS_ERR(nh))
- ret = PTR_ERR(nh);
- rcu_read_unlock();
- if (!ret)
- ret = srcu_notifier_chain_unregister(nh, &devfreq->nb);
-
- return ret;
+ return dev_pm_opp_unregister_notifier(dev, &devfreq->nb);
}
EXPORT_SYMBOL(devfreq_unregister_opp_notifier);
#include <linux/module.h>
#include <linux/of_address.h>
#include <linux/platform_device.h>
+#include <linux/regmap.h>
#include <linux/suspend.h>
#include <linux/devfreq-event.h>
#include "exynos-ppmu.h"
struct exynos_ppmu_data {
- void __iomem *base;
struct clk *clk;
};
unsigned int num_events;
struct device *dev;
+ struct regmap *regmap;
struct exynos_ppmu_data ppmu;
};
static int exynos_ppmu_disable(struct devfreq_event_dev *edev)
{
struct exynos_ppmu *info = devfreq_event_get_drvdata(edev);
+ int ret;
u32 pmnc;
/* Disable all counters */
- __raw_writel(PPMU_CCNT_MASK |
- PPMU_PMCNT0_MASK |
- PPMU_PMCNT1_MASK |
- PPMU_PMCNT2_MASK |
- PPMU_PMCNT3_MASK,
- info->ppmu.base + PPMU_CNTENC);
+ ret = regmap_write(info->regmap, PPMU_CNTENC,
+ PPMU_CCNT_MASK |
+ PPMU_PMCNT0_MASK |
+ PPMU_PMCNT1_MASK |
+ PPMU_PMCNT2_MASK |
+ PPMU_PMCNT3_MASK);
+ if (ret < 0)
+ return ret;
/* Disable PPMU */
- pmnc = __raw_readl(info->ppmu.base + PPMU_PMNC);
+ ret = regmap_read(info->regmap, PPMU_PMNC, &pmnc);
+ if (ret < 0)
+ return ret;
+
pmnc &= ~PPMU_PMNC_ENABLE_MASK;
- __raw_writel(pmnc, info->ppmu.base + PPMU_PMNC);
+ ret = regmap_write(info->regmap, PPMU_PMNC, pmnc);
+ if (ret < 0)
+ return ret;
return 0;
}
{
struct exynos_ppmu *info = devfreq_event_get_drvdata(edev);
int id = exynos_ppmu_find_ppmu_id(edev);
+ int ret;
u32 pmnc, cntens;
if (id < 0)
return id;
/* Enable specific counter */
- cntens = __raw_readl(info->ppmu.base + PPMU_CNTENS);
+ ret = regmap_read(info->regmap, PPMU_CNTENS, &cntens);
+ if (ret < 0)
+ return ret;
+
cntens |= (PPMU_CCNT_MASK | (PPMU_ENABLE << id));
- __raw_writel(cntens, info->ppmu.base + PPMU_CNTENS);
+ ret = regmap_write(info->regmap, PPMU_CNTENS, cntens);
+ if (ret < 0)
+ return ret;
/* Set the event of Read/Write data count */
- __raw_writel(PPMU_RO_DATA_CNT | PPMU_WO_DATA_CNT,
- info->ppmu.base + PPMU_BEVTxSEL(id));
+ ret = regmap_write(info->regmap, PPMU_BEVTxSEL(id),
+ PPMU_RO_DATA_CNT | PPMU_WO_DATA_CNT);
+ if (ret < 0)
+ return ret;
/* Reset cycle counter/performance counter and enable PPMU */
- pmnc = __raw_readl(info->ppmu.base + PPMU_PMNC);
+ ret = regmap_read(info->regmap, PPMU_PMNC, &pmnc);
+ if (ret < 0)
+ return ret;
+
pmnc &= ~(PPMU_PMNC_ENABLE_MASK
| PPMU_PMNC_COUNTER_RESET_MASK
| PPMU_PMNC_CC_RESET_MASK);
pmnc |= (PPMU_ENABLE << PPMU_PMNC_ENABLE_SHIFT);
pmnc |= (PPMU_ENABLE << PPMU_PMNC_COUNTER_RESET_SHIFT);
pmnc |= (PPMU_ENABLE << PPMU_PMNC_CC_RESET_SHIFT);
- __raw_writel(pmnc, info->ppmu.base + PPMU_PMNC);
+ ret = regmap_write(info->regmap, PPMU_PMNC, pmnc);
+ if (ret < 0)
+ return ret;
return 0;
}
{
struct exynos_ppmu *info = devfreq_event_get_drvdata(edev);
int id = exynos_ppmu_find_ppmu_id(edev);
- u32 pmnc, cntenc;
+ unsigned int total_count, load_count;
+ unsigned int pmcnt3_high, pmcnt3_low;
+ unsigned int pmnc, cntenc;
+ int ret;
if (id < 0)
return -EINVAL;
/* Disable PPMU */
- pmnc = __raw_readl(info->ppmu.base + PPMU_PMNC);
+ ret = regmap_read(info->regmap, PPMU_PMNC, &pmnc);
+ if (ret < 0)
+ return ret;
+
pmnc &= ~PPMU_PMNC_ENABLE_MASK;
- __raw_writel(pmnc, info->ppmu.base + PPMU_PMNC);
+ ret = regmap_write(info->regmap, PPMU_PMNC, pmnc);
+ if (ret < 0)
+ return ret;
/* Read cycle count */
- edata->total_count = __raw_readl(info->ppmu.base + PPMU_CCNT);
+ ret = regmap_read(info->regmap, PPMU_CCNT, &total_count);
+ if (ret < 0)
+ return ret;
+ edata->total_count = total_count;
/* Read performance count */
switch (id) {
case PPMU_PMNCNT0:
case PPMU_PMNCNT1:
case PPMU_PMNCNT2:
- edata->load_count
- = __raw_readl(info->ppmu.base + PPMU_PMNCT(id));
+ ret = regmap_read(info->regmap, PPMU_PMNCT(id), &load_count);
+ if (ret < 0)
+ return ret;
+ edata->load_count = load_count;
break;
case PPMU_PMNCNT3:
- edata->load_count =
- ((__raw_readl(info->ppmu.base + PPMU_PMCNT3_HIGH) << 8)
- | __raw_readl(info->ppmu.base + PPMU_PMCNT3_LOW));
+ ret = regmap_read(info->regmap, PPMU_PMCNT3_HIGH, &pmcnt3_high);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_read(info->regmap, PPMU_PMCNT3_LOW, &pmcnt3_low);
+ if (ret < 0)
+ return ret;
+
+ edata->load_count = ((pmcnt3_high << 8) | pmcnt3_low);
break;
default:
return -EINVAL;
}
/* Disable specific counter */
- cntenc = __raw_readl(info->ppmu.base + PPMU_CNTENC);
+ ret = regmap_read(info->regmap, PPMU_CNTENC, &cntenc);
+ if (ret < 0)
+ return ret;
+
cntenc |= (PPMU_CCNT_MASK | (PPMU_ENABLE << id));
- __raw_writel(cntenc, info->ppmu.base + PPMU_CNTENC);
+ ret = regmap_write(info->regmap, PPMU_CNTENC, cntenc);
+ if (ret < 0)
+ return ret;
dev_dbg(&edev->dev, "%s (event: %ld/%ld)\n", edev->desc->name,
edata->load_count, edata->total_count);
static int exynos_ppmu_v2_disable(struct devfreq_event_dev *edev)
{
struct exynos_ppmu *info = devfreq_event_get_drvdata(edev);
+ int ret;
u32 pmnc, clear;
/* Disable all counters */
clear = (PPMU_CCNT_MASK | PPMU_PMCNT0_MASK | PPMU_PMCNT1_MASK
| PPMU_PMCNT2_MASK | PPMU_PMCNT3_MASK);
+ ret = regmap_write(info->regmap, PPMU_V2_FLAG, clear);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_INTENC, clear);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CNTENC, clear);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CNT_RESET, clear);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CIG_CFG0, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CIG_CFG1, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CIG_CFG2, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CIG_RESULT, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CNT_AUTO, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CH_EV0_TYPE, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CH_EV1_TYPE, 0x0);
+ if (ret < 0)
+ return ret;
- __raw_writel(clear, info->ppmu.base + PPMU_V2_FLAG);
- __raw_writel(clear, info->ppmu.base + PPMU_V2_INTENC);
- __raw_writel(clear, info->ppmu.base + PPMU_V2_CNTENC);
- __raw_writel(clear, info->ppmu.base + PPMU_V2_CNT_RESET);
-
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CIG_CFG0);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CIG_CFG1);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CIG_CFG2);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CIG_RESULT);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CNT_AUTO);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CH_EV0_TYPE);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CH_EV1_TYPE);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CH_EV2_TYPE);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_CH_EV3_TYPE);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_SM_ID_V);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_SM_ID_A);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_SM_OTHERS_V);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_SM_OTHERS_A);
- __raw_writel(0x0, info->ppmu.base + PPMU_V2_INTERRUPT_RESET);
+ ret = regmap_write(info->regmap, PPMU_V2_CH_EV2_TYPE, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_CH_EV3_TYPE, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_SM_ID_V, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_SM_ID_A, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_SM_OTHERS_V, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_SM_OTHERS_A, 0x0);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_write(info->regmap, PPMU_V2_INTERRUPT_RESET, 0x0);
+ if (ret < 0)
+ return ret;
/* Disable PPMU */
- pmnc = __raw_readl(info->ppmu.base + PPMU_V2_PMNC);
+ ret = regmap_read(info->regmap, PPMU_V2_PMNC, &pmnc);
+ if (ret < 0)
+ return ret;
+
pmnc &= ~PPMU_PMNC_ENABLE_MASK;
- __raw_writel(pmnc, info->ppmu.base + PPMU_V2_PMNC);
+ ret = regmap_write(info->regmap, PPMU_V2_PMNC, pmnc);
+ if (ret < 0)
+ return ret;
return 0;
}
static int exynos_ppmu_v2_set_event(struct devfreq_event_dev *edev)
{
struct exynos_ppmu *info = devfreq_event_get_drvdata(edev);
+ unsigned int pmnc, cntens;
int id = exynos_ppmu_find_ppmu_id(edev);
- u32 pmnc, cntens;
+ int ret;
/* Enable all counters */
- cntens = __raw_readl(info->ppmu.base + PPMU_V2_CNTENS);
+ ret = regmap_read(info->regmap, PPMU_V2_CNTENS, &cntens);
+ if (ret < 0)
+ return ret;
+
cntens |= (PPMU_CCNT_MASK | (PPMU_ENABLE << id));
- __raw_writel(cntens, info->ppmu.base + PPMU_V2_CNTENS);
+ ret = regmap_write(info->regmap, PPMU_V2_CNTENS, cntens);
+ if (ret < 0)
+ return ret;
/* Set the event of Read/Write data count */
switch (id) {
case PPMU_PMNCNT0:
case PPMU_PMNCNT1:
case PPMU_PMNCNT2:
- __raw_writel(PPMU_V2_RO_DATA_CNT | PPMU_V2_WO_DATA_CNT,
- info->ppmu.base + PPMU_V2_CH_EVx_TYPE(id));
+ ret = regmap_write(info->regmap, PPMU_V2_CH_EVx_TYPE(id),
+ PPMU_V2_RO_DATA_CNT | PPMU_V2_WO_DATA_CNT);
+ if (ret < 0)
+ return ret;
break;
case PPMU_PMNCNT3:
- __raw_writel(PPMU_V2_EVT3_RW_DATA_CNT,
- info->ppmu.base + PPMU_V2_CH_EVx_TYPE(id));
+ ret = regmap_write(info->regmap, PPMU_V2_CH_EVx_TYPE(id),
+ PPMU_V2_EVT3_RW_DATA_CNT);
+ if (ret < 0)
+ return ret;
break;
}
/* Reset cycle counter/performance counter and enable PPMU */
- pmnc = __raw_readl(info->ppmu.base + PPMU_V2_PMNC);
+ ret = regmap_read(info->regmap, PPMU_V2_PMNC, &pmnc);
+ if (ret < 0)
+ return ret;
+
pmnc &= ~(PPMU_PMNC_ENABLE_MASK
| PPMU_PMNC_COUNTER_RESET_MASK
| PPMU_PMNC_CC_RESET_MASK
pmnc |= (PPMU_ENABLE << PPMU_PMNC_COUNTER_RESET_SHIFT);
pmnc |= (PPMU_ENABLE << PPMU_PMNC_CC_RESET_SHIFT);
pmnc |= (PPMU_V2_MODE_MANUAL << PPMU_V2_PMNC_START_MODE_SHIFT);
- __raw_writel(pmnc, info->ppmu.base + PPMU_V2_PMNC);
+
+ ret = regmap_write(info->regmap, PPMU_V2_PMNC, pmnc);
+ if (ret < 0)
+ return ret;
return 0;
}
{
struct exynos_ppmu *info = devfreq_event_get_drvdata(edev);
int id = exynos_ppmu_find_ppmu_id(edev);
- u32 pmnc, cntenc;
- u32 pmcnt_high, pmcnt_low;
- u64 load_count = 0;
+ int ret;
+ unsigned int pmnc, cntenc;
+ unsigned int pmcnt_high, pmcnt_low;
+ unsigned int total_count, count;
+ unsigned long load_count = 0;
/* Disable PPMU */
- pmnc = __raw_readl(info->ppmu.base + PPMU_V2_PMNC);
+ ret = regmap_read(info->regmap, PPMU_V2_PMNC, &pmnc);
+ if (ret < 0)
+ return ret;
+
pmnc &= ~PPMU_PMNC_ENABLE_MASK;
- __raw_writel(pmnc, info->ppmu.base + PPMU_V2_PMNC);
+ ret = regmap_write(info->regmap, PPMU_V2_PMNC, pmnc);
+ if (ret < 0)
+ return ret;
/* Read cycle count and performance count */
- edata->total_count = __raw_readl(info->ppmu.base + PPMU_V2_CCNT);
+ ret = regmap_read(info->regmap, PPMU_V2_CCNT, &total_count);
+ if (ret < 0)
+ return ret;
+ edata->total_count = total_count;
switch (id) {
case PPMU_PMNCNT0:
case PPMU_PMNCNT1:
case PPMU_PMNCNT2:
- load_count = __raw_readl(info->ppmu.base + PPMU_V2_PMNCT(id));
+ ret = regmap_read(info->regmap, PPMU_V2_PMNCT(id), &count);
+ if (ret < 0)
+ return ret;
+ load_count = count;
break;
case PPMU_PMNCNT3:
- pmcnt_high = __raw_readl(info->ppmu.base + PPMU_V2_PMCNT3_HIGH);
- pmcnt_low = __raw_readl(info->ppmu.base + PPMU_V2_PMCNT3_LOW);
- load_count = ((u64)((pmcnt_high & 0xff)) << 32)
- + (u64)pmcnt_low;
+ ret = regmap_read(info->regmap, PPMU_V2_PMCNT3_HIGH,
+ &pmcnt_high);
+ if (ret < 0)
+ return ret;
+
+ ret = regmap_read(info->regmap, PPMU_V2_PMCNT3_LOW, &pmcnt_low);
+ if (ret < 0)
+ return ret;
+
+ load_count = ((u64)((pmcnt_high & 0xff)) << 32)+ (u64)pmcnt_low;
break;
}
edata->load_count = load_count;
/* Disable all counters */
- cntenc = __raw_readl(info->ppmu.base + PPMU_V2_CNTENC);
+ ret = regmap_read(info->regmap, PPMU_V2_CNTENC, &cntenc);
+ if (ret < 0)
+ return 0;
+
cntenc |= (PPMU_CCNT_MASK | (PPMU_ENABLE << id));
- __raw_writel(cntenc, info->ppmu.base + PPMU_V2_CNTENC);
+ ret = regmap_write(info->regmap, PPMU_V2_CNTENC, cntenc);
+ if (ret < 0)
+ return ret;
dev_dbg(&edev->dev, "%25s (load: %ld / %ld)\n", edev->desc->name,
edata->load_count, edata->total_count);
return 0;
}
-static int exynos_ppmu_parse_dt(struct exynos_ppmu *info)
+static struct regmap_config exynos_ppmu_regmap_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+};
+
+static int exynos_ppmu_parse_dt(struct platform_device *pdev,
+ struct exynos_ppmu *info)
{
struct device *dev = info->dev;
struct device_node *np = dev->of_node;
+ struct resource *res;
+ void __iomem *base;
int ret = 0;
if (!np) {
}
/* Maps the memory mapped IO to control PPMU register */
- info->ppmu.base = of_iomap(np, 0);
- if (IS_ERR_OR_NULL(info->ppmu.base)) {
- dev_err(dev, "failed to map memory region\n");
- return -ENOMEM;
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ exynos_ppmu_regmap_config.max_register = resource_size(res) - 4;
+ info->regmap = devm_regmap_init_mmio(dev, base,
+ &exynos_ppmu_regmap_config);
+ if (IS_ERR(info->regmap)) {
+ dev_err(dev, "failed to initialize regmap\n");
+ return PTR_ERR(info->regmap);
}
info->ppmu.clk = devm_clk_get(dev, "ppmu");
ret = of_get_devfreq_events(np, info);
if (ret < 0) {
dev_err(dev, "failed to parse exynos ppmu dt node\n");
- goto err;
+ return ret;
}
return 0;
-
-err:
- iounmap(info->ppmu.base);
-
- return ret;
}
static int exynos_ppmu_probe(struct platform_device *pdev)
info->dev = &pdev->dev;
/* Parse dt data to get resource */
- ret = exynos_ppmu_parse_dt(info);
+ ret = exynos_ppmu_parse_dt(pdev, info);
if (ret < 0) {
dev_err(&pdev->dev,
"failed to parse devicetree for resource\n");
if (!info->edev) {
dev_err(&pdev->dev,
"failed to allocate memory devfreq-event devices\n");
- ret = -ENOMEM;
- goto err;
+ return -ENOMEM;
}
edev = info->edev;
platform_set_drvdata(pdev, info);
ret = PTR_ERR(edev[i]);
dev_err(&pdev->dev,
"failed to add devfreq-event device\n");
- goto err;
+ return PTR_ERR(edev[i]);
}
+
+ pr_info("exynos-ppmu: new PPMU device registered %s (%s)\n",
+ dev_name(&pdev->dev), desc[i].name);
}
clk_prepare_enable(info->ppmu.clk);
return 0;
-err:
- iounmap(info->ppmu.base);
-
- return ret;
}
static int exynos_ppmu_remove(struct platform_device *pdev)
struct exynos_ppmu *info = platform_get_drvdata(pdev);
clk_disable_unprepare(info->ppmu.clk);
- iounmap(info->ppmu.base);
return 0;
}
int ret = 0;
/* Get new opp-bus instance according to new bus clock */
- rcu_read_lock();
new_opp = devfreq_recommended_opp(dev, freq, flags);
if (IS_ERR(new_opp)) {
dev_err(dev, "failed to get recommended opp instance\n");
- rcu_read_unlock();
return PTR_ERR(new_opp);
}
new_freq = dev_pm_opp_get_freq(new_opp);
new_volt = dev_pm_opp_get_voltage(new_opp);
+ dev_pm_opp_put(new_opp);
+
old_freq = bus->curr_freq;
- rcu_read_unlock();
if (old_freq == new_freq)
return 0;
}
bus->curr_freq = new_freq;
- dev_dbg(dev, "Set the frequency of bus (%lukHz -> %lukHz)\n",
- old_freq/1000, new_freq/1000);
+ dev_dbg(dev, "Set the frequency of bus (%luHz -> %luHz, %luHz)\n",
+ old_freq, new_freq, clk_get_rate(bus->clk));
out:
mutex_unlock(&bus->lock);
int ret = 0;
/* Get new opp-bus instance according to new bus clock */
- rcu_read_lock();
new_opp = devfreq_recommended_opp(dev, freq, flags);
if (IS_ERR(new_opp)) {
dev_err(dev, "failed to get recommended opp instance\n");
- rcu_read_unlock();
return PTR_ERR(new_opp);
}
new_freq = dev_pm_opp_get_freq(new_opp);
+ dev_pm_opp_put(new_opp);
+
old_freq = bus->curr_freq;
- rcu_read_unlock();
if (old_freq == new_freq)
return 0;
*freq = new_freq;
bus->curr_freq = new_freq;
- dev_dbg(dev, "Set the frequency of bus (%lukHz -> %lukHz)\n",
- old_freq/1000, new_freq/1000);
+ dev_dbg(dev, "Set the frequency of bus (%luHz -> %luHz, %luHz)\n",
+ old_freq, new_freq, clk_get_rate(bus->clk));
out:
mutex_unlock(&bus->lock);
rate = clk_get_rate(bus->clk);
- rcu_read_lock();
opp = devfreq_recommended_opp(dev, &rate, 0);
if (IS_ERR(opp)) {
dev_err(dev, "failed to find dev_pm_opp\n");
- rcu_read_unlock();
ret = PTR_ERR(opp);
goto err_opp;
}
bus->curr_freq = dev_pm_opp_get_freq(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
return 0;
extern int devfreq_add_governor(struct devfreq_governor *governor);
extern int devfreq_remove_governor(struct devfreq_governor *governor);
+extern int devfreq_update_status(struct devfreq *devfreq, unsigned long freq);
+
#endif /* _GOVERNOR_H */
* list of parent device. Because in this case, *freq is temporary
* value which is decided by ondemand governor.
*/
- rcu_read_lock();
opp = devfreq_recommended_opp(parent_devfreq->dev.parent, freq, 0);
- rcu_read_unlock();
if (IS_ERR(opp)) {
ret = PTR_ERR(opp);
goto out;
}
+ dev_pm_opp_put(opp);
+
/*
* Get the OPP table's index of decided freqeuncy by governor
* of parent device.
if (ret < 0)
goto out;
+ if (devfreq->profile->freq_table
+ && (devfreq_update_status(devfreq, freq)))
+ dev_err(&devfreq->dev,
+ "Couldn't update frequency transition information.\n");
+
devfreq->previous_freq = freq;
out:
static struct devfreq_governor devfreq_passive = {
.name = "passive",
+ .immutable = 1,
.get_target_freq = devfreq_passive_get_target_freq,
.event_handler = devfreq_passive_event_handler,
};
/*
- * linux/drivers/devfreq/governor_simpleondemand.c
+ * linux/drivers/devfreq/governor_userspace.c
*
* Copyright (C) 2011 Samsung Electronics
* MyungJoo Ham <myungjoo.ham@samsung.com>
unsigned long wanted;
int err = 0;
-
mutex_lock(&devfreq->lock);
data = devfreq->data;
static void userspace_exit(struct devfreq *devfreq)
{
- sysfs_remove_group(&devfreq->dev.kobj, &dev_attr_group);
+ /*
+ * Remove the sysfs entry, unless this is being called after
+ * device_del(), which should have done this already via kobject_del().
+ */
+ if (devfreq->dev.kobj.sd)
+ sysfs_remove_group(&devfreq->dev.kobj, &dev_attr_group);
+
kfree(devfreq->data);
devfreq->data = NULL;
}
unsigned long target_volt, target_rate;
int err;
- rcu_read_lock();
opp = devfreq_recommended_opp(dev, freq, flags);
- if (IS_ERR(opp)) {
- rcu_read_unlock();
+ if (IS_ERR(opp))
return PTR_ERR(opp);
- }
target_rate = dev_pm_opp_get_freq(opp);
target_volt = dev_pm_opp_get_voltage(opp);
-
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
if (dmcfreq->rate == target_rate)
return 0;
data->rate = clk_get_rate(data->dmc_clk);
- rcu_read_lock();
opp = devfreq_recommended_opp(dev, &data->rate, 0);
- if (IS_ERR(opp)) {
- rcu_read_unlock();
+ if (IS_ERR(opp))
return PTR_ERR(opp);
- }
+
data->rate = dev_pm_opp_get_freq(opp);
data->volt = dev_pm_opp_get_voltage(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
rk3399_devfreq_dmc_profile.initial_freq = data->rate;
struct dev_pm_opp *opp;
unsigned long rate = *freq * KHZ;
- rcu_read_lock();
opp = devfreq_recommended_opp(dev, &rate, flags);
if (IS_ERR(opp)) {
- rcu_read_unlock();
dev_err(dev, "Failed to find opp for %lu KHz\n", *freq);
return PTR_ERR(opp);
}
rate = dev_pm_opp_get_freq(opp);
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
clk_set_min_rate(tegra->emc_clock, rate);
clk_set_rate(tegra->emc_clock, 0);
/* context for suspend/resume */
unsigned int dma_tdfdq;
+
+ bool is_suspended;
};
#define FIST_COMPLETION_QUEUE 93
BUG_ON(desc_num >= ALLOC_DECS_NUM);
c = cdd->chan_busy[desc_num];
cdd->chan_busy[desc_num] = NULL;
+
+ /* Usecount for chan_busy[], paired with push_desc_queue() */
+ pm_runtime_put(cdd->ddev.dev);
+
return c;
}
while (val) {
u32 desc, len;
- int error;
- error = pm_runtime_get(cdd->ddev.dev);
- if (error < 0)
- dev_err(cdd->ddev.dev, "%s pm runtime get: %i\n",
- __func__, error);
+ /*
+ * This should never trigger, see the comments in
+ * push_desc_queue()
+ */
+ WARN_ON(cdd->is_suspended);
q_num = __fls(val);
val &= ~(1 << q_num);
c->residue = pd_trans_len(c->desc->pd6) - len;
dma_cookie_complete(&c->txd);
dmaengine_desc_get_callback_invoke(&c->txd, NULL);
-
- pm_runtime_mark_last_busy(cdd->ddev.dev);
- pm_runtime_put_autosuspend(cdd->ddev.dev);
}
}
return IRQ_HANDLED;
*/
__iowmb();
+ /*
+ * DMA transfers can take at least 200ms to complete with USB mass
+ * storage connected. To prevent autosuspend timeouts, we must use
+ * pm_runtime_get/put() when chan_busy[] is modified. This will get
+ * cleared in desc_to_chan() or cppi41_stop_chan() depending on the
+ * outcome of the transfer.
+ */
+ pm_runtime_get(cdd->ddev.dev);
+
desc_phys = lower_32_bits(c->desc_phys);
desc_num = (desc_phys - cdd->descs_phys) / sizeof(struct cppi41_desc);
WARN_ON(cdd->chan_busy[desc_num]);
cppi_writel(reg, cdd->qmgr_mem + QMGR_QUEUE_D(c->q_num));
}
-static void pending_desc(struct cppi41_channel *c)
+/*
+ * Caller must hold cdd->lock to prevent push_desc_queue()
+ * getting called out of order. We have both cppi41_dma_issue_pending()
+ * and cppi41_runtime_resume() call this function.
+ */
+static void cppi41_run_queue(struct cppi41_dd *cdd)
{
- struct cppi41_dd *cdd = c->cdd;
- unsigned long flags;
+ struct cppi41_channel *c, *_c;
- spin_lock_irqsave(&cdd->lock, flags);
- list_add_tail(&c->node, &cdd->pending);
- spin_unlock_irqrestore(&cdd->lock, flags);
+ list_for_each_entry_safe(c, _c, &cdd->pending, node) {
+ push_desc_queue(c);
+ list_del(&c->node);
+ }
}
static void cppi41_dma_issue_pending(struct dma_chan *chan)
{
struct cppi41_channel *c = to_cpp41_chan(chan);
struct cppi41_dd *cdd = c->cdd;
+ unsigned long flags;
int error;
error = pm_runtime_get(cdd->ddev.dev);
return;
}
- if (likely(pm_runtime_active(cdd->ddev.dev)))
- push_desc_queue(c);
- else
- pending_desc(c);
+ spin_lock_irqsave(&cdd->lock, flags);
+ list_add_tail(&c->node, &cdd->pending);
+ if (!cdd->is_suspended)
+ cppi41_run_queue(cdd);
+ spin_unlock_irqrestore(&cdd->lock, flags);
pm_runtime_mark_last_busy(cdd->ddev.dev);
pm_runtime_put_autosuspend(cdd->ddev.dev);
WARN_ON(!cdd->chan_busy[desc_num]);
cdd->chan_busy[desc_num] = NULL;
+ /* Usecount for chan_busy[], paired with push_desc_queue() */
+ pm_runtime_put(cdd->ddev.dev);
+
return 0;
}
static int __maybe_unused cppi41_runtime_suspend(struct device *dev)
{
struct cppi41_dd *cdd = dev_get_drvdata(dev);
+ unsigned long flags;
+ spin_lock_irqsave(&cdd->lock, flags);
+ cdd->is_suspended = true;
WARN_ON(!list_empty(&cdd->pending));
+ spin_unlock_irqrestore(&cdd->lock, flags);
return 0;
}
static int __maybe_unused cppi41_runtime_resume(struct device *dev)
{
struct cppi41_dd *cdd = dev_get_drvdata(dev);
- struct cppi41_channel *c, *_c;
unsigned long flags;
spin_lock_irqsave(&cdd->lock, flags);
- list_for_each_entry_safe(c, _c, &cdd->pending, node) {
- push_desc_queue(c);
- list_del(&c->node);
- }
+ cdd->is_suspended = false;
+ cppi41_run_queue(cdd);
spin_unlock_irqrestore(&cdd->lock, flags);
return 0;
static struct pl330_thread *pl330_request_channel(struct pl330_dmac *pl330)
{
struct pl330_thread *thrd = NULL;
- unsigned long flags;
int chans, i;
if (pl330->state == DYING)
chans = pl330->pcfg.num_chan;
- spin_lock_irqsave(&pl330->lock, flags);
-
for (i = 0; i < chans; i++) {
thrd = &pl330->channels[i];
if ((thrd->free) && (!_manager_ns(thrd) ||
thrd = NULL;
}
- spin_unlock_irqrestore(&pl330->lock, flags);
-
return thrd;
}
static void pl330_release_channel(struct pl330_thread *thrd)
{
struct pl330_dmac *pl330;
- unsigned long flags;
if (!thrd || thrd->free)
return;
pl330 = thrd->dmac;
- spin_lock_irqsave(&pl330->lock, flags);
_free_event(thrd, thrd->ev);
thrd->free = true;
- spin_unlock_irqrestore(&pl330->lock, flags);
}
/* Initialize the structure for PL330 configuration, that can be used
struct pl330_dmac *pl330 = pch->dmac;
unsigned long flags;
- spin_lock_irqsave(&pch->lock, flags);
+ spin_lock_irqsave(&pl330->lock, flags);
dma_cookie_init(chan);
pch->cyclic = false;
pch->thread = pl330_request_channel(pl330);
if (!pch->thread) {
- spin_unlock_irqrestore(&pch->lock, flags);
+ spin_unlock_irqrestore(&pl330->lock, flags);
return -ENOMEM;
}
tasklet_init(&pch->task, pl330_tasklet, (unsigned long) pch);
- spin_unlock_irqrestore(&pch->lock, flags);
+ spin_unlock_irqrestore(&pl330->lock, flags);
return 1;
}
static void pl330_free_chan_resources(struct dma_chan *chan)
{
struct dma_pl330_chan *pch = to_pchan(chan);
+ struct pl330_dmac *pl330 = pch->dmac;
unsigned long flags;
tasklet_kill(&pch->task);
pm_runtime_get_sync(pch->dmac->ddma.dev);
- spin_lock_irqsave(&pch->lock, flags);
+ spin_lock_irqsave(&pl330->lock, flags);
pl330_release_channel(pch->thread);
pch->thread = NULL;
if (pch->cyclic)
list_splice_tail_init(&pch->work_list, &pch->dmac->desc_pool);
- spin_unlock_irqrestore(&pch->lock, flags);
+ spin_unlock_irqrestore(&pl330->lock, flags);
pm_runtime_mark_last_busy(pch->dmac->ddma.dev);
pm_runtime_put_autosuspend(pch->dmac->ddma.dev);
}
struct exit_boot_struct {
efi_memory_desc_t *runtime_map;
int *runtime_entry_count;
+ void *new_fdt_addr;
};
static efi_status_t exit_boot_func(efi_system_table_t *sys_table_arg,
efi_get_virtmap(*map->map, *map->map_size, *map->desc_size,
p->runtime_map, p->runtime_entry_count);
- return EFI_SUCCESS;
+ return update_fdt_memmap(p->new_fdt_addr, map);
}
/*
priv.runtime_map = runtime_map;
priv.runtime_entry_count = &runtime_entry_count;
+ priv.new_fdt_addr = (void *)*new_fdt_addr;
status = efi_exit_boot_services(sys_table, handle, &map, &priv,
exit_boot_func);
if (status == EFI_SUCCESS) {
efi_set_virtual_address_map_t *svam;
- status = update_fdt_memmap((void *)*new_fdt_addr, &map);
- if (status != EFI_SUCCESS) {
- /*
- * The kernel won't get far without the memory map, but
- * may still be able to print something meaningful so
- * return success here.
- */
- return EFI_SUCCESS;
- }
-
/* Install the new virtual address map */
svam = sys_table->runtime->set_virtual_address_map;
status = svam(runtime_entry_count * desc_size, desc_size,
}
/**
- * _gpiochip_irqchip_add() - adds an irqchip to a gpiochip
+ * gpiochip_irqchip_add_key() - adds an irqchip to a gpiochip
* @gpiochip: the gpiochip to add the irqchip to
* @irqchip: the irqchip to add to the gpiochip
* @first_irq: if not dynamically assigned, the base (first) IRQ to
* the pins on the gpiochip can generate a unique IRQ. Everything else
* need to be open coded.
*/
-int _gpiochip_irqchip_add(struct gpio_chip *gpiochip,
- struct irq_chip *irqchip,
- unsigned int first_irq,
- irq_flow_handler_t handler,
- unsigned int type,
- bool nested,
- struct lock_class_key *lock_key)
+int gpiochip_irqchip_add_key(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type,
+ bool nested,
+ struct lock_class_key *lock_key)
{
struct device_node *of_node;
bool irq_base_set = false;
return 0;
}
-EXPORT_SYMBOL_GPL(_gpiochip_irqchip_add);
+EXPORT_SYMBOL_GPL(gpiochip_irqchip_add_key);
#else /* CONFIG_GPIOLIB_IRQCHIP */
}
break;
}
+
+ if (!(*out_ring && (*out_ring)->adev)) {
+ DRM_ERROR("Ring %d is not initialized on IP %d\n",
+ ring, ip_type);
+ return -EINVAL;
+ }
+
return 0;
}
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((amdgpu_crtc->cursor_width - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v10_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v10_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v10_0_show_cursor(crtc);
static void dce_v10_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v10_0_lock_cursor(crtc, true);
dce_v10_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v10_0_show_cursor(crtc);
dce_v10_0_lock_cursor(crtc, false);
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((amdgpu_crtc->cursor_width - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v11_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v11_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v11_0_show_cursor(crtc);
static void dce_v11_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v11_0_lock_cursor(crtc, true);
dce_v11_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v11_0_show_cursor(crtc);
dce_v11_0_lock_cursor(crtc, false);
struct amdgpu_device *adev = crtc->dev->dev_private;
int xorigin = 0, yorigin = 0;
+ int w = amdgpu_crtc->cursor_width;
+
amdgpu_crtc->cursor_x = x;
amdgpu_crtc->cursor_y = y;
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((w - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v6_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v6_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v6_0_show_cursor(crtc);
static void dce_v6_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v6_0_lock_cursor(crtc, true);
dce_v6_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v6_0_show_cursor(crtc);
dce_v6_0_lock_cursor(crtc, false);
}
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((amdgpu_crtc->cursor_width - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v8_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v8_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v8_0_show_cursor(crtc);
static void dce_v8_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v8_0_lock_cursor(crtc, true);
dce_v8_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v8_0_show_cursor(crtc);
dce_v8_0_lock_cursor(crtc, false);
static void dce_virtual_encoder_destroy(struct drm_encoder *encoder)
{
- struct amdgpu_encoder *amdgpu_encoder = to_amdgpu_encoder(encoder);
-
- kfree(amdgpu_encoder->enc_priv);
drm_encoder_cleanup(encoder);
- kfree(amdgpu_encoder);
+ kfree(encoder);
}
static const struct drm_encoder_funcs dce_virtual_encoder_funcs = {
MODULE_FIRMWARE("radeon/pitcairn_mc.bin");
MODULE_FIRMWARE("radeon/verde_mc.bin");
MODULE_FIRMWARE("radeon/oland_mc.bin");
+MODULE_FIRMWARE("radeon/si58_mc.bin");
#define MC_SEQ_MISC0__MT__MASK 0xf0000000
#define MC_SEQ_MISC0__MT__GDDR1 0x10000000
const char *chip_name;
char fw_name[30];
int err;
+ bool is_58_fw = false;
DRM_DEBUG("\n");
default: BUG();
}
- snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", chip_name);
+ /* this memory configuration requires special firmware */
+ if (((RREG32(mmMC_SEQ_MISC0) & 0xff000000) >> 24) == 0x58)
+ is_58_fw = true;
+
+ if (is_58_fw)
+ snprintf(fw_name, sizeof(fw_name), "radeon/si58_mc.bin");
+ else
+ snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", chip_name);
err = request_firmware(&adev->mc.fw, fw_name, adev->dev);
if (err)
goto out;
}
WREG32(mmHDP_REG_COHERENCY_FLUSH_CNTL, 0);
+ if (adev->mode_info.num_crtc)
+ amdgpu_display_set_vga_render_state(adev, false);
+
gmc_v6_0_mc_stop(adev, &save);
if (gmc_v6_0_wait_for_idle((void *)adev)) {
dev_warn(adev->dev, "Wait for MC idle timedout !\n");
}
gmc_v6_0_mc_resume(adev, &save);
- amdgpu_display_set_vga_render_state(adev, false);
}
static int gmc_v6_0_mc_init(struct amdgpu_device *adev)
WREG32(mmVM_CONTEXT1_CNTL,
VM_CONTEXT1_CNTL__ENABLE_CONTEXT_MASK |
(1UL << VM_CONTEXT1_CNTL__PAGE_TABLE_DEPTH__SHIFT) |
- ((amdgpu_vm_block_size - 9) << VM_CONTEXT1_CNTL__PAGE_TABLE_BLOCK_SIZE__SHIFT) |
- VM_CONTEXT1_CNTL__RANGE_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__RANGE_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__DUMMY_PAGE_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__DUMMY_PAGE_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__PDE0_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__PDE0_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__VALID_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__VALID_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__READ_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__READ_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__WRITE_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__WRITE_PROTECTION_FAULT_ENABLE_DEFAULT_MASK);
+ ((amdgpu_vm_block_size - 9) << VM_CONTEXT1_CNTL__PAGE_TABLE_BLOCK_SIZE__SHIFT));
+ if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_ALWAYS)
+ gmc_v6_0_set_fault_enable_default(adev, false);
+ else
+ gmc_v6_0_set_fault_enable_default(adev, true);
gmc_v6_0_gart_flush_gpu_tlb(adev, 0);
dev_info(adev->dev, "PCIE GART of %uM enabled (table at 0x%016llX).\n",
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- return amdgpu_irq_get(adev, &adev->mc.vm_fault, 0);
+ if (amdgpu_vm_fault_stop != AMDGPU_VM_FAULT_STOP_ALWAYS)
+ return amdgpu_irq_get(adev, &adev->mc.vm_fault, 0);
+ else
+ return 0;
}
static int gmc_v6_0_sw_init(void *handle)
MODULE_FIRMWARE("radeon/oland_k_smc.bin");
MODULE_FIRMWARE("radeon/hainan_smc.bin");
MODULE_FIRMWARE("radeon/hainan_k_smc.bin");
+MODULE_FIRMWARE("radeon/banks_k_2_smc.bin");
union power_info {
struct _ATOM_POWERPLAY_INFO info;
(adev->pdev->device == 0x6817) ||
(adev->pdev->device == 0x6806))
max_mclk = 120000;
- } else if (adev->asic_type == CHIP_OLAND) {
- if ((adev->pdev->revision == 0xC7) ||
- (adev->pdev->revision == 0x80) ||
- (adev->pdev->revision == 0x81) ||
- (adev->pdev->revision == 0x83) ||
- (adev->pdev->revision == 0x87) ||
- (adev->pdev->device == 0x6604) ||
- (adev->pdev->device == 0x6605)) {
- max_sclk = 75000;
- max_mclk = 80000;
- }
} else if (adev->asic_type == CHIP_HAINAN) {
if ((adev->pdev->revision == 0x81) ||
(adev->pdev->revision == 0x83) ||
(adev->pdev->device == 0x6665) ||
(adev->pdev->device == 0x6667)) {
max_sclk = 75000;
- max_mclk = 80000;
}
}
/* Apply dpm quirks */
((adev->pdev->device == 0x6660) ||
(adev->pdev->device == 0x6663) ||
(adev->pdev->device == 0x6665) ||
- (adev->pdev->device == 0x6667))) ||
- ((adev->pdev->revision == 0xc3) &&
- (adev->pdev->device == 0x6665)))
+ (adev->pdev->device == 0x6667))))
chip_name = "hainan_k";
+ else if ((adev->pdev->revision == 0xc3) &&
+ (adev->pdev->device == 0x6665))
+ chip_name = "banks_k_2";
else
chip_name = "hainan";
break;
#include "smu/smu_7_0_1_sh_mask.h"
static void uvd_v4_2_mc_resume(struct amdgpu_device *adev);
-static void uvd_v4_2_init_cg(struct amdgpu_device *adev);
static void uvd_v4_2_set_ring_funcs(struct amdgpu_device *adev);
static void uvd_v4_2_set_irq_funcs(struct amdgpu_device *adev);
static int uvd_v4_2_start(struct amdgpu_device *adev);
static void uvd_v4_2_stop(struct amdgpu_device *adev);
static int uvd_v4_2_set_clockgating_state(void *handle,
enum amd_clockgating_state state);
+static void uvd_v4_2_set_dcm(struct amdgpu_device *adev,
+ bool sw_mode);
/**
* uvd_v4_2_ring_get_rptr - get read pointer
*
return r;
}
-
+static void uvd_v4_2_enable_mgcg(struct amdgpu_device *adev,
+ bool enable);
/**
* uvd_v4_2_hw_init - start and test UVD block
*
uint32_t tmp;
int r;
- uvd_v4_2_init_cg(adev);
- uvd_v4_2_set_clockgating_state(adev, AMD_CG_STATE_GATE);
+ uvd_v4_2_enable_mgcg(adev, true);
amdgpu_asic_set_uvd_clocks(adev, 10000, 10000);
r = uvd_v4_2_start(adev);
if (r)
struct amdgpu_ring *ring = &adev->uvd.ring;
uint32_t rb_bufsz;
int i, j, r;
-
/* disable byte swapping */
u32 lmi_swap_cntl = 0;
u32 mp_swap_cntl = 0;
+ WREG32(mmUVD_CGC_GATE, 0);
+ uvd_v4_2_set_dcm(adev, true);
+
uvd_v4_2_mc_resume(adev);
/* disable interupt */
/* Unstall UMC and register bus */
WREG32_P(mmUVD_LMI_CTRL2, 0, ~(1 << 8));
+
+ uvd_v4_2_set_dcm(adev, false);
}
/**
WREG32_UVD_CTX(ixUVD_CGC_CTRL2, tmp2);
}
-static void uvd_v4_2_init_cg(struct amdgpu_device *adev)
-{
- bool hw_mode = true;
-
- if (hw_mode) {
- uvd_v4_2_set_dcm(adev, false);
- } else {
- u32 tmp = RREG32(mmUVD_CGC_CTRL);
- tmp &= ~UVD_CGC_CTRL__DYN_CLOCK_MODE_MASK;
- WREG32(mmUVD_CGC_CTRL, tmp);
- }
-}
-
static bool uvd_v4_2_is_idle(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
static int uvd_v4_2_set_clockgating_state(void *handle,
enum amd_clockgating_state state)
{
- bool gate = false;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- if (!(adev->cg_flags & AMD_CG_SUPPORT_UVD_MGCG))
- return 0;
-
- if (state == AMD_CG_STATE_GATE)
- gate = true;
-
- uvd_v4_2_enable_mgcg(adev, gate);
-
return 0;
}
*/
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- if (!(adev->pg_flags & AMD_PG_SUPPORT_UVD))
- return 0;
-
if (state == AMD_PG_STATE_GATE) {
uvd_v4_2_stop(adev);
return 0;
#define GRBM_GFX_INDEX__VCE_INSTANCE__SHIFT 0x04
#define GRBM_GFX_INDEX__VCE_INSTANCE_MASK 0x10
+#define GRBM_GFX_INDEX__VCE_ALL_PIPE 0x07
+
#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR0 0x8616
#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR1 0x8617
#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR2 0x8618
+#define mmGRBM_GFX_INDEX_DEFAULT 0xE0000000
+
#define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02
#define VCE_V3_0_FW_SIZE (384 * 1024)
#define FW_52_8_3 ((52 << 24) | (8 << 16) | (3 << 8))
+#define GET_VCE_INSTANCE(i) ((i) << GRBM_GFX_INDEX__VCE_INSTANCE__SHIFT \
+ | GRBM_GFX_INDEX__VCE_ALL_PIPE)
+
static void vce_v3_0_mc_resume(struct amdgpu_device *adev, int idx);
static void vce_v3_0_set_ring_funcs(struct amdgpu_device *adev);
static void vce_v3_0_set_irq_funcs(struct amdgpu_device *adev);
WREG32(mmVCE_UENC_CLOCK_GATING_2, data);
data = RREG32(mmVCE_UENC_REG_CLOCK_GATING);
- data &= ~0xffc00000;
+ data &= ~0x3ff;
WREG32(mmVCE_UENC_REG_CLOCK_GATING, data);
data = RREG32(mmVCE_UENC_DMA_DCLK_CTRL);
if (adev->vce.harvest_config & (1 << idx))
continue;
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, idx);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(idx));
vce_v3_0_mc_resume(adev, idx);
WREG32_FIELD(VCE_STATUS, JOB_BUSY, 1);
}
}
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, 0);
+ WREG32(mmGRBM_GFX_INDEX, mmGRBM_GFX_INDEX_DEFAULT);
mutex_unlock(&adev->grbm_idx_mutex);
return 0;
if (adev->vce.harvest_config & (1 << idx))
continue;
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, idx);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(idx));
if (adev->asic_type >= CHIP_STONEY)
WREG32_P(mmVCE_VCPU_CNTL, 0, ~0x200001);
vce_v3_0_set_vce_sw_clock_gating(adev, false);
}
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, 0);
+ WREG32(mmGRBM_GFX_INDEX, mmGRBM_GFX_INDEX_DEFAULT);
mutex_unlock(&adev->grbm_idx_mutex);
return 0;
* VCE team suggest use bit 3--bit 6 for busy status check
*/
mutex_lock(&adev->grbm_idx_mutex);
- WREG32_FIELD(GRBM_GFX_INDEX, INSTANCE_INDEX, 0);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(0));
if (RREG32(mmVCE_STATUS) & AMDGPU_VCE_STATUS_BUSY_MASK) {
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE0, 1);
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE1, 1);
}
- WREG32_FIELD(GRBM_GFX_INDEX, INSTANCE_INDEX, 0x10);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(1));
if (RREG32(mmVCE_STATUS) & AMDGPU_VCE_STATUS_BUSY_MASK) {
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE0, 1);
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE1, 1);
}
- WREG32_FIELD(GRBM_GFX_INDEX, INSTANCE_INDEX, 0);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(0));
mutex_unlock(&adev->grbm_idx_mutex);
if (srbm_soft_reset) {
if (adev->vce.harvest_config & (1 << i))
continue;
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, i);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(i));
if (enable) {
/* initialize VCE_CLOCK_GATING_A: Clock ON/OFF delay */
vce_v3_0_set_vce_sw_clock_gating(adev, enable);
}
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, 0);
+ WREG32(mmGRBM_GFX_INDEX, mmGRBM_GFX_INDEX_DEFAULT);
mutex_unlock(&adev->grbm_idx_mutex);
return 0;
cgs_set_clockgating_state(
hwmgr->device,
AMD_IP_BLOCK_TYPE_VCE,
- AMD_CG_STATE_UNGATE);
+ AMD_CG_STATE_GATE);
cgs_set_powergating_state(
hwmgr->device,
AMD_IP_BLOCK_TYPE_VCE,
cgs_set_clockgating_state(
hwmgr->device,
AMD_IP_BLOCK_TYPE_VCE,
- AMD_PG_STATE_GATE);
+ AMD_PG_STATE_UNGATE);
cz_dpm_update_vce_dpm(hwmgr);
cz_enable_disable_vce_dpm(hwmgr, true);
return 0;
cz_hwmgr->vce_dpm.hard_min_clk,
PPSMC_MSG_SetEclkHardMin));
} else {
- /*EPR# 419220 -HW limitation to to */
- cz_hwmgr->vce_dpm.hard_min_clk = hwmgr->vce_arbiter.ecclk;
- smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
- PPSMC_MSG_SetEclkHardMin,
- cz_get_eclk_level(hwmgr,
- cz_hwmgr->vce_dpm.hard_min_clk,
- PPSMC_MSG_SetEclkHardMin));
-
+ /*Program HardMin based on the vce_arbiter.ecclk */
+ if (hwmgr->vce_arbiter.ecclk == 0) {
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetEclkHardMin, 0);
+ /* disable ECLK DPM 0. Otherwise VCE could hang if
+ * switching SCLK from DPM 0 to 6/7 */
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetEclkSoftMin, 1);
+ } else {
+ cz_hwmgr->vce_dpm.hard_min_clk = hwmgr->vce_arbiter.ecclk;
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetEclkHardMin,
+ cz_get_eclk_level(hwmgr,
+ cz_hwmgr->vce_dpm.hard_min_clk,
+ PPSMC_MSG_SetEclkHardMin));
+ }
}
return 0;
}
struct ttm_bo_kmap_obj cache_kmap;
int next_cursor;
bool support_wide_screen;
+ bool DisableP2A;
enum ast_tx_chip tx_chip_type;
u8 dp501_maxclk;
} else
*need_post = false;
+ /* Check P2A Access */
+ ast->DisableP2A = true;
+ data = ast_read32(ast, 0xf004);
+ if (data != 0xFFFFFFFF)
+ ast->DisableP2A = false;
+
/* Check if we support wide screen */
switch (ast->chip) {
case AST1180:
ast->support_wide_screen = true;
else {
ast->support_wide_screen = false;
- /* Read SCU7c (silicon revision register) */
- ast_write32(ast, 0xf004, 0x1e6e0000);
- ast_write32(ast, 0xf000, 0x1);
- data = ast_read32(ast, 0x1207c);
- data &= 0x300;
- if (ast->chip == AST2300 && data == 0x0) /* ast1300 */
- ast->support_wide_screen = true;
- if (ast->chip == AST2400 && data == 0x100) /* ast1400 */
- ast->support_wide_screen = true;
+ if (ast->DisableP2A == false) {
+ /* Read SCU7c (silicon revision register) */
+ ast_write32(ast, 0xf004, 0x1e6e0000);
+ ast_write32(ast, 0xf000, 0x1);
+ data = ast_read32(ast, 0x1207c);
+ data &= 0x300;
+ if (ast->chip == AST2300 && data == 0x0) /* ast1300 */
+ ast->support_wide_screen = true;
+ if (ast->chip == AST2400 && data == 0x100) /* ast1400 */
+ ast->support_wide_screen = true;
+ }
}
break;
}
uint32_t data, data2;
uint32_t denum, num, div, ref_pll;
- ast_write32(ast, 0xf004, 0x1e6e0000);
- ast_write32(ast, 0xf000, 0x1);
-
-
- ast_write32(ast, 0x10000, 0xfc600309);
-
- do {
- if (pci_channel_offline(dev->pdev))
- return -EIO;
- } while (ast_read32(ast, 0x10000) != 0x01);
- data = ast_read32(ast, 0x10004);
-
- if (data & 0x40)
+ if (ast->DisableP2A)
+ {
ast->dram_bus_width = 16;
+ ast->dram_type = AST_DRAM_1Gx16;
+ ast->mclk = 396;
+ }
else
- ast->dram_bus_width = 32;
+ {
+ ast_write32(ast, 0xf004, 0x1e6e0000);
+ ast_write32(ast, 0xf000, 0x1);
+ data = ast_read32(ast, 0x10004);
+
+ if (data & 0x40)
+ ast->dram_bus_width = 16;
+ else
+ ast->dram_bus_width = 32;
+
+ if (ast->chip == AST2300 || ast->chip == AST2400) {
+ switch (data & 0x03) {
+ case 0:
+ ast->dram_type = AST_DRAM_512Mx16;
+ break;
+ default:
+ case 1:
+ ast->dram_type = AST_DRAM_1Gx16;
+ break;
+ case 2:
+ ast->dram_type = AST_DRAM_2Gx16;
+ break;
+ case 3:
+ ast->dram_type = AST_DRAM_4Gx16;
+ break;
+ }
+ } else {
+ switch (data & 0x0c) {
+ case 0:
+ case 4:
+ ast->dram_type = AST_DRAM_512Mx16;
+ break;
+ case 8:
+ if (data & 0x40)
+ ast->dram_type = AST_DRAM_1Gx16;
+ else
+ ast->dram_type = AST_DRAM_512Mx32;
+ break;
+ case 0xc:
+ ast->dram_type = AST_DRAM_1Gx32;
+ break;
+ }
+ }
- if (ast->chip == AST2300 || ast->chip == AST2400) {
- switch (data & 0x03) {
- case 0:
- ast->dram_type = AST_DRAM_512Mx16;
- break;
- default:
- case 1:
- ast->dram_type = AST_DRAM_1Gx16;
- break;
- case 2:
- ast->dram_type = AST_DRAM_2Gx16;
- break;
+ data = ast_read32(ast, 0x10120);
+ data2 = ast_read32(ast, 0x10170);
+ if (data2 & 0x2000)
+ ref_pll = 14318;
+ else
+ ref_pll = 12000;
+
+ denum = data & 0x1f;
+ num = (data & 0x3fe0) >> 5;
+ data = (data & 0xc000) >> 14;
+ switch (data) {
case 3:
- ast->dram_type = AST_DRAM_4Gx16;
- break;
- }
- } else {
- switch (data & 0x0c) {
- case 0:
- case 4:
- ast->dram_type = AST_DRAM_512Mx16;
+ div = 0x4;
break;
- case 8:
- if (data & 0x40)
- ast->dram_type = AST_DRAM_1Gx16;
- else
- ast->dram_type = AST_DRAM_512Mx32;
+ case 2:
+ case 1:
+ div = 0x2;
break;
- case 0xc:
- ast->dram_type = AST_DRAM_1Gx32;
+ default:
+ div = 0x1;
break;
}
+ ast->mclk = ref_pll * (num + 2) / (denum + 2) * (div * 1000);
}
-
- data = ast_read32(ast, 0x10120);
- data2 = ast_read32(ast, 0x10170);
- if (data2 & 0x2000)
- ref_pll = 14318;
- else
- ref_pll = 12000;
-
- denum = data & 0x1f;
- num = (data & 0x3fe0) >> 5;
- data = (data & 0xc000) >> 14;
- switch (data) {
- case 3:
- div = 0x4;
- break;
- case 2:
- case 1:
- div = 0x2;
- break;
- default:
- div = 0x1;
- break;
- }
- ast->mclk = ref_pll * (num + 2) / (denum + 2) * (div * 1000);
return 0;
}
ast_open_key(ast);
ast_set_def_ext_reg(dev);
- if (ast->chip == AST2300 || ast->chip == AST2400)
- ast_init_dram_2300(dev);
- else
- ast_init_dram_reg(dev);
+ if (ast->DisableP2A == false)
+ {
+ if (ast->chip == AST2300 || ast->chip == AST2400)
+ ast_init_dram_2300(dev);
+ else
+ ast_init_dram_reg(dev);
- ast_init_3rdtx(dev);
+ ast_init_3rdtx(dev);
+ }
+ else
+ {
+ if (ast->tx_chip_type != AST_TX_NONE)
+ ast_set_index_reg_mask(ast, AST_IO_CRTC_PORT, 0xa3, 0xcf, 0x80); /* Enable DVO */
+ }
}
/* AST 2300 DRAM settings */
pm_runtime_enable(dev);
+ pm_runtime_get_sync(dev);
phy_power_on(dp->phy);
analogix_dp_init_dp(dp);
goto err_disable_pm_runtime;
}
+ phy_power_off(dp->phy);
+ pm_runtime_put(dev);
+
return 0;
err_disable_pm_runtime:
+
+ phy_power_off(dp->phy);
+ pm_runtime_put(dev);
pm_runtime_disable(dev);
return ret;
This is a KMS driver for emulated cirrus device in qemu.
It is *NOT* intended for real cirrus devices. This requires
the modesetting userspace X.org driver.
+
+ Cirrus is obsolete, the hardware was designed in the 90ies
+ and can't keep up with todays needs. More background:
+ https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/
+
+ Better alternatives are:
+ - stdvga (DRM_BOCHS, qemu -vga std, default in qemu 2.2+)
+ - qxl (DRM_QXL, qemu -vga qxl, works best with spice)
+ - virtio (DRM_VIRTIO_GPU), qemu -vga virtio)
EXPORT_SYMBOL(drm_atomic_get_crtc_state);
static void set_out_fence_for_crtc(struct drm_atomic_state *state,
- struct drm_crtc *crtc, s64 __user *fence_ptr)
+ struct drm_crtc *crtc, s32 __user *fence_ptr)
{
state->crtcs[drm_crtc_index(crtc)].out_fence_ptr = fence_ptr;
}
-static s64 __user *get_out_fence_for_crtc(struct drm_atomic_state *state,
+static s32 __user *get_out_fence_for_crtc(struct drm_atomic_state *state,
struct drm_crtc *crtc)
{
- s64 __user *fence_ptr;
+ s32 __user *fence_ptr;
fence_ptr = state->crtcs[drm_crtc_index(crtc)].out_fence_ptr;
state->crtcs[drm_crtc_index(crtc)].out_fence_ptr = NULL;
state->color_mgmt_changed |= replaced;
return ret;
} else if (property == config->prop_out_fence_ptr) {
- s64 __user *fence_ptr = u64_to_user_ptr(val);
+ s32 __user *fence_ptr = u64_to_user_ptr(val);
if (!fence_ptr)
return 0;
*/
struct drm_out_fence_state {
- s64 __user *out_fence_ptr;
+ s32 __user *out_fence_ptr;
struct sync_file *sync_file;
int fd;
};
return 0;
for_each_crtc_in_state(state, crtc, crtc_state, i) {
- u64 __user *fence_ptr;
+ s32 __user *fence_ptr;
fence_ptr = get_out_fence_for_crtc(crtc_state->state, crtc);
}
for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ struct drm_pending_vblank_event *event = crtc_state->event;
/*
- * TEST_ONLY and PAGE_FLIP_EVENT are mutually
- * exclusive, if they weren't, this code should be
- * called on success for TEST_ONLY too.
+ * Free the allocated event. drm_atomic_helper_setup_commit
+ * can allocate an event too, so only free it if it's ours
+ * to prevent a double free in drm_atomic_state_clear.
*/
- if (crtc_state->event)
- drm_event_cancel_free(dev, &crtc_state->event->base);
+ if (event && (event->base.fence || event->base.file_priv)) {
+ drm_event_cancel_free(dev, &event->base);
+ crtc_state->event = NULL;
+ }
}
if (!fence_state)
funcs = plane->helper_private;
- if (!drm_atomic_helper_framebuffer_changed(dev, state, plane_state->crtc))
- continue;
-
if (funcs->prepare_fb) {
ret = funcs->prepare_fb(plane, plane_state);
if (ret)
if (j >= i)
continue;
- if (!drm_atomic_helper_framebuffer_changed(dev, state, plane_state->crtc))
- continue;
-
funcs = plane->helper_private;
if (funcs->cleanup_fb)
for_each_plane_in_state(old_state, plane, plane_state, i) {
const struct drm_plane_helper_funcs *funcs;
- if (!drm_atomic_helper_framebuffer_changed(dev, old_state, plane_state->crtc))
- continue;
-
funcs = plane->helper_private;
if (funcs->cleanup_fb)
INIT_LIST_HEAD(&connector->probed_modes);
INIT_LIST_HEAD(&connector->modes);
+ mutex_init(&connector->mutex);
connector->edid_blob_ptr = NULL;
connector->status = connector_status_unknown;
connector->funcs->atomic_destroy_state(connector,
connector->state);
+ mutex_destroy(&connector->mutex);
+
memset(connector, 0, sizeof(*connector));
}
EXPORT_SYMBOL(drm_connector_cleanup);
*/
int drm_connector_register(struct drm_connector *connector)
{
- int ret;
+ int ret = 0;
- if (connector->registered)
+ if (!connector->dev->registered)
return 0;
+ mutex_lock(&connector->mutex);
+ if (connector->registered)
+ goto unlock;
+
ret = drm_sysfs_connector_add(connector);
if (ret)
- return ret;
+ goto unlock;
ret = drm_debugfs_connector_add(connector);
if (ret) {
drm_mode_object_register(connector->dev, &connector->base);
connector->registered = true;
- return 0;
+ goto unlock;
err_debugfs:
drm_debugfs_connector_remove(connector);
err_sysfs:
drm_sysfs_connector_remove(connector);
+unlock:
+ mutex_unlock(&connector->mutex);
return ret;
}
EXPORT_SYMBOL(drm_connector_register);
*/
void drm_connector_unregister(struct drm_connector *connector)
{
- if (!connector->registered)
+ mutex_lock(&connector->mutex);
+ if (!connector->registered) {
+ mutex_unlock(&connector->mutex);
return;
+ }
if (connector->funcs->early_unregister)
connector->funcs->early_unregister(connector);
drm_debugfs_connector_remove(connector);
connector->registered = false;
+ mutex_unlock(&connector->mutex);
}
EXPORT_SYMBOL(drm_connector_unregister);
if (ret)
goto err_minors;
+ dev->registered = true;
+
if (dev->driver->load) {
ret = dev->driver->load(dev, flags);
if (ret)
drm_lastclose(dev);
+ dev->registered = false;
+
if (drm_core_check_feature(dev, DRIVER_MODESET))
drm_modeset_unregister_all(dev);
return NULL;
mode->type |= DRM_MODE_TYPE_USERDEF;
+ /* fix up 1368x768: GFT/CVT can't express 1366 width due to alignment */
+ if (cmd->xres == 1366 && mode->hdisplay == 1368) {
+ mode->hdisplay = 1366;
+ mode->hsync_start--;
+ mode->hsync_end--;
+ drm_mode_set_name(mode);
+ }
drm_mode_set_crtcinfo(mode, CRTC_INTERLACE_HALVE_V);
return mode;
}
}
if (dev->mode_config.delayed_event) {
+ /*
+ * FIXME:
+ *
+ * Use short (1s) delay to handle the initial delayed event.
+ * This delay should not be needed, but Optimus/nouveau will
+ * fail in a mysterious way if the delayed event is handled as
+ * soon as possible like it is done in
+ * drm_helper_probe_single_connector_modes() in case the poll
+ * was enabled before.
+ */
poll = true;
- delay = 0;
+ delay = HZ;
}
if (poll)
struct list_head list;
bool found;
+ /*
+ * XXX: The DRM_MM_SEARCH_BELOW is really a hack to trick
+ * drm_mm into giving out a low IOVA after address space
+ * rollover. This needs a proper fix.
+ */
ret = drm_mm_insert_node_in_range(&mmu->mm, node,
size, 0, mmu->last_iova, ~0UL,
- DRM_MM_SEARCH_DEFAULT);
+ mmu->last_iova ? DRM_MM_SEARCH_DEFAULT : DRM_MM_SEARCH_BELOW);
if (ret != -ENOSPC)
break;
BIT_CLKS_ENABLED,
BIT_IRQS_ENABLED,
BIT_WIN_UPDATED,
- BIT_SUSPENDED
+ BIT_SUSPENDED,
+ BIT_REQUEST_UPDATE
};
struct decon_context {
m->crtc_vsync_end = m->crtc_vsync_start + 1;
}
- decon_set_bits(ctx, DECON_VIDCON0, VIDCON0_ENVID, 0);
-
- /* enable clock gate */
- val = CMU_CLKGAGE_MODE_SFR_F | CMU_CLKGAGE_MODE_MEM_F;
- writel(val, ctx->addr + DECON_CMU);
-
if (ctx->out_type & (IFTYPE_I80 | I80_HW_TRG))
decon_setup_trigger(ctx);
/* window enable */
decon_set_bits(ctx, DECON_WINCONx(win), WINCONx_ENWIN_F, ~0);
+ set_bit(BIT_REQUEST_UPDATE, &ctx->flags);
}
static void decon_disable_plane(struct exynos_drm_crtc *crtc,
return;
decon_set_bits(ctx, DECON_WINCONx(win), WINCONx_ENWIN_F, 0);
+ set_bit(BIT_REQUEST_UPDATE, &ctx->flags);
}
static void decon_atomic_flush(struct exynos_drm_crtc *crtc)
for (i = ctx->first_win; i < WINDOWS_NR; i++)
decon_shadow_protect_win(ctx, i, false);
- /* standalone update */
- decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
+ if (test_and_clear_bit(BIT_REQUEST_UPDATE, &ctx->flags))
+ decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
if (ctx->out_type & IFTYPE_I80)
set_bit(BIT_WIN_UPDATED, &ctx->flags);
#include "i915_drv.h"
#include "gvt.h"
-#define MB_TO_BYTES(mb) ((mb) << 20ULL)
-#define BYTES_TO_MB(b) ((b) >> 20ULL)
-
-#define HOST_LOW_GM_SIZE MB_TO_BYTES(128)
-#define HOST_HIGH_GM_SIZE MB_TO_BYTES(384)
-#define HOST_FENCE 4
-
static int alloc_gm(struct intel_vgpu *vgpu, bool high_gm)
{
struct intel_gvt *gvt = vgpu->gvt;
POSTING_READ(fence_reg_lo);
}
+static void _clear_vgpu_fence(struct intel_vgpu *vgpu)
+{
+ int i;
+
+ for (i = 0; i < vgpu_fence_sz(vgpu); i++)
+ intel_vgpu_write_fence(vgpu, i, 0);
+}
+
static void free_vgpu_fence(struct intel_vgpu *vgpu)
{
struct intel_gvt *gvt = vgpu->gvt;
intel_runtime_pm_get(dev_priv);
mutex_lock(&dev_priv->drm.struct_mutex);
+ _clear_vgpu_fence(vgpu);
for (i = 0; i < vgpu_fence_sz(vgpu); i++) {
reg = vgpu->fence.regs[i];
- intel_vgpu_write_fence(vgpu, i, 0);
list_add_tail(®->link,
&dev_priv->mm.fence_list);
}
continue;
list_del(pos);
vgpu->fence.regs[i] = reg;
- intel_vgpu_write_fence(vgpu, i, 0);
if (++i == vgpu_fence_sz(vgpu))
break;
}
if (i != vgpu_fence_sz(vgpu))
goto out_free_fence;
+ _clear_vgpu_fence(vgpu);
+
mutex_unlock(&dev_priv->drm.struct_mutex);
intel_runtime_pm_put(dev_priv);
return 0;
free_resource(vgpu);
}
+/**
+ * intel_vgpu_reset_resource - reset resource state owned by a vGPU
+ * @vgpu: a vGPU
+ *
+ * This function is used to reset resource state owned by a vGPU.
+ *
+ */
+void intel_vgpu_reset_resource(struct intel_vgpu *vgpu)
+{
+ struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
+
+ intel_runtime_pm_get(dev_priv);
+ _clear_vgpu_fence(vgpu);
+ intel_runtime_pm_put(dev_priv);
+}
+
/**
* intel_alloc_vgpu_resource - allocate HW resource for a vGPU
* @vgpu: vGPU
}
return 0;
}
+
+/**
+ * intel_vgpu_init_cfg_space - init vGPU configuration space when create vGPU
+ *
+ * @vgpu: a vGPU
+ * @primary: is the vGPU presented as primary
+ *
+ */
+void intel_vgpu_init_cfg_space(struct intel_vgpu *vgpu,
+ bool primary)
+{
+ struct intel_gvt *gvt = vgpu->gvt;
+ const struct intel_gvt_device_info *info = &gvt->device_info;
+ u16 *gmch_ctl;
+ int i;
+
+ memcpy(vgpu_cfg_space(vgpu), gvt->firmware.cfg_space,
+ info->cfg_space_size);
+
+ if (!primary) {
+ vgpu_cfg_space(vgpu)[PCI_CLASS_DEVICE] =
+ INTEL_GVT_PCI_CLASS_VGA_OTHER;
+ vgpu_cfg_space(vgpu)[PCI_CLASS_PROG] =
+ INTEL_GVT_PCI_CLASS_VGA_OTHER;
+ }
+
+ /* Show guest that there isn't any stolen memory.*/
+ gmch_ctl = (u16 *)(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_GMCH_CONTROL);
+ *gmch_ctl &= ~(BDW_GMCH_GMS_MASK << BDW_GMCH_GMS_SHIFT);
+
+ intel_vgpu_write_pci_bar(vgpu, PCI_BASE_ADDRESS_2,
+ gvt_aperture_pa_base(gvt), true);
+
+ vgpu_cfg_space(vgpu)[PCI_COMMAND] &= ~(PCI_COMMAND_IO
+ | PCI_COMMAND_MEMORY
+ | PCI_COMMAND_MASTER);
+ /*
+ * Clear the bar upper 32bit and let guest to assign the new value
+ */
+ memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_1, 0, 4);
+ memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_3, 0, 4);
+ memset(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_OPREGION, 0, 4);
+
+ for (i = 0; i < INTEL_GVT_MAX_BAR_NUM; i++) {
+ vgpu->cfg_space.bar[i].size = pci_resource_len(
+ gvt->dev_priv->drm.pdev, i * 2);
+ vgpu->cfg_space.bar[i].tracked = false;
+ }
+}
+
+/**
+ * intel_vgpu_reset_cfg_space - reset vGPU configuration space
+ *
+ * @vgpu: a vGPU
+ *
+ */
+void intel_vgpu_reset_cfg_space(struct intel_vgpu *vgpu)
+{
+ u8 cmd = vgpu_cfg_space(vgpu)[PCI_COMMAND];
+ bool primary = vgpu_cfg_space(vgpu)[PCI_CLASS_DEVICE] !=
+ INTEL_GVT_PCI_CLASS_VGA_OTHER;
+
+ if (cmd & PCI_COMMAND_MEMORY) {
+ trap_gttmmio(vgpu, false);
+ map_aperture(vgpu, false);
+ }
+
+ /**
+ * Currently we only do such reset when vGPU is not
+ * owned by any VM, so we simply restore entire cfg
+ * space to default value.
+ */
+ intel_vgpu_init_cfg_space(vgpu, primary);
+}
(s->vgpu->gvt->device_info.gmadr_bytes_in_cmd >> 2)
static unsigned long bypass_scan_mask = 0;
-static bool bypass_batch_buffer_scan = true;
/* ring ALL, type = 0 */
static struct sub_op_bits sub_op_mi[] = {
{
struct intel_gvt *gvt = s->vgpu->gvt;
- if (bypass_batch_buffer_scan)
- return 0;
-
if (IS_BROADWELL(gvt->dev_priv) || IS_SKYLAKE(gvt->dev_priv)) {
/* BDW decides privilege based on address space */
if (cmd_val(s, 0) & (1 << 8))
#define get_desc_from_elsp_dwords(ed, i) \
((struct execlist_ctx_descriptor_format *)&((ed)->data[i * 2]))
-
-#define BATCH_BUFFER_ADDR_MASK ((1UL << 32) - (1U << 2))
-#define BATCH_BUFFER_ADDR_HIGH_MASK ((1UL << 16) - (1U))
-static int set_gma_to_bb_cmd(struct intel_shadow_bb_entry *entry_obj,
- unsigned long add, int gmadr_bytes)
-{
- if (WARN_ON(gmadr_bytes != 4 && gmadr_bytes != 8))
- return -1;
-
- *((u32 *)(entry_obj->bb_start_cmd_va + (1 << 2))) = add &
- BATCH_BUFFER_ADDR_MASK;
- if (gmadr_bytes == 8) {
- *((u32 *)(entry_obj->bb_start_cmd_va + (2 << 2))) =
- add & BATCH_BUFFER_ADDR_HIGH_MASK;
- }
-
- return 0;
-}
-
static void prepare_shadow_batch_buffer(struct intel_vgpu_workload *workload)
{
- int gmadr_bytes = workload->vgpu->gvt->device_info.gmadr_bytes_in_cmd;
+ const int gmadr_bytes = workload->vgpu->gvt->device_info.gmadr_bytes_in_cmd;
+ struct intel_shadow_bb_entry *entry_obj;
/* pin the gem object to ggtt */
- if (!list_empty(&workload->shadow_bb)) {
- struct intel_shadow_bb_entry *entry_obj =
- list_first_entry(&workload->shadow_bb,
- struct intel_shadow_bb_entry,
- list);
- struct intel_shadow_bb_entry *temp;
+ list_for_each_entry(entry_obj, &workload->shadow_bb, list) {
+ struct i915_vma *vma;
- list_for_each_entry_safe(entry_obj, temp, &workload->shadow_bb,
- list) {
- struct i915_vma *vma;
-
- vma = i915_gem_object_ggtt_pin(entry_obj->obj, NULL, 0,
- 4, 0);
- if (IS_ERR(vma)) {
- gvt_err("Cannot pin\n");
- return;
- }
-
- /* FIXME: we are not tracking our pinned VMA leaving it
- * up to the core to fix up the stray pin_count upon
- * free.
- */
-
- /* update the relocate gma with shadow batch buffer*/
- set_gma_to_bb_cmd(entry_obj,
- i915_ggtt_offset(vma),
- gmadr_bytes);
+ vma = i915_gem_object_ggtt_pin(entry_obj->obj, NULL, 0, 4, 0);
+ if (IS_ERR(vma)) {
+ gvt_err("Cannot pin\n");
+ return;
}
+
+ /* FIXME: we are not tracking our pinned VMA leaving it
+ * up to the core to fix up the stray pin_count upon
+ * free.
+ */
+
+ /* update the relocate gma with shadow batch buffer*/
+ entry_obj->bb_start_cmd_va[1] = i915_ggtt_offset(vma);
+ if (gmadr_bytes == 8)
+ entry_obj->bb_start_cmd_va[2] = 0;
}
}
INIT_LIST_HEAD(&vgpu->workload_q_head[i]);
}
- vgpu->workloads = kmem_cache_create("gvt-g vgpu workload",
+ vgpu->workloads = kmem_cache_create("gvt-g_vgpu_workload",
sizeof(struct intel_vgpu_workload), 0,
SLAB_HWCACHE_ALIGN,
NULL);
static u64 read_pte64(struct drm_i915_private *dev_priv, unsigned long index)
{
void __iomem *addr = (gen8_pte_t __iomem *)dev_priv->ggtt.gsm + index;
- u64 pte;
-#ifdef readq
- pte = readq(addr);
-#else
- pte = ioread32(addr);
- pte |= (u64)ioread32(addr + 4) << 32;
-#endif
- return pte;
+ return readq(addr);
}
static void write_pte64(struct drm_i915_private *dev_priv,
{
void __iomem *addr = (gen8_pte_t __iomem *)dev_priv->ggtt.gsm + index;
-#ifdef writeq
writeq(pte, addr);
-#else
- iowrite32((u32)pte, addr);
- iowrite32(pte >> 32, addr + 4);
-#endif
+
I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
POSTING_READ(GFX_FLSH_CNTL_GEN6);
}
info->gtt_entry_size;
mem = kzalloc(mm->has_shadow_page_table ?
mm->page_table_entry_size * 2
- : mm->page_table_entry_size,
- GFP_ATOMIC);
+ : mm->page_table_entry_size, GFP_KERNEL);
if (!mem)
return -ENOMEM;
mm->virtual_page_table = mem;
struct intel_vgpu_mm *mm;
int ret;
- mm = kzalloc(sizeof(*mm), GFP_ATOMIC);
+ mm = kzalloc(sizeof(*mm), GFP_KERNEL);
if (!mm) {
ret = -ENOMEM;
goto fail;
struct intel_gvt_gtt_pte_ops *ops = vgpu->gvt->gtt.pte_ops;
int page_entry_num = GTT_PAGE_SIZE >>
vgpu->gvt->device_info.gtt_entry_size_shift;
- struct page *scratch_pt;
+ void *scratch_pt;
unsigned long mfn;
int i;
- void *p;
if (WARN_ON(type < GTT_TYPE_PPGTT_PTE_PT || type >= GTT_TYPE_MAX))
return -EINVAL;
- scratch_pt = alloc_page(GFP_KERNEL | GFP_ATOMIC | __GFP_ZERO);
+ scratch_pt = (void *)get_zeroed_page(GFP_KERNEL);
if (!scratch_pt) {
gvt_err("fail to allocate scratch page\n");
return -ENOMEM;
}
- p = kmap_atomic(scratch_pt);
- mfn = intel_gvt_hypervisor_virt_to_mfn(p);
+ mfn = intel_gvt_hypervisor_virt_to_mfn(scratch_pt);
if (mfn == INTEL_GVT_INVALID_ADDR) {
- gvt_err("fail to translate vaddr:0x%llx\n", (u64)p);
- kunmap_atomic(p);
- __free_page(scratch_pt);
+ gvt_err("fail to translate vaddr:0x%lx\n", (unsigned long)scratch_pt);
+ free_page((unsigned long)scratch_pt);
return -EFAULT;
}
gtt->scratch_pt[type].page_mfn = mfn;
- gtt->scratch_pt[type].page = scratch_pt;
+ gtt->scratch_pt[type].page = virt_to_page(scratch_pt);
gvt_dbg_mm("vgpu%d create scratch_pt: type %d mfn=0x%lx\n",
vgpu->id, type, mfn);
* scratch_pt[type] indicate the scratch pt/scratch page used by the
* 'type' pt.
* e.g. scratch_pt[GTT_TYPE_PPGTT_PDE_PT] is used by
- * GTT_TYPE_PPGTT_PDE_PT level pt, that means this scatch_pt it self
+ * GTT_TYPE_PPGTT_PDE_PT level pt, that means this scratch_pt it self
* is GTT_TYPE_PPGTT_PTE_PT, and full filled by scratch page mfn.
*/
if (type > GTT_TYPE_PPGTT_PTE_PT && type < GTT_TYPE_MAX) {
se.val64 |= PPAT_CACHED_INDEX;
for (i = 0; i < page_entry_num; i++)
- ops->set_entry(p, &se, i, false, 0, vgpu);
+ ops->set_entry(scratch_pt, &se, i, false, 0, vgpu);
}
- kunmap_atomic(p);
-
return 0;
}
int intel_gvt_init_gtt(struct intel_gvt *gvt)
{
int ret;
- void *page_addr;
+ void *page;
gvt_dbg_core("init gtt\n");
return -ENODEV;
}
- gvt->gtt.scratch_ggtt_page =
- alloc_page(GFP_KERNEL | GFP_ATOMIC | __GFP_ZERO);
- if (!gvt->gtt.scratch_ggtt_page) {
+ page = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!page) {
gvt_err("fail to allocate scratch ggtt page\n");
return -ENOMEM;
}
+ gvt->gtt.scratch_ggtt_page = virt_to_page(page);
- page_addr = page_address(gvt->gtt.scratch_ggtt_page);
-
- gvt->gtt.scratch_ggtt_mfn =
- intel_gvt_hypervisor_virt_to_mfn(page_addr);
+ gvt->gtt.scratch_ggtt_mfn = intel_gvt_hypervisor_virt_to_mfn(page);
if (gvt->gtt.scratch_ggtt_mfn == INTEL_GVT_INVALID_ADDR) {
gvt_err("fail to translate scratch ggtt page\n");
__free_page(gvt->gtt.scratch_ggtt_page);
for (offset = 0; offset < num_entries; offset++)
ops->set_entry(NULL, &e, index + offset, false, 0, vgpu);
}
+
+/**
+ * intel_vgpu_reset_gtt - reset the all GTT related status
+ * @vgpu: a vGPU
+ * @dmlr: true for vGPU Device Model Level Reset, false for GT Reset
+ *
+ * This function is called from vfio core to reset reset all
+ * GTT related status, including GGTT, PPGTT, scratch page.
+ *
+ */
+void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr)
+{
+ int i;
+
+ ppgtt_free_all_shadow_page(vgpu);
+ if (!dmlr)
+ return;
+
+ intel_vgpu_reset_ggtt(vgpu);
+
+ /* clear scratch page for security */
+ for (i = GTT_TYPE_PPGTT_PTE_PT; i < GTT_TYPE_MAX; i++) {
+ if (vgpu->gtt.scratch_pt[i].page != NULL)
+ memset(page_address(vgpu->gtt.scratch_pt[i].page),
+ 0, PAGE_SIZE);
+ }
+}
void intel_vgpu_reset_ggtt(struct intel_vgpu *vgpu);
extern int intel_gvt_init_gtt(struct intel_gvt *gvt);
+extern void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr);
extern void intel_gvt_clean_gtt(struct intel_gvt *gvt);
extern struct intel_vgpu_mm *intel_gvt_find_ppgtt_mm(struct intel_vgpu *vgpu,
intel_gvt_hypervisor_host_exit(&dev_priv->drm.pdev->dev, gvt);
intel_gvt_clean_vgpu_types(gvt);
+ idr_destroy(&gvt->vgpu_idr);
+
kfree(dev_priv->gvt);
dev_priv->gvt = NULL;
}
gvt_dbg_core("init gvt device\n");
+ idr_init(&gvt->vgpu_idr);
+
mutex_init(&gvt->lock);
gvt->dev_priv = dev_priv;
ret = intel_gvt_setup_mmio_info(gvt);
if (ret)
- return ret;
+ goto out_clean_idr;
ret = intel_gvt_load_firmware(gvt);
if (ret)
intel_gvt_free_firmware(gvt);
out_clean_mmio_info:
intel_gvt_clean_mmio_info(gvt);
+out_clean_idr:
+ idr_destroy(&gvt->vgpu_idr);
kfree(gvt);
return ret;
}
int intel_vgpu_alloc_resource(struct intel_vgpu *vgpu,
struct intel_vgpu_creation_params *param);
+void intel_vgpu_reset_resource(struct intel_vgpu *vgpu);
void intel_vgpu_free_resource(struct intel_vgpu *vgpu);
void intel_vgpu_write_fence(struct intel_vgpu *vgpu,
u32 fence, u64 value);
struct intel_vgpu *intel_gvt_create_vgpu(struct intel_gvt *gvt,
struct intel_vgpu_type *type);
void intel_gvt_destroy_vgpu(struct intel_vgpu *vgpu);
+void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr,
+ unsigned int engine_mask);
void intel_gvt_reset_vgpu(struct intel_vgpu *vgpu);
int intel_gvt_ggtt_h2g_index(struct intel_vgpu *vgpu, unsigned long h_index,
unsigned long *g_index);
+void intel_vgpu_init_cfg_space(struct intel_vgpu *vgpu,
+ bool primary);
+void intel_vgpu_reset_cfg_space(struct intel_vgpu *vgpu);
+
int intel_vgpu_emulate_cfg_read(struct intel_vgpu *vgpu, unsigned int offset,
void *p_data, unsigned int bytes);
int intel_vgpu_init_opregion(struct intel_vgpu *vgpu, u32 gpa);
int intel_vgpu_emulate_opregion_request(struct intel_vgpu *vgpu, u32 swsci);
-int setup_vgpu_mmio(struct intel_vgpu *vgpu);
void populate_pvinfo_page(struct intel_vgpu *vgpu);
struct intel_gvt_ops {
static int new_mmio_info(struct intel_gvt *gvt,
u32 offset, u32 flags, u32 size,
u32 addr_mask, u32 ro_mask, u32 device,
- void *read, void *write)
+ int (*read)(struct intel_vgpu *, unsigned int, void *, unsigned int),
+ int (*write)(struct intel_vgpu *, unsigned int, void *, unsigned int))
{
struct intel_gvt_mmio_info *info, *p;
u32 start, end, i;
default:
/*should not hit here*/
gvt_err("invalid forcewake offset 0x%x\n", offset);
- return 1;
+ return -EINVAL;
}
} else {
ack_reg_offset = FORCEWAKE_ACK_HSW_REG;
return 0;
}
-static int handle_device_reset(struct intel_vgpu *vgpu, unsigned int offset,
- void *p_data, unsigned int bytes, unsigned long bitmap)
-{
- struct intel_gvt_workload_scheduler *scheduler =
- &vgpu->gvt->scheduler;
-
- vgpu->resetting = true;
-
- intel_vgpu_stop_schedule(vgpu);
- /*
- * The current_vgpu will set to NULL after stopping the
- * scheduler when the reset is triggered by current vgpu.
- */
- if (scheduler->current_vgpu == NULL) {
- mutex_unlock(&vgpu->gvt->lock);
- intel_gvt_wait_vgpu_idle(vgpu);
- mutex_lock(&vgpu->gvt->lock);
- }
-
- intel_vgpu_reset_execlist(vgpu, bitmap);
-
- /* full GPU reset */
- if (bitmap == 0xff) {
- mutex_unlock(&vgpu->gvt->lock);
- intel_vgpu_clean_gtt(vgpu);
- mutex_lock(&vgpu->gvt->lock);
- setup_vgpu_mmio(vgpu);
- populate_pvinfo_page(vgpu);
- intel_vgpu_init_gtt(vgpu);
- }
-
- vgpu->resetting = false;
-
- return 0;
-}
-
static int gdrst_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
- void *p_data, unsigned int bytes)
+ void *p_data, unsigned int bytes)
{
+ unsigned int engine_mask = 0;
u32 data;
- u64 bitmap = 0;
write_vreg(vgpu, offset, p_data, bytes);
data = vgpu_vreg(vgpu, offset);
if (data & GEN6_GRDOM_FULL) {
gvt_dbg_mmio("vgpu%d: request full GPU reset\n", vgpu->id);
- bitmap = 0xff;
- }
- if (data & GEN6_GRDOM_RENDER) {
- gvt_dbg_mmio("vgpu%d: request RCS reset\n", vgpu->id);
- bitmap |= (1 << RCS);
- }
- if (data & GEN6_GRDOM_MEDIA) {
- gvt_dbg_mmio("vgpu%d: request VCS reset\n", vgpu->id);
- bitmap |= (1 << VCS);
- }
- if (data & GEN6_GRDOM_BLT) {
- gvt_dbg_mmio("vgpu%d: request BCS Reset\n", vgpu->id);
- bitmap |= (1 << BCS);
- }
- if (data & GEN6_GRDOM_VECS) {
- gvt_dbg_mmio("vgpu%d: request VECS Reset\n", vgpu->id);
- bitmap |= (1 << VECS);
- }
- if (data & GEN8_GRDOM_MEDIA2) {
- gvt_dbg_mmio("vgpu%d: request VCS2 Reset\n", vgpu->id);
- if (HAS_BSD2(vgpu->gvt->dev_priv))
- bitmap |= (1 << VCS2);
+ engine_mask = ALL_ENGINES;
+ } else {
+ if (data & GEN6_GRDOM_RENDER) {
+ gvt_dbg_mmio("vgpu%d: request RCS reset\n", vgpu->id);
+ engine_mask |= (1 << RCS);
+ }
+ if (data & GEN6_GRDOM_MEDIA) {
+ gvt_dbg_mmio("vgpu%d: request VCS reset\n", vgpu->id);
+ engine_mask |= (1 << VCS);
+ }
+ if (data & GEN6_GRDOM_BLT) {
+ gvt_dbg_mmio("vgpu%d: request BCS Reset\n", vgpu->id);
+ engine_mask |= (1 << BCS);
+ }
+ if (data & GEN6_GRDOM_VECS) {
+ gvt_dbg_mmio("vgpu%d: request VECS Reset\n", vgpu->id);
+ engine_mask |= (1 << VECS);
+ }
+ if (data & GEN8_GRDOM_MEDIA2) {
+ gvt_dbg_mmio("vgpu%d: request VCS2 Reset\n", vgpu->id);
+ if (HAS_BSD2(vgpu->gvt->dev_priv))
+ engine_mask |= (1 << VCS2);
+ }
}
- return handle_device_reset(vgpu, offset, p_data, bytes, bitmap);
+
+ intel_gvt_reset_vgpu_locked(vgpu, false, engine_mask);
+
+ return 0;
}
static int gmbus_mmio_read(struct intel_vgpu *vgpu, unsigned int offset,
return 0;
}
-static bool sbi_ctl_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
+static int sbi_ctl_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
void *p_data, unsigned int bytes)
{
u32 data;
static int gvt_reg_tlb_control_handler(struct intel_vgpu *vgpu,
unsigned int offset, void *p_data, unsigned int bytes)
{
- int rc = 0;
unsigned int id = 0;
write_vreg(vgpu, offset, p_data, bytes);
id = VECS;
break;
default:
- rc = -EINVAL;
- break;
+ return -EINVAL;
}
set_bit(id, (void *)vgpu->tlb_handle_pending);
- return rc;
+ return 0;
}
static int ring_reset_ctl_write(struct intel_vgpu *vgpu,
return NULL;
}
-static ssize_t available_instance_show(struct kobject *kobj, struct device *dev,
- char *buf)
+static ssize_t available_instances_show(struct kobject *kobj,
+ struct device *dev, char *buf)
{
struct intel_vgpu_type *type;
unsigned int num = 0;
type->fence);
}
-static MDEV_TYPE_ATTR_RO(available_instance);
+static MDEV_TYPE_ATTR_RO(available_instances);
static MDEV_TYPE_ATTR_RO(device_api);
static MDEV_TYPE_ATTR_RO(description);
static struct attribute *type_attrs[] = {
- &mdev_type_attr_available_instance.attr,
+ &mdev_type_attr_available_instances.attr,
&mdev_type_attr_device_api.attr,
&mdev_type_attr_description.attr,
NULL,
struct intel_vgpu_type *type;
struct device *pdev;
void *gvt;
+ int ret;
pdev = mdev_parent_dev(mdev);
gvt = kdev_to_i915(pdev)->gvt;
if (!type) {
gvt_err("failed to find type %s to create\n",
kobject_name(kobj));
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
vgpu = intel_gvt_ops->vgpu_create(gvt, type);
if (IS_ERR_OR_NULL(vgpu)) {
- gvt_err("create intel vgpu failed\n");
- return -EINVAL;
+ ret = vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
+ gvt_err("failed to create intel vgpu: %d\n", ret);
+ goto out;
}
INIT_WORK(&vgpu->vdev.release_work, intel_vgpu_release_work);
gvt_dbg_core("intel_vgpu_create succeeded for mdev: %s\n",
dev_name(mdev_dev(mdev)));
- return 0;
+ ret = 0;
+
+out:
+ return ret;
}
static int intel_vgpu_remove(struct mdev_device *mdev)
if (WARN_ON(!reg_is_mmio(gvt, offset + bytes - 1)))
goto err;
- mmio = intel_gvt_find_mmio_info(gvt, rounddown(offset, 4));
- if (!mmio && !vgpu->mmio.disable_warn_untrack) {
- gvt_err("vgpu%d: read untracked MMIO %x len %d val %x\n",
- vgpu->id, offset, bytes, *(u32 *)p_data);
-
- if (offset == 0x206c) {
- gvt_err("------------------------------------------\n");
- gvt_err("vgpu%d: likely triggers a gfx reset\n",
- vgpu->id);
- gvt_err("------------------------------------------\n");
- vgpu->mmio.disable_warn_untrack = true;
- }
- }
-
if (!intel_gvt_mmio_is_unalign(gvt, offset)) {
if (WARN_ON(!IS_ALIGNED(offset, bytes)))
goto err;
}
+ mmio = intel_gvt_find_mmio_info(gvt, rounddown(offset, 4));
if (mmio) {
if (!intel_gvt_mmio_is_unalign(gvt, mmio->offset)) {
if (WARN_ON(offset + bytes > mmio->offset + mmio->size))
goto err;
}
ret = mmio->read(vgpu, offset, p_data, bytes);
- } else
+ } else {
ret = intel_vgpu_default_mmio_read(vgpu, offset, p_data, bytes);
+ if (!vgpu->mmio.disable_warn_untrack) {
+ gvt_err("vgpu%d: read untracked MMIO %x(%dB) val %x\n",
+ vgpu->id, offset, bytes, *(u32 *)p_data);
+
+ if (offset == 0x206c) {
+ gvt_err("------------------------------------------\n");
+ gvt_err("vgpu%d: likely triggers a gfx reset\n",
+ vgpu->id);
+ gvt_err("------------------------------------------\n");
+ vgpu->mmio.disable_warn_untrack = true;
+ }
+ }
+ }
+
if (ret)
goto err;
mutex_unlock(&gvt->lock);
return ret;
}
+
+
+/**
+ * intel_vgpu_reset_mmio - reset virtual MMIO space
+ * @vgpu: a vGPU
+ *
+ */
+void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu)
+{
+ struct intel_gvt *gvt = vgpu->gvt;
+ const struct intel_gvt_device_info *info = &gvt->device_info;
+
+ memcpy(vgpu->mmio.vreg, gvt->firmware.mmio, info->mmio_size);
+ memcpy(vgpu->mmio.sreg, gvt->firmware.mmio, info->mmio_size);
+
+ vgpu_vreg(vgpu, GEN6_GT_THREAD_STATUS_REG) = 0;
+
+ /* set the bit 0:2(Core C-State ) to C0 */
+ vgpu_vreg(vgpu, GEN6_GT_CORE_STATUS) = 0;
+}
+
+/**
+ * intel_vgpu_init_mmio - init MMIO space
+ * @vgpu: a vGPU
+ *
+ * Returns:
+ * Zero on success, negative error code if failed
+ */
+int intel_vgpu_init_mmio(struct intel_vgpu *vgpu)
+{
+ const struct intel_gvt_device_info *info = &vgpu->gvt->device_info;
+
+ vgpu->mmio.vreg = vzalloc(info->mmio_size * 2);
+ if (!vgpu->mmio.vreg)
+ return -ENOMEM;
+
+ vgpu->mmio.sreg = vgpu->mmio.vreg + info->mmio_size;
+
+ intel_vgpu_reset_mmio(vgpu);
+
+ return 0;
+}
+
+/**
+ * intel_vgpu_clean_mmio - clean MMIO space
+ * @vgpu: a vGPU
+ *
+ */
+void intel_vgpu_clean_mmio(struct intel_vgpu *vgpu)
+{
+ vfree(vgpu->mmio.vreg);
+ vgpu->mmio.vreg = vgpu->mmio.sreg = NULL;
+}
*offset; \
})
+int intel_vgpu_init_mmio(struct intel_vgpu *vgpu);
+void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu);
+void intel_vgpu_clean_mmio(struct intel_vgpu *vgpu);
+
int intel_vgpu_gpa_to_mmio_offset(struct intel_vgpu *vgpu, u64 gpa);
int intel_vgpu_emulate_mmio_read(struct intel_vgpu *vgpu, u64 pa,
vgpu->id))
return -EINVAL;
- vgpu_opregion(vgpu)->va = (void *)__get_free_pages(GFP_ATOMIC |
- GFP_DMA32 | __GFP_ZERO,
- INTEL_GVT_OPREGION_PORDER);
+ vgpu_opregion(vgpu)->va = (void *)__get_free_pages(GFP_KERNEL |
+ __GFP_ZERO,
+ get_order(INTEL_GVT_OPREGION_SIZE));
if (!vgpu_opregion(vgpu)->va)
return -ENOMEM;
if (intel_gvt_host.hypervisor_type == INTEL_GVT_HYPERVISOR_XEN) {
map_vgpu_opregion(vgpu, false);
free_pages((unsigned long)vgpu_opregion(vgpu)->va,
- INTEL_GVT_OPREGION_PORDER);
+ get_order(INTEL_GVT_OPREGION_SIZE));
vgpu_opregion(vgpu)->va = NULL;
}
#define INTEL_GVT_OPREGION_PARM 0x204
#define INTEL_GVT_OPREGION_PAGES 2
-#define INTEL_GVT_OPREGION_PORDER 1
-#define INTEL_GVT_OPREGION_SIZE (2 * 4096)
+#define INTEL_GVT_OPREGION_SIZE (INTEL_GVT_OPREGION_PAGES * PAGE_SIZE)
#define VGT_SPRSTRIDE(pipe) _PIPE(pipe, _SPRA_STRIDE, _PLANE_STRIDE_2_B)
{
struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler;
struct intel_vgpu_workload *workload;
+ struct intel_vgpu *vgpu;
int event;
mutex_lock(&gvt->lock);
workload = scheduler->current_workload[ring_id];
+ vgpu = workload->vgpu;
- if (!workload->status && !workload->vgpu->resetting) {
+ if (!workload->status && !vgpu->resetting) {
wait_event(workload->shadow_ctx_status_wq,
!atomic_read(&workload->shadow_ctx_active));
for_each_set_bit(event, workload->pending_events,
INTEL_GVT_EVENT_MAX)
- intel_vgpu_trigger_virtual_event(workload->vgpu,
- event);
+ intel_vgpu_trigger_virtual_event(vgpu, event);
}
gvt_dbg_sched("ring id %d complete workload %p status %d\n",
scheduler->current_workload[ring_id] = NULL;
- atomic_dec(&workload->vgpu->running_workload_num);
-
list_del_init(&workload->list);
workload->complete(workload);
+ atomic_dec(&vgpu->running_workload_num);
wake_up(&scheduler->workload_complete_wq);
mutex_unlock(&gvt->lock);
}
gvt_dbg_sched("will complete workload %p\n, status: %d\n",
workload, workload->status);
- complete_current_workload(gvt, ring_id);
-
if (workload->req)
i915_gem_request_put(fetch_and_zero(&workload->req));
+ complete_current_workload(gvt, ring_id);
+
if (need_force_wake)
intel_uncore_forcewake_put(gvt->dev_priv,
FORCEWAKE_ALL);
struct drm_i915_gem_object *obj;
void *va;
unsigned long len;
- void *bb_start_cmd_va;
+ u32 *bb_start_cmd_va;
};
#define workload_q_head(vgpu, ring_id) \
#include "gvt.h"
#include "i915_pvinfo.h"
-static void clean_vgpu_mmio(struct intel_vgpu *vgpu)
-{
- vfree(vgpu->mmio.vreg);
- vgpu->mmio.vreg = vgpu->mmio.sreg = NULL;
-}
-
-int setup_vgpu_mmio(struct intel_vgpu *vgpu)
-{
- struct intel_gvt *gvt = vgpu->gvt;
- const struct intel_gvt_device_info *info = &gvt->device_info;
-
- if (vgpu->mmio.vreg)
- memset(vgpu->mmio.vreg, 0, info->mmio_size * 2);
- else {
- vgpu->mmio.vreg = vzalloc(info->mmio_size * 2);
- if (!vgpu->mmio.vreg)
- return -ENOMEM;
- }
-
- vgpu->mmio.sreg = vgpu->mmio.vreg + info->mmio_size;
-
- memcpy(vgpu->mmio.vreg, gvt->firmware.mmio, info->mmio_size);
- memcpy(vgpu->mmio.sreg, gvt->firmware.mmio, info->mmio_size);
-
- vgpu_vreg(vgpu, GEN6_GT_THREAD_STATUS_REG) = 0;
-
- /* set the bit 0:2(Core C-State ) to C0 */
- vgpu_vreg(vgpu, GEN6_GT_CORE_STATUS) = 0;
- return 0;
-}
-
-static void setup_vgpu_cfg_space(struct intel_vgpu *vgpu,
- struct intel_vgpu_creation_params *param)
-{
- struct intel_gvt *gvt = vgpu->gvt;
- const struct intel_gvt_device_info *info = &gvt->device_info;
- u16 *gmch_ctl;
- int i;
-
- memcpy(vgpu_cfg_space(vgpu), gvt->firmware.cfg_space,
- info->cfg_space_size);
-
- if (!param->primary) {
- vgpu_cfg_space(vgpu)[PCI_CLASS_DEVICE] =
- INTEL_GVT_PCI_CLASS_VGA_OTHER;
- vgpu_cfg_space(vgpu)[PCI_CLASS_PROG] =
- INTEL_GVT_PCI_CLASS_VGA_OTHER;
- }
-
- /* Show guest that there isn't any stolen memory.*/
- gmch_ctl = (u16 *)(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_GMCH_CONTROL);
- *gmch_ctl &= ~(BDW_GMCH_GMS_MASK << BDW_GMCH_GMS_SHIFT);
-
- intel_vgpu_write_pci_bar(vgpu, PCI_BASE_ADDRESS_2,
- gvt_aperture_pa_base(gvt), true);
-
- vgpu_cfg_space(vgpu)[PCI_COMMAND] &= ~(PCI_COMMAND_IO
- | PCI_COMMAND_MEMORY
- | PCI_COMMAND_MASTER);
- /*
- * Clear the bar upper 32bit and let guest to assign the new value
- */
- memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_1, 0, 4);
- memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_3, 0, 4);
- memset(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_OPREGION, 0, 4);
-
- for (i = 0; i < INTEL_GVT_MAX_BAR_NUM; i++) {
- vgpu->cfg_space.bar[i].size = pci_resource_len(
- gvt->dev_priv->drm.pdev, i * 2);
- vgpu->cfg_space.bar[i].tracked = false;
- }
-}
-
void populate_pvinfo_page(struct intel_vgpu *vgpu)
{
/* setup the ballooning information */
if (low_avail / min_low == 0)
break;
gvt->types[i].low_gm_size = min_low;
- gvt->types[i].high_gm_size = 3 * gvt->types[i].low_gm_size;
+ gvt->types[i].high_gm_size = max((min_low<<3), MB_TO_BYTES(384U));
gvt->types[i].fence = 4;
gvt->types[i].max_instance = low_avail / min_low;
gvt->types[i].avail_instance = gvt->types[i].max_instance;
*/
low_gm_avail = MB_TO_BYTES(256) - HOST_LOW_GM_SIZE -
gvt->gm.vgpu_allocated_low_gm_size;
- high_gm_avail = MB_TO_BYTES(256) * 3 - HOST_HIGH_GM_SIZE -
+ high_gm_avail = MB_TO_BYTES(256) * 8UL - HOST_HIGH_GM_SIZE -
gvt->gm.vgpu_allocated_high_gm_size;
fence_avail = gvt_fence_sz(gvt) - HOST_FENCE -
gvt->fence.vgpu_allocated_fence_num;
intel_vgpu_clean_gtt(vgpu);
intel_gvt_hypervisor_detach_vgpu(vgpu);
intel_vgpu_free_resource(vgpu);
- clean_vgpu_mmio(vgpu);
+ intel_vgpu_clean_mmio(vgpu);
vfree(vgpu);
intel_gvt_update_vgpu_types(gvt);
vgpu->gvt = gvt;
bitmap_zero(vgpu->tlb_handle_pending, I915_NUM_ENGINES);
- setup_vgpu_cfg_space(vgpu, param);
+ intel_vgpu_init_cfg_space(vgpu, param->primary);
- ret = setup_vgpu_mmio(vgpu);
+ ret = intel_vgpu_init_mmio(vgpu);
if (ret)
- goto out_free_vgpu;
+ goto out_clean_idr;
ret = intel_vgpu_alloc_resource(vgpu, param);
if (ret)
out_clean_vgpu_resource:
intel_vgpu_free_resource(vgpu);
out_clean_vgpu_mmio:
- clean_vgpu_mmio(vgpu);
+ intel_vgpu_clean_mmio(vgpu);
+out_clean_idr:
+ idr_remove(&gvt->vgpu_idr, vgpu->id);
out_free_vgpu:
vfree(vgpu);
mutex_unlock(&gvt->lock);
}
/**
- * intel_gvt_reset_vgpu - reset a virtual GPU
+ * intel_gvt_reset_vgpu_locked - reset a virtual GPU by DMLR or GT reset
+ * @vgpu: virtual GPU
+ * @dmlr: vGPU Device Model Level Reset or GT Reset
+ * @engine_mask: engines to reset for GT reset
+ *
+ * This function is called when user wants to reset a virtual GPU through
+ * device model reset or GT reset. The caller should hold the gvt lock.
+ *
+ * vGPU Device Model Level Reset (DMLR) simulates the PCI level reset to reset
+ * the whole vGPU to default state as when it is created. This vGPU function
+ * is required both for functionary and security concerns.The ultimate goal
+ * of vGPU FLR is that reuse a vGPU instance by virtual machines. When we
+ * assign a vGPU to a virtual machine we must isse such reset first.
+ *
+ * Full GT Reset and Per-Engine GT Reset are soft reset flow for GPU engines
+ * (Render, Blitter, Video, Video Enhancement). It is defined by GPU Spec.
+ * Unlike the FLR, GT reset only reset particular resource of a vGPU per
+ * the reset request. Guest driver can issue a GT reset by programming the
+ * virtual GDRST register to reset specific virtual GPU engine or all
+ * engines.
+ *
+ * The parameter dev_level is to identify if we will do DMLR or GT reset.
+ * The parameter engine_mask is to specific the engines that need to be
+ * resetted. If value ALL_ENGINES is given for engine_mask, it means
+ * the caller requests a full GT reset that we will reset all virtual
+ * GPU engines. For FLR, engine_mask is ignored.
+ */
+void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr,
+ unsigned int engine_mask)
+{
+ struct intel_gvt *gvt = vgpu->gvt;
+ struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler;
+
+ gvt_dbg_core("------------------------------------------\n");
+ gvt_dbg_core("resseting vgpu%d, dmlr %d, engine_mask %08x\n",
+ vgpu->id, dmlr, engine_mask);
+ vgpu->resetting = true;
+
+ intel_vgpu_stop_schedule(vgpu);
+ /*
+ * The current_vgpu will set to NULL after stopping the
+ * scheduler when the reset is triggered by current vgpu.
+ */
+ if (scheduler->current_vgpu == NULL) {
+ mutex_unlock(&gvt->lock);
+ intel_gvt_wait_vgpu_idle(vgpu);
+ mutex_lock(&gvt->lock);
+ }
+
+ intel_vgpu_reset_execlist(vgpu, dmlr ? ALL_ENGINES : engine_mask);
+
+ /* full GPU reset or device model level reset */
+ if (engine_mask == ALL_ENGINES || dmlr) {
+ intel_vgpu_reset_gtt(vgpu, dmlr);
+ intel_vgpu_reset_resource(vgpu);
+ intel_vgpu_reset_mmio(vgpu);
+ populate_pvinfo_page(vgpu);
+
+ if (dmlr)
+ intel_vgpu_reset_cfg_space(vgpu);
+ }
+
+ vgpu->resetting = false;
+ gvt_dbg_core("reset vgpu%d done\n", vgpu->id);
+ gvt_dbg_core("------------------------------------------\n");
+}
+
+/**
+ * intel_gvt_reset_vgpu - reset a virtual GPU (Function Level)
* @vgpu: virtual GPU
*
* This function is called when user wants to reset a virtual GPU.
*/
void intel_gvt_reset_vgpu(struct intel_vgpu *vgpu)
{
+ mutex_lock(&vgpu->gvt->lock);
+ intel_gvt_reset_vgpu_locked(vgpu, true, 0);
+ mutex_unlock(&vgpu->gvt->lock);
}
} else if (id == INTEL_PCH_KBP_DEVICE_ID_TYPE) {
dev_priv->pch_type = PCH_KBP;
DRM_DEBUG_KMS("Found KabyPoint PCH\n");
- WARN_ON(!IS_KABYLAKE(dev_priv));
+ WARN_ON(!IS_SKYLAKE(dev_priv) &&
+ !IS_KABYLAKE(dev_priv));
} else if ((id == INTEL_PCH_P2X_DEVICE_ID_TYPE) ||
(id == INTEL_PCH_P3X_DEVICE_ID_TYPE) ||
((id == INTEL_PCH_QEMU_DEVICE_ID_TYPE) &&
assert_forcewakes_inactive(dev_priv);
- if (!IS_VALLEYVIEW(dev_priv) || !IS_CHERRYVIEW(dev_priv))
+ if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv))
intel_hpd_poll_init(dev_priv);
DRM_DEBUG_KMS("Device suspended\n");
* we can do is to hope that things will still work (and disable RPM).
*/
i915_gem_init_swizzling(dev_priv);
+ i915_gem_restore_fences(dev_priv);
intel_runtime_pm_enable_interrupts(dev_priv);
struct work_struct underrun_work;
struct intel_fbc_state_cache {
+ struct i915_vma *vma;
+
struct {
unsigned int mode_flags;
uint32_t hsw_bdw_pixel_rate;
} plane;
struct {
- u64 ilk_ggtt_offset;
uint32_t pixel_format;
unsigned int stride;
- int fence_reg;
- unsigned int tiling_mode;
} fb;
} state_cache;
struct intel_fbc_reg_params {
+ struct i915_vma *vma;
+
struct {
enum pipe pipe;
enum plane plane;
} crtc;
struct {
- u64 ggtt_offset;
uint32_t pixel_format;
unsigned int stride;
- int fence_reg;
} fb;
int cfb_size;
struct i915_frontbuffer_tracking fb_tracking;
+ struct intel_atomic_helper {
+ struct llist_head free_list;
+ struct work_struct free_work;
+ } atomic_helper;
+
u16 orig_clock;
bool mchbar_need_disable;
return i915_gem_obj_to_vma(obj, &to_i915(obj->base.dev)->ggtt.base, view);
}
-static inline unsigned long
-i915_gem_object_ggtt_offset(struct drm_i915_gem_object *o,
- const struct i915_ggtt_view *view)
-{
- return i915_ggtt_offset(i915_gem_object_to_ggtt(o, view));
-}
-
/* i915_gem_fence_reg.c */
int __must_check i915_vma_get_fence(struct i915_vma *vma);
int __must_check i915_vma_put_fence(struct i915_vma *vma);
struct drm_i915_gem_pwrite *args,
struct drm_file *file)
{
- struct drm_device *dev = obj->base.dev;
void *vaddr = obj->phys_handle->vaddr + args->offset;
char __user *user_data = u64_to_user_ptr(args->data_ptr);
- int ret;
/* We manually control the domain here and pretend that it
* remains coherent i.e. in the GTT domain, like shmem_pwrite.
*/
- lockdep_assert_held(&obj->base.dev->struct_mutex);
- ret = i915_gem_object_wait(obj,
- I915_WAIT_INTERRUPTIBLE |
- I915_WAIT_LOCKED |
- I915_WAIT_ALL,
- MAX_SCHEDULE_TIMEOUT,
- to_rps_client(file));
- if (ret)
- return ret;
-
intel_fb_obj_invalidate(obj, ORIGIN_CPU);
- if (__copy_from_user_inatomic_nocache(vaddr, user_data, args->size)) {
- unsigned long unwritten;
-
- /* The physical object once assigned is fixed for the lifetime
- * of the obj, so we can safely drop the lock and continue
- * to access vaddr.
- */
- mutex_unlock(&dev->struct_mutex);
- unwritten = copy_from_user(vaddr, user_data, args->size);
- mutex_lock(&dev->struct_mutex);
- if (unwritten) {
- ret = -EFAULT;
- goto out;
- }
- }
+ if (copy_from_user(vaddr, user_data, args->size))
+ return -EFAULT;
drm_clflush_virt_range(vaddr, args->size);
- i915_gem_chipset_flush(to_i915(dev));
+ i915_gem_chipset_flush(to_i915(obj->base.dev));
-out:
intel_fb_obj_flush(obj, false, ORIGIN_CPU);
- return ret;
+ return 0;
}
void *i915_gem_object_alloc(struct drm_device *dev)
for (i = 0; i < dev_priv->num_fence_regs; i++) {
struct drm_i915_fence_reg *reg = &dev_priv->fence_regs[i];
- if (WARN_ON(reg->pin_count))
- continue;
+ /* Ideally we want to assert that the fence register is not
+ * live at this point (i.e. that no piece of code will be
+ * trying to write through fence + GTT, as that both violates
+ * our tracking of activity and associated locking/barriers,
+ * but also is illegal given that the hw is powered down).
+ *
+ * Previously we used reg->pin_count as a "liveness" indicator.
+ * That is not sufficient, and we need a more fine-grained
+ * tool if we want to have a sanity check here.
+ */
if (!reg->vma)
continue;
vma->display_alignment = max_t(u64, vma->display_alignment, alignment);
/* Treat this as an end-of-frame, like intel_user_framebuffer_dirty() */
- if (obj->cache_dirty) {
+ if (obj->cache_dirty || obj->base.write_domain == I915_GEM_DOMAIN_CPU) {
i915_gem_clflush_object(obj, true);
intel_fb_obj_flush(obj, false, ORIGIN_DIRTYFB);
}
}
/* Unbinding will emit any required flushes */
+ ret = 0;
while (!list_empty(&eviction_list)) {
vma = list_first_entry(&eviction_list,
struct i915_vma,
if (exec[i].offset !=
gen8_canonical_addr(exec[i].offset & PAGE_MASK))
return -EINVAL;
-
- /* From drm_mm perspective address space is continuous,
- * so from this point we're always using non-canonical
- * form internally.
- */
- exec[i].offset = gen8_noncanonical_addr(exec[i].offset);
}
+ /* From drm_mm perspective address space is continuous,
+ * so from this point we're always using non-canonical
+ * form internally.
+ */
+ exec[i].offset = gen8_noncanonical_addr(exec[i].offset);
+
if (exec[i].alignment && !is_power_of_2(exec[i].alignment))
return -EINVAL;
max_order = MAX_ORDER;
#ifdef CONFIG_SWIOTLB
- if (swiotlb_nr_tbl()) /* minimum max swiotlb size is IO_TLB_SEGSIZE */
- max_order = min(max_order, ilog2(IO_TLB_SEGPAGES));
+ if (swiotlb_nr_tbl()) {
+ unsigned int max_segment;
+
+ max_segment = swiotlb_max_segment();
+ if (max_segment) {
+ max_segment = max_t(unsigned int, max_segment,
+ PAGE_SIZE) >> PAGE_SHIFT;
+ max_order = min(max_order, ilog2(max_segment));
+ }
+ }
#endif
gfp = GFP_KERNEL | __GFP_HIGHMEM | __GFP_RECLAIMABLE;
return ret;
}
+ trace_i915_vma_bind(vma, bind_flags);
ret = vma->vm->bind_vma(vma, cache_level, bind_flags);
if (ret)
return ret;
__drm_atomic_helper_plane_duplicate_state(plane, state);
+ intel_state->vma = NULL;
+
return state;
}
intel_plane_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
+ struct i915_vma *vma;
+
+ vma = fetch_and_zero(&to_intel_plane_state(state)->vma);
+
+ /*
+ * FIXME: Normally intel_cleanup_plane_fb handles destruction of vma.
+ * We currently don't clear all planes during driver unload, so we have
+ * to be able to unpin vma here for now.
+ *
+ * Normally this can only happen during unload when kmscon is disabled
+ * and userspace doesn't attempt to set a framebuffer at all.
+ */
+ if (vma) {
+ mutex_lock(&plane->dev->struct_mutex);
+ intel_unpin_fb_vma(vma);
+ mutex_unlock(&plane->dev->struct_mutex);
+ }
+
drm_atomic_helper_plane_destroy_state(plane, state);
}
struct drm_i915_private *dev_priv = to_i915(crt->base.base.dev);
struct edid *edid;
struct i2c_adapter *i2c;
+ bool ret = false;
BUG_ON(crt->base.type != INTEL_OUTPUT_ANALOG);
*/
if (!is_digital) {
DRM_DEBUG_KMS("CRT detected via DDC:0x50 [EDID]\n");
- return true;
+ ret = true;
+ } else {
+ DRM_DEBUG_KMS("CRT not detected via DDC:0x50 [EDID reports a digital panel]\n");
}
-
- DRM_DEBUG_KMS("CRT not detected via DDC:0x50 [EDID reports a digital panel]\n");
} else {
DRM_DEBUG_KMS("CRT not detected via DDC:0x50 [no valid EDID found]\n");
}
kfree(edid);
- return false;
+ return ret;
}
static enum drm_connector_status
i915_vma_pin_fence(vma);
}
+ i915_vma_get(vma);
err:
intel_runtime_pm_put(dev_priv);
return vma;
}
-void intel_unpin_fb_obj(struct drm_framebuffer *fb, unsigned int rotation)
+void intel_unpin_fb_vma(struct i915_vma *vma)
{
- struct drm_i915_gem_object *obj = intel_fb_obj(fb);
- struct i915_ggtt_view view;
- struct i915_vma *vma;
+ lockdep_assert_held(&vma->vm->dev->struct_mutex);
- WARN_ON(!mutex_is_locked(&obj->base.dev->struct_mutex));
-
- intel_fill_fb_ggtt_view(&view, fb, rotation);
- vma = i915_gem_object_to_ggtt(obj, &view);
+ if (WARN_ON_ONCE(!vma))
+ return;
i915_vma_unpin_fence(vma);
i915_gem_object_unpin_from_display_plane(vma);
+ i915_vma_put(vma);
}
static int intel_fb_pitch(const struct drm_framebuffer *fb, int plane,
* We only keep the x/y offsets, so push all of the
* gtt offset into the x/y offsets.
*/
- _intel_adjust_tile_offset(&x, &y, tile_size,
- tile_width, tile_height, pitch_tiles,
+ _intel_adjust_tile_offset(&x, &y,
+ tile_width, tile_height,
+ tile_size, pitch_tiles,
gtt_offset_rotated * tile_size, 0);
gtt_offset_rotated += rot_info->plane[i].width * rot_info->plane[i].height;
struct drm_device *dev = intel_crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_crtc *c;
- struct intel_crtc *i;
struct drm_i915_gem_object *obj;
struct drm_plane *primary = intel_crtc->base.primary;
struct drm_plane_state *plane_state = primary->state;
* an fb with another CRTC instead
*/
for_each_crtc(dev, c) {
- i = to_intel_crtc(c);
+ struct intel_plane_state *state;
if (c == &intel_crtc->base)
continue;
- if (!i->active)
+ if (!to_intel_crtc(c)->active)
continue;
- fb = c->primary->fb;
- if (!fb)
+ state = to_intel_plane_state(c->primary->state);
+ if (!state->vma)
continue;
- obj = intel_fb_obj(fb);
- if (i915_gem_object_ggtt_offset(obj, NULL) == plane_config->base) {
+ if (intel_plane_ggtt_offset(state) == plane_config->base) {
+ fb = c->primary->fb;
drm_framebuffer_reference(fb);
goto valid_fb;
}
return;
valid_fb:
+ mutex_lock(&dev->struct_mutex);
+ intel_state->vma =
+ intel_pin_and_fence_fb_obj(fb, primary->state->rotation);
+ mutex_unlock(&dev->struct_mutex);
+ if (IS_ERR(intel_state->vma)) {
+ DRM_ERROR("failed to pin boot fb on pipe %d: %li\n",
+ intel_crtc->pipe, PTR_ERR(intel_state->vma));
+
+ intel_state->vma = NULL;
+ drm_framebuffer_unreference(fb);
+ return;
+ }
+
plane_state->src_x = 0;
plane_state->src_y = 0;
plane_state->src_w = fb->width << 16;
unsigned int rotation = plane_state->base.rotation;
int ret;
+ if (!plane_state->base.visible)
+ return 0;
+
/* Rotate src coordinates to match rotated GTT view */
if (drm_rotation_90_or_270(rotation))
drm_rect_rotate(&plane_state->base.src,
I915_WRITE(DSPSTRIDE(plane), fb->pitches[0]);
if (INTEL_GEN(dev_priv) >= 4) {
I915_WRITE(DSPSURF(plane),
- intel_fb_gtt_offset(fb, rotation) +
+ intel_plane_ggtt_offset(plane_state) +
intel_crtc->dspaddr_offset);
I915_WRITE(DSPTILEOFF(plane), (y << 16) | x);
I915_WRITE(DSPLINOFF(plane), linear_offset);
} else {
I915_WRITE(DSPADDR(plane),
- intel_fb_gtt_offset(fb, rotation) +
+ intel_plane_ggtt_offset(plane_state) +
intel_crtc->dspaddr_offset);
}
POSTING_READ(reg);
I915_WRITE(DSPSTRIDE(plane), fb->pitches[0]);
I915_WRITE(DSPSURF(plane),
- intel_fb_gtt_offset(fb, rotation) +
+ intel_plane_ggtt_offset(plane_state) +
intel_crtc->dspaddr_offset);
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
I915_WRITE(DSPOFFSET(plane), (y << 16) | x);
}
}
-u32 intel_fb_gtt_offset(struct drm_framebuffer *fb,
- unsigned int rotation)
-{
- struct drm_i915_gem_object *obj = intel_fb_obj(fb);
- struct i915_ggtt_view view;
- struct i915_vma *vma;
-
- intel_fill_fb_ggtt_view(&view, fb, rotation);
-
- vma = i915_gem_object_to_ggtt(obj, &view);
- if (WARN(!vma, "ggtt vma for display object not found! (view=%u)\n",
- view.type))
- return -1;
-
- return i915_ggtt_offset(vma);
-}
-
static void skl_detach_scaler(struct intel_crtc *intel_crtc, int id)
{
struct drm_device *dev = intel_crtc->base.dev;
}
I915_WRITE(PLANE_SURF(pipe, 0),
- intel_fb_gtt_offset(fb, rotation) + surf_addr);
+ intel_plane_ggtt_offset(plane_state) + surf_addr);
POSTING_READ(PLANE_SURF(pipe, 0));
}
drm_crtc_vblank_put(&intel_crtc->base);
wake_up_all(&dev_priv->pending_flip_queue);
- queue_work(dev_priv->wq, &work->unpin_work);
-
trace_i915_flip_complete(intel_crtc->plane,
work->pending_flip_obj);
+
+ queue_work(dev_priv->wq, &work->unpin_work);
}
static int intel_crtc_wait_for_pending_flips(struct drm_crtc *crtc)
}
state = drm_atomic_state_alloc(crtc->dev);
+ if (!state) {
+ DRM_DEBUG_KMS("failed to disable [CRTC:%d:%s], out of memory",
+ crtc->base.id, crtc->name);
+ return;
+ }
+
state->acquire_ctx = crtc->dev->mode_config.acquire_ctx;
/* Everything's already locked, -EDEADLK can't happen. */
}
old->restore_state = restore_state;
+ drm_atomic_state_put(state);
/* let the connector get through one full cycle before testing */
intel_wait_for_vblank(dev_priv, intel_crtc->pipe);
flush_work(&work->mmio_work);
mutex_lock(&dev->struct_mutex);
- intel_unpin_fb_obj(work->old_fb, primary->state->rotation);
+ intel_unpin_fb_vma(work->old_vma);
i915_gem_object_put(work->pending_flip_obj);
mutex_unlock(&dev->struct_mutex);
goto cleanup_pending;
}
- work->gtt_offset = intel_fb_gtt_offset(fb, primary->state->rotation);
- work->gtt_offset += intel_crtc->dspaddr_offset;
+ work->old_vma = to_intel_plane_state(primary->state)->vma;
+ to_intel_plane_state(primary->state)->vma = vma;
+
+ work->gtt_offset = i915_ggtt_offset(vma) + intel_crtc->dspaddr_offset;
work->rotation = crtc->primary->state->rotation;
/*
cleanup_request:
i915_add_request_no_flush(request);
cleanup_unpin:
- intel_unpin_fb_obj(fb, crtc->primary->state->rotation);
+ to_intel_plane_state(primary->state)->vma = work->old_vma;
+ intel_unpin_fb_vma(vma);
cleanup_pending:
atomic_dec(&intel_crtc->unpin_work_count);
unlock:
break;
case FENCE_FREE:
- drm_atomic_state_put(&state->base);
- break;
+ {
+ struct intel_atomic_helper *helper =
+ &to_i915(state->base.dev)->atomic_helper;
+
+ if (llist_add(&state->freed, &helper->free_list))
+ schedule_work(&helper->free_work);
+ break;
+ }
}
return NOTIFY_DONE;
DRM_DEBUG_KMS("failed to pin object\n");
return PTR_ERR(vma);
}
+
+ to_intel_plane_state(new_state)->vma = vma;
}
return 0;
intel_cleanup_plane_fb(struct drm_plane *plane,
struct drm_plane_state *old_state)
{
- struct drm_i915_private *dev_priv = to_i915(plane->dev);
- struct intel_plane_state *old_intel_state;
- struct drm_i915_gem_object *old_obj = intel_fb_obj(old_state->fb);
- struct drm_i915_gem_object *obj = intel_fb_obj(plane->state->fb);
-
- old_intel_state = to_intel_plane_state(old_state);
-
- if (!obj && !old_obj)
- return;
+ struct i915_vma *vma;
- if (old_obj && (plane->type != DRM_PLANE_TYPE_CURSOR ||
- !INTEL_INFO(dev_priv)->cursor_needs_physical))
- intel_unpin_fb_obj(old_state->fb, old_state->rotation);
+ /* Should only be called after a successful intel_prepare_plane_fb()! */
+ vma = fetch_and_zero(&to_intel_plane_state(old_state)->vma);
+ if (vma)
+ intel_unpin_fb_vma(vma);
}
int
if (!obj)
addr = 0;
else if (!INTEL_INFO(dev_priv)->cursor_needs_physical)
- addr = i915_gem_object_ggtt_offset(obj, NULL);
+ addr = intel_plane_ggtt_offset(state);
else
addr = obj->phys_handle->busaddr;
drm_modeset_acquire_fini(&ctx);
}
+static void intel_atomic_helper_free_state(struct work_struct *work)
+{
+ struct drm_i915_private *dev_priv =
+ container_of(work, typeof(*dev_priv), atomic_helper.free_work);
+ struct intel_atomic_state *state, *next;
+ struct llist_node *freed;
+
+ freed = llist_del_all(&dev_priv->atomic_helper.free_list);
+ llist_for_each_entry_safe(state, next, freed, freed)
+ drm_atomic_state_put(&state->base);
+}
+
int intel_modeset_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
dev->mode_config.funcs = &intel_mode_funcs;
+ INIT_WORK(&dev_priv->atomic_helper.free_work,
+ intel_atomic_helper_free_state);
+
intel_init_quirks(dev);
intel_init_pm(dev_priv);
if (ret)
DRM_ERROR("Restoring old state failed with %i\n", ret);
- drm_atomic_state_put(state);
+ if (state)
+ drm_atomic_state_put(state);
}
void intel_modeset_gem_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct drm_crtc *c;
- struct drm_i915_gem_object *obj;
intel_init_gt_powersave(dev_priv);
intel_modeset_init_hw(dev);
intel_setup_overlay(dev_priv);
-
- /*
- * Make sure any fbs we allocated at startup are properly
- * pinned & fenced. When we do the allocation it's too early
- * for this.
- */
- for_each_crtc(dev, c) {
- struct i915_vma *vma;
-
- obj = intel_fb_obj(c->primary->fb);
- if (obj == NULL)
- continue;
-
- mutex_lock(&dev->struct_mutex);
- vma = intel_pin_and_fence_fb_obj(c->primary->fb,
- c->primary->state->rotation);
- mutex_unlock(&dev->struct_mutex);
- if (IS_ERR(vma)) {
- DRM_ERROR("failed to pin boot fb on pipe %d\n",
- to_intel_crtc(c)->pipe);
- drm_framebuffer_unreference(c->primary->fb);
- c->primary->fb = NULL;
- c->primary->crtc = c->primary->state->crtc = NULL;
- update_state_fb(c->primary);
- c->state->plane_mask &= ~(1 << drm_plane_index(c->primary));
- }
- }
}
int intel_connector_register(struct drm_connector *connector)
{
struct drm_i915_private *dev_priv = to_i915(dev);
+ flush_work(&dev_priv->atomic_helper.free_work);
+ WARN_ON(!llist_empty(&dev_priv->atomic_helper.free_list));
+
intel_disable_gt_powersave(dev_priv);
/*
return NULL;
if ((encoder->type == INTEL_OUTPUT_DP ||
- encoder->type == INTEL_OUTPUT_EDP) &&
+ encoder->type == INTEL_OUTPUT_EDP ||
+ encoder->type == INTEL_OUTPUT_DP_MST) &&
!bxt_ddi_dp_set_dpll_hw_state(clock, &dpll_hw_state))
return NULL;
struct skl_wm_values wm_results;
struct i915_sw_fence commit_ready;
+
+ struct llist_node freed;
};
struct intel_plane_state {
struct drm_plane_state base;
struct drm_rect clip;
+ struct i915_vma *vma;
struct {
u32 offset;
struct work_struct mmio_work;
struct drm_crtc *crtc;
+ struct i915_vma *old_vma;
struct drm_framebuffer *old_fb;
struct drm_i915_gem_object *pending_flip_obj;
struct drm_pending_vblank_event *event;
struct drm_modeset_acquire_ctx *ctx);
struct i915_vma *
intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb, unsigned int rotation);
-void intel_unpin_fb_obj(struct drm_framebuffer *fb, unsigned int rotation);
+void intel_unpin_fb_vma(struct i915_vma *vma);
struct drm_framebuffer *
__intel_framebuffer_create(struct drm_device *dev,
struct drm_mode_fb_cmd2 *mode_cmd,
int skl_update_scaler_crtc(struct intel_crtc_state *crtc_state);
int skl_max_scale(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state);
-u32 intel_fb_gtt_offset(struct drm_framebuffer *fb, unsigned int rotation);
+static inline u32 intel_plane_ggtt_offset(const struct intel_plane_state *state)
+{
+ return i915_ggtt_offset(state->vma);
+}
u32 skl_plane_ctl_format(uint32_t pixel_format);
u32 skl_plane_ctl_tiling(uint64_t fb_modifier);
if (IS_I945GM(dev_priv))
fbc_ctl |= FBC_CTL_C3_IDLE; /* 945 needs special SR handling */
fbc_ctl |= (cfb_pitch & 0xff) << FBC_CTL_STRIDE_SHIFT;
- fbc_ctl |= params->fb.fence_reg;
+ fbc_ctl |= params->vma->fence->id;
I915_WRITE(FBC_CONTROL, fbc_ctl);
}
else
dpfc_ctl |= DPFC_CTL_LIMIT_1X;
- if (params->fb.fence_reg != I915_FENCE_REG_NONE) {
- dpfc_ctl |= DPFC_CTL_FENCE_EN | params->fb.fence_reg;
+ if (params->vma->fence) {
+ dpfc_ctl |= DPFC_CTL_FENCE_EN | params->vma->fence->id;
I915_WRITE(DPFC_FENCE_YOFF, params->crtc.fence_y_offset);
} else {
I915_WRITE(DPFC_FENCE_YOFF, 0);
break;
}
- if (params->fb.fence_reg != I915_FENCE_REG_NONE) {
+ if (params->vma->fence) {
dpfc_ctl |= DPFC_CTL_FENCE_EN;
if (IS_GEN5(dev_priv))
- dpfc_ctl |= params->fb.fence_reg;
+ dpfc_ctl |= params->vma->fence->id;
if (IS_GEN6(dev_priv)) {
I915_WRITE(SNB_DPFC_CTL_SA,
- SNB_CPU_FENCE_ENABLE | params->fb.fence_reg);
+ SNB_CPU_FENCE_ENABLE |
+ params->vma->fence->id);
I915_WRITE(DPFC_CPU_FENCE_OFFSET,
params->crtc.fence_y_offset);
}
}
I915_WRITE(ILK_DPFC_FENCE_YOFF, params->crtc.fence_y_offset);
- I915_WRITE(ILK_FBC_RT_BASE, params->fb.ggtt_offset | ILK_FBC_RT_VALID);
+ I915_WRITE(ILK_FBC_RT_BASE,
+ i915_ggtt_offset(params->vma) | ILK_FBC_RT_VALID);
/* enable it... */
I915_WRITE(ILK_DPFC_CONTROL, dpfc_ctl | DPFC_CTL_EN);
break;
}
- if (params->fb.fence_reg != I915_FENCE_REG_NONE) {
+ if (params->vma->fence) {
dpfc_ctl |= IVB_DPFC_CTL_FENCE_EN;
I915_WRITE(SNB_DPFC_CTL_SA,
- SNB_CPU_FENCE_ENABLE | params->fb.fence_reg);
+ SNB_CPU_FENCE_ENABLE |
+ params->vma->fence->id);
I915_WRITE(DPFC_CPU_FENCE_OFFSET, params->crtc.fence_y_offset);
} else {
I915_WRITE(SNB_DPFC_CTL_SA,0);
return effective_w <= max_w && effective_h <= max_h;
}
-/* XXX replace me when we have VMA tracking for intel_plane_state */
-static int get_fence_id(struct drm_framebuffer *fb)
-{
- struct i915_vma *vma = i915_gem_object_to_ggtt(intel_fb_obj(fb), NULL);
-
- return vma && vma->fence ? vma->fence->id : I915_FENCE_REG_NONE;
-}
-
static void intel_fbc_update_state_cache(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state,
struct intel_plane_state *plane_state)
struct intel_fbc *fbc = &dev_priv->fbc;
struct intel_fbc_state_cache *cache = &fbc->state_cache;
struct drm_framebuffer *fb = plane_state->base.fb;
- struct drm_i915_gem_object *obj;
+
+ cache->vma = NULL;
cache->crtc.mode_flags = crtc_state->base.adjusted_mode.flags;
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
if (!cache->plane.visible)
return;
- obj = intel_fb_obj(fb);
-
- /* FIXME: We lack the proper locking here, so only run this on the
- * platforms that need. */
- if (IS_GEN(dev_priv, 5, 6))
- cache->fb.ilk_ggtt_offset = i915_gem_object_ggtt_offset(obj, NULL);
cache->fb.pixel_format = fb->pixel_format;
cache->fb.stride = fb->pitches[0];
- cache->fb.fence_reg = get_fence_id(fb);
- cache->fb.tiling_mode = i915_gem_object_get_tiling(obj);
+
+ cache->vma = plane_state->vma;
}
static bool intel_fbc_can_activate(struct intel_crtc *crtc)
return false;
}
- if (!cache->plane.visible) {
+ if (!cache->vma) {
fbc->no_fbc_reason = "primary plane not visible";
return false;
}
* so have no fence associated with it) due to aperture constaints
* at the time of pinning.
*/
- if (cache->fb.tiling_mode != I915_TILING_X ||
- cache->fb.fence_reg == I915_FENCE_REG_NONE) {
+ if (!cache->vma->fence) {
fbc->no_fbc_reason = "framebuffer not tiled or fenced";
return false;
}
* zero. */
memset(params, 0, sizeof(*params));
+ params->vma = cache->vma;
+
params->crtc.pipe = crtc->pipe;
params->crtc.plane = crtc->plane;
params->crtc.fence_y_offset = get_crtc_fence_y_offset(crtc);
params->fb.pixel_format = cache->fb.pixel_format;
params->fb.stride = cache->fb.stride;
- params->fb.fence_reg = cache->fb.fence_reg;
params->cfb_size = intel_fbc_calculate_cfb_size(dev_priv, cache);
-
- params->fb.ggtt_offset = cache->fb.ilk_ggtt_offset;
}
static bool intel_fbc_reg_params_equal(struct intel_fbc_reg_params *params1,
out_destroy_fbi:
drm_fb_helper_release_fbi(helper);
out_unpin:
- intel_unpin_fb_obj(&ifbdev->fb->base, DRM_ROTATE_0);
+ intel_unpin_fb_vma(vma);
out_unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
if (ifbdev->fb) {
mutex_lock(&ifbdev->helper.dev->struct_mutex);
- intel_unpin_fb_obj(&ifbdev->fb->base, DRM_ROTATE_0);
+ intel_unpin_fb_vma(ifbdev->vma);
mutex_unlock(&ifbdev->helper.dev->struct_mutex);
drm_framebuffer_remove(&ifbdev->fb->base);
{
struct intel_fbdev *ifbdev = to_i915(dev)->fbdev;
+ if (!ifbdev)
+ return;
+
ifbdev->cookie = async_schedule(intel_fbdev_initial_config, ifbdev);
}
uint32_t *batch,
uint32_t index)
{
- struct drm_i915_private *dev_priv = engine->i915;
uint32_t l3sqc4_flush = (0x40400000 | GEN8_LQSC_FLUSH_COHERENT_LINES);
- /*
- * WaDisableLSQCROPERFforOCL:kbl
- * This WA is implemented in skl_init_clock_gating() but since
- * this batch updates GEN8_L3SQCREG4 with default value we need to
- * set this bit here to retain the WA during flush.
- */
- if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_E0))
- l3sqc4_flush |= GEN8_LQSC_RO_PERF_DIS;
-
wa_ctx_emit(batch, index, (MI_STORE_REGISTER_MEM_GEN8 |
MI_SRM_LRM_GLOBAL_GTT));
wa_ctx_emit_reg(batch, index, GEN8_L3SQCREG4);
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FENCE_DEST_SLM_DISABLE);
- /* GEN8_L3SQCREG4 has a dependency with WA batch so any new changes
- * involving this register should also be added to WA batch as required.
- */
- if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_E0))
- /* WaDisableLSQCROPERFforOCL:kbl */
- I915_WRITE(GEN8_L3SQCREG4, I915_READ(GEN8_L3SQCREG4) |
- GEN8_LQSC_RO_PERF_DIS);
-
/* WaToEnableHwFixForPushConstHWBug:kbl */
if (IS_KBL_REVID(dev_priv, KBL_REVID_C0, REVID_FOREVER))
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
I915_WRITE(PLANE_CTL(pipe, plane), plane_ctl);
I915_WRITE(PLANE_SURF(pipe, plane),
- intel_fb_gtt_offset(fb, rotation) + surf_addr);
+ intel_plane_ggtt_offset(plane_state) + surf_addr);
POSTING_READ(PLANE_SURF(pipe, plane));
}
I915_WRITE(SPSIZE(pipe, plane), (crtc_h << 16) | crtc_w);
I915_WRITE(SPCNTR(pipe, plane), sprctl);
I915_WRITE(SPSURF(pipe, plane),
- intel_fb_gtt_offset(fb, rotation) + sprsurf_offset);
+ intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
POSTING_READ(SPSURF(pipe, plane));
}
I915_WRITE(SPRSCALE(pipe), sprscale);
I915_WRITE(SPRCTL(pipe), sprctl);
I915_WRITE(SPRSURF(pipe),
- intel_fb_gtt_offset(fb, rotation) + sprsurf_offset);
+ intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
POSTING_READ(SPRSURF(pipe));
}
I915_WRITE(DVSSCALE(pipe), dvsscale);
I915_WRITE(DVSCNTR(pipe), dvscntr);
I915_WRITE(DVSSURF(pipe),
- intel_fb_gtt_offset(fb, rotation) + dvssurf_offset);
+ intel_plane_ggtt_offset(plane_state) + dvssurf_offset);
POSTING_READ(DVSSURF(pipe));
}
{
struct adreno_platform_config *config = pdev->dev.platform_data;
struct msm_gpu *gpu = &adreno_gpu->base;
- struct msm_mmu *mmu;
int ret;
adreno_gpu->funcs = funcs;
return ret;
}
- mmu = gpu->aspace->mmu;
- if (mmu) {
+ if (gpu->aspace && gpu->aspace->mmu) {
+ struct msm_mmu *mmu = gpu->aspace->mmu;
ret = mmu->funcs->attach(mmu, iommu_ports,
ARRAY_SIZE(iommu_ports));
if (ret)
static void mdp5_complete_commit(struct msm_kms *kms, struct drm_atomic_state *state)
{
- int i;
struct mdp5_kms *mdp5_kms = to_mdp5_kms(to_mdp_kms(kms));
- struct drm_plane *plane;
- struct drm_plane_state *plane_state;
-
- for_each_plane_in_state(state, plane, plane_state, i)
- mdp5_plane_complete_commit(plane, plane_state);
if (mdp5_kms->smp)
mdp5_smp_complete_commit(mdp5_kms->smp, &mdp5_kms->state->smp);
/* assigned by crtc blender */
enum mdp_mixer_stage_id stage;
-
- bool pending : 1;
};
#define to_mdp5_plane_state(x) \
container_of(x, struct mdp5_plane_state, base)
void mdp5_irq_domain_fini(struct mdp5_kms *mdp5_kms);
uint32_t mdp5_plane_get_flush(struct drm_plane *plane);
-void mdp5_plane_complete_commit(struct drm_plane *plane,
- struct drm_plane_state *state);
enum mdp5_pipe mdp5_plane_pipe(struct drm_plane *plane);
struct drm_plane *mdp5_plane_init(struct drm_device *dev, bool primary);
drm_printf(p, "\tzpos=%u\n", pstate->zpos);
drm_printf(p, "\talpha=%u\n", pstate->alpha);
drm_printf(p, "\tstage=%s\n", stage2name(pstate->stage));
- drm_printf(p, "\tpending=%u\n", pstate->pending);
}
static void mdp5_plane_reset(struct drm_plane *plane)
if (mdp5_state && mdp5_state->base.fb)
drm_framebuffer_reference(mdp5_state->base.fb);
- mdp5_state->pending = false;
-
return &mdp5_state->base;
}
DBG("%s: check (%d -> %d)", plane->name,
plane_enabled(old_state), plane_enabled(state));
- /* We don't allow faster-than-vblank updates.. if we did add this
- * some day, we would need to disallow in cases where hwpipe
- * changes
- */
- if (WARN_ON(to_mdp5_plane_state(old_state)->pending))
- return -EBUSY;
-
max_width = config->hw->lm.max_width << 16;
max_height = config->hw->lm.max_height << 16;
struct drm_plane_state *old_state)
{
struct drm_plane_state *state = plane->state;
- struct mdp5_plane_state *mdp5_state = to_mdp5_plane_state(state);
DBG("%s: update", plane->name);
- mdp5_state->pending = true;
-
if (plane_enabled(state)) {
int ret;
return pstate->hwpipe->flush_mask;
}
-/* called after vsync in thread context */
-void mdp5_plane_complete_commit(struct drm_plane *plane,
- struct drm_plane_state *state)
-{
- struct mdp5_plane_state *pstate = to_mdp5_plane_state(plane->state);
-
- pstate->pending = false;
-}
-
/* initialize plane */
struct drm_plane *mdp5_plane_init(struct drm_device *dev, bool primary)
{
WARN_ON(!mutex_is_locked(&dev->struct_mutex));
for (id = 0; id < ARRAY_SIZE(msm_obj->domain); id++) {
+ if (!priv->aspace[id])
+ continue;
msm_gem_unmap_vma(priv->aspace[id],
&msm_obj->domain[id], msm_obj->sgt);
}
uint32_t mpllP;
pci_read_config_dword(pci_get_bus_and_slot(0, 3), 0x6c, &mpllP);
+ mpllP = (mpllP >> 8) & 0xf;
if (!mpllP)
mpllP = 4;
uint32_t clock;
pci_read_config_dword(pci_get_bus_and_slot(0, 5), 0x4c, &clock);
- return clock;
+ return clock / 1000;
}
ret = nouveau_hw_get_pllvals(dev, plltype, &pllvals);
return ret;
/* enable polling for external displays */
- drm_kms_helper_poll_enable(dev);
+ if (!dev->mode_config.poll_enabled)
+ drm_kms_helper_poll_enable(dev);
/* enable hotplug interrupts */
list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
pci_set_master(pdev);
ret = nouveau_do_resume(drm_dev, true);
- drm_kms_helper_poll_enable(drm_dev);
+
+ if (!drm_dev->mode_config.poll_enabled)
+ drm_kms_helper_poll_enable(drm_dev);
+
/* do magic */
nvif_mask(&device->object, 0x088488, (1 << 25), (1 << 25));
vga_switcheroo_set_dynamic_switch(pdev, VGA_SWITCHEROO_ON);
struct backlight_device *backlight;
struct list_head bl_connectors;
struct work_struct hpd_work;
+ struct work_struct fbcon_work;
+ int fbcon_new_state;
#ifdef CONFIG_ACPI
struct notifier_block acpi_nb;
#endif
.fb_probe = nouveau_fbcon_create,
};
+static void
+nouveau_fbcon_set_suspend_work(struct work_struct *work)
+{
+ struct nouveau_drm *drm = container_of(work, typeof(*drm), fbcon_work);
+ int state = READ_ONCE(drm->fbcon_new_state);
+
+ if (state == FBINFO_STATE_RUNNING)
+ pm_runtime_get_sync(drm->dev->dev);
+
+ console_lock();
+ if (state == FBINFO_STATE_RUNNING)
+ nouveau_fbcon_accel_restore(drm->dev);
+ drm_fb_helper_set_suspend(&drm->fbcon->helper, state);
+ if (state != FBINFO_STATE_RUNNING)
+ nouveau_fbcon_accel_save_disable(drm->dev);
+ console_unlock();
+
+ if (state == FBINFO_STATE_RUNNING) {
+ pm_runtime_mark_last_busy(drm->dev->dev);
+ pm_runtime_put_sync(drm->dev->dev);
+ }
+}
+
void
nouveau_fbcon_set_suspend(struct drm_device *dev, int state)
{
struct nouveau_drm *drm = nouveau_drm(dev);
- if (drm->fbcon) {
- console_lock();
- if (state == FBINFO_STATE_RUNNING)
- nouveau_fbcon_accel_restore(dev);
- drm_fb_helper_set_suspend(&drm->fbcon->helper, state);
- if (state != FBINFO_STATE_RUNNING)
- nouveau_fbcon_accel_save_disable(dev);
- console_unlock();
- }
+
+ if (!drm->fbcon)
+ return;
+
+ drm->fbcon_new_state = state;
+ /* Since runtime resume can happen as a result of a sysfs operation,
+ * it's possible we already have the console locked. So handle fbcon
+ * init/deinit from a seperate work thread
+ */
+ schedule_work(&drm->fbcon_work);
}
int
return -ENOMEM;
drm->fbcon = fbcon;
+ INIT_WORK(&drm->fbcon_work, nouveau_fbcon_set_suspend_work);
drm_fb_helper_prepare(dev, &fbcon->helper, &nouveau_fbcon_helper_funcs);
struct nouveau_bo *bo;
struct nouveau_bo *bo_gart;
u32 *suspend;
+ struct mutex mutex;
};
int nv84_fence_context_new(struct nouveau_channel *);
}
/* nouveau_led.c */
-#if IS_ENABLED(CONFIG_LEDS_CLASS)
+#if IS_REACHABLE(CONFIG_LEDS_CLASS)
int nouveau_led_init(struct drm_device *dev);
void nouveau_led_suspend(struct drm_device *dev);
void nouveau_led_resume(struct drm_device *dev);
if (!(ret = nvif_unpack(-ENOSYS, &data, &size, argv->v0, 0, 0, true))) {
/* block access to objects not created via this interface */
owner = argv->v0.owner;
- if (argv->v0.object == 0ULL)
+ if (argv->v0.object == 0ULL &&
+ argv->v0.type != NVIF_IOCTL_V0_DEL)
argv->v0.owner = NVDRM_OBJECT_ANY; /* except client */
else
argv->v0.owner = NVDRM_OBJECT_USIF;
}
}
+ for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ if (crtc->state->event)
+ drm_crtc_vblank_get(crtc);
+ }
+
/* Update plane(s). */
for_each_plane_in_state(state, plane, plane_state, i) {
struct nv50_wndw_atom *asyw = nv50_wndw_atom(plane->state);
drm_crtc_send_vblank_event(crtc, crtc->state->event);
spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
crtc->state->event = NULL;
+ drm_crtc_vblank_put(crtc);
}
}
struct nv84_fence_chan *fctx = chan->fence;
nouveau_bo_wr32(priv->bo, chan->chid * 16 / 4, fctx->base.sequence);
+ mutex_lock(&priv->mutex);
nouveau_bo_vma_del(priv->bo, &fctx->vma_gart);
nouveau_bo_vma_del(priv->bo, &fctx->vma);
+ mutex_unlock(&priv->mutex);
nouveau_fence_context_del(&fctx->base);
chan->fence = NULL;
nouveau_fence_context_free(&fctx->base);
fctx->base.sync32 = nv84_fence_sync32;
fctx->base.sequence = nv84_fence_read(chan);
+ mutex_lock(&priv->mutex);
ret = nouveau_bo_vma_add(priv->bo, cli->vm, &fctx->vma);
if (ret == 0) {
ret = nouveau_bo_vma_add(priv->bo_gart, cli->vm,
&fctx->vma_gart);
}
+ mutex_unlock(&priv->mutex);
if (ret)
nv84_fence_context_del(chan);
priv->base.context_base = dma_fence_context_alloc(priv->base.contexts);
priv->base.uevent = true;
+ mutex_init(&priv->mutex);
+
/* Use VRAM if there is any ; otherwise fallback to system memory */
domain = drm->device.info.ram_size != 0 ? TTM_PL_FLAG_VRAM :
/*
);
}
for (i = 0; i < size; i++)
- nvkm_wr32(device, 0x61c440 + soff, (i << 8) | args->v0.data[0]);
+ nvkm_wr32(device, 0x61c440 + soff, (i << 8) | args->v0.data[i]);
for (; i < 0x60; i++)
nvkm_wr32(device, 0x61c440 + soff, (i << 8));
nvkm_mask(device, 0x61c448 + soff, 0x80000003, 0x80000003);
case 0x94:
case 0x96:
case 0x98:
- case 0xaa:
- case 0xac:
return true;
default:
break;
* 2.46.0 - Add PFP_SYNC_ME support on evergreen
* 2.47.0 - Add UVD_NO_OP register support
* 2.48.0 - TA_CS_BC_BASE_ADDR allowed on SI
+ * 2.49.0 - DRM_RADEON_GEM_INFO ioctl returns correct vram_size/visible values
*/
#define KMS_DRIVER_MAJOR 2
-#define KMS_DRIVER_MINOR 48
+#define KMS_DRIVER_MINOR 49
#define KMS_DRIVER_PATCHLEVEL 0
int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags);
int radeon_driver_unload_kms(struct drm_device *dev);
radeon_pci_shutdown(struct pci_dev *pdev)
{
/* if we are running in a VM, make sure the device
- * torn down properly on reboot/shutdown.
- * unfortunately we can't detect certain
- * hypervisors so just do this all the time.
+ * torn down properly on reboot/shutdown
*/
- radeon_pci_remove(pdev);
+ if (radeon_device_is_virtual())
+ radeon_pci_remove(pdev);
}
static int radeon_pmops_suspend(struct device *dev)
man = &rdev->mman.bdev.man[TTM_PL_VRAM];
- args->vram_size = rdev->mc.real_vram_size;
- args->vram_visible = (u64)man->size << PAGE_SHIFT;
+ args->vram_size = (u64)man->size << PAGE_SHIFT;
+ args->vram_visible = rdev->mc.visible_vram_size;
args->vram_visible -= rdev->vram_pin_size;
args->gart_size = rdev->mc.gtt_size;
args->gart_size -= rdev->gart_pin_size;
MODULE_FIRMWARE("radeon/hainan_rlc.bin");
MODULE_FIRMWARE("radeon/hainan_smc.bin");
MODULE_FIRMWARE("radeon/hainan_k_smc.bin");
+MODULE_FIRMWARE("radeon/banks_k_2_smc.bin");
+
+MODULE_FIRMWARE("radeon/si58_mc.bin");
static u32 si_get_cu_active_bitmap(struct radeon_device *rdev, u32 se, u32 sh);
static void si_pcie_gen3_enable(struct radeon_device *rdev);
int err;
int new_fw = 0;
bool new_smc = false;
+ bool si58_fw = false;
+ bool banks2_fw = false;
DRM_DEBUG("\n");
((rdev->pdev->device == 0x6660) ||
(rdev->pdev->device == 0x6663) ||
(rdev->pdev->device == 0x6665) ||
- (rdev->pdev->device == 0x6667))) ||
- ((rdev->pdev->revision == 0xc3) &&
- (rdev->pdev->device == 0x6665)))
+ (rdev->pdev->device == 0x6667))))
new_smc = true;
+ else if ((rdev->pdev->revision == 0xc3) &&
+ (rdev->pdev->device == 0x6665))
+ banks2_fw = true;
new_chip_name = "hainan";
pfp_req_size = SI_PFP_UCODE_SIZE * 4;
me_req_size = SI_PM4_UCODE_SIZE * 4;
default: BUG();
}
+ /* this memory configuration requires special firmware */
+ if (((RREG32(MC_SEQ_MISC0) & 0xff000000) >> 24) == 0x58)
+ si58_fw = true;
+
DRM_INFO("Loading %s Microcode\n", new_chip_name);
snprintf(fw_name, sizeof(fw_name), "radeon/%s_pfp.bin", new_chip_name);
}
}
- snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", new_chip_name);
+ if (si58_fw)
+ snprintf(fw_name, sizeof(fw_name), "radeon/si58_mc.bin");
+ else
+ snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", new_chip_name);
err = request_firmware(&rdev->mc_fw, fw_name, rdev->dev);
if (err) {
snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc2.bin", chip_name);
}
}
- if (new_smc)
+ if (banks2_fw)
+ snprintf(fw_name, sizeof(fw_name), "radeon/banks_k_2_smc.bin");
+ else if (new_smc)
snprintf(fw_name, sizeof(fw_name), "radeon/%s_k_smc.bin", new_chip_name);
else
snprintf(fw_name, sizeof(fw_name), "radeon/%s_smc.bin", new_chip_name);
(rdev->pdev->device == 0x6817) ||
(rdev->pdev->device == 0x6806))
max_mclk = 120000;
- } else if (rdev->family == CHIP_OLAND) {
- if ((rdev->pdev->revision == 0xC7) ||
- (rdev->pdev->revision == 0x80) ||
- (rdev->pdev->revision == 0x81) ||
- (rdev->pdev->revision == 0x83) ||
- (rdev->pdev->revision == 0x87) ||
- (rdev->pdev->device == 0x6604) ||
- (rdev->pdev->device == 0x6605)) {
- max_sclk = 75000;
- max_mclk = 80000;
- }
} else if (rdev->family == CHIP_HAINAN) {
if ((rdev->pdev->revision == 0x81) ||
(rdev->pdev->revision == 0x83) ||
(rdev->pdev->device == 0x6665) ||
(rdev->pdev->device == 0x6667)) {
max_sclk = 75000;
- max_mclk = 80000;
}
}
/* Apply dpm quirks */
}
- __drm_atomic_helper_crtc_destroy_state(state);
+ drm_atomic_helper_crtc_destroy_state(crtc, state);
}
static const struct drm_crtc_funcs vc4_crtc_funcs = {
args->shader_rec_count);
struct vc4_bo *bo;
- if (uniforms_offset < shader_rec_offset ||
+ if (shader_rec_offset < args->bin_cl_size ||
+ uniforms_offset < shader_rec_offset ||
exec_size < uniforms_offset ||
args->shader_rec_count >= (UINT_MAX /
sizeof(struct vc4_shader_state)) ||
temp_size < exec_size) {
DRM_ERROR("overflow in exec arguments\n");
+ ret = -EINVAL;
goto fail;
}
}
}
plane = &vc4_plane->base;
- ret = drm_universal_plane_init(dev, plane, 0xff,
+ ret = drm_universal_plane_init(dev, plane, 0,
&vc4_plane_funcs,
formats, num_formats,
type, NULL);
}
ret = vc4_full_res_bounds_check(exec, *obj, surf);
- if (!ret)
+ if (ret)
return ret;
return 0;
info->fbops = &virtio_gpufb_ops;
info->pixmap.flags = FB_PIXMAP_SYSTEM;
- info->screen_base = obj->vmap;
+ info->screen_buffer = obj->vmap;
info->screen_size = obj->gem_base.size;
drm_fb_helper_fill_fix(info, fb->pitches[0], fb->depth);
drm_fb_helper_fill_var(info, &vfbdev->helper,
mode_cmd.height = var->yres;
mode_cmd.pitches[0] = ((var->bits_per_pixel + 7) / 8) * mode_cmd.width;
mode_cmd.pixel_format =
- drm_mode_legacy_fb_format(var->bits_per_pixel,
- ((var->bits_per_pixel + 7) / 8) * mode_cmd.width);
+ drm_mode_legacy_fb_format(var->bits_per_pixel, depth);
cur_fb = par->set_fb;
if (cur_fb && cur_fb->width == mode_cmd.width &&
atomic_t xfer_avail;
struct gpio_chip gc;
u8 *in_out_buffer;
- spinlock_t lock;
+ struct mutex lock;
struct gpio_desc *desc[8];
bool gpio_poll;
struct cp2112_device *dev = gpiochip_get_data(chip);
struct hid_device *hdev = dev->hdev;
u8 *buf = dev->in_out_buffer;
- unsigned long flags;
int ret;
- spin_lock_irqsave(&dev->lock, flags);
+ mutex_lock(&dev->lock);
ret = hid_hw_raw_request(hdev, CP2112_GPIO_CONFIG, buf,
CP2112_GPIO_CONFIG_LENGTH, HID_FEATURE_REPORT,
ret = 0;
exit:
- spin_unlock_irqrestore(&dev->lock, flags);
- return ret <= 0 ? ret : -EIO;
+ mutex_unlock(&dev->lock);
+ return ret < 0 ? ret : -EIO;
}
static void cp2112_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
struct cp2112_device *dev = gpiochip_get_data(chip);
struct hid_device *hdev = dev->hdev;
u8 *buf = dev->in_out_buffer;
- unsigned long flags;
int ret;
- spin_lock_irqsave(&dev->lock, flags);
+ mutex_lock(&dev->lock);
buf[0] = CP2112_GPIO_SET;
buf[1] = value ? 0xff : 0;
if (ret < 0)
hid_err(hdev, "error setting GPIO values: %d\n", ret);
- spin_unlock_irqrestore(&dev->lock, flags);
+ mutex_unlock(&dev->lock);
}
static int cp2112_gpio_get_all(struct gpio_chip *chip)
struct cp2112_device *dev = gpiochip_get_data(chip);
struct hid_device *hdev = dev->hdev;
u8 *buf = dev->in_out_buffer;
- unsigned long flags;
int ret;
- spin_lock_irqsave(&dev->lock, flags);
+ mutex_lock(&dev->lock);
ret = hid_hw_raw_request(hdev, CP2112_GPIO_GET, buf,
CP2112_GPIO_GET_LENGTH, HID_FEATURE_REPORT,
ret = buf[1];
exit:
- spin_unlock_irqrestore(&dev->lock, flags);
+ mutex_unlock(&dev->lock);
return ret;
}
struct cp2112_device *dev = gpiochip_get_data(chip);
struct hid_device *hdev = dev->hdev;
u8 *buf = dev->in_out_buffer;
- unsigned long flags;
int ret;
- spin_lock_irqsave(&dev->lock, flags);
+ mutex_lock(&dev->lock);
ret = hid_hw_raw_request(hdev, CP2112_GPIO_CONFIG, buf,
CP2112_GPIO_CONFIG_LENGTH, HID_FEATURE_REPORT,
goto fail;
}
- spin_unlock_irqrestore(&dev->lock, flags);
+ mutex_unlock(&dev->lock);
/*
* Set gpio value when output direction is already set,
return 0;
fail:
- spin_unlock_irqrestore(&dev->lock, flags);
+ mutex_unlock(&dev->lock);
return ret < 0 ? ret : -EIO;
}
if (!dev->in_out_buffer)
return -ENOMEM;
- spin_lock_init(&dev->lock);
+ mutex_init(&dev->lock);
ret = hid_parse(hdev);
if (ret) {
#define USB_VENDOR_ID_ALPS_JP 0x044E
#define HID_DEVICE_ID_ALPS_U1_DUAL 0x120B
+#define USB_VENDOR_ID_AMI 0x046b
+#define USB_DEVICE_ID_AMI_VIRT_KEYBOARD_AND_MOUSE 0xff10
+
#define USB_VENDOR_ID_ANTON 0x1130
#define USB_DEVICE_ID_ANTON_TOUCH_PAD 0x3101
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_WINGMAN_FFG),
.driver_data = LG_NOGET | LG_FF4 },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_RUMBLEPAD2),
- .driver_data = LG_FF2 },
+ .driver_data = LG_NOGET | LG_FF2 },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_FLIGHT_SYSTEM_G940),
.driver_data = LG_FF3 },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_SPACENAVIGATOR),
{ USB_VENDOR_ID_AIREN, USB_DEVICE_ID_AIREN_SLIMPLUS, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_AKAI, USB_DEVICE_ID_AKAI_MPKMINI2, HID_QUIRK_NO_INIT_REPORTS },
{ USB_VENDOR_ID_AKAI_09E8, USB_DEVICE_ID_AKAI_09E8_MIDIMIX, HID_QUIRK_NO_INIT_REPORTS },
+ { USB_VENDOR_ID_AMI, USB_DEVICE_ID_AMI_VIRT_KEYBOARD_AND_MOUSE, HID_QUIRK_ALWAYS_POLL },
{ USB_VENDOR_ID_ATEN, USB_DEVICE_ID_ATEN_UC100KM, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_ATEN, USB_DEVICE_ID_ATEN_CS124U, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_ATEN, USB_DEVICE_ID_ATEN_2PORTKVM, HID_QUIRK_NOGET },
wacom->id[0] = STYLUS_DEVICE_ID;
}
- pressure = (signed char)((data[7] << 1) | ((data[4] >> 2) & 1));
- if (features->pressure_max > 255)
- pressure = (pressure << 1) | ((data[4] >> 6) & 1);
- pressure += (features->pressure_max + 1) / 2;
-
- input_report_abs(input, ABS_X, data[3] | (data[2] << 7) | ((data[1] & 0x03) << 14));
- input_report_abs(input, ABS_Y, data[6] | (data[5] << 7) | ((data[4] & 0x03) << 14));
- input_report_abs(input, ABS_PRESSURE, pressure);
-
- input_report_key(input, BTN_TOUCH, data[4] & 0x08);
- input_report_key(input, BTN_STYLUS, data[4] & 0x10);
- /* Only allow the stylus2 button to be reported for the pen tool. */
- input_report_key(input, BTN_STYLUS2, (wacom->tool[0] == BTN_TOOL_PEN) && (data[4] & 0x20));
+ if (prox) {
+ pressure = (signed char)((data[7] << 1) | ((data[4] >> 2) & 1));
+ if (features->pressure_max > 255)
+ pressure = (pressure << 1) | ((data[4] >> 6) & 1);
+ pressure += (features->pressure_max + 1) / 2;
+
+ input_report_abs(input, ABS_X, data[3] | (data[2] << 7) | ((data[1] & 0x03) << 14));
+ input_report_abs(input, ABS_Y, data[6] | (data[5] << 7) | ((data[4] & 0x03) << 14));
+ input_report_abs(input, ABS_PRESSURE, pressure);
+
+ input_report_key(input, BTN_TOUCH, data[4] & 0x08);
+ input_report_key(input, BTN_STYLUS, data[4] & 0x10);
+ /* Only allow the stylus2 button to be reported for the pen tool. */
+ input_report_key(input, BTN_STYLUS2, (wacom->tool[0] == BTN_TOOL_PEN) && (data[4] & 0x20));
+ }
if (!prox)
wacom->id[0] = 0;
return ret;
}
+ init_cached_read_index(channel);
next_read_location = hv_get_next_read_location(inring_info);
next_read_location = hv_copyfrom_ringbuffer(inring_info, &desc,
sizeof(desc),
goto err_clk_dis;
}
- ret = i2c_add_adapter(&id->adap);
- if (ret < 0)
- goto err_clk_dis;
-
/*
* Cadence I2C controller has a bug wherein it generates
* invalid read transaction after HW timeout in master receiver mode.
*/
cdns_i2c_writereg(CDNS_I2C_TIMEOUT_MAX, CDNS_I2C_TIME_OUT_OFFSET);
+ ret = i2c_add_adapter(&id->adap);
+ if (ret < 0)
+ goto err_clk_dis;
+
dev_info(&pdev->dev, "%u kHz mmio %08lx irq %d\n",
id->i2c_clk / 1000, (unsigned long)r_mem->start, id->irq);
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/pinctrl/consumer.h>
#include <linux/platform_device.h>
#include <linux/sched.h>
#include <linux/slab.h>
return 0;
}
+#ifdef CONFIG_PM_SLEEP
+static int lpi2c_imx_suspend(struct device *dev)
+{
+ pinctrl_pm_select_sleep_state(dev);
+
+ return 0;
+}
+
+static int lpi2c_imx_resume(struct device *dev)
+{
+ pinctrl_pm_select_default_state(dev);
+
+ return 0;
+}
+#endif
+
+static SIMPLE_DEV_PM_OPS(imx_lpi2c_pm, lpi2c_imx_suspend, lpi2c_imx_resume);
+
static struct platform_driver lpi2c_imx_driver = {
.probe = lpi2c_imx_probe,
.remove = lpi2c_imx_remove,
.driver = {
.name = DRIVER_NAME,
.of_match_table = lpi2c_imx_of_match,
+ .pm = &imx_lpi2c_pm,
},
};
#define SMBSLVDAT (0xC + piix4_smba)
/* count for request_region */
-#define SMBIOSIZE 8
+#define SMBIOSIZE 9
/* PCI Address Constants */
#define SMBBA 0x090
u8 port;
int retval;
+ mutex_lock(&piix4_mutex_sb800);
+
/* Request the SMBUS semaphore, avoid conflicts with the IMC */
smbslvcnt = inb_p(SMBSLVCNT);
do {
usleep_range(1000, 2000);
} while (--retries);
/* SMBus is still owned by the IMC, we give up */
- if (!retries)
+ if (!retries) {
+ mutex_unlock(&piix4_mutex_sb800);
return -EBUSY;
-
- mutex_lock(&piix4_mutex_sb800);
+ }
outb_p(piix4_port_sel_sb800, SB800_PIIX4_SMB_IDX);
smba_en_lo = inb_p(SB800_PIIX4_SMB_IDX + 1);
outb_p(smba_en_lo, SB800_PIIX4_SMB_IDX + 1);
- mutex_unlock(&piix4_mutex_sb800);
-
/* Release the semaphore */
outb_p(smbslvcnt | 0x20, SMBSLVCNT);
+ mutex_unlock(&piix4_mutex_sb800);
+
return retval;
}
static int palmas_gpadc_suspend(struct device *dev)
{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct palmas_gpadc *adc = iio_priv(indio_dev);
int wakeup = adc->wakeup1_enable || adc->wakeup2_enable;
int ret;
static int palmas_gpadc_resume(struct device *dev)
{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct palmas_gpadc *adc = iio_priv(indio_dev);
int wakeup = adc->wakeup1_enable || adc->wakeup2_enable;
int ret;
static int __maybe_unused afe4403_suspend(struct device *dev)
{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev *indio_dev = spi_get_drvdata(to_spi_device(dev));
struct afe4403_data *afe = iio_priv(indio_dev);
int ret;
static int __maybe_unused afe4403_resume(struct device *dev)
{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev *indio_dev = spi_get_drvdata(to_spi_device(dev));
struct afe4403_data *afe = iio_priv(indio_dev);
int ret;
static int __maybe_unused afe4404_suspend(struct device *dev)
{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
struct afe4404_data *afe = iio_priv(indio_dev);
int ret;
static int __maybe_unused afe4404_resume(struct device *dev)
{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
struct afe4404_data *afe = iio_priv(indio_dev);
int ret;
mutex_lock(&data->lock);
- while (cnt || (cnt = max30100_fifo_count(data) > 0)) {
+ while (cnt || (cnt = max30100_fifo_count(data)) > 0) {
ret = max30100_read_measurement(data);
if (ret)
break;
* a) select an implementation using busy loop polling on those systems
* b) use the checksum to do some probabilistic decoding
*/
-#define DHT11_START_TRANSMISSION 18 /* ms */
+#define DHT11_START_TRANSMISSION_MIN 18000 /* us */
+#define DHT11_START_TRANSMISSION_MAX 20000 /* us */
#define DHT11_MIN_TIMERES 34000 /* ns */
#define DHT11_THRESHOLD 49000 /* ns */
#define DHT11_AMBIG_LOW 23000 /* ns */
ret = gpio_direction_output(dht11->gpio, 0);
if (ret)
goto err;
- msleep(DHT11_START_TRANSMISSION);
+ usleep_range(DHT11_START_TRANSMISSION_MIN,
+ DHT11_START_TRANSMISSION_MAX);
ret = gpio_direction_input(dht11->gpio);
if (ret)
goto err;
if (!src_addr || !src_addr->sa_family) {
src_addr = (struct sockaddr *) &id->route.addr.src_addr;
src_addr->sa_family = dst_addr->sa_family;
- if (dst_addr->sa_family == AF_INET6) {
+ if (IS_ENABLED(CONFIG_IPV6) &&
+ dst_addr->sa_family == AF_INET6) {
struct sockaddr_in6 *src_addr6 = (struct sockaddr_in6 *) src_addr;
struct sockaddr_in6 *dst_addr6 = (struct sockaddr_in6 *) dst_addr;
src_addr6->sin6_scope_id = dst_addr6->sin6_scope_id;
IB_ACCESS_REMOTE_ATOMIC | IB_ACCESS_MW_BIND));
if (access & IB_ACCESS_ON_DEMAND) {
+ put_pid(umem->pid);
ret = ib_umem_odp_get(context, umem);
if (ret) {
kfree(umem);
page_list = (struct page **) __get_free_page(GFP_KERNEL);
if (!page_list) {
+ put_pid(umem->pid);
kfree(umem);
return ERR_PTR(-ENOMEM);
}
memset(props, 0, sizeof(struct ib_port_attr));
props->max_mtu = IB_MTU_4096;
- if (netdev->mtu >= 4096)
- props->active_mtu = IB_MTU_4096;
- else if (netdev->mtu >= 2048)
- props->active_mtu = IB_MTU_2048;
- else if (netdev->mtu >= 1024)
- props->active_mtu = IB_MTU_1024;
- else if (netdev->mtu >= 512)
- props->active_mtu = IB_MTU_512;
- else
- props->active_mtu = IB_MTU_256;
+ props->active_mtu = ib_mtu_int_to_enum(netdev->mtu);
if (!netif_carrier_ok(netdev))
props->state = IB_PORT_DOWN;
skb_trim(skb, dlen);
mutex_lock(&ep->com.mutex);
- /* update RX credits */
- update_rx_credits(ep, dlen);
-
switch (ep->com.state) {
case MPA_REQ_SENT:
+ update_rx_credits(ep, dlen);
ep->rcv_seq += dlen;
disconnect = process_mpa_reply(ep, skb);
break;
case MPA_REQ_WAIT:
+ update_rx_credits(ep, dlen);
ep->rcv_seq += dlen;
disconnect = process_mpa_request(ep, skb);
break;
case FPDU_MODE: {
struct c4iw_qp_attributes attrs;
+
+ update_rx_credits(ep, dlen);
BUG_ON(!ep->com.qp);
if (status)
pr_err("%s Unexpected streaming data." \
goto skip_cqe;
}
+ /*
+ * Special cqe for drain WR completions...
+ */
+ if (CQE_OPCODE(hw_cqe) == C4IW_DRAIN_OPCODE) {
+ *cookie = CQE_DRAIN_COOKIE(hw_cqe);
+ *cqe = *hw_cqe;
+ goto skip_cqe;
+ }
+
/*
* Gotta tweak READ completions:
* 1) the cqe doesn't contain the sq_wptr from the wr.
c4iw_invalidate_mr(qhp->rhp,
CQE_WRID_FR_STAG(&cqe));
break;
+ case C4IW_DRAIN_OPCODE:
+ wc->opcode = IB_WC_SEND;
+ break;
default:
printk(KERN_ERR MOD "Unexpected opcode %d "
"in the CQE received for QPID=0x%0x\n",
}
}
out:
- if (wq) {
- if (unlikely(qhp->attr.state != C4IW_QP_STATE_RTS)) {
- if (t4_sq_empty(wq))
- complete(&qhp->sq_drained);
- if (t4_rq_empty(wq))
- complete(&qhp->rq_drained);
- }
+ if (wq)
spin_unlock(&qhp->lock);
- }
return ret;
}
}
}
+ rdev->free_workq = create_singlethread_workqueue("iw_cxgb4_free");
+ if (!rdev->free_workq) {
+ err = -ENOMEM;
+ goto err_free_status_page;
+ }
+
rdev->status_page->db_off = 0;
return 0;
+err_free_status_page:
+ free_page((unsigned long)rdev->status_page);
destroy_ocqp_pool:
c4iw_ocqp_pool_destroy(rdev);
destroy_rqtpool:
static void c4iw_rdev_close(struct c4iw_rdev *rdev)
{
+ destroy_workqueue(rdev->free_workq);
kfree(rdev->wr_log);
free_page((unsigned long)rdev->status_page);
c4iw_pblpool_destroy(rdev);
#include <linux/kref.h>
#include <linux/timer.h>
#include <linux/io.h>
+#include <linux/workqueue.h>
#include <asm/byteorder.h>
struct list_head qpids;
struct list_head cqids;
struct mutex lock;
+ struct kref kref;
};
enum c4iw_rdev_flags {
atomic_t wr_log_idx;
struct wr_log_entry *wr_log;
int wr_log_size;
+ struct workqueue_struct *free_workq;
};
static inline int c4iw_fatal_error(struct c4iw_rdev *rdev)
wait_queue_head_t wait;
struct timer_list timer;
int sq_sig_all;
- struct completion rq_drained;
- struct completion sq_drained;
+ struct work_struct free_work;
+ struct c4iw_ucontext *ucontext;
};
static inline struct c4iw_qp *to_c4iw_qp(struct ib_qp *ibqp)
u32 key;
spinlock_t mmap_lock;
struct list_head mmaps;
+ struct kref kref;
};
static inline struct c4iw_ucontext *to_c4iw_ucontext(struct ib_ucontext *c)
return container_of(c, struct c4iw_ucontext, ibucontext);
}
+void _c4iw_free_ucontext(struct kref *kref);
+
+static inline void c4iw_put_ucontext(struct c4iw_ucontext *ucontext)
+{
+ kref_put(&ucontext->kref, _c4iw_free_ucontext);
+}
+
+static inline void c4iw_get_ucontext(struct c4iw_ucontext *ucontext)
+{
+ kref_get(&ucontext->kref);
+}
+
struct c4iw_mm_entry {
struct list_head entry;
u64 addr;
return IB_QPS_ERR;
}
+#define C4IW_DRAIN_OPCODE FW_RI_SGE_EC_CR_RETURN
+
static inline u32 c4iw_ib_to_tpt_access(int a)
{
return (a & IB_ACCESS_REMOTE_WRITE ? FW_RI_MEM_ACCESS_REM_WRITE : 0) |
extern int db_fc_threshold;
extern int db_coalescing_threshold;
extern int use_dsgl;
-void c4iw_drain_rq(struct ib_qp *qp);
-void c4iw_drain_sq(struct ib_qp *qp);
void c4iw_invalidate_mr(struct c4iw_dev *rhp, u32 rkey);
#endif
return -ENOSYS;
}
-static int c4iw_dealloc_ucontext(struct ib_ucontext *context)
+void _c4iw_free_ucontext(struct kref *kref)
{
- struct c4iw_dev *rhp = to_c4iw_dev(context->device);
- struct c4iw_ucontext *ucontext = to_c4iw_ucontext(context);
+ struct c4iw_ucontext *ucontext;
+ struct c4iw_dev *rhp;
struct c4iw_mm_entry *mm, *tmp;
- PDBG("%s context %p\n", __func__, context);
+ ucontext = container_of(kref, struct c4iw_ucontext, kref);
+ rhp = to_c4iw_dev(ucontext->ibucontext.device);
+
+ PDBG("%s ucontext %p\n", __func__, ucontext);
list_for_each_entry_safe(mm, tmp, &ucontext->mmaps, entry)
kfree(mm);
c4iw_release_dev_ucontext(&rhp->rdev, &ucontext->uctx);
kfree(ucontext);
+}
+
+static int c4iw_dealloc_ucontext(struct ib_ucontext *context)
+{
+ struct c4iw_ucontext *ucontext = to_c4iw_ucontext(context);
+
+ PDBG("%s context %p\n", __func__, context);
+ c4iw_put_ucontext(ucontext);
return 0;
}
c4iw_init_dev_ucontext(&rhp->rdev, &context->uctx);
INIT_LIST_HEAD(&context->mmaps);
spin_lock_init(&context->mmap_lock);
+ kref_init(&context->kref);
if (udata->outlen < sizeof(uresp) - sizeof(uresp.reserved)) {
if (!warned++)
memset(props, 0, sizeof(struct ib_port_attr));
props->max_mtu = IB_MTU_4096;
- if (netdev->mtu >= 4096)
- props->active_mtu = IB_MTU_4096;
- else if (netdev->mtu >= 2048)
- props->active_mtu = IB_MTU_2048;
- else if (netdev->mtu >= 1024)
- props->active_mtu = IB_MTU_1024;
- else if (netdev->mtu >= 512)
- props->active_mtu = IB_MTU_512;
- else
- props->active_mtu = IB_MTU_256;
+ props->active_mtu = ib_mtu_int_to_enum(netdev->mtu);
if (!netif_carrier_ok(netdev))
props->state = IB_PORT_DOWN;
dev->ibdev.uverbs_abi_ver = C4IW_UVERBS_ABI_VERSION;
dev->ibdev.get_port_immutable = c4iw_port_immutable;
dev->ibdev.get_dev_fw_str = get_dev_fw_str;
- dev->ibdev.drain_sq = c4iw_drain_sq;
- dev->ibdev.drain_rq = c4iw_drain_rq;
dev->ibdev.iwcm = kmalloc(sizeof(struct iw_cm_verbs), GFP_KERNEL);
if (!dev->ibdev.iwcm)
return 0;
}
-static void _free_qp(struct kref *kref)
+static void free_qp_work(struct work_struct *work)
+{
+ struct c4iw_ucontext *ucontext;
+ struct c4iw_qp *qhp;
+ struct c4iw_dev *rhp;
+
+ qhp = container_of(work, struct c4iw_qp, free_work);
+ ucontext = qhp->ucontext;
+ rhp = qhp->rhp;
+
+ PDBG("%s qhp %p ucontext %p\n", __func__, qhp, ucontext);
+ destroy_qp(&rhp->rdev, &qhp->wq,
+ ucontext ? &ucontext->uctx : &rhp->rdev.uctx);
+
+ if (ucontext)
+ c4iw_put_ucontext(ucontext);
+ kfree(qhp);
+}
+
+static void queue_qp_free(struct kref *kref)
{
struct c4iw_qp *qhp;
qhp = container_of(kref, struct c4iw_qp, kref);
PDBG("%s qhp %p\n", __func__, qhp);
- kfree(qhp);
+ queue_work(qhp->rhp->rdev.free_workq, &qhp->free_work);
}
void c4iw_qp_add_ref(struct ib_qp *qp)
void c4iw_qp_rem_ref(struct ib_qp *qp)
{
PDBG("%s ib_qp %p\n", __func__, qp);
- kref_put(&to_c4iw_qp(qp)->kref, _free_qp);
+ kref_put(&to_c4iw_qp(qp)->kref, queue_qp_free);
}
static void add_to_fc_list(struct list_head *head, struct list_head *entry)
return 0;
}
+static void complete_sq_drain_wr(struct c4iw_qp *qhp, struct ib_send_wr *wr)
+{
+ struct t4_cqe cqe = {};
+ struct c4iw_cq *schp;
+ unsigned long flag;
+ struct t4_cq *cq;
+
+ schp = to_c4iw_cq(qhp->ibqp.send_cq);
+ cq = &schp->cq;
+
+ cqe.u.drain_cookie = wr->wr_id;
+ cqe.header = cpu_to_be32(CQE_STATUS_V(T4_ERR_SWFLUSH) |
+ CQE_OPCODE_V(C4IW_DRAIN_OPCODE) |
+ CQE_TYPE_V(1) |
+ CQE_SWCQE_V(1) |
+ CQE_QPID_V(qhp->wq.sq.qid));
+
+ spin_lock_irqsave(&schp->lock, flag);
+ cqe.bits_type_ts = cpu_to_be64(CQE_GENBIT_V((u64)cq->gen));
+ cq->sw_queue[cq->sw_pidx] = cqe;
+ t4_swcq_produce(cq);
+ spin_unlock_irqrestore(&schp->lock, flag);
+
+ spin_lock_irqsave(&schp->comp_handler_lock, flag);
+ (*schp->ibcq.comp_handler)(&schp->ibcq,
+ schp->ibcq.cq_context);
+ spin_unlock_irqrestore(&schp->comp_handler_lock, flag);
+}
+
+static void complete_rq_drain_wr(struct c4iw_qp *qhp, struct ib_recv_wr *wr)
+{
+ struct t4_cqe cqe = {};
+ struct c4iw_cq *rchp;
+ unsigned long flag;
+ struct t4_cq *cq;
+
+ rchp = to_c4iw_cq(qhp->ibqp.recv_cq);
+ cq = &rchp->cq;
+
+ cqe.u.drain_cookie = wr->wr_id;
+ cqe.header = cpu_to_be32(CQE_STATUS_V(T4_ERR_SWFLUSH) |
+ CQE_OPCODE_V(C4IW_DRAIN_OPCODE) |
+ CQE_TYPE_V(0) |
+ CQE_SWCQE_V(1) |
+ CQE_QPID_V(qhp->wq.sq.qid));
+
+ spin_lock_irqsave(&rchp->lock, flag);
+ cqe.bits_type_ts = cpu_to_be64(CQE_GENBIT_V((u64)cq->gen));
+ cq->sw_queue[cq->sw_pidx] = cqe;
+ t4_swcq_produce(cq);
+ spin_unlock_irqrestore(&rchp->lock, flag);
+
+ spin_lock_irqsave(&rchp->comp_handler_lock, flag);
+ (*rchp->ibcq.comp_handler)(&rchp->ibcq,
+ rchp->ibcq.cq_context);
+ spin_unlock_irqrestore(&rchp->comp_handler_lock, flag);
+}
+
int c4iw_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
struct ib_send_wr **bad_wr)
{
spin_lock_irqsave(&qhp->lock, flag);
if (t4_wq_in_error(&qhp->wq)) {
spin_unlock_irqrestore(&qhp->lock, flag);
- *bad_wr = wr;
- return -EINVAL;
+ complete_sq_drain_wr(qhp, wr);
+ return err;
}
num_wrs = t4_sq_avail(&qhp->wq);
if (num_wrs == 0) {
spin_lock_irqsave(&qhp->lock, flag);
if (t4_wq_in_error(&qhp->wq)) {
spin_unlock_irqrestore(&qhp->lock, flag);
- *bad_wr = wr;
- return -EINVAL;
+ complete_rq_drain_wr(qhp, wr);
+ return err;
}
num_wrs = t4_rq_avail(&qhp->wq);
if (num_wrs == 0) {
}
break;
case C4IW_QP_STATE_CLOSING:
- if (!internal) {
+
+ /*
+ * Allow kernel users to move to ERROR for qp draining.
+ */
+ if (!internal && (qhp->ibqp.uobject || attrs->next_state !=
+ C4IW_QP_STATE_ERROR)) {
ret = -EINVAL;
goto out;
}
struct c4iw_dev *rhp;
struct c4iw_qp *qhp;
struct c4iw_qp_attributes attrs;
- struct c4iw_ucontext *ucontext;
qhp = to_c4iw_qp(ib_qp);
rhp = qhp->rhp;
spin_unlock_irq(&rhp->lock);
free_ird(rhp, qhp->attr.max_ird);
- ucontext = ib_qp->uobject ?
- to_c4iw_ucontext(ib_qp->uobject->context) : NULL;
- destroy_qp(&rhp->rdev, &qhp->wq,
- ucontext ? &ucontext->uctx : &rhp->rdev.uctx);
-
c4iw_qp_rem_ref(ib_qp);
PDBG("%s ib_qp %p qpid 0x%0x\n", __func__, ib_qp, qhp->wq.sq.qid);
qhp->attr.max_ird = 0;
qhp->sq_sig_all = attrs->sq_sig_type == IB_SIGNAL_ALL_WR;
spin_lock_init(&qhp->lock);
- init_completion(&qhp->sq_drained);
- init_completion(&qhp->rq_drained);
mutex_init(&qhp->mutex);
init_waitqueue_head(&qhp->wait);
kref_init(&qhp->kref);
+ INIT_WORK(&qhp->free_work, free_qp_work);
ret = insert_handle(rhp, &rhp->qpidr, qhp, qhp->wq.sq.qid);
if (ret)
ma_sync_key_mm->len = PAGE_SIZE;
insert_mmap(ucontext, ma_sync_key_mm);
}
+
+ c4iw_get_ucontext(ucontext);
+ qhp->ucontext = ucontext;
}
qhp->ibqp.qp_num = qhp->wq.sq.qid;
init_timer(&(qhp->timer));
init_attr->sq_sig_type = qhp->sq_sig_all ? IB_SIGNAL_ALL_WR : 0;
return 0;
}
-
-static void move_qp_to_err(struct c4iw_qp *qp)
-{
- struct c4iw_qp_attributes attrs = { .next_state = C4IW_QP_STATE_ERROR };
-
- (void)c4iw_modify_qp(qp->rhp, qp, C4IW_QP_ATTR_NEXT_STATE, &attrs, 1);
-}
-
-void c4iw_drain_sq(struct ib_qp *ibqp)
-{
- struct c4iw_qp *qp = to_c4iw_qp(ibqp);
- unsigned long flag;
- bool need_to_wait;
-
- move_qp_to_err(qp);
- spin_lock_irqsave(&qp->lock, flag);
- need_to_wait = !t4_sq_empty(&qp->wq);
- spin_unlock_irqrestore(&qp->lock, flag);
-
- if (need_to_wait)
- wait_for_completion(&qp->sq_drained);
-}
-
-void c4iw_drain_rq(struct ib_qp *ibqp)
-{
- struct c4iw_qp *qp = to_c4iw_qp(ibqp);
- unsigned long flag;
- bool need_to_wait;
-
- move_qp_to_err(qp);
- spin_lock_irqsave(&qp->lock, flag);
- need_to_wait = !t4_rq_empty(&qp->wq);
- spin_unlock_irqrestore(&qp->lock, flag);
-
- if (need_to_wait)
- wait_for_completion(&qp->rq_drained);
-}
__be32 wrid_hi;
__be32 wrid_low;
} gen;
+ u64 drain_cookie;
} u;
__be64 reserved;
__be64 bits_type_ts;
/* generic accessor macros */
#define CQE_WRID_HI(x) (be32_to_cpu((x)->u.gen.wrid_hi))
#define CQE_WRID_LOW(x) (be32_to_cpu((x)->u.gen.wrid_low))
+#define CQE_DRAIN_COOKIE(x) ((x)->u.drain_cookie)
/* macros for flit 3 of the cqe */
#define CQE_GENBIT_S 63
memset(props, 0, sizeof(*props));
props->max_mtu = IB_MTU_4096;
- if (netdev->mtu >= 4096)
- props->active_mtu = IB_MTU_4096;
- else if (netdev->mtu >= 2048)
- props->active_mtu = IB_MTU_2048;
- else if (netdev->mtu >= 1024)
- props->active_mtu = IB_MTU_1024;
- else if (netdev->mtu >= 512)
- props->active_mtu = IB_MTU_512;
- else
- props->active_mtu = IB_MTU_256;
+ props->active_mtu = ib_mtu_int_to_enum(netdev->mtu);
props->lid = 1;
if (netif_carrier_ok(iwdev->netdev))
memset(props, 0, sizeof(*props));
props->max_mtu = IB_MTU_4096;
-
- if (netdev->mtu >= 4096)
- props->active_mtu = IB_MTU_4096;
- else if (netdev->mtu >= 2048)
- props->active_mtu = IB_MTU_2048;
- else if (netdev->mtu >= 1024)
- props->active_mtu = IB_MTU_1024;
- else if (netdev->mtu >= 512)
- props->active_mtu = IB_MTU_512;
- else
- props->active_mtu = IB_MTU_256;
+ props->active_mtu = ib_mtu_int_to_enum(netdev->mtu);
props->lid = 1;
props->lmc = 0;
return 0;
}
-void qedr_unaffiliated_event(void *context,
- u8 event_code)
+void qedr_unaffiliated_event(void *context, u8 event_code)
{
pr_err("unaffiliated event not implemented yet\n");
}
if (device_create_file(&dev->ibdev.dev, qedr_attributes[i]))
goto sysfs_err;
+ if (!test_and_set_bit(QEDR_ENET_STATE_BIT, &dev->enet_state))
+ qedr_ib_dispatch_event(dev, QEDR_PORT, IB_EVENT_PORT_ACTIVE);
+
DP_DEBUG(dev, QEDR_MSG_INIT, "qedr driver loaded successfully\n");
return dev;
ib_dealloc_device(&dev->ibdev);
}
-static int qedr_close(struct qedr_dev *dev)
+static void qedr_close(struct qedr_dev *dev)
{
- qedr_ib_dispatch_event(dev, 1, IB_EVENT_PORT_ERR);
-
- return 0;
+ if (test_and_clear_bit(QEDR_ENET_STATE_BIT, &dev->enet_state))
+ qedr_ib_dispatch_event(dev, QEDR_PORT, IB_EVENT_PORT_ERR);
}
static void qedr_shutdown(struct qedr_dev *dev)
qedr_remove(dev);
}
+static void qedr_open(struct qedr_dev *dev)
+{
+ if (!test_and_set_bit(QEDR_ENET_STATE_BIT, &dev->enet_state))
+ qedr_ib_dispatch_event(dev, QEDR_PORT, IB_EVENT_PORT_ACTIVE);
+}
+
static void qedr_mac_address_change(struct qedr_dev *dev)
{
union ib_gid *sgid = &dev->sgid_tbl[0];
ether_addr_copy(dev->gsi_ll2_mac_address, dev->ndev->dev_addr);
- qedr_ib_dispatch_event(dev, 1, IB_EVENT_GID_CHANGE);
+ qedr_ib_dispatch_event(dev, QEDR_PORT, IB_EVENT_GID_CHANGE);
if (rc)
DP_ERR(dev, "Error updating mac filter\n");
{
switch (event) {
case QEDE_UP:
- qedr_ib_dispatch_event(dev, 1, IB_EVENT_PORT_ACTIVE);
+ qedr_open(dev);
break;
case QEDE_DOWN:
qedr_close(dev);
struct qed_rdma_events events;
};
+#define QEDR_ENET_STATE_BIT (0)
+
struct qedr_dev {
struct ib_device ibdev;
struct qed_dev *cdev;
struct qedr_cq *gsi_sqcq;
struct qedr_cq *gsi_rqcq;
struct qedr_qp *gsi_qp;
+
+ unsigned long enet_state;
};
#define QEDR_MAX_SQ_PBL (0x8000)
#define QEDR_ROCE_MAX_CNQ_SIZE (0x4000)
#define QEDR_MAX_PORT (1)
+#define QEDR_PORT (1)
#define QEDR_UVERBS(CMD_NAME) (1ull << IB_USER_VERBS_CMD_##CMD_NAME)
u16 icid;
- /* Lock to protect completion handler */
- spinlock_t comp_handler_lock;
-
/* Lock to protect multiplem CQ's */
spinlock_t cq_lock;
u8 arm_flags;
qedr_inc_sw_gsi_cons(&qp->sq);
spin_unlock_irqrestore(&qp->q_lock, flags);
- if (cq->ibcq.comp_handler) {
- spin_lock_irqsave(&cq->comp_handler_lock, flags);
+ if (cq->ibcq.comp_handler)
(*cq->ibcq.comp_handler) (&cq->ibcq, cq->ibcq.cq_context);
- spin_unlock_irqrestore(&cq->comp_handler_lock, flags);
- }
}
void qedr_ll2_rx_cb(void *_dev, struct qed_roce_ll2_packet *pkt,
spin_unlock_irqrestore(&qp->q_lock, flags);
- if (cq->ibcq.comp_handler) {
- spin_lock_irqsave(&cq->comp_handler_lock, flags);
+ if (cq->ibcq.comp_handler)
(*cq->ibcq.comp_handler) (&cq->ibcq, cq->ibcq.cq_context);
- spin_unlock_irqrestore(&cq->comp_handler_lock, flags);
- }
}
static void qedr_destroy_gsi_cq(struct qedr_dev *dev,
}
if (ether_addr_equal(udh.eth.smac_h, udh.eth.dmac_h))
- packet->tx_dest = QED_ROCE_LL2_TX_DEST_NW;
- else
packet->tx_dest = QED_ROCE_LL2_TX_DEST_LB;
+ else
+ packet->tx_dest = QED_ROCE_LL2_TX_DEST_NW;
packet->roce_mode = roce_mode;
memcpy(packet->header.vaddr, ud_header_buffer, header_size);
struct ib_ucontext *context, struct ib_udata *udata)
{
struct qedr_dev *dev = get_qedr_dev(ibdev);
- struct qedr_ucontext *uctx = NULL;
- struct qedr_alloc_pd_uresp uresp;
struct qedr_pd *pd;
u16 pd_id;
int rc;
if (!pd)
return ERR_PTR(-ENOMEM);
- dev->ops->rdma_alloc_pd(dev->rdma_ctx, &pd_id);
+ rc = dev->ops->rdma_alloc_pd(dev->rdma_ctx, &pd_id);
+ if (rc)
+ goto err;
- uresp.pd_id = pd_id;
pd->pd_id = pd_id;
if (udata && context) {
+ struct qedr_alloc_pd_uresp uresp;
+
+ uresp.pd_id = pd_id;
+
rc = ib_copy_to_udata(udata, &uresp, sizeof(uresp));
- if (rc)
+ if (rc) {
DP_ERR(dev, "copy error pd_id=0x%x.\n", pd_id);
- uctx = get_qedr_ucontext(context);
- uctx->pd = pd;
- pd->uctx = uctx;
+ dev->ops->rdma_dealloc_pd(dev->rdma_ctx, pd_id);
+ goto err;
+ }
+
+ pd->uctx = get_qedr_ucontext(context);
+ pd->uctx->pd = pd;
}
return &pd->ibpd;
+
+err:
+ kfree(pd);
+ return ERR_PTR(rc);
}
int qedr_dealloc_pd(struct ib_pd *ibpd)
return ERR_PTR(-EFAULT);
}
-enum ib_qp_state qedr_get_ibqp_state(enum qed_roce_qp_state qp_state)
+static enum ib_qp_state qedr_get_ibqp_state(enum qed_roce_qp_state qp_state)
{
switch (qp_state) {
case QED_ROCE_QP_STATE_RESET:
return IB_QPS_ERR;
}
-enum qed_roce_qp_state qedr_get_state_from_ibqp(enum ib_qp_state qp_state)
+static enum qed_roce_qp_state qedr_get_state_from_ibqp(
+ enum ib_qp_state qp_state)
{
switch (qp_state) {
case IB_QPS_RESET:
int status = 0;
if (new_state == qp->state)
- return 1;
+ return 0;
switch (qp->state) {
case QED_ROCE_QP_STATE_RESET:
/* ERR->XXX */
switch (new_state) {
case QED_ROCE_QP_STATE_RESET:
+ if ((qp->rq.prod != qp->rq.cons) ||
+ (qp->sq.prod != qp->sq.cons)) {
+ DP_NOTICE(dev,
+ "Error->Reset with rq/sq not empty rq.prod=%x rq.cons=%x sq.prod=%x sq.cons=%x\n",
+ qp->rq.prod, qp->rq.cons, qp->sq.prod,
+ qp->sq.cons);
+ status = -EINVAL;
+ }
break;
default:
status = -EINVAL;
qp_params.sgid.dwords[2], qp_params.sgid.dwords[3]);
DP_DEBUG(dev, QEDR_MSG_QP, "remote_mac=[%pM]\n",
qp_params.remote_mac_addr);
-;
qp_params.mtu = qp->mtu;
qp_params.lb_indication = false;
qp_attr->qp_state = qedr_get_ibqp_state(params.state);
qp_attr->cur_qp_state = qedr_get_ibqp_state(params.state);
- qp_attr->path_mtu = iboe_get_mtu(params.mtu);
+ qp_attr->path_mtu = ib_mtu_int_to_enum(params.mtu);
qp_attr->path_mig_state = IB_MIG_MIGRATED;
qp_attr->rq_psn = params.rq_psn;
qp_attr->sq_psn = params.sq_psn;
qp_attr->cap.max_recv_wr = qp->rq.max_wr;
qp_attr->cap.max_send_sge = qp->sq.max_sges;
qp_attr->cap.max_recv_sge = qp->rq.max_sges;
- qp_attr->cap.max_inline_data = qp->max_inline_data;
+ qp_attr->cap.max_inline_data = ROCE_REQ_MAX_INLINE_DATA_SIZE;
qp_init_attr->cap = qp_attr->cap;
memcpy(&qp_attr->ah_attr.grh.dgid.raw[0], ¶ms.dgid.bytes[0],
return rc;
}
-struct qedr_mr *__qedr_alloc_mr(struct ib_pd *ibpd, int max_page_list_len)
+static struct qedr_mr *__qedr_alloc_mr(struct ib_pd *ibpd,
+ int max_page_list_len)
{
struct qedr_pd *pd = get_qedr_pd(ibpd);
struct qedr_dev *dev = get_qedr_dev(ibpd->device);
return 0;
}
-enum ib_wc_opcode qedr_ib_to_wc_opcode(enum ib_wr_opcode opcode)
+static enum ib_wc_opcode qedr_ib_to_wc_opcode(enum ib_wr_opcode opcode)
{
switch (opcode) {
case IB_WR_RDMA_WRITE:
}
}
-inline bool qedr_can_post_send(struct qedr_qp *qp, struct ib_send_wr *wr)
+static inline bool qedr_can_post_send(struct qedr_qp *qp, struct ib_send_wr *wr)
{
int wq_is_full, err_wr, pbl_is_full;
struct qedr_dev *dev = qp->dev;
return true;
}
-int __qedr_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
+static int __qedr_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
struct ib_send_wr **bad_wr)
{
struct qedr_dev *dev = get_qedr_dev(ibqp->device);
IB_WC_SUCCESS, 0);
break;
case RDMA_CQE_REQ_STS_WORK_REQUEST_FLUSHED_ERR:
- DP_ERR(dev,
- "Error: POLL CQ with RDMA_CQE_REQ_STS_WORK_REQUEST_FLUSHED_ERR. CQ icid=0x%x, QP icid=0x%x\n",
- cq->icid, qp->icid);
+ if (qp->state != QED_ROCE_QP_STATE_ERR)
+ DP_ERR(dev,
+ "Error: POLL CQ with RDMA_CQE_REQ_STS_WORK_REQUEST_FLUSHED_ERR. CQ icid=0x%x, QP icid=0x%x\n",
+ cq->icid, qp->icid);
cnt = process_req(dev, qp, cq, num_entries, wc, req->sq_cons,
IB_WC_WR_FLUSH_ERR, 1);
break;
if (ret) {
dev_err(&pdev->dev, "failed to allocate interrupts\n");
ret = -ENOMEM;
- goto err_netdevice;
+ goto err_free_cq_ring;
}
/* Allocate UAR table. */
err_free_intrs:
pvrdma_free_irq(dev);
pvrdma_disable_msi_all(dev);
-err_netdevice:
- unregister_netdevice_notifier(&dev->nb_netdev);
err_free_cq_ring:
pvrdma_page_dir_cleanup(dev, &dev->cq_pdir);
err_free_async_ring:
union pvrdma_cmd_resp rsp;
struct pvrdma_cmd_create_uc *cmd = &req.create_uc;
struct pvrdma_cmd_create_uc_resp *resp = &rsp.create_uc_resp;
- struct pvrdma_alloc_ucontext_resp uresp;
+ struct pvrdma_alloc_ucontext_resp uresp = {0};
int ret;
void *ptr;
case RXE_MEM_TYPE_MR:
case RXE_MEM_TYPE_FMR:
- return ((iova < mem->iova) ||
- ((iova + length) > (mem->iova + mem->length))) ?
- -EFAULT : 0;
+ if (iova < mem->iova ||
+ length > mem->length ||
+ iova > mem->iova + mem->length - length)
+ return -EFAULT;
+ return 0;
default:
return -EFAULT;
}
spin_lock_bh(&dev_list_lock);
- list_add_tail(&rxe_dev_list, &rxe->list);
+ list_add_tail(&rxe->list, &rxe_dev_list);
spin_unlock_bh(&dev_list_lock);
return rxe;
}
del_timer_sync(&qp->rnr_nak_timer);
rxe_cleanup_task(&qp->req.task);
- if (qp_type(qp) == IB_QPT_RC)
- rxe_cleanup_task(&qp->comp.task);
+ rxe_cleanup_task(&qp->comp.task);
/* flush out any receive wr's or pending requests */
__rxe_do_task(&qp->req.task);
goto err2;
}
- resid = mtu;
+ qp->resp.resid = mtu;
} else {
if (pktlen != resid) {
state = RESPST_ERR_LENGTH;
SHOST_DIX_GUARD_CRC);
}
- /*
- * Limit the sg_tablesize and max_sectors based on the device
- * max fastreg page list length.
- */
- shost->sg_tablesize = min_t(unsigned short, shost->sg_tablesize,
- ib_conn->device->ib_device->attrs.max_fast_reg_page_list_len);
-
if (iscsi_host_add(shost,
ib_conn->device->ib_device->dma_device)) {
mutex_unlock(&iser_conn->state_mutex);
max_fr_sectors = ((shost->sg_tablesize - 1) * PAGE_SIZE) >> 9;
shost->max_sectors = min(iser_max_sectors, max_fr_sectors);
+ iser_dbg("iser_conn %p, sg_tablesize %u, max_sectors %u\n",
+ iser_conn, shost->sg_tablesize,
+ shost->max_sectors);
+
if (cmds_max > max_cmds) {
iser_info("cmds_max changed from %u to %u\n",
cmds_max, max_cmds);
* @rx_descs: rx buffers array (cyclic buffer)
* @num_rx_descs: number of rx descriptors
* @scsi_sg_tablesize: scsi host sg_tablesize
- * @scsi_max_sectors: scsi host max sectors
*/
struct iser_conn {
struct ib_conn ib_conn;
struct iser_rx_desc *rx_descs;
u32 num_rx_descs;
unsigned short scsi_sg_tablesize;
- unsigned int scsi_max_sectors;
bool snd_w_inv;
};
sup_sg_tablesize = min_t(unsigned, ISCSI_ISER_MAX_SG_TABLESIZE,
device->ib_device->attrs.max_fast_reg_page_list_len);
- if (sg_tablesize > sup_sg_tablesize) {
- sg_tablesize = sup_sg_tablesize;
- iser_conn->scsi_max_sectors = sg_tablesize * SIZE_4K / 512;
- } else {
- iser_conn->scsi_max_sectors = max_sectors;
- }
-
- iser_conn->scsi_sg_tablesize = sg_tablesize;
-
- iser_dbg("iser_conn %p, sg_tablesize %u, max_sectors %u\n",
- iser_conn, iser_conn->scsi_sg_tablesize,
- iser_conn->scsi_max_sectors);
+ iser_conn->scsi_sg_tablesize = min(sg_tablesize, sup_sg_tablesize);
}
/**
struct srp_fr_desc *d;
struct ib_mr *mr;
int i, ret = -EINVAL;
+ enum ib_mr_type mr_type;
if (pool_size <= 0)
goto err;
spin_lock_init(&pool->lock);
INIT_LIST_HEAD(&pool->free_list);
+ if (device->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG)
+ mr_type = IB_MR_TYPE_SG_GAPS;
+ else
+ mr_type = IB_MR_TYPE_MEM_REG;
+
for (i = 0, d = &pool->desc[0]; i < pool->size; i++, d++) {
- mr = ib_alloc_mr(pd, IB_MR_TYPE_MEM_REG,
- max_page_list_len);
+ mr = ib_alloc_mr(pd, mr_type, max_page_list_len);
if (IS_ERR(mr)) {
ret = PTR_ERR(mr);
if (ret == -ENOMEM)
indirect_sg_entries = cmd_sg_entries;
}
+ if (indirect_sg_entries > SG_MAX_SEGMENTS) {
+ pr_warn("Clamping indirect_sg_entries to %u\n",
+ SG_MAX_SEGMENTS);
+ indirect_sg_entries = SG_MAX_SEGMENTS;
+ }
+
srp_remove_wq = create_workqueue("srp_remove");
if (!srp_remove_wq) {
ret = -ENOMEM;
return -EINVAL;
}
- if (test_bit(ABS_MT_SLOT, dev->absbit)) {
- nslot = input_abs_get_max(dev, ABS_MT_SLOT) + 1;
- error = input_mt_init_slots(dev, nslot, 0);
- if (error)
+ if (test_bit(EV_ABS, dev->evbit)) {
+ input_alloc_absinfo(dev);
+ if (!dev->absinfo) {
+ error = -EINVAL;
goto fail1;
- } else if (test_bit(ABS_MT_POSITION_X, dev->absbit)) {
- input_set_events_per_packet(dev, 60);
+ }
+
+ if (test_bit(ABS_MT_SLOT, dev->absbit)) {
+ nslot = input_abs_get_max(dev, ABS_MT_SLOT) + 1;
+ error = input_mt_init_slots(dev, nslot, 0);
+ if (error)
+ goto fail1;
+ } else if (test_bit(ABS_MT_POSITION_X, dev->absbit)) {
+ input_set_events_per_packet(dev, 60);
+ }
}
if (test_bit(EV_FF, dev->evbit) && !udev->ff_effects_max) {
config RMI4_F03
bool "RMI4 Function 03 (PS2 Guest)"
depends on RMI4_CORE
- depends on SERIO=y || RMI4_CORE=SERIO
help
Say Y here if you want to add support for RMI4 function 03.
Function 03 provides PS2 guest support for RMI4 devices. This
includes support for TrackPoints on TouchPads.
+config RMI4_F03_SERIO
+ tristate
+ depends on RMI4_CORE
+ depends on RMI4_F03
+ default RMI4_CORE
+ select SERIO
+
config RMI4_2D_SENSOR
bool
depends on RMI4_CORE
data->enabled = true;
if (clear_wake && device_may_wakeup(rmi_dev->xport->dev)) {
retval = disable_irq_wake(irq);
- if (!retval)
+ if (retval)
dev_warn(&rmi_dev->dev,
"Failed to disable irq for wake: %d\n",
retval);
disable_irq(irq);
if (enable_wake && device_may_wakeup(rmi_dev->xport->dev)) {
retval = enable_irq_wake(irq);
- if (!retval)
+ if (retval)
dev_warn(&rmi_dev->dev,
"Failed to enable irq for wake: %d\n",
retval);
}
platform_set_drvdata(wm->battery_dev, wm);
wm->battery_dev->dev.parent = dev;
- wm->battery_dev->dev.platform_data = pdata->batt_pdata;
+ wm->battery_dev->dev.platform_data = pdata ? pdata->batt_pdata : NULL;
ret = platform_device_add(wm->battery_dev);
if (ret < 0)
goto batt_reg_err;
#include <linux/bitops.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/interrupt.h>
#include <linux/irqdomain.h>
#include <linux/irqchip.h>
-#include <linux/irqchip/chained_irq.h>
#include <linux/of.h>
#include <linux/of_platform.h>
#include <linux/mfd/syscon.h>
struct irq_domain *irqd;
struct regmap *devctrl_regs;
u32 devctrl_offset;
+ raw_spinlock_t wa_lock;
};
static inline u32 keystone_irq_readl(struct keystone_irq_device *kirq)
/* nothing to do here */
}
-static void keystone_irq_handler(struct irq_desc *desc)
+static irqreturn_t keystone_irq_handler(int irq, void *keystone_irq)
{
- unsigned int irq = irq_desc_get_irq(desc);
- struct keystone_irq_device *kirq = irq_desc_get_handler_data(desc);
+ struct keystone_irq_device *kirq = keystone_irq;
+ unsigned long wa_lock_flags;
unsigned long pending;
int src, virq;
dev_dbg(kirq->dev, "start irq %d\n", irq);
- chained_irq_enter(irq_desc_get_chip(desc), desc);
-
pending = keystone_irq_readl(kirq);
keystone_irq_writel(kirq, pending);
if (!virq)
dev_warn(kirq->dev, "spurious irq detected hwirq %d, virq %d\n",
src, virq);
+ raw_spin_lock_irqsave(&kirq->wa_lock, wa_lock_flags);
generic_handle_irq(virq);
+ raw_spin_unlock_irqrestore(&kirq->wa_lock,
+ wa_lock_flags);
}
}
- chained_irq_exit(irq_desc_get_chip(desc), desc);
-
dev_dbg(kirq->dev, "end irq %d\n", irq);
+ return IRQ_HANDLED;
}
static int keystone_irq_map(struct irq_domain *h, unsigned int virq,
return -ENODEV;
}
+ raw_spin_lock_init(&kirq->wa_lock);
+
platform_set_drvdata(pdev, kirq);
- irq_set_chained_handler_and_data(kirq->irq, keystone_irq_handler, kirq);
+ ret = request_irq(kirq->irq, keystone_irq_handler,
+ 0, dev_name(dev), kirq);
+ if (ret) {
+ irq_domain_remove(kirq->irqd);
+ return ret;
+ }
/* clear all source bits */
keystone_irq_writel(kirq, ~0x0);
struct keystone_irq_device *kirq = platform_get_drvdata(pdev);
int hwirq;
+ free_irq(kirq->irq, kirq);
+
for (hwirq = 0; hwirq < KEYSTONE_N_IRQ; hwirq++)
irq_dispose_mapping(irq_find_mapping(kirq->irqd, hwirq));
.irq_ack = icoll_ack_irq,
.irq_mask = icoll_mask_irq,
.irq_unmask = icoll_unmask_irq,
+ .flags = IRQCHIP_MASK_ON_SUSPEND |
+ IRQCHIP_SKIP_SET_WAKE,
};
static struct irq_chip asm9260_icoll_chip = {
.irq_ack = icoll_ack_irq,
.irq_mask = asm9260_mask_irq,
.irq_unmask = asm9260_unmask_irq,
+ .flags = IRQCHIP_MASK_ON_SUSPEND |
+ IRQCHIP_SKIP_SET_WAKE,
};
asmlinkage void __exception_irq_entry icoll_handle_irq(struct pt_regs *regs)
((CAPI_MSG *) msg)->header.ncci = 0;
((CAPI_MSG *) msg)->info.facility_req.Selector = SELECTOR_LINE_INTERCONNECT;
((CAPI_MSG *) msg)->info.facility_req.structs[0] = 3;
- PUT_WORD(&(((CAPI_MSG *) msg)->info.facility_req.structs[1]), LI_REQ_SILENT_UPDATE);
+ ((CAPI_MSG *) msg)->info.facility_req.structs[1] = LI_REQ_SILENT_UPDATE & 0xff;
+ ((CAPI_MSG *) msg)->info.facility_req.structs[2] = LI_REQ_SILENT_UPDATE >> 8;
((CAPI_MSG *) msg)->info.facility_req.structs[3] = 0;
w = api_put(notify_plci->appl, (CAPI_MSG *) msg);
if (w != _QUEUE_FULL)
return PTR_ERR(key);
}
- rcu_read_lock();
+ down_read(&key->sem);
ukp = user_key_payload(key);
if (!ukp) {
- rcu_read_unlock();
+ up_read(&key->sem);
key_put(key);
kzfree(new_key_string);
return -EKEYREVOKED;
}
if (cc->key_size != ukp->datalen) {
- rcu_read_unlock();
+ up_read(&key->sem);
key_put(key);
kzfree(new_key_string);
return -EINVAL;
memcpy(cc->key, ukp->data, cc->key_size);
- rcu_read_unlock();
+ up_read(&key->sem);
key_put(key);
/* clear the flag since following operations may invalidate previously valid key */
unsigned long flags;
struct priority_group *pg;
struct pgpath *pgpath;
- bool bypassed = true;
+ unsigned bypassed = 1;
if (!atomic_read(&m->nr_valid_paths)) {
clear_bit(MPATHF_QUEUE_IO, &m->flags);
*/
do {
list_for_each_entry(pg, &m->priority_groups, list) {
- if (pg->bypassed == bypassed)
+ if (pg->bypassed == !!bypassed)
continue;
pgpath = choose_path_in_pg(m, pg, nr_bytes);
if (!IS_ERR_OR_NULL(pgpath)) {
int srcu_idx;
struct dm_table *map = dm_get_live_table(md, &srcu_idx);
+ if (unlikely(!map)) {
+ dm_put_live_table(md, srcu_idx);
+ return;
+ }
ti = dm_table_find_target(map, pos);
dm_put_live_table(md, srcu_idx);
}
if (start_readonly && mddev->ro == 0)
mddev->ro = 2; /* read-only, but switch on first write */
+ /*
+ * NOTE: some pers->run(), for example r5l_recovery_log(), wakes
+ * up mddev->thread. It is important to initialize critical
+ * resources for mddev->thread BEFORE calling pers->run().
+ */
err = pers->run(mddev);
if (err)
pr_warn("md: pers->run() failed ...\n");
/* to submit async io_units, to fulfill ordering of flush */
struct work_struct deferred_io_work;
+ /* to disable write back during in degraded mode */
+ struct work_struct disable_writeback_work;
};
/*
r5l_do_submit_io(log, io);
}
+static void r5c_disable_writeback_async(struct work_struct *work)
+{
+ struct r5l_log *log = container_of(work, struct r5l_log,
+ disable_writeback_work);
+ struct mddev *mddev = log->rdev->mddev;
+
+ if (log->r5c_journal_mode == R5C_JOURNAL_MODE_WRITE_THROUGH)
+ return;
+ pr_info("md/raid:%s: Disabling writeback cache for degraded array.\n",
+ mdname(mddev));
+ mddev_suspend(mddev);
+ log->r5c_journal_mode = R5C_JOURNAL_MODE_WRITE_THROUGH;
+ mddev_resume(mddev);
+}
+
static void r5l_submit_current_io(struct r5l_log *log)
{
struct r5l_io_unit *io = log->current_io;
next_checkpoint = r5c_calculate_new_cp(conf);
spin_unlock_irq(&log->io_list_lock);
- BUG_ON(reclaimable < 0);
-
if (reclaimable == 0 || !write_super)
return;
r5c_recovery_rewrite_data_only_stripes(struct r5l_log *log,
struct r5l_recovery_ctx *ctx)
{
- struct stripe_head *sh, *next;
+ struct stripe_head *sh;
struct mddev *mddev = log->rdev->mddev;
struct page *page;
sector_t next_checkpoint = MaxSector;
WARN_ON(list_empty(&ctx->cached_list));
- list_for_each_entry_safe(sh, next, &ctx->cached_list, lru) {
+ list_for_each_entry(sh, &ctx->cached_list, lru) {
struct r5l_meta_block *mb;
int i;
int offset;
ctx->pos = write_pos;
ctx->seq += 1;
next_checkpoint = sh->log_start;
- list_del_init(&sh->lru);
- raid5_release_stripe(sh);
}
log->next_checkpoint = next_checkpoint;
__free_page(page);
return 0;
}
+static void r5c_recovery_flush_data_only_stripes(struct r5l_log *log,
+ struct r5l_recovery_ctx *ctx)
+{
+ struct mddev *mddev = log->rdev->mddev;
+ struct r5conf *conf = mddev->private;
+ struct stripe_head *sh, *next;
+
+ if (ctx->data_only_stripes == 0)
+ return;
+
+ log->r5c_journal_mode = R5C_JOURNAL_MODE_WRITE_BACK;
+
+ list_for_each_entry_safe(sh, next, &ctx->cached_list, lru) {
+ r5c_make_stripe_write_out(sh);
+ set_bit(STRIPE_HANDLE, &sh->state);
+ list_del_init(&sh->lru);
+ raid5_release_stripe(sh);
+ }
+
+ md_wakeup_thread(conf->mddev->thread);
+ /* reuse conf->wait_for_quiescent in recovery */
+ wait_event(conf->wait_for_quiescent,
+ atomic_read(&conf->active_stripes) == 0);
+
+ log->r5c_journal_mode = R5C_JOURNAL_MODE_WRITE_THROUGH;
+}
+
static int r5l_recovery_log(struct r5l_log *log)
{
struct mddev *mddev = log->rdev->mddev;
pos = ctx.pos;
ctx.seq += 10000;
- if (ctx.data_only_stripes == 0) {
- log->next_checkpoint = ctx.pos;
- r5l_log_write_empty_meta_block(log, ctx.pos, ctx.seq++);
- ctx.pos = r5l_ring_add(log, ctx.pos, BLOCK_SECTORS);
- }
if ((ctx.data_only_stripes == 0) && (ctx.data_parity_stripes == 0))
pr_debug("md/raid:%s: starting from clean shutdown\n",
mdname(mddev));
- else {
+ else
pr_debug("md/raid:%s: recovering %d data-only stripes and %d data-parity stripes\n",
mdname(mddev), ctx.data_only_stripes,
ctx.data_parity_stripes);
- if (ctx.data_only_stripes > 0)
- if (r5c_recovery_rewrite_data_only_stripes(log, &ctx)) {
- pr_err("md/raid:%s: failed to rewrite stripes to journal\n",
- mdname(mddev));
- return -EIO;
- }
+ if (ctx.data_only_stripes == 0) {
+ log->next_checkpoint = ctx.pos;
+ r5l_log_write_empty_meta_block(log, ctx.pos, ctx.seq++);
+ ctx.pos = r5l_ring_add(log, ctx.pos, BLOCK_SECTORS);
+ } else if (r5c_recovery_rewrite_data_only_stripes(log, &ctx)) {
+ pr_err("md/raid:%s: failed to rewrite stripes to journal\n",
+ mdname(mddev));
+ return -EIO;
}
log->log_start = ctx.pos;
log->seq = ctx.seq;
log->last_checkpoint = pos;
r5l_write_super(log, pos);
+
+ r5c_recovery_flush_data_only_stripes(log, &ctx);
return 0;
}
val > R5C_JOURNAL_MODE_WRITE_BACK)
return -EINVAL;
+ if (raid5_calc_degraded(conf) > 0 &&
+ val == R5C_JOURNAL_MODE_WRITE_BACK)
+ return -EINVAL;
+
mddev_suspend(mddev);
conf->log->r5c_journal_mode = val;
mddev_resume(mddev);
set_bit(STRIPE_R5C_CACHING, &sh->state);
}
+ /*
+ * When run in degraded mode, array is set to write-through mode.
+ * This check helps drain pending write safely in the transition to
+ * write-through mode.
+ */
+ if (s->failed) {
+ r5c_make_stripe_write_out(sh);
+ return -EAGAIN;
+ }
+
for (i = disks; i--; ) {
dev = &sh->dev[i];
/* if non-overwrite, use writing-out phase */
struct page *p = sh->dev[i].orig_page;
sh->dev[i].orig_page = sh->dev[i].page;
+ clear_bit(R5_OrigPageUPTDODATE, &sh->dev[i].flags);
+
if (!using_disk_info_extra_page)
put_page(p);
}
return ret;
}
+void r5c_update_on_rdev_error(struct mddev *mddev)
+{
+ struct r5conf *conf = mddev->private;
+ struct r5l_log *log = conf->log;
+
+ if (!log)
+ return;
+
+ if (raid5_calc_degraded(conf) > 0 &&
+ conf->log->r5c_journal_mode == R5C_JOURNAL_MODE_WRITE_BACK)
+ schedule_work(&log->disable_writeback_work);
+}
+
int r5l_init_log(struct r5conf *conf, struct md_rdev *rdev)
{
struct request_queue *q = bdev_get_queue(rdev->bdev);
spin_lock_init(&log->no_space_stripes_lock);
INIT_WORK(&log->deferred_io_work, r5l_submit_io_async);
+ INIT_WORK(&log->disable_writeback_work, r5c_disable_writeback_async);
log->r5c_journal_mode = R5C_JOURNAL_MODE_WRITE_THROUGH;
INIT_LIST_HEAD(&log->stripe_in_journal_list);
void r5l_exit_log(struct r5l_log *log)
{
+ flush_work(&log->disable_writeback_work);
md_unregister_thread(&log->reclaim_thread);
mempool_destroy(log->meta_pool);
bioset_free(log->bs);
* of the two sections, and some non-in_sync devices may
* be insync in the section most affected by failed devices.
*/
-static int calc_degraded(struct r5conf *conf)
+int raid5_calc_degraded(struct r5conf *conf)
{
int degraded, degraded2;
int i;
if (conf->mddev->reshape_position == MaxSector)
return conf->mddev->degraded > conf->max_degraded;
- degraded = calc_degraded(conf);
+ degraded = raid5_calc_degraded(conf);
if (degraded > conf->max_degraded)
return 1;
return 0;
if (test_bit(R5_SkipCopy, &sh->dev[i].flags))
WARN_ON(test_bit(R5_UPTODATE, &sh->dev[i].flags));
- sh->dev[i].vec.bv_page = sh->dev[i].page;
+
+ if (!op_is_write(op) &&
+ test_bit(R5_InJournal, &sh->dev[i].flags))
+ /*
+ * issuing read for a page in journal, this
+ * must be preparing for prexor in rmw; read
+ * the data into orig_page
+ */
+ sh->dev[i].vec.bv_page = sh->dev[i].orig_page;
+ else
+ sh->dev[i].vec.bv_page = sh->dev[i].page;
bi->bi_vcnt = 1;
bi->bi_io_vec[0].bv_len = STRIPE_SIZE;
bi->bi_io_vec[0].bv_offset = 0;
} else if (test_bit(R5_ReadNoMerge, &sh->dev[i].flags))
clear_bit(R5_ReadNoMerge, &sh->dev[i].flags);
+ if (test_bit(R5_InJournal, &sh->dev[i].flags))
+ /*
+ * end read for a page in journal, this
+ * must be preparing for prexor in rmw
+ */
+ set_bit(R5_OrigPageUPTDODATE, &sh->dev[i].flags);
+
if (atomic_read(&rdev->read_errors))
atomic_set(&rdev->read_errors, 0);
} else {
spin_lock_irqsave(&conf->device_lock, flags);
clear_bit(In_sync, &rdev->flags);
- mddev->degraded = calc_degraded(conf);
+ mddev->degraded = raid5_calc_degraded(conf);
spin_unlock_irqrestore(&conf->device_lock, flags);
set_bit(MD_RECOVERY_INTR, &mddev->recovery);
bdevname(rdev->bdev, b),
mdname(mddev),
conf->raid_disks - mddev->degraded);
+ r5c_update_on_rdev_error(mddev);
}
/*
return r_sector;
}
+/*
+ * There are cases where we want handle_stripe_dirtying() and
+ * schedule_reconstruction() to delay towrite to some dev of a stripe.
+ *
+ * This function checks whether we want to delay the towrite. Specifically,
+ * we delay the towrite when:
+ *
+ * 1. degraded stripe has a non-overwrite to the missing dev, AND this
+ * stripe has data in journal (for other devices).
+ *
+ * In this case, when reading data for the non-overwrite dev, it is
+ * necessary to handle complex rmw of write back cache (prexor with
+ * orig_page, and xor with page). To keep read path simple, we would
+ * like to flush data in journal to RAID disks first, so complex rmw
+ * is handled in the write patch (handle_stripe_dirtying).
+ *
+ */
+static inline bool delay_towrite(struct r5dev *dev,
+ struct stripe_head_state *s)
+{
+ return !test_bit(R5_OVERWRITE, &dev->flags) &&
+ !test_bit(R5_Insync, &dev->flags) && s->injournal;
+}
+
static void
schedule_reconstruction(struct stripe_head *sh, struct stripe_head_state *s,
int rcw, int expand)
for (i = disks; i--; ) {
struct r5dev *dev = &sh->dev[i];
- if (dev->towrite) {
+ if (dev->towrite && !delay_towrite(dev, s)) {
set_bit(R5_LOCKED, &dev->flags);
set_bit(R5_Wantdrain, &dev->flags);
if (!expand)
return rv;
}
-/* fetch_block - checks the given member device to see if its data needs
- * to be read or computed to satisfy a request.
- *
- * Returns 1 when no more member devices need to be checked, otherwise returns
- * 0 to tell the loop in handle_stripe_fill to continue
- */
-
static int need_this_block(struct stripe_head *sh, struct stripe_head_state *s,
int disk_idx, int disks)
{
return 0;
}
+/* fetch_block - checks the given member device to see if its data needs
+ * to be read or computed to satisfy a request.
+ *
+ * Returns 1 when no more member devices need to be checked, otherwise returns
+ * 0 to tell the loop in handle_stripe_fill to continue
+ */
static int fetch_block(struct stripe_head *sh, struct stripe_head_state *s,
int disk_idx, int disks)
{
* midst of changing due to a write
*/
if (!test_bit(STRIPE_COMPUTE_RUN, &sh->state) && !sh->check_state &&
- !sh->reconstruct_state)
+ !sh->reconstruct_state) {
+
+ /*
+ * For degraded stripe with data in journal, do not handle
+ * read requests yet, instead, flush the stripe to raid
+ * disks first, this avoids handling complex rmw of write
+ * back cache (prexor with orig_page, and then xor with
+ * page) in the read path
+ */
+ if (s->injournal && s->failed) {
+ if (test_bit(STRIPE_R5C_CACHING, &sh->state))
+ r5c_make_stripe_write_out(sh);
+ goto out;
+ }
+
for (i = disks; i--; )
if (fetch_block(sh, s, i, disks))
break;
+ }
+out:
set_bit(STRIPE_HANDLE, &sh->state);
}
break_stripe_batch_list(head_sh, STRIPE_EXPAND_SYNC_FLAGS);
}
+/*
+ * For RMW in write back cache, we need extra page in prexor to store the
+ * old data. This page is stored in dev->orig_page.
+ *
+ * This function checks whether we have data for prexor. The exact logic
+ * is:
+ * R5_UPTODATE && (!R5_InJournal || R5_OrigPageUPTDODATE)
+ */
+static inline bool uptodate_for_rmw(struct r5dev *dev)
+{
+ return (test_bit(R5_UPTODATE, &dev->flags)) &&
+ (!test_bit(R5_InJournal, &dev->flags) ||
+ test_bit(R5_OrigPageUPTDODATE, &dev->flags));
+}
+
static int handle_stripe_dirtying(struct r5conf *conf,
struct stripe_head *sh,
struct stripe_head_state *s,
} else for (i = disks; i--; ) {
/* would I have to read this buffer for read_modify_write */
struct r5dev *dev = &sh->dev[i];
- if ((dev->towrite || i == sh->pd_idx || i == sh->qd_idx ||
+ if (((dev->towrite && !delay_towrite(dev, s)) ||
+ i == sh->pd_idx || i == sh->qd_idx ||
test_bit(R5_InJournal, &dev->flags)) &&
!test_bit(R5_LOCKED, &dev->flags) &&
- !((test_bit(R5_UPTODATE, &dev->flags) &&
- (!test_bit(R5_InJournal, &dev->flags) ||
- dev->page != dev->orig_page)) ||
+ !(uptodate_for_rmw(dev) ||
test_bit(R5_Wantcompute, &dev->flags))) {
if (test_bit(R5_Insync, &dev->flags))
rmw++;
i != sh->pd_idx && i != sh->qd_idx &&
!test_bit(R5_LOCKED, &dev->flags) &&
!(test_bit(R5_UPTODATE, &dev->flags) ||
- test_bit(R5_InJournal, &dev->flags) ||
test_bit(R5_Wantcompute, &dev->flags))) {
if (test_bit(R5_Insync, &dev->flags))
rcw++;
for (i = disks; i--; ) {
struct r5dev *dev = &sh->dev[i];
- if ((dev->towrite ||
+ if (((dev->towrite && !delay_towrite(dev, s)) ||
i == sh->pd_idx || i == sh->qd_idx ||
test_bit(R5_InJournal, &dev->flags)) &&
!test_bit(R5_LOCKED, &dev->flags) &&
- !((test_bit(R5_UPTODATE, &dev->flags) &&
- (!test_bit(R5_InJournal, &dev->flags) ||
- dev->page != dev->orig_page)) ||
+ !(uptodate_for_rmw(dev) ||
test_bit(R5_Wantcompute, &dev->flags)) &&
test_bit(R5_Insync, &dev->flags)) {
if (test_bit(STRIPE_PREREAD_ACTIVE,
i != sh->pd_idx && i != sh->qd_idx &&
!test_bit(R5_LOCKED, &dev->flags) &&
!(test_bit(R5_UPTODATE, &dev->flags) ||
- test_bit(R5_InJournal, &dev->flags) ||
test_bit(R5_Wantcompute, &dev->flags))) {
rcw++;
if (test_bit(R5_Insync, &dev->flags) &&
/*
* 0 for a fully functional array, 1 or 2 for a degraded array.
*/
- mddev->degraded = calc_degraded(conf);
+ mddev->degraded = raid5_calc_degraded(conf);
if (has_failed(conf)) {
pr_crit("md/raid:%s: not enough operational devices (%d/%d failed)\n",
}
}
spin_lock_irqsave(&conf->device_lock, flags);
- mddev->degraded = calc_degraded(conf);
+ mddev->degraded = raid5_calc_degraded(conf);
spin_unlock_irqrestore(&conf->device_lock, flags);
print_raid5_conf(conf);
return count;
* pre and post number of devices.
*/
spin_lock_irqsave(&conf->device_lock, flags);
- mddev->degraded = calc_degraded(conf);
+ mddev->degraded = raid5_calc_degraded(conf);
spin_unlock_irqrestore(&conf->device_lock, flags);
}
mddev->raid_disks = conf->raid_disks;
} else {
int d;
spin_lock_irq(&conf->device_lock);
- mddev->degraded = calc_degraded(conf);
+ mddev->degraded = raid5_calc_degraded(conf);
spin_unlock_irq(&conf->device_lock);
for (d = conf->raid_disks ;
d < conf->raid_disks - mddev->delta_disks;
* data and parity being written are in the journal
* device
*/
+ R5_OrigPageUPTDODATE, /* with write back cache, we read old data into
+ * dev->orig_page for prexor. When this flag is
+ * set, orig_page contains latest data in the
+ * raid disk.
+ */
};
/*
extern struct stripe_head *
raid5_get_active_stripe(struct r5conf *conf, sector_t sector,
int previous, int noblock, int noquiesce);
+extern int raid5_calc_degraded(struct r5conf *conf);
extern int r5l_init_log(struct r5conf *conf, struct md_rdev *rdev);
extern void r5l_exit_log(struct r5l_log *log);
extern int r5l_write_stripe(struct r5l_log *log, struct stripe_head *head_sh);
extern void r5c_check_stripe_cache_usage(struct r5conf *conf);
extern void r5c_check_cached_full_stripe(struct r5conf *conf);
extern struct md_sysfs_entry r5c_journal_mode;
+extern void r5c_update_on_rdev_error(struct mddev *mddev);
#endif
#include "cec-priv.h"
-static int cec_report_features(struct cec_adapter *adap, unsigned int la_idx);
-static int cec_report_phys_addr(struct cec_adapter *adap, unsigned int la_idx);
+static void cec_fill_msg_report_features(struct cec_adapter *adap,
+ struct cec_msg *msg,
+ unsigned int la_idx);
/*
* 400 ms is the time it takes for one 16 byte message to be
/* Mark it as an error */
data->msg.tx_ts = ktime_get_ns();
- data->msg.tx_status = CEC_TX_STATUS_ERROR |
- CEC_TX_STATUS_MAX_RETRIES;
+ data->msg.tx_status |= CEC_TX_STATUS_ERROR |
+ CEC_TX_STATUS_MAX_RETRIES;
+ data->msg.tx_error_cnt++;
data->attempts = 0;
- data->msg.tx_error_cnt = 1;
/* Queue transmitted message for monitoring purposes */
cec_queue_msg_monitor(data->adap, &data->msg, 1);
[CEC_MSG_REQUEST_ARC_TERMINATION] = 2 | DIRECTED,
[CEC_MSG_TERMINATE_ARC] = 2 | DIRECTED,
[CEC_MSG_REQUEST_CURRENT_LATENCY] = 4 | BCAST,
- [CEC_MSG_REPORT_CURRENT_LATENCY] = 7 | BCAST,
+ [CEC_MSG_REPORT_CURRENT_LATENCY] = 6 | BCAST,
[CEC_MSG_CDC_MESSAGE] = 2 | BCAST,
};
las->log_addr[i] = CEC_LOG_ADDR_INVALID;
if (last_la == CEC_LOG_ADDR_INVALID ||
last_la == CEC_LOG_ADDR_UNREGISTERED ||
- !(last_la & type2mask[type]))
+ !((1 << last_la) & type2mask[type]))
last_la = la_list[0];
err = cec_config_log_addr(adap, i, last_la);
for (i = 1; i < las->num_log_addrs; i++)
las->log_addr[i] = CEC_LOG_ADDR_INVALID;
}
+ for (i = las->num_log_addrs; i < CEC_MAX_LOG_ADDRS; i++)
+ las->log_addr[i] = CEC_LOG_ADDR_INVALID;
adap->is_configured = true;
adap->is_configuring = false;
cec_post_state_event(adap);
- mutex_unlock(&adap->lock);
+ /*
+ * Now post the Report Features and Report Physical Address broadcast
+ * messages. Note that these are non-blocking transmits, meaning that
+ * they are just queued up and once adap->lock is unlocked the main
+ * thread will kick in and start transmitting these.
+ *
+ * If after this function is done (but before one or more of these
+ * messages are actually transmitted) the CEC adapter is unconfigured,
+ * then any remaining messages will be dropped by the main thread.
+ */
for (i = 0; i < las->num_log_addrs; i++) {
+ struct cec_msg msg = {};
+
if (las->log_addr[i] == CEC_LOG_ADDR_INVALID ||
(las->flags & CEC_LOG_ADDRS_FL_CDC_ONLY))
continue;
- /*
- * Report Features must come first according
- * to CEC 2.0
- */
- if (las->log_addr[i] != CEC_LOG_ADDR_UNREGISTERED)
- cec_report_features(adap, i);
- cec_report_phys_addr(adap, i);
+ msg.msg[0] = (las->log_addr[i] << 4) | 0x0f;
+
+ /* Report Features must come first according to CEC 2.0 */
+ if (las->log_addr[i] != CEC_LOG_ADDR_UNREGISTERED &&
+ adap->log_addrs.cec_version >= CEC_OP_CEC_VERSION_2_0) {
+ cec_fill_msg_report_features(adap, &msg, i);
+ cec_transmit_msg_fh(adap, &msg, NULL, false);
+ }
+
+ /* Report Physical Address */
+ cec_msg_report_physical_addr(&msg, adap->phys_addr,
+ las->primary_device_type[i]);
+ dprintk(2, "config: la %d pa %x.%x.%x.%x\n",
+ las->log_addr[i],
+ cec_phys_addr_exp(adap->phys_addr));
+ cec_transmit_msg_fh(adap, &msg, NULL, false);
}
- for (i = las->num_log_addrs; i < CEC_MAX_LOG_ADDRS; i++)
- las->log_addr[i] = CEC_LOG_ADDR_INVALID;
- mutex_lock(&adap->lock);
adap->kthread_config = NULL;
- mutex_unlock(&adap->lock);
complete(&adap->config_completion);
+ mutex_unlock(&adap->lock);
return 0;
unconfigure:
/* High-level core CEC message handling */
-/* Transmit the Report Features message */
-static int cec_report_features(struct cec_adapter *adap, unsigned int la_idx)
+/* Fill in the Report Features message */
+static void cec_fill_msg_report_features(struct cec_adapter *adap,
+ struct cec_msg *msg,
+ unsigned int la_idx)
{
- struct cec_msg msg = { };
const struct cec_log_addrs *las = &adap->log_addrs;
const u8 *features = las->features[la_idx];
bool op_is_dev_features = false;
unsigned int idx;
- /* This is 2.0 and up only */
- if (adap->log_addrs.cec_version < CEC_OP_CEC_VERSION_2_0)
- return 0;
-
/* Report Features */
- msg.msg[0] = (las->log_addr[la_idx] << 4) | 0x0f;
- msg.len = 4;
- msg.msg[1] = CEC_MSG_REPORT_FEATURES;
- msg.msg[2] = adap->log_addrs.cec_version;
- msg.msg[3] = las->all_device_types[la_idx];
+ msg->msg[0] = (las->log_addr[la_idx] << 4) | 0x0f;
+ msg->len = 4;
+ msg->msg[1] = CEC_MSG_REPORT_FEATURES;
+ msg->msg[2] = adap->log_addrs.cec_version;
+ msg->msg[3] = las->all_device_types[la_idx];
/* Write RC Profiles first, then Device Features */
for (idx = 0; idx < ARRAY_SIZE(las->features[0]); idx++) {
- msg.msg[msg.len++] = features[idx];
+ msg->msg[msg->len++] = features[idx];
if ((features[idx] & CEC_OP_FEAT_EXT) == 0) {
if (op_is_dev_features)
break;
op_is_dev_features = true;
}
}
- return cec_transmit_msg(adap, &msg, false);
-}
-
-/* Transmit the Report Physical Address message */
-static int cec_report_phys_addr(struct cec_adapter *adap, unsigned int la_idx)
-{
- const struct cec_log_addrs *las = &adap->log_addrs;
- struct cec_msg msg = { };
-
- /* Report Physical Address */
- msg.msg[0] = (las->log_addr[la_idx] << 4) | 0x0f;
- cec_msg_report_physical_addr(&msg, adap->phys_addr,
- las->primary_device_type[la_idx]);
- dprintk(2, "config: la %d pa %x.%x.%x.%x\n",
- las->log_addr[la_idx],
- cec_phys_addr_exp(adap->phys_addr));
- return cec_transmit_msg(adap, &msg, false);
}
/* Transmit the Feature Abort message */
}
case CEC_MSG_GIVE_FEATURES:
- if (adap->log_addrs.cec_version >= CEC_OP_CEC_VERSION_2_0)
- return cec_report_features(adap, la_idx);
- return 0;
+ if (adap->log_addrs.cec_version < CEC_OP_CEC_VERSION_2_0)
+ return cec_feature_abort(adap, msg);
+ cec_fill_msg_report_features(adap, &tx_cec_msg, la_idx);
+ return cec_transmit_msg(adap, &tx_cec_msg, false);
default:
/*
skb_copy_from_linear_data(h->priv->ule_skb, dest_addr,
ETH_ALEN);
skb_pull(h->priv->ule_skb, ETH_ALEN);
+ } else {
+ /* dest_addr buffer is only valid if h->priv->ule_dbit == 0 */
+ eth_zero_addr(dest_addr);
}
/* Handle ULE Extension Headers. */
if (!h->priv->ule_bridged) {
skb_push(h->priv->ule_skb, ETH_HLEN);
h->ethh = (struct ethhdr *)h->priv->ule_skb->data;
- if (!h->priv->ule_dbit) {
- /*
- * dest_addr buffer is only valid if
- * h->priv->ule_dbit == 0
- */
- memcpy(h->ethh->h_dest, dest_addr, ETH_ALEN);
- eth_zero_addr(h->ethh->h_source);
- } else /* zeroize source and dest */
- memset(h->ethh, 0, ETH_ALEN * 2);
-
+ memcpy(h->ethh->h_dest, dest_addr, ETH_ALEN);
+ eth_zero_addr(h->ethh->h_source);
h->ethh->h_proto = htons(h->priv->ule_sndu_type);
}
/* else: skb is in correct state; nothing to do. */
config VIDEO_S5K4ECGX
tristate "Samsung S5K4ECGX sensor support"
depends on I2C && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+ select CRC32
---help---
This is a V4L2 sensor-level driver for Samsung S5K4ECGX 5M
camera sensor with an embedded SoC image signal processor.
* I2C Driver
*/
-#ifdef CONFIG_PM
-
-static int smiapp_suspend(struct device *dev)
+static int __maybe_unused smiapp_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
struct v4l2_subdev *subdev = i2c_get_clientdata(client);
return 0;
}
-static int smiapp_resume(struct device *dev)
+static int __maybe_unused smiapp_resume(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
struct v4l2_subdev *subdev = i2c_get_clientdata(client);
return rval;
}
-#else
-
-#define smiapp_suspend NULL
-#define smiapp_resume NULL
-
-#endif /* CONFIG_PM */
-
static struct smiapp_hwconfig *smiapp_get_hwconfig(struct device *dev)
{
struct smiapp_hwconfig *hwcfg;
if (IS_ERR(sensor->xshutdown))
return PTR_ERR(sensor->xshutdown);
- pm_runtime_enable(&client->dev);
-
- rval = pm_runtime_get_sync(&client->dev);
- if (rval < 0) {
- rval = -ENODEV;
- goto out_power_off;
- }
+ rval = smiapp_power_on(&client->dev);
+ if (rval < 0)
+ return rval;
rval = smiapp_identify_module(sensor);
if (rval) {
if (rval < 0)
goto out_media_entity_cleanup;
+ pm_runtime_set_active(&client->dev);
+ pm_runtime_get_noresume(&client->dev);
+ pm_runtime_enable(&client->dev);
pm_runtime_set_autosuspend_delay(&client->dev, 1000);
pm_runtime_use_autosuspend(&client->dev);
pm_runtime_put_autosuspend(&client->dev);
smiapp_cleanup(sensor);
out_power_off:
- pm_runtime_put(&client->dev);
- pm_runtime_disable(&client->dev);
+ smiapp_power_off(&client->dev);
return rval;
}
v4l2_async_unregister_subdev(subdev);
- pm_runtime_suspend(&client->dev);
pm_runtime_disable(&client->dev);
+ if (!pm_runtime_status_suspended(&client->dev))
+ smiapp_power_off(&client->dev);
+ pm_runtime_set_suspended(&client->dev);
for (i = 0; i < sensor->ssds_used; i++) {
v4l2_device_unregister_subdev(&sensor->ssds[i].sd);
tvp5150_write(sd, TVP5150_OP_MODE_CTL, opmode);
tvp5150_write(sd, TVP5150_VD_IN_SRC_SEL_1, input);
- /* Svideo should enable YCrCb output and disable GPCL output
- * For Composite and TV, it should be the reverse
+ /*
+ * Setup the FID/GLCO/VLK/HVLK and INTREQ/GPCL/VBLK output signals. For
+ * S-Video we output the vertical lock (VLK) signal on FID/GLCO/VLK/HVLK
+ * and set INTREQ/GPCL/VBLK to logic 0. For composite we output the
+ * field indicator (FID) signal on FID/GLCO/VLK/HVLK and set
+ * INTREQ/GPCL/VBLK to logic 1.
*/
val = tvp5150_read(sd, TVP5150_MISC_CTL);
if (val < 0) {
}
if (decoder->input == TVP5150_SVIDEO)
- val = (val & ~0x40) | 0x10;
+ val = (val & ~TVP5150_MISC_CTL_GPCL) | TVP5150_MISC_CTL_HVLK;
else
- val = (val & ~0x10) | 0x40;
+ val = (val & ~TVP5150_MISC_CTL_HVLK) | TVP5150_MISC_CTL_GPCL;
tvp5150_write(sd, TVP5150_MISC_CTL, val);
};
},{ /* Automatic offset and AGC enabled */
TVP5150_ANAL_CHL_CTL, 0x15
},{ /* Activate YCrCb output 0x9 or 0xd ? */
- TVP5150_MISC_CTL, 0x6f
+ TVP5150_MISC_CTL, TVP5150_MISC_CTL_GPCL |
+ TVP5150_MISC_CTL_INTREQ_OE |
+ TVP5150_MISC_CTL_YCBCR_OE |
+ TVP5150_MISC_CTL_SYNC_OE |
+ TVP5150_MISC_CTL_VBLANK |
+ TVP5150_MISC_CTL_CLOCK_OE,
},{ /* Activates video std autodetection for all standards */
TVP5150_AUTOSW_MSK, 0x0
},{ /* Default format: 0x47. For 4:2:2: 0x40 */
f = &format->format;
- tvp5150_reset(sd, 0);
-
f->width = decoder->rect.width;
f->height = decoder->rect.height / 2;
static int tvp5150_s_stream(struct v4l2_subdev *sd, int enable)
{
struct tvp5150 *decoder = to_tvp5150(sd);
- /* Output format: 8-bit ITU-R BT.656 with embedded syncs */
- int val = 0x09;
-
- /* Output format: 8-bit 4:2:2 YUV with discrete sync */
- if (decoder->mbus_type == V4L2_MBUS_PARALLEL)
- val = 0x0d;
+ int val;
- /* Initializes TVP5150 to its default values */
- /* # set PCLK (27MHz) */
- tvp5150_write(sd, TVP5150_CONF_SHARED_PIN, 0x00);
+ /* Enable or disable the video output signals. */
+ val = tvp5150_read(sd, TVP5150_MISC_CTL);
+ if (val < 0)
+ return val;
+
+ val &= ~(TVP5150_MISC_CTL_YCBCR_OE | TVP5150_MISC_CTL_SYNC_OE |
+ TVP5150_MISC_CTL_CLOCK_OE);
+
+ if (enable) {
+ /*
+ * Enable the YCbCr and clock outputs. In discrete sync mode
+ * (non-BT.656) additionally enable the the sync outputs.
+ */
+ val |= TVP5150_MISC_CTL_YCBCR_OE | TVP5150_MISC_CTL_CLOCK_OE;
+ if (decoder->mbus_type == V4L2_MBUS_PARALLEL)
+ val |= TVP5150_MISC_CTL_SYNC_OE;
+ }
- if (enable)
- tvp5150_write(sd, TVP5150_MISC_CTL, val);
- else
- tvp5150_write(sd, TVP5150_MISC_CTL, 0x00);
+ tvp5150_write(sd, TVP5150_MISC_CTL, val);
return 0;
}
res = core->hdl.error;
goto err;
}
- v4l2_ctrl_handler_setup(&core->hdl);
/* Default is no cropping */
core->rect.top = 0;
core->rect.left = 0;
core->rect.width = TVP5150_H_MAX;
+ tvp5150_reset(sd, 0); /* Calls v4l2_ctrl_handler_setup() */
+
res = v4l2_async_register_subdev(sd);
if (res < 0)
goto err;
#define TVP5150_ANAL_CHL_CTL 0x01 /* Analog channel controls */
#define TVP5150_OP_MODE_CTL 0x02 /* Operation mode controls */
#define TVP5150_MISC_CTL 0x03 /* Miscellaneous controls */
+#define TVP5150_MISC_CTL_VBLK_GPCL BIT(7)
+#define TVP5150_MISC_CTL_GPCL BIT(6)
+#define TVP5150_MISC_CTL_INTREQ_OE BIT(5)
+#define TVP5150_MISC_CTL_HVLK BIT(4)
+#define TVP5150_MISC_CTL_YCBCR_OE BIT(3)
+#define TVP5150_MISC_CTL_SYNC_OE BIT(2)
+#define TVP5150_MISC_CTL_VBLANK BIT(1)
+#define TVP5150_MISC_CTL_CLOCK_OE BIT(0)
+
#define TVP5150_AUTOSW_MSK 0x04 /* Autoswitch mask: TVP5150A / TVP5150AM */
/* Reserved 05h */
static void cobalt_free_msi(struct cobalt *cobalt, struct pci_dev *pci_dev)
{
free_irq(pci_dev->irq, (void *)cobalt);
-
- if (cobalt->msi_enabled)
- pci_disable_msi(pci_dev);
+ pci_free_irq_vectors(pci_dev);
}
static int cobalt_setup_pci(struct cobalt *cobalt, struct pci_dev *pci_dev,
from being generated. */
cobalt_set_interrupt(cobalt, false);
- if (pci_enable_msi_range(pci_dev, 1, 1) < 1) {
+ if (pci_alloc_irq_vectors(pci_dev, 1, 1, PCI_IRQ_MSI) < 1) {
cobalt_err("Could not enable MSI\n");
- cobalt->msi_enabled = false;
ret = -EIO;
goto err_release;
}
msi_config_show(cobalt, pci_dev);
- cobalt->msi_enabled = true;
/* Register IRQ */
if (request_irq(pci_dev->irq, cobalt_irq_handler, IRQF_SHARED,
u32 irq_none;
u32 irq_full_fifo;
- bool msi_enabled;
-
/* omnitek dma */
int dma_channels;
int first_fifo_channel;
u8 c; /* transaction counter, wraps around... */
u8 initialized; /* set to 1 if 0x15 has been sent */
u16 last_rc_key;
-
- unsigned char data[80];
};
static int tt3650_ci_msg(struct dvb_usb_device *d, u8 cmd, u8 *data,
unsigned int write_len, unsigned int read_len)
{
struct pctv452e_state *state = (struct pctv452e_state *)d->priv;
+ u8 *buf;
u8 id;
unsigned int rlen;
int ret;
return -EIO;
}
- mutex_lock(&state->ca_mutex);
+ buf = kmalloc(64, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
id = state->c++;
- state->data[0] = SYNC_BYTE_OUT;
- state->data[1] = id;
- state->data[2] = cmd;
- state->data[3] = write_len;
+ buf[0] = SYNC_BYTE_OUT;
+ buf[1] = id;
+ buf[2] = cmd;
+ buf[3] = write_len;
- memcpy(state->data + 4, data, write_len);
+ memcpy(buf + 4, data, write_len);
rlen = (read_len > 0) ? 64 : 0;
- ret = dvb_usb_generic_rw(d, state->data, 4 + write_len,
- state->data, rlen, /* delay_ms */ 0);
+ ret = dvb_usb_generic_rw(d, buf, 4 + write_len,
+ buf, rlen, /* delay_ms */ 0);
if (0 != ret)
goto failed;
ret = -EIO;
- if (SYNC_BYTE_IN != state->data[0] || id != state->data[1])
+ if (SYNC_BYTE_IN != buf[0] || id != buf[1])
goto failed;
- memcpy(data, state->data + 4, read_len);
+ memcpy(data, buf + 4, read_len);
- mutex_unlock(&state->ca_mutex);
+ kfree(buf);
return 0;
failed:
err("CI error %d; %02X %02X %02X -> %*ph.",
- ret, SYNC_BYTE_OUT, id, cmd, 3, state->data);
+ ret, SYNC_BYTE_OUT, id, cmd, 3, buf);
- mutex_unlock(&state->ca_mutex);
+ kfree(buf);
return ret;
}
u8 *rcv_buf, u8 rcv_len)
{
struct pctv452e_state *state = (struct pctv452e_state *)d->priv;
+ u8 *buf;
u8 id;
int ret;
- mutex_lock(&state->ca_mutex);
+ buf = kmalloc(64, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
id = state->c++;
ret = -EINVAL;
if (snd_len > 64 - 7 || rcv_len > 64 - 7)
goto failed;
- state->data[0] = SYNC_BYTE_OUT;
- state->data[1] = id;
- state->data[2] = PCTV_CMD_I2C;
- state->data[3] = snd_len + 3;
- state->data[4] = addr << 1;
- state->data[5] = snd_len;
- state->data[6] = rcv_len;
+ buf[0] = SYNC_BYTE_OUT;
+ buf[1] = id;
+ buf[2] = PCTV_CMD_I2C;
+ buf[3] = snd_len + 3;
+ buf[4] = addr << 1;
+ buf[5] = snd_len;
+ buf[6] = rcv_len;
- memcpy(state->data + 7, snd_buf, snd_len);
+ memcpy(buf + 7, snd_buf, snd_len);
- ret = dvb_usb_generic_rw(d, state->data, 7 + snd_len,
- state->data, /* rcv_len */ 64,
+ ret = dvb_usb_generic_rw(d, buf, 7 + snd_len,
+ buf, /* rcv_len */ 64,
/* delay_ms */ 0);
if (ret < 0)
goto failed;
/* TT USB protocol error. */
ret = -EIO;
- if (SYNC_BYTE_IN != state->data[0] || id != state->data[1])
+ if (SYNC_BYTE_IN != buf[0] || id != buf[1])
goto failed;
/* I2C device didn't respond as expected. */
ret = -EREMOTEIO;
- if (state->data[5] < snd_len || state->data[6] < rcv_len)
+ if (buf[5] < snd_len || buf[6] < rcv_len)
goto failed;
- memcpy(rcv_buf, state->data + 7, rcv_len);
- mutex_unlock(&state->ca_mutex);
+ memcpy(rcv_buf, buf + 7, rcv_len);
+ kfree(buf);
return rcv_len;
failed:
err("I2C error %d; %02X %02X %02X %02X %02X -> %*ph",
ret, SYNC_BYTE_OUT, id, addr << 1, snd_len, rcv_len,
- 7, state->data);
+ 7, buf);
- mutex_unlock(&state->ca_mutex);
+ kfree(buf);
return ret;
}
static int pctv452e_power_ctrl(struct dvb_usb_device *d, int i)
{
struct pctv452e_state *state = (struct pctv452e_state *)d->priv;
- u8 *rx;
+ u8 *b0, *rx;
int ret;
info("%s: %d\n", __func__, i);
if (state->initialized)
return 0;
- rx = kmalloc(PCTV_ANSWER_LEN, GFP_KERNEL);
- if (!rx)
+ b0 = kmalloc(5 + PCTV_ANSWER_LEN, GFP_KERNEL);
+ if (!b0)
return -ENOMEM;
- mutex_lock(&state->ca_mutex);
+ rx = b0 + 5;
+
/* hmm where shoud this should go? */
ret = usb_set_interface(d->udev, 0, ISOC_INTERFACE_ALTERNATIVE);
if (ret != 0)
__func__, ret);
/* this is a one-time initialization, dont know where to put */
- state->data[0] = 0xaa;
- state->data[1] = state->c++;
- state->data[2] = PCTV_CMD_RESET;
- state->data[3] = 1;
- state->data[4] = 0;
+ b0[0] = 0xaa;
+ b0[1] = state->c++;
+ b0[2] = PCTV_CMD_RESET;
+ b0[3] = 1;
+ b0[4] = 0;
/* reset board */
- ret = dvb_usb_generic_rw(d, state->data, 5, rx, PCTV_ANSWER_LEN, 0);
+ ret = dvb_usb_generic_rw(d, b0, 5, rx, PCTV_ANSWER_LEN, 0);
if (ret)
goto ret;
- state->data[1] = state->c++;
- state->data[4] = 1;
+ b0[1] = state->c++;
+ b0[4] = 1;
/* reset board (again?) */
- ret = dvb_usb_generic_rw(d, state->data, 5, rx, PCTV_ANSWER_LEN, 0);
+ ret = dvb_usb_generic_rw(d, b0, 5, rx, PCTV_ANSWER_LEN, 0);
if (ret)
goto ret;
state->initialized = 1;
ret:
- mutex_unlock(&state->ca_mutex);
- kfree(rx);
+ kfree(b0);
return ret;
}
static int pctv452e_rc_query(struct dvb_usb_device *d)
{
struct pctv452e_state *state = (struct pctv452e_state *)d->priv;
+ u8 *b, *rx;
int ret, i;
u8 id;
- mutex_lock(&state->ca_mutex);
+ b = kmalloc(CMD_BUFFER_SIZE + PCTV_ANSWER_LEN, GFP_KERNEL);
+ if (!b)
+ return -ENOMEM;
+
+ rx = b + CMD_BUFFER_SIZE;
+
id = state->c++;
/* prepare command header */
- state->data[0] = SYNC_BYTE_OUT;
- state->data[1] = id;
- state->data[2] = PCTV_CMD_IR;
- state->data[3] = 0;
+ b[0] = SYNC_BYTE_OUT;
+ b[1] = id;
+ b[2] = PCTV_CMD_IR;
+ b[3] = 0;
/* send ir request */
- ret = dvb_usb_generic_rw(d, state->data, 4,
- state->data, PCTV_ANSWER_LEN, 0);
+ ret = dvb_usb_generic_rw(d, b, 4, rx, PCTV_ANSWER_LEN, 0);
if (ret != 0)
goto ret;
if (debug > 3) {
- info("%s: read: %2d: %*ph: ", __func__, ret, 3, state->data);
- for (i = 0; (i < state->data[3]) && ((i + 3) < PCTV_ANSWER_LEN); i++)
- info(" %02x", state->data[i + 3]);
+ info("%s: read: %2d: %*ph: ", __func__, ret, 3, rx);
+ for (i = 0; (i < rx[3]) && ((i+3) < PCTV_ANSWER_LEN); i++)
+ info(" %02x", rx[i+3]);
info("\n");
}
- if ((state->data[3] == 9) && (state->data[12] & 0x01)) {
+ if ((rx[3] == 9) && (rx[12] & 0x01)) {
/* got a "press" event */
- state->last_rc_key = RC_SCANCODE_RC5(state->data[7], state->data[6]);
+ state->last_rc_key = RC_SCANCODE_RC5(rx[7], rx[6]);
if (debug > 2)
info("%s: cmd=0x%02x sys=0x%02x\n",
- __func__, state->data[6], state->data[7]);
+ __func__, rx[6], rx[7]);
rc_keydown(d->rc_dev, RC_TYPE_RC5, state->last_rc_key, 0);
} else if (state->last_rc_key) {
state->last_rc_key = 0;
}
ret:
- mutex_unlock(&state->ca_mutex);
+ kfree(b);
return ret;
}
struct ms_id_register id_reg;
if (!(*mrq)) {
- memstick_init_req(&card->current_mrq, MS_TPC_READ_REG, NULL,
+ memstick_init_req(&card->current_mrq, MS_TPC_READ_REG, &id_reg,
sizeof(struct ms_id_register));
*mrq = &card->current_mrq;
return 0;
if (!slot)
continue;
- if (slot->mmc->pm_flags & MMC_PM_KEEP_POWER) {
+ if (slot->mmc->pm_flags & MMC_PM_KEEP_POWER)
dw_mci_set_ios(slot->mmc, &slot->mmc->ios);
- dw_mci_setup_bus(slot, true);
- }
+
+ /* Force setup bus to guarantee available clock output */
+ dw_mci_setup_bus(slot, true);
}
/* Now that slots are all setup, we can enable card detect */
if (!host->busy_status && busy_resp &&
!(status & (MCI_CMDCRCFAIL|MCI_CMDTIMEOUT)) &&
(readl(base + MMCISTATUS) & host->variant->busy_detect_flag)) {
- /* Unmask the busy IRQ */
+
+ /* Clear the busy start IRQ */
+ writel(host->variant->busy_detect_mask,
+ host->base + MMCICLEAR);
+
+ /* Unmask the busy end IRQ */
writel(readl(base + MMCIMASK0) |
host->variant->busy_detect_mask,
base + MMCIMASK0);
/*
* At this point we are not busy with a command, we have
- * not received a new busy request, mask the busy IRQ and
- * fall through to process the IRQ.
+ * not received a new busy request, clear and mask the busy
+ * end IRQ and fall through to process the IRQ.
*/
if (host->busy_status) {
+
+ writel(host->variant->busy_detect_mask,
+ host->base + MMCICLEAR);
+
writel(readl(base + MMCIMASK0) &
~host->variant->busy_detect_mask,
base + MMCIMASK0);
}
/*
- * We intentionally clear the MCI_ST_CARDBUSY IRQ here (if it's
- * enabled) since the HW seems to be triggering the IRQ on both
- * edges while monitoring DAT0 for busy completion.
+ * We intentionally clear the MCI_ST_CARDBUSY IRQ (if it's
+ * enabled) in mmci_cmd_irq() function where ST Micro busy
+ * detection variant is handled. Considering the HW seems to be
+ * triggering the IRQ on both edges while monitoring DAT0 for
+ * busy completion and that same status bit is used to monitor
+ * start and end of busy detection, special care must be taken
+ * to make sure that both start and end interrupts are always
+ * cleared one after the other.
*/
status &= readl(host->base + MMCIMASK0);
- writel(status, host->base + MMCICLEAR);
+ if (host->variant->busy_detect)
+ writel(status & ~host->variant->busy_detect_mask,
+ host->base + MMCICLEAR);
+ else
+ writel(status, host->base + MMCICLEAR);
dev_dbg(mmc_dev(host->mmc), "irq0 (data+cmd) %08x\n", status);
if (intmask & SDHCI_INT_RETUNE)
mmc_retune_needed(host->mmc);
- if (intmask & SDHCI_INT_CARD_INT) {
+ if ((intmask & SDHCI_INT_CARD_INT) &&
+ (host->ier & SDHCI_INT_CARD_INT)) {
sdhci_enable_sdio_irq_nolock(host, false);
host->thread_isr |= SDHCI_INT_CARD_INT;
result = IRQ_WAKE_THREAD;
dev->irq = pdev->irq;
priv->base = addr;
+ priv->device = &pdev->dev;
if (!c_can_pci_data->freq) {
dev_err(&pdev->dev, "no clock frequency defined\n");
netif_napi_add(ndev, &priv->napi, ti_hecc_rx_poll,
HECC_DEF_NAPI_WEIGHT);
- clk_enable(priv->clk);
+ err = clk_prepare_enable(priv->clk);
+ if (err) {
+ dev_err(&pdev->dev, "clk_prepare_enable() failed\n");
+ goto probe_exit_clk;
+ }
+
err = register_candev(ndev);
if (err) {
dev_err(&pdev->dev, "register_candev() failed\n");
struct ti_hecc_priv *priv = netdev_priv(ndev);
unregister_candev(ndev);
- clk_disable(priv->clk);
+ clk_disable_unprepare(priv->clk);
clk_put(priv->clk);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
iounmap(priv->base);
hecc_set_bit(priv, HECC_CANMC, HECC_CANMC_PDR);
priv->can.state = CAN_STATE_SLEEPING;
- clk_disable(priv->clk);
+ clk_disable_unprepare(priv->clk);
return 0;
}
{
struct net_device *dev = platform_get_drvdata(pdev);
struct ti_hecc_priv *priv = netdev_priv(dev);
+ int err;
- clk_enable(priv->clk);
+ err = clk_prepare_enable(priv->clk);
+ if (err)
+ return err;
hecc_clear_bit(priv, HECC_CANMC, HECC_CANMC_PDR);
priv->can.state = CAN_STATE_ERROR_ACTIVE;
if (skb == NULL)
break;
np->rx_info[i].mapping = pci_map_single(np->pci_dev, skb->data, np->rx_buf_sz, PCI_DMA_FROMDEVICE);
+ if (pci_dma_mapping_error(np->pci_dev,
+ np->rx_info[i].mapping)) {
+ dev_kfree_skb(skb);
+ np->rx_info[i].skb = NULL;
+ break;
+ }
/* Grrr, we cannot offset to correctly align the IP header. */
np->rx_ring[i].rxaddr = cpu_to_dma(np->rx_info[i].mapping | RxDescValid);
}
{
struct netdev_private *np = netdev_priv(dev);
unsigned int entry;
+ unsigned int prev_tx;
u32 status;
- int i;
+ int i, j;
/*
* be cautious here, wrapping the queue has weird semantics
}
#endif /* ZEROCOPY && HAS_BROKEN_FIRMWARE */
+ prev_tx = np->cur_tx;
entry = np->cur_tx % TX_RING_SIZE;
for (i = 0; i < skb_num_frags(skb); i++) {
int wrap_ring = 0;
skb_frag_size(this_frag),
PCI_DMA_TODEVICE);
}
+ if (pci_dma_mapping_error(np->pci_dev,
+ np->tx_info[entry].mapping)) {
+ dev->stats.tx_dropped++;
+ goto err_out;
+ }
np->tx_ring[entry].addr = cpu_to_dma(np->tx_info[entry].mapping);
np->tx_ring[entry].status = cpu_to_le32(status);
netif_stop_queue(dev);
return NETDEV_TX_OK;
-}
+err_out:
+ entry = prev_tx % TX_RING_SIZE;
+ np->tx_info[entry].skb = NULL;
+ if (i > 0) {
+ pci_unmap_single(np->pci_dev,
+ np->tx_info[entry].mapping,
+ skb_first_frag_len(skb),
+ PCI_DMA_TODEVICE);
+ np->tx_info[entry].mapping = 0;
+ entry = (entry + np->tx_info[entry].used_slots) % TX_RING_SIZE;
+ for (j = 1; j < i; j++) {
+ pci_unmap_single(np->pci_dev,
+ np->tx_info[entry].mapping,
+ skb_frag_size(
+ &skb_shinfo(skb)->frags[j-1]),
+ PCI_DMA_TODEVICE);
+ entry++;
+ }
+ }
+ dev_kfree_skb_any(skb);
+ np->cur_tx = prev_tx;
+ return NETDEV_TX_OK;
+}
/* The interrupt handler does all of the Rx thread work and cleans up
after the Tx thread. */
break; /* Better luck next round. */
np->rx_info[entry].mapping =
pci_map_single(np->pci_dev, skb->data, np->rx_buf_sz, PCI_DMA_FROMDEVICE);
+ if (pci_dma_mapping_error(np->pci_dev,
+ np->rx_info[entry].mapping)) {
+ dev_kfree_skb(skb);
+ np->rx_info[entry].skb = NULL;
+ break;
+ }
np->rx_ring[entry].rxaddr =
cpu_to_dma(np->rx_info[entry].mapping | RxDescValid);
}
#define PCS_V1_WINDOW_SELECT 0x03fc
#define PCS_V2_WINDOW_DEF 0x9060
#define PCS_V2_WINDOW_SELECT 0x9064
+#define PCS_V2_RV_WINDOW_DEF 0x1060
+#define PCS_V2_RV_WINDOW_SELECT 0x1064
/* PCS register entry bit positions and sizes */
#define PCS_V2_WINDOW_DEF_OFFSET_INDEX 6
offset = pdata->xpcs_window + (mmd_address & pdata->xpcs_window_mask);
spin_lock_irqsave(&pdata->xpcs_lock, flags);
- XPCS32_IOWRITE(pdata, PCS_V2_WINDOW_SELECT, index);
+ XPCS32_IOWRITE(pdata, pdata->xpcs_window_sel_reg, index);
mmd_data = XPCS16_IOREAD(pdata, offset);
spin_unlock_irqrestore(&pdata->xpcs_lock, flags);
offset = pdata->xpcs_window + (mmd_address & pdata->xpcs_window_mask);
spin_lock_irqsave(&pdata->xpcs_lock, flags);
- XPCS32_IOWRITE(pdata, PCS_V2_WINDOW_SELECT, index);
+ XPCS32_IOWRITE(pdata, pdata->xpcs_window_sel_reg, index);
XPCS16_IOWRITE(pdata, offset, mmd_data);
spin_unlock_irqrestore(&pdata->xpcs_lock, flags);
}
/* Flush Tx queues */
ret = xgbe_flush_tx_queues(pdata);
- if (ret)
+ if (ret) {
+ netdev_err(pdata->netdev, "error flushing TX queues\n");
return ret;
+ }
/*
* Initialize DMA related features
DBGPR("-->xgbe_start\n");
- hw_if->init(pdata);
+ ret = hw_if->init(pdata);
+ if (ret)
+ return ret;
xgbe_napi_enable(pdata, 1);
struct xgbe_prv_data *pdata;
struct device *dev = &pdev->dev;
void __iomem * const *iomap_table;
+ struct pci_dev *rdev;
unsigned int ma_lo, ma_hi;
unsigned int reg;
int bar_mask;
if (netif_msg_probe(pdata))
dev_dbg(dev, "xpcs_regs = %p\n", pdata->xpcs_regs);
+ /* Set the PCS indirect addressing definition registers */
+ rdev = pci_get_domain_bus_and_slot(0, 0, PCI_DEVFN(0, 0));
+ if (rdev &&
+ (rdev->vendor == PCI_VENDOR_ID_AMD) && (rdev->device == 0x15d0)) {
+ pdata->xpcs_window_def_reg = PCS_V2_RV_WINDOW_DEF;
+ pdata->xpcs_window_sel_reg = PCS_V2_RV_WINDOW_SELECT;
+ } else {
+ pdata->xpcs_window_def_reg = PCS_V2_WINDOW_DEF;
+ pdata->xpcs_window_sel_reg = PCS_V2_WINDOW_SELECT;
+ }
+ pci_dev_put(rdev);
+
/* Configure the PCS indirect addressing support */
- reg = XPCS32_IOREAD(pdata, PCS_V2_WINDOW_DEF);
+ reg = XPCS32_IOREAD(pdata, pdata->xpcs_window_def_reg);
pdata->xpcs_window = XPCS_GET_BITS(reg, PCS_V2_WINDOW_DEF, OFFSET);
pdata->xpcs_window <<= 6;
pdata->xpcs_window_size = XPCS_GET_BITS(reg, PCS_V2_WINDOW_DEF, SIZE);
/* XPCS indirect addressing lock */
spinlock_t xpcs_lock;
+ unsigned int xpcs_window_def_reg;
+ unsigned int xpcs_window_sel_reg;
unsigned int xpcs_window;
unsigned int xpcs_window_size;
unsigned int xpcs_window_mask;
return -ENOMEM;
}
- alx_reinit_rings(alx);
-
return 0;
}
if (alx->qnapi[0] && alx->qnapi[0]->rxq)
kfree(alx->qnapi[0]->rxq->bufs);
- if (!alx->descmem.virt)
+ if (alx->descmem.virt)
dma_free_coherent(&alx->hw.pdev->dev,
alx->descmem.size,
alx->descmem.virt,
alx_free_rings(alx);
alx_free_napis(alx);
alx_disable_advanced_intr(alx);
+ alx_init_intr(alx, false);
err = alx_alloc_napis(alx);
if (err)
if (err)
goto out_free_rings;
+ /* must be called after alx_request_irq because the chip stops working
+ * if we copy the dma addresses in alx_init_ring_ptrs twice when
+ * requesting msi-x interrupts failed
+ */
+ alx_reinit_rings(alx);
+
netif_set_real_num_tx_queues(alx->dev, alx->num_txq);
netif_set_real_num_rx_queues(alx->dev, alx->num_rxq);
priv->old_link = 0;
priv->old_duplex = -1;
priv->old_pause = -1;
+ } else {
+ phydev = NULL;
}
/* mask all interrupts and request them */
enet_dmac_writel(priv, priv->dma_chan_int_mask,
ENETDMAC_IRMASK, priv->tx_chan);
- if (priv->has_phy)
+ if (phydev)
phy_start(phydev);
else
bcm_enet_adjust_link(dev);
free_irq(dev->irq, dev);
out_phy_disconnect:
- if (priv->has_phy)
+ if (phydev)
phy_disconnect(phydev);
return ret;
{
#ifdef CONFIG_INET
struct tcphdr *th;
- int len, nw_off, tcp_opt_len;
+ int len, nw_off, tcp_opt_len = 0;
if (tcp_ts)
tcp_opt_len = 12;
if ((link_info->support_auto_speeds | diff) !=
link_info->support_auto_speeds) {
/* An advertised speed is no longer supported, so we need to
- * update the advertisement settings. See bnxt_reset() for
- * comments about the rtnl_lock() sequence below.
+ * update the advertisement settings. Caller holds RTNL
+ * so we can modify link settings.
*/
- clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
- rtnl_lock();
link_info->advertising = link_info->support_auto_speeds;
- if (test_bit(BNXT_STATE_OPEN, &bp->state) &&
- (link_info->autoneg & BNXT_AUTONEG_SPEED))
+ if (link_info->autoneg & BNXT_AUTONEG_SPEED)
bnxt_hwrm_set_link_setting(bp, true, false);
- set_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
- rtnl_unlock();
}
return 0;
}
mod_timer(&bp->timer, jiffies + bp->current_interval);
}
-/* Only called from bnxt_sp_task() */
-static void bnxt_reset(struct bnxt *bp, bool silent)
+static void bnxt_rtnl_lock_sp(struct bnxt *bp)
{
- /* bnxt_reset_task() calls bnxt_close_nic() which waits
- * for BNXT_STATE_IN_SP_TASK to clear.
- * If there is a parallel dev_close(), bnxt_close() may be holding
+ /* We are called from bnxt_sp_task which has BNXT_STATE_IN_SP_TASK
+ * set. If the device is being closed, bnxt_close() may be holding
* rtnl() and waiting for BNXT_STATE_IN_SP_TASK to clear. So we
* must clear BNXT_STATE_IN_SP_TASK before holding rtnl().
*/
clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
rtnl_lock();
- if (test_bit(BNXT_STATE_OPEN, &bp->state))
- bnxt_reset_task(bp, silent);
+}
+
+static void bnxt_rtnl_unlock_sp(struct bnxt *bp)
+{
set_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
rtnl_unlock();
}
+/* Only called from bnxt_sp_task() */
+static void bnxt_reset(struct bnxt *bp, bool silent)
+{
+ bnxt_rtnl_lock_sp(bp);
+ if (test_bit(BNXT_STATE_OPEN, &bp->state))
+ bnxt_reset_task(bp, silent);
+ bnxt_rtnl_unlock_sp(bp);
+}
+
static void bnxt_cfg_ntp_filters(struct bnxt *);
static void bnxt_sp_task(struct work_struct *work)
{
struct bnxt *bp = container_of(work, struct bnxt, sp_task);
- int rc;
set_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
smp_mb__after_atomic();
if (test_and_clear_bit(BNXT_RX_NTP_FLTR_SP_EVENT, &bp->sp_event))
bnxt_cfg_ntp_filters(bp);
- if (test_and_clear_bit(BNXT_LINK_CHNG_SP_EVENT, &bp->sp_event)) {
- if (test_and_clear_bit(BNXT_LINK_SPEED_CHNG_SP_EVENT,
- &bp->sp_event))
- bnxt_hwrm_phy_qcaps(bp);
-
- rc = bnxt_update_link(bp, true);
- if (rc)
- netdev_err(bp->dev, "SP task can't update link (rc: %x)\n",
- rc);
- }
if (test_and_clear_bit(BNXT_HWRM_EXEC_FWD_REQ_SP_EVENT, &bp->sp_event))
bnxt_hwrm_exec_fwd_req(bp);
if (test_and_clear_bit(BNXT_VXLAN_ADD_PORT_SP_EVENT, &bp->sp_event)) {
bnxt_hwrm_tunnel_dst_port_free(
bp, TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_GENEVE);
}
+ if (test_and_clear_bit(BNXT_PERIODIC_STATS_SP_EVENT, &bp->sp_event))
+ bnxt_hwrm_port_qstats(bp);
+
+ /* These functions below will clear BNXT_STATE_IN_SP_TASK. They
+ * must be the last functions to be called before exiting.
+ */
+ if (test_and_clear_bit(BNXT_LINK_CHNG_SP_EVENT, &bp->sp_event)) {
+ int rc = 0;
+
+ if (test_and_clear_bit(BNXT_LINK_SPEED_CHNG_SP_EVENT,
+ &bp->sp_event))
+ bnxt_hwrm_phy_qcaps(bp);
+
+ bnxt_rtnl_lock_sp(bp);
+ if (test_bit(BNXT_STATE_OPEN, &bp->state))
+ rc = bnxt_update_link(bp, true);
+ bnxt_rtnl_unlock_sp(bp);
+ if (rc)
+ netdev_err(bp->dev, "SP task can't update link (rc: %x)\n",
+ rc);
+ }
+ if (test_and_clear_bit(BNXT_HWRM_PORT_MODULE_SP_EVENT, &bp->sp_event)) {
+ bnxt_rtnl_lock_sp(bp);
+ if (test_bit(BNXT_STATE_OPEN, &bp->state))
+ bnxt_get_port_module_status(bp);
+ bnxt_rtnl_unlock_sp(bp);
+ }
if (test_and_clear_bit(BNXT_RESET_TASK_SP_EVENT, &bp->sp_event))
bnxt_reset(bp, false);
if (test_and_clear_bit(BNXT_RESET_TASK_SILENT_SP_EVENT, &bp->sp_event))
bnxt_reset(bp, true);
- if (test_and_clear_bit(BNXT_HWRM_PORT_MODULE_SP_EVENT, &bp->sp_event))
- bnxt_get_port_module_status(bp);
-
- if (test_and_clear_bit(BNXT_PERIODIC_STATS_SP_EVENT, &bp->sp_event))
- bnxt_hwrm_port_qstats(bp);
-
smp_mb__before_atomic();
clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
}
#define DEFAULT_RX_RING_SIZE 512 /* must be power of 2 */
#define MIN_RX_RING_SIZE 64
#define MAX_RX_RING_SIZE 8192
-#define RX_RING_BYTES(bp) (sizeof(struct macb_dma_desc) \
+#define RX_RING_BYTES(bp) (macb_dma_desc_get_size(bp) \
* (bp)->rx_ring_size)
#define DEFAULT_TX_RING_SIZE 512 /* must be power of 2 */
#define MIN_TX_RING_SIZE 64
#define MAX_TX_RING_SIZE 4096
-#define TX_RING_BYTES(bp) (sizeof(struct macb_dma_desc) \
+#define TX_RING_BYTES(bp) (macb_dma_desc_get_size(bp) \
* (bp)->tx_ring_size)
/* level of occupied TX descriptors under which we wake up TX process */
*/
#define MACB_HALT_TIMEOUT 1230
+/* DMA buffer descriptor might be different size
+ * depends on hardware configuration.
+ */
+static unsigned int macb_dma_desc_get_size(struct macb *bp)
+{
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ return sizeof(struct macb_dma_desc) + sizeof(struct macb_dma_desc_64);
+#endif
+ return sizeof(struct macb_dma_desc);
+}
+
+static unsigned int macb_adj_dma_desc_idx(struct macb *bp, unsigned int idx)
+{
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ /* Dma buffer descriptor is 4 words length (instead of 2 words)
+ * for 64b GEM.
+ */
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ idx <<= 1;
+#endif
+ return idx;
+}
+
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+static struct macb_dma_desc_64 *macb_64b_desc(struct macb *bp, struct macb_dma_desc *desc)
+{
+ return (struct macb_dma_desc_64 *)((void *)desc + sizeof(struct macb_dma_desc));
+}
+#endif
+
/* Ring buffer accessors */
static unsigned int macb_tx_ring_wrap(struct macb *bp, unsigned int index)
{
static struct macb_dma_desc *macb_tx_desc(struct macb_queue *queue,
unsigned int index)
{
- return &queue->tx_ring[macb_tx_ring_wrap(queue->bp, index)];
+ index = macb_tx_ring_wrap(queue->bp, index);
+ index = macb_adj_dma_desc_idx(queue->bp, index);
+ return &queue->tx_ring[index];
}
static struct macb_tx_skb *macb_tx_skb(struct macb_queue *queue,
dma_addr_t offset;
offset = macb_tx_ring_wrap(queue->bp, index) *
- sizeof(struct macb_dma_desc);
+ macb_dma_desc_get_size(queue->bp);
return queue->tx_ring_dma + offset;
}
static struct macb_dma_desc *macb_rx_desc(struct macb *bp, unsigned int index)
{
- return &bp->rx_ring[macb_rx_ring_wrap(bp, index)];
+ index = macb_rx_ring_wrap(bp, index);
+ index = macb_adj_dma_desc_idx(bp, index);
+ return &bp->rx_ring[index];
}
static void *macb_rx_buffer(struct macb *bp, unsigned int index)
}
}
-static inline void macb_set_addr(struct macb_dma_desc *desc, dma_addr_t addr)
+static void macb_set_addr(struct macb *bp, struct macb_dma_desc *desc, dma_addr_t addr)
{
- desc->addr = (u32)addr;
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- desc->addrh = (u32)(addr >> 32);
+ struct macb_dma_desc_64 *desc_64;
+
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B) {
+ desc_64 = macb_64b_desc(bp, desc);
+ desc_64->addrh = upper_32_bits(addr);
+ }
#endif
+ desc->addr = lower_32_bits(addr);
+}
+
+static dma_addr_t macb_get_addr(struct macb *bp, struct macb_dma_desc *desc)
+{
+ dma_addr_t addr = 0;
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ struct macb_dma_desc_64 *desc_64;
+
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B) {
+ desc_64 = macb_64b_desc(bp, desc);
+ addr = ((u64)(desc_64->addrh) << 32);
+ }
+#endif
+ addr |= MACB_BF(RX_WADDR, MACB_BFEXT(RX_WADDR, desc->addr));
+ return addr;
}
static void macb_tx_error_task(struct work_struct *work)
/* Set end of TX queue */
desc = macb_tx_desc(queue, 0);
- macb_set_addr(desc, 0);
+ macb_set_addr(bp, desc, 0);
desc->ctrl = MACB_BIT(TX_USED);
/* Make descriptor updates visible to hardware */
wmb();
/* Reinitialize the TX desc queue */
- queue_writel(queue, TBQP, (u32)(queue->tx_ring_dma));
+ queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- queue_writel(queue, TBQPH, (u32)(queue->tx_ring_dma >> 32));
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
#endif
/* Make TX ring reflect state of hardware */
queue->tx_head = 0;
unsigned int entry;
struct sk_buff *skb;
dma_addr_t paddr;
+ struct macb_dma_desc *desc;
while (CIRC_SPACE(bp->rx_prepared_head, bp->rx_tail,
bp->rx_ring_size) > 0) {
rmb();
bp->rx_prepared_head++;
+ desc = macb_rx_desc(bp, entry);
if (!bp->rx_skbuff[entry]) {
/* allocate sk_buff for this free entry in ring */
if (entry == bp->rx_ring_size - 1)
paddr |= MACB_BIT(RX_WRAP);
- macb_set_addr(&(bp->rx_ring[entry]), paddr);
- bp->rx_ring[entry].ctrl = 0;
+ macb_set_addr(bp, desc, paddr);
+ desc->ctrl = 0;
/* properly align Ethernet header */
skb_reserve(skb, NET_IP_ALIGN);
} else {
- bp->rx_ring[entry].addr &= ~MACB_BIT(RX_USED);
- bp->rx_ring[entry].ctrl = 0;
+ desc->addr &= ~MACB_BIT(RX_USED);
+ desc->ctrl = 0;
}
}
bool rxused;
entry = macb_rx_ring_wrap(bp, bp->rx_tail);
- desc = &bp->rx_ring[entry];
+ desc = macb_rx_desc(bp, entry);
/* Make hw descriptor updates visible to CPU */
rmb();
rxused = (desc->addr & MACB_BIT(RX_USED)) ? true : false;
- addr = MACB_BF(RX_WADDR, MACB_BFEXT(RX_WADDR, desc->addr));
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- addr |= ((u64)(desc->addrh) << 32);
-#endif
+ addr = macb_get_addr(bp, desc);
ctrl = desc->ctrl;
if (!rxused)
static inline void macb_init_rx_ring(struct macb *bp)
{
dma_addr_t addr;
+ struct macb_dma_desc *desc = NULL;
int i;
addr = bp->rx_buffers_dma;
for (i = 0; i < bp->rx_ring_size; i++) {
- bp->rx_ring[i].addr = addr;
- bp->rx_ring[i].ctrl = 0;
+ desc = macb_rx_desc(bp, i);
+ macb_set_addr(bp, desc, addr);
+ desc->ctrl = 0;
addr += bp->rx_buffer_size;
}
- bp->rx_ring[bp->rx_ring_size - 1].addr |= MACB_BIT(RX_WRAP);
+ desc->addr |= MACB_BIT(RX_WRAP);
bp->rx_tail = 0;
}
for (tail = bp->rx_tail; budget > 0; tail++) {
struct macb_dma_desc *desc = macb_rx_desc(bp, tail);
- u32 addr, ctrl;
+ u32 ctrl;
/* Make hw descriptor updates visible to CPU */
rmb();
- addr = desc->addr;
ctrl = desc->ctrl;
- if (!(addr & MACB_BIT(RX_USED)))
+ if (!(desc->addr & MACB_BIT(RX_USED)))
break;
if (ctrl & MACB_BIT(RX_SOF)) {
i = tx_head;
entry = macb_tx_ring_wrap(bp, i);
ctrl = MACB_BIT(TX_USED);
- desc = &queue->tx_ring[entry];
+ desc = macb_tx_desc(queue, entry);
desc->ctrl = ctrl;
if (lso_ctrl) {
i--;
entry = macb_tx_ring_wrap(bp, i);
tx_skb = &queue->tx_skb[entry];
- desc = &queue->tx_ring[entry];
+ desc = macb_tx_desc(queue, entry);
ctrl = (u32)tx_skb->size;
if (eof) {
ctrl |= MACB_BF(MSS_MFS, mss_mfs);
/* Set TX buffer descriptor */
- macb_set_addr(desc, tx_skb->mapping);
+ macb_set_addr(bp, desc, tx_skb->mapping);
/* desc->addr must be visible to hardware before clearing
* 'TX_USED' bit in desc->ctrl.
*/
if (!skb)
continue;
- desc = &bp->rx_ring[i];
- addr = MACB_BF(RX_WADDR, MACB_BFEXT(RX_WADDR, desc->addr));
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- addr |= ((u64)(desc->addrh) << 32);
-#endif
+ desc = macb_rx_desc(bp, i);
+ addr = macb_get_addr(bp, desc);
+
dma_unmap_single(&bp->pdev->dev, addr, bp->rx_buffer_size,
DMA_FROM_DEVICE);
dev_kfree_skb_any(skb);
static void gem_init_rings(struct macb *bp)
{
struct macb_queue *queue;
+ struct macb_dma_desc *desc = NULL;
unsigned int q;
int i;
for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
for (i = 0; i < bp->tx_ring_size; i++) {
- queue->tx_ring[i].addr = 0;
- queue->tx_ring[i].ctrl = MACB_BIT(TX_USED);
+ desc = macb_tx_desc(queue, i);
+ macb_set_addr(bp, desc, 0);
+ desc->ctrl = MACB_BIT(TX_USED);
}
- queue->tx_ring[bp->tx_ring_size - 1].ctrl |= MACB_BIT(TX_WRAP);
+ desc->ctrl |= MACB_BIT(TX_WRAP);
queue->tx_head = 0;
queue->tx_tail = 0;
}
static void macb_init_rings(struct macb *bp)
{
int i;
+ struct macb_dma_desc *desc = NULL;
macb_init_rx_ring(bp);
for (i = 0; i < bp->tx_ring_size; i++) {
- bp->queues[0].tx_ring[i].addr = 0;
- bp->queues[0].tx_ring[i].ctrl = MACB_BIT(TX_USED);
+ desc = macb_tx_desc(&bp->queues[0], i);
+ macb_set_addr(bp, desc, 0);
+ desc->ctrl = MACB_BIT(TX_USED);
}
bp->queues[0].tx_head = 0;
bp->queues[0].tx_tail = 0;
- bp->queues[0].tx_ring[bp->tx_ring_size - 1].ctrl |= MACB_BIT(TX_WRAP);
+ desc->ctrl |= MACB_BIT(TX_WRAP);
}
static void macb_reset_hw(struct macb *bp)
dmacfg &= ~GEM_BIT(TXCOEN);
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- dmacfg |= GEM_BIT(ADDR64);
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ dmacfg |= GEM_BIT(ADDR64);
#endif
netdev_dbg(bp->dev, "Cadence configure DMA with 0x%08x\n",
dmacfg);
macb_configure_dma(bp);
/* Initialize TX and RX buffers */
- macb_writel(bp, RBQP, (u32)(bp->rx_ring_dma));
+ macb_writel(bp, RBQP, lower_32_bits(bp->rx_ring_dma));
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- macb_writel(bp, RBQPH, (u32)(bp->rx_ring_dma >> 32));
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ macb_writel(bp, RBQPH, upper_32_bits(bp->rx_ring_dma));
#endif
for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- queue_writel(queue, TBQP, (u32)(queue->tx_ring_dma));
+ queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- queue_writel(queue, TBQPH, (u32)(queue->tx_ring_dma >> 32));
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
#endif
/* Enable interrupts */
queue->IMR = GEM_IMR(hw_q - 1);
queue->TBQP = GEM_TBQP(hw_q - 1);
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- queue->TBQPH = GEM_TBQPH(hw_q -1);
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ queue->TBQPH = GEM_TBQPH(hw_q - 1);
#endif
} else {
/* queue0 uses legacy registers */
queue->IMR = MACB_IMR;
queue->TBQP = MACB_TBQP;
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- queue->TBQPH = MACB_TBQPH;
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ queue->TBQPH = MACB_TBQPH;
#endif
}
static int at91ether_start(struct net_device *dev)
{
struct macb *lp = netdev_priv(dev);
+ struct macb_dma_desc *desc;
dma_addr_t addr;
u32 ctl;
int i;
lp->rx_ring = dma_alloc_coherent(&lp->pdev->dev,
(AT91ETHER_MAX_RX_DESCR *
- sizeof(struct macb_dma_desc)),
+ macb_dma_desc_get_size(lp)),
&lp->rx_ring_dma, GFP_KERNEL);
if (!lp->rx_ring)
return -ENOMEM;
if (!lp->rx_buffers) {
dma_free_coherent(&lp->pdev->dev,
AT91ETHER_MAX_RX_DESCR *
- sizeof(struct macb_dma_desc),
+ macb_dma_desc_get_size(lp),
lp->rx_ring, lp->rx_ring_dma);
lp->rx_ring = NULL;
return -ENOMEM;
addr = lp->rx_buffers_dma;
for (i = 0; i < AT91ETHER_MAX_RX_DESCR; i++) {
- lp->rx_ring[i].addr = addr;
- lp->rx_ring[i].ctrl = 0;
+ desc = macb_rx_desc(lp, i);
+ macb_set_addr(lp, desc, addr);
+ desc->ctrl = 0;
addr += AT91ETHER_MAX_RBUFF_SZ;
}
/* Set the Wrap bit on the last descriptor */
- lp->rx_ring[AT91ETHER_MAX_RX_DESCR - 1].addr |= MACB_BIT(RX_WRAP);
+ desc->addr |= MACB_BIT(RX_WRAP);
/* Reset buffer index */
lp->rx_tail = 0;
dma_free_coherent(&lp->pdev->dev,
AT91ETHER_MAX_RX_DESCR *
- sizeof(struct macb_dma_desc),
+ macb_dma_desc_get_size(lp),
lp->rx_ring, lp->rx_ring_dma);
lp->rx_ring = NULL;
static void at91ether_rx(struct net_device *dev)
{
struct macb *lp = netdev_priv(dev);
+ struct macb_dma_desc *desc;
unsigned char *p_recv;
struct sk_buff *skb;
unsigned int pktlen;
- while (lp->rx_ring[lp->rx_tail].addr & MACB_BIT(RX_USED)) {
+ desc = macb_rx_desc(lp, lp->rx_tail);
+ while (desc->addr & MACB_BIT(RX_USED)) {
p_recv = lp->rx_buffers + lp->rx_tail * AT91ETHER_MAX_RBUFF_SZ;
- pktlen = MACB_BF(RX_FRMLEN, lp->rx_ring[lp->rx_tail].ctrl);
+ pktlen = MACB_BF(RX_FRMLEN, desc->ctrl);
skb = netdev_alloc_skb(dev, pktlen + 2);
if (skb) {
skb_reserve(skb, 2);
lp->stats.rx_dropped++;
}
- if (lp->rx_ring[lp->rx_tail].ctrl & MACB_BIT(RX_MHASH_MATCH))
+ if (desc->ctrl & MACB_BIT(RX_MHASH_MATCH))
lp->stats.multicast++;
/* reset ownership bit */
- lp->rx_ring[lp->rx_tail].addr &= ~MACB_BIT(RX_USED);
+ desc->addr &= ~MACB_BIT(RX_USED);
/* wrap after last buffer */
if (lp->rx_tail == AT91ETHER_MAX_RX_DESCR - 1)
lp->rx_tail = 0;
else
lp->rx_tail++;
+
+ desc = macb_rx_desc(lp, lp->rx_tail);
}
}
device_init_wakeup(&pdev->dev, bp->wol & MACB_WOL_HAS_MAGIC_PACKET);
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (GEM_BFEXT(DBWDEF, gem_readl(bp, DCFG1)) > GEM_DBW32)
+ if (GEM_BFEXT(DAW64, gem_readl(bp, DCFG6))) {
dma_set_mask(&pdev->dev, DMA_BIT_MASK(44));
+ bp->hw_dma_cap = HW_DMA_CAP_64B;
+ } else
+ bp->hw_dma_cap = HW_DMA_CAP_32B;
#endif
spin_lock_init(&bp->lock);
/* Bitfields in DCFG6. */
#define GEM_PBUF_LSO_OFFSET 27
#define GEM_PBUF_LSO_SIZE 1
+#define GEM_DAW64_OFFSET 23
+#define GEM_DAW64_SIZE 1
/* Constants for CLK */
#define MACB_CLK_DIV8 0
struct macb_dma_desc {
u32 addr;
u32 ctrl;
+};
+
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- u32 addrh;
- u32 resvd;
-#endif
+enum macb_hw_dma_cap {
+ HW_DMA_CAP_32B,
+ HW_DMA_CAP_64B,
};
+struct macb_dma_desc_64 {
+ u32 addrh;
+ u32 resvd;
+};
+#endif
+
/* DMA descriptor bitfields */
#define MACB_RX_USED_OFFSET 0
#define MACB_RX_USED_SIZE 1
unsigned int jumbo_max_len;
u32 wol;
+
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ enum macb_hw_dma_cap hw_dma_cap;
+#endif
};
static inline bool macb_is_gem(struct macb *bp)
u8 lmac_type;
u8 lane_to_sds;
bool use_training;
+ bool autoneg;
bool link_up;
int lmacid; /* ID within BGX */
int lmacid_bd; /* ID on board */
/* power down, reset autoneg, autoneg enable */
cfg = bgx_reg_read(bgx, lmacid, BGX_GMP_PCS_MRX_CTL);
cfg &= ~PCS_MRX_CTL_PWR_DN;
- cfg |= (PCS_MRX_CTL_RST_AN | PCS_MRX_CTL_AN_EN);
+ cfg |= PCS_MRX_CTL_RST_AN;
+ if (lmac->phydev) {
+ cfg |= PCS_MRX_CTL_AN_EN;
+ } else {
+ /* In scenarios where PHY driver is not present or it's a
+ * non-standard PHY, FW sets AN_EN to inform Linux driver
+ * to do auto-neg and link polling or not.
+ */
+ if (cfg & PCS_MRX_CTL_AN_EN)
+ lmac->autoneg = true;
+ }
bgx_reg_write(bgx, lmacid, BGX_GMP_PCS_MRX_CTL, cfg);
if (lmac->lmac_type == BGX_MODE_QSGMII) {
return 0;
}
- if (lmac->lmac_type == BGX_MODE_SGMII) {
+ if ((lmac->lmac_type == BGX_MODE_SGMII) && lmac->phydev) {
if (bgx_poll_reg(bgx, lmacid, BGX_GMP_PCS_MRX_STATUS,
PCS_MRX_STATUS_AN_CPT, false)) {
dev_err(&bgx->pdev->dev, "BGX AN_CPT not completed\n");
return -1;
}
+static void bgx_poll_for_sgmii_link(struct lmac *lmac)
+{
+ u64 pcs_link, an_result;
+ u8 speed;
+
+ pcs_link = bgx_reg_read(lmac->bgx, lmac->lmacid,
+ BGX_GMP_PCS_MRX_STATUS);
+
+ /*Link state bit is sticky, read it again*/
+ if (!(pcs_link & PCS_MRX_STATUS_LINK))
+ pcs_link = bgx_reg_read(lmac->bgx, lmac->lmacid,
+ BGX_GMP_PCS_MRX_STATUS);
+
+ if (bgx_poll_reg(lmac->bgx, lmac->lmacid, BGX_GMP_PCS_MRX_STATUS,
+ PCS_MRX_STATUS_AN_CPT, false)) {
+ lmac->link_up = false;
+ lmac->last_speed = SPEED_UNKNOWN;
+ lmac->last_duplex = DUPLEX_UNKNOWN;
+ goto next_poll;
+ }
+
+ lmac->link_up = ((pcs_link & PCS_MRX_STATUS_LINK) != 0) ? true : false;
+ an_result = bgx_reg_read(lmac->bgx, lmac->lmacid,
+ BGX_GMP_PCS_ANX_AN_RESULTS);
+
+ speed = (an_result >> 3) & 0x3;
+ lmac->last_duplex = (an_result >> 1) & 0x1;
+ switch (speed) {
+ case 0:
+ lmac->last_speed = 10;
+ break;
+ case 1:
+ lmac->last_speed = 100;
+ break;
+ case 2:
+ lmac->last_speed = 1000;
+ break;
+ default:
+ lmac->link_up = false;
+ lmac->last_speed = SPEED_UNKNOWN;
+ lmac->last_duplex = DUPLEX_UNKNOWN;
+ break;
+ }
+
+next_poll:
+
+ if (lmac->last_link != lmac->link_up) {
+ if (lmac->link_up)
+ bgx_sgmii_change_link_state(lmac);
+ lmac->last_link = lmac->link_up;
+ }
+
+ queue_delayed_work(lmac->check_link, &lmac->dwork, HZ * 3);
+}
+
static void bgx_poll_for_link(struct work_struct *work)
{
struct lmac *lmac;
u64 spu_link, smu_link;
lmac = container_of(work, struct lmac, dwork.work);
+ if (lmac->is_sgmii) {
+ bgx_poll_for_sgmii_link(lmac);
+ return;
+ }
/* Receive link is latching low. Force it high and verify it */
bgx_reg_modify(lmac->bgx, lmac->lmacid,
(lmac->lmac_type != BGX_MODE_XLAUI) &&
(lmac->lmac_type != BGX_MODE_40G_KR) &&
(lmac->lmac_type != BGX_MODE_10G_KR)) {
- if (!lmac->phydev)
- return -ENODEV;
-
+ if (!lmac->phydev) {
+ if (lmac->autoneg) {
+ bgx_reg_write(bgx, lmacid,
+ BGX_GMP_PCS_LINKX_TIMER,
+ PCS_LINKX_TIMER_COUNT);
+ goto poll;
+ } else {
+ /* Default to below link speed and duplex */
+ lmac->link_up = true;
+ lmac->last_speed = 1000;
+ lmac->last_duplex = 1;
+ bgx_sgmii_change_link_state(lmac);
+ return 0;
+ }
+ }
lmac->phydev->dev_flags = 0;
if (phy_connect_direct(&lmac->netdev, lmac->phydev,
return -ENODEV;
phy_start_aneg(lmac->phydev);
- } else {
- lmac->check_link = alloc_workqueue("check_link", WQ_UNBOUND |
- WQ_MEM_RECLAIM, 1);
- if (!lmac->check_link)
- return -ENOMEM;
- INIT_DELAYED_WORK(&lmac->dwork, bgx_poll_for_link);
- queue_delayed_work(lmac->check_link, &lmac->dwork, 0);
+ return 0;
}
+poll:
+ lmac->check_link = alloc_workqueue("check_link", WQ_UNBOUND |
+ WQ_MEM_RECLAIM, 1);
+ if (!lmac->check_link)
+ return -ENOMEM;
+ INIT_DELAYED_WORK(&lmac->dwork, bgx_poll_for_link);
+ queue_delayed_work(lmac->check_link, &lmac->dwork, 0);
+
return 0;
}
#define PCS_MRX_CTL_LOOPBACK1 BIT_ULL(14)
#define PCS_MRX_CTL_RESET BIT_ULL(15)
#define BGX_GMP_PCS_MRX_STATUS 0x30008
+#define PCS_MRX_STATUS_LINK BIT_ULL(2)
#define PCS_MRX_STATUS_AN_CPT BIT_ULL(5)
+#define BGX_GMP_PCS_ANX_ADV 0x30010
#define BGX_GMP_PCS_ANX_AN_RESULTS 0x30020
+#define BGX_GMP_PCS_LINKX_TIMER 0x30040
+#define PCS_LINKX_TIMER_COUNT 0x1E84
#define BGX_GMP_PCS_SGM_AN_ADV 0x30068
#define BGX_GMP_PCS_MISCX_CTL 0x30078
+#define PCS_MISC_CTL_MODE BIT_ULL(8)
#define PCS_MISC_CTL_DISP_EN BIT_ULL(13)
#define PCS_MISC_CTL_GMX_ENO BIT_ULL(11)
#define PCS_MISC_CTL_SAMP_PT_MASK 0x7Full
int speed = 2;
if (!xcv) {
- dev_err(&xcv->pdev->dev,
- "XCV init not done, probe may have failed\n");
+ pr_err("XCV init not done, probe may have failed\n");
return;
}
status = -EPERM;
goto err;
}
-done:
+
+ /* Remember currently programmed MAC */
ether_addr_copy(adapter->dev_mac, addr->sa_data);
+done:
ether_addr_copy(netdev->dev_addr, addr->sa_data);
dev_info(dev, "MAC address changed to %pM\n", addr->sa_data);
return 0;
{
/* Don't delete MAC on BE3 VFs without FILTMGMT privilege */
if (!BEx_chip(adapter) || !be_virtfn(adapter) ||
- check_privilege(adapter, BE_PRIV_FILTMGMT))
+ check_privilege(adapter, BE_PRIV_FILTMGMT)) {
be_dev_mac_del(adapter, adapter->pmac_id[0]);
+ eth_zero_addr(adapter->dev_mac);
+ }
be_clear_uc_list(adapter);
be_clear_mc_list(adapter);
if (status)
return status;
- /* Don't add MAC on BE3 VFs without FILTMGMT privilege */
- if (!BEx_chip(adapter) || !be_virtfn(adapter) ||
- check_privilege(adapter, BE_PRIV_FILTMGMT)) {
+ /* Normally this condition usually true as the ->dev_mac is zeroed.
+ * But on BE3 VFs the initial MAC is pre-programmed by PF and
+ * subsequent be_dev_mac_add() can fail (after fresh boot)
+ */
+ if (!ether_addr_equal(adapter->dev_mac, adapter->netdev->dev_addr)) {
+ int old_pmac_id = -1;
+
+ /* Remember old programmed MAC if any - can happen on BE3 VF */
+ if (!is_zero_ether_addr(adapter->dev_mac))
+ old_pmac_id = adapter->pmac_id[0];
+
status = be_dev_mac_add(adapter, adapter->netdev->dev_addr);
if (status)
return status;
+
+ /* Delete the old programmed MAC as we successfully programmed
+ * a new MAC
+ */
+ if (old_pmac_id >= 0 && old_pmac_id != adapter->pmac_id[0])
+ be_dev_mac_del(adapter, old_pmac_id);
+
ether_addr_copy(adapter->dev_mac, adapter->netdev->dev_addr);
}
memcpy(adapter->netdev->dev_addr, mac, ETH_ALEN);
memcpy(adapter->netdev->perm_addr, mac, ETH_ALEN);
+
+ /* Initial MAC for BE3 VFs is already programmed by PF */
+ if (BEx_chip(adapter) && be_virtfn(adapter))
+ memcpy(adapter->dev_mac, mac, ETH_ALEN);
}
return 0;
if (!rxb->page)
continue;
- dma_unmap_single(rx_queue->dev, rxb->dma,
- PAGE_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_page(rx_queue->dev, rxb->dma,
+ PAGE_SIZE, DMA_FROM_DEVICE);
__free_page(rxb->page);
rxb->page = NULL;
}
/* try reuse page */
- if (unlikely(page_count(page) != 1))
+ if (unlikely(page_count(page) != 1 || page_is_pfmemalloc(page)))
return false;
/* change offset to the other half */
static inline void dsaf_write_reg(void __iomem *base, u32 reg, u32 value)
{
- u8 __iomem *reg_addr = ACCESS_ONCE(base);
-
- writel(value, reg_addr + reg);
+ writel(value, base + reg);
}
#define dsaf_write_dev(a, reg, value) \
static inline u32 dsaf_read_reg(u8 __iomem *base, u32 reg)
{
- u8 __iomem *reg_addr = ACCESS_ONCE(base);
-
- return readl(reg_addr + reg);
+ return readl(base + reg);
}
static inline void dsaf_write_syscon(struct regmap *base, u32 reg, u32 value)
struct hns_nic_ring_data *ring_data)
{
struct hns_nic_priv *priv = netdev_priv(ndev);
- struct device *dev = priv->dev;
struct hnae_ring *ring = ring_data->ring;
+ struct device *dev = ring_to_dev(ring);
struct netdev_queue *dev_queue;
struct skb_frag_struct *frag;
int buf_num;
netdev->netdev_ops = &ibmveth_netdev_ops;
netdev->ethtool_ops = &netdev_ethtool_ops;
SET_NETDEV_DEV(netdev, &dev->dev);
- netdev->hw_features = NETIF_F_SG | NETIF_F_RXCSUM |
- NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
+ netdev->hw_features = NETIF_F_SG;
+ if (vio_get_attribute(dev, "ibm,illan-options", NULL) != NULL) {
+ netdev->hw_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
+ NETIF_F_RXCSUM;
+ }
netdev->features |= netdev->hw_features;
}
const struct of_device_id of_mtk_match[] = {
- { .compatible = "mediatek,mt7623-eth" },
+ { .compatible = "mediatek,mt2701-eth" },
{},
};
MODULE_DEVICE_TABLE(of, of_mtk_match);
return -ETIMEDOUT;
}
-static int mlx4_comm_internal_err(u32 slave_read)
+int mlx4_comm_internal_err(u32 slave_read)
{
return (u32)COMM_CHAN_EVENT_INTERNAL_ERR ==
(slave_read & (u32)COMM_CHAN_EVENT_INTERNAL_ERR) ? 1 : 0;
memcpy(&new_prof, priv->prof, sizeof(struct mlx4_en_port_profile));
new_prof.tx_ring_size = tx_size;
new_prof.rx_ring_size = rx_size;
- err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof);
+ err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof, true);
if (err)
goto out;
{
struct mlx4_en_priv *priv = netdev_priv(dev);
- memset(channel, 0, sizeof(*channel));
-
channel->max_rx = MAX_RX_RINGS;
channel->max_tx = MLX4_EN_MAX_TX_RING_P_UP;
int xdp_count;
int err = 0;
- if (channel->other_count || channel->combined_count ||
- channel->tx_count > MLX4_EN_MAX_TX_RING_P_UP ||
- channel->rx_count > MAX_RX_RINGS ||
- !channel->tx_count || !channel->rx_count)
+ if (!channel->tx_count || !channel->rx_count)
return -EINVAL;
tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
new_prof.tx_ring_num[TX_XDP] = xdp_count;
new_prof.rx_ring_num = channel->rx_count;
- err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof);
+ err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof, true);
if (err)
goto out;
if (priv->tx_cq[t] && priv->tx_cq[t][i])
mlx4_en_destroy_cq(priv, &priv->tx_cq[t][i]);
}
+ kfree(priv->tx_ring[t]);
+ kfree(priv->tx_cq[t]);
}
for (i = 0; i < priv->rx_ring_num; i++) {
int mlx4_en_try_alloc_resources(struct mlx4_en_priv *priv,
struct mlx4_en_priv *tmp,
- struct mlx4_en_port_profile *prof)
+ struct mlx4_en_port_profile *prof,
+ bool carry_xdp_prog)
{
- int t;
+ struct bpf_prog *xdp_prog;
+ int i, t;
mlx4_en_copy_priv(tmp, priv, prof);
}
return -ENOMEM;
}
+
+ /* All rx_rings has the same xdp_prog. Pick the first one. */
+ xdp_prog = rcu_dereference_protected(
+ priv->rx_ring[0]->xdp_prog,
+ lockdep_is_held(&priv->mdev->state_lock));
+
+ if (xdp_prog && carry_xdp_prog) {
+ xdp_prog = bpf_prog_add(xdp_prog, tmp->rx_ring_num);
+ if (IS_ERR(xdp_prog)) {
+ mlx4_en_free_resources(tmp);
+ return PTR_ERR(xdp_prog);
+ }
+ for (i = 0; i < tmp->rx_ring_num; i++)
+ rcu_assign_pointer(tmp->rx_ring[i]->xdp_prog,
+ xdp_prog);
+ }
+
return 0;
}
{
struct mlx4_en_priv *priv = netdev_priv(dev);
struct mlx4_en_dev *mdev = priv->mdev;
- int t;
en_dbg(DRV, priv, "Destroying netdev on port:%d\n", priv->port);
mlx4_en_free_resources(priv);
mutex_unlock(&mdev->state_lock);
- for (t = 0; t < MLX4_EN_NUM_TX_TYPES; t++) {
- kfree(priv->tx_ring[t]);
- kfree(priv->tx_cq[t]);
- }
-
free_netdev(dev);
}
en_warn(priv, "Reducing the number of TX rings, to not exceed the max total rings number.\n");
}
- err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof);
+ err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof, false);
if (err) {
if (prog)
bpf_prog_sub(prog, priv->rx_ring_num - 1);
memcpy(&new_prof, priv->prof, sizeof(struct mlx4_en_port_profile));
memcpy(&new_prof.hwtstamp_config, &ts_config, sizeof(ts_config));
- err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof);
+ err = mlx4_en_try_alloc_resources(priv, tmp, &new_prof, true);
if (err)
goto out;
return;
for (ring = 0; ring < priv->rx_ring_num; ring++) {
- if (mlx4_en_is_ring_empty(priv->rx_ring[ring]))
+ if (mlx4_en_is_ring_empty(priv->rx_ring[ring])) {
+ local_bh_disable();
napi_reschedule(&priv->rx_cq[ring]->napi);
+ local_bh_enable();
+ }
}
}
return;
mlx4_stop_catas_poll(dev);
+ if (dev->persist->interface_state & MLX4_INTERFACE_STATE_DELETION &&
+ mlx4_is_slave(dev)) {
+ /* In mlx4_remove_one on a VF */
+ u32 slave_read =
+ swab32(readl(&mlx4_priv(dev)->mfunc.comm->slave_read));
+
+ if (mlx4_comm_internal_err(slave_read)) {
+ mlx4_dbg(dev, "%s: comm channel is down, entering error state.\n",
+ __func__);
+ mlx4_enter_error_state(dev->persist);
+ }
+ }
mutex_lock(&intf_mutex);
list_for_each_entry(intf, &intf_list, list)
void mlx4_srq_event(struct mlx4_dev *dev, u32 srqn, int event_type);
void mlx4_enter_error_state(struct mlx4_dev_persistent *persist);
+int mlx4_comm_internal_err(u32 slave_read);
int mlx4_SENSE_PORT(struct mlx4_dev *dev, int port,
enum mlx4_port_type *type);
int mlx4_en_try_alloc_resources(struct mlx4_en_priv *priv,
struct mlx4_en_priv *tmp,
- struct mlx4_en_port_profile *prof);
+ struct mlx4_en_port_profile *prof,
+ bool carry_xdp_prog);
void mlx4_en_safe_replace_resources(struct mlx4_en_priv *priv,
struct mlx4_en_priv *tmp);
if (cmd->cmdif_rev > CMD_IF_REV) {
dev_err(&dev->pdev->dev, "driver does not support command interface version. driver %d, firmware %d\n",
CMD_IF_REV, cmd->cmdif_rev);
- err = -ENOTSUPP;
+ err = -EOPNOTSUPP;
goto err_free_page;
}
int mlx5e_modify_rqs_vsd(struct mlx5e_priv *priv, bool vsd);
int mlx5e_redirect_rqt(struct mlx5e_priv *priv, u32 rqtn, int sz, int ix);
-void mlx5e_build_tir_ctx_hash(void *tirc, struct mlx5e_priv *priv);
+void mlx5e_build_indir_tir_ctx_hash(struct mlx5e_priv *priv, void *tirc,
+ enum mlx5e_traffic_types tt);
int mlx5e_open_locked(struct net_device *netdev);
int mlx5e_close_locked(struct net_device *netdev);
static inline int mlx5e_arfs_enable(struct mlx5e_priv *priv)
{
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
}
static inline int mlx5e_arfs_disable(struct mlx5e_priv *priv)
{
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
}
#else
int mlx5e_arfs_create_tables(struct mlx5e_priv *priv);
int i;
if (!MLX5_CAP_GEN(priv->mdev, ets))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
ets->ets_cap = mlx5_max_tc(priv->mdev) + 1;
for (i = 0; i < ets->ets_cap; i++) {
int err;
if (!MLX5_CAP_GEN(priv->mdev, ets))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
err = mlx5e_dbcnl_validate_ets(netdev, ets);
if (err)
struct mlx5_core_dev *mdev = priv->mdev;
struct ieee_ets ets;
struct ieee_pfc pfc;
- int err = -ENOTSUPP;
+ int err = -EOPNOTSUPP;
int i;
if (!MLX5_CAP_GEN(mdev, ets))
struct mlx5e_priv *priv = netdev_priv(netdev);
struct mlx5_core_dev *mdev = priv->mdev;
+ if (!MLX5_CAP_GEN(priv->mdev, ets)) {
+ netdev_err(netdev, "%s, ets is not supported\n", __func__);
+ return;
+ }
+
if (priority >= CEE_DCBX_MAX_PRIO) {
netdev_err(netdev,
"%s, priority is out of range\n", __func__);
struct ethtool_channels *ch)
{
struct mlx5e_priv *priv = netdev_priv(dev);
- int ncv = mlx5e_get_max_num_channels(priv->mdev);
unsigned int count = ch->combined_count;
bool arfs_enabled;
bool was_opened;
__func__);
return -EINVAL;
}
- if (ch->rx_count || ch->tx_count) {
- netdev_info(dev, "%s: separate rx/tx count not supported\n",
- __func__);
- return -EINVAL;
- }
- if (count > ncv) {
- netdev_info(dev, "%s: count (%d) > max (%d)\n",
- __func__, count, ncv);
- return -EINVAL;
- }
if (priv->params.num_channels == count)
return 0;
struct mlx5e_priv *priv = netdev_priv(netdev);
if (!MLX5_CAP_GEN(priv->mdev, cq_moderation))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
coal->rx_coalesce_usecs = priv->params.rx_cq_moderation.usec;
coal->rx_max_coalesced_frames = priv->params.rx_cq_moderation.pkts;
int i;
if (!MLX5_CAP_GEN(mdev, cq_moderation))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
mutex_lock(&priv->state_lock);
static void mlx5e_modify_tirs_hash(struct mlx5e_priv *priv, void *in, int inlen)
{
- struct mlx5_core_dev *mdev = priv->mdev;
void *tirc = MLX5_ADDR_OF(modify_tir_in, in, ctx);
- int i;
+ struct mlx5_core_dev *mdev = priv->mdev;
+ int ctxlen = MLX5_ST_SZ_BYTES(tirc);
+ int tt;
MLX5_SET(modify_tir_in, in, bitmask.hash, 1);
- mlx5e_build_tir_ctx_hash(tirc, priv);
- for (i = 0; i < MLX5E_NUM_INDIR_TIRS; i++)
- mlx5_core_modify_tir(mdev, priv->indir_tir[i].tirn, in, inlen);
+ for (tt = 0; tt < MLX5E_NUM_INDIR_TIRS; tt++) {
+ memset(tirc, 0, ctxlen);
+ mlx5e_build_indir_tir_ctx_hash(priv, tirc, tt);
+ mlx5_core_modify_tir(mdev, priv->indir_tir[tt].tirn, in, inlen);
+ }
}
static int mlx5e_set_rxfh(struct net_device *dev, const u32 *indir,
{
struct mlx5e_priv *priv = netdev_priv(dev);
int inlen = MLX5_ST_SZ_BYTES(modify_tir_in);
+ bool hash_changed = false;
void *in;
if ((hfunc != ETH_RSS_HASH_NO_CHANGE) &&
mlx5e_redirect_rqt(priv, rqtn, MLX5E_INDIR_RQT_SIZE, 0);
}
- if (key)
+ if (hfunc != ETH_RSS_HASH_NO_CHANGE &&
+ hfunc != priv->params.rss_hfunc) {
+ priv->params.rss_hfunc = hfunc;
+ hash_changed = true;
+ }
+
+ if (key) {
memcpy(priv->params.toeplitz_hash_key, key,
sizeof(priv->params.toeplitz_hash_key));
+ hash_changed = hash_changed ||
+ priv->params.rss_hfunc == ETH_RSS_HASH_TOP;
+ }
- if (hfunc != ETH_RSS_HASH_NO_CHANGE)
- priv->params.rss_hfunc = hfunc;
-
- mlx5e_modify_tirs_hash(priv, in, inlen);
+ if (hash_changed)
+ mlx5e_modify_tirs_hash(priv, in, inlen);
mutex_unlock(&priv->state_lock);
u32 mlx5_wol_mode;
if (!wol_supported)
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
if (wol->wolopts & ~wol_supported)
return -EINVAL;
if (rx_cq_period_mode == MLX5_CQ_PERIOD_MODE_START_FROM_CQE &&
!MLX5_CAP_GEN(mdev, cq_period_start_from_cqe))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
if (!rx_mode_changed)
return 0;
bool reset;
if (!MLX5_CAP_GEN(mdev, cqe_compression))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
if (enable && priv->tstamp.hwtstamp_config.rx_filter != HWTSTAMP_FILTER_NONE) {
netdev_err(netdev, "Can't enable cqe compression while timestamping is enabled.\n");
MLX5_FLOW_NAMESPACE_KERNEL);
if (!priv->fs.ns)
- return -EINVAL;
+ return -EOPNOTSUPP;
err = mlx5e_arfs_create_tables(priv);
if (err) {
ns = mlx5_get_flow_namespace(priv->mdev,
MLX5_FLOW_NAMESPACE_ETHTOOL);
if (!ns)
- return ERR_PTR(-ENOTSUPP);
+ return ERR_PTR(-EOPNOTSUPP);
table_size = min_t(u32, BIT(MLX5_CAP_FLOWTABLE(priv->mdev,
flow_table_properties_nic_receive.log_max_ft_size)),
MLX5_SET(tirc, tirc, lro_timeout_period_usecs, priv->params.lro_timeout);
}
-void mlx5e_build_tir_ctx_hash(void *tirc, struct mlx5e_priv *priv)
+void mlx5e_build_indir_tir_ctx_hash(struct mlx5e_priv *priv, void *tirc,
+ enum mlx5e_traffic_types tt)
{
+ void *hfso = MLX5_ADDR_OF(tirc, tirc, rx_hash_field_selector_outer);
+
+#define MLX5_HASH_IP (MLX5_HASH_FIELD_SEL_SRC_IP |\
+ MLX5_HASH_FIELD_SEL_DST_IP)
+
+#define MLX5_HASH_IP_L4PORTS (MLX5_HASH_FIELD_SEL_SRC_IP |\
+ MLX5_HASH_FIELD_SEL_DST_IP |\
+ MLX5_HASH_FIELD_SEL_L4_SPORT |\
+ MLX5_HASH_FIELD_SEL_L4_DPORT)
+
+#define MLX5_HASH_IP_IPSEC_SPI (MLX5_HASH_FIELD_SEL_SRC_IP |\
+ MLX5_HASH_FIELD_SEL_DST_IP |\
+ MLX5_HASH_FIELD_SEL_IPSEC_SPI)
+
MLX5_SET(tirc, tirc, rx_hash_fn,
mlx5e_rx_hash_fn(priv->params.rss_hfunc));
if (priv->params.rss_hfunc == ETH_RSS_HASH_TOP) {
MLX5_SET(tirc, tirc, rx_hash_symmetric, 1);
memcpy(rss_key, priv->params.toeplitz_hash_key, len);
}
+
+ switch (tt) {
+ case MLX5E_TT_IPV4_TCP:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV4);
+ MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
+ MLX5_L4_PROT_TYPE_TCP);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_L4PORTS);
+ break;
+
+ case MLX5E_TT_IPV6_TCP:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV6);
+ MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
+ MLX5_L4_PROT_TYPE_TCP);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_L4PORTS);
+ break;
+
+ case MLX5E_TT_IPV4_UDP:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV4);
+ MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
+ MLX5_L4_PROT_TYPE_UDP);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_L4PORTS);
+ break;
+
+ case MLX5E_TT_IPV6_UDP:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV6);
+ MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
+ MLX5_L4_PROT_TYPE_UDP);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_L4PORTS);
+ break;
+
+ case MLX5E_TT_IPV4_IPSEC_AH:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV4);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_IPSEC_SPI);
+ break;
+
+ case MLX5E_TT_IPV6_IPSEC_AH:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV6);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_IPSEC_SPI);
+ break;
+
+ case MLX5E_TT_IPV4_IPSEC_ESP:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV4);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_IPSEC_SPI);
+ break;
+
+ case MLX5E_TT_IPV6_IPSEC_ESP:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV6);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP_IPSEC_SPI);
+ break;
+
+ case MLX5E_TT_IPV4:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV4);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP);
+ break;
+
+ case MLX5E_TT_IPV6:
+ MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
+ MLX5_L3_PROT_TYPE_IPV6);
+ MLX5_SET(rx_hash_field_select, hfso, selected_fields,
+ MLX5_HASH_IP);
+ break;
+ default:
+ WARN_ONCE(true, "%s: bad traffic type!\n", __func__);
+ }
}
static int mlx5e_modify_tirs_lro(struct mlx5e_priv *priv)
static void mlx5e_build_indir_tir_ctx(struct mlx5e_priv *priv, u32 *tirc,
enum mlx5e_traffic_types tt)
{
- void *hfso = MLX5_ADDR_OF(tirc, tirc, rx_hash_field_selector_outer);
-
MLX5_SET(tirc, tirc, transport_domain, priv->mdev->mlx5e_res.td.tdn);
-#define MLX5_HASH_IP (MLX5_HASH_FIELD_SEL_SRC_IP |\
- MLX5_HASH_FIELD_SEL_DST_IP)
-
-#define MLX5_HASH_IP_L4PORTS (MLX5_HASH_FIELD_SEL_SRC_IP |\
- MLX5_HASH_FIELD_SEL_DST_IP |\
- MLX5_HASH_FIELD_SEL_L4_SPORT |\
- MLX5_HASH_FIELD_SEL_L4_DPORT)
-
-#define MLX5_HASH_IP_IPSEC_SPI (MLX5_HASH_FIELD_SEL_SRC_IP |\
- MLX5_HASH_FIELD_SEL_DST_IP |\
- MLX5_HASH_FIELD_SEL_IPSEC_SPI)
-
mlx5e_build_tir_ctx_lro(tirc, priv);
MLX5_SET(tirc, tirc, disp_type, MLX5_TIRC_DISP_TYPE_INDIRECT);
MLX5_SET(tirc, tirc, indirect_table, priv->indir_rqt.rqtn);
- mlx5e_build_tir_ctx_hash(tirc, priv);
-
- switch (tt) {
- case MLX5E_TT_IPV4_TCP:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV4);
- MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
- MLX5_L4_PROT_TYPE_TCP);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_L4PORTS);
- break;
-
- case MLX5E_TT_IPV6_TCP:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV6);
- MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
- MLX5_L4_PROT_TYPE_TCP);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_L4PORTS);
- break;
-
- case MLX5E_TT_IPV4_UDP:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV4);
- MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
- MLX5_L4_PROT_TYPE_UDP);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_L4PORTS);
- break;
-
- case MLX5E_TT_IPV6_UDP:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV6);
- MLX5_SET(rx_hash_field_select, hfso, l4_prot_type,
- MLX5_L4_PROT_TYPE_UDP);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_L4PORTS);
- break;
-
- case MLX5E_TT_IPV4_IPSEC_AH:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV4);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_IPSEC_SPI);
- break;
-
- case MLX5E_TT_IPV6_IPSEC_AH:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV6);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_IPSEC_SPI);
- break;
-
- case MLX5E_TT_IPV4_IPSEC_ESP:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV4);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_IPSEC_SPI);
- break;
-
- case MLX5E_TT_IPV6_IPSEC_ESP:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV6);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP_IPSEC_SPI);
- break;
-
- case MLX5E_TT_IPV4:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV4);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP);
- break;
-
- case MLX5E_TT_IPV6:
- MLX5_SET(rx_hash_field_select, hfso, l3_prot_type,
- MLX5_L3_PROT_TYPE_IPV6);
- MLX5_SET(rx_hash_field_select, hfso, selected_fields,
- MLX5_HASH_IP);
- break;
- default:
- WARN_ONCE(true,
- "mlx5e_build_indir_tir_ctx: bad traffic type!\n");
- }
+ mlx5e_build_indir_tir_ctx_hash(priv, tirc, tt);
}
static void mlx5e_build_direct_tir_ctx(struct mlx5e_priv *priv, u32 *tirc,
static int mlx5e_check_required_hca_cap(struct mlx5_core_dev *mdev)
{
if (MLX5_CAP_GEN(mdev, port_type) != MLX5_CAP_PORT_TYPE_ETH)
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
if (!MLX5_CAP_GEN(mdev, eth_net_offloads) ||
!MLX5_CAP_GEN(mdev, nic_flow_table) ||
!MLX5_CAP_ETH(mdev, csum_cap) ||
< 3) {
mlx5_core_warn(mdev,
"Not creating net device, some required device capabilities are missing\n");
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
}
if (!MLX5_CAP_ETH(mdev, self_lb_en_modifiable))
mlx5_core_warn(mdev, "Self loop back prevention is not supported\n");
return false;
}
+ if (unlikely(page_is_pfmemalloc(dma_info->page)))
+ return false;
+
cache->page_cache[cache->tail] = *dma_info;
cache->tail = tail_next;
return true;
__be32 *saddr,
int *out_ttl)
{
+ struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
struct rtable *rt;
struct neighbour *n = NULL;
int ttl;
#else
return -EOPNOTSUPP;
#endif
-
- if (!switchdev_port_same_parent_id(priv->netdev, rt->dst.dev)) {
- pr_warn("%s: can't offload, devices not on same HW e-switch\n", __func__);
- ip_rt_put(rt);
- return -EOPNOTSUPP;
- }
+ /* if the egress device isn't on the same HW e-switch, we use the uplink */
+ if (!switchdev_port_same_parent_id(priv->netdev, rt->dst.dev))
+ *out_dev = mlx5_eswitch_get_uplink_netdev(esw);
+ else
+ *out_dev = rt->dst.dev;
ttl = ip4_dst_hoplimit(&rt->dst);
n = dst_neigh_lookup(&rt->dst, &fl4->daddr);
*out_n = n;
*saddr = fl4->saddr;
*out_ttl = ttl;
- *out_dev = rt->dst.dev;
return 0;
}
if (!MLX5_CAP_ESW(dev, vport_cvlan_strip) ||
!MLX5_CAP_ESW(dev, vport_cvlan_insert_if_not_exist))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
esw_debug(dev, "Set Vport[%d] VLAN %d qos %d set=%x\n",
vport, vlan, qos, set_flags);
root_ns = mlx5_get_flow_namespace(dev, MLX5_FLOW_NAMESPACE_FDB);
if (!root_ns) {
esw_warn(dev, "Failed to get FDB flow namespace\n");
- return -ENOMEM;
+ return -EOPNOTSUPP;
}
flow_group_in = mlx5_vzalloc(inlen);
root_ns = mlx5_get_flow_namespace(dev, MLX5_FLOW_NAMESPACE_ESW_EGRESS);
if (!root_ns) {
esw_warn(dev, "Failed to get E-Switch egress flow namespace\n");
- return -EIO;
+ return -EOPNOTSUPP;
}
flow_group_in = mlx5_vzalloc(inlen);
root_ns = mlx5_get_flow_namespace(dev, MLX5_FLOW_NAMESPACE_ESW_INGRESS);
if (!root_ns) {
esw_warn(dev, "Failed to get E-Switch ingress flow namespace\n");
- return -EIO;
+ return -EOPNOTSUPP;
}
flow_group_in = mlx5_vzalloc(inlen);
if (!MLX5_CAP_GEN(esw->dev, eswitch_flow_table) ||
!MLX5_CAP_ESW_FLOWTABLE_FDB(esw->dev, ft_support)) {
esw_warn(esw->dev, "E-Switch FDB is not supported, aborting ...\n");
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
}
if (!MLX5_CAP_ESW_INGRESS_ACL(esw->dev, ft_support))
return 0;
out_notsupp:
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
}
int mlx5_eswitch_add_vlan_action(struct mlx5_eswitch *esw,
root_ns = mlx5_get_flow_namespace(dev, MLX5_FLOW_NAMESPACE_FDB);
if (!root_ns) {
esw_warn(dev, "Failed to get FDB flow namespace\n");
+ err = -EOPNOTSUPP;
goto ns_err;
}
ns = mlx5_get_flow_namespace(dev, MLX5_FLOW_NAMESPACE_OFFLOADS);
if (!ns) {
esw_warn(esw->dev, "Failed to get offloads flow namespace\n");
- return -ENOMEM;
+ return -EOPNOTSUPP;
}
ft_offloads = mlx5_create_flow_table(ns, 0, dev->priv.sriov.num_vfs + 2, 0, 0);
esw_warn(esw->dev, "Failed setting eswitch to offloads, err %d\n", err);
err1 = mlx5_eswitch_enable_sriov(esw, num_vfs, SRIOV_LEGACY);
if (err1)
- esw_warn(esw->dev, "Failed setting eswitch back to legacy, err %d\n", err);
+ esw_warn(esw->dev, "Failed setting eswitch back to legacy, err %d\n", err1);
}
if (esw->offloads.inline_mode == MLX5_INLINE_MODE_NONE) {
if (mlx5_eswitch_inline_mode_get(esw,
int vport;
int err;
+ /* disable PF RoCE so missed packets don't go through RoCE steering */
+ mlx5_dev_list_lock();
+ mlx5_remove_dev_by_protocol(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
+ mlx5_dev_list_unlock();
+
err = esw_create_offloads_fdb_table(esw, nvports);
if (err)
- return err;
+ goto create_fdb_err;
err = esw_create_offloads_table(esw);
if (err)
goto err_reps;
}
- /* disable PF RoCE so missed packets don't go through RoCE steering */
- mlx5_dev_list_lock();
- mlx5_remove_dev_by_protocol(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
- mlx5_dev_list_unlock();
-
return 0;
err_reps:
create_ft_err:
esw_destroy_offloads_fdb_table(esw);
+
+create_fdb_err:
+ /* enable back PF RoCE */
+ mlx5_dev_list_lock();
+ mlx5_add_dev_by_protocol(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
+ mlx5_dev_list_unlock();
+
return err;
}
{
int err, err1, num_vfs = esw->dev->priv.sriov.num_vfs;
- /* enable back PF RoCE */
- mlx5_dev_list_lock();
- mlx5_add_dev_by_protocol(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
- mlx5_dev_list_unlock();
-
mlx5_eswitch_disable_sriov(esw);
err = mlx5_eswitch_enable_sriov(esw, num_vfs, SRIOV_LEGACY);
if (err) {
esw_warn(esw->dev, "Failed setting eswitch back to offloads, err %d\n", err);
}
+ /* enable back PF RoCE */
+ mlx5_dev_list_lock();
+ mlx5_add_dev_by_protocol(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
+ mlx5_dev_list_unlock();
+
return err;
}
flow_table_properties_nic_receive.
flow_modify_en);
if (!atomic_mod_cap)
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
opmod = 1;
return mlx5_cmd_set_fte(dev, opmod, modify_mask, ft, group_id, fte);
struct mlx5_flow_table *ft;
ns = mlx5_get_flow_namespace(steering->dev, MLX5_FLOW_NAMESPACE_ANCHOR);
- if (!ns)
+ if (WARN_ON(!ns))
return -EINVAL;
ft = mlx5_create_flow_table(ns, ANCHOR_PRIO, ANCHOR_SIZE, ANCHOR_LEVEL, 0);
if (IS_ERR(ft)) {
return 0;
}
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
}
u32 out[MLX5_ST_SZ_DW(qtct_reg)];
if (!MLX5_CAP_GEN(mdev, ets))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
return mlx5_core_access_reg(mdev, in, inlen, out, sizeof(out),
MLX5_REG_QETCR, 0, 1);
u32 in[MLX5_ST_SZ_DW(qtct_reg)];
if (!MLX5_CAP_GEN(mdev, ets))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
memset(in, 0, sizeof(in));
return mlx5_core_access_reg(mdev, in, sizeof(in), out, outlen,
if (!MLX5_CAP_GEN(mdev, vport_group_manager))
return -EACCES;
if (!MLX5_CAP_ESW(mdev, nic_vport_node_guid_modify))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
in = mlx5_vzalloc(inlen);
if (!in)
static int
mlxsw_sp_nexthop_group_mac_update(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_nexthop_group *nh_grp)
+ struct mlxsw_sp_nexthop_group *nh_grp,
+ bool reallocate)
{
u32 adj_index = nh_grp->adj_index; /* base */
struct mlxsw_sp_nexthop *nh;
continue;
}
- if (nh->update) {
+ if (nh->update || reallocate) {
err = mlxsw_sp_nexthop_mac_update(mlxsw_sp,
adj_index, nh);
if (err)
/* Nothing was added or removed, so no need to reallocate. Just
* update MAC on existing adjacency indexes.
*/
- err = mlxsw_sp_nexthop_group_mac_update(mlxsw_sp, nh_grp);
+ err = mlxsw_sp_nexthop_group_mac_update(mlxsw_sp, nh_grp,
+ false);
if (err) {
dev_warn(mlxsw_sp->bus_info->dev, "Failed to update neigh MAC in adjacency table.\n");
goto set_trap;
nh_grp->adj_index_valid = 1;
nh_grp->adj_index = adj_index;
nh_grp->ecmp_size = ecmp_size;
- err = mlxsw_sp_nexthop_group_mac_update(mlxsw_sp, nh_grp);
+ err = mlxsw_sp_nexthop_group_mac_update(mlxsw_sp, nh_grp, true);
if (err) {
dev_warn(mlxsw_sp->bus_info->dev, "Failed to update neigh MAC in adjacency table.\n");
goto set_trap;
list_del(&p_pkt->list_entry);
b_last_packet = list_empty(&p_tx->active_descq);
list_add_tail(&p_pkt->list_entry, &p_tx->free_descq);
- if (p_ll2_conn->conn_type == QED_LL2_TYPE_ISCSI_OOO) {
+ if (p_ll2_conn->conn.conn_type == QED_LL2_TYPE_ISCSI_OOO) {
struct qed_ooo_buffer *p_buffer;
p_buffer = (struct qed_ooo_buffer *)p_pkt->cookie;
b_last_frag =
p_tx->cur_completing_bd_idx == p_pkt->bd_used;
tx_frag = p_pkt->bds_set[0].tx_frag;
- if (p_ll2_conn->gsi_enable)
+ if (p_ll2_conn->conn.gsi_enable)
qed_ll2b_release_tx_gsi_packet(p_hwfn,
p_ll2_conn->
my_id,
spin_unlock_irqrestore(&p_tx->lock, flags);
tx_frag = p_pkt->bds_set[0].tx_frag;
- if (p_ll2_conn->gsi_enable)
+ if (p_ll2_conn->conn.gsi_enable)
qed_ll2b_complete_tx_gsi_packet(p_hwfn,
p_ll2_conn->my_id,
p_pkt->cookie,
list_move_tail(&p_pkt->list_entry, &p_rx->free_descq);
- if (p_ll2_conn->conn_type == QED_LL2_TYPE_ISCSI_OOO) {
+ if (p_ll2_conn->conn.conn_type == QED_LL2_TYPE_ISCSI_OOO) {
struct qed_ooo_buffer *p_buffer;
p_buffer = (struct qed_ooo_buffer *)p_pkt->cookie;
rc = qed_ll2_prepare_tx_packet(p_hwfn, p_ll2_conn->my_id, 1,
p_buffer->vlan, bd_flags,
l4_hdr_offset_w,
- p_ll2_conn->tx_dest, 0,
+ p_ll2_conn->conn.tx_dest, 0,
first_frag,
p_buffer->packet_length,
p_buffer, true);
u16 buf_idx;
int rc = 0;
- if (p_ll2_info->conn_type != QED_LL2_TYPE_ISCSI_OOO)
+ if (p_ll2_info->conn.conn_type != QED_LL2_TYPE_ISCSI_OOO)
return rc;
if (!rx_num_ooo_buffers)
qed_ll2_establish_connection_ooo(struct qed_hwfn *p_hwfn,
struct qed_ll2_info *p_ll2_conn)
{
- if (p_ll2_conn->conn_type != QED_LL2_TYPE_ISCSI_OOO)
+ if (p_ll2_conn->conn.conn_type != QED_LL2_TYPE_ISCSI_OOO)
return;
qed_ooo_release_all_isles(p_hwfn, p_hwfn->p_ooo_info);
{
struct qed_ooo_buffer *p_buffer;
- if (p_ll2_conn->conn_type != QED_LL2_TYPE_ISCSI_OOO)
+ if (p_ll2_conn->conn.conn_type != QED_LL2_TYPE_ISCSI_OOO)
return;
qed_ooo_release_all_isles(p_hwfn, p_hwfn->p_ooo_info);
{
struct qed_hwfn *hwfn = QED_LEADING_HWFN(cdev);
u8 *handle = &hwfn->pf_params.iscsi_pf_params.ll2_ooo_queue_id;
- struct qed_ll2_info *ll2_info;
+ struct qed_ll2_conn ll2_info;
int rc;
- ll2_info = kzalloc(sizeof(*ll2_info), GFP_KERNEL);
- if (!ll2_info)
- return -ENOMEM;
- ll2_info->conn_type = QED_LL2_TYPE_ISCSI_OOO;
- ll2_info->mtu = params->mtu;
- ll2_info->rx_drop_ttl0_flg = params->drop_ttl0_packets;
- ll2_info->rx_vlan_removal_en = params->rx_vlan_stripping;
- ll2_info->tx_tc = OOO_LB_TC;
- ll2_info->tx_dest = CORE_TX_DEST_LB;
-
- rc = qed_ll2_acquire_connection(hwfn, ll2_info,
+ ll2_info.conn_type = QED_LL2_TYPE_ISCSI_OOO;
+ ll2_info.mtu = params->mtu;
+ ll2_info.rx_drop_ttl0_flg = params->drop_ttl0_packets;
+ ll2_info.rx_vlan_removal_en = params->rx_vlan_stripping;
+ ll2_info.tx_tc = OOO_LB_TC;
+ ll2_info.tx_dest = CORE_TX_DEST_LB;
+
+ rc = qed_ll2_acquire_connection(hwfn, &ll2_info,
QED_LL2_RX_SIZE, QED_LL2_TX_SIZE,
handle);
- kfree(ll2_info);
if (rc) {
DP_INFO(cdev, "Failed to acquire LL2 OOO connection\n");
goto out;
struct qed_ll2_info *p_ll2_conn,
u8 action_on_error)
{
- enum qed_ll2_conn_type conn_type = p_ll2_conn->conn_type;
+ enum qed_ll2_conn_type conn_type = p_ll2_conn->conn.conn_type;
struct qed_ll2_rx_queue *p_rx = &p_ll2_conn->rx_queue;
struct core_rx_start_ramrod_data *p_ramrod = NULL;
struct qed_spq_entry *p_ent = NULL;
p_ramrod->sb_index = p_rx->rx_sb_index;
p_ramrod->complete_event_flg = 1;
- p_ramrod->mtu = cpu_to_le16(p_ll2_conn->mtu);
+ p_ramrod->mtu = cpu_to_le16(p_ll2_conn->conn.mtu);
DMA_REGPAIR_LE(p_ramrod->bd_base,
p_rx->rxq_chain.p_phys_addr);
cqe_pbl_size = (u16)qed_chain_get_page_cnt(&p_rx->rcq_chain);
DMA_REGPAIR_LE(p_ramrod->cqe_pbl_addr,
qed_chain_get_pbl_phys(&p_rx->rcq_chain));
- p_ramrod->drop_ttl0_flg = p_ll2_conn->rx_drop_ttl0_flg;
- p_ramrod->inner_vlan_removal_en = p_ll2_conn->rx_vlan_removal_en;
+ p_ramrod->drop_ttl0_flg = p_ll2_conn->conn.rx_drop_ttl0_flg;
+ p_ramrod->inner_vlan_removal_en = p_ll2_conn->conn.rx_vlan_removal_en;
p_ramrod->queue_id = p_ll2_conn->queue_id;
p_ramrod->main_func_queue = (conn_type == QED_LL2_TYPE_ISCSI_OOO) ? 0
: 1;
}
p_ramrod->action_on_error.error_type = action_on_error;
- p_ramrod->gsi_offload_flag = p_ll2_conn->gsi_enable;
+ p_ramrod->gsi_offload_flag = p_ll2_conn->conn.gsi_enable;
return qed_spq_post(p_hwfn, p_ent, NULL);
}
static int qed_sp_ll2_tx_queue_start(struct qed_hwfn *p_hwfn,
struct qed_ll2_info *p_ll2_conn)
{
- enum qed_ll2_conn_type conn_type = p_ll2_conn->conn_type;
+ enum qed_ll2_conn_type conn_type = p_ll2_conn->conn.conn_type;
struct qed_ll2_tx_queue *p_tx = &p_ll2_conn->tx_queue;
struct core_tx_start_ramrod_data *p_ramrod = NULL;
struct qed_spq_entry *p_ent = NULL;
if (!QED_LL2_TX_REGISTERED(p_ll2_conn))
return 0;
- if (p_ll2_conn->conn_type == QED_LL2_TYPE_ISCSI_OOO)
+ if (p_ll2_conn->conn.conn_type == QED_LL2_TYPE_ISCSI_OOO)
p_ll2_conn->tx_stats_en = 0;
else
p_ll2_conn->tx_stats_en = 1;
p_ramrod->sb_id = cpu_to_le16(qed_int_get_sp_sb_id(p_hwfn));
p_ramrod->sb_index = p_tx->tx_sb_index;
- p_ramrod->mtu = cpu_to_le16(p_ll2_conn->mtu);
+ p_ramrod->mtu = cpu_to_le16(p_ll2_conn->conn.mtu);
p_ramrod->stats_en = p_ll2_conn->tx_stats_en;
p_ramrod->stats_id = p_ll2_conn->tx_stats_id;
p_ramrod->pbl_size = cpu_to_le16(pbl_size);
memset(&pq_params, 0, sizeof(pq_params));
- pq_params.core.tc = p_ll2_conn->tx_tc;
+ pq_params.core.tc = p_ll2_conn->conn.tx_tc;
pq_id = qed_get_qm_pq(p_hwfn, PROTOCOLID_CORE, &pq_params);
p_ramrod->qm_pq_id = cpu_to_le16(pq_id);
DP_NOTICE(p_hwfn, "Unknown connection type: %d\n", conn_type);
}
- p_ramrod->gsi_offload_flag = p_ll2_conn->gsi_enable;
+ p_ramrod->gsi_offload_flag = p_ll2_conn->conn.gsi_enable;
return qed_spq_post(p_hwfn, p_ent, NULL);
}
DP_VERBOSE(p_hwfn, QED_MSG_LL2,
"Allocated LL2 Rxq [Type %08x] with 0x%08x buffers\n",
- p_ll2_info->conn_type, rx_num_desc);
+ p_ll2_info->conn.conn_type, rx_num_desc);
out:
return rc;
DP_VERBOSE(p_hwfn, QED_MSG_LL2,
"Allocated LL2 Txq [Type %08x] with 0x%08x buffers\n",
- p_ll2_info->conn_type, tx_num_desc);
+ p_ll2_info->conn.conn_type, tx_num_desc);
out:
if (rc)
}
int qed_ll2_acquire_connection(struct qed_hwfn *p_hwfn,
- struct qed_ll2_info *p_params,
+ struct qed_ll2_conn *p_params,
u16 rx_num_desc,
u16 tx_num_desc,
u8 *p_connection_handle)
if (!p_ll2_info)
return -EBUSY;
- p_ll2_info->conn_type = p_params->conn_type;
- p_ll2_info->mtu = p_params->mtu;
- p_ll2_info->rx_drop_ttl0_flg = p_params->rx_drop_ttl0_flg;
- p_ll2_info->rx_vlan_removal_en = p_params->rx_vlan_removal_en;
- p_ll2_info->tx_tc = p_params->tx_tc;
- p_ll2_info->tx_dest = p_params->tx_dest;
- p_ll2_info->ai_err_packet_too_big = p_params->ai_err_packet_too_big;
- p_ll2_info->ai_err_no_buf = p_params->ai_err_no_buf;
- p_ll2_info->gsi_enable = p_params->gsi_enable;
+ p_ll2_info->conn = *p_params;
rc = qed_ll2_acquire_connection_rx(p_hwfn, p_ll2_info, rx_num_desc);
if (rc)
SET_FIELD(action_on_error,
CORE_RX_ACTION_ON_ERROR_PACKET_TOO_BIG,
- p_ll2_conn->ai_err_packet_too_big);
+ p_ll2_conn->conn.ai_err_packet_too_big);
SET_FIELD(action_on_error,
- CORE_RX_ACTION_ON_ERROR_NO_BUFF, p_ll2_conn->ai_err_no_buf);
+ CORE_RX_ACTION_ON_ERROR_NO_BUFF, p_ll2_conn->conn.ai_err_no_buf);
return qed_sp_ll2_rx_queue_start(p_hwfn, p_ll2_conn, action_on_error);
}
"LL2 [q 0x%02x cid 0x%08x type 0x%08x] Tx Producer at [0x%04x] - set with a %04x bytes %02x BDs buffer at %08x:%08x\n",
p_ll2->queue_id,
p_ll2->cid,
- p_ll2->conn_type,
+ p_ll2->conn.conn_type,
prod_idx,
first_frag_len,
num_of_bds,
(NETIF_MSG_TX_QUEUED | QED_MSG_LL2),
"LL2 [q 0x%02x cid 0x%08x type 0x%08x] Doorbelled [producer 0x%04x]\n",
p_ll2_conn->queue_id,
- p_ll2_conn->cid, p_ll2_conn->conn_type, db_msg.spq_prod);
+ p_ll2_conn->cid, p_ll2_conn->conn.conn_type, db_msg.spq_prod);
}
int qed_ll2_prepare_tx_packet(struct qed_hwfn *p_hwfn,
qed_ll2_rxq_flush(p_hwfn, connection_handle);
}
- if (p_ll2_conn->conn_type == QED_LL2_TYPE_ISCSI_OOO)
+ if (p_ll2_conn->conn.conn_type == QED_LL2_TYPE_ISCSI_OOO)
qed_ooo_release_all_isles(p_hwfn, p_hwfn->p_ooo_info);
return rc;
static int qed_ll2_start(struct qed_dev *cdev, struct qed_ll2_params *params)
{
- struct qed_ll2_info ll2_info;
+ struct qed_ll2_conn ll2_info;
struct qed_ll2_buffer *buffer, *tmp_buffer;
enum qed_ll2_conn_type conn_type;
struct qed_ptt *p_ptt;
/* Prepare the temporary ll2 information */
memset(&ll2_info, 0, sizeof(ll2_info));
+
ll2_info.conn_type = conn_type;
ll2_info.mtu = params->mtu;
ll2_info.rx_drop_ttl0_flg = params->drop_ttl0_packets;
}
ether_addr_copy(cdev->ll2_mac_address, params->ll2_mac_address);
-
return 0;
release_terminate_all:
bool b_completing_packet;
};
-struct qed_ll2_info {
- /* Lock protecting the state of LL2 */
- struct mutex mutex;
+struct qed_ll2_conn {
enum qed_ll2_conn_type conn_type;
- u32 cid;
- u8 my_id;
- u8 queue_id;
- u8 tx_stats_id;
- bool b_active;
u16 mtu;
u8 rx_drop_ttl0_flg;
u8 rx_vlan_removal_en;
enum core_tx_dest tx_dest;
enum core_error_handle ai_err_packet_too_big;
enum core_error_handle ai_err_no_buf;
+ u8 gsi_enable;
+};
+
+struct qed_ll2_info {
+ /* Lock protecting the state of LL2 */
+ struct mutex mutex;
+ struct qed_ll2_conn conn;
+ u32 cid;
+ u8 my_id;
+ u8 queue_id;
+ u8 tx_stats_id;
+ bool b_active;
u8 tx_stats_en;
struct qed_ll2_rx_queue rx_queue;
struct qed_ll2_tx_queue tx_queue;
- u8 gsi_enable;
};
/**
* @return 0 on success, failure otherwise
*/
int qed_ll2_acquire_connection(struct qed_hwfn *p_hwfn,
- struct qed_ll2_info *p_params,
+ struct qed_ll2_conn *p_params,
u16 rx_num_desc,
u16 tx_num_desc,
u8 *p_connection_handle);
{
struct qed_hwfn *hwfn = QED_LEADING_HWFN(cdev);
struct qed_roce_ll2_info *roce_ll2;
- struct qed_ll2_info ll2_params;
+ struct qed_ll2_conn ll2_params;
int rc;
if (!params) {
.get_mdio_data = ravb_get_mdio_data,
};
+/* Free TX skb function for AVB-IP */
+static int ravb_tx_free(struct net_device *ndev, int q, bool free_txed_only)
+{
+ struct ravb_private *priv = netdev_priv(ndev);
+ struct net_device_stats *stats = &priv->stats[q];
+ struct ravb_tx_desc *desc;
+ int free_num = 0;
+ int entry;
+ u32 size;
+
+ for (; priv->cur_tx[q] - priv->dirty_tx[q] > 0; priv->dirty_tx[q]++) {
+ bool txed;
+
+ entry = priv->dirty_tx[q] % (priv->num_tx_ring[q] *
+ NUM_TX_DESC);
+ desc = &priv->tx_ring[q][entry];
+ txed = desc->die_dt == DT_FEMPTY;
+ if (free_txed_only && !txed)
+ break;
+ /* Descriptor type must be checked before all other reads */
+ dma_rmb();
+ size = le16_to_cpu(desc->ds_tagl) & TX_DS;
+ /* Free the original skb. */
+ if (priv->tx_skb[q][entry / NUM_TX_DESC]) {
+ dma_unmap_single(ndev->dev.parent, le32_to_cpu(desc->dptr),
+ size, DMA_TO_DEVICE);
+ /* Last packet descriptor? */
+ if (entry % NUM_TX_DESC == NUM_TX_DESC - 1) {
+ entry /= NUM_TX_DESC;
+ dev_kfree_skb_any(priv->tx_skb[q][entry]);
+ priv->tx_skb[q][entry] = NULL;
+ if (txed)
+ stats->tx_packets++;
+ }
+ free_num++;
+ }
+ if (txed)
+ stats->tx_bytes += size;
+ desc->die_dt = DT_EEMPTY;
+ }
+ return free_num;
+}
+
/* Free skb's and DMA buffers for Ethernet AVB */
static void ravb_ring_free(struct net_device *ndev, int q)
{
kfree(priv->rx_skb[q]);
priv->rx_skb[q] = NULL;
- /* Free TX skb ringbuffer */
- if (priv->tx_skb[q]) {
- for (i = 0; i < priv->num_tx_ring[q]; i++)
- dev_kfree_skb(priv->tx_skb[q][i]);
- }
- kfree(priv->tx_skb[q]);
- priv->tx_skb[q] = NULL;
-
/* Free aligned TX buffers */
kfree(priv->tx_align[q]);
priv->tx_align[q] = NULL;
if (priv->rx_ring[q]) {
+ for (i = 0; i < priv->num_rx_ring[q]; i++) {
+ struct ravb_ex_rx_desc *desc = &priv->rx_ring[q][i];
+
+ if (!dma_mapping_error(ndev->dev.parent,
+ le32_to_cpu(desc->dptr)))
+ dma_unmap_single(ndev->dev.parent,
+ le32_to_cpu(desc->dptr),
+ PKT_BUF_SZ,
+ DMA_FROM_DEVICE);
+ }
ring_size = sizeof(struct ravb_ex_rx_desc) *
(priv->num_rx_ring[q] + 1);
dma_free_coherent(ndev->dev.parent, ring_size, priv->rx_ring[q],
}
if (priv->tx_ring[q]) {
+ ravb_tx_free(ndev, q, false);
+
ring_size = sizeof(struct ravb_tx_desc) *
(priv->num_tx_ring[q] * NUM_TX_DESC + 1);
dma_free_coherent(ndev->dev.parent, ring_size, priv->tx_ring[q],
priv->tx_desc_dma[q]);
priv->tx_ring[q] = NULL;
}
+
+ /* Free TX skb ringbuffer.
+ * SKBs are freed by ravb_tx_free() call above.
+ */
+ kfree(priv->tx_skb[q]);
+ priv->tx_skb[q] = NULL;
}
/* Format skb and descriptor buffer for Ethernet AVB */
return 0;
}
-/* Free TX skb function for AVB-IP */
-static int ravb_tx_free(struct net_device *ndev, int q)
-{
- struct ravb_private *priv = netdev_priv(ndev);
- struct net_device_stats *stats = &priv->stats[q];
- struct ravb_tx_desc *desc;
- int free_num = 0;
- int entry;
- u32 size;
-
- for (; priv->cur_tx[q] - priv->dirty_tx[q] > 0; priv->dirty_tx[q]++) {
- entry = priv->dirty_tx[q] % (priv->num_tx_ring[q] *
- NUM_TX_DESC);
- desc = &priv->tx_ring[q][entry];
- if (desc->die_dt != DT_FEMPTY)
- break;
- /* Descriptor type must be checked before all other reads */
- dma_rmb();
- size = le16_to_cpu(desc->ds_tagl) & TX_DS;
- /* Free the original skb. */
- if (priv->tx_skb[q][entry / NUM_TX_DESC]) {
- dma_unmap_single(ndev->dev.parent, le32_to_cpu(desc->dptr),
- size, DMA_TO_DEVICE);
- /* Last packet descriptor? */
- if (entry % NUM_TX_DESC == NUM_TX_DESC - 1) {
- entry /= NUM_TX_DESC;
- dev_kfree_skb_any(priv->tx_skb[q][entry]);
- priv->tx_skb[q][entry] = NULL;
- stats->tx_packets++;
- }
- free_num++;
- }
- stats->tx_bytes += size;
- desc->die_dt = DT_EEMPTY;
- }
- return free_num;
-}
-
static void ravb_get_tx_tstamp(struct net_device *ndev)
{
struct ravb_private *priv = netdev_priv(ndev);
spin_lock_irqsave(&priv->lock, flags);
/* Clear TX interrupt */
ravb_write(ndev, ~mask, TIS);
- ravb_tx_free(ndev, q);
+ ravb_tx_free(ndev, q, true);
netif_wake_subqueue(ndev, q);
mmiowb();
spin_unlock_irqrestore(&priv->lock, flags);
priv->cur_tx[q] += NUM_TX_DESC;
if (priv->cur_tx[q] - priv->dirty_tx[q] >
- (priv->num_tx_ring[q] - 1) * NUM_TX_DESC && !ravb_tx_free(ndev, q))
+ (priv->num_tx_ring[q] - 1) * NUM_TX_DESC &&
+ !ravb_tx_free(ndev, q, true))
netif_stop_subqueue(ndev, q);
exit:
{
void __iomem *ioaddr = hw->pcsr;
u32 intr_status = readl(ioaddr + GMAC_INT_STATUS);
+ u32 intr_mask = readl(ioaddr + GMAC_INT_MASK);
int ret = 0;
+ /* Discard masked bits */
+ intr_status &= ~intr_mask;
+
/* Not used events (e.g. MMC interrupts) are not handled. */
if ((intr_status & GMAC_INT_STATUS_MMCTIS))
x->mmc_tx_irq_n++;
if (of_phy_is_fixed_link(np))
of_phy_deregister_fixed_link(np);
of_node_put(plat->phy_node);
+ of_node_put(plat->mdio_node);
}
#else
struct plat_stmmacenet_data *
struct socket *sock0;
struct socket *sock1u;
- struct net *net;
struct net_device *dev;
unsigned int hash_size;
netdev_dbg(gtp->dev, "encap_recv sk=%p\n", sk);
- xnet = !net_eq(gtp->net, dev_net(gtp->dev));
+ xnet = !net_eq(sock_net(sk), dev_net(gtp->dev));
switch (udp_sk(sk)->encap_type) {
case UDP_ENCAP_GTP0:
pktinfo.fl4.saddr, pktinfo.fl4.daddr,
pktinfo.iph->tos,
ip4_dst_hoplimit(&pktinfo.rt->dst),
- htons(IP_DF),
+ 0,
pktinfo.gtph_port, pktinfo.gtph_port,
true, false);
break;
static int gtp_hashtable_new(struct gtp_dev *gtp, int hsize);
static void gtp_hashtable_free(struct gtp_dev *gtp);
static int gtp_encap_enable(struct net_device *dev, struct gtp_dev *gtp,
- int fd_gtp0, int fd_gtp1, struct net *src_net);
+ int fd_gtp0, int fd_gtp1);
static int gtp_newlink(struct net *src_net, struct net_device *dev,
struct nlattr *tb[], struct nlattr *data[])
fd0 = nla_get_u32(data[IFLA_GTP_FD0]);
fd1 = nla_get_u32(data[IFLA_GTP_FD1]);
- err = gtp_encap_enable(dev, gtp, fd0, fd1, src_net);
+ err = gtp_encap_enable(dev, gtp, fd0, fd1);
if (err < 0)
goto out_err;
}
static int gtp_encap_enable(struct net_device *dev, struct gtp_dev *gtp,
- int fd_gtp0, int fd_gtp1, struct net *src_net)
+ int fd_gtp0, int fd_gtp1)
{
struct udp_tunnel_sock_cfg tuncfg = {NULL};
struct socket *sock0, *sock1u;
gtp->sock0 = sock0;
gtp->sock1u = sock1u;
- gtp->net = src_net;
tuncfg.sk_user_data = gtp;
tuncfg.encap_rcv = gtp_encap_recv;
MODULE_AUTHOR("Harald Welte <hwelte@sysmocom.de>");
MODULE_DESCRIPTION("Interface driver for GTP encapsulated traffic");
MODULE_ALIAS_RTNL_LINK("gtp");
+MODULE_ALIAS_GENL_FAMILY("gtp");
{
/* Finish setting up the DEVICE info. */
dev->mtu = AX_MTU;
- dev->hard_header_len = 0;
- dev->addr_len = 0;
+ dev->hard_header_len = AX25_MAX_HEADER_LEN;
+ dev->addr_len = AX25_ADDR_LEN;
dev->type = ARPHRD_AX25;
dev->tx_queue_len = 10;
dev->header_ops = &ax25_header_ops;
ndev = hv_get_drvdata(device);
buffer = get_per_channel_state(channel);
+ /* commit_rd_index() -> hv_signal_on_read() needs this. */
+ init_cached_read_index(channel);
+
do {
desc = get_next_pkt_raw(channel);
if (desc != NULL) {
bufferlen = bytes_recvd;
}
+
+ init_cached_read_index(channel);
+
} while (1);
if (bufferlen > NETVSC_PACKET_SIZE)
{
dev->mtu = 64 * 1024;
dev->hard_header_len = ETH_HLEN; /* 14 */
+ dev->min_header_len = ETH_HLEN; /* 14 */
dev->addr_len = ETH_ALEN; /* 6 */
dev->type = ARPHRD_LOOPBACK; /* 0x0001*/
dev->flags = IFF_LOOPBACK;
size_t linear;
if (q->flags & IFF_VNET_HDR) {
- vnet_hdr_len = q->vnet_hdr_sz;
+ vnet_hdr_len = READ_ONCE(q->vnet_hdr_sz);
err = -EINVAL;
if (len < vnet_hdr_len)
if (q->flags & IFF_VNET_HDR) {
struct virtio_net_hdr vnet_hdr;
- vnet_hdr_len = q->vnet_hdr_sz;
+ vnet_hdr_len = READ_ONCE(q->vnet_hdr_sz);
if (iov_iter_count(iter) < vnet_hdr_len)
return -EINVAL;
if (virtio_net_hdr_from_skb(skb, &vnet_hdr,
- macvtap_is_little_endian(q)))
+ macvtap_is_little_endian(q), true))
BUG();
if (copy_to_iter(&vnet_hdr, sizeof(vnet_hdr), iter) !=
MODULE_AUTHOR("Maxime Bizon <mbizon@freebox.fr>");
MODULE_LICENSE("GPL");
+static int bcm63xx_config_intr(struct phy_device *phydev)
+{
+ int reg, err;
+
+ reg = phy_read(phydev, MII_BCM63XX_IR);
+ if (reg < 0)
+ return reg;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
+ reg &= ~MII_BCM63XX_IR_GMASK;
+ else
+ reg |= MII_BCM63XX_IR_GMASK;
+
+ err = phy_write(phydev, MII_BCM63XX_IR, reg);
+ return err;
+}
+
static int bcm63xx_config_init(struct phy_device *phydev)
{
int reg, err;
.config_aneg = genphy_config_aneg,
.read_status = genphy_read_status,
.ack_interrupt = bcm_phy_ack_intr,
- .config_intr = bcm_phy_config_intr,
+ .config_intr = bcm63xx_config_intr,
}, {
/* same phy as above, with just a different OUI */
.phy_id = 0x002bdc00,
.config_aneg = genphy_config_aneg,
.read_status = genphy_read_status,
.ack_interrupt = bcm_phy_ack_intr,
- .config_intr = bcm_phy_config_intr,
+ .config_intr = bcm63xx_config_intr,
} };
module_phy_driver(bcm63xx_driver);
#include <linux/phy.h>
#define TI_DP83848C_PHY_ID 0x20005ca0
+#define TI_DP83620_PHY_ID 0x20005ce0
#define NS_DP83848C_PHY_ID 0x20005c90
#define TLK10X_PHY_ID 0x2000a210
#define TI_DP83822_PHY_ID 0x2000a240
static struct mdio_device_id __maybe_unused dp83848_tbl[] = {
{ TI_DP83848C_PHY_ID, 0xfffffff0 },
{ NS_DP83848C_PHY_ID, 0xfffffff0 },
+ { TI_DP83620_PHY_ID, 0xfffffff0 },
{ TLK10X_PHY_ID, 0xfffffff0 },
{ TI_DP83822_PHY_ID, 0xfffffff0 },
{ }
static struct phy_driver dp83848_driver[] = {
DP83848_PHY_DRIVER(TI_DP83848C_PHY_ID, "TI DP83848C 10/100 Mbps PHY"),
DP83848_PHY_DRIVER(NS_DP83848C_PHY_ID, "NS DP83848C 10/100 Mbps PHY"),
+ DP83848_PHY_DRIVER(TI_DP83620_PHY_ID, "TI DP83620 10/100 Mbps PHY"),
DP83848_PHY_DRIVER(TLK10X_PHY_ID, "TI TLK10X 10/100 Mbps PHY"),
DP83848_PHY_DRIVER(TI_DP83822_PHY_ID, "TI DP83822 10/100 Mbps PHY"),
};
.ack_interrupt = &marvell_ack_interrupt,
.config_intr = &marvell_config_intr,
.did_interrupt = &m88e1121_did_interrupt,
+ .get_wol = &m88e1318_get_wol,
+ .set_wol = &m88e1318_set_wol,
.resume = &marvell_resume,
.suspend = &marvell_suspend,
.get_sset_count = marvell_get_sset_count,
if (rc)
return rc;
- iproc_mdio_config_clk(priv->base);
-
/* Prepare the read operation */
cmd = (MII_DATA_TA_VAL << MII_DATA_TA_SHIFT) |
(reg << MII_DATA_RA_SHIFT) |
if (rc)
return rc;
- iproc_mdio_config_clk(priv->base);
-
/* Prepare the write operation */
cmd = (MII_DATA_TA_VAL << MII_DATA_TA_SHIFT) |
(reg << MII_DATA_RA_SHIFT) |
bus->read = iproc_mdio_read;
bus->write = iproc_mdio_write;
+ iproc_mdio_config_clk(priv->base);
+
rc = of_mdiobus_register(bus, pdev->dev.of_node);
if (rc) {
dev_err(&pdev->dev, "MDIO bus registration failed\n");
.get_stats = kszphy_get_stats,
.suspend = genphy_suspend,
.resume = genphy_resume,
+}, {
+ .phy_id = PHY_ID_KSZ8795,
+ .phy_id_mask = MICREL_PHY_ID_MASK,
+ .name = "Micrel KSZ8795",
+ .features = PHY_BASIC_FEATURES,
+ .flags = PHY_HAS_MAGICANEG | PHY_HAS_INTERRUPT,
+ .config_init = kszphy_config_init,
+ .config_aneg = ksz8873mll_config_aneg,
+ .read_status = ksz8873mll_read_status,
+ .get_sset_count = kszphy_get_sset_count,
+ .get_strings = kszphy_get_strings,
+ .get_stats = kszphy_get_stats,
+ .suspend = genphy_suspend,
+ .resume = genphy_resume,
} };
module_phy_driver(ksphy_driver);
#include <linux/mii.h>
#include <linux/ethtool.h>
#include <linux/phy.h>
+#include <linux/phy_led_triggers.h>
#include <linux/timer.h>
#include <linux/workqueue.h>
#include <linux/mdio.h>
* phy_trigger_machine - trigger the state machine to run
*
* @phydev: the phy_device struct
+ * @sync: indicate whether we should wait for the workqueue cancelation
*
* Description: There has been a change in state which requires that the
* state machine runs.
*/
-static void phy_trigger_machine(struct phy_device *phydev)
+static void phy_trigger_machine(struct phy_device *phydev, bool sync)
{
- cancel_delayed_work_sync(&phydev->state_queue);
+ if (sync)
+ cancel_delayed_work_sync(&phydev->state_queue);
+ else
+ cancel_delayed_work(&phydev->state_queue);
queue_delayed_work(system_power_efficient_wq, &phydev->state_queue, 0);
}
phydev->state = PHY_HALTED;
mutex_unlock(&phydev->lock);
- phy_trigger_machine(phydev);
+ phy_trigger_machine(phydev, false);
}
/**
}
/* reschedule state queue work to run as soon as possible */
- phy_trigger_machine(phydev);
+ phy_trigger_machine(phydev, true);
return;
ignore:
if (do_resume)
phy_resume(phydev);
- phy_trigger_machine(phydev);
+ phy_trigger_machine(phydev, true);
}
EXPORT_SYMBOL(phy_start);
struct module *ndev_owner = dev->dev.parent->driver->owner;
struct mii_bus *bus = phydev->mdio.bus;
struct device *d = &phydev->mdio.dev;
+ bool using_genphy = false;
int err;
/* For Ethernet device drivers that register their own MDIO bus, we
d->driver =
&genphy_driver[GENPHY_DRV_1G].mdiodrv.driver;
+ using_genphy = true;
+ }
+
+ if (!try_module_get(d->driver->owner)) {
+ dev_err(&dev->dev, "failed to get the device driver module\n");
+ err = -EIO;
+ goto error_put_device;
+ }
+
+ if (using_genphy) {
err = d->driver->probe(d);
if (err >= 0)
err = device_bind_driver(d);
if (err)
- goto error;
+ goto error_module_put;
}
if (phydev->attached_dev) {
return err;
error:
+ /* phy_detach() does all of the cleanup below */
phy_detach(phydev);
+ return err;
+
+error_module_put:
+ module_put(d->driver->owner);
+error_put_device:
put_device(d);
if (ndev_owner != bus->owner)
module_put(bus->owner);
phy_led_triggers_unregister(phydev);
+ module_put(phydev->mdio.dev.driver->owner);
+
/* If the device had no specific driver before (i.e. - it
* was using the generic driver), we unbind the device
* from the generic driver so that there's a chance a
*/
#include <linux/leds.h>
#include <linux/phy.h>
+#include <linux/phy_led_triggers.h>
#include <linux/netdevice.h>
static struct phy_led_trigger *phy_speed_to_led_trigger(struct phy_device *phy,
sizeof(struct phy_led_trigger) *
phy->phy_num_led_triggers,
GFP_KERNEL);
- if (!phy->phy_led_triggers)
- return -ENOMEM;
+ if (!phy->phy_led_triggers) {
+ err = -ENOMEM;
+ goto out_clear;
+ }
for (i = 0; i < phy->phy_num_led_triggers; i++) {
err = phy_led_trigger_register(phy, &phy->phy_led_triggers[i],
while (i--)
phy_led_trigger_unregister(&phy->phy_led_triggers[i]);
devm_kfree(&phy->mdio.dev, phy->phy_led_triggers);
+out_clear:
+ phy->phy_num_led_triggers = 0;
return err;
}
EXPORT_SYMBOL_GPL(phy_led_triggers_register);
}
if (tun->flags & IFF_VNET_HDR) {
- if (len < tun->vnet_hdr_sz)
+ int vnet_hdr_sz = READ_ONCE(tun->vnet_hdr_sz);
+
+ if (len < vnet_hdr_sz)
return -EINVAL;
- len -= tun->vnet_hdr_sz;
+ len -= vnet_hdr_sz;
if (!copy_from_iter_full(&gso, sizeof(gso), from))
return -EFAULT;
if (tun16_to_cpu(tun, gso.hdr_len) > len)
return -EINVAL;
- iov_iter_advance(from, tun->vnet_hdr_sz - sizeof(gso));
+ iov_iter_advance(from, vnet_hdr_sz - sizeof(gso));
}
if ((tun->flags & TUN_TYPE_MASK) == IFF_TAP) {
vlan_hlen = VLAN_HLEN;
if (tun->flags & IFF_VNET_HDR)
- vnet_hdr_sz = tun->vnet_hdr_sz;
+ vnet_hdr_sz = READ_ONCE(tun->vnet_hdr_sz);
total = skb->len + vlan_hlen + vnet_hdr_sz;
return -EINVAL;
if (virtio_net_hdr_from_skb(skb, &gso,
- tun_is_little_endian(tun))) {
+ tun_is_little_endian(tun), true)) {
struct skb_shared_info *sinfo = skb_shinfo(skb);
pr_err("unexpected GSO type: "
"0x%x, gso_size %d, hdr_len %d\n",
struct net_device *netdev;
struct catc *catc;
u8 broadcast[ETH_ALEN];
- int i, pktsz;
+ int pktsz, ret;
if (usb_set_interface(usbdev,
intf->altsetting->desc.bInterfaceNumber, 1)) {
if ((!catc->ctrl_urb) || (!catc->tx_urb) ||
(!catc->rx_urb) || (!catc->irq_urb)) {
dev_err(&intf->dev, "No free urbs available.\n");
- usb_free_urb(catc->ctrl_urb);
- usb_free_urb(catc->tx_urb);
- usb_free_urb(catc->rx_urb);
- usb_free_urb(catc->irq_urb);
- free_netdev(netdev);
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto fail_free;
}
/* The F5U011 has the same vendor/product as the netmate but a device version of 0x130 */
catc->irq_buf, 2, catc_irq_done, catc, 1);
if (!catc->is_f5u011) {
+ u32 *buf;
+ int i;
+
dev_dbg(dev, "Checking memory size\n");
- i = 0x12345678;
- catc_write_mem(catc, 0x7a80, &i, 4);
- i = 0x87654321;
- catc_write_mem(catc, 0xfa80, &i, 4);
- catc_read_mem(catc, 0x7a80, &i, 4);
+ buf = kmalloc(4, GFP_KERNEL);
+ if (!buf) {
+ ret = -ENOMEM;
+ goto fail_free;
+ }
+
+ *buf = 0x12345678;
+ catc_write_mem(catc, 0x7a80, buf, 4);
+ *buf = 0x87654321;
+ catc_write_mem(catc, 0xfa80, buf, 4);
+ catc_read_mem(catc, 0x7a80, buf, 4);
- switch (i) {
+ switch (*buf) {
case 0x12345678:
catc_set_reg(catc, TxBufCount, 8);
catc_set_reg(catc, RxBufCount, 32);
dev_dbg(dev, "32k Memory\n");
break;
}
+
+ kfree(buf);
dev_dbg(dev, "Getting MAC from SEEROM.\n");
usb_set_intfdata(intf, catc);
SET_NETDEV_DEV(netdev, &intf->dev);
- if (register_netdev(netdev) != 0) {
- usb_set_intfdata(intf, NULL);
- usb_free_urb(catc->ctrl_urb);
- usb_free_urb(catc->tx_urb);
- usb_free_urb(catc->rx_urb);
- usb_free_urb(catc->irq_urb);
- free_netdev(netdev);
- return -EIO;
- }
+ ret = register_netdev(netdev);
+ if (ret)
+ goto fail_clear_intfdata;
+
return 0;
+
+fail_clear_intfdata:
+ usb_set_intfdata(intf, NULL);
+fail_free:
+ usb_free_urb(catc->ctrl_urb);
+ usb_free_urb(catc->tx_urb);
+ usb_free_urb(catc->rx_urb);
+ usb_free_urb(catc->irq_urb);
+ free_netdev(netdev);
+ return ret;
}
static void catc_disconnect(struct usb_interface *intf)
#define SAMSUNG_VENDOR_ID 0x04e8
#define LENOVO_VENDOR_ID 0x17ef
#define NVIDIA_VENDOR_ID 0x0955
+#define HP_VENDOR_ID 0x03f0
static const struct usb_device_id products[] = {
/* BLACKLIST !!
.driver_info = 0,
},
+/* HP lt2523 (Novatel E371) - handled by qmi_wwan */
+{
+ USB_DEVICE_AND_INTERFACE_INFO(HP_VENDOR_ID, 0x421d, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_ETHERNET, USB_CDC_PROTO_NONE),
+ .driver_info = 0,
+},
+
/* AnyDATA ADU960S - handled by qmi_wwan */
{
USB_DEVICE_AND_INTERFACE_INFO(0x16d5, 0x650a, USB_CLASS_COMM,
static int get_registers(pegasus_t *pegasus, __u16 indx, __u16 size, void *data)
{
+ u8 *buf;
int ret;
+ buf = kmalloc(size, GFP_NOIO);
+ if (!buf)
+ return -ENOMEM;
+
ret = usb_control_msg(pegasus->usb, usb_rcvctrlpipe(pegasus->usb, 0),
PEGASUS_REQ_GET_REGS, PEGASUS_REQT_READ, 0,
- indx, data, size, 1000);
+ indx, buf, size, 1000);
if (ret < 0)
netif_dbg(pegasus, drv, pegasus->net,
"%s returned %d\n", __func__, ret);
+ else if (ret <= size)
+ memcpy(data, buf, ret);
+ kfree(buf);
return ret;
}
-static int set_registers(pegasus_t *pegasus, __u16 indx, __u16 size, void *data)
+static int set_registers(pegasus_t *pegasus, __u16 indx, __u16 size,
+ const void *data)
{
+ u8 *buf;
int ret;
+ buf = kmemdup(data, size, GFP_NOIO);
+ if (!buf)
+ return -ENOMEM;
+
ret = usb_control_msg(pegasus->usb, usb_sndctrlpipe(pegasus->usb, 0),
PEGASUS_REQ_SET_REGS, PEGASUS_REQT_WRITE, 0,
- indx, data, size, 100);
+ indx, buf, size, 100);
if (ret < 0)
netif_dbg(pegasus, drv, pegasus->net,
"%s returned %d\n", __func__, ret);
+ kfree(buf);
return ret;
}
static int set_register(pegasus_t *pegasus, __u16 indx, __u8 data)
{
+ u8 *buf;
int ret;
+ buf = kmemdup(&data, 1, GFP_NOIO);
+ if (!buf)
+ return -ENOMEM;
+
ret = usb_control_msg(pegasus->usb, usb_sndctrlpipe(pegasus->usb, 0),
PEGASUS_REQ_SET_REG, PEGASUS_REQT_WRITE, data,
- indx, &data, 1, 1000);
+ indx, buf, 1, 1000);
if (ret < 0)
netif_dbg(pegasus, drv, pegasus->net,
"%s returned %d\n", __func__, ret);
+ kfree(buf);
return ret;
}
USB_CDC_PROTO_NONE),
.driver_info = (unsigned long)&qmi_wwan_info,
},
+ { /* HP lt2523 (Novatel E371) */
+ USB_DEVICE_AND_INTERFACE_INFO(0x03f0, 0x421d,
+ USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_ETHERNET,
+ USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long)&qmi_wwan_info,
+ },
{ /* HP lt4112 LTE/HSPA+ Gobi 4G Module (Huawei me906e) */
USB_DEVICE_AND_INTERFACE_INFO(0x03f0, 0x581d, USB_CLASS_VENDOR_SPEC, 1, 7),
.driver_info = (unsigned long)&qmi_wwan_info,
#define NETNEXT_VERSION "08"
/* Information for net */
-#define NET_VERSION "6"
+#define NET_VERSION "8"
#define DRIVER_VERSION "v1." NETNEXT_VERSION "." NET_VERSION
#define DRIVER_AUTHOR "Realtek linux nic maintainers <nic_swsd@realtek.com>"
napi_complete(napi);
if (!list_empty(&tp->rx_done))
napi_schedule(napi);
+ else if (!skb_queue_empty(&tp->tx_queue) &&
+ !list_empty(&tp->tx_free))
+ napi_schedule(napi);
}
return work_done;
if (!netif_carrier_ok(netdev)) {
tp->rtl_ops.enable(tp);
set_bit(RTL8152_SET_RX_MODE, &tp->flags);
+ netif_stop_queue(netdev);
napi_disable(&tp->napi);
netif_carrier_on(netdev);
rtl_start_rx(tp);
napi_enable(&tp->napi);
+ netif_wake_queue(netdev);
+ netif_info(tp, link, netdev, "carrier on\n");
}
} else {
if (netif_carrier_ok(netdev)) {
napi_disable(&tp->napi);
tp->rtl_ops.disable(tp);
napi_enable(&tp->napi);
+ netif_info(tp, link, netdev, "carrier off\n");
}
}
}
if (!netif_running(netdev))
return 0;
+ netif_stop_queue(netdev);
napi_disable(&tp->napi);
clear_bit(WORK_ENABLE, &tp->flags);
usb_kill_urb(tp->intr_urb);
cancel_delayed_work_sync(&tp->schedule);
if (netif_carrier_ok(netdev)) {
- netif_stop_queue(netdev);
mutex_lock(&tp->control);
tp->rtl_ops.disable(tp);
mutex_unlock(&tp->control);
if (netif_carrier_ok(netdev)) {
mutex_lock(&tp->control);
tp->rtl_ops.enable(tp);
+ rtl_start_rx(tp);
rtl8152_set_rx_mode(netdev);
mutex_unlock(&tp->control);
- netif_wake_queue(netdev);
}
napi_enable(&tp->napi);
+ netif_wake_queue(netdev);
+ usb_submit_urb(tp->intr_urb, GFP_KERNEL);
+
+ if (!list_empty(&tp->rx_done))
+ napi_schedule(&tp->napi);
return 0;
}
*/
if (!sw_linking && tp->rtl_ops.in_nway(tp))
return true;
+ else if (!skb_queue_empty(&tp->tx_queue))
+ return true;
else
return false;
}
struct net_device *netdev = tp->netdev;
int ret = 0;
+ set_bit(SELECTIVE_SUSPEND, &tp->flags);
+ smp_mb__after_atomic();
+
if (netif_running(netdev) && test_bit(WORK_ENABLE, &tp->flags)) {
u32 rcr = 0;
if (delay_autosuspend(tp)) {
+ clear_bit(SELECTIVE_SUSPEND, &tp->flags);
+ smp_mb__after_atomic();
ret = -EBUSY;
goto out1;
}
if (!(ocp_data & RXFIFO_EMPTY)) {
rxdy_gated_en(tp, false);
ocp_write_dword(tp, MCU_TYPE_PLA, PLA_RCR, rcr);
+ clear_bit(SELECTIVE_SUSPEND, &tp->flags);
+ smp_mb__after_atomic();
ret = -EBUSY;
goto out1;
}
}
}
- set_bit(SELECTIVE_SUSPEND, &tp->flags);
-
out1:
return ret;
}
if (netif_running(tp->netdev) && tp->netdev->flags & IFF_UP) {
if (test_bit(SELECTIVE_SUSPEND, &tp->flags)) {
tp->rtl_ops.autosuspend_en(tp, false);
- clear_bit(SELECTIVE_SUSPEND, &tp->flags);
napi_disable(&tp->napi);
set_bit(WORK_ENABLE, &tp->flags);
if (netif_carrier_ok(tp->netdev))
rtl_start_rx(tp);
napi_enable(&tp->napi);
+ clear_bit(SELECTIVE_SUSPEND, &tp->flags);
+ smp_mb__after_atomic();
+ if (!list_empty(&tp->rx_done))
+ napi_schedule(&tp->napi);
} else {
tp->rtl_ops.up(tp);
netif_carrier_off(tp->netdev);
*/
static int get_registers(rtl8150_t * dev, u16 indx, u16 size, void *data)
{
- return usb_control_msg(dev->udev, usb_rcvctrlpipe(dev->udev, 0),
- RTL8150_REQ_GET_REGS, RTL8150_REQT_READ,
- indx, 0, data, size, 500);
+ void *buf;
+ int ret;
+
+ buf = kmalloc(size, GFP_NOIO);
+ if (!buf)
+ return -ENOMEM;
+
+ ret = usb_control_msg(dev->udev, usb_rcvctrlpipe(dev->udev, 0),
+ RTL8150_REQ_GET_REGS, RTL8150_REQT_READ,
+ indx, 0, buf, size, 500);
+ if (ret > 0 && ret <= size)
+ memcpy(data, buf, ret);
+ kfree(buf);
+ return ret;
}
-static int set_registers(rtl8150_t * dev, u16 indx, u16 size, void *data)
+static int set_registers(rtl8150_t * dev, u16 indx, u16 size, const void *data)
{
- return usb_control_msg(dev->udev, usb_sndctrlpipe(dev->udev, 0),
- RTL8150_REQ_SET_REGS, RTL8150_REQT_WRITE,
- indx, 0, data, size, 500);
+ void *buf;
+ int ret;
+
+ buf = kmemdup(data, size, GFP_NOIO);
+ if (!buf)
+ return -ENOMEM;
+
+ ret = usb_control_msg(dev->udev, usb_sndctrlpipe(dev->udev, 0),
+ RTL8150_REQ_SET_REGS, RTL8150_REQT_WRITE,
+ indx, 0, buf, size, 500);
+ kfree(buf);
+ return ret;
}
static void async_set_reg_cb(struct urb *urb)
/* Private data structure */
struct sierra_net_data {
- u8 ethr_hdr_tmpl[ETH_HLEN]; /* ethernet header template for rx'd pkts */
-
u16 link_up; /* air link up or down */
u8 tx_hdr_template[4]; /* part of HIP hdr for tx'd packets */
/* LSI Protocol types */
#define SIERRA_NET_PROTOCOL_UMTS 0x01
+#define SIERRA_NET_PROTOCOL_UMTS_DS 0x04
/* LSI Coverage */
#define SIERRA_NET_COVERAGE_NONE 0x00
#define SIERRA_NET_COVERAGE_NOPACKET 0x01
/* LSI Session */
#define SIERRA_NET_SESSION_IDLE 0x00
/* LSI Link types */
-#define SIERRA_NET_AS_LINK_TYPE_IPv4 0x00
+#define SIERRA_NET_AS_LINK_TYPE_IPV4 0x00
+#define SIERRA_NET_AS_LINK_TYPE_IPV6 0x02
struct lsi_umts {
u8 protocol;
__be16 length;
/* eventually use a union for the rest - assume umts for now */
u8 coverage;
- u8 unused2[41];
+ u8 network_len; /* network name len */
+ u8 network[40]; /* network name (UCS2, bigendian) */
u8 session_state;
u8 unused3[33];
+} __packed;
+
+struct lsi_umts_single {
+ struct lsi_umts lsi;
u8 link_type;
u8 pdp_addr_len; /* NW-supplied PDP address len */
u8 pdp_addr[16]; /* NW-supplied PDP address (bigendian)) */
u8 reserved[8];
} __packed;
+struct lsi_umts_dual {
+ struct lsi_umts lsi;
+ u8 pdp_addr4_len; /* NW-supplied PDP IPv4 address len */
+ u8 pdp_addr4[4]; /* NW-supplied PDP IPv4 address (bigendian)) */
+ u8 pdp_addr6_len; /* NW-supplied PDP IPv6 address len */
+ u8 pdp_addr6[16]; /* NW-supplied PDP IPv6 address (bigendian)) */
+ u8 unused4[23];
+ u8 dns1_addr4_len; /* NW-supplied 1st DNS v4 address len (bigendian) */
+ u8 dns1_addr4[4]; /* NW-supplied 1st DNS v4 address */
+ u8 dns1_addr6_len; /* NW-supplied 1st DNS v6 address len */
+ u8 dns1_addr6[16]; /* NW-supplied 1st DNS v6 address (bigendian)*/
+ u8 dns2_addr4_len; /* NW-supplied 2nd DNS v4 address len (bigendian) */
+ u8 dns2_addr4[4]; /* NW-supplied 2nd DNS v4 address */
+ u8 dns2_addr6_len; /* NW-supplied 2nd DNS v6 address len */
+ u8 dns2_addr6[16]; /* NW-supplied 2nd DNS v6 address (bigendian)*/
+ u8 unused5[68];
+} __packed;
+
#define SIERRA_NET_LSI_COMMON_LEN 4
-#define SIERRA_NET_LSI_UMTS_LEN (sizeof(struct lsi_umts))
+#define SIERRA_NET_LSI_UMTS_LEN (sizeof(struct lsi_umts_single))
#define SIERRA_NET_LSI_UMTS_STATUS_LEN \
(SIERRA_NET_LSI_UMTS_LEN - SIERRA_NET_LSI_COMMON_LEN)
+#define SIERRA_NET_LSI_UMTS_DS_LEN (sizeof(struct lsi_umts_dual))
+#define SIERRA_NET_LSI_UMTS_DS_STATUS_LEN \
+ (SIERRA_NET_LSI_UMTS_DS_LEN - SIERRA_NET_LSI_COMMON_LEN)
/* Forward definitions */
static void sierra_sync_timer(unsigned long syncdata);
dev->data[0] = (unsigned long)priv;
}
-/* is packet IPv4 */
+/* is packet IPv4/IPv6 */
static inline int is_ip(struct sk_buff *skb)
{
- return skb->protocol == cpu_to_be16(ETH_P_IP);
+ return skb->protocol == cpu_to_be16(ETH_P_IP) ||
+ skb->protocol == cpu_to_be16(ETH_P_IPV6);
}
/*
static int sierra_net_parse_lsi(struct usbnet *dev, char *data, int datalen)
{
struct lsi_umts *lsi = (struct lsi_umts *)data;
+ u32 expected_length;
- if (datalen < sizeof(struct lsi_umts)) {
- netdev_err(dev->net, "%s: Data length %d, exp %Zu\n",
- __func__, datalen,
- sizeof(struct lsi_umts));
+ if (datalen < sizeof(struct lsi_umts_single)) {
+ netdev_err(dev->net, "%s: Data length %d, exp >= %Zu\n",
+ __func__, datalen, sizeof(struct lsi_umts_single));
return -1;
}
- if (lsi->length != cpu_to_be16(SIERRA_NET_LSI_UMTS_STATUS_LEN)) {
- netdev_err(dev->net, "%s: LSI_UMTS_STATUS_LEN %d, exp %u\n",
- __func__, be16_to_cpu(lsi->length),
- (u32)SIERRA_NET_LSI_UMTS_STATUS_LEN);
- return -1;
+ /* Validate the session state */
+ if (lsi->session_state == SIERRA_NET_SESSION_IDLE) {
+ netdev_err(dev->net, "Session idle, 0x%02x\n",
+ lsi->session_state);
+ return 0;
}
/* Validate the protocol - only support UMTS for now */
- if (lsi->protocol != SIERRA_NET_PROTOCOL_UMTS) {
+ if (lsi->protocol == SIERRA_NET_PROTOCOL_UMTS) {
+ struct lsi_umts_single *single = (struct lsi_umts_single *)lsi;
+
+ /* Validate the link type */
+ if (single->link_type != SIERRA_NET_AS_LINK_TYPE_IPV4 &&
+ single->link_type != SIERRA_NET_AS_LINK_TYPE_IPV6) {
+ netdev_err(dev->net, "Link type unsupported: 0x%02x\n",
+ single->link_type);
+ return -1;
+ }
+ expected_length = SIERRA_NET_LSI_UMTS_STATUS_LEN;
+ } else if (lsi->protocol == SIERRA_NET_PROTOCOL_UMTS_DS) {
+ expected_length = SIERRA_NET_LSI_UMTS_DS_STATUS_LEN;
+ } else {
netdev_err(dev->net, "Protocol unsupported, 0x%02x\n",
- lsi->protocol);
+ lsi->protocol);
return -1;
}
- /* Validate the link type */
- if (lsi->link_type != SIERRA_NET_AS_LINK_TYPE_IPv4) {
- netdev_err(dev->net, "Link type unsupported: 0x%02x\n",
- lsi->link_type);
+ if (be16_to_cpu(lsi->length) != expected_length) {
+ netdev_err(dev->net, "%s: LSI_UMTS_STATUS_LEN %d, exp %u\n",
+ __func__, be16_to_cpu(lsi->length), expected_length);
return -1;
}
/* Validate the coverage */
- if (lsi->coverage == SIERRA_NET_COVERAGE_NONE
- || lsi->coverage == SIERRA_NET_COVERAGE_NOPACKET) {
+ if (lsi->coverage == SIERRA_NET_COVERAGE_NONE ||
+ lsi->coverage == SIERRA_NET_COVERAGE_NOPACKET) {
netdev_err(dev->net, "No coverage, 0x%02x\n", lsi->coverage);
return 0;
}
- /* Validate the session state */
- if (lsi->session_state == SIERRA_NET_SESSION_IDLE) {
- netdev_err(dev->net, "Session idle, 0x%02x\n",
- lsi->session_state);
- return 0;
- }
-
/* Set link_sense true */
return 1;
}
u8 numendpoints;
u16 fwattr = 0;
int status;
- struct ethhdr *eth;
struct sierra_net_data *priv;
static const u8 sync_tmplate[sizeof(priv->sync_msg)] = {
0x00, 0x00, SIERRA_NET_HIP_MSYNC_ID, 0x00};
dev->net->dev_addr[ETH_ALEN-2] = atomic_inc_return(&iface_counter);
dev->net->dev_addr[ETH_ALEN-1] = ifacenum;
- /* we will have to manufacture ethernet headers, prepare template */
- eth = (struct ethhdr *)priv->ethr_hdr_tmpl;
- memcpy(ð->h_dest, dev->net->dev_addr, ETH_ALEN);
- eth->h_proto = cpu_to_be16(ETH_P_IP);
-
/* prepare shutdown message template */
memcpy(priv->shdwn_msg, shdwn_tmplate, sizeof(priv->shdwn_msg));
/* set context index initially to 0 - prepares tx hdr template */
skb_pull(skb, hh.hdrlen);
- /* We are going to accept this packet, prepare it */
- memcpy(skb->data, sierra_net_get_private(dev)->ethr_hdr_tmpl,
- ETH_HLEN);
+ /* We are going to accept this packet, prepare it.
+ * In case protocol is IPv6, keep it, otherwise force IPv4.
+ */
+ skb_reset_mac_header(skb);
+ if (eth_hdr(skb)->h_proto != cpu_to_be16(ETH_P_IPV6))
+ eth_hdr(skb)->h_proto = cpu_to_be16(ETH_P_IP);
+ eth_zero_addr(eth_hdr(skb)->h_source);
+ memcpy(eth_hdr(skb)->h_dest, dev->net->dev_addr, ETH_ALEN);
/* Last packet in batch handled by usbnet */
if (hh.payload_len.word == skb->len)
*/
DECLARE_EWMA(pkt_len, 1, 64)
+/* With mergeable buffers we align buffer address and use the low bits to
+ * encode its true size. Buffer size is up to 1 page so we need to align to
+ * square root of page size to ensure we reserve enough bits to encode the true
+ * size.
+ */
+#define MERGEABLE_BUFFER_MIN_ALIGN_SHIFT ((PAGE_SHIFT + 1) / 2)
+
/* Minimum alignment for mergeable packet buffers. */
-#define MERGEABLE_BUFFER_ALIGN max(L1_CACHE_BYTES, 256)
+#define MERGEABLE_BUFFER_ALIGN max(L1_CACHE_BYTES, \
+ 1 << MERGEABLE_BUFFER_MIN_ALIGN_SHIFT)
#define VIRTNET_DRIVER_VERSION "1.0.0"
hdr = skb_vnet_hdr(skb);
if (virtio_net_hdr_from_skb(skb, &hdr->hdr,
- virtio_is_little_endian(vi->vdev)))
+ virtio_is_little_endian(vi->vdev), false))
BUG();
if (vi->mergeable_rx_bufs)
u16 xdp_qp = 0, curr_qp;
int i, err;
+ if (prog && prog->xdp_adjust_head) {
+ netdev_warn(dev, "Does not support bpf_xdp_adjust_head()\n");
+ return -EOPNOTSUPP;
+ }
+
if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO4) ||
virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO6) ||
virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_ECN) ||
put_page(vi->rq[i].alloc_frag.page);
}
-static bool is_xdp_queue(struct virtnet_info *vi, int q)
+static bool is_xdp_raw_buffer_queue(struct virtnet_info *vi, int q)
{
+ /* For small receive mode always use kfree_skb variants */
+ if (!vi->mergeable_rx_bufs)
+ return false;
+
if (q < (vi->curr_queue_pairs - vi->xdp_queue_pairs))
return false;
else if (q < vi->curr_queue_pairs)
for (i = 0; i < vi->max_queue_pairs; i++) {
struct virtqueue *vq = vi->sq[i].vq;
while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
- if (!is_xdp_queue(vi, i))
+ if (!is_xdp_raw_buffer_queue(vi, i))
dev_kfree_skb(buf);
else
put_page(virt_to_head_page(buf));
= container_of(p, struct vxlan_fdb, hlist);
unsigned long timeout;
- if (f->state & NUD_PERMANENT)
+ if (f->state & (NUD_PERMANENT | NUD_NOARP))
continue;
timeout = f->used + vxlan->cfg.age_interval * HZ;
}
/* Purge the forwarding table */
-static void vxlan_flush(struct vxlan_dev *vxlan)
+static void vxlan_flush(struct vxlan_dev *vxlan, bool do_all)
{
unsigned int h;
hlist_for_each_safe(p, n, &vxlan->fdb_head[h]) {
struct vxlan_fdb *f
= container_of(p, struct vxlan_fdb, hlist);
+ if (!do_all && (f->state & (NUD_PERMANENT | NUD_NOARP)))
+ continue;
/* the all_zeros_mac entry is deleted at vxlan_uninit */
if (!is_zero_ether_addr(f->eth_addr))
vxlan_fdb_destroy(vxlan, f);
del_timer_sync(&vxlan->age_timer);
- vxlan_flush(vxlan);
+ vxlan_flush(vxlan, false);
vxlan_sock_release(vxlan);
return ret;
memcpy(&vxlan->cfg, conf, sizeof(*conf));
if (!vxlan->cfg.dst_port) {
if (conf->flags & VXLAN_F_GPE)
- vxlan->cfg.dst_port = 4790; /* IANA assigned VXLAN-GPE port */
+ vxlan->cfg.dst_port = htons(4790); /* IANA VXLAN-GPE port */
else
vxlan->cfg.dst_port = default_port;
}
struct vxlan_dev *vxlan = netdev_priv(dev);
struct vxlan_net *vn = net_generic(vxlan->net, vxlan_net_id);
+ vxlan_flush(vxlan, true);
+
spin_lock(&vn->sock_lock);
if (!hlist_unhashed(&vxlan->hlist))
hlist_del_rcu(&vxlan->hlist);
#define IWL8000_FW_PRE "iwlwifi-8000C-"
#define IWL8000_MODULE_FIRMWARE(api) \
- IWL8000_FW_PRE "-" __stringify(api) ".ucode"
+ IWL8000_FW_PRE __stringify(api) ".ucode"
#define IWL8265_FW_PRE "iwlwifi-8265-"
#define IWL8265_MODULE_FIRMWARE(api) \
.frame_limit = IWL_FRAME_LIMIT,
};
- /* Make sure reserved queue is still marked as such (or allocated) */
- mvm->queue_info[mvm_sta->reserved_queue].status =
- IWL_MVM_QUEUE_RESERVED;
+ /* Make sure reserved queue is still marked as such (if allocated) */
+ if (mvm_sta->reserved_queue != IEEE80211_INVAL_HW_QUEUE)
+ mvm->queue_info[mvm_sta->reserved_queue].status =
+ IWL_MVM_QUEUE_RESERVED;
for (i = 0; i <= IWL_MAX_TID_COUNT; i++) {
struct iwl_mvm_tid_data *tid_data = &mvm_sta->tid_data[i];
return;
IWL_DEBUG_TEMP(mvm, "Thermal zone device unregister\n");
- thermal_zone_device_unregister(mvm->tz_device.tzone);
- mvm->tz_device.tzone = NULL;
+ if (mvm->tz_device.tzone) {
+ thermal_zone_device_unregister(mvm->tz_device.tzone);
+ mvm->tz_device.tzone = NULL;
+ }
}
static void iwl_mvm_cooling_device_unregister(struct iwl_mvm *mvm)
return;
IWL_DEBUG_TEMP(mvm, "Cooling device unregister\n");
- thermal_cooling_device_unregister(mvm->cooling_dev.cdev);
- mvm->cooling_dev.cdev = NULL;
+ if (mvm->cooling_dev.cdev) {
+ thermal_cooling_device_unregister(mvm->cooling_dev.cdev);
+ mvm->cooling_dev.cdev = NULL;
+ }
}
#endif /* CONFIG_THERMAL */
struct rtl_priv *rtlpriv = rtl_priv(hw);
struct rtl_pci *rtlpci = rtl_pcidev(rtl_pcipriv(hw));
struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw));
- char *fw_name = "rtlwifi/rtl8192cfwU.bin";
+ char *fw_name;
rtl8192ce_bt_reg_init(hw);
}
/* request fw */
- if (IS_81XXC_VENDOR_UMC_B_CUT(rtlhal->version))
+ if (IS_VENDOR_UMC_A_CUT(rtlhal->version) &&
+ !IS_92C_SERIAL(rtlhal->version))
+ fw_name = "rtlwifi/rtl8192cfwU.bin";
+ else if (IS_81XXC_VENDOR_UMC_B_CUT(rtlhal->version))
fw_name = "rtlwifi/rtl8192cfwU_B.bin";
+ else
+ fw_name = "rtlwifi/rtl8192cfw.bin";
rtlpriv->max_fw_size = 0x4000;
pr_info("Using firmware %s\n", fw_name);
{
struct xenvif *vif = netdev_priv(dev);
struct xenvif_queue *queue = NULL;
- unsigned int num_queues = vif->num_queues;
unsigned long rx_bytes = 0;
unsigned long rx_packets = 0;
unsigned long tx_bytes = 0;
unsigned long tx_packets = 0;
unsigned int index;
+ spin_lock(&vif->lock);
if (vif->queues == NULL)
goto out;
/* Aggregate tx and rx stats from each queue */
- for (index = 0; index < num_queues; ++index) {
+ for (index = 0; index < vif->num_queues; ++index) {
queue = &vif->queues[index];
rx_bytes += queue->stats.rx_bytes;
rx_packets += queue->stats.rx_packets;
}
out:
+ spin_unlock(&vif->lock);
+
vif->dev->stats.rx_bytes = rx_bytes;
vif->dev->stats.rx_packets = rx_packets;
vif->dev->stats.tx_bytes = tx_bytes;
static void backend_disconnect(struct backend_info *be)
{
if (be->vif) {
+ unsigned int queue_index;
+
xen_unregister_watchers(be->vif);
#ifdef CONFIG_DEBUG_FS
xenvif_debugfs_delif(be->vif);
#endif /* CONFIG_DEBUG_FS */
xenvif_disconnect_data(be->vif);
+ for (queue_index = 0; queue_index < be->vif->num_queues; ++queue_index)
+ xenvif_deinit_queue(&be->vif->queues[queue_index]);
+
+ spin_lock(&be->vif->lock);
+ vfree(be->vif->queues);
+ be->vif->num_queues = 0;
+ be->vif->queues = NULL;
+ spin_unlock(&be->vif->lock);
+
xenvif_disconnect_ctrl(be->vif);
}
}
err:
if (be->vif->num_queues > 0)
xenvif_disconnect_data(be->vif); /* Clean up existing queues */
+ for (queue_index = 0; queue_index < be->vif->num_queues; ++queue_index)
+ xenvif_deinit_queue(&be->vif->queues[queue_index]);
vfree(be->vif->queues);
be->vif->queues = NULL;
be->vif->num_queues = 0;
{
RING_IDX req_prod = queue->rx.req_prod_pvt;
int notify;
+ int err = 0;
if (unlikely(!netif_carrier_ok(queue->info->netdev)))
return;
struct xen_netif_rx_request *req;
skb = xennet_alloc_one_rx_buffer(queue);
- if (!skb)
+ if (!skb) {
+ err = -ENOMEM;
break;
+ }
id = xennet_rxidx(req_prod);
queue->rx.req_prod_pvt = req_prod;
- /* Not enough requests? Try again later. */
- if (req_prod - queue->rx.rsp_cons < NET_RX_SLOTS_MIN) {
+ /* Try again later if there are not enough requests or skb allocation
+ * failed.
+ * Enough requests is quantified as the sum of newly created slots and
+ * the unconsumed slots at the backend.
+ */
+ if (req_prod - queue->rx.rsp_cons < NET_RX_SLOTS_MIN ||
+ unlikely(err)) {
mod_timer(&queue->rx_refill_timer, jiffies + (HZ/10));
return;
}
for (i = 0; i < num_queues && info->queues; ++i) {
struct netfront_queue *queue = &info->queues[i];
+ del_timer_sync(&queue->rx_refill_timer);
+
if (queue->tx_irq && (queue->tx_irq == queue->rx_irq))
unbind_from_irqhandler(queue->tx_irq, queue);
if (queue->tx_irq && (queue->tx_irq != queue->rx_irq)) {
if (netif_running(info->netdev))
napi_disable(&queue->napi);
- del_timer_sync(&queue->rx_refill_timer);
netif_napi_del(&queue->napi);
}
xennet_destroy_queues(info);
err = xennet_create_queues(info, &num_queues);
- if (err < 0)
- goto destroy_ring;
+ if (err < 0) {
+ xenbus_dev_fatal(dev, err, "creating queues");
+ kfree(info->queues);
+ info->queues = NULL;
+ goto out;
+ }
/* Create shared ring, alloc event channel -- for each queue */
for (i = 0; i < num_queues; ++i) {
queue = &info->queues[i];
err = setup_netfront(dev, queue, feature_split_evtchn);
- if (err) {
- /* setup_netfront() will tidy up the current
- * queue on error, but we need to clean up
- * those already allocated.
- */
- if (i > 0) {
- rtnl_lock();
- netif_set_real_num_tx_queues(info->netdev, i);
- rtnl_unlock();
- goto destroy_ring;
- } else {
- goto out;
- }
- }
+ if (err)
+ goto destroy_ring;
}
again:
xenbus_transaction_end(xbt, 1);
destroy_ring:
xennet_disconnect_backend(info);
- kfree(info->queues);
- info->queues = NULL;
+ xennet_destroy_queues(info);
out:
+ unregister_netdev(info->netdev);
+ xennet_free_netdev(info->netdev);
return err;
}
kfree(nsblk);
}
-static struct device_type namespace_io_device_type = {
+static const struct device_type namespace_io_device_type = {
.name = "nd_namespace_io",
.release = namespace_io_release,
};
-static struct device_type namespace_pmem_device_type = {
+static const struct device_type namespace_pmem_device_type = {
.name = "nd_namespace_pmem",
.release = namespace_pmem_release,
};
-static struct device_type namespace_blk_device_type = {
+static const struct device_type namespace_blk_device_type = {
.name = "nd_namespace_blk",
.release = namespace_blk_release,
};
struct nvdimm_drvdata *ndd;
struct nd_label_id label_id;
u32 flags = 0, remainder;
+ int rc, i, id = -1;
u8 *uuid = NULL;
- int rc, i;
if (dev->driver || ndns->claim)
return -EBUSY;
struct nd_namespace_pmem *nspm = to_nd_namespace_pmem(dev);
uuid = nspm->uuid;
+ id = nspm->id;
} else if (is_namespace_blk(dev)) {
struct nd_namespace_blk *nsblk = to_nd_namespace_blk(dev);
uuid = nsblk->uuid;
flags = NSLABEL_FLAG_LOCAL;
+ id = nsblk->id;
}
/*
/*
* Try to delete the namespace if we deleted all of its
- * allocation, this is not the seed device for the region, and
- * it is not actively claimed by a btt instance.
+ * allocation, this is not the seed or 0th device for the
+ * region, and it is not actively claimed by a btt, pfn, or dax
+ * instance.
*/
- if (val == 0 && nd_region->ns_seed != dev && !ndns->claim)
+ if (val == 0 && id != 0 && nd_region->ns_seed != dev && !ndns->claim)
nd_device_unregister(dev, ND_ASYNC);
return rc;
size = resource_size(&nsio->res);
npfns = (size - start_pad - end_trunc - SZ_8K) / SZ_4K;
if (nd_pfn->mode == PFN_MODE_PMEM) {
- unsigned long memmap_size;
-
/*
* vmemmap_populate_hugepages() allocates the memmap array in
* HPAGE_SIZE chunks.
*/
- memmap_size = ALIGN(64 * npfns, HPAGE_SIZE);
- offset = ALIGN(start + SZ_8K + memmap_size + dax_label_reserve,
- nd_pfn->align) - start;
+ offset = ALIGN(start + SZ_8K + 64 * npfns + dax_label_reserve,
+ max(nd_pfn->align, HPAGE_SIZE)) - start;
} else if (nd_pfn->mode == PFN_MODE_RAM)
offset = ALIGN(start + SZ_8K + dax_label_reserve,
nd_pfn->align) - start;
return 0;
freq->sg_table.sgl = freq->first_sgl;
- ret = sg_alloc_table_chained(&freq->sg_table, rq->nr_phys_segments,
- freq->sg_table.sgl);
+ ret = sg_alloc_table_chained(&freq->sg_table,
+ blk_rq_nr_phys_segments(rq), freq->sg_table.sgl);
if (ret)
return -ENOMEM;
op->nents = blk_rq_map_sg(rq->q, rq, freq->sg_table.sgl);
- WARN_ON(op->nents > rq->nr_phys_segments);
+ WARN_ON(op->nents > blk_rq_nr_phys_segments(rq));
dir = (rq_data_dir(rq) == WRITE) ? DMA_TO_DEVICE : DMA_FROM_DEVICE;
freq->sg_cnt = fc_dma_map_sg(ctrl->lport->dev, freq->sg_table.sgl,
op->nents, dir);
{
struct nvmet_subsys *subsys = to_subsys(item);
+ nvmet_subsys_del_ctrls(subsys);
nvmet_subsys_put(subsys);
}
pr_err("ctrl %d keep-alive timer (%d seconds) expired!\n",
ctrl->cntlid, ctrl->kato);
- ctrl->ops->delete_ctrl(ctrl);
+ nvmet_ctrl_fatal_error(ctrl);
}
static void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl)
list_del(&ctrl->subsys_entry);
mutex_unlock(&subsys->lock);
+ flush_work(&ctrl->async_event_work);
+ cancel_work_sync(&ctrl->fatal_err_work);
+
ida_simple_remove(&subsys->cntlid_ida, ctrl->cntlid);
nvmet_subsys_put(subsys);
kfree(subsys);
}
+void nvmet_subsys_del_ctrls(struct nvmet_subsys *subsys)
+{
+ struct nvmet_ctrl *ctrl;
+
+ mutex_lock(&subsys->lock);
+ list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry)
+ ctrl->ops->delete_ctrl(ctrl);
+ mutex_unlock(&subsys->lock);
+}
+
void nvmet_subsys_put(struct nvmet_subsys *subsys)
{
kref_put(&subsys->ref, nvmet_subsys_free);
(struct fcnvme_ls_disconnect_rqst *)iod->rqstbuf;
struct fcnvme_ls_disconnect_acc *acc =
(struct fcnvme_ls_disconnect_acc *)iod->rspbuf;
- struct nvmet_fc_tgt_queue *queue;
+ struct nvmet_fc_tgt_queue *queue = NULL;
struct nvmet_fc_tgt_assoc *assoc;
int ret = 0;
bool del_assoc = false;
assoc = nvmet_fc_find_target_assoc(tgtport,
be64_to_cpu(rqst->associd.association_id));
iod->assoc = assoc;
- if (!assoc)
+ if (assoc) {
+ if (rqst->discon_cmd.scope ==
+ FCNVME_DISCONN_CONNECTION) {
+ queue = nvmet_fc_find_target_queue(tgtport,
+ be64_to_cpu(
+ rqst->discon_cmd.id));
+ if (!queue) {
+ nvmet_fc_tgt_a_put(assoc);
+ ret = VERR_NO_CONN;
+ }
+ }
+ } else
ret = VERR_NO_ASSOC;
}
FCNVME_LS_DISCONNECT);
- if (rqst->discon_cmd.scope == FCNVME_DISCONN_CONNECTION) {
- queue = nvmet_fc_find_target_queue(tgtport,
- be64_to_cpu(rqst->discon_cmd.id));
- if (queue) {
- int qid = queue->qid;
+ /* are we to delete a Connection ID (queue) */
+ if (queue) {
+ int qid = queue->qid;
- nvmet_fc_delete_target_queue(queue);
+ nvmet_fc_delete_target_queue(queue);
- /* release the get taken by find_target_queue */
- nvmet_fc_tgt_q_put(queue);
+ /* release the get taken by find_target_queue */
+ nvmet_fc_tgt_q_put(queue);
- /* tear association down if io queue terminated */
- if (!qid)
- del_assoc = true;
- }
+ /* tear association down if io queue terminated */
+ if (!qid)
+ del_assoc = true;
}
/* release get taken in nvmet_fc_find_target_assoc */
struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
enum nvme_subsys_type type);
void nvmet_subsys_put(struct nvmet_subsys *subsys);
+void nvmet_subsys_del_ctrls(struct nvmet_subsys *subsys);
struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl, __le32 nsid);
void nvmet_put_namespace(struct nvmet_ns *ns);
{
struct ib_recv_wr *bad_wr;
+ ib_dma_sync_single_for_device(ndev->device,
+ cmd->sge[0].addr, cmd->sge[0].length,
+ DMA_FROM_DEVICE);
+
if (ndev->srq)
return ib_post_srq_recv(ndev->srq, &cmd->wr, &bad_wr);
return ib_post_recv(cmd->queue->cm_id->qp, &cmd->wr, &bad_wr);
first_wr = &rsp->send_wr;
nvmet_rdma_post_recv(rsp->queue->dev, rsp->cmd);
+
+ ib_dma_sync_single_for_device(rsp->queue->dev->device,
+ rsp->send_sge.addr, rsp->send_sge.length,
+ DMA_TO_DEVICE);
+
if (ib_post_send(cm_id->qp, first_wr, &bad_wr)) {
pr_err("sending cmd response failed\n");
nvmet_rdma_release_rsp(rsp);
cmd->n_rdma = 0;
cmd->req.port = queue->port;
+
+ ib_dma_sync_single_for_cpu(queue->dev->device,
+ cmd->cmd->sge[0].addr, cmd->cmd->sge[0].length,
+ DMA_FROM_DEVICE);
+ ib_dma_sync_single_for_cpu(queue->dev->device,
+ cmd->send_sge.addr, cmd->send_sge.length,
+ DMA_TO_DEVICE);
+
if (!nvmet_req_init(&cmd->req, &queue->nvme_cq,
&queue->nvme_sq, &nvmet_rdma_ops))
return;
p->irq = PARPORT_IRQ_NONE;
}
if (p->irq != PARPORT_IRQ_NONE) {
- printk(", irq %d", p->irq);
+ pr_cont(", irq %d", p->irq);
if (p->dma == PARPORT_DMA_AUTO) {
p->dma = PARPORT_DMA_NONE;
is mandatory (see above) */
p->dma = PARPORT_DMA_NONE;
- printk(" [");
-#define printmode(x) {if(p->modes&PARPORT_MODE_##x){printk("%s%s",f?",":"",#x);f++;}}
+ pr_cont(" [");
+#define printmode(x) {if(p->modes&PARPORT_MODE_##x){pr_cont("%s%s",f?",":"",#x);f++;}}
{
int f = 0;
printmode(PCSPP);
// printmode(DMA);
}
#undef printmode
- printk("]\n");
+ pr_cont("]\n");
if (p->irq != PARPORT_IRQ_NONE) {
if (request_irq (p->irq, parport_irq_handler,
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/slab.h>
-#include <linux/pm_runtime.h>
#include <linux/pci.h>
#include "../pci.h"
#include "pciehp.h"
pciehp_green_led_blink(p_slot);
/* Check link training status */
- pm_runtime_get_sync(&ctrl->pcie->port->dev);
retval = pciehp_check_link_status(ctrl);
if (retval) {
ctrl_err(ctrl, "Failed to check link status\n");
if (retval != -EEXIST)
goto err_exit;
}
- pm_runtime_put(&ctrl->pcie->port->dev);
pciehp_green_led_on(p_slot);
pciehp_set_attention_status(p_slot, 0);
return 0;
err_exit:
- pm_runtime_put(&ctrl->pcie->port->dev);
set_slot_off(ctrl, p_slot);
return retval;
}
int retval;
struct controller *ctrl = p_slot->ctrl;
- pm_runtime_get_sync(&ctrl->pcie->port->dev);
retval = pciehp_unconfigure_device(p_slot);
- pm_runtime_put(&ctrl->pcie->port->dev);
if (retval)
return retval;
if (flags & PCI_IRQ_AFFINITY) {
if (!affd)
affd = &msi_default_affd;
+
+ if (affd->pre_vectors + affd->post_vectors > min_vecs)
+ return -EINVAL;
+
+ /*
+ * If there aren't any vectors left after applying the pre/post
+ * vectors don't bother with assigning affinity.
+ */
+ if (affd->pre_vectors + affd->post_vectors == min_vecs)
+ affd = NULL;
} else {
if (WARN_ON(affd))
affd = NULL;
return false;
/*
- * Hotplug ports handled by firmware in System Management Mode
+ * Hotplug interrupts cannot be delivered if the link is down,
+ * so parents of a hotplug port must stay awake. In addition,
+ * hotplug ports handled by firmware in System Management Mode
* may not be put into D3 by the OS (Thunderbolt on non-Macs).
+ * For simplicity, disallow in general for now.
*/
- if (bridge->is_hotplug_bridge && !pciehp_is_native(bridge))
+ if (bridge->is_hotplug_bridge)
return false;
if (pci_bridge_d3_force)
!pci_pme_capable(dev, PCI_D3cold)) ||
/* If it is a bridge it must be allowed to go to D3. */
- !pci_power_manageable(dev) ||
-
- /* Hotplug interrupts cannot be delivered if the link is down. */
- dev->is_hotplug_bridge)
+ !pci_power_manageable(dev))
*d3cold_ok = false;
link = kzalloc(sizeof(*link), GFP_KERNEL);
if (!link)
return NULL;
+
INIT_LIST_HEAD(&link->sibling);
INIT_LIST_HEAD(&link->children);
INIT_LIST_HEAD(&link->link);
link->pdev = pdev;
- if (pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT) {
+
+ /*
+ * Root Ports and PCI/PCI-X to PCIe Bridges are roots of PCIe
+ * hierarchies.
+ */
+ if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT ||
+ pci_pcie_type(pdev) == PCI_EXP_TYPE_PCIE_BRIDGE) {
+ link->root = link;
+ } else {
struct pcie_link_state *parent;
+
parent = pdev->bus->parent->self->link_state;
if (!parent) {
kfree(link);
return NULL;
}
+
link->parent = parent;
+ link->root = link->parent->root;
list_add(&link->link, &parent->children);
}
- /* Setup a pointer to the root port link */
- if (!link->parent)
- link->root = link;
- else
- link->root = link->parent->root;
list_add(&link->sibling, &link_list);
pdev->link_state = link;
BERLIN_PINCTRL_GROUP("SCRD0_CRD_PRES", 0xc, 0x3, 0x15,
BERLIN_PINCTRL_FUNCTION(0x0, "gpio"), /* GPIO20 */
BERLIN_PINCTRL_FUNCTION(0x1, "scrd0"), /* crd pres */
- BERLIN_PINCTRL_FUNCTION(0x1, "sd1a")), /* DAT3 */
+ BERLIN_PINCTRL_FUNCTION(0x3, "sd1a")), /* DAT3 */
BERLIN_PINCTRL_GROUP("SPI1_SS0n", 0xc, 0x3, 0x18,
BERLIN_PINCTRL_FUNCTION(0x0, "spi1"), /* SS0n */
BERLIN_PINCTRL_FUNCTION(0x1, "gpio"), /* GPIO37 */
int reg)
{
struct byt_community *comm = byt_get_community(vg, offset);
- u32 reg_offset = 0;
+ u32 reg_offset;
if (!comm)
return NULL;
offset -= comm->pin_base;
- if (reg == BYT_INT_STAT_REG)
+ switch (reg) {
+ case BYT_INT_STAT_REG:
reg_offset = (offset / 32) * 4;
- else
+ break;
+ case BYT_DEBOUNCE_REG:
+ reg_offset = 0;
+ break;
+ default:
reg_offset = comm->pad_map[offset] * 16;
+ break;
+ }
return comm->reg_base + reg_offset + reg;
}
enum pin_config_param param = pinconf_to_config_param(*config);
void __iomem *conf_reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
void __iomem *val_reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
+ void __iomem *db_reg = byt_gpio_reg(vg, offset, BYT_DEBOUNCE_REG);
unsigned long flags;
u32 conf, pull, val, debounce;
u16 arg = 0;
return -EINVAL;
raw_spin_lock_irqsave(&vg->lock, flags);
- debounce = readl(byt_gpio_reg(vg, offset, BYT_DEBOUNCE_REG));
+ debounce = readl(db_reg);
raw_spin_unlock_irqrestore(&vg->lock, flags);
switch (debounce & BYT_DEBOUNCE_PULSE_MASK) {
unsigned int param, arg;
void __iomem *conf_reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
void __iomem *val_reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
+ void __iomem *db_reg = byt_gpio_reg(vg, offset, BYT_DEBOUNCE_REG);
unsigned long flags;
u32 conf, val, debounce;
int i, ret = 0;
break;
case PIN_CONFIG_INPUT_DEBOUNCE:
- debounce = readl(byt_gpio_reg(vg, offset,
- BYT_DEBOUNCE_REG));
- conf &= ~BYT_DEBOUNCE_PULSE_MASK;
+ debounce = readl(db_reg);
+ debounce &= ~BYT_DEBOUNCE_PULSE_MASK;
+
+ if (arg)
+ conf |= BYT_DEBOUNCE_EN;
+ else
+ conf &= ~BYT_DEBOUNCE_EN;
switch (arg) {
case 375:
- conf |= BYT_DEBOUNCE_PULSE_375US;
+ debounce |= BYT_DEBOUNCE_PULSE_375US;
break;
case 750:
- conf |= BYT_DEBOUNCE_PULSE_750US;
+ debounce |= BYT_DEBOUNCE_PULSE_750US;
break;
case 1500:
- conf |= BYT_DEBOUNCE_PULSE_1500US;
+ debounce |= BYT_DEBOUNCE_PULSE_1500US;
break;
case 3000:
- conf |= BYT_DEBOUNCE_PULSE_3MS;
+ debounce |= BYT_DEBOUNCE_PULSE_3MS;
break;
case 6000:
- conf |= BYT_DEBOUNCE_PULSE_6MS;
+ debounce |= BYT_DEBOUNCE_PULSE_6MS;
break;
case 12000:
- conf |= BYT_DEBOUNCE_PULSE_12MS;
+ debounce |= BYT_DEBOUNCE_PULSE_12MS;
break;
case 24000:
- conf |= BYT_DEBOUNCE_PULSE_24MS;
+ debounce |= BYT_DEBOUNCE_PULSE_24MS;
break;
default:
- ret = -EINVAL;
+ if (arg)
+ ret = -EINVAL;
+ break;
}
+ if (!ret)
+ writel(debounce, db_reg);
break;
default:
ret = -ENOTSUPP;
continue;
}
+ raw_spin_lock(&vg->lock);
pending = readl(reg);
+ raw_spin_unlock(&vg->lock);
for_each_set_bit(pin, &pending, 32) {
virq = irq_find_mapping(vg->chip.irqdomain, base + pin);
generic_handle_irq(virq);
static void byt_gpio_irq_init_hw(struct byt_gpio *vg)
{
+ struct gpio_chip *gc = &vg->chip;
+ struct device *dev = &vg->pdev->dev;
void __iomem *reg;
u32 base, value;
int i;
}
value = readl(reg);
- if ((value & BYT_PIN_MUX) == byt_get_gpio_mux(vg, i) &&
- !(value & BYT_DIRECT_IRQ_EN)) {
+ if (value & BYT_DIRECT_IRQ_EN) {
+ clear_bit(i, gc->irq_valid_mask);
+ dev_dbg(dev, "excluding GPIO %d from IRQ domain\n", i);
+ } else if ((value & BYT_PIN_MUX) == byt_get_gpio_mux(vg, i)) {
byt_gpio_clear_triggering(vg, i);
- dev_dbg(&vg->pdev->dev, "disabling GPIO %d\n", i);
+ dev_dbg(dev, "disabling GPIO %d\n", i);
}
}
gc->can_sleep = false;
gc->parent = &vg->pdev->dev;
gc->ngpio = vg->soc_data->npins;
+ gc->irq_need_valid_mask = true;
#ifdef CONFIG_PM_SLEEP
vg->saved_context = devm_kcalloc(&vg->pdev->dev, gc->ngpio,
#define BXT_PAD_OWN 0x020
#define BXT_HOSTSW_OWN 0x080
-#define BXT_PADCFGLOCK 0x090
+#define BXT_PADCFGLOCK 0x060
#define BXT_GPI_IE 0x110
#define BXT_COMMUNITY(s, e) \
return 0;
}
+static void __intel_gpio_set_direction(void __iomem *padcfg0, bool input)
+{
+ u32 value;
+
+ value = readl(padcfg0);
+ if (input) {
+ value &= ~PADCFG0_GPIORXDIS;
+ value |= PADCFG0_GPIOTXDIS;
+ } else {
+ value &= ~PADCFG0_GPIOTXDIS;
+ value |= PADCFG0_GPIORXDIS;
+ }
+ writel(value, padcfg0);
+}
+
static int intel_gpio_request_enable(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range,
unsigned pin)
/* Disable SCI/SMI/NMI generation */
value &= ~(PADCFG0_GPIROUTIOXAPIC | PADCFG0_GPIROUTSCI);
value &= ~(PADCFG0_GPIROUTSMI | PADCFG0_GPIROUTNMI);
- /* Disable TX buffer and enable RX (this will be input) */
- value &= ~PADCFG0_GPIORXDIS;
- value |= PADCFG0_GPIOTXDIS;
writel(value, padcfg0);
+ /* Disable TX buffer and enable RX (this will be input) */
+ __intel_gpio_set_direction(padcfg0, true);
+
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
return 0;
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
void __iomem *padcfg0;
unsigned long flags;
- u32 value;
raw_spin_lock_irqsave(&pctrl->lock, flags);
padcfg0 = intel_get_padcfg(pctrl, pin, PADCFG0);
-
- value = readl(padcfg0);
- if (input)
- value |= PADCFG0_GPIOTXDIS;
- else
- value &= ~PADCFG0_GPIOTXDIS;
- writel(value, padcfg0);
+ __intel_gpio_set_direction(padcfg0, input);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
unsigned int i;
int ret;
+ if (!mrfld_buf_available(mp, pin))
+ return -ENOTSUPP;
+
for (i = 0; i < nconfigs; i++) {
switch (pinconf_to_config_param(configs[i])) {
case PIN_CONFIG_BIAS_DISABLE:
static const unsigned int uart_rx_ao_a_pins[] = { PIN(GPIOAO_1, 0) };
static const unsigned int uart_cts_ao_a_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_a_pins[] = { PIN(GPIOAO_3, 0) };
-static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_0, 0) };
-static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_1, 0),
- PIN(GPIOAO_5, 0) };
+static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_4, 0) };
+static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_5, 0) };
static const unsigned int uart_cts_ao_b_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_b_pins[] = { PIN(GPIOAO_3, 0) };
GPIO_GROUP(GPIOAO_13, 0),
/* bank AO */
- GROUP(uart_tx_ao_b, 0, 26),
+ GROUP(uart_tx_ao_b, 0, 24),
GROUP(uart_rx_ao_b, 0, 25),
GROUP(uart_tx_ao_a, 0, 12),
GROUP(uart_rx_ao_a, 0, 11),
static const unsigned int uart_rx_ao_a_pins[] = { PIN(GPIOAO_1, 0) };
static const unsigned int uart_cts_ao_a_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_a_pins[] = { PIN(GPIOAO_3, 0) };
-static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_0, 0) };
-static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_1, 0),
- PIN(GPIOAO_5, 0) };
+static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_4, 0) };
+static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_5, 0) };
static const unsigned int uart_cts_ao_b_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_b_pins[] = { PIN(GPIOAO_3, 0) };
GPIO_GROUP(GPIOAO_9, 0),
/* bank AO */
- GROUP(uart_tx_ao_b, 0, 26),
+ GROUP(uart_tx_ao_b, 0, 24),
GROUP(uart_rx_ao_b, 0, 25),
GROUP(uart_tx_ao_a, 0, 12),
GROUP(uart_rx_ao_a, 0, 11),
i = 128;
pin_num = AMD_GPIO_PINS_BANK2 + i;
break;
+ default:
+ return;
}
for (; i < pin_num; i++) {
val = arg / 10 - 1;
break;
case PIN_CONFIG_BIAS_DISABLE:
- val = 0;
- break;
+ continue;
case PIN_CONFIG_BIAS_PULL_UP:
if (arg == 0)
return -EINVAL;
0, 0, 0, 0};
static const unsigned ether_rmii_pins[] = {30, 31, 32, 33, 34, 35, 36, 37, 39,
41, 42, 45};
-static const int ether_rmii_muxvals[] = {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1};
+static const int ether_rmii_muxvals[] = {0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1};
static const unsigned i2c0_pins[] = {63, 64};
static const int i2c0_muxvals[] = {0, 0};
static const unsigned i2c1_pins[] = {65, 66};
case 8:
case 7:
case 6:
+ case 1:
ideapad_input_report(priv, vpc_bit);
break;
case 5:
input_set_capability(input, EV_KEY, KEY_POWER);
- error = request_threaded_irq(irq, NULL, mfld_pb_isr, 0,
+ error = request_threaded_irq(irq, NULL, mfld_pb_isr, IRQF_ONESHOT,
DRIVER_NAME, input);
if (error) {
dev_err(&pdev->dev, "Unable to request irq %d for mfld power"
return 0;
fail_platform_mux_register:
- for (i--; i > 0 ; i--)
+ while (--i >= 0)
platform_device_unregister(priv->pdev_mux[i]);
platform_device_unregister(priv->pdev_i2c);
fail_alloc:
static int s3_wmi_check_platform_device(struct device *dev, void *data)
{
- struct acpi_device *adev, *ts_adev;
+ struct acpi_device *adev, *ts_adev = NULL;
acpi_handle handle;
acpi_status status;
return 0;
}
-#ifdef CONFIG_PM
-static int s3_wmi_resume(struct device *dev)
+static int __maybe_unused s3_wmi_resume(struct device *dev)
{
s3_wmi_send_lid_state();
return 0;
}
-#endif
static SIMPLE_DEV_PM_OPS(s3_wmi_pm, NULL, s3_wmi_resume);
static struct platform_driver s3_wmi_driver = {
64, AXP806_DCDCD_V_CTRL, 0x3f, AXP806_PWR_OUT_CTRL1,
BIT(3)),
AXP_DESC(AXP806, DCDCE, "dcdce", "vine", 1100, 3400, 100,
- AXP806_DCDCB_V_CTRL, 0x1f, AXP806_PWR_OUT_CTRL1, BIT(4)),
+ AXP806_DCDCE_V_CTRL, 0x1f, AXP806_PWR_OUT_CTRL1, BIT(4)),
AXP_DESC(AXP806, ALDO1, "aldo1", "aldoin", 700, 3300, 100,
AXP806_ALDO1_V_CTRL, 0x1f, AXP806_PWR_OUT_CTRL1, BIT(5)),
AXP_DESC(AXP806, ALDO2, "aldo2", "aldoin", 700, 3400, 100,
#include <linux/of_gpio.h>
#include <linux/regulator/of_regulator.h>
#include <linux/regulator/machine.h>
-#include <linux/acpi.h>
-#include <linux/property.h>
-#include <linux/gpio/consumer.h>
struct fixed_voltage_data {
struct regulator_desc desc;
return config;
}
-/**
- * acpi_get_fixed_voltage_config - extract fixed_voltage_config structure info
- * @dev: device requesting for fixed_voltage_config
- * @desc: regulator description
- *
- * Populates fixed_voltage_config structure by extracting data through ACPI
- * interface, returns a pointer to the populated structure of NULL if memory
- * alloc fails.
- */
-static struct fixed_voltage_config *
-acpi_get_fixed_voltage_config(struct device *dev,
- const struct regulator_desc *desc)
-{
- struct fixed_voltage_config *config;
- const char *supply_name;
- struct gpio_desc *gpiod;
- int ret;
-
- config = devm_kzalloc(dev, sizeof(*config), GFP_KERNEL);
- if (!config)
- return ERR_PTR(-ENOMEM);
-
- ret = device_property_read_string(dev, "supply-name", &supply_name);
- if (!ret)
- config->supply_name = supply_name;
-
- gpiod = gpiod_get(dev, "gpio", GPIOD_ASIS);
- if (IS_ERR(gpiod))
- return ERR_PTR(-ENODEV);
-
- config->gpio = desc_to_gpio(gpiod);
- config->enable_high = device_property_read_bool(dev,
- "enable-active-high");
- gpiod_put(gpiod);
-
- return config;
-}
-
static struct regulator_ops fixed_voltage_ops = {
};
&drvdata->desc);
if (IS_ERR(config))
return PTR_ERR(config);
- } else if (ACPI_HANDLE(&pdev->dev)) {
- config = acpi_get_fixed_voltage_config(&pdev->dev,
- &drvdata->desc);
- if (IS_ERR(config))
- return PTR_ERR(config);
} else {
config = dev_get_platdata(&pdev->dev);
}
vsel = 62;
else if ((min_uV > 1800000) && (min_uV <= 1900000))
vsel = 61;
- else if ((min_uV > 1350000) && (min_uV <= 1800000))
+ else if ((min_uV > 1500000) && (min_uV <= 1800000))
vsel = 60;
else if ((min_uV > 1350000) && (min_uV <= 1500000))
vsel = 59;
will be called rtc-mpc5121.
config RTC_DRV_JZ4740
- bool "Ingenic JZ4740 SoC"
+ tristate "Ingenic JZ4740 SoC"
depends on MACH_INGENIC || COMPILE_TEST
help
If you say yes here you get support for the Ingenic JZ47xx SoCs RTC
controllers.
+ This driver can also be buillt as a module. If so, the module
+ will be called rtc-jz4740.
+
config RTC_DRV_LPC24XX
tristate "NXP RTC for LPC178x/18xx/408x/43xx"
depends on ARCH_LPC18XX || COMPILE_TEST
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
#include <linux/reboot.h>
JZ_REG_RTC_RESET_COUNTER, reset_counter_ticks);
jz4740_rtc_poweroff(dev_for_power_off);
- machine_halt();
+ kernel_halt();
}
static const struct of_device_id jz4740_rtc_of_match[] = {
{ .compatible = "ingenic,jz4780-rtc", .data = (void *)ID_JZ4780 },
{},
};
+MODULE_DEVICE_TABLE(of, jz4740_rtc_of_match);
static int jz4740_rtc_probe(struct platform_device *pdev)
{
{ "jz4780-rtc", ID_JZ4780 },
{}
};
+MODULE_DEVICE_TABLE(platform, jz4740_rtc_ids);
static struct platform_driver jz4740_rtc_driver = {
.probe = jz4740_rtc_probe,
.id_table = jz4740_rtc_ids,
};
-builtin_platform_driver(jz4740_rtc_driver);
+module_platform_driver(jz4740_rtc_driver);
+
+MODULE_AUTHOR("Lars-Peter Clausen <lars@metafoo.de>");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("RTC driver for the JZ4740 SoC\n");
+MODULE_ALIAS("platform:jz4740-rtc");
int zfcp_fsf_open_wka_port(struct zfcp_fc_wka_port *wka_port)
{
struct zfcp_qdio *qdio = wka_port->adapter->qdio;
- struct zfcp_fsf_req *req = NULL;
+ struct zfcp_fsf_req *req;
int retval = -EIO;
spin_lock_irq(&qdio->req_q_lock);
zfcp_fsf_req_free(req);
out:
spin_unlock_irq(&qdio->req_q_lock);
- if (req && !IS_ERR(req))
+ if (!retval)
zfcp_dbf_rec_run_wka("fsowp_1", wka_port, req->req_id);
return retval;
}
int zfcp_fsf_close_wka_port(struct zfcp_fc_wka_port *wka_port)
{
struct zfcp_qdio *qdio = wka_port->adapter->qdio;
- struct zfcp_fsf_req *req = NULL;
+ struct zfcp_fsf_req *req;
int retval = -EIO;
spin_lock_irq(&qdio->req_q_lock);
zfcp_fsf_req_free(req);
out:
spin_unlock_irq(&qdio->req_q_lock);
- if (req && !IS_ERR(req))
+ if (!retval)
zfcp_dbf_rec_run_wka("fscwp_1", wka_port, req->req_id);
return retval;
}
static inline int aac_is_msix_mode(struct aac_dev *dev)
{
- u32 status;
+ u32 status = 0;
- status = src_readl(dev, MUnit.OMR);
+ if (dev->pdev->device == PMC_DEVICE_S6 ||
+ dev->pdev->device == PMC_DEVICE_S7 ||
+ dev->pdev->device == PMC_DEVICE_S8) {
+ status = src_readl(dev, MUnit.OMR);
+ }
return (status & AAC_INT_MODE_MSIX);
}
static const struct target_core_fabric_ops ibmvscsis_ops = {
.module = THIS_MODULE,
.name = "ibmvscsis",
+ .max_data_sg_nents = MAX_TXU / PAGE_SIZE,
.get_fabric_name = ibmvscsis_get_fabric_name,
.tpg_get_wwn = ibmvscsis_get_fabric_wwn,
.tpg_get_tag = ibmvscsis_get_tag,
#include <linux/workqueue.h>
#include <linux/delay.h>
#include <linux/pci.h>
+#include <linux/pci-aspm.h>
#include <linux/interrupt.h>
#include <linux/aer.h>
#include <linux/raid_class.h>
struct MPT3SAS_DEVICE *sas_device_priv_data;
u32 response_code = 0;
unsigned long flags;
+ unsigned int sector_sz;
mpi_reply = mpt3sas_base_get_reply_virt_addr(ioc, reply);
scmd = _scsih_scsi_lookup_get_clear(ioc, smid);
}
xfer_cnt = le32_to_cpu(mpi_reply->TransferCount);
+
+ /* In case of bogus fw or device, we could end up having
+ * unaligned partial completion. We can force alignment here,
+ * then scsi-ml does not need to handle this misbehavior.
+ */
+ sector_sz = scmd->device->sector_size;
+ if (unlikely(scmd->request->cmd_type == REQ_TYPE_FS && sector_sz &&
+ xfer_cnt % sector_sz)) {
+ sdev_printk(KERN_INFO, scmd->device,
+ "unaligned partial completion avoided (xfer_cnt=%u, sector_sz=%u)\n",
+ xfer_cnt, sector_sz);
+ xfer_cnt = round_down(xfer_cnt, sector_sz);
+ }
+
scsi_set_resid(scmd, scsi_bufflen(scmd) - xfer_cnt);
if (ioc_status & MPI2_IOCSTATUS_FLAG_LOG_INFO_AVAILABLE)
log_info = le32_to_cpu(mpi_reply->IOCLogInfo);
switch (hba_mpi_version) {
case MPI2_VERSION:
+ pci_disable_link_state(pdev, PCIE_LINK_STATE_L0S |
+ PCIE_LINK_STATE_L1 | PCIE_LINK_STATE_CLKPM);
/* Use mpt2sas driver host template for SAS 2.0 HBA's */
shost = scsi_host_alloc(&mpt2sas_driver_template,
sizeof(struct MPT3SAS_ADAPTER));
* from a probe failure context.
*/
if (!ha->rsp_q_map || !ha->rsp_q_map[0])
- return;
+ goto free_irqs;
rsp = ha->rsp_q_map[0];
if (ha->flags.msix_enabled) {
free_irq(pci_irq_vector(ha->pdev, 0), rsp);
}
+free_irqs:
pci_free_irq_vectors(ha->pdev);
}
/* Don't abort commands in adapter during EEH
* recovery as it's not accessible/responding.
*/
- if (!ha->flags.eeh_busy) {
+ if (GET_CMD_SP(sp) && !ha->flags.eeh_busy) {
/* Get a reference to the sp and drop the lock.
* The reference ensures this sp->done() call
* - and not the call in qla2xxx_eh_abort() -
struct bio *bio = rq->bio;
sector_t sector = blk_rq_pos(rq);
unsigned int nr_sectors = blk_rq_sectors(rq);
+ unsigned int nr_bytes = blk_rq_bytes(rq);
int ret;
if (sdkp->device->no_write_same)
cmd->transfersize = sdp->sector_size;
cmd->allowed = SD_MAX_RETRIES;
- return scsi_init_io(cmd);
+
+ /*
+ * For WRITE SAME the data transferred via the DATA OUT buffer is
+ * different from the amount of data actually written to the target.
+ *
+ * We set up __data_len to the amount of data transferred via the
+ * DATA OUT buffer so that blk_rq_map_sg sets up the proper S/G list
+ * to transfer a single sector of data first, but then reset it to
+ * the amount of data to be written right after so that the I/O path
+ * knows how much to actually write.
+ */
+ rq->__data_len = sdp->sector_size;
+ ret = scsi_init_io(cmd);
+ rq->__data_len = nr_bytes;
+ return ret;
}
static int sd_setup_flush_cmnd(struct scsi_cmnd *cmd)
{
struct Scsi_Host *shost = virtio_scsi_host(vscsi->vdev);
struct virtio_scsi_cmd *cmd = scsi_cmd_priv(sc);
+ unsigned long flags;
int req_size;
+ int ret;
BUG_ON(scsi_sg_count(sc) > shost->sg_tablesize);
req_size = sizeof(cmd->req.cmd);
}
- if (virtscsi_kick_cmd(req_vq, cmd, req_size, sizeof(cmd->resp.cmd)) != 0)
+ ret = virtscsi_kick_cmd(req_vq, cmd, req_size, sizeof(cmd->resp.cmd));
+ if (ret == -EIO) {
+ cmd->resp.cmd.response = VIRTIO_SCSI_S_BAD_TARGET;
+ spin_lock_irqsave(&req_vq->vq_lock, flags);
+ virtscsi_complete_cmd(vscsi, cmd);
+ spin_unlock_irqrestore(&req_vq->vq_lock, flags);
+ } else if (ret != 0) {
return SCSI_MLQUEUE_HOST_BUSY;
+ }
return 0;
}
int gb_timesync_platform_lock_bus(struct gb_timesync_svc *pdata)
{
+ if (!arche_platform_change_state_cb)
+ return 0;
+
return arche_platform_change_state_cb(ARCHE_PLATFORM_STATE_TIME_SYNC,
pdata);
}
void gb_timesync_platform_unlock_bus(void)
{
+ if (!arche_platform_change_state_cb)
+ return;
+
arche_platform_change_state_cb(ARCHE_PLATFORM_STATE_ACTIVE, NULL);
}
result = VM_FAULT_LOCKED;
break;
case -ENODATA:
+ case -EAGAIN:
case -EFAULT:
result = VM_FAULT_NOPAGE;
break;
case -ENOMEM:
result = VM_FAULT_OOM;
break;
- case -EAGAIN:
- result = VM_FAULT_RETRY;
- break;
default:
result = VM_FAULT_SIGBUS;
break;
kfree(new);
return -EINVAL;
}
- BUG_ON(orig->se_lun_acl != NULL);
+ if (orig->se_lun_acl != NULL) {
+ pr_warn_ratelimited("Detected existing explicit"
+ " se_lun_acl->se_lun_group reference for %s"
+ " mapped_lun: %llu, failing\n",
+ nacl->initiatorname, mapped_lun);
+ mutex_unlock(&nacl->lun_entry_mutex);
+ kfree(new);
+ return -EINVAL;
+ }
rcu_assign_pointer(new->se_lun, lun);
rcu_assign_pointer(new->se_lun_acl, lun_acl);
int *post_ret)
{
struct se_device *dev = cmd->se_dev;
+ sense_reason_t ret = TCM_NO_SENSE;
/*
* Only set SCF_COMPARE_AND_WRITE_POST to force a response fall-through
* sent to the backend driver.
*/
spin_lock_irq(&cmd->t_state_lock);
- if ((cmd->transport_state & CMD_T_SENT) && !cmd->scsi_status) {
+ if (cmd->transport_state & CMD_T_SENT) {
cmd->se_cmd_flags |= SCF_COMPARE_AND_WRITE_POST;
*post_ret = 1;
+
+ if (cmd->scsi_status == SAM_STAT_CHECK_CONDITION)
+ ret = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
}
spin_unlock_irq(&cmd->t_state_lock);
*/
up(&dev->caw_sem);
- return TCM_NO_SENSE;
+ return ret;
}
static sense_reason_t compare_and_write_callback(struct se_cmd *cmd, bool success,
{
struct se_node_acl *nacl = container_of(kref,
struct se_node_acl, acl_kref);
+ struct se_portal_group *se_tpg = nacl->se_tpg;
- complete(&nacl->acl_free_comp);
+ if (!nacl->dynamic_stop) {
+ complete(&nacl->acl_free_comp);
+ return;
+ }
+
+ mutex_lock(&se_tpg->acl_node_mutex);
+ list_del(&nacl->acl_list);
+ mutex_unlock(&se_tpg->acl_node_mutex);
+
+ core_tpg_wait_for_nacl_pr_ref(nacl);
+ core_free_device_list_for_node(nacl, se_tpg);
+ kfree(nacl);
}
void target_put_nacl(struct se_node_acl *nacl)
void transport_free_session(struct se_session *se_sess)
{
struct se_node_acl *se_nacl = se_sess->se_node_acl;
+
/*
* Drop the se_node_acl->nacl_kref obtained from within
* core_tpg_get_initiator_node_acl().
*/
if (se_nacl) {
+ struct se_portal_group *se_tpg = se_nacl->se_tpg;
+ const struct target_core_fabric_ops *se_tfo = se_tpg->se_tpg_tfo;
+ unsigned long flags;
+
se_sess->se_node_acl = NULL;
+
+ /*
+ * Also determine if we need to drop the extra ->cmd_kref if
+ * it had been previously dynamically generated, and
+ * the endpoint is not caching dynamic ACLs.
+ */
+ mutex_lock(&se_tpg->acl_node_mutex);
+ if (se_nacl->dynamic_node_acl &&
+ !se_tfo->tpg_check_demo_mode_cache(se_tpg)) {
+ spin_lock_irqsave(&se_nacl->nacl_sess_lock, flags);
+ if (list_empty(&se_nacl->acl_sess_list))
+ se_nacl->dynamic_stop = true;
+ spin_unlock_irqrestore(&se_nacl->nacl_sess_lock, flags);
+
+ if (se_nacl->dynamic_stop)
+ list_del(&se_nacl->acl_list);
+ }
+ mutex_unlock(&se_tpg->acl_node_mutex);
+
+ if (se_nacl->dynamic_stop)
+ target_put_nacl(se_nacl);
+
target_put_nacl(se_nacl);
}
if (se_sess->sess_cmd_map) {
void transport_deregister_session(struct se_session *se_sess)
{
struct se_portal_group *se_tpg = se_sess->se_tpg;
- const struct target_core_fabric_ops *se_tfo;
- struct se_node_acl *se_nacl;
unsigned long flags;
- bool drop_nacl = false;
if (!se_tpg) {
transport_free_session(se_sess);
return;
}
- se_tfo = se_tpg->se_tpg_tfo;
spin_lock_irqsave(&se_tpg->session_lock, flags);
list_del(&se_sess->sess_list);
se_sess->fabric_sess_ptr = NULL;
spin_unlock_irqrestore(&se_tpg->session_lock, flags);
- /*
- * Determine if we need to do extra work for this initiator node's
- * struct se_node_acl if it had been previously dynamically generated.
- */
- se_nacl = se_sess->se_node_acl;
-
- mutex_lock(&se_tpg->acl_node_mutex);
- if (se_nacl && se_nacl->dynamic_node_acl) {
- if (!se_tfo->tpg_check_demo_mode_cache(se_tpg)) {
- list_del(&se_nacl->acl_list);
- drop_nacl = true;
- }
- }
- mutex_unlock(&se_tpg->acl_node_mutex);
-
- if (drop_nacl) {
- core_tpg_wait_for_nacl_pr_ref(se_nacl);
- core_free_device_list_for_node(se_nacl, se_tpg);
- se_sess->se_node_acl = NULL;
- kfree(se_nacl);
- }
pr_debug("TARGET_CORE[%s]: Deregistered fabric_sess\n",
se_tpg->se_tpg_tfo->get_fabric_name());
/*
* If last kref is dropping now for an explicit NodeACL, awake sleeping
* ->acl_free_comp caller to wakeup configfs se_node_acl->acl_group
* removal context from within transport_free_session() code.
+ *
+ * For dynamic ACL, target_put_nacl() uses target_complete_nacl()
+ * to release all remaining generate_node_acl=1 created ACL resources.
*/
transport_free_session(se_sess);
spin_unlock_irqrestore(&cmd->t_state_lock, flags);
goto check_stop;
}
- cmd->t_state = TRANSPORT_ISTATE_PROCESSING;
spin_unlock_irqrestore(&cmd->t_state_lock, flags);
cmd->se_tfo->queue_tm_rsp(cmd);
struct se_cmd *cmd)
{
unsigned long flags;
+ bool aborted = false;
spin_lock_irqsave(&cmd->t_state_lock, flags);
- cmd->transport_state |= CMD_T_ACTIVE;
+ if (cmd->transport_state & CMD_T_ABORTED) {
+ aborted = true;
+ } else {
+ cmd->t_state = TRANSPORT_ISTATE_PROCESSING;
+ cmd->transport_state |= CMD_T_ACTIVE;
+ }
spin_unlock_irqrestore(&cmd->t_state_lock, flags);
+ if (aborted) {
+ pr_warn_ratelimited("handle_tmr caught CMD_T_ABORTED TMR %d"
+ "ref_tag: %llu tag: %llu\n", cmd->se_tmr_req->function,
+ cmd->se_tmr_req->ref_task_tag, cmd->tag);
+ transport_cmd_check_stop_to_fabric(cmd);
+ return 0;
+ }
+
INIT_WORK(&cmd->work, target_tmr_work);
queue_work(cmd->se_dev->tmr_wq, &cmd->work);
return 0;
" CHECK_CONDITION -> sending response\n", rc);
ec_cmd->scsi_status = SAM_STAT_CHECK_CONDITION;
}
- target_complete_cmd(ec_cmd, SAM_STAT_CHECK_CONDITION);
+ target_complete_cmd(ec_cmd, ec_cmd->scsi_status);
}
sense_reason_t target_do_xcopy(struct se_cmd *se_cmd)
if (!power_table)
return -ENOMEM;
- rcu_read_lock();
-
for (freq = 0, i = 0;
opp = dev_pm_opp_find_freq_ceil(dev, &freq), !IS_ERR(opp);
freq++, i++) {
u64 power;
if (i >= num_opps) {
- rcu_read_unlock();
ret = -EAGAIN;
goto free_power_table;
}
freq_mhz = freq / 1000000;
voltage_mv = dev_pm_opp_get_voltage(opp) / 1000;
+ dev_pm_opp_put(opp);
/*
* Do the multiplication with MHz and millivolt so as
power_table[i].power = power;
}
- rcu_read_unlock();
-
if (i != num_opps) {
ret = PTR_ERR(opp);
goto free_power_table;
return 0;
}
- rcu_read_lock();
-
opp = dev_pm_opp_find_freq_exact(cpufreq_device->cpu_dev, freq_hz,
true);
voltage = dev_pm_opp_get_voltage(opp);
-
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
if (voltage == 0) {
dev_warn_ratelimited(cpufreq_device->cpu_dev,
unsigned int freq = dfc->freq_table[i];
bool want_enable = i >= cdev_state ? true : false;
- rcu_read_lock();
opp = dev_pm_opp_find_freq_exact(dev, freq, !want_enable);
- rcu_read_unlock();
if (PTR_ERR(opp) == -ERANGE)
continue;
else if (IS_ERR(opp))
return PTR_ERR(opp);
+ dev_pm_opp_put(opp);
+
if (want_enable)
ret = dev_pm_opp_enable(dev, freq);
else
if (!dfc->power_ops->get_static_power)
return 0;
- rcu_read_lock();
-
opp = dev_pm_opp_find_freq_exact(dev, freq, true);
if (IS_ERR(opp) && (PTR_ERR(opp) == -ERANGE))
opp = dev_pm_opp_find_freq_exact(dev, freq, false);
voltage = dev_pm_opp_get_voltage(opp) / 1000; /* mV */
-
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
if (voltage == 0) {
dev_warn_ratelimited(dev,
unsigned long power_dyn, voltage;
struct dev_pm_opp *opp;
- rcu_read_lock();
-
opp = dev_pm_opp_find_freq_floor(dev, &freq);
if (IS_ERR(opp)) {
- rcu_read_unlock();
ret = PTR_ERR(opp);
goto free_tables;
}
voltage = dev_pm_opp_get_voltage(opp) / 1000; /* mV */
-
- rcu_read_unlock();
+ dev_pm_opp_put(opp);
if (dfc->power_ops) {
power_dyn = get_dynamic_power(dfc, freq, voltage);
static DEFINE_MUTEX(thermal_hwmon_list_lock);
+static ssize_t
+name_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct thermal_hwmon_device *hwmon = dev_get_drvdata(dev);
+ return sprintf(buf, "%s\n", hwmon->type);
+}
+static DEVICE_ATTR_RO(name);
+
static ssize_t
temp_input_show(struct device *dev, struct device_attribute *attr, char *buf)
{
INIT_LIST_HEAD(&hwmon->tz_list);
strlcpy(hwmon->type, tz->type, THERMAL_NAME_LENGTH);
- hwmon->device = hwmon_device_register_with_info(NULL, hwmon->type,
- hwmon, NULL, NULL);
+ hwmon->device = hwmon_device_register(NULL);
if (IS_ERR(hwmon->device)) {
result = PTR_ERR(hwmon->device);
goto free_mem;
}
+ dev_set_drvdata(hwmon->device, hwmon);
+ result = device_create_file(hwmon->device, &dev_attr_name);
+ if (result)
+ goto free_mem;
register_sys_interface:
temp = kzalloc(sizeof(*temp), GFP_KERNEL);
free_temp_mem:
kfree(temp);
unregister_name:
- if (new_hwmon_device)
+ if (new_hwmon_device) {
+ device_remove_file(hwmon->device, &dev_attr_name);
hwmon_device_unregister(hwmon->device);
+ }
free_mem:
if (new_hwmon_device)
kfree(hwmon);
list_del(&hwmon->node);
mutex_unlock(&thermal_hwmon_list_lock);
+ device_remove_file(hwmon->device, &dev_attr_name);
hwmon_device_unregister(hwmon->device);
kfree(hwmon);
}
/* CBM - Flash disk */
{ USB_DEVICE(0x0204, 0x6025), .driver_info = USB_QUIRK_RESET_RESUME },
+ /* WORLDE easy key (easykey.25) MIDI controller */
+ { USB_DEVICE(0x0218, 0x0401), .driver_info =
+ USB_QUIRK_CONFIG_INTF_STRINGS },
+
/* HP 5300/5370C scanner */
{ USB_DEVICE(0x03f0, 0x0701), .driver_info =
USB_QUIRK_STRING_FETCH_255 },
if (len < sizeof(*d) || h->interface >= ffs->interfaces_count)
return -EINVAL;
length = le32_to_cpu(d->dwSize);
+ if (len < length)
+ return -EINVAL;
type = le32_to_cpu(d->dwPropertyDataType);
if (type < USB_EXT_PROP_UNICODE ||
type > USB_EXT_PROP_UNICODE_MULTI) {
return -EINVAL;
}
pnl = le16_to_cpu(d->wPropertyNameLength);
+ if (length < 14 + pnl) {
+ pr_vdebug("invalid os descriptor length: %d pnl:%d (descriptor %d)\n",
+ length, pnl, type);
+ return -EINVAL;
+ }
pdl = le32_to_cpu(*(u32 *)((u8 *)data + 10 + pnl));
if (length != 14 + pnl + pdl) {
pr_vdebug("invalid os descriptor length: %d pnl:%d pdl:%d (descriptor %d)\n",
}
}
if (flags & (1 << i)) {
+ if (len < 4) {
+ goto error;
+ }
os_descs_count = get_unaligned_le32(data);
data += 4;
len -= 4;
ENTER();
- if (unlikely(get_unaligned_le32(data) != FUNCTIONFS_STRINGS_MAGIC ||
+ if (unlikely(len < 16 ||
+ get_unaligned_le32(data) != FUNCTIONFS_STRINGS_MAGIC ||
get_unaligned_le32(data + 4) != len))
goto error;
str_count = get_unaligned_le32(data + 8);
| MUSB_PORT_STAT_RESUME;
musb->rh_timer = jiffies
+ msecs_to_jiffies(USB_RESUME_TIMEOUT);
- musb->need_finish_resume = 1;
-
musb->xceiv->otg->state = OTG_STATE_A_HOST;
musb->is_active = 1;
musb_host_resume_root_hub(musb);
+ schedule_delayed_work(&musb->finish_resume_work,
+ msecs_to_jiffies(USB_RESUME_TIMEOUT));
break;
case OTG_STATE_B_WAIT_ACON:
musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
static void musb_irq_work(struct work_struct *data)
{
struct musb *musb = container_of(data, struct musb, irq_work.work);
+ int error;
+
+ error = pm_runtime_get_sync(musb->controller);
+ if (error < 0) {
+ dev_err(musb->controller, "Could not enable: %i\n", error);
+
+ return;
+ }
musb_pm_runtime_check_session(musb);
musb->xceiv_old_state = musb->xceiv->otg->state;
sysfs_notify(&musb->controller->kobj, NULL, "mode");
}
+
+ pm_runtime_mark_last_busy(musb->controller);
+ pm_runtime_put_autosuspend(musb->controller);
}
static void musb_recover_from_babble(struct musb *musb)
mask = MUSB_DEVCTL_BDEVICE | MUSB_DEVCTL_FSDEV | MUSB_DEVCTL_LSDEV;
if ((devctl & mask) != (musb->context.devctl & mask))
musb->port1_status = 0;
- if (musb->need_finish_resume) {
- musb->need_finish_resume = 0;
- schedule_delayed_work(&musb->finish_resume_work,
- msecs_to_jiffies(USB_RESUME_TIMEOUT));
- }
/*
* The USB HUB code expects the device to be in RPM_ACTIVE once it came
musb_restore_context(musb);
- if (musb->need_finish_resume) {
- musb->need_finish_resume = 0;
- schedule_delayed_work(&musb->finish_resume_work,
- msecs_to_jiffies(USB_RESUME_TIMEOUT));
- }
-
spin_lock_irqsave(&musb->lock, flags);
error = musb_run_resume_work(musb);
if (error)
/* is_suspended means USB B_PERIPHERAL suspend */
unsigned is_suspended:1;
- unsigned need_finish_resume :1;
/* may_wakeup means remote wakeup is enabled */
unsigned may_wakeup:1;
{ USB_DEVICE_AND_INTERFACE_INFO(WETELECOM_VENDOR_ID, WETELECOM_PRODUCT_WMD200, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(WETELECOM_VENDOR_ID, WETELECOM_PRODUCT_6802, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(WETELECOM_VENDOR_ID, WETELECOM_PRODUCT_WMD300, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(0x03f0, 0x421d, 0xff, 0xff, 0xff) }, /* HP lt2523 (Novatel E371) */
{ } /* Terminating entry */
};
MODULE_DEVICE_TABLE(usb, option_ids);
{ USB_DEVICE(IODATA_VENDOR_ID, IODATA_PRODUCT_ID) },
{ USB_DEVICE(IODATA_VENDOR_ID, IODATA_PRODUCT_ID_RSAQ5) },
{ USB_DEVICE(ATEN_VENDOR_ID, ATEN_PRODUCT_ID) },
+ { USB_DEVICE(ATEN_VENDOR_ID, ATEN_PRODUCT_ID2) },
{ USB_DEVICE(ATEN_VENDOR_ID2, ATEN_PRODUCT_ID) },
{ USB_DEVICE(ELCOM_VENDOR_ID, ELCOM_PRODUCT_ID) },
{ USB_DEVICE(ELCOM_VENDOR_ID, ELCOM_PRODUCT_ID_UCSGT) },
#define ATEN_VENDOR_ID 0x0557
#define ATEN_VENDOR_ID2 0x0547
#define ATEN_PRODUCT_ID 0x2008
+#define ATEN_PRODUCT_ID2 0x2118
#define IODATA_VENDOR_ID 0x04bb
#define IODATA_PRODUCT_ID 0x0a03
{USB_DEVICE(0x1410, 0xa021)}, /* Novatel Gobi 3000 Composite */
{USB_DEVICE(0x413c, 0x8193)}, /* Dell Gobi 3000 QDL */
{USB_DEVICE(0x413c, 0x8194)}, /* Dell Gobi 3000 Composite */
+ {USB_DEVICE(0x413c, 0x81a6)}, /* Dell DW5570 QDL (MC8805) */
{USB_DEVICE(0x1199, 0x68a4)}, /* Sierra Wireless QDL */
{USB_DEVICE(0x1199, 0x68a5)}, /* Sierra Wireless Modem */
{USB_DEVICE(0x1199, 0x68a8)}, /* Sierra Wireless QDL */
mutex_lock(&container->lock);
ret = tce_iommu_create_default_window(container);
- if (ret)
- return ret;
-
- ret = tce_iommu_create_window(container, create.page_shift,
- create.window_size, create.levels,
- &create.start_addr);
+ if (!ret)
+ ret = tce_iommu_create_window(container,
+ create.page_shift,
+ create.window_size, create.levels,
+ &create.start_addr);
mutex_unlock(&container->lock);
static long tce_iommu_take_ownership_ddw(struct tce_container *container,
struct iommu_table_group *table_group)
{
+ long i, ret = 0;
+
if (!table_group->ops->create_table || !table_group->ops->set_window ||
!table_group->ops->release_ownership) {
WARN_ON_ONCE(1);
table_group->ops->take_ownership(table_group);
+ /* Set all windows to the new group */
+ for (i = 0; i < IOMMU_TABLE_GROUP_MAX_TABLES; ++i) {
+ struct iommu_table *tbl = container->tables[i];
+
+ if (!tbl)
+ continue;
+
+ ret = table_group->ops->set_window(table_group, i, tbl);
+ if (ret)
+ goto release_exit;
+ }
+
return 0;
+
+release_exit:
+ for (i = 0; i < IOMMU_TABLE_GROUP_MAX_TABLES; ++i)
+ table_group->ops->unset_window(table_group, i);
+
+ table_group->ops->release_ownership(table_group);
+
+ return ret;
}
static int tce_iommu_attach_group(void *iommu_data,
/* pr_debug("tce_vfio: Attaching group #%u to iommu %p\n",
iommu_group_id(iommu_group), iommu_group); */
table_group = iommu_group_get_iommudata(iommu_group);
+ if (!table_group) {
+ ret = -ENODEV;
+ goto unlock_exit;
+ }
if (tce_groups_attached(container) && (!table_group->ops ||
!table_group->ops->take_ownership ||
static void vhost_init_is_le(struct vhost_virtqueue *vq)
{
- if (vhost_has_feature(vq, VIRTIO_F_VERSION_1))
- vq->is_le = true;
+ vq->is_le = vhost_has_feature(vq, VIRTIO_F_VERSION_1)
+ || virtio_legacy_is_little_endian();
}
#endif /* CONFIG_VHOST_CROSS_ENDIAN_LEGACY */
static void vhost_reset_is_le(struct vhost_virtqueue *vq)
{
- vq->is_le = virtio_legacy_is_little_endian();
+ vhost_init_is_le(vq);
}
struct vhost_flush_struct {
int r;
bool is_le = vq->is_le;
- if (!vq->private_data) {
- vhost_reset_is_le(vq);
+ if (!vq->private_data)
return 0;
- }
vhost_init_is_le(vq);
static int vhost_vsock_start(struct vhost_vsock *vsock)
{
+ struct vhost_virtqueue *vq;
size_t i;
int ret;
goto err;
for (i = 0; i < ARRAY_SIZE(vsock->vqs); i++) {
- struct vhost_virtqueue *vq = &vsock->vqs[i];
+ vq = &vsock->vqs[i];
mutex_lock(&vq->mutex);
if (!vhost_vq_access_ok(vq)) {
ret = -EFAULT;
- mutex_unlock(&vq->mutex);
goto err_vq;
}
if (!vq->private_data) {
vq->private_data = vsock;
- vhost_vq_init_access(vq);
+ ret = vhost_vq_init_access(vq);
+ if (ret)
+ goto err_vq;
}
mutex_unlock(&vq->mutex);
return 0;
err_vq:
+ vq->private_data = NULL;
+ mutex_unlock(&vq->mutex);
+
for (i = 0; i < ARRAY_SIZE(vsock->vqs); i++) {
- struct vhost_virtqueue *vq = &vsock->vqs[i];
+ vq = &vsock->vqs[i];
mutex_lock(&vq->mutex);
vq->private_data = NULL;
int fb_copy_cmap(const struct fb_cmap *from, struct fb_cmap *to)
{
- int tooff = 0, fromoff = 0;
- int size;
+ unsigned int tooff = 0, fromoff = 0;
+ size_t size;
if (to->start > from->start)
fromoff = to->start - from->start;
else
tooff = from->start - to->start;
- size = to->len - tooff;
- if (size > (int) (from->len - fromoff))
- size = from->len - fromoff;
- if (size <= 0)
+ if (fromoff >= from->len || tooff >= to->len)
+ return -EINVAL;
+
+ size = min_t(size_t, to->len - tooff, from->len - fromoff);
+ if (size == 0)
return -EINVAL;
size *= sizeof(u16);
int fb_cmap_to_user(const struct fb_cmap *from, struct fb_cmap_user *to)
{
- int tooff = 0, fromoff = 0;
- int size;
+ unsigned int tooff = 0, fromoff = 0;
+ size_t size;
if (to->start > from->start)
fromoff = to->start - from->start;
else
tooff = from->start - to->start;
- size = to->len - tooff;
- if (size > (int) (from->len - fromoff))
- size = from->len - fromoff;
- if (size <= 0)
+ if (fromoff >= from->len || tooff >= to->len)
+ return -EINVAL;
+
+ size = min_t(size_t, to->len - tooff, from->len - fromoff);
+ if (size == 0)
return -EINVAL;
size *= sizeof(u16);
#define pr_fmt(fmt) "virtio-mmio: " fmt
#include <linux/acpi.h>
+#include <linux/dma-mapping.h>
#include <linux/highmem.h>
#include <linux/interrupt.h>
#include <linux/io.h>
struct virtio_mmio_device *vm_dev;
struct resource *mem;
unsigned long magic;
+ int rc;
mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!mem)
}
vm_dev->vdev.id.vendor = readl(vm_dev->base + VIRTIO_MMIO_VENDOR_ID);
- if (vm_dev->version == 1)
+ if (vm_dev->version == 1) {
writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_GUEST_PAGE_SIZE);
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
+ /*
+ * In the legacy case, ensure our coherently-allocated virtio
+ * ring will be at an address expressable as a 32-bit PFN.
+ */
+ if (!rc)
+ dma_set_coherent_mask(&pdev->dev,
+ DMA_BIT_MASK(32 + PAGE_SHIFT));
+ } else {
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ }
+ if (rc)
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (rc)
+ dev_warn(&pdev->dev, "Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work.\n");
+
platform_set_drvdata(pdev, vm_dev);
return register_virtio_device(&vm_dev->vdev);
if (map == SWIOTLB_MAP_ERROR)
return DMA_ERROR_CODE;
+ dev_addr = xen_phys_to_bus(map);
xen_dma_map_page(dev, pfn_to_page(map >> PAGE_SHIFT),
dev_addr, map & ~PAGE_MASK, size, dir, attrs);
- dev_addr = xen_phys_to_bus(map);
/*
* Ensure that the address returned is DMA'ble
sg_dma_len(sgl) = 0;
return 0;
}
+ dev_addr = xen_phys_to_bus(map);
xen_dma_map_page(hwdev, pfn_to_page(map >> PAGE_SHIFT),
dev_addr,
map & ~PAGE_MASK,
sg->length,
dir,
attrs);
- sg->dma_address = xen_phys_to_bus(map);
+ sg->dma_address = dev_addr;
} else {
/* we are not interested in the dma_addr returned by
* xen_dma_map_page, only in the potential cache flushes executed
bool "Direct Access (DAX) support"
depends on MMU
depends on !(ARM || MIPS || SPARC)
+ select FS_IOMAP
help
Direct Access (DAX) can be used on memory-backed block devices.
If the block device supports DAX and the filesystem supports DAX,
struct blk_plug plug;
struct blkdev_dio *dio;
struct bio *bio;
- bool is_read = (iov_iter_rw(iter) == READ);
+ bool is_read = (iov_iter_rw(iter) == READ), is_sync;
loff_t pos = iocb->ki_pos;
blk_qc_t qc = BLK_QC_T_NONE;
int ret;
bio_get(bio); /* extra ref for the completion handler */
dio = container_of(bio, struct blkdev_dio, bio);
- dio->is_sync = is_sync_kiocb(iocb);
+ dio->is_sync = is_sync = is_sync_kiocb(iocb);
if (dio->is_sync)
dio->waiter = current;
else
}
blk_finish_plug(&plug);
- if (!dio->is_sync)
+ if (!is_sync)
return -EIOCBQUEUED;
for (;;) {
unsigned long buf_offset;
unsigned long current_buf_start;
unsigned long start_byte;
+ unsigned long prev_start_byte;
unsigned long working_bytes = total_out - buf_start;
unsigned long bytes;
char *kaddr;
if (!bio->bi_iter.bi_size)
return 0;
bvec = bio_iter_iovec(bio, bio->bi_iter);
-
+ prev_start_byte = start_byte;
start_byte = page_offset(bvec.bv_page) - disk_start;
/*
- * make sure our new page is covered by this
- * working buffer
+ * We need to make sure we're only adjusting
+ * our offset into compression working buffer when
+ * we're switching pages. Otherwise we can incorrectly
+ * keep copying when we were actually done.
*/
- if (total_out <= start_byte)
- return 1;
+ if (start_byte != prev_start_byte) {
+ /*
+ * make sure our new page is covered by this
+ * working buffer
+ */
+ if (total_out <= start_byte)
+ return 1;
- /*
- * the next page in the biovec might not be adjacent
- * to the last page, but it might still be found
- * inside this working buffer. bump our offset pointer
- */
- if (total_out > start_byte &&
- current_buf_start < start_byte) {
- buf_offset = start_byte - buf_start;
- working_bytes = total_out - start_byte;
- current_buf_start = buf_start + buf_offset;
+ /*
+ * the next page in the biovec might not be adjacent
+ * to the last page, but it might still be found
+ * inside this working buffer. bump our offset pointer
+ */
+ if (total_out > start_byte &&
+ current_buf_start < start_byte) {
+ buf_offset = start_byte - buf_start;
+ working_bytes = total_out - start_byte;
+ current_buf_start = buf_start + buf_offset;
+ }
}
}
break;
case S_IFDIR:
inode->i_fop = &btrfs_dir_file_operations;
- if (root == fs_info->tree_root)
- inode->i_op = &btrfs_dir_ro_inode_operations;
- else
- inode->i_op = &btrfs_dir_inode_operations;
+ inode->i_op = &btrfs_dir_inode_operations;
break;
case S_IFLNK:
inode->i_op = &btrfs_symlink_inode_operations;
if (found_type > min_type) {
del_item = 1;
} else {
- if (item_end < new_size)
+ if (item_end < new_size) {
+ /*
+ * With NO_HOLES mode, for the following mapping
+ *
+ * [0-4k][hole][8k-12k]
+ *
+ * if truncating isize down to 6k, it ends up
+ * isize being 8k.
+ */
+ if (btrfs_fs_incompat(root->fs_info, NO_HOLES))
+ last_size = new_size;
break;
+ }
if (found_key.offset >= new_size)
del_item = 1;
else
inode->i_ino = BTRFS_EMPTY_SUBVOL_DIR_OBJECTID;
inode->i_op = &btrfs_dir_ro_inode_operations;
+ inode->i_opflags &= ~IOP_XATTR;
inode->i_fop = &simple_dir_operations;
inode->i_mode = S_IFDIR | S_IRUGO | S_IWUSR | S_IXUGO;
inode->i_mtime = current_time(inode);
struct extent_map *em = NULL;
int ret;
- down_read(&BTRFS_I(inode)->dio_sem);
if (type != BTRFS_ORDERED_NOCOW) {
em = create_pinned_em(inode, start, len, orig_start,
block_start, block_len, orig_block_len,
em = ERR_PTR(ret);
}
out:
- up_read(&BTRFS_I(inode)->dio_sem);
return em;
}
dio_data.unsubmitted_oe_range_start = (u64)offset;
dio_data.unsubmitted_oe_range_end = (u64)offset;
current->journal_info = &dio_data;
+ down_read(&BTRFS_I(inode)->dio_sem);
} else if (test_bit(BTRFS_INODE_READDIO_NEED_LOCK,
&BTRFS_I(inode)->runtime_flags)) {
inode_dio_end(inode);
iter, btrfs_get_blocks_direct, NULL,
btrfs_submit_direct, flags);
if (iov_iter_rw(iter) == WRITE) {
+ up_read(&BTRFS_I(inode)->dio_sem);
current->journal_info = NULL;
if (ret < 0 && ret != -EIOCBQUEUED) {
if (dio_data.reserve)
break;
}
+ btrfs_block_rsv_release(fs_info, rsv, -1);
ret = btrfs_block_rsv_migrate(&fs_info->trans_block_rsv,
rsv, min_size, 0);
BUG_ON(ret); /* shouldn't happen */
static const struct inode_operations btrfs_dir_ro_inode_operations = {
.lookup = btrfs_lookup,
.permission = btrfs_permission,
- .get_acl = btrfs_get_acl,
- .set_acl = btrfs_set_acl,
.update_time = btrfs_update_time,
};
#ifdef CONFIG_COMPAT
long btrfs_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
+ /*
+ * These all access 32-bit values anyway so no further
+ * handling is necessary.
+ */
switch (cmd) {
case FS_IOC32_GETFLAGS:
cmd = FS_IOC_GETFLAGS;
case FS_IOC32_GETVERSION:
cmd = FS_IOC_GETVERSION;
break;
- default:
- return -ENOIOCTLCMD;
}
return btrfs_ioctl(file, cmd, (unsigned long) compat_ptr(arg));
rc = -ENOMEM;
goto error_exit;
}
+ spin_lock_init(&cifsFile->file_info_lock);
file->private_data = cifsFile;
cifsFile->tlink = cifs_get_tlink(tlink);
tcon = tlink_tcon(tlink);
}
EXPORT_SYMBOL_GPL(__dax_zero_page_range);
-#ifdef CONFIG_FS_IOMAP
static sector_t dax_iomap_sector(struct iomap *iomap, loff_t pos)
{
return iomap->blkno + (((pos & PAGE_MASK) - iomap->offset) >> 9);
struct blk_dax_ctl dax = { 0 };
ssize_t map_len;
+ if (fatal_signal_pending(current)) {
+ ret = -EINTR;
+ break;
+ }
+
dax.sector = dax_iomap_sector(iomap, pos);
dax.size = (length + offset + PAGE_SIZE - 1) & PAGE_MASK;
map_len = dax_map_atomic(iomap->bdev, &dax);
}
EXPORT_SYMBOL_GPL(dax_iomap_pmd_fault);
#endif /* CONFIG_FS_DAX_PMD */
-#endif /* CONFIG_FS_IOMAP */
config EXT2_FS
tristate "Second extended fs support"
- select FS_IOMAP if FS_DAX
help
Ext2 is a standard Linux file system for hard disks.
select CRC16
select CRYPTO
select CRYPTO_CRC32C
- select FS_IOMAP if FS_DAX
help
This is the next generation of the ext3 filesystem.
hlist_for_each_entry(object, &cookie->backing_objects, cookie_link) {
if (invalidate)
set_bit(FSCACHE_OBJECT_RETIRED, &object->flags);
+ clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
fscache_raise_event(object, FSCACHE_OBJECT_EV_KILL);
}
} else {
wait_on_atomic_t(&cookie->n_active, fscache_wait_atomic_t,
TASK_UNINTERRUPTIBLE);
+ /* Make sure any pending writes are cancelled. */
+ if (cookie->def->type != FSCACHE_COOKIE_TYPE_INDEX)
+ fscache_invalidate_writes(cookie);
+
/* Reset the cookie state if it wasn't relinquished */
if (!test_bit(FSCACHE_COOKIE_RELINQUISHED, &cookie->flags)) {
atomic_inc(&cookie->n_active);
cookie->flags = 1 << FSCACHE_COOKIE_ENABLED;
spin_lock_init(&cookie->lock);
+ spin_lock_init(&cookie->stores_lock);
INIT_HLIST_HEAD(&cookie->backing_objects);
/* check the netfs type is not already present */
static const struct fscache_state *fscache_object_available(struct fscache_object *, int);
static const struct fscache_state *fscache_parent_ready(struct fscache_object *, int);
static const struct fscache_state *fscache_update_object(struct fscache_object *, int);
+static const struct fscache_state *fscache_object_dead(struct fscache_object *, int);
#define __STATE_NAME(n) fscache_osm_##n
#define STATE(n) (&__STATE_NAME(n))
static WORK_STATE(KILL_OBJECT, "KILL", fscache_kill_object);
static WORK_STATE(KILL_DEPENDENTS, "KDEP", fscache_kill_dependents);
static WORK_STATE(DROP_OBJECT, "DROP", fscache_drop_object);
-static WORK_STATE(OBJECT_DEAD, "DEAD", (void*)2UL);
+static WORK_STATE(OBJECT_DEAD, "DEAD", fscache_object_dead);
static WAIT_STATE(WAIT_FOR_INIT, "?INI",
TRANSIT_TO(INIT_OBJECT, 1 << FSCACHE_OBJECT_EV_NEW_CHILD));
event = -1;
if (new_state == NO_TRANSIT) {
_debug("{OBJ%x} %s notrans", object->debug_id, state->name);
+ if (unlikely(state == STATE(OBJECT_DEAD))) {
+ _leave(" [dead]");
+ return;
+ }
fscache_enqueue_object(object);
event_mask = object->oob_event_mask;
goto unmask_events;
object->state = state = new_state;
if (state->work) {
- if (unlikely(state->work == ((void *)2UL))) {
+ if (unlikely(state == STATE(OBJECT_DEAD))) {
_leave(" [dead]");
return;
}
fscache_mark_object_dead(object);
object->oob_event_mask = 0;
+ if (test_bit(FSCACHE_OBJECT_RETIRED, &object->flags)) {
+ /* Reject any new read/write ops and abort any that are pending. */
+ clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
+ fscache_cancel_all_ops(object);
+ }
+
if (list_empty(&object->dependents) &&
object->n_ops == 0 &&
object->n_children == 0)
}
}
EXPORT_SYMBOL(fscache_object_mark_killed);
+
+/*
+ * The object is dead. We can get here if an object gets queued by an event
+ * that would lead to its death (such as EV_KILL) when the dispatcher is
+ * already running (and so can be requeued) but hasn't yet cleared the event
+ * mask.
+ */
+static const struct fscache_state *fscache_object_dead(struct fscache_object *object,
+ int event)
+{
+ if (!test_and_set_bit(FSCACHE_OBJECT_RUN_AFTER_DEAD,
+ &object->flags))
+ return NO_TRANSIT;
+
+ WARN(true, "FS-Cache object redispatched after death");
+ return NO_TRANSIT;
+}
BUG_ON(pos + len > iomap->offset + iomap->length);
+ if (fatal_signal_pending(current))
+ return -EINTR;
+
page = grab_cache_page_write_begin(inode->i_mapping, index, flags);
if (!page)
return -ENOMEM;
sattr->ia_valid |= ATTR_MTIME;
/* Except MODE, it seems harmless of setting twice. */
- if ((attrset[1] & FATTR4_WORD1_MODE))
+ if (opendata->o_arg.createmode != NFS4_CREATE_EXCLUSIVE &&
+ attrset[1] & FATTR4_WORD1_MODE)
sattr->ia_valid &= ~ATTR_MODE;
if (attrset[2] & FATTR4_WORD2_SECURITY_LABEL)
goto out;
}
+ nfs4_sequence_free_slot(&lgp->res.seq_res);
err = nfs4_handle_exception(server, nfs4err, exception);
if (!status) {
if (exception->retry)
case -NFS4ERR_BADXDR:
case -NFS4ERR_RESOURCE:
case -NFS4ERR_NOFILEHANDLE:
+ case -NFS4ERR_MOVED:
/* Non-seqid mutating errors */
return;
};
send = pnfs_prepare_layoutreturn(lo, &stateid, NULL);
spin_unlock(&ino->i_lock);
- pnfs_free_lseg_list(&tmp_list);
if (send)
status = pnfs_send_layoutreturn(lo, &stateid, IOMODE_ANY, true);
out_put_layout_hdr:
+ pnfs_free_lseg_list(&tmp_list);
pnfs_put_layout_hdr(lo);
out:
dprintk("<-- %s status: %d\n", __func__, status);
struct nfs4_layout_stateid *ls;
struct nfs4_stid *stp;
- stp = nfs4_alloc_stid(cstate->clp, nfs4_layout_stateid_cache);
+ stp = nfs4_alloc_stid(cstate->clp, nfs4_layout_stateid_cache,
+ nfsd4_free_layout_stateid);
if (!stp)
return NULL;
- stp->sc_free = nfsd4_free_layout_stateid;
+
get_nfs4_file(fp);
stp->sc_file = fp;
return co;
}
-struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl,
- struct kmem_cache *slab)
+struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl, struct kmem_cache *slab,
+ void (*sc_free)(struct nfs4_stid *))
{
struct nfs4_stid *stid;
int new_id;
idr_preload_end();
if (new_id < 0)
goto out_free;
+
+ stid->sc_free = sc_free;
stid->sc_client = cl;
stid->sc_stateid.si_opaque.so_id = new_id;
stid->sc_stateid.si_opaque.so_clid = cl->cl_clientid;
static struct nfs4_ol_stateid * nfs4_alloc_open_stateid(struct nfs4_client *clp)
{
struct nfs4_stid *stid;
- struct nfs4_ol_stateid *stp;
- stid = nfs4_alloc_stid(clp, stateid_slab);
+ stid = nfs4_alloc_stid(clp, stateid_slab, nfs4_free_ol_stateid);
if (!stid)
return NULL;
- stp = openlockstateid(stid);
- stp->st_stid.sc_free = nfs4_free_ol_stateid;
- return stp;
+ return openlockstateid(stid);
}
static void nfs4_free_deleg(struct nfs4_stid *stid)
goto out_dec;
if (delegation_blocked(¤t_fh->fh_handle))
goto out_dec;
- dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab));
+ dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab, nfs4_free_deleg));
if (dp == NULL)
goto out_dec;
- dp->dl_stid.sc_free = nfs4_free_deleg;
/*
* delegation seqid's are never incremented. The 4.1 special
* meaning of seqid 0 isn't meaningful, really, but let's avoid
stp->st_stateowner = nfs4_get_stateowner(&lo->lo_owner);
get_nfs4_file(fp);
stp->st_stid.sc_file = fp;
- stp->st_stid.sc_free = nfs4_free_lock_stateid;
stp->st_access_bmap = 0;
stp->st_deny_bmap = open_stp->st_deny_bmap;
stp->st_openstp = open_stp;
lst = find_lock_stateid(lo, fi);
if (lst == NULL) {
spin_unlock(&clp->cl_lock);
- ns = nfs4_alloc_stid(clp, stateid_slab);
+ ns = nfs4_alloc_stid(clp, stateid_slab, nfs4_free_lock_stateid);
if (ns == NULL)
return NULL;
__be32 nfsd4_lookup_stateid(struct nfsd4_compound_state *cstate,
stateid_t *stateid, unsigned char typemask,
struct nfs4_stid **s, struct nfsd_net *nn);
-struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl,
- struct kmem_cache *slab);
+struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl, struct kmem_cache *slab,
+ void (*sc_free)(struct nfs4_stid *));
void nfs4_unhash_stid(struct nfs4_stid *s);
void nfs4_put_stid(struct nfs4_stid *s);
void nfs4_inc_and_copy_stateid(stateid_t *dst, struct nfs4_stid *stid);
iter.tgid += 1, iter = next_tgid(ns, iter)) {
char name[PROC_NUMBUF];
int len;
+
+ cond_resched();
if (!has_pid_permissions(ns, iter.task, 2))
continue;
u |= kpf_copy_bit(k, KPF_ACTIVE, PG_active);
u |= kpf_copy_bit(k, KPF_RECLAIM, PG_reclaim);
- u |= kpf_copy_bit(k, KPF_SWAPCACHE, PG_swapcache);
+ if (PageSwapCache(page))
+ u |= 1 << KPF_SWAPCACHE;
u |= kpf_copy_bit(k, KPF_SWAPBACKED, PG_swapbacked);
u |= kpf_copy_bit(k, KPF_UNEVICTABLE, PG_unevictable);
1, id, type, PSTORE_TYPE_PMSG, 0);
/* ftrace is last since it may want to dynamically allocate memory. */
- if (!prz_ok(prz)) {
+ if (!prz_ok(prz) && cxt->fprzs) {
if (!(cxt->flags & RAMOOPS_FLAG_FTRACE_PER_CPU)) {
prz = ramoops_get_next_prz(cxt->fprzs,
&cxt->ftrace_read_cnt, 1, id, type,
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/uaccess.h>
+#include <linux/major.h>
#include "internal.h"
static struct kmem_cache *romfs_inode_cachep;
static int romfs_statfs(struct dentry *dentry, struct kstatfs *buf)
{
struct super_block *sb = dentry->d_sb;
- u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
+ u64 id = 0;
+
+ /* When calling huge_encode_dev(),
+ * use sb->s_bdev->bd_dev when,
+ * - CONFIG_ROMFS_ON_BLOCK defined
+ * use sb->s_dev when,
+ * - CONFIG_ROMFS_ON_BLOCK undefined and
+ * - CONFIG_ROMFS_ON_MTD defined
+ * leave id as 0 when,
+ * - CONFIG_ROMFS_ON_BLOCK undefined and
+ * - CONFIG_ROMFS_ON_MTD undefined
+ */
+ if (sb->s_bdev)
+ id = huge_encode_dev(sb->s_bdev->bd_dev);
+ else if (sb->s_dev)
+ id = huge_encode_dev(sb->s_dev);
buf->f_type = ROMFS_MAGIC;
buf->f_namelen = ROMFS_MAXFN;
sb->s_flags |= MS_RDONLY | MS_NOATIME;
sb->s_op = &romfs_super_ops;
+#ifdef CONFIG_ROMFS_ON_MTD
+ /* Use same dev ID from the underlying mtdblock device */
+ if (sb->s_mtd)
+ sb->s_dev = MKDEV(MTD_BLOCK_MAJOR, sb->s_mtd->index);
+#endif
/* read the image superblock and check it */
rsb = kmalloc(512, GFP_KERNEL);
if (!rsb)
struct uffd_msg msg;
wait_queue_t wq;
struct userfaultfd_ctx *ctx;
+ bool waken;
};
struct userfaultfd_wake_range {
if (len && (start > uwq->msg.arg.pagefault.address ||
start + len <= uwq->msg.arg.pagefault.address))
goto out;
+ WRITE_ONCE(uwq->waken, true);
+ /*
+ * The implicit smp_mb__before_spinlock in try_to_wake_up()
+ * renders uwq->waken visible to other CPUs before the task is
+ * waken.
+ */
ret = wake_up_state(wq->private, mode);
if (ret)
/*
struct userfaultfd_wait_queue uwq;
int ret;
bool must_wait, return_to_userland;
+ long blocking_state;
BUG_ON(!rwsem_is_locked(&mm->mmap_sem));
uwq.wq.private = current;
uwq.msg = userfault_msg(vmf->address, vmf->flags, reason);
uwq.ctx = ctx;
+ uwq.waken = false;
return_to_userland =
(vmf->flags & (FAULT_FLAG_USER|FAULT_FLAG_KILLABLE)) ==
(FAULT_FLAG_USER|FAULT_FLAG_KILLABLE);
+ blocking_state = return_to_userland ? TASK_INTERRUPTIBLE :
+ TASK_KILLABLE;
spin_lock(&ctx->fault_pending_wqh.lock);
/*
* following the spin_unlock to happen before the list_add in
* __add_wait_queue.
*/
- set_current_state(return_to_userland ? TASK_INTERRUPTIBLE :
- TASK_KILLABLE);
+ set_current_state(blocking_state);
spin_unlock(&ctx->fault_pending_wqh.lock);
must_wait = userfaultfd_must_wait(ctx, vmf->address, vmf->flags,
wake_up_poll(&ctx->fd_wqh, POLLIN);
schedule();
ret |= VM_FAULT_MAJOR;
+
+ /*
+ * False wakeups can orginate even from rwsem before
+ * up_read() however userfaults will wait either for a
+ * targeted wakeup on the specific uwq waitqueue from
+ * wake_userfault() or for signals or for uffd
+ * release.
+ */
+ while (!READ_ONCE(uwq.waken)) {
+ /*
+ * This needs the full smp_store_mb()
+ * guarantee as the state write must be
+ * visible to other CPUs before reading
+ * uwq.waken from other CPUs.
+ */
+ set_current_state(blocking_state);
+ if (READ_ONCE(uwq.waken) ||
+ READ_ONCE(ctx->released) ||
+ (return_to_userland ? signal_pending(current) :
+ fatal_signal_pending(current)))
+ break;
+ schedule();
+ }
}
__set_current_state(TASK_RUNNING);
#include "xfs_rmap_btree.h"
#include "xfs_btree.h"
#include "xfs_refcount_btree.h"
+#include "xfs_ialloc_btree.h"
/*
* Per-AG Block Reservations
struct xfs_mount *mp = pag->pag_mount;
struct xfs_ag_resv *resv;
int error;
+ xfs_extlen_t reserved;
- resv = xfs_perag_resv(pag, type);
if (used > ask)
ask = used;
- resv->ar_asked = ask;
- resv->ar_reserved = resv->ar_orig_reserved = ask - used;
- mp->m_ag_max_usable -= ask;
+ reserved = ask - used;
- trace_xfs_ag_resv_init(pag, type, ask);
-
- error = xfs_mod_fdblocks(mp, -(int64_t)resv->ar_reserved, true);
- if (error)
+ error = xfs_mod_fdblocks(mp, -(int64_t)reserved, true);
+ if (error) {
trace_xfs_ag_resv_init_error(pag->pag_mount, pag->pag_agno,
error, _RET_IP_);
+ xfs_warn(mp,
+"Per-AG reservation for AG %u failed. Filesystem may run out of space.",
+ pag->pag_agno);
+ return error;
+ }
- return error;
+ mp->m_ag_max_usable -= ask;
+
+ resv = xfs_perag_resv(pag, type);
+ resv->ar_asked = ask;
+ resv->ar_reserved = resv->ar_orig_reserved = reserved;
+
+ trace_xfs_ag_resv_init(pag, type, ask);
+ return 0;
}
/* Create a per-AG block reservation. */
xfs_ag_resv_init(
struct xfs_perag *pag)
{
+ struct xfs_mount *mp = pag->pag_mount;
+ xfs_agnumber_t agno = pag->pag_agno;
xfs_extlen_t ask;
xfs_extlen_t used;
int error = 0;
if (pag->pag_meta_resv.ar_asked == 0) {
ask = used = 0;
- error = xfs_refcountbt_calc_reserves(pag->pag_mount,
- pag->pag_agno, &ask, &used);
+ error = xfs_refcountbt_calc_reserves(mp, agno, &ask, &used);
if (error)
goto out;
- error = __xfs_ag_resv_init(pag, XFS_AG_RESV_METADATA,
- ask, used);
+ error = xfs_finobt_calc_reserves(mp, agno, &ask, &used);
if (error)
goto out;
+
+ error = __xfs_ag_resv_init(pag, XFS_AG_RESV_METADATA,
+ ask, used);
+ if (error) {
+ /*
+ * Because we didn't have per-AG reservations when the
+ * finobt feature was added we might not be able to
+ * reserve all needed blocks. Warn and fall back to the
+ * old and potentially buggy code in that case, but
+ * ensure we do have the reservation for the refcountbt.
+ */
+ ask = used = 0;
+
+ mp->m_inotbt_nores = true;
+
+ error = xfs_refcountbt_calc_reserves(mp, agno, &ask,
+ &used);
+ if (error)
+ goto out;
+
+ error = __xfs_ag_resv_init(pag, XFS_AG_RESV_METADATA,
+ ask, used);
+ if (error)
+ goto out;
+ }
}
/* Create the AGFL metadata reservation */
if (pag->pag_agfl_resv.ar_asked == 0) {
ask = used = 0;
- error = xfs_rmapbt_calc_reserves(pag->pag_mount, pag->pag_agno,
- &ask, &used);
+ error = xfs_rmapbt_calc_reserves(mp, agno, &ask, &used);
if (error)
goto out;
goto out;
}
+#ifdef DEBUG
+ /* need to read in the AGF for the ASSERT below to work */
+ error = xfs_alloc_pagf_init(pag->pag_mount, NULL, pag->pag_agno, 0);
+ if (error)
+ return error;
+
ASSERT(xfs_perag_resv(pag, XFS_AG_RESV_METADATA)->ar_reserved +
xfs_perag_resv(pag, XFS_AG_RESV_AGFL)->ar_reserved <=
pag->pagf_freeblks + pag->pagf_flcount);
+#endif
out:
return error;
}
if (XFS_FORCED_SHUTDOWN(ip->i_mount))
return -EIO;
- if (!xfs_inode_hasattr(ip))
- return -ENOATTR;
-
error = xfs_attr_args_init(&args, ip, name, flags);
if (error)
return error;
if (XFS_FORCED_SHUTDOWN(dp->i_mount))
return -EIO;
- if (!xfs_inode_hasattr(dp))
- return -ENOATTR;
-
error = xfs_attr_args_init(&args, dp, name, flags);
if (error)
return error;
align = xfs_get_cowextsz_hint(ap->ip);
else if (xfs_alloc_is_userdata(ap->datatype))
align = xfs_get_extsz_hint(ap->ip);
- if (unlikely(align)) {
+ if (align) {
error = xfs_bmap_extsize_align(mp, &ap->got, &ap->prev,
align, 0, ap->eof, 0, ap->conv,
&ap->offset, &ap->length);
args.minlen = ap->minlen;
}
/* apply extent size hints if obtained earlier */
- if (unlikely(align)) {
+ if (align) {
args.prod = align;
if ((args.mod = (xfs_extlen_t)do_mod(ap->offset, args.prod)))
args.mod = (xfs_extlen_t)(args.prod - args.mod);
int n; /* current extent index */
xfs_fileoff_t obno; /* old block number (offset) */
int whichfork; /* data or attr fork */
- char inhole; /* current location is hole in file */
- char wasdelay; /* old extent was delayed */
#ifdef DEBUG
xfs_fileoff_t orig_bno; /* original block number value */
bma.firstblock = firstblock;
while (bno < end && n < *nmap) {
- inhole = eof || bma.got.br_startoff > bno;
- wasdelay = !inhole && isnullstartblock(bma.got.br_startblock);
+ bool need_alloc = false, wasdelay = false;
- /*
- * Make sure we only reflink into a hole.
- */
- if (flags & XFS_BMAPI_REMAP)
- ASSERT(inhole);
- if (flags & XFS_BMAPI_COWFORK)
- ASSERT(!inhole);
+ /* in hole or beyoned EOF? */
+ if (eof || bma.got.br_startoff > bno) {
+ if (flags & XFS_BMAPI_DELALLOC) {
+ /*
+ * For the COW fork we can reasonably get a
+ * request for converting an extent that races
+ * with other threads already having converted
+ * part of it, as there converting COW to
+ * regular blocks is not protected using the
+ * IOLOCK.
+ */
+ ASSERT(flags & XFS_BMAPI_COWFORK);
+ if (!(flags & XFS_BMAPI_COWFORK)) {
+ error = -EIO;
+ goto error0;
+ }
+
+ if (eof || bno >= end)
+ break;
+ } else {
+ need_alloc = true;
+ }
+ } else {
+ /*
+ * Make sure we only reflink into a hole.
+ */
+ ASSERT(!(flags & XFS_BMAPI_REMAP));
+ if (isnullstartblock(bma.got.br_startblock))
+ wasdelay = true;
+ }
/*
* First, deal with the hole before the allocated space
* that we found, if any.
*/
- if (inhole || wasdelay) {
+ if (need_alloc || wasdelay) {
bma.eof = eof;
bma.conv = !!(flags & XFS_BMAPI_CONVERT);
bma.wasdel = wasdelay;
/* Map something in the CoW fork. */
#define XFS_BMAPI_COWFORK 0x200
+/* Only convert delalloc space, don't allocate entirely new extents */
+#define XFS_BMAPI_DELALLOC 0x400
+
#define XFS_BMAPI_FLAGS \
{ XFS_BMAPI_ENTIRE, "ENTIRE" }, \
{ XFS_BMAPI_METADATA, "METADATA" }, \
{ XFS_BMAPI_CONVERT, "CONVERT" }, \
{ XFS_BMAPI_ZERO, "ZERO" }, \
{ XFS_BMAPI_REMAP, "REMAP" }, \
- { XFS_BMAPI_COWFORK, "COWFORK" }
+ { XFS_BMAPI_COWFORK, "COWFORK" }, \
+ { XFS_BMAPI_DELALLOC, "DELALLOC" }
static inline int xfs_bmapi_aflag(int w)
}
STATIC int
-xfs_inobt_alloc_block(
+__xfs_inobt_alloc_block(
struct xfs_btree_cur *cur,
union xfs_btree_ptr *start,
union xfs_btree_ptr *new,
- int *stat)
+ int *stat,
+ enum xfs_ag_resv_type resv)
{
xfs_alloc_arg_t args; /* block allocation args */
int error; /* error return value */
args.maxlen = 1;
args.prod = 1;
args.type = XFS_ALLOCTYPE_NEAR_BNO;
+ args.resv = resv;
error = xfs_alloc_vextent(&args);
if (error) {
return 0;
}
+STATIC int
+xfs_inobt_alloc_block(
+ struct xfs_btree_cur *cur,
+ union xfs_btree_ptr *start,
+ union xfs_btree_ptr *new,
+ int *stat)
+{
+ return __xfs_inobt_alloc_block(cur, start, new, stat, XFS_AG_RESV_NONE);
+}
+
+STATIC int
+xfs_finobt_alloc_block(
+ struct xfs_btree_cur *cur,
+ union xfs_btree_ptr *start,
+ union xfs_btree_ptr *new,
+ int *stat)
+{
+ return __xfs_inobt_alloc_block(cur, start, new, stat,
+ XFS_AG_RESV_METADATA);
+}
+
STATIC int
xfs_inobt_free_block(
struct xfs_btree_cur *cur,
.dup_cursor = xfs_inobt_dup_cursor,
.set_root = xfs_finobt_set_root,
- .alloc_block = xfs_inobt_alloc_block,
+ .alloc_block = xfs_finobt_alloc_block,
.free_block = xfs_inobt_free_block,
.get_minrecs = xfs_inobt_get_minrecs,
.get_maxrecs = xfs_inobt_get_maxrecs,
return 0;
}
#endif /* DEBUG */
+
+static xfs_extlen_t
+xfs_inobt_max_size(
+ struct xfs_mount *mp)
+{
+ /* Bail out if we're uninitialized, which can happen in mkfs. */
+ if (mp->m_inobt_mxr[0] == 0)
+ return 0;
+
+ return xfs_btree_calc_size(mp, mp->m_inobt_mnr,
+ (uint64_t)mp->m_sb.sb_agblocks * mp->m_sb.sb_inopblock /
+ XFS_INODES_PER_CHUNK);
+}
+
+static int
+xfs_inobt_count_blocks(
+ struct xfs_mount *mp,
+ xfs_agnumber_t agno,
+ xfs_btnum_t btnum,
+ xfs_extlen_t *tree_blocks)
+{
+ struct xfs_buf *agbp;
+ struct xfs_btree_cur *cur;
+ int error;
+
+ error = xfs_ialloc_read_agi(mp, NULL, agno, &agbp);
+ if (error)
+ return error;
+
+ cur = xfs_inobt_init_cursor(mp, NULL, agbp, agno, btnum);
+ error = xfs_btree_count_blocks(cur, tree_blocks);
+ xfs_btree_del_cursor(cur, error ? XFS_BTREE_ERROR : XFS_BTREE_NOERROR);
+ xfs_buf_relse(agbp);
+
+ return error;
+}
+
+/*
+ * Figure out how many blocks to reserve and how many are used by this btree.
+ */
+int
+xfs_finobt_calc_reserves(
+ struct xfs_mount *mp,
+ xfs_agnumber_t agno,
+ xfs_extlen_t *ask,
+ xfs_extlen_t *used)
+{
+ xfs_extlen_t tree_len = 0;
+ int error;
+
+ if (!xfs_sb_version_hasfinobt(&mp->m_sb))
+ return 0;
+
+ error = xfs_inobt_count_blocks(mp, agno, XFS_BTNUM_FINO, &tree_len);
+ if (error)
+ return error;
+
+ *ask += xfs_inobt_max_size(mp);
+ *used += tree_len;
+ return 0;
+}
#define xfs_inobt_rec_check_count(mp, rec) 0
#endif /* DEBUG */
+int xfs_finobt_calc_reserves(struct xfs_mount *mp, xfs_agnumber_t agno,
+ xfs_extlen_t *ask, xfs_extlen_t *used);
+
#endif /* __XFS_IALLOC_BTREE_H__ */
sbp->sb_blocklog < XFS_MIN_BLOCKSIZE_LOG ||
sbp->sb_blocklog > XFS_MAX_BLOCKSIZE_LOG ||
sbp->sb_blocksize != (1 << sbp->sb_blocklog) ||
- sbp->sb_dirblklog > XFS_MAX_BLOCKSIZE_LOG ||
+ sbp->sb_dirblklog + sbp->sb_blocklog > XFS_MAX_BLOCKSIZE_LOG ||
sbp->sb_inodesize < XFS_DINODE_MIN_SIZE ||
sbp->sb_inodesize > XFS_DINODE_MAX_SIZE ||
sbp->sb_inodelog < XFS_DINODE_MIN_LOG ||
xfs_bmbt_irec_t *map; /* buffer for user's data */
xfs_mount_t *mp; /* file system mount point */
int nex; /* # of user extents can do */
- int nexleft; /* # of user extents left */
int subnex; /* # of bmapi's can do */
int nmap; /* number of map entries */
struct getbmapx *out; /* output structure */
goto out_free_map;
}
- nexleft = nex;
-
do {
- nmap = (nexleft > subnex) ? subnex : nexleft;
+ nmap = (nex> subnex) ? subnex : nex;
error = xfs_bmapi_read(ip, XFS_BB_TO_FSBT(mp, bmv->bmv_offset),
XFS_BB_TO_FSB(mp, bmv->bmv_length),
map, &nmap, bmapi_flags);
goto out_free_map;
ASSERT(nmap <= subnex);
- for (i = 0; i < nmap && nexleft && bmv->bmv_length &&
- cur_ext < bmv->bmv_count; i++) {
+ for (i = 0; i < nmap && bmv->bmv_length &&
+ cur_ext < bmv->bmv_count - 1; i++) {
out[cur_ext].bmv_oflags = 0;
if (map[i].br_state == XFS_EXT_UNWRITTEN)
out[cur_ext].bmv_oflags |= BMV_OF_PREALLOC;
continue;
}
+ /*
+ * In order to report shared extents accurately,
+ * we report each distinct shared/unshared part
+ * of a single bmbt record using multiple bmap
+ * extents. To make that happen, we iterate the
+ * same map array item multiple times, each
+ * time trimming out the subextent that we just
+ * reported.
+ *
+ * Because of this, we must check the out array
+ * index (cur_ext) directly against bmv_count-1
+ * to avoid overflows.
+ */
if (inject_map.br_startblock != NULLFSBLOCK) {
map[i] = inject_map;
i--;
- } else
- nexleft--;
+ }
bmv->bmv_entries++;
cur_ext++;
}
- } while (nmap && nexleft && bmv->bmv_length &&
- cur_ext < bmv->bmv_count);
+ } while (nmap && bmv->bmv_length && cur_ext < bmv->bmv_count - 1);
out_free_map:
kmem_free(map);
out_free_pages:
for (i = 0; i < bp->b_page_count; i++)
__free_page(bp->b_pages[i]);
+ bp->b_flags &= ~_XBF_PAGES;
return error;
}
int error;
/*
- * The ifree transaction might need to allocate blocks for record
- * insertion to the finobt. We don't want to fail here at ENOSPC, so
- * allow ifree to dip into the reserved block pool if necessary.
- *
- * Freeing large sets of inodes generally means freeing inode chunks,
- * directory and file data blocks, so this should be relatively safe.
- * Only under severe circumstances should it be possible to free enough
- * inodes to exhaust the reserve block pool via finobt expansion while
- * at the same time not creating free space in the filesystem.
+ * We try to use a per-AG reservation for any block needed by the finobt
+ * tree, but as the finobt feature predates the per-AG reservation
+ * support a degraded file system might not have enough space for the
+ * reservation at mount time. In that case try to dip into the reserved
+ * pool and pray.
*
* Send a warning if the reservation does happen to fail, as the inode
* now remains allocated and sits on the unlinked list until the fs is
* repaired.
*/
- error = xfs_trans_alloc(mp, &M_RES(mp)->tr_ifree,
- XFS_IFREE_SPACE_RES(mp), 0, XFS_TRANS_RESERVE, &tp);
+ if (unlikely(mp->m_inotbt_nores)) {
+ error = xfs_trans_alloc(mp, &M_RES(mp)->tr_ifree,
+ XFS_IFREE_SPACE_RES(mp), 0, XFS_TRANS_RESERVE,
+ &tp);
+ } else {
+ error = xfs_trans_alloc(mp, &M_RES(mp)->tr_ifree, 0, 0, 0, &tp);
+ }
if (error) {
if (error == -ENOSPC) {
xfs_warn_ratelimited(mp,
xfs_trans_t *tp;
int nimaps;
int error = 0;
- int flags = 0;
+ int flags = XFS_BMAPI_DELALLOC;
int nres;
if (whichfork == XFS_COW_FORK)
int m_fixedfsid[2]; /* unchanged for life of FS */
uint m_dmevmask; /* DMI events for this FS */
__uint64_t m_flags; /* global mount flags */
+ bool m_inotbt_nores; /* no per-AG finobt resv. */
int m_ialloc_inos; /* inodes in inode allocation */
int m_ialloc_blks; /* blocks in inode allocation */
int m_ialloc_min_blks;/* min blocks in sparse inode
* the case in all other instances. It's OK that we do this because
* quotacheck is done only at mount time.
*/
- error = xfs_iget(mp, NULL, ino, 0, XFS_ILOCK_EXCL, &ip);
+ error = xfs_iget(mp, NULL, ino, XFS_IGET_DONTCACHE, XFS_ILOCK_EXCL,
+ &ip);
if (error) {
*res = BULKSTAT_RV_NOTHING;
return error;
#ifndef KSYM_ALIGN
#define KSYM_ALIGN 8
#endif
-#ifndef KCRC_ALIGN
-#define KCRC_ALIGN 8
-#endif
#else
#define __put .long
#ifndef KSYM_ALIGN
#define KSYM_ALIGN 4
#endif
+#endif
#ifndef KCRC_ALIGN
#define KCRC_ALIGN 4
#endif
-#endif
#ifdef CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX
#define KSYM(name) _##name
.section ___kcrctab\sec+\name,"a"
.balign KCRC_ALIGN
KSYM(__kcrctab_\name):
- __put KSYM(__crc_\name)
+#if defined(CONFIG_MODULE_REL_CRCS)
+ .long KSYM(__crc_\name) - .
+#else
+ .long KSYM(__crc_\name)
+#endif
.weak KSYM(__crc_\name)
.previous
#endif
struct drm_minor *control; /**< Control node */
struct drm_minor *primary; /**< Primary node */
struct drm_minor *render; /**< Render node */
+ bool registered;
/* currently active master for this device. Protected by master_mutex */
struct drm_master *master;
struct drm_crtc *ptr;
struct drm_crtc_state *state;
struct drm_crtc_commit *commit;
- s64 __user *out_fence_ptr;
+ s32 __user *out_fence_ptr;
};
struct __drm_connnectors_state {
* core drm connector interfaces. Everything added from this callback
* should be unregistered in the early_unregister callback.
*
+ * This is called while holding drm_connector->mutex.
+ *
* Returns:
*
* 0 on success, or a negative error code on failure.
* late_register(). It is called from drm_connector_unregister(),
* early in the driver unload sequence to disable userspace access
* before data structures are torndown.
+ *
+ * This is called while holding drm_connector->mutex.
*/
void (*early_unregister)(struct drm_connector *connector);
* @interlace_allowed: can this connector handle interlaced modes?
* @doublescan_allowed: can this connector handle doublescan?
* @stereo_allowed: can this connector handle stereo modes?
- * @registered: is this connector exposed (registered) with userspace?
* @modes: modes available on this connector (from fill_modes() + user)
* @status: one of the drm_connector_status enums (connected, not, or unknown)
* @probed_modes: list of modes derived directly from the display
char *name;
+ /**
+ * @mutex: Lock for general connector state, but currently only protects
+ * @registered. Most of the connector state is still protected by the
+ * mutex in &drm_mode_config.
+ */
+ struct mutex mutex;
+
/**
* @index: Compacted connector index, which matches the position inside
* the mode_config.list for drivers not supporting hot-add/removing. Can
bool interlace_allowed;
bool doublescan_allowed;
bool stereo_allowed;
+ /**
+ * @registered: Is this connector exposed (registered) with userspace?
+ * Protected by @mutex.
+ */
bool registered;
struct list_head modes; /* list of modes on this connector */
/**
* @prop_out_fence_ptr: Sync File fd pointer representing the
* outgoing fences for a CRTC. Userspace should provide a pointer to a
- * value of type s64, and then cast that pointer to u64.
+ * value of type s32, and then cast that pointer to u64.
*/
struct drm_property *prop_out_fence_ptr;
/**
void bpf_map_put_with_uref(struct bpf_map *map);
void bpf_map_put(struct bpf_map *map);
int bpf_map_precharge_memlock(u32 pages);
+void *bpf_map_area_alloc(size_t size);
+void bpf_map_area_free(void *base);
extern int sysctl_unprivileged_bpf_disabled;
{
if (err == 0)
return VM_FAULT_LOCKED;
- if (err == -EFAULT)
+ if (err == -EFAULT || err == -EAGAIN)
return VM_FAULT_NOPAGE;
if (err == -ENOMEM)
return VM_FAULT_OOM;
- if (err == -EAGAIN)
- return VM_FAULT_RETRY;
/* -ENOSPC, -EDQUOT, -EIO ... */
return VM_FAULT_SIGBUS;
}
extern int can_proto_register(const struct can_proto *cp);
extern void can_proto_unregister(const struct can_proto *cp);
-extern int can_rx_register(struct net_device *dev, canid_t can_id,
- canid_t mask,
- void (*func)(struct sk_buff *, void *),
- void *data, char *ident);
+int can_rx_register(struct net_device *dev, canid_t can_id, canid_t mask,
+ void (*func)(struct sk_buff *, void *),
+ void *data, char *ident, struct sock *sk);
extern void can_rx_unregister(struct net_device *dev, canid_t can_id,
canid_t mask,
#define CPUFREQ_ETERNAL (-1)
#define CPUFREQ_NAME_LEN 16
-/* Print length for names. Extra 1 space for accomodating '\n' in prints */
+/* Print length for names. Extra 1 space for accommodating '\n' in prints */
#define CPUFREQ_NAME_PLEN (CPUFREQ_NAME_LEN + 1)
struct cpufreq_governor;
* guarantee that frequency can be changed on any CPU sharing the
* policy and that the change will affect all of the policy CPUs then.
* - fast_switch_enabled is to be set by governors that support fast
- * freqnency switching with the help of cpufreq_enable_fast_switch().
+ * frequency switching with the help of cpufreq_enable_fast_switch().
*/
bool fast_switch_possible;
bool fast_switch_enabled;
/* Policy Notifiers */
#define CPUFREQ_ADJUST (0)
#define CPUFREQ_NOTIFY (1)
-#define CPUFREQ_START (2)
-#define CPUFREQ_CREATE_POLICY (3)
-#define CPUFREQ_REMOVE_POLICY (4)
#ifdef CONFIG_CPU_FREQ
int cpufreq_register_notifier(struct notifier_block *nb, unsigned int list);
CPUHP_CREATE_THREADS,
CPUHP_PERF_PREPARE,
CPUHP_PERF_X86_PREPARE,
- CPUHP_PERF_X86_UNCORE_PREP,
CPUHP_PERF_X86_AMD_UNCORE_PREP,
- CPUHP_PERF_X86_RAPL_PREP,
CPUHP_PERF_BFIN,
CPUHP_PERF_POWER,
CPUHP_PERF_SUPERH,
CPUHP_AP_IRQ_ARMADA_XP_STARTING,
CPUHP_AP_IRQ_BCM2836_STARTING,
CPUHP_AP_ARM_MVEBU_COHERENCY,
- CPUHP_AP_PERF_X86_UNCORE_STARTING,
CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
CPUHP_AP_PERF_X86_STARTING,
CPUHP_AP_PERF_X86_AMD_IBS_STARTING,
static inline int cpumask_parse_user(const char __user *buf, int len,
struct cpumask *dstp)
{
- return bitmap_parse_user(buf, len, cpumask_bits(dstp), nr_cpu_ids);
+ return bitmap_parse_user(buf, len, cpumask_bits(dstp), nr_cpumask_bits);
}
/**
struct cpumask *dstp)
{
return bitmap_parselist_user(buf, len, cpumask_bits(dstp),
- nr_cpu_ids);
+ nr_cpumask_bits);
}
/**
char *nl = strchr(buf, '\n');
unsigned int len = nl ? (unsigned int)(nl - buf) : strlen(buf);
- return bitmap_parse(buf, len, cpumask_bits(dstp), nr_cpu_ids);
+ return bitmap_parse(buf, len, cpumask_bits(dstp), nr_cpumask_bits);
}
/**
*/
static inline int cpulist_parse(const char *buf, struct cpumask *dstp)
{
- return bitmap_parselist(buf, cpumask_bits(dstp), nr_cpu_ids);
+ return bitmap_parselist(buf, cpumask_bits(dstp), nr_cpumask_bits);
}
/**
* struct devfreq_governor - Devfreq policy governor
* @node: list node - contains registered devfreq governors
* @name: Governor's name
+ * @immutable: Immutable flag for governor. If the value is 1,
+ * this govenror is never changeable to other governor.
* @get_target_freq: Returns desired operating frequency for the device.
* Basically, get_target_freq will run
* devfreq_dev_profile.get_dev_status() to get the
struct list_head node;
const char name[DEVFREQ_NAME_LEN];
+ const unsigned int immutable;
int (*get_target_freq)(struct devfreq *this, unsigned long *freq);
int (*event_handler)(struct devfreq *devfreq,
unsigned int event, void *data);
#ifdef CONFIG_MODVERSIONS
/* Mark the CRC weak since genksyms apparently decides not to
* generate a checksums for some symbols */
+#if defined(CONFIG_MODULE_REL_CRCS)
#define __CRC_SYMBOL(sym, sec) \
- extern __visible void *__crc_##sym __attribute__((weak)); \
- static const unsigned long __kcrctab_##sym \
- __used \
- __attribute__((section("___kcrctab" sec "+" #sym), used)) \
- = (unsigned long) &__crc_##sym;
+ asm(" .section \"___kcrctab" sec "+" #sym "\", \"a\" \n" \
+ " .weak " VMLINUX_SYMBOL_STR(__crc_##sym) " \n" \
+ " .long " VMLINUX_SYMBOL_STR(__crc_##sym) " - . \n" \
+ " .previous \n");
+#else
+#define __CRC_SYMBOL(sym, sec) \
+ asm(" .section \"___kcrctab" sec "+" #sym "\", \"a\" \n" \
+ " .weak " VMLINUX_SYMBOL_STR(__crc_##sym) " \n" \
+ " .long " VMLINUX_SYMBOL_STR(__crc_##sym) " \n" \
+ " .previous \n");
+#endif
#else
#define __CRC_SYMBOL(sym, sec)
#endif
#define FSCACHE_OBJECT_IS_AVAILABLE 5 /* T if object has become active */
#define FSCACHE_OBJECT_RETIRED 6 /* T if object was retired on relinquishment */
#define FSCACHE_OBJECT_KILLED_BY_CACHE 7 /* T if object was killed by the cache */
+#define FSCACHE_OBJECT_RUN_AFTER_DEAD 8 /* T if object has been dispatched after death */
struct list_head cache_link; /* link in cache->object_list */
struct hlist_node cookie_link; /* link in cookie->backing_objects */
struct irq_chip *irqchip,
int parent_irq);
-int _gpiochip_irqchip_add(struct gpio_chip *gpiochip,
+int gpiochip_irqchip_add_key(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type,
+ bool nested,
+ struct lock_class_key *lock_key);
+
+#ifdef CONFIG_LOCKDEP
+
+/*
+ * Lockdep requires that each irqchip instance be created with a
+ * unique key so as to avoid unnecessary warnings. This upfront
+ * boilerplate static inlines provides such a key for each
+ * unique instance.
+ */
+static inline int gpiochip_irqchip_add(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type)
+{
+ static struct lock_class_key key;
+
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, false, &key);
+}
+
+static inline int gpiochip_irqchip_add_nested(struct gpio_chip *gpiochip,
struct irq_chip *irqchip,
unsigned int first_irq,
irq_flow_handler_t handler,
- unsigned int type,
- bool nested,
- struct lock_class_key *lock_key);
+ unsigned int type)
+{
+
+ static struct lock_class_key key;
+
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, true, &key);
+}
+#else
+static inline int gpiochip_irqchip_add(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type)
+{
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, false, NULL);
+}
-/* FIXME: I assume threaded IRQchips do not have the lockdep problem */
static inline int gpiochip_irqchip_add_nested(struct gpio_chip *gpiochip,
struct irq_chip *irqchip,
unsigned int first_irq,
irq_flow_handler_t handler,
unsigned int type)
{
- return _gpiochip_irqchip_add(gpiochip, irqchip, first_irq,
- handler, type, true, NULL);
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, true, NULL);
}
-
-#ifdef CONFIG_LOCKDEP
-#define gpiochip_irqchip_add(...) \
-( \
- ({ \
- static struct lock_class_key _key; \
- _gpiochip_irqchip_add(__VA_ARGS__, false, &_key); \
- }) \
-)
-#else
-#define gpiochip_irqchip_add(...) \
- _gpiochip_irqchip_add(__VA_ARGS__, false, NULL)
-#endif
+#endif /* CONFIG_LOCKDEP */
#endif /* CONFIG_GPIOLIB_IRQCHIP */
u32 ring_data_startoffset;
u32 priv_write_index;
u32 priv_read_index;
+ u32 cached_read_index;
};
/*
return write;
}
+static inline u32 hv_get_cached_bytes_to_write(
+ const struct hv_ring_buffer_info *rbi)
+{
+ u32 read_loc, write_loc, dsize, write;
+
+ dsize = rbi->ring_datasize;
+ read_loc = rbi->cached_read_index;
+ write_loc = rbi->ring_buffer->write_index;
+
+ write = write_loc >= read_loc ? dsize - (write_loc - read_loc) :
+ read_loc - write_loc;
+ return write;
+}
/*
* VMBUS version is 32 bit entity broken up into
* two 16 bit quantities: major_number. minor_number.
static inline void hv_signal_on_read(struct vmbus_channel *channel)
{
- u32 cur_write_sz;
+ u32 cur_write_sz, cached_write_sz;
u32 pending_sz;
struct hv_ring_buffer_info *rbi = &channel->inbound;
cur_write_sz = hv_get_bytes_to_write(rbi);
- if (cur_write_sz >= pending_sz)
+ if (cur_write_sz < pending_sz)
+ return;
+
+ cached_write_sz = hv_get_cached_bytes_to_write(rbi);
+ if (cached_write_sz < pending_sz)
vmbus_setevent(channel);
return;
}
+static inline void
+init_cached_read_index(struct vmbus_channel *channel)
+{
+ struct hv_ring_buffer_info *rbi = &channel->inbound;
+
+ rbi->cached_read_index = rbi->ring_buffer->read_index;
+}
+
/*
* An API to support in-place processing of incoming VMBUS packets.
*/
* This call commits the read index and potentially signals the host.
* Here is the pattern for using the "in-place" consumption APIs:
*
+ * init_cached_read_index();
+ *
* while (get_next_pkt_raw() {
* process the packet "in-place";
* put_pkt_raw();
*
* IRQD_TRIGGER_MASK - Mask for the trigger type bits
* IRQD_SETAFFINITY_PENDING - Affinity setting is pending
+ * IRQD_ACTIVATED - Interrupt has already been activated
* IRQD_NO_BALANCING - Balancing disabled for this IRQ
* IRQD_PER_CPU - Interrupt is per cpu
* IRQD_AFFINITY_SET - Interrupt affinity was set
enum {
IRQD_TRIGGER_MASK = 0xf,
IRQD_SETAFFINITY_PENDING = (1 << 8),
+ IRQD_ACTIVATED = (1 << 9),
IRQD_NO_BALANCING = (1 << 10),
IRQD_PER_CPU = (1 << 11),
IRQD_AFFINITY_SET = (1 << 12),
return __irqd_to_state(d) & IRQD_AFFINITY_MANAGED;
}
+static inline bool irqd_is_activated(struct irq_data *d)
+{
+ return __irqd_to_state(d) & IRQD_ACTIVATED;
+}
+
+static inline void irqd_set_activated(struct irq_data *d)
+{
+ __irqd_to_state(d) |= IRQD_ACTIVATED;
+}
+
+static inline void irqd_clr_activated(struct irq_data *d)
+{
+ __irqd_to_state(d) &= ~IRQD_ACTIVATED;
+}
+
#undef __irqd_to_state
static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
* ... and so on.
*/
-#define order_base_2(n) ilog2(roundup_pow_of_two(n))
+static inline __attribute_const__
+int __order_base_2(unsigned long n)
+{
+ return n > 1 ? ilog2(n - 1) + 1 : 0;
+}
+#define order_base_2(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ ((n) == 0 || (n) == 1) ? 0 : \
+ ilog2((n) - 1) + 1) : \
+ __order_base_2(n) \
+)
#endif /* _LINUX_LOG2_H */
extern int add_one_highpage(struct page *page, int pfn, int bad_ppro);
/* VM interface that may be used by firmware interface */
extern int online_pages(unsigned long, unsigned long, int);
-extern int test_pages_in_a_zone(unsigned long, unsigned long);
+extern int test_pages_in_a_zone(unsigned long start_pfn, unsigned long end_pfn,
+ unsigned long *valid_start, unsigned long *valid_end);
extern void __offline_isolated_pages(unsigned long, unsigned long);
typedef void (*online_page_callback_t)(struct page *page);
unsigned long map_offset);
extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
unsigned long pnum);
-extern int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
- enum zone_type target);
+extern bool zone_can_shift(unsigned long pfn, unsigned long nr_pages,
+ enum zone_type target, int *zone_shift);
#endif /* __LINUX_MEMORY_HOTPLUG_H */
#define PHY_ID_KSZ886X 0x00221430
#define PHY_ID_KSZ8863 0x00221435
+#define PHY_ID_KSZ8795 0x00221550
+
/* struct phy_device dev_flags definitions */
#define MICREL_PHY_50MHZ_CLK 0x00000001
#define MICREL_PHY_FXEN 0x00000002
* @zonelist - The zonelist to search for a suitable zone
* @highest_zoneidx - The zone index of the highest zone to return
* @nodes - An optional nodemask to filter the zonelist with
- * @zone - The first suitable zone found is returned via this parameter
+ * @return - Zoneref pointer for the first suitable zone found (see below)
*
* This function returns the first zone at or below a given zone index that is
* within the allowed nodemask. The zoneref returned is a cursor that can be
* used to iterate the zonelist with next_zones_zonelist by advancing it by
* one before calling.
+ *
+ * When no eligible zone is found, zoneref->zone is NULL (zoneref itself is
+ * never NULL). This may happen either genuinely, or due to concurrent nodemask
+ * update due to cpuset modification.
*/
static inline struct zoneref *first_zones_zonelist(struct zonelist *zonelist,
enum zone_type highest_zoneidx,
/* Exported symbols */
const struct kernel_symbol *syms;
- const unsigned long *crcs;
+ const s32 *crcs;
unsigned int num_syms;
/* Kernel parameters. */
/* GPL-only exported symbols. */
unsigned int num_gpl_syms;
const struct kernel_symbol *gpl_syms;
- const unsigned long *gpl_crcs;
+ const s32 *gpl_crcs;
#ifdef CONFIG_UNUSED_SYMBOLS
/* unused exported symbols. */
const struct kernel_symbol *unused_syms;
- const unsigned long *unused_crcs;
+ const s32 *unused_crcs;
unsigned int num_unused_syms;
/* GPL-only, unused exported symbols. */
unsigned int num_unused_gpl_syms;
const struct kernel_symbol *unused_gpl_syms;
- const unsigned long *unused_gpl_crcs;
+ const s32 *unused_gpl_crcs;
#endif
#ifdef CONFIG_MODULE_SIG
/* symbols that will be GPL-only in the near future. */
const struct kernel_symbol *gpl_future_syms;
- const unsigned long *gpl_future_crcs;
+ const s32 *gpl_future_crcs;
unsigned int num_gpl_future_syms;
/* Exception table */
struct symsearch {
const struct kernel_symbol *start, *stop;
- const unsigned long *crcs;
+ const s32 *crcs;
enum {
NOT_GPL_ONLY,
GPL_ONLY,
*/
const struct kernel_symbol *find_symbol(const char *name,
struct module **owner,
- const unsigned long **crc,
+ const s32 **crc,
bool gplok,
bool warn);
* of useless work if you return NETDEV_TX_BUSY.
* Required; cannot be NULL.
*
- * netdev_features_t (*ndo_fix_features)(struct net_device *dev,
- * netdev_features_t features);
- * Adjusts the requested feature flags according to device-specific
- * constraints, and returns the resulting flags. Must not modify
- * the device state.
+ * netdev_features_t (*ndo_features_check)(struct sk_buff *skb,
+ * struct net_device *dev
+ * netdev_features_t features);
+ * Called by core transmit path to determine if device is capable of
+ * performing offload operations on a given packet. This is to give
+ * the device an opportunity to implement any restrictions that cannot
+ * be otherwise expressed by feature flags. The check is called with
+ * the set of features that the stack has calculated and it returns
+ * those the driver believes to be appropriate.
*
* u16 (*ndo_select_queue)(struct net_device *dev, struct sk_buff *skb,
* void *accel_priv, select_queue_fallback_t fallback);
* Called to release previously enslaved netdev.
*
* Feature/offload setting functions.
+ * netdev_features_t (*ndo_fix_features)(struct net_device *dev,
+ * netdev_features_t features);
+ * Adjusts the requested feature flags according to device-specific
+ * constraints, and returns the resulting flags. Must not modify
+ * the device state.
+ *
* int (*ndo_set_features)(struct net_device *dev, netdev_features_t features);
* Called to update device configuration to new features. Passed
* feature set might be less than what was returned by ndo_fix_features()).
* Callback to use for xmit over the accelerated station. This
* is used in place of ndo_start_xmit on accelerated net
* devices.
- * netdev_features_t (*ndo_features_check)(struct sk_buff *skb,
- * struct net_device *dev
- * netdev_features_t features);
- * Called by core transmit path to determine if device is capable of
- * performing offload operations on a given packet. This is to give
- * the device an opportunity to implement any restrictions that cannot
- * be otherwise expressed by feature flags. The check is called with
- * the set of features that the stack has calculated and it returns
- * those the driver believes to be appropriate.
* int (*ndo_set_tx_maxrate)(struct net_device *dev,
* int queue_index, u32 maxrate);
* Called when a user wants to set a max-rate limitation of specific
* @max_mtu: Interface Maximum MTU value
* @type: Interface hardware type
* @hard_header_len: Maximum hardware header length.
+ * @min_header_len: Minimum hardware header length
*
* @needed_headroom: Extra headroom the hardware may need, but not in all
* cases can this be guaranteed
unsigned int max_mtu;
unsigned short type;
unsigned short hard_header_len;
+ unsigned short min_header_len;
unsigned short needed_headroom;
unsigned short needed_tailroom;
{
if (likely(len >= dev->hard_header_len))
return true;
+ if (len < dev->min_header_len)
+ return false;
if (capable(CAP_SYS_RAWIO)) {
memset(ll_header + len, 0, dev->hard_header_len - len);
static inline bool seqid_mutating_err(u32 err)
{
- /* rfc 3530 section 8.1.5: */
+ /* See RFC 7530, section 9.1.7 */
switch (err) {
case NFS4ERR_STALE_CLIENTID:
case NFS4ERR_STALE_STATEID:
case NFS4ERR_BADXDR:
case NFS4ERR_RESOURCE:
case NFS4ERR_NOFILEHANDLE:
+ case NFS4ERR_MOVED:
return false;
};
return true;
extern int watchdog_thresh;
extern unsigned long watchdog_enabled;
extern unsigned long *watchdog_cpumask_bits;
+extern atomic_t watchdog_park_in_progress;
#ifdef CONFIG_SMP
extern int sysctl_softlockup_all_cpu_backtrace;
extern int sysctl_hardlockup_all_cpu_backtrace;
static inline bool percpu_ref_tryget(struct percpu_ref *ref)
{
unsigned long __percpu *percpu_count;
- int ret;
+ bool ret;
rcu_read_lock_sched();
static inline bool percpu_ref_tryget_live(struct percpu_ref *ref)
{
unsigned long __percpu *percpu_count;
- int ret = false;
+ bool ret = false;
rcu_read_lock_sched();
#include <linux/timer.h>
#include <linux/workqueue.h>
#include <linux/mod_devicetable.h>
-#include <linux/phy_led_triggers.h>
#include <linux/atomic.h>
#ifdef CONFIG_LED_TRIGGER_PHY
#include <linux/leds.h>
+#include <linux/phy.h>
#define PHY_LED_TRIGGER_SPEED_SUFFIX_SIZE 10
-#define PHY_MII_BUS_ID_SIZE (20 - 3)
-#define PHY_LINK_LED_TRIGGER_NAME_SIZE (PHY_MII_BUS_ID_SIZE + \
+#define PHY_LINK_LED_TRIGGER_NAME_SIZE (MII_BUS_ID_SIZE + \
FIELD_SIZEOF(struct mdio_device, addr)+\
PHY_LED_TRIGGER_SPEED_SUFFIX_SIZE)
{
return -ENOTSUPP;
}
+
+#define simple_qos_governor (*(struct dev_power_governor *)(NULL))
+#define pm_domain_always_on_gov (*(struct dev_power_governor *)(NULL))
#endif
static inline int pm_genpd_add_device(struct generic_pm_domain *genpd,
#if defined(CONFIG_PM_OPP)
+struct opp_table *dev_pm_opp_get_opp_table(struct device *dev);
+void dev_pm_opp_put_opp_table(struct opp_table *opp_table);
+
unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp);
unsigned long dev_pm_opp_get_freq(struct dev_pm_opp *opp);
unsigned long dev_pm_opp_get_max_clock_latency(struct device *dev);
unsigned long dev_pm_opp_get_max_volt_latency(struct device *dev);
unsigned long dev_pm_opp_get_max_transition_latency(struct device *dev);
-struct dev_pm_opp *dev_pm_opp_get_suspend_opp(struct device *dev);
+unsigned long dev_pm_opp_get_suspend_opp_freq(struct device *dev);
struct dev_pm_opp *dev_pm_opp_find_freq_exact(struct device *dev,
unsigned long freq,
struct dev_pm_opp *dev_pm_opp_find_freq_ceil(struct device *dev,
unsigned long *freq);
+void dev_pm_opp_put(struct dev_pm_opp *opp);
int dev_pm_opp_add(struct device *dev, unsigned long freq,
unsigned long u_volt);
int dev_pm_opp_disable(struct device *dev, unsigned long freq);
-struct srcu_notifier_head *dev_pm_opp_get_notifier(struct device *dev);
-int dev_pm_opp_set_supported_hw(struct device *dev, const u32 *versions,
- unsigned int count);
-void dev_pm_opp_put_supported_hw(struct device *dev);
-int dev_pm_opp_set_prop_name(struct device *dev, const char *name);
-void dev_pm_opp_put_prop_name(struct device *dev);
+int dev_pm_opp_register_notifier(struct device *dev, struct notifier_block *nb);
+int dev_pm_opp_unregister_notifier(struct device *dev, struct notifier_block *nb);
+
+struct opp_table *dev_pm_opp_set_supported_hw(struct device *dev, const u32 *versions, unsigned int count);
+void dev_pm_opp_put_supported_hw(struct opp_table *opp_table);
+struct opp_table *dev_pm_opp_set_prop_name(struct device *dev, const char *name);
+void dev_pm_opp_put_prop_name(struct opp_table *opp_table);
struct opp_table *dev_pm_opp_set_regulators(struct device *dev, const char * const names[], unsigned int count);
void dev_pm_opp_put_regulators(struct opp_table *opp_table);
-int dev_pm_opp_register_set_opp_helper(struct device *dev, int (*set_opp)(struct dev_pm_set_opp_data *data));
-void dev_pm_opp_register_put_opp_helper(struct device *dev);
+struct opp_table *dev_pm_opp_register_set_opp_helper(struct device *dev, int (*set_opp)(struct dev_pm_set_opp_data *data));
+void dev_pm_opp_register_put_opp_helper(struct opp_table *opp_table);
int dev_pm_opp_set_rate(struct device *dev, unsigned long target_freq);
int dev_pm_opp_set_sharing_cpus(struct device *cpu_dev, const struct cpumask *cpumask);
int dev_pm_opp_get_sharing_cpus(struct device *cpu_dev, struct cpumask *cpumask);
void dev_pm_opp_remove_table(struct device *dev);
void dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask);
#else
+static inline struct opp_table *dev_pm_opp_get_opp_table(struct device *dev)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
+
+static inline void dev_pm_opp_put_opp_table(struct opp_table *opp_table) {}
+
static inline unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp)
{
return 0;
return 0;
}
-static inline struct dev_pm_opp *dev_pm_opp_get_suspend_opp(struct device *dev)
+static inline unsigned long dev_pm_opp_get_suspend_opp_freq(struct device *dev)
{
- return NULL;
+ return 0;
}
static inline struct dev_pm_opp *dev_pm_opp_find_freq_exact(struct device *dev,
return ERR_PTR(-ENOTSUPP);
}
+static inline void dev_pm_opp_put(struct dev_pm_opp *opp) {}
+
static inline int dev_pm_opp_add(struct device *dev, unsigned long freq,
unsigned long u_volt)
{
return 0;
}
-static inline struct srcu_notifier_head *dev_pm_opp_get_notifier(
- struct device *dev)
+static inline int dev_pm_opp_register_notifier(struct device *dev, struct notifier_block *nb)
{
- return ERR_PTR(-ENOTSUPP);
+ return -ENOTSUPP;
}
-static inline int dev_pm_opp_set_supported_hw(struct device *dev,
- const u32 *versions,
- unsigned int count)
+static inline int dev_pm_opp_unregister_notifier(struct device *dev, struct notifier_block *nb)
{
return -ENOTSUPP;
}
-static inline void dev_pm_opp_put_supported_hw(struct device *dev) {}
+static inline struct opp_table *dev_pm_opp_set_supported_hw(struct device *dev,
+ const u32 *versions,
+ unsigned int count)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
-static inline int dev_pm_opp_register_set_opp_helper(struct device *dev,
+static inline void dev_pm_opp_put_supported_hw(struct opp_table *opp_table) {}
+
+static inline struct opp_table *dev_pm_opp_register_set_opp_helper(struct device *dev,
int (*set_opp)(struct dev_pm_set_opp_data *data))
{
- return -ENOTSUPP;
+ return ERR_PTR(-ENOTSUPP);
}
-static inline void dev_pm_opp_register_put_opp_helper(struct device *dev) {}
+static inline void dev_pm_opp_register_put_opp_helper(struct opp_table *opp_table) {}
-static inline int dev_pm_opp_set_prop_name(struct device *dev, const char *name)
+static inline struct opp_table *dev_pm_opp_set_prop_name(struct device *dev, const char *name)
{
- return -ENOTSUPP;
+ return ERR_PTR(-ENOTSUPP);
}
-static inline void dev_pm_opp_put_prop_name(struct device *dev) {}
+static inline void dev_pm_opp_put_prop_name(struct opp_table *opp_table) {}
static inline struct opp_table *dev_pm_opp_set_regulators(struct device *dev, const char * const names[], unsigned int count)
{
int dev_pm_opp_of_cpumask_add_table(const struct cpumask *cpumask);
void dev_pm_opp_of_cpumask_remove_table(const struct cpumask *cpumask);
int dev_pm_opp_of_get_sharing_cpus(struct device *cpu_dev, struct cpumask *cpumask);
+struct device_node *dev_pm_opp_of_get_opp_desc_node(struct device *dev);
#else
static inline int dev_pm_opp_of_add_table(struct device *dev)
{
{
return -ENOTSUPP;
}
+
+static inline struct device_node *dev_pm_opp_of_get_opp_desc_node(struct device *dev)
+{
+ return NULL;
+}
#endif
#endif /* __LINUX_OPP_H__ */
void rpc_clnt_xprt_switch_add_xprt(struct rpc_clnt *, struct rpc_xprt *);
bool rpc_clnt_xprt_switch_has_addr(struct rpc_clnt *clnt,
const struct sockaddr *sap);
+void rpc_cleanup_clids(void);
#endif /* __KERNEL__ */
#endif /* _LINUX_SUNRPC_CLNT_H */
};
#ifdef CONFIG_SUSPEND
-extern suspend_state_t mem_sleep_default;
-
/**
* suspend_set_ops - set platform dependent suspend operations
* @ops: The new suspend operations to set.
static inline int virtio_net_hdr_from_skb(const struct sk_buff *skb,
struct virtio_net_hdr *hdr,
- bool little_endian)
+ bool little_endian,
+ bool has_data_valid)
{
memset(hdr, 0, sizeof(*hdr)); /* no info leak */
skb_checksum_start_offset(skb));
hdr->csum_offset = __cpu_to_virtio16(little_endian,
skb->csum_offset);
- } else if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
+ } else if (has_data_valid &&
+ skb->ip_summed == CHECKSUM_UNNECESSARY) {
hdr->flags = VIRTIO_NET_HDR_F_DATA_VALID;
} /* else everything is zero */
}
for (opt_iter = 6; opt_iter < opt_len;) {
+ if (opt_iter + 1 == opt_len) {
+ err_offset = opt_iter;
+ goto out;
+ }
tag_len = opt[opt_iter + 1];
if ((tag_len == 0) || (tag_len > (opt_len - opt_iter))) {
err_offset = opt_iter + 1;
{
u32 hash;
+ /* @flowlabel may include more than a flow label, eg, the traffic class.
+ * Here we want only the flow label value.
+ */
+ flowlabel &= IPV6_FLOWLABEL_MASK;
+
if (flowlabel ||
net->ipv6.sysctl.auto_flowlabels == IP6_AUTO_FLOW_LABEL_OFF ||
(!autolabel &&
* upper-layer output functions
*/
int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,
- struct ipv6_txoptions *opt, int tclass);
+ __u32 mark, struct ipv6_txoptions *opt, int tclass);
int ip6_find_1stfragopt(struct sk_buff *skb, u8 **nexthdr);
int (*get_encap_size)(struct lwtunnel_state *lwtstate);
int (*cmp_encap)(struct lwtunnel_state *a, struct lwtunnel_state *b);
int (*xmit)(struct sk_buff *skb);
+
+ struct module *owner;
};
#ifdef CONFIG_LWTUNNEL
unsigned int num);
int lwtunnel_encap_del_ops(const struct lwtunnel_encap_ops *op,
unsigned int num);
+int lwtunnel_valid_encap_type(u16 encap_type);
+int lwtunnel_valid_encap_type_attr(struct nlattr *attr, int len);
int lwtunnel_build_state(struct net_device *dev, u16 encap_type,
struct nlattr *encap,
unsigned int family, const void *cfg,
return -EOPNOTSUPP;
}
+static inline int lwtunnel_valid_encap_type(u16 encap_type)
+{
+ return -EOPNOTSUPP;
+}
+static inline int lwtunnel_valid_encap_type_attr(struct nlattr *attr, int len)
+{
+ /* return 0 since we are not walking attr looking for
+ * RTA_ENCAP_TYPE attribute on nexthops.
+ */
+ return 0;
+}
+
static inline int lwtunnel_build_state(struct net_device *dev, u16 encap_type,
struct nlattr *encap,
unsigned int family, const void *cfg,
unsigned int skip;
int err;
int (*fn)(const struct nft_ctx *ctx,
- const struct nft_set *set,
+ struct nft_set *set,
const struct nft_set_iter *iter,
- const struct nft_set_elem *elem);
+ struct nft_set_elem *elem);
};
/**
void (*remove)(const struct nft_set *set,
const struct nft_set_elem *elem);
void (*walk)(const struct nft_ctx *ctx,
- const struct nft_set *set,
+ struct nft_set *set,
struct nft_set_iter *iter);
unsigned int (*privsize)(const struct nlattr * const nla[]);
extern const struct nla_policy nft_fib_policy[];
+static inline bool
+nft_fib_is_loopback(const struct sk_buff *skb, const struct net_device *in)
+{
+ return skb->pkt_type == PACKET_LOOPBACK || in->flags & IFF_LOOPBACK;
+}
+
int nft_fib_dump(struct sk_buff *skb, const struct nft_expr *expr);
int nft_fib_init(const struct nft_ctx *ctx, const struct nft_expr *expr,
const struct nlattr * const tb[]);
void sk_stop_timer(struct sock *sk, struct timer_list *timer);
int __sk_queue_drop_skb(struct sock *sk, struct sk_buff *skb,
- unsigned int flags);
+ unsigned int flags,
+ void (*destructor)(struct sock *sk,
+ struct sk_buff *skb));
int __sock_queue_rcv_skb(struct sock *sk, struct sk_buff *skb);
int sock_queue_rcv_skb(struct sock *sk, struct sk_buff *skb);
}
}
+static inline enum ib_mtu ib_mtu_int_to_enum(int mtu)
+{
+ if (mtu >= 4096)
+ return IB_MTU_4096;
+ else if (mtu >= 2048)
+ return IB_MTU_2048;
+ else if (mtu >= 1024)
+ return IB_MTU_1024;
+ else if (mtu >= 512)
+ return IB_MTU_512;
+ else
+ return IB_MTU_256;
+}
+
enum ib_port_state {
IB_PORT_NOP = 0,
IB_PORT_DOWN = 1,
struct mcip_bcr {
#ifdef CONFIG_CPU_BIG_ENDIAN
- unsigned int pad3:8,
- idu:1, llm:1, num_cores:6,
- iocoh:1, gfrc:1, dbg:1, pad2:1,
- msg:1, sem:1, ipi:1, pad:1,
+ unsigned int pad4:6, pw_dom:1, pad3:1,
+ idu:1, pad2:1, num_cores:6,
+ pad:1, gfrc:1, dbg:1, pw:1,
+ msg:1, sem:1, ipi:1, slv:1,
ver:8;
#else
unsigned int ver:8,
- pad:1, ipi:1, sem:1, msg:1,
- pad2:1, dbg:1, gfrc:1, iocoh:1,
- num_cores:6, llm:1, idu:1,
- pad3:8;
+ slv:1, ipi:1, sem:1, msg:1,
+ pw:1, dbg:1, gfrc:1, pad:1,
+ num_cores:6, pad2:1, idu:1,
+ pad3:1, pw_dom:1, pad4:6;
#endif
};
char initiatorname[TRANSPORT_IQN_LEN];
/* Used to signal demo mode created ACL, disabled by default */
bool dynamic_node_acl;
+ bool dynamic_stop;
u32 queue_depth;
u32 acl_index;
enum target_prot_type saved_prot_type;
__u8 audio_out_compensated,
__u8 audio_out_delay)
{
- msg->len = 7;
+ msg->len = 6;
msg->msg[0] |= 0xf; /* broadcast */
msg->msg[1] = CEC_MSG_REPORT_CURRENT_LATENCY;
msg->msg[2] = phys_addr >> 8;
msg->msg[3] = phys_addr & 0xff;
msg->msg[4] = video_latency;
msg->msg[5] = (low_latency_mode << 2) | audio_out_compensated;
- msg->msg[6] = audio_out_delay;
+ if (audio_out_compensated == 3)
+ msg->msg[msg->len++] = audio_out_delay;
}
static inline void cec_ops_report_current_latency(const struct cec_msg *msg,
*video_latency = msg->msg[4];
*low_latency_mode = (msg->msg[5] >> 2) & 1;
*audio_out_compensated = msg->msg[5] & 3;
- *audio_out_delay = msg->msg[6];
+ if (*audio_out_compensated == 3 && msg->len >= 7)
+ *audio_out_delay = msg->msg[6];
+ else
+ *audio_out_delay = 0;
}
static inline void cec_msg_request_current_latency(struct cec_msg *msg,
ETHTOOL_LINK_MODE_10000baseLR_Full_BIT = 44,
ETHTOOL_LINK_MODE_10000baseLRM_Full_BIT = 45,
ETHTOOL_LINK_MODE_10000baseER_Full_BIT = 46,
+ ETHTOOL_LINK_MODE_2500baseT_Full_BIT = 47,
+ ETHTOOL_LINK_MODE_5000baseT_Full_BIT = 48,
/* Last allowed bit for __ETHTOOL_LINK_MODE_LEGACY_MASK is bit
*/
__ETHTOOL_LINK_MODE_LAST
- = ETHTOOL_LINK_MODE_10000baseER_Full_BIT,
+ = ETHTOOL_LINK_MODE_5000baseT_Full_BIT,
};
#define __ETHTOOL_LINK_MODE_LEGACY_MASK(base_name) \
#define NF_LOG_MACDECODE 0x20 /* Decode MAC header */
#define NF_LOG_MASK 0x2f
+#define NF_LOG_PREFIXLEN 128
+
#endif /* _NETFILTER_NF_LOG_H */
/**
* enum nft_rule_compat_attributes - nf_tables rule compat attributes
*
- * @NFTA_RULE_COMPAT_PROTO: numerice value of handled protocol (NLA_U32)
+ * @NFTA_RULE_COMPAT_PROTO: numeric value of handled protocol (NLA_U32)
* @NFTA_RULE_COMPAT_FLAGS: bitmask of enum nft_rule_compat_flags (NLA_U32)
*/
enum nft_rule_compat_attributes {
* enum nft_byteorder_ops - nf_tables byteorder operators
*
* @NFT_BYTEORDER_NTOH: network to host operator
- * @NFT_BYTEORDER_HTON: host to network opertaor
+ * @NFT_BYTEORDER_HTON: host to network operator
*/
enum nft_byteorder_ops {
NFT_BYTEORDER_NTOH,
__u8 type;
__u8 segments_left;
__u8 first_segment;
- __u8 flag_1;
- __u8 flag_2;
- __u8 reserved;
+ __u8 flags;
+ __u16 reserved;
struct in6_addr segments[0];
};
-#define SR6_FLAG1_CLEANUP (1 << 7)
#define SR6_FLAG1_PROTECTED (1 << 6)
#define SR6_FLAG1_OAM (1 << 5)
#define SR6_FLAG1_ALERT (1 << 4)
#define SR6_TLV_PADDING 4
#define SR6_TLV_HMAC 5
-#define sr_has_cleanup(srh) ((srh)->flag_1 & SR6_FLAG1_CLEANUP)
-#define sr_has_hmac(srh) ((srh)->flag_1 & SR6_FLAG1_HMAC)
+#define sr_has_hmac(srh) ((srh)->flags & SR6_FLAG1_HMAC)
struct sr6_tlv {
__u8 type;
header-y += ocrdma-abi.h
header-y += hns-abi.h
header-y += vmw_pvrdma-abi.h
+header-y += qedr-abi.h
* SOFTWARE.
*/
#ifndef CXGB3_ABI_USER_H
-#define CXBG3_ABI_USER_H
+#define CXGB3_ABI_USER_H
#include <linux/types.h>
#define IB_USER_VERBS_H
#include <linux/types.h>
-#include <rdma/ib_verbs.h>
/*
* Increment this value if any changes that break userspace ABI
};
enum {
- IB_USER_LEGACY_LAST_QP_ATTR_MASK = IB_QP_DEST_QPN
+ /*
+ * This value is equal to IB_QP_DEST_QPN.
+ */
+ IB_USER_LEGACY_LAST_QP_ATTR_MASK = 1ULL << 20,
};
enum {
- IB_USER_LAST_QP_ATTR_MASK = IB_QP_RATE_LIMIT
+ /*
+ * This value is equal to IB_QP_RATE_LIMIT.
+ */
+ IB_USER_LAST_QP_ATTR_MASK = 1ULL << 25,
};
struct ib_uverbs_ex_create_qp {
make them incompatible with the kernel you are running. If
unsure, say N.
+config MODULE_REL_CRCS
+ bool
+ depends on MODVERSIONS
+
config MODULE_SRCVERSION_ALL
bool "Source checksum for all modules"
help
*/
#include <linux/bpf.h>
#include <linux/err.h>
-#include <linux/vmalloc.h>
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/filter.h>
if (array_size >= U32_MAX - PAGE_SIZE)
return ERR_PTR(-ENOMEM);
-
/* allocate all map elements and zero-initialize them */
- array = kzalloc(array_size, GFP_USER | __GFP_NOWARN);
- if (!array) {
- array = vzalloc(array_size);
- if (!array)
- return ERR_PTR(-ENOMEM);
- }
+ array = bpf_map_area_alloc(array_size);
+ if (!array)
+ return ERR_PTR(-ENOMEM);
/* copy mandatory map attributes */
array->map.map_type = attr->map_type;
if (array_size >= U32_MAX - PAGE_SIZE ||
elem_size > PCPU_MIN_UNIT_SIZE || bpf_array_alloc_percpu(array)) {
- kvfree(array);
+ bpf_map_area_free(array);
return ERR_PTR(-ENOMEM);
}
out:
if (array->map.map_type == BPF_MAP_TYPE_PERCPU_ARRAY)
bpf_array_free_percpu(array);
- kvfree(array);
+ bpf_map_area_free(array);
}
static const struct bpf_map_ops array_ops = {
/* make sure it's empty */
for (i = 0; i < array->map.max_entries; i++)
BUG_ON(array->ptrs[i] != NULL);
- kvfree(array);
+
+ bpf_map_area_free(array);
}
static void *fd_array_map_lookup_elem(struct bpf_map *map, void *key)
#include <linux/bpf.h>
#include <linux/jhash.h>
#include <linux/filter.h>
-#include <linux/vmalloc.h>
#include "percpu_freelist.h"
#include "bpf_lru_list.h"
free_percpu(pptr);
}
free_elems:
- vfree(htab->elems);
+ bpf_map_area_free(htab->elems);
}
static struct htab_elem *prealloc_lru_pop(struct bpf_htab *htab, void *key,
{
int err = -ENOMEM, i;
- htab->elems = vzalloc(htab->elem_size * htab->map.max_entries);
+ htab->elems = bpf_map_area_alloc(htab->elem_size *
+ htab->map.max_entries);
if (!htab->elems)
return -ENOMEM;
goto free_htab;
err = -ENOMEM;
- htab->buckets = kmalloc_array(htab->n_buckets, sizeof(struct bucket),
- GFP_USER | __GFP_NOWARN);
-
- if (!htab->buckets) {
- htab->buckets = vmalloc(htab->n_buckets * sizeof(struct bucket));
- if (!htab->buckets)
- goto free_htab;
- }
+ htab->buckets = bpf_map_area_alloc(htab->n_buckets *
+ sizeof(struct bucket));
+ if (!htab->buckets)
+ goto free_htab;
for (i = 0; i < htab->n_buckets; i++) {
INIT_HLIST_HEAD(&htab->buckets[i].head);
free_extra_elems:
free_percpu(htab->extra_elems);
free_buckets:
- kvfree(htab->buckets);
+ bpf_map_area_free(htab->buckets);
free_htab:
kfree(htab);
return ERR_PTR(err);
prealloc_destroy(htab);
free_percpu(htab->extra_elems);
- kvfree(htab->buckets);
+ bpf_map_area_free(htab->buckets);
kfree(htab);
}
#include <linux/bpf.h>
#include <linux/jhash.h>
#include <linux/filter.h>
-#include <linux/vmalloc.h>
#include <linux/stacktrace.h>
#include <linux/perf_event.h>
#include "percpu_freelist.h"
u32 elem_size = sizeof(struct stack_map_bucket) + smap->map.value_size;
int err;
- smap->elems = vzalloc(elem_size * smap->map.max_entries);
+ smap->elems = bpf_map_area_alloc(elem_size * smap->map.max_entries);
if (!smap->elems)
return -ENOMEM;
return 0;
free_elems:
- vfree(smap->elems);
+ bpf_map_area_free(smap->elems);
return err;
}
if (cost >= U32_MAX - PAGE_SIZE)
return ERR_PTR(-E2BIG);
- smap = kzalloc(cost, GFP_USER | __GFP_NOWARN);
- if (!smap) {
- smap = vzalloc(cost);
- if (!smap)
- return ERR_PTR(-ENOMEM);
- }
+ smap = bpf_map_area_alloc(cost);
+ if (!smap)
+ return ERR_PTR(-ENOMEM);
err = -E2BIG;
cost += n_buckets * (value_size + sizeof(struct stack_map_bucket));
put_buffers:
put_callchain_buffers();
free_smap:
- kvfree(smap);
+ bpf_map_area_free(smap);
return ERR_PTR(err);
}
/* wait for bpf programs to complete before freeing stack map */
synchronize_rcu();
- vfree(smap->elems);
+ bpf_map_area_free(smap->elems);
pcpu_freelist_destroy(&smap->freelist);
- kvfree(smap);
+ bpf_map_area_free(smap);
put_callchain_buffers();
}
#include <linux/bpf.h>
#include <linux/syscalls.h>
#include <linux/slab.h>
+#include <linux/vmalloc.h>
+#include <linux/mmzone.h>
#include <linux/anon_inodes.h>
#include <linux/file.h>
#include <linux/license.h>
list_add(&tl->list_node, &bpf_map_types);
}
+void *bpf_map_area_alloc(size_t size)
+{
+ /* We definitely need __GFP_NORETRY, so OOM killer doesn't
+ * trigger under memory pressure as we really just want to
+ * fail instead.
+ */
+ const gfp_t flags = __GFP_NOWARN | __GFP_NORETRY | __GFP_ZERO;
+ void *area;
+
+ if (size <= (PAGE_SIZE << PAGE_ALLOC_COSTLY_ORDER)) {
+ area = kmalloc(size, GFP_USER | flags);
+ if (area != NULL)
+ return area;
+ }
+
+ return __vmalloc(size, GFP_KERNEL | __GFP_HIGHMEM | flags,
+ PAGE_KERNEL);
+}
+
+void bpf_map_area_free(void *area)
+{
+ kvfree(area);
+}
+
int bpf_map_precharge_memlock(u32 pages)
{
struct user_struct *user = get_current_user();
return ERR_PTR(err);
}
+/*
+ * The returned cgroup is fully initialized including its control mask, but
+ * it isn't associated with its kernfs_node and doesn't have the control
+ * mask applied.
+ */
static struct cgroup *cgroup_create(struct cgroup *parent)
{
struct cgroup_root *root = parent->root;
cgroup_propagate_control(cgrp);
- /* @cgrp doesn't have dir yet so the following will only create csses */
- ret = cgroup_apply_control_enable(cgrp);
- if (ret)
- goto out_destroy;
-
return cgrp;
out_cancel_ref:
out_free_cgrp:
kfree(cgrp);
return ERR_PTR(ret);
-out_destroy:
- cgroup_destroy_locked(cgrp);
- return ERR_PTR(ret);
}
static int cgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
static void
list_add_event(struct perf_event *event, struct perf_event_context *ctx)
{
-
lockdep_assert_held(&ctx->lock);
WARN_ON_ONCE(event->attach_state & PERF_ATTACH_CONTEXT);
{
struct perf_event *group_leader = event->group_leader, *pos;
+ lockdep_assert_held(&event->ctx->lock);
+
/*
* We can have double attach due to group movement in perf_event_open.
*/
struct perf_event *sibling, *tmp;
struct list_head *list = NULL;
+ lockdep_assert_held(&event->ctx->lock);
+
/*
* We can have double detach due to exit/hot-unplug + close.
*/
*/
static void perf_remove_from_context(struct perf_event *event, unsigned long flags)
{
- lockdep_assert_held(&event->ctx->mutex);
+ struct perf_event_context *ctx = event->ctx;
+
+ lockdep_assert_held(&ctx->mutex);
event_function_call(event, __perf_remove_from_context, (void *)flags);
+
+ /*
+ * The above event_function_call() can NO-OP when it hits
+ * TASK_TOMBSTONE. In that case we must already have been detached
+ * from the context (by perf_event_exit_event()) but the grouping
+ * might still be in-tact.
+ */
+ WARN_ON_ONCE(event->attach_state & PERF_ATTACH_CONTEXT);
+ if ((flags & DETACH_GROUP) &&
+ (event->attach_state & PERF_ATTACH_GROUP)) {
+ /*
+ * Since in that case we cannot possibly be scheduled, simply
+ * detach now.
+ */
+ raw_spin_lock_irq(&ctx->lock);
+ perf_group_detach(event);
+ raw_spin_unlock_irq(&ctx->lock);
+ }
}
/*
int ret;
};
-static int find_cpu_to_read(struct perf_event *event, int local_cpu)
+static int __perf_event_read_cpu(struct perf_event *event, int event_cpu)
{
- int event_cpu = event->oncpu;
u16 local_pkg, event_pkg;
if (event->group_caps & PERF_EV_CAP_READ_ACTIVE_PKG) {
- event_pkg = topology_physical_package_id(event_cpu);
- local_pkg = topology_physical_package_id(local_cpu);
+ int local_cpu = smp_processor_id();
+
+ event_pkg = topology_physical_package_id(event_cpu);
+ local_pkg = topology_physical_package_id(local_cpu);
if (event_pkg == local_pkg)
return local_cpu;
static int perf_event_read(struct perf_event *event, bool group)
{
- int ret = 0, cpu_to_read, local_cpu;
+ int event_cpu, ret = 0;
/*
* If event is enabled and currently active on a CPU, update the
.ret = 0,
};
- local_cpu = get_cpu();
- cpu_to_read = find_cpu_to_read(event, local_cpu);
- put_cpu();
+ event_cpu = READ_ONCE(event->oncpu);
+ if ((unsigned)event_cpu >= nr_cpu_ids)
+ return 0;
+
+ preempt_disable();
+ event_cpu = __perf_event_read_cpu(event, event_cpu);
/*
* Purposely ignore the smp_call_function_single() return
* value.
*
- * If event->oncpu isn't a valid CPU it means the event got
+ * If event_cpu isn't a valid CPU it means the event got
* scheduled out and that will have updated the event count.
*
* Therefore, either way, we'll have an up-to-date event count
* after this.
*/
- (void)smp_call_function_single(cpu_to_read, __perf_event_read, &data, 1);
+ (void)smp_call_function_single(event_cpu, __perf_event_read, &data, 1);
+ preempt_enable();
ret = data.ret;
} else if (event->state == PERF_EVENT_STATE_INACTIVE) {
struct perf_event_context *ctx = event->ctx;
char *buf = NULL;
char *name;
+ if (vma->vm_flags & VM_READ)
+ prot |= PROT_READ;
+ if (vma->vm_flags & VM_WRITE)
+ prot |= PROT_WRITE;
+ if (vma->vm_flags & VM_EXEC)
+ prot |= PROT_EXEC;
+
+ if (vma->vm_flags & VM_MAYSHARE)
+ flags = MAP_SHARED;
+ else
+ flags = MAP_PRIVATE;
+
+ if (vma->vm_flags & VM_DENYWRITE)
+ flags |= MAP_DENYWRITE;
+ if (vma->vm_flags & VM_MAYEXEC)
+ flags |= MAP_EXECUTABLE;
+ if (vma->vm_flags & VM_LOCKED)
+ flags |= MAP_LOCKED;
+ if (vma->vm_flags & VM_HUGETLB)
+ flags |= MAP_HUGETLB;
+
if (file) {
struct inode *inode;
dev_t dev;
maj = MAJOR(dev);
min = MINOR(dev);
- if (vma->vm_flags & VM_READ)
- prot |= PROT_READ;
- if (vma->vm_flags & VM_WRITE)
- prot |= PROT_WRITE;
- if (vma->vm_flags & VM_EXEC)
- prot |= PROT_EXEC;
-
- if (vma->vm_flags & VM_MAYSHARE)
- flags = MAP_SHARED;
- else
- flags = MAP_PRIVATE;
-
- if (vma->vm_flags & VM_DENYWRITE)
- flags |= MAP_DENYWRITE;
- if (vma->vm_flags & VM_MAYEXEC)
- flags |= MAP_EXECUTABLE;
- if (vma->vm_flags & VM_LOCKED)
- flags |= MAP_LOCKED;
- if (vma->vm_flags & VM_HUGETLB)
- flags |= MAP_HUGETLB;
-
goto got_name;
} else {
if (vma->vm_ops && vma->vm_ops->name) {
}
EXPORT_SYMBOL_GPL(irq_domain_free_irqs_parent);
+static void __irq_domain_activate_irq(struct irq_data *irq_data)
+{
+ if (irq_data && irq_data->domain) {
+ struct irq_domain *domain = irq_data->domain;
+
+ if (irq_data->parent_data)
+ __irq_domain_activate_irq(irq_data->parent_data);
+ if (domain->ops->activate)
+ domain->ops->activate(domain, irq_data);
+ }
+}
+
+static void __irq_domain_deactivate_irq(struct irq_data *irq_data)
+{
+ if (irq_data && irq_data->domain) {
+ struct irq_domain *domain = irq_data->domain;
+
+ if (domain->ops->deactivate)
+ domain->ops->deactivate(domain, irq_data);
+ if (irq_data->parent_data)
+ __irq_domain_deactivate_irq(irq_data->parent_data);
+ }
+}
+
/**
* irq_domain_activate_irq - Call domain_ops->activate recursively to activate
* interrupt
*/
void irq_domain_activate_irq(struct irq_data *irq_data)
{
- if (irq_data && irq_data->domain) {
- struct irq_domain *domain = irq_data->domain;
-
- if (irq_data->parent_data)
- irq_domain_activate_irq(irq_data->parent_data);
- if (domain->ops->activate)
- domain->ops->activate(domain, irq_data);
+ if (!irqd_is_activated(irq_data)) {
+ __irq_domain_activate_irq(irq_data);
+ irqd_set_activated(irq_data);
}
}
*/
void irq_domain_deactivate_irq(struct irq_data *irq_data)
{
- if (irq_data && irq_data->domain) {
- struct irq_domain *domain = irq_data->domain;
-
- if (domain->ops->deactivate)
- domain->ops->deactivate(domain, irq_data);
- if (irq_data->parent_data)
- irq_domain_deactivate_irq(irq_data->parent_data);
+ if (irqd_is_activated(irq_data)) {
+ __irq_domain_deactivate_irq(irq_data);
+ irqd_clr_activated(irq_data);
}
}
extern const struct kernel_symbol __stop___ksymtab_gpl[];
extern const struct kernel_symbol __start___ksymtab_gpl_future[];
extern const struct kernel_symbol __stop___ksymtab_gpl_future[];
-extern const unsigned long __start___kcrctab[];
-extern const unsigned long __start___kcrctab_gpl[];
-extern const unsigned long __start___kcrctab_gpl_future[];
+extern const s32 __start___kcrctab[];
+extern const s32 __start___kcrctab_gpl[];
+extern const s32 __start___kcrctab_gpl_future[];
#ifdef CONFIG_UNUSED_SYMBOLS
extern const struct kernel_symbol __start___ksymtab_unused[];
extern const struct kernel_symbol __stop___ksymtab_unused[];
extern const struct kernel_symbol __start___ksymtab_unused_gpl[];
extern const struct kernel_symbol __stop___ksymtab_unused_gpl[];
-extern const unsigned long __start___kcrctab_unused[];
-extern const unsigned long __start___kcrctab_unused_gpl[];
+extern const s32 __start___kcrctab_unused[];
+extern const s32 __start___kcrctab_unused_gpl[];
#endif
#ifndef CONFIG_MODVERSIONS
/* Output */
struct module *owner;
- const unsigned long *crc;
+ const s32 *crc;
const struct kernel_symbol *sym;
};
* (optional) module which owns it. Needs preempt disabled or module_mutex. */
const struct kernel_symbol *find_symbol(const char *name,
struct module **owner,
- const unsigned long **crc,
+ const s32 **crc,
bool gplok,
bool warn)
{
}
#ifdef CONFIG_MODVERSIONS
-/* If the arch applies (non-zero) relocations to kernel kcrctab, unapply it. */
-static unsigned long maybe_relocated(unsigned long crc,
- const struct module *crc_owner)
+
+static u32 resolve_rel_crc(const s32 *crc)
{
-#ifdef ARCH_RELOCATES_KCRCTAB
- if (crc_owner == NULL)
- return crc - (unsigned long)reloc_start;
-#endif
- return crc;
+ return *(u32 *)((void *)crc + *crc);
}
static int check_version(Elf_Shdr *sechdrs,
unsigned int versindex,
const char *symname,
struct module *mod,
- const unsigned long *crc,
- const struct module *crc_owner)
+ const s32 *crc)
{
unsigned int i, num_versions;
struct modversion_info *versions;
/ sizeof(struct modversion_info);
for (i = 0; i < num_versions; i++) {
+ u32 crcval;
+
if (strcmp(versions[i].name, symname) != 0)
continue;
- if (versions[i].crc == maybe_relocated(*crc, crc_owner))
+ if (IS_ENABLED(CONFIG_MODULE_REL_CRCS))
+ crcval = resolve_rel_crc(crc);
+ else
+ crcval = *crc;
+ if (versions[i].crc == crcval)
return 1;
- pr_debug("Found checksum %lX vs module %lX\n",
- maybe_relocated(*crc, crc_owner), versions[i].crc);
+ pr_debug("Found checksum %X vs module %lX\n",
+ crcval, versions[i].crc);
goto bad_version;
}
unsigned int versindex,
struct module *mod)
{
- const unsigned long *crc;
+ const s32 *crc;
/*
* Since this should be found in kernel (which can't be removed), no
}
preempt_enable();
return check_version(sechdrs, versindex,
- VMLINUX_SYMBOL_STR(module_layout), mod, crc,
- NULL);
+ VMLINUX_SYMBOL_STR(module_layout), mod, crc);
}
/* First part is kernel version, which we ignore if module has crcs. */
unsigned int versindex,
const char *symname,
struct module *mod,
- const unsigned long *crc,
- const struct module *crc_owner)
+ const s32 *crc)
{
return 1;
}
{
struct module *owner;
const struct kernel_symbol *sym;
- const unsigned long *crc;
+ const s32 *crc;
int err;
/*
if (!sym)
goto unlock;
- if (!check_version(info->sechdrs, info->index.vers, name, mod, crc,
- owner)) {
+ if (!check_version(info->sechdrs, info->index.vers, name, mod, crc)) {
sym = ERR_PTR(-EINVAL);
goto getname;
}
* Delay timeout seconds before rebooting the machine.
* We can't use the "normal" timers since we just panicked.
*/
- pr_emerg("Rebooting in %d seconds..", panic_timeout);
+ pr_emerg("Rebooting in %d seconds..\n", panic_timeout);
for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
touch_nmi_watchdog();
const char *mem_sleep_states[PM_SUSPEND_MAX];
suspend_state_t mem_sleep_current = PM_SUSPEND_FREEZE;
-suspend_state_t mem_sleep_default = PM_SUSPEND_MAX;
+static suspend_state_t mem_sleep_default = PM_SUSPEND_MEM;
unsigned int pm_suspend_global_flags;
EXPORT_SYMBOL_GPL(pm_suspend_global_flags);
}
if (valid_state(PM_SUSPEND_MEM)) {
mem_sleep_states[PM_SUSPEND_MEM] = mem_sleep_labels[PM_SUSPEND_MEM];
- if (mem_sleep_default >= PM_SUSPEND_MEM)
+ if (mem_sleep_default == PM_SUSPEND_MEM)
mem_sleep_current = PM_SUSPEND_MEM;
}
if (WARN_ON(!trace->entries))
return;
- for (i = 0; i < trace->nr_entries; i++) {
- printk("%*c", 1 + spaces, ' ');
- print_ip_sym(trace->entries[i]);
- }
+ for (i = 0; i < trace->nr_entries; i++)
+ printk("%*c%pS\n", 1 + spaces, ' ', (void *)trace->entries[i]);
}
EXPORT_SYMBOL_GPL(print_stack_trace);
struct stack_trace *trace, int spaces)
{
int i;
- unsigned long ip;
int generated;
int total = 0;
return 0;
for (i = 0; i < trace->nr_entries; i++) {
- ip = trace->entries[i];
- generated = snprintf(buf, size, "%*c[<%p>] %pS\n",
- 1 + spaces, ' ', (void *) ip, (void *) ip);
+ generated = snprintf(buf, size, "%*c%pS\n", 1 + spaces, ' ',
+ (void *)trace->entries[i]);
total += generated;
break;
if (neg)
continue;
+ val = convmul * val / convdiv;
if ((min && val < *min) || (max && val > *max))
continue;
*i = val;
*/
if (delta == 0) {
tick_nohz_restart(ts, now);
+ /*
+ * Make sure next tick stop doesn't get fooled by past
+ * clock deadline
+ */
+ ts->next_tick = 0;
goto out;
}
}
static struct cpumask save_cpumask;
static bool disable_migrate;
-static void move_to_next_cpu(void)
+static void move_to_next_cpu(bool initmask)
{
static struct cpumask *current_mask;
int next_cpu;
return;
/* Just pick the first CPU on first iteration */
- if (!current_mask) {
+ if (initmask) {
current_mask = &save_cpumask;
get_online_cpus();
cpumask_and(current_mask, cpu_online_mask, tracing_buffer_mask);
static int kthread_fn(void *data)
{
u64 interval;
+ bool initmask = true;
while (!kthread_should_stop()) {
- move_to_next_cpu();
+ move_to_next_cpu(initmask);
+ initmask = false;
local_irq_disable();
get_sample();
return a1 + a2 + a3 + a4 + a5 + a6;
}
-static struct __init trace_event_file *
+static __init struct trace_event_file *
find_trace_probe_file(struct trace_kprobe *tk, struct trace_array *tr)
{
struct trace_event_file *file;
struct hlist_head *hashent = ucounts_hashentry(ns, uid);
struct ucounts *ucounts, *new;
- spin_lock(&ucounts_lock);
+ spin_lock_irq(&ucounts_lock);
ucounts = find_ucounts(ns, uid, hashent);
if (!ucounts) {
- spin_unlock(&ucounts_lock);
+ spin_unlock_irq(&ucounts_lock);
new = kzalloc(sizeof(*new), GFP_KERNEL);
if (!new)
new->uid = uid;
atomic_set(&new->count, 0);
- spin_lock(&ucounts_lock);
+ spin_lock_irq(&ucounts_lock);
ucounts = find_ucounts(ns, uid, hashent);
if (ucounts) {
kfree(new);
}
if (!atomic_add_unless(&ucounts->count, 1, INT_MAX))
ucounts = NULL;
- spin_unlock(&ucounts_lock);
+ spin_unlock_irq(&ucounts_lock);
return ucounts;
}
static void put_ucounts(struct ucounts *ucounts)
{
+ unsigned long flags;
+
if (atomic_dec_and_test(&ucounts->count)) {
- spin_lock(&ucounts_lock);
+ spin_lock_irqsave(&ucounts_lock, flags);
hlist_del_init(&ucounts->node);
- spin_unlock(&ucounts_lock);
+ spin_unlock_irqrestore(&ucounts_lock, flags);
kfree(ucounts);
}
* properly.
*/
user_header = register_sysctl("user", empty);
+ kmemleak_ignore(user_header);
BUG_ON(!user_header);
BUG_ON(!setup_userns_sysctls(&init_user_ns));
#endif
return 0;
}
subsys_initcall(user_namespace_sysctl_init);
-
-
#define for_each_watchdog_cpu(cpu) \
for_each_cpu_and((cpu), cpu_online_mask, &watchdog_cpumask)
+atomic_t watchdog_park_in_progress = ATOMIC_INIT(0);
+
/*
* The 'watchdog_running' variable is set to 1 when the watchdog threads
* are registered/started and is set to 0 when the watchdog threads are
int duration;
int softlockup_all_cpu_backtrace = sysctl_softlockup_all_cpu_backtrace;
+ if (atomic_read(&watchdog_park_in_progress) != 0)
+ return HRTIMER_NORESTART;
+
/* kick the hardlockup detector */
watchdog_interrupt_count();
{
int cpu, ret = 0;
+ atomic_set(&watchdog_park_in_progress, 1);
+
for_each_watchdog_cpu(cpu) {
ret = kthread_park(per_cpu(softlockup_watchdog, cpu));
if (ret)
break;
}
+ atomic_set(&watchdog_park_in_progress, 0);
+
return ret;
}
/* Ensure the watchdog never gets throttled */
event->hw.interrupts = 0;
+ if (atomic_read(&watchdog_park_in_progress) != 0)
+ return;
+
if (__this_cpu_read(watchdog_nmi_touch) == true) {
__this_cpu_write(watchdog_nmi_touch, false);
return;
return err;
}
-EXPORT_SYMBOL_GPL(ioremap_page_range);
struct radix_tree_node *old = child;
offset = child->offset + 1;
child = child->parent;
- WARN_ON_ONCE(!list_empty(&node->private_list));
+ WARN_ON_ONCE(!list_empty(&old->private_list));
radix_tree_node_free(old);
if (old == entry_to_node(node))
return;
cond_resched();
find_page:
+ if (fatal_signal_pending(current)) {
+ error = -EINTR;
+ goto out;
+ }
+
page = find_get_page(mapping, index);
if (!page) {
page_cache_sync_readahead(mapping,
assert_spin_locked(pmd_lockptr(mm, pmd));
+ /*
+ * When we COW a devmap PMD entry, we split it into PTEs, so we should
+ * not be in this function with `flags & FOLL_COW` set.
+ */
+ WARN_ONCE(flags & FOLL_COW, "mm: In follow_devmap_pmd with FOLL_COW set");
+
if (flags & FOLL_WRITE && !pmd_write(*pmd))
return NULL;
return ret;
}
+/*
+ * FOLL_FORCE can write to even unwritable pmd's, but only
+ * after we've gone through a COW cycle and they are dirty.
+ */
+static inline bool can_follow_write_pmd(pmd_t pmd, unsigned int flags)
+{
+ return pmd_write(pmd) ||
+ ((flags & FOLL_FORCE) && (flags & FOLL_COW) && pmd_dirty(pmd));
+}
+
struct page *follow_trans_huge_pmd(struct vm_area_struct *vma,
unsigned long addr,
pmd_t *pmd,
assert_spin_locked(pmd_lockptr(mm, pmd));
- if (flags & FOLL_WRITE && !pmd_write(*pmd))
+ if (flags & FOLL_WRITE && !can_follow_write_pmd(*pmd, flags))
goto out;
/* Avoid dumping huge zero page */
*
*/
+#include <linux/ftrace.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/printk.h>
if (likely(!kasan_report_enabled()))
return;
+ disable_trace_on_warning();
+
info.access_addr = (void *)addr;
info.access_size = size;
info.is_write = is_write;
return ret;
}
- /* Try charges one by one with reclaim */
+ /* Try charges one by one with reclaim, but do not retry */
while (count--) {
- ret = try_charge(mc.to, GFP_KERNEL & ~__GFP_NORETRY, 1);
+ ret = try_charge(mc.to, GFP_KERNEL | __GFP_NORETRY, 1);
if (ret)
return ret;
mc.precharge++;
node_set_state(node, N_MEMORY);
}
-int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
- enum zone_type target)
+bool zone_can_shift(unsigned long pfn, unsigned long nr_pages,
+ enum zone_type target, int *zone_shift)
{
struct zone *zone = page_zone(pfn_to_page(pfn));
enum zone_type idx = zone_idx(zone);
int i;
+ *zone_shift = 0;
+
if (idx < target) {
/* pages must be at end of current zone */
if (pfn + nr_pages != zone_end_pfn(zone))
- return 0;
+ return false;
/* no zones in use between current zone and target */
for (i = idx + 1; i < target; i++)
if (zone_is_initialized(zone - idx + i))
- return 0;
+ return false;
}
if (target < idx) {
/* pages must be at beginning of current zone */
if (pfn != zone->zone_start_pfn)
- return 0;
+ return false;
/* no zones in use between current zone and target */
for (i = target + 1; i < idx; i++)
if (zone_is_initialized(zone - idx + i))
- return 0;
+ return false;
}
- return target - idx;
+ *zone_shift = target - idx;
+ return true;
}
/* Must be protected by mem_hotplug_begin() */
!can_online_high_movable(zone))
return -EINVAL;
- if (online_type == MMOP_ONLINE_KERNEL)
- zone_shift = zone_can_shift(pfn, nr_pages, ZONE_NORMAL);
- else if (online_type == MMOP_ONLINE_MOVABLE)
- zone_shift = zone_can_shift(pfn, nr_pages, ZONE_MOVABLE);
+ if (online_type == MMOP_ONLINE_KERNEL) {
+ if (!zone_can_shift(pfn, nr_pages, ZONE_NORMAL, &zone_shift))
+ return -EINVAL;
+ } else if (online_type == MMOP_ONLINE_MOVABLE) {
+ if (!zone_can_shift(pfn, nr_pages, ZONE_MOVABLE, &zone_shift))
+ return -EINVAL;
+ }
zone = move_pfn_range(zone_shift, pfn, pfn + nr_pages);
if (!zone)
}
/*
- * Confirm all pages in a range [start, end) is belongs to the same zone.
+ * Confirm all pages in a range [start, end) belong to the same zone.
+ * When true, return its valid [start, end).
*/
-int test_pages_in_a_zone(unsigned long start_pfn, unsigned long end_pfn)
+int test_pages_in_a_zone(unsigned long start_pfn, unsigned long end_pfn,
+ unsigned long *valid_start, unsigned long *valid_end)
{
unsigned long pfn, sec_end_pfn;
+ unsigned long start, end;
struct zone *zone = NULL;
struct page *page;
int i;
- for (pfn = start_pfn, sec_end_pfn = SECTION_ALIGN_UP(start_pfn);
+ for (pfn = start_pfn, sec_end_pfn = SECTION_ALIGN_UP(start_pfn + 1);
pfn < end_pfn;
- pfn = sec_end_pfn + 1, sec_end_pfn += PAGES_PER_SECTION) {
+ pfn = sec_end_pfn, sec_end_pfn += PAGES_PER_SECTION) {
/* Make sure the memory section is present first */
if (!present_section_nr(pfn_to_section_nr(pfn)))
continue;
page = pfn_to_page(pfn + i);
if (zone && page_zone(page) != zone)
return 0;
+ if (!zone)
+ start = pfn + i;
zone = page_zone(page);
+ end = pfn + MAX_ORDER_NR_PAGES;
}
}
- return 1;
+
+ if (zone) {
+ *valid_start = start;
+ *valid_end = end;
+ return 1;
+ } else {
+ return 0;
+ }
}
/*
long offlined_pages;
int ret, drain, retry_max, node;
unsigned long flags;
+ unsigned long valid_start, valid_end;
struct zone *zone;
struct memory_notify arg;
return -EINVAL;
/* This makes hotplug much easier...and readable.
we assume this for now. .*/
- if (!test_pages_in_a_zone(start_pfn, end_pfn))
+ if (!test_pages_in_a_zone(start_pfn, end_pfn, &valid_start, &valid_end))
return -EINVAL;
- zone = page_zone(pfn_to_page(start_pfn));
+ zone = page_zone(pfn_to_page(valid_start));
node = zone_to_nid(zone);
nr_pages = end_pfn - start_pfn;
nmask = policy_nodemask(gfp, pol);
zl = policy_zonelist(gfp, pol, node);
- mpol_cond_put(pol);
page = __alloc_pages_nodemask(gfp, order, zl, nmask);
+ mpol_cond_put(pol);
out:
if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
goto retry_cpuset;
struct page *page = NULL;
unsigned int alloc_flags;
unsigned long did_some_progress;
- enum compact_priority compact_priority = DEF_COMPACT_PRIORITY;
+ enum compact_priority compact_priority;
enum compact_result compact_result;
- int compaction_retries = 0;
- int no_progress_loops = 0;
+ int compaction_retries;
+ int no_progress_loops;
unsigned long alloc_start = jiffies;
unsigned int stall_timeout = 10 * HZ;
+ unsigned int cpuset_mems_cookie;
/*
* In the slowpath, we sanity check order to avoid ever trying to
(__GFP_ATOMIC|__GFP_DIRECT_RECLAIM)))
gfp_mask &= ~__GFP_ATOMIC;
+retry_cpuset:
+ compaction_retries = 0;
+ no_progress_loops = 0;
+ compact_priority = DEF_COMPACT_PRIORITY;
+ cpuset_mems_cookie = read_mems_allowed_begin();
+ /*
+ * We need to recalculate the starting point for the zonelist iterator
+ * because we might have used different nodemask in the fast path, or
+ * there was a cpuset modification and we are retrying - otherwise we
+ * could end up iterating over non-eligible zones endlessly.
+ */
+ ac->preferred_zoneref = first_zones_zonelist(ac->zonelist,
+ ac->high_zoneidx, ac->nodemask);
+ if (!ac->preferred_zoneref->zone)
+ goto nopage;
+
+
/*
* The fast path uses conservative alloc_flags to succeed only until
* kswapd needs to be woken up, and to avoid the cost of setting up
&compaction_retries))
goto retry;
+ /*
+ * It's possible we raced with cpuset update so the OOM would be
+ * premature (see below the nopage: label for full explanation).
+ */
+ if (read_mems_allowed_retry(cpuset_mems_cookie))
+ goto retry_cpuset;
+
/* Reclaim has failed us, start killing things */
page = __alloc_pages_may_oom(gfp_mask, order, ac, &did_some_progress);
if (page)
}
nopage:
+ /*
+ * When updating a task's mems_allowed or mempolicy nodemask, it is
+ * possible to race with parallel threads in such a way that our
+ * allocation can fail while the mask is being updated. If we are about
+ * to fail, check if the cpuset changed during allocation and if so,
+ * retry.
+ */
+ if (read_mems_allowed_retry(cpuset_mems_cookie))
+ goto retry_cpuset;
+
warn_alloc(gfp_mask,
"page allocation failure: order:%u", order);
got_pg:
struct zonelist *zonelist, nodemask_t *nodemask)
{
struct page *page;
- unsigned int cpuset_mems_cookie;
unsigned int alloc_flags = ALLOC_WMARK_LOW;
gfp_t alloc_mask = gfp_mask; /* The gfp_t that was actually used for allocation */
struct alloc_context ac = {
if (IS_ENABLED(CONFIG_CMA) && ac.migratetype == MIGRATE_MOVABLE)
alloc_flags |= ALLOC_CMA;
-retry_cpuset:
- cpuset_mems_cookie = read_mems_allowed_begin();
-
/* Dirty zone balancing only done in the fast path */
ac.spread_dirty_pages = (gfp_mask & __GFP_WRITE);
*/
ac.preferred_zoneref = first_zones_zonelist(ac.zonelist,
ac.high_zoneidx, ac.nodemask);
- if (!ac.preferred_zoneref) {
+ if (!ac.preferred_zoneref->zone) {
page = NULL;
+ /*
+ * This might be due to race with cpuset_current_mems_allowed
+ * update, so make sure we retry with original nodemask in the
+ * slow path.
+ */
goto no_zone;
}
if (likely(page))
goto out;
+no_zone:
/*
* Runtime PM, block IO and its error handling path can deadlock
* because I/O on the device might not complete.
* Restore the original nodemask if it was potentially replaced with
* &cpuset_current_mems_allowed to optimize the fast-path attempt.
*/
- if (cpusets_enabled())
+ if (unlikely(ac.nodemask != nodemask))
ac.nodemask = nodemask;
- page = __alloc_pages_slowpath(alloc_mask, order, &ac);
-no_zone:
- /*
- * When updating a task's mems_allowed, it is possible to race with
- * parallel threads in such a way that an allocation can fail while
- * the mask is being updated. If a page allocation is about to fail,
- * check if the cpuset changed during allocation and if so, retry.
- */
- if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie))) {
- alloc_mask = gfp_mask;
- goto retry_cpuset;
- }
+ page = __alloc_pages_slowpath(alloc_mask, order, &ac);
out:
if (memcg_kmem_enabled() && (gfp_mask & __GFP_ACCOUNT) && page &&
.zone = page_zone(pfn_to_page(start)),
.mode = MIGRATE_SYNC,
.ignore_skip_hint = true,
+ .gfp_mask = GFP_KERNEL,
};
INIT_LIST_HEAD(&cc.migratepages);
struct shrink_control *sc, unsigned long nr_to_split)
{
LIST_HEAD(list), *pos, *next;
+ LIST_HEAD(to_remove);
struct inode *inode;
struct shmem_inode_info *info;
struct page *page;
/* Check if there's anything to gain */
if (round_up(inode->i_size, PAGE_SIZE) ==
round_up(inode->i_size, HPAGE_PMD_SIZE)) {
- list_del_init(&info->shrinklist);
+ list_move(&info->shrinklist, &to_remove);
removed++;
- iput(inode);
goto next;
}
}
spin_unlock(&sbinfo->shrinklist_lock);
+ list_for_each_safe(pos, next, &to_remove) {
+ info = list_entry(pos, struct shmem_inode_info, shrinklist);
+ inode = &info->vfs_inode;
+ list_del_init(&info->shrinklist);
+ iput(inode);
+ }
+
list_for_each_safe(pos, next, &list) {
int ret;
return 1;
}
-static void print_section(char *text, u8 *addr, unsigned int length)
+static void print_section(char *level, char *text, u8 *addr,
+ unsigned int length)
{
metadata_access_enable();
- print_hex_dump(KERN_ERR, text, DUMP_PREFIX_ADDRESS, 16, 1, addr,
+ print_hex_dump(level, text, DUMP_PREFIX_ADDRESS, 16, 1, addr,
length, 1);
metadata_access_disable();
}
p, p - addr, get_freepointer(s, p));
if (s->flags & SLAB_RED_ZONE)
- print_section("Redzone ", p - s->red_left_pad, s->red_left_pad);
+ print_section(KERN_ERR, "Redzone ", p - s->red_left_pad,
+ s->red_left_pad);
else if (p > addr + 16)
- print_section("Bytes b4 ", p - 16, 16);
+ print_section(KERN_ERR, "Bytes b4 ", p - 16, 16);
- print_section("Object ", p, min_t(unsigned long, s->object_size,
- PAGE_SIZE));
+ print_section(KERN_ERR, "Object ", p,
+ min_t(unsigned long, s->object_size, PAGE_SIZE));
if (s->flags & SLAB_RED_ZONE)
- print_section("Redzone ", p + s->object_size,
+ print_section(KERN_ERR, "Redzone ", p + s->object_size,
s->inuse - s->object_size);
if (s->offset)
if (off != size_from_object(s))
/* Beginning of the filler is the free pointer */
- print_section("Padding ", p + off, size_from_object(s) - off);
+ print_section(KERN_ERR, "Padding ", p + off,
+ size_from_object(s) - off);
dump_stack();
}
end--;
slab_err(s, page, "Padding overwritten. 0x%p-0x%p", fault, end - 1);
- print_section("Padding ", end - remainder, remainder);
+ print_section(KERN_ERR, "Padding ", end - remainder, remainder);
restore_bytes(s, "slab padding", POISON_INUSE, end - remainder, end);
return 0;
page->freelist);
if (!alloc)
- print_section("Object ", (void *)object,
+ print_section(KERN_INFO, "Object ", (void *)object,
s->object_size);
dump_stack();
int err;
unsigned long i, count = oo_objects(s->oo);
+ /* Bailout if already initialised */
+ if (s->random_seq)
+ return 0;
+
err = cache_random_seq_create(s, count, GFP_KERNEL);
if (err) {
pr_err("SLUB: Unable to initialize free list for %s\n",
/* Enable/disable zswap (disabled by default) */
static bool zswap_enabled;
-module_param_named(enabled, zswap_enabled, bool, 0644);
+static int zswap_enabled_param_set(const char *,
+ const struct kernel_param *);
+static struct kernel_param_ops zswap_enabled_param_ops = {
+ .set = zswap_enabled_param_set,
+ .get = param_get_bool,
+};
+module_param_cb(enabled, &zswap_enabled_param_ops, &zswap_enabled, 0644);
/* Crypto compressor to use */
#define ZSWAP_COMPRESSOR_DEFAULT "lzo"
/* used by param callback function */
static bool zswap_init_started;
+/* fatal error during init */
+static bool zswap_init_failed;
+
/*********************************
* helpers and fwd declarations
**********************************/
char *s = strstrip((char *)val);
int ret;
+ if (zswap_init_failed) {
+ pr_err("can't set param, initialization failed\n");
+ return -ENODEV;
+ }
+
/* no change required */
if (!strcmp(s, *(char **)kp->arg))
return 0;
return __zswap_param_set(val, kp, NULL, zswap_compressor);
}
+static int zswap_enabled_param_set(const char *val,
+ const struct kernel_param *kp)
+{
+ if (zswap_init_failed) {
+ pr_err("can't enable, initialization failed\n");
+ return -ENODEV;
+ }
+
+ return param_set_bool(val, kp);
+}
+
/*********************************
* writeback code
**********************************/
dstmem_fail:
zswap_entry_cache_destroy();
cache_fail:
+ /* if built-in, we aren't unloaded on failure; don't allow use */
+ zswap_init_failed = true;
+ zswap_enabled = false;
return -ENOMEM;
}
/* must be late so crypto has time to come up */
primary_if = batadv_primary_if_get_selected(bat_priv);
if (!primary_if) {
ret = -EINVAL;
- goto put_primary_if;
+ goto free_skb;
}
/* Create one header to be copied to all fragments */
skb_fragment = batadv_frag_create(skb, &frag_header, mtu);
if (!skb_fragment) {
ret = -ENOMEM;
- goto free_skb;
+ goto put_primary_if;
}
batadv_inc_counter(bat_priv, BATADV_CNT_FRAG_TX);
ret = batadv_send_unicast_skb(skb_fragment, neigh_node);
if (ret != NET_XMIT_SUCCESS) {
ret = NET_XMIT_DROP;
- goto free_skb;
+ goto put_primary_if;
}
frag_header.no++;
/* The initial check in this function should cover this case */
if (frag_header.no == BATADV_FRAG_MAX_FRAGMENTS - 1) {
ret = -EINVAL;
- goto free_skb;
+ goto put_primary_if;
}
}
if (batadv_skb_head_push(skb, header_size) < 0 ||
pskb_expand_head(skb, header_size + ETH_HLEN, 0, GFP_ATOMIC) < 0) {
ret = -ENOMEM;
- goto free_skb;
+ goto put_primary_if;
}
memcpy(skb->data, &frag_header, header_size);
return 0;
}
-static int br_dev_newlink(struct net *src_net, struct net_device *dev,
- struct nlattr *tb[], struct nlattr *data[])
-{
- struct net_bridge *br = netdev_priv(dev);
-
- if (tb[IFLA_ADDRESS]) {
- spin_lock_bh(&br->lock);
- br_stp_change_bridge_id(br, nla_data(tb[IFLA_ADDRESS]));
- spin_unlock_bh(&br->lock);
- }
-
- return register_netdevice(dev);
-}
-
static int br_port_slave_changelink(struct net_device *brdev,
struct net_device *dev,
struct nlattr *tb[],
return 0;
}
+static int br_dev_newlink(struct net *src_net, struct net_device *dev,
+ struct nlattr *tb[], struct nlattr *data[])
+{
+ struct net_bridge *br = netdev_priv(dev);
+ int err;
+
+ if (tb[IFLA_ADDRESS]) {
+ spin_lock_bh(&br->lock);
+ br_stp_change_bridge_id(br, nla_data(tb[IFLA_ADDRESS]));
+ spin_unlock_bh(&br->lock);
+ }
+
+ err = br_changelink(dev, tb, data);
+ if (err)
+ return err;
+
+ return register_netdevice(dev);
+}
+
static size_t br_get_size(const struct net_device *brdev)
{
return nla_total_size(sizeof(u32)) + /* IFLA_BR_FORWARD_DELAY */
* @func: callback function on filter match
* @data: returned parameter for callback function
* @ident: string for calling module identification
+ * @sk: socket pointer (might be NULL)
*
* Description:
* Invokes the callback function with the received sk_buff and the given
*/
int can_rx_register(struct net_device *dev, canid_t can_id, canid_t mask,
void (*func)(struct sk_buff *, void *), void *data,
- char *ident)
+ char *ident, struct sock *sk)
{
struct receiver *r;
struct hlist_head *rl;
r->func = func;
r->data = data;
r->ident = ident;
+ r->sk = sk;
hlist_add_head_rcu(&r->list, rl);
d->entries++;
static void can_rx_delete_receiver(struct rcu_head *rp)
{
struct receiver *r = container_of(rp, struct receiver, rcu);
+ struct sock *sk = r->sk;
kmem_cache_free(rcv_cache, r);
+ if (sk)
+ sock_put(sk);
}
/**
spin_unlock(&can_rcvlists_lock);
/* schedule the receiver item for deletion */
- if (r)
+ if (r) {
+ if (r->sk)
+ sock_hold(r->sk);
call_rcu(&r->rcu, can_rx_delete_receiver);
+ }
}
EXPORT_SYMBOL(can_rx_unregister);
struct receiver {
struct hlist_node list;
- struct rcu_head rcu;
canid_t can_id;
canid_t mask;
unsigned long matches;
void (*func)(struct sk_buff *, void *);
void *data;
char *ident;
+ struct sock *sk;
+ struct rcu_head rcu;
};
#define CAN_SFF_RCV_ARRAY_SZ (1 << CAN_SFF_ID_BITS)
static void bcm_remove_op(struct bcm_op *op)
{
- hrtimer_cancel(&op->timer);
- hrtimer_cancel(&op->thrtimer);
-
- if (op->tsklet.func)
- tasklet_kill(&op->tsklet);
+ if (op->tsklet.func) {
+ while (test_bit(TASKLET_STATE_SCHED, &op->tsklet.state) ||
+ test_bit(TASKLET_STATE_RUN, &op->tsklet.state) ||
+ hrtimer_active(&op->timer)) {
+ hrtimer_cancel(&op->timer);
+ tasklet_kill(&op->tsklet);
+ }
+ }
- if (op->thrtsklet.func)
- tasklet_kill(&op->thrtsklet);
+ if (op->thrtsklet.func) {
+ while (test_bit(TASKLET_STATE_SCHED, &op->thrtsklet.state) ||
+ test_bit(TASKLET_STATE_RUN, &op->thrtsklet.state) ||
+ hrtimer_active(&op->thrtimer)) {
+ hrtimer_cancel(&op->thrtimer);
+ tasklet_kill(&op->thrtsklet);
+ }
+ }
if ((op->frames) && (op->frames != &op->sframe))
kfree(op->frames);
err = can_rx_register(dev, op->can_id,
REGMASK(op->can_id),
bcm_rx_handler, op,
- "bcm");
+ "bcm", sk);
op->rx_reg_dev = dev;
dev_put(dev);
} else
err = can_rx_register(NULL, op->can_id,
REGMASK(op->can_id),
- bcm_rx_handler, op, "bcm");
+ bcm_rx_handler, op, "bcm", sk);
if (err) {
/* this bcm rx op is broken -> remove it */
list_del(&op->list);
{
return can_rx_register(gwj->src.dev, gwj->ccgw.filter.can_id,
gwj->ccgw.filter.can_mask, can_can_gw_rcv,
- gwj, "gw");
+ gwj, "gw", NULL);
}
static inline void cgw_unregister_filter(struct cgw_job *gwj)
for (i = 0; i < count; i++) {
err = can_rx_register(dev, filter[i].can_id,
filter[i].can_mask,
- raw_rcv, sk, "raw");
+ raw_rcv, sk, "raw", sk);
if (err) {
/* clean up successfully registered filters */
while (--i >= 0)
if (err_mask)
err = can_rx_register(dev, 0, err_mask | CAN_ERR_FLAG,
- raw_rcv, sk, "raw");
+ raw_rcv, sk, "raw", sk);
return err;
}
EXPORT_SYMBOL(__skb_free_datagram_locked);
int __sk_queue_drop_skb(struct sock *sk, struct sk_buff *skb,
- unsigned int flags)
+ unsigned int flags,
+ void (*destructor)(struct sock *sk,
+ struct sk_buff *skb))
{
int err = 0;
if (skb == skb_peek(&sk->sk_receive_queue)) {
__skb_unlink(skb, &sk->sk_receive_queue);
atomic_dec(&skb->users);
+ if (destructor)
+ destructor(sk, skb);
err = 0;
}
spin_unlock_bh(&sk->sk_receive_queue.lock);
int skb_kill_datagram(struct sock *sk, struct sk_buff *skb, unsigned int flags)
{
- int err = __sk_queue_drop_skb(sk, skb, flags);
+ int err = __sk_queue_drop_skb(sk, skb, flags, NULL);
kfree_skb(skb);
sk_mem_reclaim_partial(sk);
static struct static_key netstamp_needed __read_mostly;
#ifdef HAVE_JUMP_LABEL
-/* We are not allowed to call static_key_slow_dec() from irq context
- * If net_disable_timestamp() is called from irq context, defer the
- * static_key_slow_dec() calls.
- */
static atomic_t netstamp_needed_deferred;
-#endif
-
-void net_enable_timestamp(void)
+static void netstamp_clear(struct work_struct *work)
{
-#ifdef HAVE_JUMP_LABEL
int deferred = atomic_xchg(&netstamp_needed_deferred, 0);
- if (deferred) {
- while (--deferred)
- static_key_slow_dec(&netstamp_needed);
- return;
- }
+ while (deferred--)
+ static_key_slow_dec(&netstamp_needed);
+}
+static DECLARE_WORK(netstamp_work, netstamp_clear);
#endif
+
+void net_enable_timestamp(void)
+{
static_key_slow_inc(&netstamp_needed);
}
EXPORT_SYMBOL(net_enable_timestamp);
void net_disable_timestamp(void)
{
#ifdef HAVE_JUMP_LABEL
- if (in_interrupt()) {
- atomic_inc(&netstamp_needed_deferred);
- return;
- }
-#endif
+ /* net_disable_timestamp() can be called from non process context */
+ atomic_inc(&netstamp_needed_deferred);
+ schedule_work(&netstamp_work);
+#else
static_key_slow_dec(&netstamp_needed);
+#endif
}
EXPORT_SYMBOL(net_disable_timestamp);
if (skb->ip_summed != CHECKSUM_NONE &&
!can_checksum_protocol(features, type)) {
features &= ~(NETIF_F_CSUM_MASK | NETIF_F_GSO_MASK);
- } else if (illegal_highdma(skb->dev, skb)) {
- features &= ~NETIF_F_SG;
}
+ if (illegal_highdma(skb->dev, skb))
+ features &= ~NETIF_F_SG;
return features;
}
if (regs.len > reglen)
regs.len = reglen;
- regbuf = vzalloc(reglen);
- if (reglen && !regbuf)
- return -ENOMEM;
+ regbuf = NULL;
+ if (reglen) {
+ regbuf = vzalloc(reglen);
+ if (!regbuf)
+ return -ENOMEM;
+ }
ops->get_regs(dev, ®s, regbuf);
static noinline_for_stack int ethtool_set_channels(struct net_device *dev,
void __user *useraddr)
{
- struct ethtool_channels channels, max;
+ struct ethtool_channels channels, max = { .cmd = ETHTOOL_GCHANNELS };
u32 max_rx_in_use = 0;
if (!dev->ethtool_ops->set_channels || !dev->ethtool_ops->get_channels)
.fill_encap = bpf_fill_encap_info,
.get_encap_size = bpf_encap_nlsize,
.cmp_encap = bpf_encap_cmp,
+ .owner = THIS_MODULE,
};
static int __init bpf_lwt_init(void)
#include <net/lwtunnel.h>
#include <net/rtnetlink.h>
#include <net/ip6_fib.h>
+#include <net/nexthop.h>
#ifdef CONFIG_MODULES
ret = -EOPNOTSUPP;
rcu_read_lock();
ops = rcu_dereference(lwtun_encaps[encap_type]);
+ if (likely(ops && ops->build_state && try_module_get(ops->owner))) {
+ ret = ops->build_state(dev, encap, family, cfg, lws);
+ if (ret)
+ module_put(ops->owner);
+ }
+ rcu_read_unlock();
+
+ return ret;
+}
+EXPORT_SYMBOL(lwtunnel_build_state);
+
+int lwtunnel_valid_encap_type(u16 encap_type)
+{
+ const struct lwtunnel_encap_ops *ops;
+ int ret = -EINVAL;
+
+ if (encap_type == LWTUNNEL_ENCAP_NONE ||
+ encap_type > LWTUNNEL_ENCAP_MAX)
+ return ret;
+
+ rcu_read_lock();
+ ops = rcu_dereference(lwtun_encaps[encap_type]);
+ rcu_read_unlock();
#ifdef CONFIG_MODULES
if (!ops) {
const char *encap_type_str = lwtunnel_encap_str(encap_type);
if (encap_type_str) {
- rcu_read_unlock();
+ __rtnl_unlock();
request_module("rtnl-lwt-%s", encap_type_str);
+ rtnl_lock();
+
rcu_read_lock();
ops = rcu_dereference(lwtun_encaps[encap_type]);
+ rcu_read_unlock();
}
}
#endif
- if (likely(ops && ops->build_state))
- ret = ops->build_state(dev, encap, family, cfg, lws);
- rcu_read_unlock();
+ return ops ? 0 : -EOPNOTSUPP;
+}
+EXPORT_SYMBOL(lwtunnel_valid_encap_type);
- return ret;
+int lwtunnel_valid_encap_type_attr(struct nlattr *attr, int remaining)
+{
+ struct rtnexthop *rtnh = (struct rtnexthop *)attr;
+ struct nlattr *nla_entype;
+ struct nlattr *attrs;
+ struct nlattr *nla;
+ u16 encap_type;
+ int attrlen;
+
+ while (rtnh_ok(rtnh, remaining)) {
+ attrlen = rtnh_attrlen(rtnh);
+ if (attrlen > 0) {
+ attrs = rtnh_attrs(rtnh);
+ nla = nla_find(attrs, attrlen, RTA_ENCAP);
+ nla_entype = nla_find(attrs, attrlen, RTA_ENCAP_TYPE);
+
+ if (nla_entype) {
+ encap_type = nla_get_u16(nla_entype);
+
+ if (lwtunnel_valid_encap_type(encap_type) != 0)
+ return -EOPNOTSUPP;
+ }
+ }
+ rtnh = rtnh_next(rtnh, &remaining);
+ }
+
+ return 0;
}
-EXPORT_SYMBOL(lwtunnel_build_state);
+EXPORT_SYMBOL(lwtunnel_valid_encap_type_attr);
void lwtstate_free(struct lwtunnel_state *lws)
{
} else {
kfree(lws);
}
+ module_put(ops->owner);
}
EXPORT_SYMBOL(lwtstate_free);
opt = ireq->ipv6_opt;
if (!opt)
opt = rcu_dereference(np->opt);
- err = ip6_xmit(sk, skb, &fl6, opt, np->tclass);
+ err = ip6_xmit(sk, skb, &fl6, sk->sk_mark, opt, np->tclass);
rcu_read_unlock();
err = net_xmit_eval(err);
}
dst = ip6_dst_lookup_flow(ctl_sk, &fl6, NULL);
if (!IS_ERR(dst)) {
skb_dst_set(skb, dst);
- ip6_xmit(ctl_sk, skb, &fl6, NULL, 0);
+ ip6_xmit(ctl_sk, skb, &fl6, 0, NULL, 0);
DCCP_INC_STATS(DCCP_MIB_OUTSEGS);
DCCP_INC_STATS(DCCP_MIB_OUTRSTS);
return;
if (err) {
dev_warn(ds->dev, "Failed to create slave %d: %d\n",
index, err);
+ ds->ports[index].netdev = NULL;
return err;
}
/* Use already configured phy mode */
if (p->phy_interface == PHY_INTERFACE_MODE_NA)
p->phy_interface = p->phy->interface;
- phy_connect_direct(slave_dev, p->phy, dsa_slave_adjust_link,
- p->phy_interface);
-
- return 0;
+ return phy_connect_direct(slave_dev, p->phy, dsa_slave_adjust_link,
+ p->phy_interface);
}
static int dsa_slave_phy_setup(struct dsa_slave_priv *p,
{
struct dsa_slave_priv *p = netdev_priv(slave_dev);
+ netif_device_detach(slave_dev);
+
if (p->phy) {
phy_stop(p->phy);
p->old_pause = -1;
dev->header_ops = ð_header_ops;
dev->type = ARPHRD_ETHER;
dev->hard_header_len = ETH_HLEN;
+ dev->min_header_len = ETH_HLEN;
dev->mtu = ETH_DATA_LEN;
dev->min_mtu = ETH_MIN_MTU;
dev->max_mtu = ETH_DATA_LEN;
goto validate_return_locked;
}
+ if (opt_iter + 1 == opt_len) {
+ err_offset = opt_iter;
+ goto validate_return_locked;
+ }
tag_len = tag[1];
if (tag_len > (opt_len - opt_iter)) {
err_offset = opt_iter + 1;
#include <net/rtnetlink.h>
#include <net/xfrm.h>
#include <net/l3mdev.h>
+#include <net/lwtunnel.h>
#include <trace/events/fib.h>
#ifndef CONFIG_IP_MULTIPLE_TABLES
cfg->fc_mx_len = nla_len(attr);
break;
case RTA_MULTIPATH:
+ err = lwtunnel_valid_encap_type_attr(nla_data(attr),
+ nla_len(attr));
+ if (err < 0)
+ goto errout;
cfg->fc_mp = nla_data(attr);
cfg->fc_mp_len = nla_len(attr);
break;
break;
case RTA_ENCAP_TYPE:
cfg->fc_encap_type = nla_get_u16(attr);
+ err = lwtunnel_valid_encap_type(cfg->fc_encap_type);
+ if (err < 0)
+ goto errout;
break;
}
}
psf->sf_crcount = im->crcount;
}
in_dev_put(pmc->interface);
+ kfree(pmc);
}
spin_unlock_bh(&im->lock);
}
sk->sk_protocol = ip_hdr(skb)->protocol;
sk->sk_bound_dev_if = arg->bound_dev_if;
sk->sk_sndbuf = sysctl_wmem_default;
+ sk->sk_mark = fl4.flowi4_mark;
err = ip_append_data(sk, &fl4, ip_reply_glue_bits, arg->iov->iov_base,
len, 0, &ipc, &rt, MSG_DONTWAIT);
if (unlikely(err)) {
pktinfo->ipi_ifindex = 0;
pktinfo->ipi_spec_dst.s_addr = 0;
}
- skb_dst_drop(skb);
+ /* We need to keep the dst for __ip_options_echo()
+ * We could restrict the test to opt.ts_needtime || opt.srr,
+ * but the following is good enough as IP options are not often used.
+ */
+ if (unlikely(IPCB(skb)->opt.optlen))
+ skb_dst_force(skb);
+ else
+ skb_dst_drop(skb);
}
int ip_setsockopt(struct sock *sk, int level,
.fill_encap = ip_tun_fill_encap_info,
.get_encap_size = ip_tun_encap_nlsize,
.cmp_encap = ip_tun_cmp_encap,
+ .owner = THIS_MODULE,
};
static const struct nla_policy ip6_tun_policy[LWTUNNEL_IP6_MAX + 1] = {
.fill_encap = ip6_tun_fill_encap_info,
.get_encap_size = ip6_tun_encap_nlsize,
.cmp_encap = ip_tun_cmp_encap,
+ .owner = THIS_MODULE,
};
void __init ip_tunnel_core_init(void)
rcu_read_lock_bh();
c = __clusterip_config_find(net, clusterip);
if (c) {
- if (!c->pde || unlikely(!atomic_inc_not_zero(&c->refcount)))
+#ifdef CONFIG_PROC_FS
+ if (!c->pde)
+ c = NULL;
+ else
+#endif
+ if (unlikely(!atomic_inc_not_zero(&c->refcount)))
c = NULL;
else if (entry)
atomic_inc(&c->entries);
return dev_match || flags & XT_RPFILTER_LOOSE;
}
-static bool rpfilter_is_local(const struct sk_buff *skb)
+static bool
+rpfilter_is_loopback(const struct sk_buff *skb, const struct net_device *in)
{
- const struct rtable *rt = skb_rtable(skb);
- return rt && (rt->rt_flags & RTCF_LOCAL);
+ return skb->pkt_type == PACKET_LOOPBACK || in->flags & IFF_LOOPBACK;
}
static bool rpfilter_mt(const struct sk_buff *skb, struct xt_action_param *par)
info = par->matchinfo;
invert = info->flags & XT_RPFILTER_INVERT;
- if (rpfilter_is_local(skb))
+ if (rpfilter_is_loopback(skb, xt_in(par)))
return true ^ invert;
iph = ip_hdr(skb);
/* ip_route_me_harder expects skb->dst to be set */
skb_dst_set_noref(nskb, skb_dst(oldskb));
+ nskb->mark = IP4_REPLY_MARK(net, oldskb->mark);
+
skb_reserve(nskb, LL_MAX_HEADER);
niph = nf_reject_iphdr_put(nskb, oldskb, IPPROTO_TCP,
ip4_dst_hoplimit(skb_dst(nskb)));
return addr;
}
-static bool fib4_is_local(const struct sk_buff *skb)
-{
- const struct rtable *rt = skb_rtable(skb);
-
- return rt && (rt->rt_flags & RTCF_LOCAL);
-}
-
#define DSCP_BITS 0xfc
void nft_fib4_eval_type(const struct nft_expr *expr, struct nft_regs *regs,
else
oif = NULL;
- if (nft_hook(pkt) == NF_INET_PRE_ROUTING && fib4_is_local(pkt->skb)) {
- nft_fib_store_result(dest, priv->result, pkt, LOOPBACK_IFINDEX);
+ if (nft_hook(pkt) == NF_INET_PRE_ROUTING &&
+ nft_fib_is_loopback(pkt->skb, nft_in(pkt))) {
+ nft_fib_store_result(dest, priv->result, pkt,
+ nft_in(pkt)->ifindex);
return;
}
switch (res.type) {
case RTN_UNICAST:
break;
- case RTN_LOCAL: /* should not appear here, see fib4_is_local() above */
+ case RTN_LOCAL: /* Should not see RTN_LOCAL here */
return;
default:
break;
{
struct sk_buff *skb = skb_peek(&sk->sk_write_queue);
+ if (!skb)
+ return 0;
pfh->wcheck = csum_partial((char *)&pfh->icmph,
sizeof(struct icmphdr), pfh->wcheck);
pfh->icmph.checksum = csum_fold(pfh->wcheck);
ret = -EAGAIN;
break;
}
+ /* if __tcp_splice_read() got nothing while we have
+ * an skb in receive queue, we do not want to loop.
+ * This might happen with URG data.
+ */
+ if (!skb_queue_empty(&sk->sk_receive_queue))
+ break;
sk_wait_data(sk, &timeo, NULL);
if (signal_pending(current)) {
ret = sock_intr_errno(timeo);
* scaled. So correct it appropriately.
*/
tp->snd_wnd = ntohs(tcp_hdr(skb)->window);
+ tp->max_window = tp->snd_wnd;
/* Activate the retrans timer so that SYNACK can be retransmitted.
* The request socket is not added to the ehash
if (sock_flag(sk, SOCK_QUEUE_SHRUNK)) {
sock_reset_flag(sk, SOCK_QUEUE_SHRUNK);
/* pairs with tcp_poll() */
- smp_mb__after_atomic();
+ smp_mb();
if (sk->sk_socket &&
test_bit(SOCK_NOSPACE, &sk->sk_socket->flags)) {
tcp_new_space(sk);
int full_space = min_t(int, tp->window_clamp, allowed_space);
int window;
- if (mss > full_space)
+ if (unlikely(mss > full_space)) {
mss = full_space;
-
+ if (mss <= 0)
+ return 0;
+ }
if (free_space < (full_space >> 1)) {
icsk->icsk_ack.quick = 0;
return err;
csum_copy_err:
- if (!__sk_queue_drop_skb(sk, skb, flags)) {
+ if (!__sk_queue_drop_skb(sk, skb, flags, udp_skb_destructor)) {
UDP_INC_STATS(sock_net(sk), UDP_MIB_CSUMERRORS, is_udplite);
UDP_INC_STATS(sock_net(sk), UDP_MIB_INERRORS, is_udplite);
}
}
if (idev) {
- if (idev->if_flags & IF_READY)
- /* device is already configured. */
+ if (idev->if_flags & IF_READY) {
+ /* device is already configured -
+ * but resend MLD reports, we might
+ * have roamed and need to update
+ * multicast snooping switches
+ */
+ ipv6_mc_up(idev);
break;
+ }
idev->if_flags |= IF_READY;
}
if (bump_id)
rt_genid_bump_ipv6(dev_net(dev));
+
+ /* Make sure that a new temporary address will be created
+ * before this temporary address becomes deprecated.
+ */
+ if (ifp->flags & IFA_F_TEMPORARY)
+ addrconf_verify_rtnl();
}
static void addrconf_dad_run(struct inet6_dev *idev)
struct net_device *dev;
struct inet6_dev *idev;
- rcu_read_lock();
- for_each_netdev_rcu(net, dev) {
+ for_each_netdev(net, dev) {
idev = __in6_dev_get(dev);
if (idev) {
int changed = (!idev->cnf.disable_ipv6) ^ (!newf);
dev_disable_change(idev);
}
}
- rcu_read_unlock();
}
static int addrconf_disable_ipv6(struct ctl_table *table, int *p, int newf)
struct ipv6_sr_hdr *hdr;
struct inet6_dev *idev;
struct in6_addr *addr;
- bool cleanup = false;
int accept_seg6;
hdr = (struct ipv6_sr_hdr *)skb_transport_header(skb);
#endif
looped_back:
- if (hdr->segments_left > 0) {
- if (hdr->nexthdr != NEXTHDR_IPV6 && hdr->segments_left == 1 &&
- sr_has_cleanup(hdr))
- cleanup = true;
- } else {
+ if (hdr->segments_left == 0) {
if (hdr->nexthdr == NEXTHDR_IPV6) {
int offset = (hdr->hdrlen + 1) << 3;
ipv6_hdr(skb)->daddr = *addr;
- if (cleanup) {
- int srhlen = (hdr->hdrlen + 1) << 3;
- int nh = hdr->nexthdr;
-
- skb_pull_rcsum(skb, sizeof(struct ipv6hdr) + srhlen);
- memmove(skb_network_header(skb) + srhlen,
- skb_network_header(skb),
- (unsigned char *)hdr - skb_network_header(skb));
- skb->network_header += srhlen;
- ipv6_hdr(skb)->nexthdr = nh;
- ipv6_hdr(skb)->payload_len = htons(skb->len -
- sizeof(struct ipv6hdr));
- skb_push_rcsum(skb, sizeof(struct ipv6hdr));
- }
-
skb_dst_drop(skb);
ip6_route_input(skb);
}
ipv6_hdr(skb)->hop_limit--;
- /* be sure that srh is still present before reinjecting */
- if (!cleanup) {
- skb_pull(skb, sizeof(struct ipv6hdr));
- goto looped_back;
- }
- skb_set_transport_header(skb, sizeof(struct ipv6hdr));
- IP6CB(skb)->nhoff = offsetof(struct ipv6hdr, nexthdr);
+ skb_pull(skb, sizeof(struct ipv6hdr));
+ goto looped_back;
}
dst_input(skb);
.fill_encap = ila_fill_encap_info,
.get_encap_size = ila_encap_nlsize,
.cmp_encap = ila_encap_cmp,
+ .owner = THIS_MODULE,
};
int ila_lwt_init(void)
/* Restore final destination back after routing done */
fl6.daddr = sk->sk_v6_daddr;
- res = ip6_xmit(sk, skb, &fl6, rcu_dereference(np->opt),
+ res = ip6_xmit(sk, skb, &fl6, sk->sk_mark, rcu_dereference(np->opt),
np->tclass);
rcu_read_unlock();
return res;
static void ip6gre_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
- u8 type, u8 code, int offset, __be32 info)
+ u8 type, u8 code, int offset, __be32 info)
{
- const struct ipv6hdr *ipv6h = (const struct ipv6hdr *)skb->data;
- __be16 *p = (__be16 *)(skb->data + offset);
- int grehlen = offset + 4;
+ const struct gre_base_hdr *greh;
+ const struct ipv6hdr *ipv6h;
+ int grehlen = sizeof(*greh);
struct ip6_tnl *t;
+ int key_off = 0;
__be16 flags;
+ __be32 key;
- flags = p[0];
- if (flags&(GRE_CSUM|GRE_KEY|GRE_SEQ|GRE_ROUTING|GRE_VERSION)) {
- if (flags&(GRE_VERSION|GRE_ROUTING))
- return;
- if (flags&GRE_KEY) {
- grehlen += 4;
- if (flags&GRE_CSUM)
- grehlen += 4;
- }
+ if (!pskb_may_pull(skb, offset + grehlen))
+ return;
+ greh = (const struct gre_base_hdr *)(skb->data + offset);
+ flags = greh->flags;
+ if (flags & (GRE_VERSION | GRE_ROUTING))
+ return;
+ if (flags & GRE_CSUM)
+ grehlen += 4;
+ if (flags & GRE_KEY) {
+ key_off = grehlen + offset;
+ grehlen += 4;
}
- /* If only 8 bytes returned, keyed message will be dropped here */
- if (!pskb_may_pull(skb, grehlen))
+ if (!pskb_may_pull(skb, offset + grehlen))
return;
ipv6h = (const struct ipv6hdr *)skb->data;
- p = (__be16 *)(skb->data + offset);
+ greh = (const struct gre_base_hdr *)(skb->data + offset);
+ key = key_off ? *(__be32 *)(skb->data + key_off) : 0;
t = ip6gre_tunnel_lookup(skb->dev, &ipv6h->daddr, &ipv6h->saddr,
- flags & GRE_KEY ?
- *(((__be32 *)p) + (grehlen / 4) - 1) : 0,
- p[1]);
+ key, greh->protocol);
if (!t)
return;
return -1;
offset = ip6_tnl_parse_tlv_enc_lim(skb, skb_network_header(skb));
+ /* ip6_tnl_parse_tlv_enc_lim() might have reallocated skb->head */
+ ipv6h = ipv6_hdr(skb);
+
if (offset > 0) {
struct ipv6_tlv_tnl_enc_lim *tel;
tel = (struct ipv6_tlv_tnl_enc_lim *)&skb_network_header(skb)[offset];
* which are using proper atomic operations or spinlocks.
*/
int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,
- struct ipv6_txoptions *opt, int tclass)
+ __u32 mark, struct ipv6_txoptions *opt, int tclass)
{
struct net *net = sock_net(sk);
const struct ipv6_pinfo *np = inet6_sk(sk);
skb->protocol = htons(ETH_P_IPV6);
skb->priority = sk->sk_priority;
- skb->mark = sk->sk_mark;
+ skb->mark = mark;
mtu = dst_mtu(dst);
if ((skb->len <= mtu) || skb->ignore_df || skb_is_gso(skb)) {
*/
if (transhdrlen && sk->sk_protocol == IPPROTO_UDP &&
headersize == sizeof(struct ipv6hdr) &&
- length < mtu - headersize &&
+ length <= mtu - headersize &&
!(flags & MSG_MORE) &&
rt->dst.dev->features & (NETIF_F_IPV6_CSUM | NETIF_F_HW_CSUM))
csummode = CHECKSUM_PARTIAL;
__u16 ip6_tnl_parse_tlv_enc_lim(struct sk_buff *skb, __u8 *raw)
{
- const struct ipv6hdr *ipv6h = (const struct ipv6hdr *) raw;
- __u8 nexthdr = ipv6h->nexthdr;
- __u16 off = sizeof(*ipv6h);
+ const struct ipv6hdr *ipv6h = (const struct ipv6hdr *)raw;
+ unsigned int nhoff = raw - skb->data;
+ unsigned int off = nhoff + sizeof(*ipv6h);
+ u8 next, nexthdr = ipv6h->nexthdr;
while (ipv6_ext_hdr(nexthdr) && nexthdr != NEXTHDR_NONE) {
- __u16 optlen = 0;
struct ipv6_opt_hdr *hdr;
- if (raw + off + sizeof(*hdr) > skb->data &&
- !pskb_may_pull(skb, raw - skb->data + off + sizeof (*hdr)))
+ u16 optlen;
+
+ if (!pskb_may_pull(skb, off + sizeof(*hdr)))
break;
- hdr = (struct ipv6_opt_hdr *) (raw + off);
+ hdr = (struct ipv6_opt_hdr *)(skb->data + off);
if (nexthdr == NEXTHDR_FRAGMENT) {
struct frag_hdr *frag_hdr = (struct frag_hdr *) hdr;
if (frag_hdr->frag_off)
} else {
optlen = ipv6_optlen(hdr);
}
+ /* cache hdr->nexthdr, since pskb_may_pull() might
+ * invalidate hdr
+ */
+ next = hdr->nexthdr;
if (nexthdr == NEXTHDR_DEST) {
- __u16 i = off + 2;
+ u16 i = 2;
+
+ /* Remember : hdr is no longer valid at this point. */
+ if (!pskb_may_pull(skb, off + optlen))
+ break;
+
while (1) {
struct ipv6_tlv_tnl_enc_lim *tel;
/* No more room for encapsulation limit */
- if (i + sizeof (*tel) > off + optlen)
+ if (i + sizeof(*tel) > optlen)
break;
- tel = (struct ipv6_tlv_tnl_enc_lim *) &raw[i];
+ tel = (struct ipv6_tlv_tnl_enc_lim *)(skb->data + off + i);
/* return index of option if found and valid */
if (tel->type == IPV6_TLV_TNL_ENCAP_LIMIT &&
tel->length == 1)
- return i;
+ return i + off - nhoff;
/* else jump to next option */
if (tel->type)
i += tel->length + 2;
i++;
}
}
- nexthdr = hdr->nexthdr;
+ nexthdr = next;
off += optlen;
}
return 0;
fl6.flowlabel = key->label;
} else {
offset = ip6_tnl_parse_tlv_enc_lim(skb, skb_network_header(skb));
+ /* ip6_tnl_parse_tlv_enc_lim() might have reallocated skb->head */
+ ipv6h = ipv6_hdr(skb);
if (offset > 0) {
struct ipv6_tlv_tnl_enc_lim *tel;
psf->sf_crcount = im->mca_crcount;
}
in6_dev_put(pmc->idev);
+ kfree(pmc);
}
spin_unlock_bh(&im->mca_lock);
}
return ret;
}
-static bool rpfilter_is_local(const struct sk_buff *skb)
+static bool
+rpfilter_is_loopback(const struct sk_buff *skb, const struct net_device *in)
{
- const struct rt6_info *rt = (const void *) skb_dst(skb);
- return rt && (rt->rt6i_flags & RTF_LOCAL);
+ return skb->pkt_type == PACKET_LOOPBACK || in->flags & IFF_LOOPBACK;
}
static bool rpfilter_mt(const struct sk_buff *skb, struct xt_action_param *par)
struct ipv6hdr *iph;
bool invert = info->flags & XT_RPFILTER_INVERT;
- if (rpfilter_is_local(skb))
+ if (rpfilter_is_loopback(skb, xt_in(par)))
return true ^ invert;
iph = ipv6_hdr(skb);
fl6.fl6_sport = otcph->dest;
fl6.fl6_dport = otcph->source;
fl6.flowi6_oif = l3mdev_master_ifindex(skb_dst(oldskb)->dev);
+ fl6.flowi6_mark = IP6_REPLY_MARK(net, oldskb->mark);
security_skb_classify_flow(oldskb, flowi6_to_flowi(&fl6));
dst = ip6_route_output(net, NULL, &fl6);
if (dst->error) {
skb_dst_set(nskb, dst);
+ nskb->mark = fl6.flowi6_mark;
+
skb_reserve(nskb, hh_len + dst->header_len);
ip6h = nf_reject_ip6hdr_put(nskb, oldskb, IPPROTO_TCP,
ip6_dst_hoplimit(dst));
#include <net/ip6_fib.h>
#include <net/ip6_route.h>
-static bool fib6_is_local(const struct sk_buff *skb)
-{
- const struct rt6_info *rt = (const void *)skb_dst(skb);
-
- return rt && (rt->rt6i_flags & RTF_LOCAL);
-}
-
static int get_ifindex(const struct net_device *dev)
{
return dev ? dev->ifindex : 0;
lookup_flags = nft_fib6_flowi_init(&fl6, priv, pkt, oif);
- if (nft_hook(pkt) == NF_INET_PRE_ROUTING && fib6_is_local(pkt->skb)) {
- nft_fib_store_result(dest, priv->result, pkt, LOOPBACK_IFINDEX);
+ if (nft_hook(pkt) == NF_INET_PRE_ROUTING &&
+ nft_fib_is_loopback(pkt->skb, nft_in(pkt))) {
+ nft_fib_store_result(dest, priv->result, pkt,
+ nft_in(pkt)->ifindex);
return;
}
if (tb[RTA_MULTIPATH]) {
cfg->fc_mp = nla_data(tb[RTA_MULTIPATH]);
cfg->fc_mp_len = nla_len(tb[RTA_MULTIPATH]);
+
+ err = lwtunnel_valid_encap_type_attr(cfg->fc_mp,
+ cfg->fc_mp_len);
+ if (err < 0)
+ goto errout;
}
if (tb[RTA_PREF]) {
if (tb[RTA_ENCAP])
cfg->fc_encap = tb[RTA_ENCAP];
- if (tb[RTA_ENCAP_TYPE])
+ if (tb[RTA_ENCAP_TYPE]) {
cfg->fc_encap_type = nla_get_u16(tb[RTA_ENCAP_TYPE]);
+ err = lwtunnel_valid_encap_type(cfg->fc_encap_type);
+ if (err < 0)
+ goto errout;
+ }
+
if (tb[RTA_EXPIRES]) {
unsigned long timeout = addrconf_timeout_fixup(nla_get_u32(tb[RTA_EXPIRES]), HZ);
val = nla_data(info->attrs[SEG6_ATTR_DST]);
t_new = kmemdup(val, sizeof(*val), GFP_KERNEL);
+ if (!t_new)
+ return -ENOMEM;
mutex_lock(&sdata->lock);
* hash function (RadioGatun) with up to 1216 bits
*/
- /* saddr(16) + first_seg(1) + cleanup(1) + keyid(4) + seglist(16n) */
+ /* saddr(16) + first_seg(1) + flags(1) + keyid(4) + seglist(16n) */
plen = 16 + 1 + 1 + 4 + (hdr->first_segment + 1) * 16;
/* this limit allows for 14 segments */
*
* 1. Source IPv6 address (128 bits)
* 2. first_segment value (8 bits)
- * 3. cleanup flag (8 bits: highest bit is cleanup value, others are 0)
+ * 3. Flags (8 bits)
* 4. HMAC Key ID (32 bits)
* 5. All segments in the segments list (n * 128 bits)
*/
/* first_segment value */
*off++ = hdr->first_segment;
- /* cleanup flag */
- *off++ = !!(sr_has_cleanup(hdr)) << 7;
+ /* flags */
+ *off++ = hdr->flags;
/* HMAC Key ID */
memcpy(off, &hmackeyid, 4);
.fill_encap = seg6_fill_encap_info,
.get_encap_size = seg6_encap_nlsize,
.cmp_encap = seg6_encap_cmp,
+ .owner = THIS_MODULE,
};
int __init seg6_iptunnel_init(void)
err = dst_cache_init(&tunnel->dst_cache, GFP_KERNEL);
if (err) {
free_percpu(dev->tstats);
+ dev->tstats = NULL;
return err;
}
opt = ireq->ipv6_opt;
if (!opt)
opt = rcu_dereference(np->opt);
- err = ip6_xmit(sk, skb, fl6, opt, np->tclass);
+ err = ip6_xmit(sk, skb, fl6, sk->sk_mark, opt, np->tclass);
rcu_read_unlock();
err = net_xmit_eval(err);
}
dst = ip6_dst_lookup_flow(ctl_sk, &fl6, NULL);
if (!IS_ERR(dst)) {
skb_dst_set(buff, dst);
- ip6_xmit(ctl_sk, buff, &fl6, NULL, tclass);
+ ip6_xmit(ctl_sk, buff, &fl6, fl6.flowi6_mark, NULL, tclass);
TCP_INC_STATS(net, TCP_MIB_OUTSEGS);
if (rst)
TCP_INC_STATS(net, TCP_MIB_OUTRSTS);
return 0; /* don't send reset */
}
+static void tcp_v6_restore_cb(struct sk_buff *skb)
+{
+ /* We need to move header back to the beginning if xfrm6_policy_check()
+ * and tcp_v6_fill_cb() are going to be called again.
+ * ip6_datagram_recv_specific_ctl() also expects IP6CB to be there.
+ */
+ memmove(IP6CB(skb), &TCP_SKB_CB(skb)->header.h6,
+ sizeof(struct inet6_skb_parm));
+}
+
static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req,
struct dst_entry *dst,
sk_gfp_mask(sk, GFP_ATOMIC));
consume_skb(ireq->pktopts);
ireq->pktopts = NULL;
- if (newnp->pktoptions)
+ if (newnp->pktoptions) {
+ tcp_v6_restore_cb(newnp->pktoptions);
skb_set_owner_r(newnp->pktoptions, newsk);
+ }
}
}
return NULL;
}
-static void tcp_v6_restore_cb(struct sk_buff *skb)
-{
- /* We need to move header back to the beginning if xfrm6_policy_check()
- * and tcp_v6_fill_cb() are going to be called again.
- * ip6_datagram_recv_specific_ctl() also expects IP6CB to be there.
- */
- memmove(IP6CB(skb), &TCP_SKB_CB(skb)->header.h6,
- sizeof(struct inet6_skb_parm));
-}
-
/* The socket must have it's spinlock held when we get
* here, unless it is a TCP_LISTEN socket.
*
return err;
csum_copy_err:
- if (!__sk_queue_drop_skb(sk, skb, flags)) {
+ if (!__sk_queue_drop_skb(sk, skb, flags, udp_skb_destructor)) {
if (is_udp4) {
UDP_INC_STATS(sock_net(sk),
UDP_MIB_CSUMERRORS, is_udplite);
goto out_error;
}
- /* New message, alloc head skb */
- head = alloc_skb(0, sk->sk_allocation);
- while (!head) {
- kcm_push(kcm);
- err = sk_stream_wait_memory(sk, &timeo);
- if (err)
- goto out_error;
-
+ if (msg_data_left(msg)) {
+ /* New message, alloc head skb */
head = alloc_skb(0, sk->sk_allocation);
- }
+ while (!head) {
+ kcm_push(kcm);
+ err = sk_stream_wait_memory(sk, &timeo);
+ if (err)
+ goto out_error;
- skb = head;
+ head = alloc_skb(0, sk->sk_allocation);
+ }
- /* Set ip_summed to CHECKSUM_UNNECESSARY to avoid calling
- * csum_and_copy_from_iter from skb_do_copy_data_nocache.
- */
- skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb = head;
+
+ /* Set ip_summed to CHECKSUM_UNNECESSARY to avoid calling
+ * csum_and_copy_from_iter from skb_do_copy_data_nocache.
+ */
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ }
start:
while (msg_data_left(msg)) {
if (eor) {
bool not_busy = skb_queue_empty(&sk->sk_write_queue);
- /* Message complete, queue it on send buffer */
- __skb_queue_tail(&sk->sk_write_queue, head);
- kcm->seq_skb = NULL;
- KCM_STATS_INCR(kcm->stats.tx_msgs);
+ if (head) {
+ /* Message complete, queue it on send buffer */
+ __skb_queue_tail(&sk->sk_write_queue, head);
+ kcm->seq_skb = NULL;
+ KCM_STATS_INCR(kcm->stats.tx_msgs);
+ }
if (msg->msg_flags & MSG_BATCH) {
kcm->tx_wait_more = true;
int l2tp_nl_register_ops(enum l2tp_pwtype pw_type,
const struct l2tp_nl_cmd_ops *ops);
void l2tp_nl_unregister_ops(enum l2tp_pwtype pw_type);
+int l2tp_ioctl(struct sock *sk, int cmd, unsigned long arg);
/* Session reference counts. Incremented when code obtains a reference
* to a session.
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <asm/ioctls.h>
#include <linux/icmp.h>
#include <linux/module.h>
#include <linux/skbuff.h>
return err ? err : copied;
}
+int l2tp_ioctl(struct sock *sk, int cmd, unsigned long arg)
+{
+ struct sk_buff *skb;
+ int amount;
+
+ switch (cmd) {
+ case SIOCOUTQ:
+ amount = sk_wmem_alloc_get(sk);
+ break;
+ case SIOCINQ:
+ spin_lock_bh(&sk->sk_receive_queue.lock);
+ skb = skb_peek(&sk->sk_receive_queue);
+ amount = skb ? skb->len : 0;
+ spin_unlock_bh(&sk->sk_receive_queue.lock);
+ break;
+
+ default:
+ return -ENOIOCTLCMD;
+ }
+
+ return put_user(amount, (int __user *)arg);
+}
+EXPORT_SYMBOL(l2tp_ioctl);
+
static struct proto l2tp_ip_prot = {
.name = "L2TP/IP",
.owner = THIS_MODULE,
.bind = l2tp_ip_bind,
.connect = l2tp_ip_connect,
.disconnect = l2tp_ip_disconnect,
- .ioctl = udp_ioctl,
+ .ioctl = l2tp_ioctl,
.destroy = l2tp_ip_destroy_sock,
.setsockopt = ip_setsockopt,
.getsockopt = ip_getsockopt,
.bind = l2tp_ip6_bind,
.connect = l2tp_ip6_connect,
.disconnect = l2tp_ip6_disconnect,
- .ioctl = udp_ioctl,
+ .ioctl = l2tp_ioctl,
.destroy = l2tp_ip6_destroy_sock,
.setsockopt = ipv6_setsockopt,
.getsockopt = ipv6_getsockopt,
/* CTR */
- tfm2 = crypto_alloc_skcipher("ctr(aes)", 0, 0);
+ tfm2 = crypto_alloc_skcipher("ctr(aes)", 0, CRYPTO_ALG_ASYNC);
if (IS_ERR(tfm2)) {
kfree(tmp);
return PTR_ERR(tfm2);
/* CTR */
- tfm2 = crypto_alloc_skcipher("ctr(aes)", 0, 0);
+ tfm2 = crypto_alloc_skcipher("ctr(aes)", 0, CRYPTO_ALG_ASYNC);
if (IS_ERR(tfm2))
return PTR_ERR(tfm2);
/* K2 for CTR */
crypt_len = skb->data + skb->len - encr;
skb_put(skb, AES_BLOCK_SIZE);
return aes_siv_encrypt(assoc_data->fils_kek, assoc_data->fils_kek_len,
- encr, crypt_len, 1, addr, len, encr);
+ encr, crypt_len, 5, addr, len, encr);
}
int fils_decrypt_assoc_resp(struct ieee80211_sub_if_data *sdata,
/* fast-forward to vendor IEs */
offset = ieee80211_ie_split_vendor(ifmsh->ie, ifmsh->ie_len, 0);
- if (offset) {
+ if (offset < ifmsh->ie_len) {
len = ifmsh->ie_len - offset;
data = ifmsh->ie + offset;
if (skb_tailroom(skb) < len)
ieee80211_sta_set_rx_nss(sta);
- ieee80211_recalc_min_chandef(sta->sdata);
-
if (!ref)
return;
}
EXPORT_SYMBOL_GPL(mpls_pkt_too_big);
-static u32 mpls_multipath_hash(struct mpls_route *rt,
- struct sk_buff *skb, bool bos)
+static u32 mpls_multipath_hash(struct mpls_route *rt, struct sk_buff *skb)
{
struct mpls_entry_decoded dec;
+ unsigned int mpls_hdr_len = 0;
struct mpls_shim_hdr *hdr;
bool eli_seen = false;
int label_index;
u32 hash = 0;
- for (label_index = 0; label_index < MAX_MP_SELECT_LABELS && !bos;
+ for (label_index = 0; label_index < MAX_MP_SELECT_LABELS;
label_index++) {
- if (!pskb_may_pull(skb, sizeof(*hdr) * label_index))
+ mpls_hdr_len += sizeof(*hdr);
+ if (!pskb_may_pull(skb, mpls_hdr_len))
break;
/* Read and decode the current label */
eli_seen = true;
}
- bos = dec.bos;
- if (bos && pskb_may_pull(skb, sizeof(*hdr) * label_index +
- sizeof(struct iphdr))) {
+ if (!dec.bos)
+ continue;
+
+ /* found bottom label; does skb have room for a header? */
+ if (pskb_may_pull(skb, mpls_hdr_len + sizeof(struct iphdr))) {
const struct iphdr *v4hdr;
- v4hdr = (const struct iphdr *)(mpls_hdr(skb) +
- label_index);
+ v4hdr = (const struct iphdr *)(hdr + 1);
if (v4hdr->version == 4) {
hash = jhash_3words(ntohl(v4hdr->saddr),
ntohl(v4hdr->daddr),
v4hdr->protocol, hash);
} else if (v4hdr->version == 6 &&
- pskb_may_pull(skb, sizeof(*hdr) * label_index +
- sizeof(struct ipv6hdr))) {
+ pskb_may_pull(skb, mpls_hdr_len +
+ sizeof(struct ipv6hdr))) {
const struct ipv6hdr *v6hdr;
- v6hdr = (const struct ipv6hdr *)(mpls_hdr(skb) +
- label_index);
-
+ v6hdr = (const struct ipv6hdr *)(hdr + 1);
hash = __ipv6_addr_jhash(&v6hdr->saddr, hash);
hash = __ipv6_addr_jhash(&v6hdr->daddr, hash);
hash = jhash_1word(v6hdr->nexthdr, hash);
}
}
+
+ break;
}
return hash;
}
static struct mpls_nh *mpls_select_multipath(struct mpls_route *rt,
- struct sk_buff *skb, bool bos)
+ struct sk_buff *skb)
{
int alive = ACCESS_ONCE(rt->rt_nhn_alive);
u32 hash = 0;
if (alive <= 0)
return NULL;
- hash = mpls_multipath_hash(rt, skb, bos);
+ hash = mpls_multipath_hash(rt, skb);
nh_index = hash % alive;
if (alive == rt->rt_nhn)
goto out;
hdr = mpls_hdr(skb);
dec = mpls_entry_decode(hdr);
- /* Pop the label */
- skb_pull(skb, sizeof(*hdr));
- skb_reset_network_header(skb);
-
- skb_orphan(skb);
-
rt = mpls_route_input_rcu(net, dec.label);
if (!rt)
goto drop;
- nh = mpls_select_multipath(rt, skb, dec.bos);
+ nh = mpls_select_multipath(rt, skb);
if (!nh)
goto drop;
if (!mpls_output_possible(out_dev))
goto drop;
+ /* Pop the label */
+ skb_pull(skb, sizeof(*hdr));
+ skb_reset_network_header(skb);
+
+ skb_orphan(skb);
+
if (skb_warn_if_lro(skb))
goto drop;
.fill_encap = mpls_fill_encap_info,
.get_encap_size = mpls_encap_nlsize,
.cmp_encap = mpls_encap_cmp,
+ .owner = THIS_MODULE,
};
static int __init mpls_iptunnel_init(void)
depends on NF_CONNTRACK
tristate "Netfilter nf_tables conntrack module"
help
- This option adds the "meta" expression that you can use to match
+ This option adds the "ct" expression that you can use to match
connection tracking information such as the flow state.
config NFT_SET_RBTREE
static __read_mostly bool nf_conntrack_locks_all;
/* every gc cycle scans at most 1/GC_MAX_BUCKETS_DIV part of table */
-#define GC_MAX_BUCKETS_DIV 64u
-/* upper bound of scan intervals */
-#define GC_INTERVAL_MAX (2 * HZ)
-/* maximum conntracks to evict per gc run */
-#define GC_MAX_EVICTS 256u
+#define GC_MAX_BUCKETS_DIV 128u
+/* upper bound of full table scan */
+#define GC_MAX_SCAN_JIFFIES (16u * HZ)
+/* desired ratio of entries found to be expired */
+#define GC_EVICT_RATIO 50u
static struct conntrack_gc_work conntrack_gc_work;
static void gc_worker(struct work_struct *work)
{
+ unsigned int min_interval = max(HZ / GC_MAX_BUCKETS_DIV, 1u);
unsigned int i, goal, buckets = 0, expired_count = 0;
struct conntrack_gc_work *gc_work;
unsigned int ratio, scanned = 0;
*/
rcu_read_unlock();
cond_resched_rcu_qs();
- } while (++buckets < goal &&
- expired_count < GC_MAX_EVICTS);
+ } while (++buckets < goal);
if (gc_work->exiting)
return;
* 1. Minimize time until we notice a stale entry
* 2. Maximize scan intervals to not waste cycles
*
- * Normally, expired_count will be 0, this increases the next_run time
- * to priorize 2) above.
+ * Normally, expire ratio will be close to 0.
*
- * As soon as a timed-out entry is found, move towards 1) and increase
- * the scan frequency.
- * In case we have lots of evictions next scan is done immediately.
+ * As soon as a sizeable fraction of the entries have expired
+ * increase scan frequency.
*/
ratio = scanned ? expired_count * 100 / scanned : 0;
- if (ratio >= 90 || expired_count == GC_MAX_EVICTS) {
- gc_work->next_gc_run = 0;
- next_run = 0;
- } else if (expired_count) {
- gc_work->next_gc_run /= 2U;
- next_run = msecs_to_jiffies(1);
+ if (ratio > GC_EVICT_RATIO) {
+ gc_work->next_gc_run = min_interval;
} else {
- if (gc_work->next_gc_run < GC_INTERVAL_MAX)
- gc_work->next_gc_run += msecs_to_jiffies(1);
+ unsigned int max = GC_MAX_SCAN_JIFFIES / GC_MAX_BUCKETS_DIV;
- next_run = gc_work->next_gc_run;
+ BUILD_BUG_ON((GC_MAX_SCAN_JIFFIES / GC_MAX_BUCKETS_DIV) == 0);
+
+ gc_work->next_gc_run += min_interval;
+ if (gc_work->next_gc_run > max)
+ gc_work->next_gc_run = max;
}
+ next_run = gc_work->next_gc_run;
gc_work->last_bucket = i;
queue_delayed_work(system_long_wq, &gc_work->dwork, next_run);
}
static void conntrack_gc_work_init(struct conntrack_gc_work *gc_work)
{
INIT_DELAYED_WORK(&gc_work->dwork, gc_worker);
- gc_work->next_gc_run = GC_INTERVAL_MAX;
+ gc_work->next_gc_run = HZ;
gc_work->exiting = false;
}
nf_ct_untracked_status_or(IPS_CONFIRMED | IPS_UNTRACKED);
conntrack_gc_work_init(&conntrack_gc_work);
- queue_delayed_work(system_long_wq, &conntrack_gc_work.dwork, GC_INTERVAL_MAX);
+ queue_delayed_work(system_long_wq, &conntrack_gc_work.dwork, HZ);
return 0;
/* Internal logging interface, which relies on the real
LOG target modules */
-#define NF_LOG_PREFIXLEN 128
#define NFLOGGER_NAME_LEN 64
static struct nf_logger __rcu *loggers[NFPROTO_NUMPROTO][NF_LOG_TYPE_MAX] __read_mostly;
}
static const struct nla_policy nft_chain_policy[NFTA_CHAIN_MAX + 1] = {
- [NFTA_CHAIN_TABLE] = { .type = NLA_STRING },
+ [NFTA_CHAIN_TABLE] = { .type = NLA_STRING,
+ .len = NFT_TABLE_MAXNAMELEN - 1 },
[NFTA_CHAIN_HANDLE] = { .type = NLA_U64 },
[NFTA_CHAIN_NAME] = { .type = NLA_STRING,
.len = NFT_CHAIN_MAXNAMELEN - 1 },
}
static const struct nla_policy nft_rule_policy[NFTA_RULE_MAX + 1] = {
- [NFTA_RULE_TABLE] = { .type = NLA_STRING },
+ [NFTA_RULE_TABLE] = { .type = NLA_STRING,
+ .len = NFT_TABLE_MAXNAMELEN - 1 },
[NFTA_RULE_CHAIN] = { .type = NLA_STRING,
.len = NFT_CHAIN_MAXNAMELEN - 1 },
[NFTA_RULE_HANDLE] = { .type = NLA_U64 },
}
static const struct nla_policy nft_set_policy[NFTA_SET_MAX + 1] = {
- [NFTA_SET_TABLE] = { .type = NLA_STRING },
+ [NFTA_SET_TABLE] = { .type = NLA_STRING,
+ .len = NFT_TABLE_MAXNAMELEN - 1 },
[NFTA_SET_NAME] = { .type = NLA_STRING,
.len = NFT_SET_MAXNAMELEN - 1 },
[NFTA_SET_FLAGS] = { .type = NLA_U32 },
}
static int nf_tables_bind_check_setelem(const struct nft_ctx *ctx,
- const struct nft_set *set,
+ struct nft_set *set,
const struct nft_set_iter *iter,
- const struct nft_set_elem *elem)
+ struct nft_set_elem *elem)
{
const struct nft_set_ext *ext = nft_set_elem_ext(set, elem->priv);
enum nft_registers dreg;
};
static const struct nla_policy nft_set_elem_list_policy[NFTA_SET_ELEM_LIST_MAX + 1] = {
- [NFTA_SET_ELEM_LIST_TABLE] = { .type = NLA_STRING },
- [NFTA_SET_ELEM_LIST_SET] = { .type = NLA_STRING },
+ [NFTA_SET_ELEM_LIST_TABLE] = { .type = NLA_STRING,
+ .len = NFT_TABLE_MAXNAMELEN - 1 },
+ [NFTA_SET_ELEM_LIST_SET] = { .type = NLA_STRING,
+ .len = NFT_SET_MAXNAMELEN - 1 },
[NFTA_SET_ELEM_LIST_ELEMENTS] = { .type = NLA_NESTED },
[NFTA_SET_ELEM_LIST_SET_ID] = { .type = NLA_U32 },
};
};
static int nf_tables_dump_setelem(const struct nft_ctx *ctx,
- const struct nft_set *set,
+ struct nft_set *set,
const struct nft_set_iter *iter,
- const struct nft_set_elem *elem)
+ struct nft_set_elem *elem)
{
struct nft_set_dump_args *args;
{
struct net *net = sock_net(skb->sk);
u8 genmask = nft_genmask_cur(net);
- const struct nft_set *set;
+ struct nft_set *set;
struct nft_set_dump_args args;
struct nft_ctx ctx;
struct nlattr *nla[NFTA_SET_ELEM_LIST_MAX + 1];
goto err5;
}
+ if (set->size &&
+ !atomic_add_unless(&set->nelems, 1, set->size + set->ndeact)) {
+ err = -ENFILE;
+ goto err6;
+ }
+
nft_trans_elem(trans) = elem;
list_add_tail(&trans->list, &ctx->net->nft.commit_list);
return 0;
+err6:
+ set->ops->remove(set, &elem);
err5:
kfree(trans);
err4:
return -EBUSY;
nla_for_each_nested(attr, nla[NFTA_SET_ELEM_LIST_ELEMENTS], rem) {
- if (set->size &&
- !atomic_add_unless(&set->nelems, 1, set->size + set->ndeact))
- return -ENFILE;
-
err = nft_add_set_elem(&ctx, set, attr, nlh->nlmsg_flags);
- if (err < 0) {
- atomic_dec(&set->nelems);
+ if (err < 0)
break;
- }
}
return err;
}
}
static int nft_flush_set(const struct nft_ctx *ctx,
- const struct nft_set *set,
+ struct nft_set *set,
const struct nft_set_iter *iter,
- const struct nft_set_elem *elem)
+ struct nft_set_elem *elem)
{
struct nft_trans *trans;
int err;
err = -ENOENT;
goto err1;
}
+ set->ndeact++;
- nft_trans_elem_set(trans) = (struct nft_set *)set;
- nft_trans_elem(trans) = *((struct nft_set_elem *)elem);
+ nft_trans_elem_set(trans) = set;
+ nft_trans_elem(trans) = *elem;
list_add_tail(&trans->list, &ctx->net->nft.commit_list);
return 0;
EXPORT_SYMBOL_GPL(nf_tables_obj_lookup);
static const struct nla_policy nft_obj_policy[NFTA_OBJ_MAX + 1] = {
- [NFTA_OBJ_TABLE] = { .type = NLA_STRING },
- [NFTA_OBJ_NAME] = { .type = NLA_STRING },
+ [NFTA_OBJ_TABLE] = { .type = NLA_STRING,
+ .len = NFT_TABLE_MAXNAMELEN - 1 },
+ [NFTA_OBJ_NAME] = { .type = NLA_STRING,
+ .len = NFT_OBJ_MAXNAMELEN - 1 },
[NFTA_OBJ_TYPE] = { .type = NLA_U32 },
[NFTA_OBJ_DATA] = { .type = NLA_NESTED },
};
if (idx > s_idx)
memset(&cb->args[1], 0,
sizeof(cb->args) - sizeof(cb->args[0]));
- if (filter->table[0] &&
+ if (filter && filter->table[0] &&
strcmp(filter->table, table->name))
goto cont;
- if (filter->type != NFT_OBJECT_UNSPEC &&
+ if (filter &&
+ filter->type != NFT_OBJECT_UNSPEC &&
obj->type->type != filter->type)
goto cont;
const struct nft_chain *chain);
static int nf_tables_loop_check_setelem(const struct nft_ctx *ctx,
- const struct nft_set *set,
+ struct nft_set *set,
const struct nft_set_iter *iter,
- const struct nft_set_elem *elem)
+ struct nft_set_elem *elem)
{
const struct nft_set_ext *ext = nft_set_elem_ext(set, elem->priv);
const struct nft_data *data;
{
const struct nft_rule *rule;
const struct nft_expr *expr, *last;
- const struct nft_set *set;
+ struct nft_set *set;
struct nft_set_binding *binding;
struct nft_set_iter iter;
}
static const struct nla_policy nft_dynset_policy[NFTA_DYNSET_MAX + 1] = {
- [NFTA_DYNSET_SET_NAME] = { .type = NLA_STRING },
+ [NFTA_DYNSET_SET_NAME] = { .type = NLA_STRING,
+ .len = NFT_SET_MAXNAMELEN - 1 },
[NFTA_DYNSET_SET_ID] = { .type = NLA_U32 },
[NFTA_DYNSET_OP] = { .type = NLA_U32 },
[NFTA_DYNSET_SREG_KEY] = { .type = NLA_U32 },
static const struct nla_policy nft_log_policy[NFTA_LOG_MAX + 1] = {
[NFTA_LOG_GROUP] = { .type = NLA_U16 },
- [NFTA_LOG_PREFIX] = { .type = NLA_STRING },
+ [NFTA_LOG_PREFIX] = { .type = NLA_STRING,
+ .len = NF_LOG_PREFIXLEN - 1 },
[NFTA_LOG_SNAPLEN] = { .type = NLA_U32 },
[NFTA_LOG_QTHRESHOLD] = { .type = NLA_U16 },
[NFTA_LOG_LEVEL] = { .type = NLA_U32 },
}
static const struct nla_policy nft_lookup_policy[NFTA_LOOKUP_MAX + 1] = {
- [NFTA_LOOKUP_SET] = { .type = NLA_STRING },
+ [NFTA_LOOKUP_SET] = { .type = NLA_STRING,
+ .len = NFT_SET_MAXNAMELEN - 1 },
[NFTA_LOOKUP_SET_ID] = { .type = NLA_U32 },
[NFTA_LOOKUP_SREG] = { .type = NLA_U32 },
[NFTA_LOOKUP_DREG] = { .type = NLA_U32 },
}
static const struct nla_policy nft_objref_policy[NFTA_OBJREF_MAX + 1] = {
- [NFTA_OBJREF_IMM_NAME] = { .type = NLA_STRING },
+ [NFTA_OBJREF_IMM_NAME] = { .type = NLA_STRING,
+ .len = NFT_OBJ_MAXNAMELEN - 1 },
[NFTA_OBJREF_IMM_TYPE] = { .type = NLA_U32 },
[NFTA_OBJREF_SET_SREG] = { .type = NLA_U32 },
- [NFTA_OBJREF_SET_NAME] = { .type = NLA_STRING },
+ [NFTA_OBJREF_SET_NAME] = { .type = NLA_STRING,
+ .len = NFT_SET_MAXNAMELEN - 1 },
[NFTA_OBJREF_SET_ID] = { .type = NLA_U32 },
};
rhashtable_remove_fast(&priv->ht, &he->node, nft_hash_params);
}
-static void nft_hash_walk(const struct nft_ctx *ctx, const struct nft_set *set,
+static void nft_hash_walk(const struct nft_ctx *ctx, struct nft_set *set,
struct nft_set_iter *iter)
{
struct nft_hash *priv = nft_set_priv(set);
}
static void nft_rbtree_walk(const struct nft_ctx *ctx,
- const struct nft_set *set,
+ struct nft_set *set,
struct nft_set_iter *iter)
{
const struct nft_rbtree *priv = nft_set_priv(set);
return -EINVAL;
*len -= sizeof(vnet_hdr);
- if (virtio_net_hdr_from_skb(skb, &vnet_hdr, vio_le()))
+ if (virtio_net_hdr_from_skb(skb, &vnet_hdr, vio_le(), true))
return -EINVAL;
return memcpy_to_msg(msg, (void *)&vnet_hdr, sizeof(vnet_hdr));
if (po->has_vnet_hdr) {
if (virtio_net_hdr_from_skb(skb, h.raw + macoff -
sizeof(struct virtio_net_hdr),
- vio_le())) {
+ vio_le(), true)) {
spin_lock(&sk->sk_receive_queue.lock);
goto drop_n_account;
}
struct virtio_net_hdr vnet_hdr = { 0 };
int offset = 0;
struct packet_sock *po = pkt_sk(sk);
- int hlen, tlen;
+ int hlen, tlen, linear;
int extra_len = 0;
/*
err = -ENOBUFS;
hlen = LL_RESERVED_SPACE(dev);
tlen = dev->needed_tailroom;
- skb = packet_alloc_skb(sk, hlen + tlen, hlen, len,
- __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len),
+ linear = __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len);
+ linear = max(linear, min_t(int, len, dev->hard_header_len));
+ skb = packet_alloc_skb(sk, hlen + tlen, hlen, len, linear,
msg->msg_flags & MSG_DONTWAIT, &err);
if (skb == NULL)
goto out_unlock;
&mask->icmp.type,
TCA_FLOWER_KEY_ICMPV6_TYPE_MASK,
sizeof(key->icmp.type));
- fl_set_key_val(tb, &key->icmp.code, TCA_FLOWER_KEY_ICMPV4_CODE,
+ fl_set_key_val(tb, &key->icmp.code, TCA_FLOWER_KEY_ICMPV6_CODE,
&mask->icmp.code,
- TCA_FLOWER_KEY_ICMPV4_CODE_MASK,
+ TCA_FLOWER_KEY_ICMPV6_CODE_MASK,
sizeof(key->icmp.code));
}
#include <net/sch_generic.h>
#include <net/pkt_cls.h>
-struct cls_mall_filter {
+struct cls_mall_head {
struct tcf_exts exts;
struct tcf_result res;
u32 handle;
- struct rcu_head rcu;
u32 flags;
-};
-
-struct cls_mall_head {
- struct cls_mall_filter *filter;
struct rcu_head rcu;
};
struct tcf_result *res)
{
struct cls_mall_head *head = rcu_dereference_bh(tp->root);
- struct cls_mall_filter *f = head->filter;
- if (tc_skip_sw(f->flags))
+ if (tc_skip_sw(head->flags))
return -1;
- return tcf_exts_exec(skb, &f->exts, res);
+ return tcf_exts_exec(skb, &head->exts, res);
}
static int mall_init(struct tcf_proto *tp)
{
- struct cls_mall_head *head;
-
- head = kzalloc(sizeof(*head), GFP_KERNEL);
- if (!head)
- return -ENOBUFS;
-
- rcu_assign_pointer(tp->root, head);
-
return 0;
}
-static void mall_destroy_filter(struct rcu_head *head)
+static void mall_destroy_rcu(struct rcu_head *rcu)
{
- struct cls_mall_filter *f = container_of(head, struct cls_mall_filter, rcu);
+ struct cls_mall_head *head = container_of(rcu, struct cls_mall_head,
+ rcu);
- tcf_exts_destroy(&f->exts);
-
- kfree(f);
+ tcf_exts_destroy(&head->exts);
+ kfree(head);
}
static int mall_replace_hw_filter(struct tcf_proto *tp,
- struct cls_mall_filter *f,
+ struct cls_mall_head *head,
unsigned long cookie)
{
struct net_device *dev = tp->q->dev_queue->dev;
offload.type = TC_SETUP_MATCHALL;
offload.cls_mall = &mall_offload;
offload.cls_mall->command = TC_CLSMATCHALL_REPLACE;
- offload.cls_mall->exts = &f->exts;
+ offload.cls_mall->exts = &head->exts;
offload.cls_mall->cookie = cookie;
return dev->netdev_ops->ndo_setup_tc(dev, tp->q->handle, tp->protocol,
}
static void mall_destroy_hw_filter(struct tcf_proto *tp,
- struct cls_mall_filter *f,
+ struct cls_mall_head *head,
unsigned long cookie)
{
struct net_device *dev = tp->q->dev_queue->dev;
{
struct cls_mall_head *head = rtnl_dereference(tp->root);
struct net_device *dev = tp->q->dev_queue->dev;
- struct cls_mall_filter *f = head->filter;
- if (!force && f)
- return false;
+ if (!head)
+ return true;
- if (f) {
- if (tc_should_offload(dev, tp, f->flags))
- mall_destroy_hw_filter(tp, f, (unsigned long) f);
+ if (tc_should_offload(dev, tp, head->flags))
+ mall_destroy_hw_filter(tp, head, (unsigned long) head);
- call_rcu(&f->rcu, mall_destroy_filter);
- }
- kfree_rcu(head, rcu);
+ call_rcu(&head->rcu, mall_destroy_rcu);
return true;
}
static unsigned long mall_get(struct tcf_proto *tp, u32 handle)
{
- struct cls_mall_head *head = rtnl_dereference(tp->root);
- struct cls_mall_filter *f = head->filter;
-
- if (f && f->handle == handle)
- return (unsigned long) f;
- return 0;
+ return 0UL;
}
static const struct nla_policy mall_policy[TCA_MATCHALL_MAX + 1] = {
};
static int mall_set_parms(struct net *net, struct tcf_proto *tp,
- struct cls_mall_filter *f,
+ struct cls_mall_head *head,
unsigned long base, struct nlattr **tb,
struct nlattr *est, bool ovr)
{
return err;
if (tb[TCA_MATCHALL_CLASSID]) {
- f->res.classid = nla_get_u32(tb[TCA_MATCHALL_CLASSID]);
- tcf_bind_filter(tp, &f->res, base);
+ head->res.classid = nla_get_u32(tb[TCA_MATCHALL_CLASSID]);
+ tcf_bind_filter(tp, &head->res, base);
}
- tcf_exts_change(tp, &f->exts, &e);
+ tcf_exts_change(tp, &head->exts, &e);
return 0;
}
unsigned long *arg, bool ovr)
{
struct cls_mall_head *head = rtnl_dereference(tp->root);
- struct cls_mall_filter *fold = (struct cls_mall_filter *) *arg;
struct net_device *dev = tp->q->dev_queue->dev;
- struct cls_mall_filter *f;
struct nlattr *tb[TCA_MATCHALL_MAX + 1];
+ struct cls_mall_head *new;
u32 flags = 0;
int err;
if (!tca[TCA_OPTIONS])
return -EINVAL;
- if (head->filter)
- return -EBUSY;
-
- if (fold)
- return -EINVAL;
+ if (head)
+ return -EEXIST;
err = nla_parse_nested(tb, TCA_MATCHALL_MAX,
tca[TCA_OPTIONS], mall_policy);
return -EINVAL;
}
- f = kzalloc(sizeof(*f), GFP_KERNEL);
- if (!f)
+ new = kzalloc(sizeof(*new), GFP_KERNEL);
+ if (!new)
return -ENOBUFS;
- tcf_exts_init(&f->exts, TCA_MATCHALL_ACT, 0);
+ tcf_exts_init(&new->exts, TCA_MATCHALL_ACT, 0);
if (!handle)
handle = 1;
- f->handle = handle;
- f->flags = flags;
+ new->handle = handle;
+ new->flags = flags;
- err = mall_set_parms(net, tp, f, base, tb, tca[TCA_RATE], ovr);
+ err = mall_set_parms(net, tp, new, base, tb, tca[TCA_RATE], ovr);
if (err)
goto errout;
if (tc_should_offload(dev, tp, flags)) {
- err = mall_replace_hw_filter(tp, f, (unsigned long) f);
+ err = mall_replace_hw_filter(tp, new, (unsigned long) new);
if (err) {
if (tc_skip_sw(flags))
goto errout;
}
}
- *arg = (unsigned long) f;
- rcu_assign_pointer(head->filter, f);
-
+ *arg = (unsigned long) head;
+ rcu_assign_pointer(tp->root, new);
+ if (head)
+ call_rcu(&head->rcu, mall_destroy_rcu);
return 0;
errout:
- kfree(f);
+ kfree(new);
return err;
}
static int mall_delete(struct tcf_proto *tp, unsigned long arg)
{
- struct cls_mall_head *head = rtnl_dereference(tp->root);
- struct cls_mall_filter *f = (struct cls_mall_filter *) arg;
- struct net_device *dev = tp->q->dev_queue->dev;
-
- if (tc_should_offload(dev, tp, f->flags))
- mall_destroy_hw_filter(tp, f, (unsigned long) f);
-
- RCU_INIT_POINTER(head->filter, NULL);
- tcf_unbind_filter(tp, &f->res);
- call_rcu(&f->rcu, mall_destroy_filter);
- return 0;
+ return -EOPNOTSUPP;
}
static void mall_walk(struct tcf_proto *tp, struct tcf_walker *arg)
{
struct cls_mall_head *head = rtnl_dereference(tp->root);
- struct cls_mall_filter *f = head->filter;
if (arg->count < arg->skip)
goto skip;
- if (arg->fn(tp, (unsigned long) f, arg) < 0)
+ if (arg->fn(tp, (unsigned long) head, arg) < 0)
arg->stop = 1;
skip:
arg->count++;
static int mall_dump(struct net *net, struct tcf_proto *tp, unsigned long fh,
struct sk_buff *skb, struct tcmsg *t)
{
- struct cls_mall_filter *f = (struct cls_mall_filter *) fh;
+ struct cls_mall_head *head = (struct cls_mall_head *) fh;
struct nlattr *nest;
- if (!f)
+ if (!head)
return skb->len;
- t->tcm_handle = f->handle;
+ t->tcm_handle = head->handle;
nest = nla_nest_start(skb, TCA_OPTIONS);
if (!nest)
goto nla_put_failure;
- if (f->res.classid &&
- nla_put_u32(skb, TCA_MATCHALL_CLASSID, f->res.classid))
+ if (head->res.classid &&
+ nla_put_u32(skb, TCA_MATCHALL_CLASSID, head->res.classid))
goto nla_put_failure;
- if (tcf_exts_dump(skb, &f->exts))
+ if (tcf_exts_dump(skb, &head->exts))
goto nla_put_failure;
nla_nest_end(skb, nest);
- if (tcf_exts_dump_stats(skb, &f->exts) < 0)
+ if (tcf_exts_dump_stats(skb, &head->exts) < 0)
goto nla_put_failure;
return skb->len;
SCTP_INC_STATS(sock_net(sk), SCTP_MIB_OUTSCTPPACKS);
rcu_read_lock();
- res = ip6_xmit(sk, skb, fl6, rcu_dereference(np->opt), np->tclass);
+ res = ip6_xmit(sk, skb, fl6, sk->sk_mark, rcu_dereference(np->opt),
+ np->tclass);
rcu_read_unlock();
return res;
}
goto out;
}
- segs = skb_segment(skb, features | NETIF_F_HW_CSUM);
+ segs = skb_segment(skb, features | NETIF_F_HW_CSUM | NETIF_F_SG);
if (IS_ERR(segs))
goto out;
sctp_assoc_t id)
{
struct sctp_association *addr_asoc = NULL, *id_asoc = NULL;
- struct sctp_transport *transport;
+ struct sctp_af *af = sctp_get_af_specific(addr->ss_family);
union sctp_addr *laddr = (union sctp_addr *)addr;
+ struct sctp_transport *transport;
+
+ if (!af || sctp_verify_addr(sk, laddr, af->sockaddr_len))
+ return NULL;
addr_asoc = sctp_endpoint_lookup_assoc(sctp_sk(sk)->ep,
laddr,
*/
release_sock(sk);
current_timeo = schedule_timeout(current_timeo);
- BUG_ON(sk != asoc->base.sk);
+ if (sk != asoc->base.sk)
+ goto do_error;
lock_sock(sk);
*timeo_p = current_timeo;
if (!oa->data)
return -ENOMEM;
- creds = kmalloc(sizeof(struct svc_cred), GFP_KERNEL);
+ creds = kzalloc(sizeof(struct svc_cred), GFP_KERNEL);
if (!creds) {
kfree(oa->data);
return -ENOMEM;
static DEFINE_IDA(rpc_clids);
+void rpc_cleanup_clids(void)
+{
+ ida_destroy(&rpc_clids);
+}
+
static int rpc_alloc_clid(struct rpc_clnt *clnt)
{
int clid;
static void __exit
cleanup_sunrpc(void)
{
+ rpc_cleanup_clids();
rpcauth_remove_module();
cleanup_socket_xprt();
svc_cleanup_xprt_sock();
write_lock_bh(&n->lock);
}
+static void tipc_node_write_unlock_fast(struct tipc_node *n)
+{
+ write_unlock_bh(&n->lock);
+}
+
static void tipc_node_write_unlock(struct tipc_node *n)
{
struct net *net = n->net;
}
tipc_node_write_lock(n);
list_add_tail(subscr, &n->publ_list);
- tipc_node_write_unlock(n);
+ tipc_node_write_unlock_fast(n);
tipc_node_put(n);
}
}
tipc_node_write_lock(n);
list_del_init(subscr);
- tipc_node_write_unlock(n);
+ tipc_node_write_unlock_fast(n);
tipc_node_put(n);
}
static void tipc_recv_work(struct work_struct *work);
static void tipc_send_work(struct work_struct *work);
static void tipc_clean_outqueues(struct tipc_conn *con);
-static void tipc_sock_release(struct tipc_conn *con);
static void tipc_conn_kref_release(struct kref *kref)
{
struct tipc_conn *con = container_of(kref, struct tipc_conn, kref);
- struct sockaddr_tipc *saddr = con->server->saddr;
+ struct tipc_server *s = con->server;
+ struct sockaddr_tipc *saddr = s->saddr;
struct socket *sock = con->sock;
struct sock *sk;
}
saddr->scope = -TIPC_NODE_SCOPE;
kernel_bind(sock, (struct sockaddr *)saddr, sizeof(*saddr));
- tipc_sock_release(con);
sock_release(sock);
con->sock = NULL;
+
+ spin_lock_bh(&s->idr_lock);
+ idr_remove(&s->conn_idr, con->conid);
+ s->idr_in_use--;
+ spin_unlock_bh(&s->idr_lock);
}
tipc_clean_outqueues(con);
spin_lock_bh(&s->idr_lock);
con = idr_find(&s->conn_idr, conid);
- if (con)
+ if (con && test_bit(CF_CONNECTED, &con->flags))
conn_get(con);
+ else
+ con = NULL;
spin_unlock_bh(&s->idr_lock);
return con;
}
write_unlock_bh(&sk->sk_callback_lock);
}
-static void tipc_sock_release(struct tipc_conn *con)
-{
- struct tipc_server *s = con->server;
-
- if (con->conid)
- s->tipc_conn_release(con->conid, con->usr_data);
-
- tipc_unregister_callbacks(con);
-}
-
static void tipc_close_conn(struct tipc_conn *con)
{
struct tipc_server *s = con->server;
if (test_and_clear_bit(CF_CONNECTED, &con->flags)) {
+ tipc_unregister_callbacks(con);
- spin_lock_bh(&s->idr_lock);
- idr_remove(&s->conn_idr, con->conid);
- s->idr_in_use--;
- spin_unlock_bh(&s->idr_lock);
+ if (con->conid)
+ s->tipc_conn_release(con->conid, con->usr_data);
/* We shouldn't flush pending works as we may be in the
* thread. In fact the races with pending rx/tx work structs
if (!con)
return -EINVAL;
+ if (!test_bit(CF_CONNECTED, &con->flags)) {
+ conn_put(con);
+ return 0;
+ }
+
e = tipc_alloc_entry(data, len);
if (!e) {
conn_put(con);
list_add_tail(&e->list, &con->outqueue);
spin_unlock_bh(&con->outqueue_lock);
- if (test_bit(CF_CONNECTED, &con->flags)) {
- if (!queue_work(s->send_wq, &con->swork))
- conn_put(con);
- } else {
+ if (!queue_work(s->send_wq, &con->swork))
conn_put(con);
- }
return 0;
}
int ret;
spin_lock_bh(&con->outqueue_lock);
- while (1) {
+ while (test_bit(CF_CONNECTED, &con->flags)) {
e = list_entry(con->outqueue.next, struct outqueue_entry,
list);
if ((struct list_head *) e == &con->outqueue)
void tipc_server_stop(struct tipc_server *s)
{
struct tipc_conn *con;
- int total = 0;
int id;
spin_lock_bh(&s->idr_lock);
- for (id = 0; total < s->idr_in_use; id++) {
+ for (id = 0; s->idr_in_use; id++) {
con = idr_find(&s->conn_idr, id);
if (con) {
- total++;
spin_unlock_bh(&s->idr_lock);
tipc_close_conn(con);
spin_lock_bh(&s->idr_lock);
static void tipc_subscrp_delete(struct tipc_subscription *sub);
static void tipc_subscrb_put(struct tipc_subscriber *subscriber);
+static void tipc_subscrp_put(struct tipc_subscription *subscription);
+static void tipc_subscrp_get(struct tipc_subscription *subscription);
/**
* htohl - convert value to endianness used by destination
{
struct tipc_name_seq seq;
+ tipc_subscrp_get(sub);
tipc_subscrp_convert_seq(&sub->evt.s.seq, sub->swap, &seq);
if (!tipc_subscrp_check_overlap(&seq, found_lower, found_upper))
return;
tipc_subscrp_send_event(sub, found_lower, found_upper, event, port_ref,
node);
+ tipc_subscrp_put(sub);
}
static void tipc_subscrp_timeout(unsigned long data)
{
struct tipc_subscription *sub = (struct tipc_subscription *)data;
- struct tipc_subscriber *subscriber = sub->subscriber;
/* Notify subscriber of timeout */
tipc_subscrp_send_event(sub, sub->evt.s.seq.lower, sub->evt.s.seq.upper,
TIPC_SUBSCR_TIMEOUT, 0, 0);
- spin_lock_bh(&subscriber->lock);
- tipc_subscrp_delete(sub);
- spin_unlock_bh(&subscriber->lock);
-
- tipc_subscrb_put(subscriber);
+ tipc_subscrp_put(sub);
}
static void tipc_subscrb_kref_release(struct kref *kref)
{
- struct tipc_subscriber *subcriber = container_of(kref,
- struct tipc_subscriber, kref);
-
- kfree(subcriber);
+ kfree(container_of(kref,struct tipc_subscriber, kref));
}
static void tipc_subscrb_put(struct tipc_subscriber *subscriber)
kref_get(&subscriber->kref);
}
+static void tipc_subscrp_kref_release(struct kref *kref)
+{
+ struct tipc_subscription *sub = container_of(kref,
+ struct tipc_subscription,
+ kref);
+ struct tipc_net *tn = net_generic(sub->net, tipc_net_id);
+ struct tipc_subscriber *subscriber = sub->subscriber;
+
+ spin_lock_bh(&subscriber->lock);
+ tipc_nametbl_unsubscribe(sub);
+ list_del(&sub->subscrp_list);
+ atomic_dec(&tn->subscription_count);
+ spin_unlock_bh(&subscriber->lock);
+ kfree(sub);
+ tipc_subscrb_put(subscriber);
+}
+
+static void tipc_subscrp_put(struct tipc_subscription *subscription)
+{
+ kref_put(&subscription->kref, tipc_subscrp_kref_release);
+}
+
+static void tipc_subscrp_get(struct tipc_subscription *subscription)
+{
+ kref_get(&subscription->kref);
+}
+
+/* tipc_subscrb_subscrp_delete - delete a specific subscription or all
+ * subscriptions for a given subscriber.
+ */
+static void tipc_subscrb_subscrp_delete(struct tipc_subscriber *subscriber,
+ struct tipc_subscr *s)
+{
+ struct list_head *subscription_list = &subscriber->subscrp_list;
+ struct tipc_subscription *sub, *temp;
+
+ spin_lock_bh(&subscriber->lock);
+ list_for_each_entry_safe(sub, temp, subscription_list, subscrp_list) {
+ if (s && memcmp(s, &sub->evt.s, sizeof(struct tipc_subscr)))
+ continue;
+
+ tipc_subscrp_get(sub);
+ spin_unlock_bh(&subscriber->lock);
+ tipc_subscrp_delete(sub);
+ tipc_subscrp_put(sub);
+ spin_lock_bh(&subscriber->lock);
+
+ if (s)
+ break;
+ }
+ spin_unlock_bh(&subscriber->lock);
+}
+
static struct tipc_subscriber *tipc_subscrb_create(int conid)
{
struct tipc_subscriber *subscriber;
pr_warn("Subscriber rejected, no memory\n");
return NULL;
}
- kref_init(&subscriber->kref);
INIT_LIST_HEAD(&subscriber->subscrp_list);
+ kref_init(&subscriber->kref);
subscriber->conid = conid;
spin_lock_init(&subscriber->lock);
static void tipc_subscrb_delete(struct tipc_subscriber *subscriber)
{
- struct tipc_subscription *sub, *temp;
- u32 timeout;
-
- spin_lock_bh(&subscriber->lock);
- /* Destroy any existing subscriptions for subscriber */
- list_for_each_entry_safe(sub, temp, &subscriber->subscrp_list,
- subscrp_list) {
- timeout = htohl(sub->evt.s.timeout, sub->swap);
- if ((timeout == TIPC_WAIT_FOREVER) || del_timer(&sub->timer)) {
- tipc_subscrp_delete(sub);
- tipc_subscrb_put(subscriber);
- }
- }
- spin_unlock_bh(&subscriber->lock);
-
+ tipc_subscrb_subscrp_delete(subscriber, NULL);
tipc_subscrb_put(subscriber);
}
static void tipc_subscrp_delete(struct tipc_subscription *sub)
{
- struct tipc_net *tn = net_generic(sub->net, tipc_net_id);
+ u32 timeout = htohl(sub->evt.s.timeout, sub->swap);
- tipc_nametbl_unsubscribe(sub);
- list_del(&sub->subscrp_list);
- kfree(sub);
- atomic_dec(&tn->subscription_count);
+ if (timeout == TIPC_WAIT_FOREVER || del_timer(&sub->timer))
+ tipc_subscrp_put(sub);
}
static void tipc_subscrp_cancel(struct tipc_subscr *s,
struct tipc_subscriber *subscriber)
{
- struct tipc_subscription *sub, *temp;
- u32 timeout;
-
- spin_lock_bh(&subscriber->lock);
- /* Find first matching subscription, exit if not found */
- list_for_each_entry_safe(sub, temp, &subscriber->subscrp_list,
- subscrp_list) {
- if (!memcmp(s, &sub->evt.s, sizeof(struct tipc_subscr))) {
- timeout = htohl(sub->evt.s.timeout, sub->swap);
- if ((timeout == TIPC_WAIT_FOREVER) ||
- del_timer(&sub->timer)) {
- tipc_subscrp_delete(sub);
- tipc_subscrb_put(subscriber);
- }
- break;
- }
- }
- spin_unlock_bh(&subscriber->lock);
+ tipc_subscrb_subscrp_delete(subscriber, s);
}
static struct tipc_subscription *tipc_subscrp_create(struct net *net,
sub->swap = swap;
memcpy(&sub->evt.s, s, sizeof(*s));
atomic_inc(&tn->subscription_count);
+ kref_init(&sub->kref);
return sub;
}
spin_lock_bh(&subscriber->lock);
list_add(&sub->subscrp_list, &subscriber->subscrp_list);
- tipc_subscrb_get(subscriber);
sub->subscriber = subscriber;
tipc_nametbl_subscribe(sub);
+ tipc_subscrb_get(subscriber);
spin_unlock_bh(&subscriber->lock);
+ setup_timer(&sub->timer, tipc_subscrp_timeout, (unsigned long)sub);
timeout = htohl(sub->evt.s.timeout, swap);
- if (timeout == TIPC_WAIT_FOREVER)
- return;
- setup_timer(&sub->timer, tipc_subscrp_timeout, (unsigned long)sub);
- mod_timer(&sub->timer, jiffies + msecs_to_jiffies(timeout));
+ if (timeout != TIPC_WAIT_FOREVER)
+ mod_timer(&sub->timer, jiffies + msecs_to_jiffies(timeout));
}
/* Handle one termination request for the subscriber */
* @evt: template for events generated by subscription
*/
struct tipc_subscription {
+ struct kref kref;
struct tipc_subscriber *subscriber;
struct net *net;
struct timer_list timer;
unsigned int hash;
struct unix_address *addr;
struct hlist_head *list;
+ struct path path = { NULL, NULL };
err = -EINVAL;
if (sunaddr->sun_family != AF_UNIX)
goto out;
addr_len = err;
+ if (sun_path[0]) {
+ umode_t mode = S_IFSOCK |
+ (SOCK_INODE(sock)->i_mode & ~current_umask());
+ err = unix_mknod(sun_path, mode, &path);
+ if (err) {
+ if (err == -EEXIST)
+ err = -EADDRINUSE;
+ goto out;
+ }
+ }
+
err = mutex_lock_interruptible(&u->bindlock);
if (err)
- goto out;
+ goto out_put;
err = -EINVAL;
if (u->addr)
atomic_set(&addr->refcnt, 1);
if (sun_path[0]) {
- struct path path;
- umode_t mode = S_IFSOCK |
- (SOCK_INODE(sock)->i_mode & ~current_umask());
- err = unix_mknod(sun_path, mode, &path);
- if (err) {
- if (err == -EEXIST)
- err = -EADDRINUSE;
- unix_release_addr(addr);
- goto out_up;
- }
addr->hash = UNIX_HASH_SIZE;
hash = d_backing_inode(path.dentry)->i_ino & (UNIX_HASH_SIZE - 1);
spin_lock(&unix_table_lock);
spin_unlock(&unix_table_lock);
out_up:
mutex_unlock(&u->bindlock);
+out_put:
+ if (err)
+ path_put(&path);
out:
return err;
}
break;
}
cfg->ht_opmode = ht_opmode;
+ mask |= (1 << (NL80211_MESHCONF_HT_OPMODE - 1));
}
FILL_IN_MESH_PARAM_IF_SET(tb, cfg, dot11MeshHWMPactivePathToRootTimeout,
1, 65535, mask,
* modify it under the terms of version 2 of the GNU General Public
* License as published by the Free Software Foundation.
*/
+#define KBUILD_MODNAME "foo"
#include <uapi/linux/bpf.h>
#include <uapi/linux/if_ether.h>
#include <uapi/linux/if_packet.h>
* encapsulating the incoming packet in an IPv4/v6 header
* and then XDP_TX it out.
*/
+#define KBUILD_MODNAME "foo"
#include <uapi/linux/bpf.h>
#include <linux/in.h>
#include <linux/if_ether.h>
$(CPP) -D__GENKSYMS__ $(c_flags) $< | \
$(GENKSYMS) $(if $(1), -T $(2)) \
$(patsubst y,-s _,$(CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX)) \
+ $(patsubst y,-R,$(CONFIG_MODULE_REL_CRCS)) \
$(if $(KBUILD_PRESERVE),-p) \
-r $(firstword $(wildcard $(2:.symtypes=.symref) /dev/null))
$(CPP) -D__GENKSYMS__ $(c_flags) -xc - | \
$(GENKSYMS) $(if $(1), -T $(2)) \
$(patsubst y,-s _,$(CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX)) \
+ $(patsubst y,-R,$(CONFIG_MODULE_REL_CRCS)) \
$(if $(KBUILD_PRESERVE),-p) \
-r $(firstword $(wildcard $(2:.symtypes=.symref) /dev/null))
int in_source_file;
static int flag_debug, flag_dump_defs, flag_reference, flag_dump_types,
- flag_preserve, flag_warnings;
+ flag_preserve, flag_warnings, flag_rel_crcs;
static const char *mod_prefix = "";
static int errors;
fputs(">\n", debugfile);
/* Used as a linker script. */
- printf("%s__crc_%s = 0x%08lx ;\n", mod_prefix, name, crc);
+ printf(!flag_rel_crcs ? "%s__crc_%s = 0x%08lx;\n" :
+ "SECTIONS { .rodata : ALIGN(4) { "
+ "%s__crc_%s = .; LONG(0x%08lx); } }\n",
+ mod_prefix, name, crc);
}
}
static void genksyms_usage(void)
{
- fputs("Usage:\n" "genksyms [-adDTwqhV] > /path/to/.tmp_obj.ver\n" "\n"
+ fputs("Usage:\n" "genksyms [-adDTwqhVR] > /path/to/.tmp_obj.ver\n" "\n"
#ifdef __GNU_LIBRARY__
" -s, --symbol-prefix Select symbol prefix\n"
" -d, --debug Increment the debug level (repeatable)\n"
" -q, --quiet Disable warnings (default)\n"
" -h, --help Print this message\n"
" -V, --version Print the release version\n"
+ " -R, --relative-crc Emit section relative symbol CRCs\n"
#else /* __GNU_LIBRARY__ */
" -s Select symbol prefix\n"
" -d Increment the debug level (repeatable)\n"
" -q Disable warnings (default)\n"
" -h Print this message\n"
" -V Print the release version\n"
+ " -R Emit section relative symbol CRCs\n"
#endif /* __GNU_LIBRARY__ */
, stderr);
}
{"preserve", 0, 0, 'p'},
{"version", 0, 0, 'V'},
{"help", 0, 0, 'h'},
+ {"relative-crc", 0, 0, 'R'},
{0, 0, 0, 0}
};
- while ((o = getopt_long(argc, argv, "s:dwqVDr:T:ph",
+ while ((o = getopt_long(argc, argv, "s:dwqVDr:T:phR",
&long_opts[0], NULL)) != EOF)
#else /* __GNU_LIBRARY__ */
- while ((o = getopt(argc, argv, "s:dwqVDr:T:ph")) != EOF)
+ while ((o = getopt(argc, argv, "s:dwqVDr:T:phR")) != EOF)
#endif /* __GNU_LIBRARY__ */
switch (o) {
case 's':
case 'h':
genksyms_usage();
return 0;
+ case 'R':
+ flag_rel_crcs = 1;
+ break;
default:
genksyms_usage();
return 1;
"_SDA2_BASE_", /* ppc */
NULL };
+ static char *special_prefixes[] = {
+ "__crc_", /* modversions */
+ NULL };
+
static char *special_suffixes[] = {
"_veneer", /* arm */
"_from_arm", /* arm */
if (strcmp(sym_name, special_symbols[i]) == 0)
return 0;
+ for (i = 0; special_prefixes[i]; i++) {
+ int l = strlen(special_prefixes[i]);
+
+ if (l <= strlen(sym_name) &&
+ strncmp(sym_name, special_prefixes[i], l) == 0)
+ return 0;
+ }
+
for (i = 0; special_suffixes[i]; i++) {
int l = strlen(sym_name) - strlen(special_suffixes[i]);
if (strncmp(symname, CRC_PFX, strlen(CRC_PFX)) == 0) {
is_crc = true;
crc = (unsigned int) sym->st_value;
+ if (sym->st_shndx != SHN_UNDEF && sym->st_shndx != SHN_ABS) {
+ unsigned int *crcp;
+
+ /* symbol points to the CRC in the ELF object */
+ crcp = (void *)info->hdr + sym->st_value +
+ info->sechdrs[sym->st_shndx].sh_offset -
+ (info->hdr->e_type != ET_REL ?
+ info->sechdrs[sym->st_shndx].sh_addr : 0);
+ crc = *crcp;
+ }
sym_update_crc(symname + strlen(CRC_PFX), mod, crc,
export);
}
return error;
/* Obtain a SID for the context, if one was specified. */
- if (size && str[1] && str[1] != '\n') {
+ if (size && str[0] && str[0] != '\n') {
if (str[size-1] == '\n') {
str[size-1] = 0;
size--;
{
unsigned long flags;
struct snd_seq_event_cell *ptr;
- int max_count = 5 * HZ;
if (snd_BUG_ON(!pool))
return -EINVAL;
if (waitqueue_active(&pool->output_sleep))
wake_up(&pool->output_sleep);
- while (atomic_read(&pool->counter) > 0) {
- if (max_count == 0) {
- pr_warn("ALSA: snd_seq_pool_done timeout: %d cells remain\n", atomic_read(&pool->counter));
- break;
- }
+ while (atomic_read(&pool->counter) > 0)
schedule_timeout_uninterruptible(1);
- max_count--;
- }
/* release all resources */
spin_lock_irqsave(&pool->lock, flags);
}
}
+static void queue_use(struct snd_seq_queue *queue, int client, int use);
+
/* allocate a new queue -
* return queue index value or negative value for error
*/
if (q == NULL)
return -ENOMEM;
q->info_flags = info_flags;
+ queue_use(q, client, 1);
if (queue_list_add(q) < 0) {
queue_delete(q);
return -ENOMEM;
}
- snd_seq_queue_use(q->queue, client, 1); /* use this queue */
return q->queue;
}
return result;
}
-
-/* use or unuse this queue -
- * if it is the first client, starts the timer.
- * if it is not longer used by any clients, stop the timer.
- */
-int snd_seq_queue_use(int queueid, int client, int use)
+/* use or unuse this queue */
+static void queue_use(struct snd_seq_queue *queue, int client, int use)
{
- struct snd_seq_queue *queue;
-
- queue = queueptr(queueid);
- if (queue == NULL)
- return -EINVAL;
- mutex_lock(&queue->timer_mutex);
if (use) {
if (!test_and_set_bit(client, queue->clients_bitmap))
queue->clients++;
} else {
snd_seq_timer_close(queue);
}
+}
+
+/* use or unuse this queue -
+ * if it is the first client, starts the timer.
+ * if it is not longer used by any clients, stop the timer.
+ */
+int snd_seq_queue_use(int queueid, int client, int use)
+{
+ struct snd_seq_queue *queue;
+
+ queue = queueptr(queueid);
+ if (queue == NULL)
+ return -EINVAL;
+ mutex_lock(&queue->timer_mutex);
+ queue_use(queue, client, use);
mutex_unlock(&queue->timer_mutex);
queuefree(queue);
return 0;
HDA_CODEC_ENTRY(0x10de0071, "GPU 71 HDMI/DP", patch_nvhdmi),
HDA_CODEC_ENTRY(0x10de0072, "GPU 72 HDMI/DP", patch_nvhdmi),
HDA_CODEC_ENTRY(0x10de007d, "GPU 7d HDMI/DP", patch_nvhdmi),
+HDA_CODEC_ENTRY(0x10de0080, "GPU 80 HDMI/DP", patch_nvhdmi),
HDA_CODEC_ENTRY(0x10de0082, "GPU 82 HDMI/DP", patch_nvhdmi),
HDA_CODEC_ENTRY(0x10de0083, "GPU 83 HDMI/DP", patch_nvhdmi),
HDA_CODEC_ENTRY(0x10de8001, "MCP73 HDMI", patch_nvhdmi_2ch),
goto error;
}
+ line6_get_interval(line6);
+
if (properties->capabilities & LINE6_CAP_CONTROL) {
- line6_get_interval(line6);
ret = line6_init_cap_control(line6);
if (ret < 0)
goto error;
*type = INSN_RETURN;
break;
- case 0xc5: /* iret */
case 0xca: /* retf */
case 0xcb: /* retf */
+ case 0xcf: /* iret */
*type = INSN_CONTEXT_SWITCH;
break;
BUG_ON(1);
}
- perf_hpp__register_sort_field(fmt);
+ perf_hpp__prepend_sort_field(fmt);
return 0;
}
list_add_tail(&format->sort_list, &list->sorts);
}
+void perf_hpp_list__prepend_sort_field(struct perf_hpp_list *list,
+ struct perf_hpp_fmt *format)
+{
+ list_add(&format->sort_list, &list->sorts);
+}
+
void perf_hpp__column_unregister(struct perf_hpp_fmt *format)
{
list_del(&format->list);
perf_hpp_list__for_each_sort_list(list, fmt) {
struct perf_hpp_fmt *pos;
+ /* skip sort-only fields ("sort_compute" in perf diff) */
+ if (!fmt->entry && !fmt->color)
+ continue;
+
perf_hpp_list__for_each_format(list, pos) {
if (fmt_equal(fmt, pos))
goto next;
}
call->ip = cursor_node->ip;
call->ms.sym = cursor_node->sym;
- call->ms.map = cursor_node->map;
+ call->ms.map = map__get(cursor_node->map);
if (cursor_node->branch) {
call->branch_count = 1;
list_for_each_entry_safe(call, tmp, &new->val, list) {
list_del(&call->list);
+ map__zput(call->ms.map);
free(call);
}
free(new);
list->ms.map, list->ms.sym,
false, NULL, 0, 0);
list_del(&list->list);
+ map__zput(list->ms.map);
free(list);
}
}
node->ip = ip;
- node->map = map;
+ map__zput(node->map);
+ node->map = map__get(map);
node->sym = sym;
node->branch = branch;
node->nr_loop_iter = nr_loop_iter;
list_for_each_entry_safe(list, tmp, &node->parent_val, list) {
list_del(&list->list);
+ map__zput(list->ms.map);
free(list);
}
list_for_each_entry_safe(list, tmp, &node->val, list) {
list_del(&list->list);
+ map__zput(list->ms.map);
free(list);
}
goto out;
*new = *chain;
new->has_children = false;
+ map__get(new->ms.map);
list_add_tail(&new->list, &head);
}
parent = parent->parent;
out:
list_for_each_entry_safe(chain, new, &head, list) {
list_del(&chain->list);
+ map__zput(chain->ms.map);
free(chain);
}
return -ENOMEM;
#include <linux/list.h>
#include <linux/rbtree.h>
#include "event.h"
+#include "map.h"
#include "symbol.h"
#define HELP_PAD "\t\t\t\t"
*/
static inline void callchain_cursor_reset(struct callchain_cursor *cursor)
{
+ struct callchain_cursor_node *node;
+
cursor->nr = 0;
cursor->last = &cursor->first;
+
+ for (node = cursor->first; node != NULL; node = node->next)
+ map__zput(node->map);
}
int callchain_cursor_append(struct callchain_cursor *cursor, u64 ip,
#include "util.h"
#include "build-id.h"
#include "hist.h"
+#include "map.h"
#include "session.h"
#include "sort.h"
#include "evlist.h"
int max_stack_depth, void *arg)
{
int err, err2;
+ struct map *alm = NULL;
+
+ if (al && al->map)
+ alm = map__get(al->map);
err = sample__resolve_callchain(iter->sample, &callchain_cursor, &iter->parent,
iter->evsel, al, max_stack_depth);
if (!err)
err = err2;
+ map__put(alm);
+
return err;
}
struct perf_hpp_fmt *format);
void perf_hpp_list__register_sort_field(struct perf_hpp_list *list,
struct perf_hpp_fmt *format);
+void perf_hpp_list__prepend_sort_field(struct perf_hpp_list *list,
+ struct perf_hpp_fmt *format);
static inline void perf_hpp__column_register(struct perf_hpp_fmt *format)
{
perf_hpp_list__register_sort_field(&perf_hpp_list, format);
}
+static inline void perf_hpp__prepend_sort_field(struct perf_hpp_fmt *format)
+{
+ perf_hpp_list__prepend_sort_field(&perf_hpp_list, format);
+}
+
#define perf_hpp_list__for_each_format(_list, format) \
list_for_each_entry(format, &(_list)->fields, list)
return map_subset(lru_map, expected) && map_subset(expected, lru_map);
}
-static int sched_next_online(int pid, int next_to_try)
+static int sched_next_online(int pid, int *next_to_try)
{
cpu_set_t cpuset;
+ int next = *next_to_try;
+ int ret = -1;
- if (next_to_try == nr_cpus)
- return -1;
-
- while (next_to_try < nr_cpus) {
+ while (next < nr_cpus) {
CPU_ZERO(&cpuset);
- CPU_SET(next_to_try++, &cpuset);
- if (!sched_setaffinity(pid, sizeof(cpuset), &cpuset))
+ CPU_SET(next++, &cpuset);
+ if (!sched_setaffinity(pid, sizeof(cpuset), &cpuset)) {
+ ret = 0;
break;
+ }
}
- return next_to_try;
+ *next_to_try = next;
+ return ret;
}
/* Size of the LRU amp is 2
{
unsigned long long key, value[nr_cpus];
int lru_map_fd, expected_map_fd;
+ int next_cpu = 0;
printf("%s (map_type:%d map_flags:0x%X): ", __func__, map_type,
map_flags);
- assert(sched_next_online(0, 0) != -1);
+ assert(sched_next_online(0, &next_cpu) != -1);
if (map_flags & BPF_F_NO_COMMON_LRU)
lru_map_fd = create_map(map_type, map_flags, 2 * nr_cpus);
int lru_map_fd, expected_map_fd;
unsigned int batch_size;
unsigned int map_size;
+ int next_cpu = 0;
if (map_flags & BPF_F_NO_COMMON_LRU)
/* Ther percpu lru list (i.e each cpu has its own LRU
printf("%s (map_type:%d map_flags:0x%X): ", __func__, map_type,
map_flags);
- assert(sched_next_online(0, 0) != -1);
+ assert(sched_next_online(0, &next_cpu) != -1);
batch_size = tgt_free / 2;
assert(batch_size * 2 == tgt_free);
int lru_map_fd, expected_map_fd;
unsigned int batch_size;
unsigned int map_size;
+ int next_cpu = 0;
if (map_flags & BPF_F_NO_COMMON_LRU)
/* Ther percpu lru list (i.e each cpu has its own LRU
printf("%s (map_type:%d map_flags:0x%X): ", __func__, map_type,
map_flags);
- assert(sched_next_online(0, 0) != -1);
+ assert(sched_next_online(0, &next_cpu) != -1);
batch_size = tgt_free / 2;
assert(batch_size * 2 == tgt_free);
int lru_map_fd, expected_map_fd;
unsigned int batch_size;
unsigned int map_size;
+ int next_cpu = 0;
printf("%s (map_type:%d map_flags:0x%X): ", __func__, map_type,
map_flags);
- assert(sched_next_online(0, 0) != -1);
+ assert(sched_next_online(0, &next_cpu) != -1);
batch_size = tgt_free / 2;
assert(batch_size * 2 == tgt_free);
int lru_map_fd, expected_map_fd;
unsigned long long key, value[nr_cpus];
unsigned long long end_key;
+ int next_cpu = 0;
printf("%s (map_type:%d map_flags:0x%X): ", __func__, map_type,
map_flags);
- assert(sched_next_online(0, 0) != -1);
+ assert(sched_next_online(0, &next_cpu) != -1);
if (map_flags & BPF_F_NO_COMMON_LRU)
lru_map_fd = create_map(map_type, map_flags,
static void test_lru_sanity5(int map_type, int map_flags)
{
unsigned long long key, value[nr_cpus];
- int next_sched_cpu = 0;
+ int next_cpu = 0;
int map_fd;
- int i;
if (map_flags & BPF_F_NO_COMMON_LRU)
return;
key = 0;
assert(!bpf_map_update(map_fd, &key, value, BPF_NOEXIST));
- for (i = 0; i < nr_cpus; i++) {
+ while (sched_next_online(0, &next_cpu) != -1) {
pid_t pid;
pid = fork();
if (pid == 0) {
- next_sched_cpu = sched_next_online(0, next_sched_cpu);
- if (next_sched_cpu != -1)
- do_test_lru_sanity5(key, map_fd);
+ do_test_lru_sanity5(key, map_fd);
exit(0);
} else if (pid == -1) {
- printf("couldn't spawn #%d process\n", i);
+ printf("couldn't spawn process to test key:%llu\n",
+ key);
exit(1);
} else {
int status;
- /* It is mostly redundant and just allow the parent
- * process to update next_shced_cpu for the next child
- * process
- */
- next_sched_cpu = sched_next_online(pid, next_sched_cpu);
-
assert(waitpid(pid, &status, 0) == pid);
assert(status == 0);
key++;
}
close(map_fd);
+ /* At least one key should be tested */
+ assert(key > 0);
printf("Pass\n");
}
projects / linux-2.6-microblaze.git / commitdiff