Andrew Murray <amurray@thegoodpenguin.co.uk> <amurray@embedded-bits.co.uk>
Andrew Murray <amurray@thegoodpenguin.co.uk> <andrew.murray@arm.com>
Andrew Vasquez <andrew.vasquez@qlogic.com>
+Andrey Konovalov <andreyknvl@gmail.com> <andreyknvl@google.com>
Andrey Ryabinin <ryabinin.a.a@gmail.com> <a.ryabinin@samsung.com>
Andrey Ryabinin <ryabinin.a.a@gmail.com> <aryabinin@virtuozzo.com>
Andy Adamson <andros@citi.umich.edu>
Changbin Du <changbin.du@intel.com> <changbin.du@intel.com>
Chao Yu <chao@kernel.org> <chao2.yu@samsung.com>
Chao Yu <chao@kernel.org> <yuchao0@huawei.com>
+Chris Chiu <chris.chiu@canonical.com> <chiu@endlessm.com>
+Chris Chiu <chris.chiu@canonical.com> <chiu@endlessos.org>
Christophe Ricard <christophe.ricard@gmail.com>
Christoph Hellwig <hch@lst.de>
Corey Minyard <minyard@acm.org>
Johan Hovold <johan@kernel.org> <johan@hovoldconsulting.com>
John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
John Stultz <johnstul@us.ibm.com>
+Jordan Crouse <jordan@cosmicpenguin.net> <jcrouse@codeaurora.org>
<josh@joshtriplett.org> <josh@freedesktop.org>
<josh@joshtriplett.org> <josh@kernel.org>
<josh@joshtriplett.org> <josht@linux.vnet.ibm.com>
Morten Welinder <welinder@darter.rentec.com>
Morten Welinder <welinder@troll.com>
Mythri P K <mythripk@ti.com>
+Nadia Yvette Chambers <nyc@holomorphy.com> William Lee Irwin III <wli@holomorphy.com>
Nathan Chancellor <nathan@kernel.org> <natechancellor@gmail.com>
Nguyen Anh Quynh <aquynh@gmail.com>
+Nicholas Piggin <npiggin@gmail.com> <npiggen@suse.de>
+Nicholas Piggin <npiggin@gmail.com> <npiggin@kernel.dk>
+Nicholas Piggin <npiggin@gmail.com> <npiggin@suse.de>
+Nicholas Piggin <npiggin@gmail.com> <nickpiggin@yahoo.com.au>
+Nicholas Piggin <npiggin@gmail.com> <piggin@cyberone.com.au>
Nicolas Ferre <nicolas.ferre@microchip.com> <nicolas.ferre@atmel.com>
Nicolas Pitre <nico@fluxnic.net> <nicolas.pitre@linaro.org>
Nicolas Pitre <nico@fluxnic.net> <nico@linaro.org>
What: /sys/kernel/debug/moxtet/input
Date: March 2019
KernelVersion: 5.3
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) Read input from the shift registers, in hexadecimal.
Returns N+1 bytes, where N is the number of Moxtet connected
modules. The first byte is from the CPU board itself.
What: /sys/kernel/debug/moxtet/output
Date: March 2019
KernelVersion: 5.3
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (RW) Read last written value to the shift registers, in
hexadecimal, or write values to the shift registers, also
in hexadecimal.
What: /sys/kernel/debug/turris-mox-rwtm/do_sign
Date: Jun 2020
KernelVersion: 5.8
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description:
======= ===========================================================
What: /sys/bus/moxtet/devices/moxtet-<name>.<addr>/module_description
Date: March 2019
KernelVersion: 5.3
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) Moxtet module description. Format: string
What: /sys/bus/moxtet/devices/moxtet-<name>.<addr>/module_id
Date: March 2019
KernelVersion: 5.3
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) Moxtet module ID. Format: %x
What: /sys/bus/moxtet/devices/moxtet-<name>.<addr>/module_name
Date: March 2019
KernelVersion: 5.3
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) Moxtet module name. Format: string
What: /sys/class/leds/<led>/device/brightness
Date: July 2020
KernelVersion: 5.9
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (RW) On the front panel of the Turris Omnia router there is also
a button which can be used to control the intensity of all the
LEDs at once, so that if they are too bright, user can dim them.
What: /sys/firmware/turris-mox-rwtm/board_version
Date: August 2019
KernelVersion: 5.4
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) Board version burned into eFuses of this Turris Mox board.
Format: %i
What: /sys/firmware/turris-mox-rwtm/mac_address*
Date: August 2019
KernelVersion: 5.4
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) MAC addresses burned into eFuses of this Turris Mox board.
Format: %pM
What: /sys/firmware/turris-mox-rwtm/pubkey
Date: August 2019
KernelVersion: 5.4
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) ECDSA public key (in pubkey hex compressed form) computed
as pair to the ECDSA private key burned into eFuses of this
Turris Mox Board.
What: /sys/firmware/turris-mox-rwtm/ram_size
Date: August 2019
KernelVersion: 5.4
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) RAM size in MiB of this Turris Mox board as was detected
during manufacturing and burned into eFuses. Can be 512 or 1024.
Format: %i
What: /sys/firmware/turris-mox-rwtm/serial_number
Date: August 2019
KernelVersion: 5.4
-Contact: Marek Behún <marek.behun@nic.cz>
+Contact: Marek Behún <kabel@kernel.org>
Description: (Read) Serial number burned into eFuses of this Turris Mox device.
Format: %016X
Description:
The current state of the log write grant head. It
represents the total log reservation of all currently
- oustanding transactions, including regrants due to
+ outstanding transactions, including regrants due to
rolling transactions. The grant head is exported in
"cycle:bytes" format.
Users: xfstests
- Recommended: BERT, EINJ, ERST, HEST, PCCT, SSDT
- - Optional: BGRT, CPEP, CSRT, DBG2, DRTM, ECDT, FACS, FPDT, IORT,
- MCHI, MPST, MSCT, NFIT, PMTT, RASF, SBST, SLIT, SPMI, SRAT, STAO,
- TCPA, TPM2, UEFI, XENV
+ - Optional: BGRT, CPEP, CSRT, DBG2, DRTM, ECDT, FACS, FPDT, IBFT,
+ IORT, MCHI, MPST, MSCT, NFIT, PMTT, RASF, SBST, SLIT, SPMI, SRAT,
+ STAO, TCPA, TPM2, UEFI, XENV
- - Not supported: BOOT, DBGP, DMAR, ETDT, HPET, IBFT, IVRS, LPIT,
- MSDM, OEMx, PSDT, RSDT, SLIC, WAET, WDAT, WDRT, WPBT
+ - Not supported: BOOT, DBGP, DMAR, ETDT, HPET, IVRS, LPIT, MSDM, OEMx,
+ PSDT, RSDT, SLIC, WAET, WDAT, WDRT, WPBT
====== ========================================================================
Table Usage for ARMv8 Linux
| Marvell | ARM-MMU-500 | #582743 | N/A |
+----------------+-----------------+-----------------+-----------------------------+
+----------------+-----------------+-----------------+-----------------------------+
+| NVIDIA | Carmel Core | N/A | NVIDIA_CARMEL_CNP_ERRATUM |
++----------------+-----------------+-----------------+-----------------------------+
++----------------+-----------------+-----------------+-----------------------------+
| Freescale/NXP | LS2080A/LS1043A | A-008585 | FSL_ERRATUM_A008585 |
+----------------+-----------------+-----------------+-----------------------------+
+----------------+-----------------+-----------------+-----------------------------+
- "#thermal-sensor-cells" Used to expose itself to thermal fw.
Read more about iio bindings at
- Documentation/devicetree/bindings/iio/iio-bindings.txt
+ https://github.com/devicetree-org/dt-schema/blob/master/schemas/iio/
Example:
ncp15wb473@0 {
title: Bindings for GPIO bitbanged I2C
maintainers:
- - Wolfram Sang <wolfram@the-dreams.de>
+ - Wolfram Sang <wsa@kernel.org>
allOf:
- $ref: /schemas/i2c/i2c-controller.yaml#
title: Freescale Inter IC (I2C) and High Speed Inter IC (HS-I2C) for i.MX
maintainers:
- - Wolfram Sang <wolfram@the-dreams.de>
+ - Oleksij Rempel <o.rempel@pengutronix.de>
allOf:
- $ref: /schemas/i2c/i2c-controller.yaml#
Industrial I/O subsystem bindings for ADC controller found in
Ingenic JZ47xx SoCs.
- ADC clients must use the format described in iio-bindings.txt, giving
- a phandle and IIO specifier pair ("io-channels") to the ADC controller.
+ ADC clients must use the format described in
+ https://github.com/devicetree-org/dt-schema/blob/master/schemas/iio/iio-consumer.yaml,
+ giving a phandle and IIO specifier pair ("io-channels") to the ADC controller.
properties:
compatible:
description: >
List of phandle and IIO specifier pairs.
Each pair defines one ADC channel to which a joystick axis is connected.
- See Documentation/devicetree/bindings/iio/iio-bindings.txt for details.
+ See
+ https://github.com/devicetree-org/dt-schema/blob/master/schemas/iio/iio-consumer.yaml
+ for details.
'#address-cells':
const: 1
- compatible: must be "resistive-adc-touch"
The device must be connected to an ADC device that provides channels for
position measurement and optional pressure.
-Refer to ../iio/iio-bindings.txt for details
+Refer to
+https://github.com/devicetree-org/dt-schema/blob/master/schemas/iio/iio-consumer.yaml
+for details
+
- iio-channels: must have at least two channels connected to an ADC device.
These should correspond to the channels exposed by the ADC device and should
have the right index as the ADC device registers them. These channels
title: CZ.NIC's Turris Omnia LEDs driver
maintainers:
- - Marek Behún <marek.behun@nic.cz>
+ - Marek Behún <kabel@kernel.org>
description:
This module adds support for the RGB LEDs found on the front panel of the
pwm|regulator|rtc|sysctrl|usb]";
A few child devices require ADC channels from the GPADC node. Those follow the
- standard bindings from iio/iio-bindings.txt and iio/adc/adc.txt
+ standard bindings from
+ https://github.com/devicetree-org/dt-schema/blob/master/schemas/iio/iio-consumer.yaml
+ and Documentation/devicetree/bindings/iio/adc/adc.yaml
abx500-temp : io-channels "aux1" and "aux2" for measuring external
temperatures.
The sub-functions of CPCAP get their own node with their own compatible values,
which are described in the following files:
-- ../power/supply/cpcap-battery.txt
-- ../power/supply/cpcap-charger.txt
-- ../regulator/cpcap-regulator.txt
-- ../phy/phy-cpcap-usb.txt
-- ../input/cpcap-pwrbutton.txt
-- ../rtc/cpcap-rtc.txt
-- ../leds/leds-cpcap.txt
-- ../iio/adc/cpcap-adc.txt
+- Documentation/devicetree/bindings/power/supply/cpcap-battery.txt
+- Documentation/devicetree/bindings/power/supply/cpcap-charger.txt
+- Documentation/devicetree/bindings/regulator/cpcap-regulator.txt
+- Documentation/devicetree/bindings/phy/phy-cpcap-usb.txt
+- Documentation/devicetree/bindings/input/cpcap-pwrbutton.txt
+- Documentation/devicetree/bindings/rtc/cpcap-rtc.txt
+- Documentation/devicetree/bindings/leds/leds-cpcap.txt
+- Documentation/devicetree/bindings/iio/adc/motorola,cpcap-adc.yaml
The only exception is the audio codec. Instead of a compatible value its
node must be named "audio-codec".
- interrupts
- interrupt-names
-additionalProperties: false
+unevaluatedProperties: false
examples:
- |
description:
Reference to an nvmem node for the MAC address
- nvmem-cells-names:
+ nvmem-cell-names:
const: mac-address
phy-connection-type:
step is 60ps. The default value is the neutral setting, so setting
rxc-skew-ps=<0> actually results in -900 picoseconds adjustment.
+ The KSZ9031 hardware supports a range of skew values from negative to
+ positive, where the specific range is property dependent. All values
+ specified in the devicetree are offset by the minimum value so they
+ can be represented as positive integers in the devicetree since it's
+ difficult to represent a negative number in the devictree.
+
+ The following 5-bit values table apply to rxc-skew-ps and txc-skew-ps.
+
+ Pad Skew Value Delay (ps) Devicetree Value
+ ------------------------------------------------------
+ 0_0000 -900ps 0
+ 0_0001 -840ps 60
+ 0_0010 -780ps 120
+ 0_0011 -720ps 180
+ 0_0100 -660ps 240
+ 0_0101 -600ps 300
+ 0_0110 -540ps 360
+ 0_0111 -480ps 420
+ 0_1000 -420ps 480
+ 0_1001 -360ps 540
+ 0_1010 -300ps 600
+ 0_1011 -240ps 660
+ 0_1100 -180ps 720
+ 0_1101 -120ps 780
+ 0_1110 -60ps 840
+ 0_1111 0ps 900
+ 1_0000 60ps 960
+ 1_0001 120ps 1020
+ 1_0010 180ps 1080
+ 1_0011 240ps 1140
+ 1_0100 300ps 1200
+ 1_0101 360ps 1260
+ 1_0110 420ps 1320
+ 1_0111 480ps 1380
+ 1_1000 540ps 1440
+ 1_1001 600ps 1500
+ 1_1010 660ps 1560
+ 1_1011 720ps 1620
+ 1_1100 780ps 1680
+ 1_1101 840ps 1740
+ 1_1110 900ps 1800
+ 1_1111 960ps 1860
+
+ The following 4-bit values table apply to the txdX-skew-ps, rxdX-skew-ps
+ data pads, and the rxdv-skew-ps, txen-skew-ps control pads.
+
+ Pad Skew Value Delay (ps) Devicetree Value
+ ------------------------------------------------------
+ 0000 -420ps 0
+ 0001 -360ps 60
+ 0010 -300ps 120
+ 0011 -240ps 180
+ 0100 -180ps 240
+ 0101 -120ps 300
+ 0110 -60ps 360
+ 0111 0ps 420
+ 1000 60ps 480
+ 1001 120ps 540
+ 1010 180ps 600
+ 1011 240ps 660
+ 1100 300ps 720
+ 1101 360ps 780
+ 1110 420ps 840
+ 1111 480ps 900
+
Optional properties:
Maximum value of 1860, default value 900:
Examples:
+ /* Attach to an Ethernet device with autodetected PHY */
+ &enet {
+ rxc-skew-ps = <1800>;
+ rxdv-skew-ps = <0>;
+ txc-skew-ps = <1800>;
+ txen-skew-ps = <0>;
+ status = "okay";
+ };
+
+ /* Attach to an explicitly-specified PHY */
mdio {
phy0: ethernet-phy@0 {
- rxc-skew-ps = <3000>;
+ rxc-skew-ps = <1800>;
rxdv-skew-ps = <0>;
- txc-skew-ps = <3000>;
+ txc-skew-ps = <1800>;
txen-skew-ps = <0>;
reg = <0>;
};
phy = <&phy0>;
phy-mode = "rgmii-id";
};
+
+References
+
+ Micrel ksz9021rl/rn Data Sheet, Revision 1.2. Dated 2/13/2014.
+ http://www.micrel.com/_PDF/Ethernet/datasheets/ksz9021rl-rn_ds.pdf
+
+ Micrel ksz9031rnx Data Sheet, Revision 2.1. Dated 11/20/2014.
+ http://www.micrel.com/_PDF/Ethernet/datasheets/KSZ9031RNX.pdf
+
+Notes:
+
+ Note that a previous version of the Micrel ksz9021rl/rn Data Sheet
+ was missing extended register 106 (transmit data pad skews), and
+ incorrectly specified the ps per step as 200ps/step instead of
+ 120ps/step. The latest update to this document reflects the latest
+ revision of the Micrel specification even though usage in the kernel
+ still reflects that incorrect document.
- fsl,vf610-spdif
- fsl,imx6sx-spdif
- fsl,imx8qm-spdif
+ - fsl,imx8qxp-spdif
+ - fsl,imx8mq-spdif
+ - fsl,imx8mm-spdif
+ - fsl,imx8mn-spdif
reg:
maxItems: 1
Tx
--
-end_start_xmit() is called by the stack. This function does the following:
+ena_start_xmit() is called by the stack. This function does the following:
- Maps data buffers (skb->data and frags).
- Populates ena_buf for the push buffer (if the driver and device are
``devlink-dpipe`` should change according to the changes done by the
standard configuration tools.
-For example, it’s quiet common to implement Access Control Lists (ACL)
+For example, it’s quite common to implement Access Control Lists (ACL)
using Ternary Content Addressable Memory (TCAM). The TCAM memory can be
divided into TCAM regions. Complex TC filters can have multiple rules with
different priorities and different lookup keys. On the other hand hardware
-------------
A subfunction devlink port is created but it is not active yet. That means the
entities are created on devlink side, the e-switch port representor is created,
-but the subfunction device itself it not created. A user might use e-switch port
+but the subfunction device itself is not created. A user might use e-switch port
representor to do settings, putting it into bridge, adding TC rules, etc. A user
might as well configure the hardware address (such as MAC address) of the
subfunction while subfunction is inactive.
* - Term
- Definitions
* - ``PCI device``
- - A physical PCI device having one or more PCI bus consists of one or
+ - A physical PCI device having one or more PCI buses consists of one or
more PCI controllers.
* - ``PCI controller``
- A controller consists of potentially multiple physical functions,
PAUSE_GET
-============
+=========
-Gets channel counts like ``ETHTOOL_GPAUSE`` ioctl request.
+Gets pause frame settings like ``ETHTOOL_GPAUSEPARAM`` ioctl request.
Request contents:
Each member has a corresponding attribute defined.
PAUSE_SET
-============
+=========
Sets pause parameters like ``ETHTOOL_GPAUSEPARAM`` ioctl request.
EEE_GET
=======
-Gets channel counts like ``ETHTOOL_GEEE`` ioctl request.
+Gets Energy Efficient Ethernet settings like ``ETHTOOL_GEEE`` ioctl request.
Request contents:
EEE_SET
=======
-Sets pause parameters like ``ETHTOOL_GEEEPARAM`` ioctl request.
+Sets Energy Efficient Ethernet parameters like ``ETHTOOL_SEEE`` ioctl request.
Request contents:
The NIC driver offering ipsec offload will need to implement these
callbacks to make the offload available to the network stack's
-XFRM subsytem. Additionally, the feature bits NETIF_F_HW_ESP and
+XFRM subsystem. Additionally, the feature bits NETIF_F_HW_ESP and
NETIF_F_HW_ESP_TX_CSUM will signal the availability of the offload.
Define which vcpu is the Bootstrap Processor (BSP). Values are the same
as the vcpu id in KVM_CREATE_VCPU. If this ioctl is not called, the default
-is vcpu 0.
+is vcpu 0. This ioctl has to be called before vcpu creation,
+otherwise it will return EBUSY error.
4.42 KVM_GET_XSAVE
allows user space to deflect and potentially handle various MSR accesses
into user space.
-If a vCPU is in running state while this ioctl is invoked, the vCPU may
-experience inconsistent filtering behavior on MSR accesses.
+Note, invoking this ioctl with a vCPU is running is inherently racy. However,
+KVM does guarantee that vCPUs will see either the previous filter or the new
+filter, e.g. MSRs with identical settings in both the old and new filter will
+have deterministic behavior.
4.127 KVM_XEN_HVM_SET_ATTR
--------------------------
M: Christian Brauner <christian@brauner.io>
M: Hridya Valsaraju <hridya@google.com>
M: Suren Baghdasaryan <surenb@google.com>
-L: devel@driverdev.osuosl.org
+L: linux-kernel@vger.kernel.org
S: Supported
T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git
F: drivers/android/
F: drivers/pinctrl/pinctrl-gemini.c
F: drivers/rtc/rtc-ftrtc010.c
-ARM/CZ.NIC TURRIS MOX SUPPORT
-M: Marek Behun <marek.behun@nic.cz>
+ARM/CZ.NIC TURRIS SUPPORT
+M: Marek Behun <kabel@kernel.org>
S: Maintained
-W: http://mox.turris.cz
+W: https://www.turris.cz/
F: Documentation/ABI/testing/debugfs-moxtet
F: Documentation/ABI/testing/sysfs-bus-moxtet-devices
F: Documentation/ABI/testing/sysfs-firmware-turris-mox-rwtm
F: Documentation/devicetree/bindings/bus/moxtet.txt
F: Documentation/devicetree/bindings/firmware/cznic,turris-mox-rwtm.txt
F: Documentation/devicetree/bindings/gpio/gpio-moxtet.txt
+F: Documentation/devicetree/bindings/leds/cznic,turris-omnia-leds.yaml
+F: Documentation/devicetree/bindings/watchdog/armada-37xx-wdt.txt
F: drivers/bus/moxtet.c
F: drivers/firmware/turris-mox-rwtm.c
+F: drivers/leds/leds-turris-omnia.c
+F: drivers/mailbox/armada-37xx-rwtm-mailbox.c
F: drivers/gpio/gpio-moxtet.c
+F: drivers/watchdog/armada_37xx_wdt.c
+F: include/dt-bindings/bus/moxtet.h
+F: include/linux/armada-37xx-rwtm-mailbox.h
F: include/linux/moxtet.h
ARM/EZX SMARTPHONES (A780, A910, A1200, E680, ROKR E2 and ROKR E6)
N: sc2731
ARM/STI ARCHITECTURE
-M: Patrice Chotard <patrice.chotard@st.com>
+M: Patrice Chotard <patrice.chotard@foss.st.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
W: http://www.stlinux.com
ARM/STM32 ARCHITECTURE
M: Maxime Coquelin <mcoquelin.stm32@gmail.com>
-M: Alexandre Torgue <alexandre.torgue@st.com>
+M: Alexandre Torgue <alexandre.torgue@foss.st.com>
L: linux-stm32@st-md-mailman.stormreply.com (moderated for non-subscribers)
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
F: drivers/md/bcache/
BDISP ST MEDIA DRIVER
-M: Fabien Dessenne <fabien.dessenne@st.com>
+M: Fabien Dessenne <fabien.dessenne@foss.st.com>
L: linux-media@vger.kernel.org
S: Supported
W: https://linuxtv.org
L: linux-pm@vger.kernel.org
S: Maintained
T: git git://github.com/broadcom/stblinux.git
-F: drivers/soc/bcm/bcm-pmb.c
+F: drivers/soc/bcm/bcm63xx/bcm-pmb.c
F: include/dt-bindings/soc/bcm-pmb.h
BROADCOM SPECIFIC AMBA DRIVER (BCMA)
F: drivers/platform/x86/dell/dell-wmi.c
DELTA ST MEDIA DRIVER
-M: Hugues Fruchet <hugues.fruchet@st.com>
+M: Hugues Fruchet <hugues.fruchet@foss.st.com>
L: linux-media@vger.kernel.org
S: Supported
W: https://linuxtv.org
DRM DRIVERS FOR STI
M: Benjamin Gaignard <benjamin.gaignard@linaro.org>
-M: Vincent Abriou <vincent.abriou@st.com>
L: dri-devel@lists.freedesktop.org
S: Maintained
T: git git://anongit.freedesktop.org/drm/drm-misc
F: drivers/gpu/drm/sti
DRM DRIVERS FOR STM
-M: Yannick Fertre <yannick.fertre@st.com>
-M: Philippe Cornu <philippe.cornu@st.com>
+M: Yannick Fertre <yannick.fertre@foss.st.com>
+M: Philippe Cornu <philippe.cornu@foss.st.com>
M: Benjamin Gaignard <benjamin.gaignard@linaro.org>
-M: Vincent Abriou <vincent.abriou@st.com>
L: dri-devel@lists.freedesktop.org
S: Maintained
T: git git://anongit.freedesktop.org/drm/drm-misc
GENERIC PHY FRAMEWORK
M: Kishon Vijay Abraham I <kishon@ti.com>
M: Vinod Koul <vkoul@kernel.org>
-L: linux-kernel@vger.kernel.org
+L: linux-phy@lists.infradead.org
S: Supported
+Q: https://patchwork.kernel.org/project/linux-phy/list/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy.git
F: Documentation/devicetree/bindings/phy/
F: drivers/phy/
HISILICON STAGING DRIVERS FOR HIKEY 960/970
M: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
-L: devel@driverdev.osuosl.org
S: Maintained
F: drivers/staging/hikey9xx/
F: mm/hugetlb.c
HVA ST MEDIA DRIVER
-M: Jean-Christophe Trotin <jean-christophe.trotin@st.com>
+M: Jean-Christophe Trotin <jean-christophe.trotin@foss.st.com>
L: linux-media@vger.kernel.org
S: Supported
W: https://linuxtv.org
M: Dany Madden <drt@linux.ibm.com>
M: Lijun Pan <ljp@linux.ibm.com>
M: Sukadev Bhattiprolu <sukadev@linux.ibm.com>
+R: Thomas Falcon <tlfalcon@linux.ibm.com>
L: netdev@vger.kernel.org
S: Supported
F: drivers/net/ethernet/ibm/ibmvnic.*
LED SUBSYSTEM
M: Pavel Machek <pavel@ucw.cz>
-R: Dan Murphy <dmurphy@ti.com>
L: linux-leds@vger.kernel.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-leds.git
F: drivers/media/radio/radio-maxiradio*
MCAN MMIO DEVICE DRIVER
-M: Dan Murphy <dmurphy@ti.com>
M: Pankaj Sharma <pankj.sharma@samsung.com>
L: linux-can@vger.kernel.org
S: Maintained
F: drivers/media/dvb-frontends/stv6111*
MEDIA DRIVERS FOR STM32 - DCMI
-M: Hugues Fruchet <hugues.fruchet@st.com>
+M: Hugues Fruchet <hugues.fruchet@foss.st.com>
L: linux-media@vger.kernel.org
S: Supported
T: git git://linuxtv.org/media_tree.git
M: Mat Martineau <mathew.j.martineau@linux.intel.com>
M: Matthieu Baerts <matthieu.baerts@tessares.net>
L: netdev@vger.kernel.org
-L: mptcp@lists.01.org
+L: mptcp@lists.linux.dev
S: Maintained
W: https://github.com/multipath-tcp/mptcp_net-next/wiki
B: https://github.com/multipath-tcp/mptcp_net-next/issues
QLOGIC QLGE 10Gb ETHERNET DRIVER
M: Manish Chopra <manishc@marvell.com>
M: GR-Linux-NIC-Dev@marvell.com
-L: netdev@vger.kernel.org
-S: Supported
-F: drivers/staging/qlge/
-
-QLOGIC QLGE 10Gb ETHERNET DRIVER
M: Coiby Xu <coiby.xu@gmail.com>
L: netdev@vger.kernel.org
-S: Maintained
+S: Supported
F: Documentation/networking/device_drivers/qlogic/qlge.rst
+F: drivers/staging/qlge/
QM1D1B0004 MEDIA DRIVER
M: Akihiro Tsukada <tskd08@gmail.com>
S: Maintained
F: drivers/iommu/arm/arm-smmu/qcom_iommu.c
+QUALCOMM IPC ROUTER (QRTR) DRIVER
+M: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
+L: linux-arm-msm@vger.kernel.org
+S: Maintained
+F: include/trace/events/qrtr.h
+F: include/uapi/linux/qrtr.h
+F: net/qrtr/
+
QUALCOMM IPCC MAILBOX DRIVER
M: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
L: linux-arm-msm@vger.kernel.org
REMOTE PROCESSOR (REMOTEPROC) SUBSYSTEM
M: Ohad Ben-Cohen <ohad@wizery.com>
M: Bjorn Andersson <bjorn.andersson@linaro.org>
+M: Mathieu Poirier <mathieu.poirier@linaro.org>
L: linux-remoteproc@vger.kernel.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/andersson/remoteproc.git rproc-next
REMOTE PROCESSOR MESSAGING (RPMSG) SUBSYSTEM
M: Ohad Ben-Cohen <ohad@wizery.com>
M: Bjorn Andersson <bjorn.andersson@linaro.org>
+M: Mathieu Poirier <mathieu.poirier@linaro.org>
L: linux-remoteproc@vger.kernel.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/andersson/remoteproc.git rpmsg-next
S390 VFIO AP DRIVER
M: Tony Krowiak <akrowiak@linux.ibm.com>
-M: Pierre Morel <pmorel@linux.ibm.com>
M: Halil Pasic <pasic@linux.ibm.com>
+M: Jason Herne <jjherne@linux.ibm.com>
L: linux-s390@vger.kernel.org
S: Supported
W: http://www.ibm.com/developerworks/linux/linux390/
S390 VFIO-CCW DRIVER
M: Cornelia Huck <cohuck@redhat.com>
M: Eric Farman <farman@linux.ibm.com>
+M: Matthew Rosato <mjrosato@linux.ibm.com>
R: Halil Pasic <pasic@linux.ibm.com>
L: linux-s390@vger.kernel.org
L: kvm@vger.kernel.org
S390 VFIO-PCI DRIVER
M: Matthew Rosato <mjrosato@linux.ibm.com>
+M: Eric Farman <farman@linux.ibm.com>
L: linux-s390@vger.kernel.org
L: kvm@vger.kernel.org
S: Supported
SPIDERNET NETWORK DRIVER for CELL
M: Ishizaki Kou <kou.ishizaki@toshiba.co.jp>
+M: Geoff Levand <geoff@infradead.org>
L: netdev@vger.kernel.org
-S: Supported
+L: linuxppc-dev@lists.ozlabs.org
+S: Maintained
F: Documentation/networking/device_drivers/ethernet/toshiba/spider_net.rst
F: drivers/net/ethernet/toshiba/spider_net*
F: drivers/media/i2c/st-mipid02.c
ST STM32 I2C/SMBUS DRIVER
-M: Pierre-Yves MORDRET <pierre-yves.mordret@st.com>
+M: Pierre-Yves MORDRET <pierre-yves.mordret@foss.st.com>
+M: Alain Volmat <alain.volmat@foss.st.com>
L: linux-i2c@vger.kernel.org
S: Maintained
F: drivers/i2c/busses/i2c-stm32*
STAGING SUBSYSTEM
M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-L: devel@driverdev.osuosl.org
+L: linux-staging@lists.linux.dev
S: Supported
T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git
F: drivers/staging/
F: kernel/static_call.c
STI AUDIO (ASoC) DRIVERS
-M: Arnaud Pouliquen <arnaud.pouliquen@st.com>
+M: Arnaud Pouliquen <arnaud.pouliquen@foss.st.com>
L: alsa-devel@alsa-project.org (moderated for non-subscribers)
S: Maintained
F: Documentation/devicetree/bindings/sound/st,sti-asoc-card.txt
F: drivers/media/usb/stk1160/
STM32 AUDIO (ASoC) DRIVERS
-M: Olivier Moysan <olivier.moysan@st.com>
-M: Arnaud Pouliquen <arnaud.pouliquen@st.com>
+M: Olivier Moysan <olivier.moysan@foss.st.com>
+M: Arnaud Pouliquen <arnaud.pouliquen@foss.st.com>
L: alsa-devel@alsa-project.org (moderated for non-subscribers)
S: Maintained
F: Documentation/devicetree/bindings/iio/adc/st,stm32-*.yaml
F: sound/soc/stm/
STM32 TIMER/LPTIMER DRIVERS
-M: Fabrice Gasnier <fabrice.gasnier@st.com>
+M: Fabrice Gasnier <fabrice.gasnier@foss.st.com>
S: Maintained
F: Documentation/ABI/testing/*timer-stm32
F: Documentation/devicetree/bindings/*/*stm32-*timer*
STMMAC ETHERNET DRIVER
M: Giuseppe Cavallaro <peppe.cavallaro@st.com>
-M: Alexandre Torgue <alexandre.torgue@st.com>
+M: Alexandre Torgue <alexandre.torgue@foss.st.com>
M: Jose Abreu <joabreu@synopsys.com>
L: netdev@vger.kernel.org
S: Supported
F: drivers/thermal/ti-soc-thermal/
TI BQ27XXX POWER SUPPLY DRIVER
-R: Dan Murphy <dmurphy@ti.com>
F: drivers/power/supply/bq27xxx_battery.c
F: drivers/power/supply/bq27xxx_battery_i2c.c
F: include/linux/power/bq27xxx_battery.h
F: sound/soc/codecs/tas571x*
TI TCAN4X5X DEVICE DRIVER
-M: Dan Murphy <dmurphy@ti.com>
L: linux-can@vger.kernel.org
S: Maintained
F: Documentation/devicetree/bindings/net/can/tcan4x5x.txt
M: Martyn Welch <martyn@welchs.me.uk>
M: Manohar Vanga <manohar.vanga@gmail.com>
M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-L: devel@driverdev.osuosl.org
+L: linux-kernel@vger.kernel.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git
F: Documentation/driver-api/vme.rst
VERSION = 5
PATCHLEVEL = 12
SUBLEVEL = 0
-EXTRAVERSION = -rc3
+EXTRAVERSION = -rc7
NAME = Frozen Wasteland
# *DOCUMENTATION*
memory {
device_type = "memory";
/* CONFIG_LINUX_RAM_BASE needs to match low mem start */
- reg = <0x0 0x80000000 0x0 0x20000000 /* 512 MB low mem */
+ reg = <0x0 0x80000000 0x0 0x40000000 /* 1 GB low mem */
0x1 0x00000000 0x0 0x40000000>; /* 1 GB highmem */
};
sizeof(sf->uc.uc_mcontext.regs.scratch));
err |= __copy_to_user(&sf->uc.uc_sigmask, set, sizeof(sigset_t));
- return err;
+ return err ? -EFAULT : 0;
}
static int restore_usr_regs(struct pt_regs *regs, struct rt_sigframe __user *sf)
&(sf->uc.uc_mcontext.regs.scratch),
sizeof(sf->uc.uc_mcontext.regs.scratch));
if (err)
- return err;
+ return -EFAULT;
set_current_blocked(&set);
regs->bta = uregs.scratch.bta;
const void *table_start, unsigned long table_size,
const u8 *header_start, unsigned long header_size)
{
- const u8 *ptr = header_start + 4;
- const u8 *end = header_start + header_size;
-
table->core.pc = (unsigned long)core_start;
table->core.range = core_size;
table->init.pc = (unsigned long)init_start;
table->init.range = init_size;
table->address = table_start;
table->size = table_size;
-
- /* See if the linker provided table looks valid. */
- if (header_size <= 4
- || header_start[0] != 1
- || (void *)read_pointer(&ptr, end, header_start[1]) != table_start
- || header_start[2] == DW_EH_PE_omit
- || read_pointer(&ptr, end, header_start[2]) <= 0
- || header_start[3] == DW_EH_PE_omit)
- header_start = NULL;
-
+ /* To avoid the pointer addition with NULL pointer.*/
+ if (header_start != NULL) {
+ const u8 *ptr = header_start + 4;
+ const u8 *end = header_start + header_size;
+ /* See if the linker provided table looks valid. */
+ if (header_size <= 4
+ || header_start[0] != 1
+ || (void *)read_pointer(&ptr, end, header_start[1])
+ != table_start
+ || header_start[2] == DW_EH_PE_omit
+ || read_pointer(&ptr, end, header_start[2]) <= 0
+ || header_start[3] == DW_EH_PE_omit)
+ header_start = NULL;
+ }
table->hdrsz = header_size;
smp_wmb();
table->header = header_start;
ethernet1 = &cpsw_emac1;
spi0 = &spi0;
spi1 = &spi1;
+ mmc0 = &mmc1;
+ mmc1 = &mmc2;
+ mmc2 = &mmc3;
};
cpus {
ranges = <MBUS_ID(0xf0, 0x01) 0 0xf1000000 0x100000
MBUS_ID(0x01, 0x1d) 0 0xfff00000 0x100000
MBUS_ID(0x09, 0x19) 0 0xf1100000 0x10000
- MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000>;
+ MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000
+ MBUS_ID(0x0c, 0x04) 0 0xf1200000 0x100000>;
internal-regs {
phy1: ethernet-phy@1 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <1>;
+ marvell,reg-init = <3 18 0 0x4985>;
/* irq is connected to &pcawan pin 7 */
};
};
&pinctrl {
- atmel,mux-mask = <
- /* A B C */
- 0xFFFFFE7F 0xC0E0397F 0xEF00019D /* pioA */
- 0x03FFFFFF 0x02FC7E68 0x00780000 /* pioB */
- 0xffffffff 0xF83FFFFF 0xB800F3FC /* pioC */
- 0x003FFFFF 0x003F8000 0x00000000 /* pioD */
- >;
-
adc {
pinctrl_adc_default: adc_default {
atmel,pins = <AT91_PIOB 15 AT91_PERIPH_A AT91_PINCTRL_NONE>;
pinctrl-0 = <&pinctrl_macb0_default>;
phy-mode = "rmii";
- ethernet-phy@0 {
- reg = <0x0>;
+ ethernet-phy@7 {
+ reg = <0x7>;
interrupt-parent = <&pioA>;
interrupts = <PIN_PD31 IRQ_TYPE_LEVEL_LOW>;
pinctrl-names = "default";
#reset-cells = <1>;
};
- bsc_intr: interrupt-controller@7ef00040 {
- compatible = "brcm,bcm2711-l2-intc", "brcm,l2-intc";
- reg = <0x7ef00040 0x30>;
- interrupts = <GIC_SPI 117 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-controller;
- #interrupt-cells = <1>;
- };
-
aon_intr: interrupt-controller@7ef00100 {
compatible = "brcm,bcm2711-l2-intc", "brcm,l2-intc";
reg = <0x7ef00100 0x30>;
reg = <0x7ef04500 0x100>, <0x7ef00b00 0x300>;
reg-names = "bsc", "auto-i2c";
clock-frequency = <97500>;
- interrupt-parent = <&bsc_intr>;
- interrupts = <0>;
status = "disabled";
};
reg = <0x7ef09500 0x100>, <0x7ef05b00 0x300>;
reg-names = "bsc", "auto-i2c";
clock-frequency = <97500>;
- interrupt-parent = <&bsc_intr>;
- interrupts = <1>;
status = "disabled";
};
};
pinctrl-0 = <&pinctrl_usdhc2>;
cd-gpios = <&gpio1 4 GPIO_ACTIVE_LOW>;
wp-gpios = <&gpio1 2 GPIO_ACTIVE_HIGH>;
+ vmmc-supply = <&vdd_sd1_reg>;
status = "disabled";
};
&pinctrl_usdhc3_cdwp>;
cd-gpios = <&gpio1 27 GPIO_ACTIVE_LOW>;
wp-gpios = <&gpio1 29 GPIO_ACTIVE_HIGH>;
+ vmmc-supply = <&vdd_sd0_reg>;
status = "disabled";
};
micrel,led-mode = <1>;
clocks = <&clks IMX6UL_CLK_ENET_REF>;
clock-names = "rmii-ref";
- reset-gpios = <&gpio_spi 1 GPIO_ACTIVE_LOW>;
- reset-assert-us = <10000>;
- reset-deassert-us = <100>;
};
micrel,led-mode = <1>;
clocks = <&clks IMX6UL_CLK_ENET2_REF>;
clock-names = "rmii-ref";
- reset-gpios = <&gpio_spi 2 GPIO_ACTIVE_LOW>;
- reset-assert-us = <10000>;
- reset-deassert-us = <100>;
};
};
};
status = "okay";
};
+&gpio_spi {
+ eth0-phy-hog {
+ gpio-hog;
+ gpios = <1 GPIO_ACTIVE_HIGH>;
+ output-high;
+ line-name = "eth0-phy";
+ };
+
+ eth1-phy-hog {
+ gpio-hog;
+ gpios = <2 GPIO_ACTIVE_HIGH>;
+ output-high;
+ line-name = "eth1-phy";
+ };
+};
+
&i2c1 {
clock-frequency = <100000>;
pinctrl-names = "default";
};
&gpmi {
+ fsl,use-minimum-ecc;
status = "okay";
};
i2c1 = &i2c2;
i2c2 = &i2c3;
i2c3 = &i2c4;
+ mmc0 = &mmc1;
+ mmc1 = &mmc2;
+ mmc2 = &mmc3;
+ mmc3 = &mmc4;
+ mmc4 = &mmc5;
serial0 = &uart1;
serial1 = &uart2;
serial2 = &uart3;
ti,max-div = <2>;
};
- sha2md5_fck: sha2md5_fck@15c8 {
- #clock-cells = <0>;
- compatible = "ti,gate-clock";
- clocks = <&l3_div_ck>;
- ti,bit-shift = <1>;
- reg = <0x15c8>;
- };
-
usb_phy_cm_clk32k: usb_phy_cm_clk32k@640 {
#clock-cells = <0>;
compatible = "ti,gate-clock";
i2c2 = &i2c3;
i2c3 = &i2c4;
i2c4 = &i2c5;
+ mmc0 = &mmc1;
+ mmc1 = &mmc2;
+ mmc2 = &mmc3;
+ mmc3 = &mmc4;
+ mmc4 = &mmc5;
serial0 = &uart1;
serial1 = &uart2;
serial2 = &uart3;
compatible = "microchip,sam9x60-pinctrl", "atmel,at91sam9x5-pinctrl", "atmel,at91rm9200-pinctrl", "simple-bus";
ranges = <0xfffff400 0xfffff400 0x800>;
+ /* mux-mask corresponding to sam9x60 SoC in TFBGA228L package */
+ atmel,mux-mask = <
+ /* A B C */
+ 0xffffffff 0xffe03fff 0xef00019d /* pioA */
+ 0x03ffffff 0x02fc7e7f 0x00780000 /* pioB */
+ 0xffffffff 0xffffffff 0xf83fffff /* pioC */
+ 0x003fffff 0x003f8000 0x00000000 /* pioD */
+ >;
+
pioA: gpio@fffff400 {
compatible = "microchip,sam9x60-gpio", "atmel,at91sam9x5-gpio", "atmel,at91rm9200-gpio";
reg = <0xfffff400 0x200>;
#include <linux/module.h>
#include <linux/irq.h>
#include <linux/irqdomain.h>
+#include <linux/irqchip.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_address.h>
* interrupts. It registers the interrupt enable and disable functions
* to the kernel for each interrupt source.
*/
-void __init mxc_init_irq(void __iomem *irqbase)
+static void __init mxc_init_irq(void __iomem *irqbase)
{
struct device_node *np;
int irq_base;
printk(KERN_INFO "MXC IRQ initialized\n");
}
+
+static int __init imx_avic_init(struct device_node *node,
+ struct device_node *parent)
+{
+ void __iomem *avic_base;
+
+ avic_base = of_iomap(node, 0);
+ BUG_ON(!avic_base);
+ mxc_init_irq(avic_base);
+ return 0;
+}
+
+IRQCHIP_DECLARE(imx_avic, "fsl,avic", imx_avic_init);
void imx21_init_early(void);
void imx31_init_early(void);
void imx35_init_early(void);
-void mxc_init_irq(void __iomem *);
void mx31_init_irq(void);
void mx35_init_irq(void);
void mxc_set_cpu_type(unsigned int type);
mxc_set_cpu_type(MXC_CPU_MX1);
}
-static void __init imx1_init_irq(void)
-{
- void __iomem *avic_addr;
-
- avic_addr = ioremap(MX1_AVIC_ADDR, SZ_4K);
- WARN_ON(!avic_addr);
-
- mxc_init_irq(avic_addr);
-}
-
static const char * const imx1_dt_board_compat[] __initconst = {
"fsl,imx1",
NULL
DT_MACHINE_START(IMX1_DT, "Freescale i.MX1 (Device Tree Support)")
.init_early = imx1_init_early,
- .init_irq = imx1_init_irq,
.dt_compat = imx1_dt_board_compat,
.restart = mxc_restart,
MACHINE_END
imx_aips_allow_unprivileged_access("fsl,imx25-aips");
}
-static void __init mx25_init_irq(void)
-{
- struct device_node *np;
- void __iomem *avic_base;
-
- np = of_find_compatible_node(NULL, NULL, "fsl,avic");
- avic_base = of_iomap(np, 0);
- BUG_ON(!avic_base);
- mxc_init_irq(avic_base);
-}
-
static const char * const imx25_dt_board_compat[] __initconst = {
"fsl,imx25",
NULL
.init_early = imx25_init_early,
.init_machine = imx25_dt_init,
.init_late = imx25_pm_init,
- .init_irq = mx25_init_irq,
.dt_compat = imx25_dt_board_compat,
MACHINE_END
mxc_set_cpu_type(MXC_CPU_MX27);
}
-static void __init mx27_init_irq(void)
-{
- void __iomem *avic_base;
- struct device_node *np;
-
- np = of_find_compatible_node(NULL, NULL, "fsl,avic");
- avic_base = of_iomap(np, 0);
- BUG_ON(!avic_base);
- mxc_init_irq(avic_base);
-}
-
static const char * const imx27_dt_board_compat[] __initconst = {
"fsl,imx27",
NULL
DT_MACHINE_START(IMX27_DT, "Freescale i.MX27 (Device Tree Support)")
.map_io = mx27_map_io,
.init_early = imx27_init_early,
- .init_irq = mx27_init_irq,
.init_late = imx27_pm_init,
.dt_compat = imx27_dt_board_compat,
MACHINE_END
DT_MACHINE_START(IMX31_DT, "Freescale i.MX31 (Device Tree Support)")
.map_io = mx31_map_io,
.init_early = imx31_init_early,
- .init_irq = mx31_init_irq,
.dt_compat = imx31_dt_board_compat,
MACHINE_END
.l2c_aux_mask = ~0,
.map_io = mx35_map_io,
.init_early = imx35_init_early,
- .init_irq = mx35_init_irq,
.dt_compat = imx35_dt_board_compat,
MACHINE_END
mx3_ccm_base = of_iomap(np, 0);
BUG_ON(!mx3_ccm_base);
}
-
-void __init mx31_init_irq(void)
-{
- void __iomem *avic_base;
- struct device_node *np;
-
- np = of_find_compatible_node(NULL, NULL, "fsl,imx31-avic");
- avic_base = of_iomap(np, 0);
- BUG_ON(!avic_base);
-
- mxc_init_irq(avic_base);
-}
#endif /* ifdef CONFIG_SOC_IMX31 */
#ifdef CONFIG_SOC_IMX35
mx3_ccm_base = of_iomap(np, 0);
BUG_ON(!mx3_ccm_base);
}
-
-void __init mx35_init_irq(void)
-{
- void __iomem *avic_base;
- struct device_node *np;
-
- np = of_find_compatible_node(NULL, NULL, "fsl,imx35-avic");
- avic_base = of_iomap(np, 0);
- BUG_ON(!avic_base);
-
- mxc_init_irq(avic_base);
-}
#endif /* ifdef CONFIG_SOC_IMX35 */
static long long __init keystone_pv_fixup(void)
{
long long offset;
- phys_addr_t mem_start, mem_end;
+ u64 mem_start, mem_end;
mem_start = memblock_start_of_DRAM();
mem_end = memblock_end_of_DRAM();
if (mem_start < KEYSTONE_HIGH_PHYS_START ||
mem_end > KEYSTONE_HIGH_PHYS_END) {
pr_crit("Invalid address space for memory (%08llx-%08llx)\n",
- (u64)mem_start, (u64)mem_end);
+ mem_start, mem_end);
return 0;
}
#include <linux/platform_data/gpio-omap.h>
#include <asm/assembler.h>
+#include <asm/irq.h>
#include "ams-delta-fiq.h"
#include "board-ams-delta.h"
*/
#include <linux/arm-smccc.h>
+#include <linux/cpu_pm.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/io.h>
#include "common.h"
#include "omap-secure.h"
+#include "soc.h"
static phys_addr_t omap_secure_memblock_base;
{
omap_optee_init_check();
}
+
+/*
+ * Dummy dispatcher call after core OSWR and MPU off. Updates the ROM return
+ * address after MMU has been re-enabled after CPU1 has been woken up again.
+ * Otherwise the ROM code will attempt to use the earlier physical return
+ * address that got set with MMU off when waking up CPU1. Only used on secure
+ * devices.
+ */
+static int cpu_notifier(struct notifier_block *nb, unsigned long cmd, void *v)
+{
+ switch (cmd) {
+ case CPU_CLUSTER_PM_EXIT:
+ omap_secure_dispatcher(OMAP4_PPA_SERVICE_0,
+ FLAG_START_CRITICAL,
+ 0, 0, 0, 0, 0);
+ break;
+ default:
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block secure_notifier_block = {
+ .notifier_call = cpu_notifier,
+};
+
+static int __init secure_pm_init(void)
+{
+ if (omap_type() == OMAP2_DEVICE_TYPE_GP || !soc_is_omap44xx())
+ return 0;
+
+ cpu_pm_register_notifier(&secure_notifier_block);
+
+ return 0;
+}
+omap_arch_initcall(secure_pm_init);
#define OMAP5_DRA7_MON_SET_ACR_INDEX 0x107
/* Secure PPA(Primary Protected Application) APIs */
+#define OMAP4_PPA_SERVICE_0 0x21
#define OMAP4_PPA_L2_POR_INDEX 0x23
#define OMAP4_PPA_CPU_ACTRL_SMP_INDEX 0x25
omap_voltage_register_pmic(voltdm, &omap443x_max8952_mpu);
if (of_machine_is_compatible("motorola,droid-bionic")) {
- voltdm = voltdm_lookup("mpu");
+ voltdm = voltdm_lookup("core");
omap_voltage_register_pmic(voltdm, &omap_cpcap_core);
- voltdm = voltdm_lookup("mpu");
+ voltdm = voltdm_lookup("iva");
omap_voltage_register_pmic(voltdm, &omap_cpcap_iva);
} else {
voltdm = voltdm_lookup("core");
extern struct omap_sr_data omap_sr_pdata[];
-static int __init sr_dev_init(struct omap_hwmod *oh, void *user)
+static int __init sr_init_by_name(const char *name, const char *voltdm)
{
struct omap_sr_data *sr_data = NULL;
struct omap_volt_data *volt_data;
- struct omap_smartreflex_dev_attr *sr_dev_attr;
static int i;
- if (!strncmp(oh->name, "smartreflex_mpu_iva", 20) ||
- !strncmp(oh->name, "smartreflex_mpu", 16))
+ if (!strncmp(name, "smartreflex_mpu_iva", 20) ||
+ !strncmp(name, "smartreflex_mpu", 16))
sr_data = &omap_sr_pdata[OMAP_SR_MPU];
- else if (!strncmp(oh->name, "smartreflex_core", 17))
+ else if (!strncmp(name, "smartreflex_core", 17))
sr_data = &omap_sr_pdata[OMAP_SR_CORE];
- else if (!strncmp(oh->name, "smartreflex_iva", 16))
+ else if (!strncmp(name, "smartreflex_iva", 16))
sr_data = &omap_sr_pdata[OMAP_SR_IVA];
if (!sr_data) {
- pr_err("%s: Unknown instance %s\n", __func__, oh->name);
+ pr_err("%s: Unknown instance %s\n", __func__, name);
return -EINVAL;
}
- sr_dev_attr = (struct omap_smartreflex_dev_attr *)oh->dev_attr;
- if (!sr_dev_attr || !sr_dev_attr->sensor_voltdm_name) {
- pr_err("%s: No voltage domain specified for %s. Cannot initialize\n",
- __func__, oh->name);
- goto exit;
- }
-
- sr_data->name = oh->name;
+ sr_data->name = name;
if (cpu_is_omap343x())
sr_data->ip_type = 1;
else
}
}
- sr_data->voltdm = voltdm_lookup(sr_dev_attr->sensor_voltdm_name);
+ sr_data->voltdm = voltdm_lookup(voltdm);
if (!sr_data->voltdm) {
pr_err("%s: Unable to get voltage domain pointer for VDD %s\n",
- __func__, sr_dev_attr->sensor_voltdm_name);
+ __func__, voltdm);
goto exit;
}
return 0;
}
+static int __init sr_dev_init(struct omap_hwmod *oh, void *user)
+{
+ struct omap_smartreflex_dev_attr *sr_dev_attr;
+
+ sr_dev_attr = (struct omap_smartreflex_dev_attr *)oh->dev_attr;
+ if (!sr_dev_attr || !sr_dev_attr->sensor_voltdm_name) {
+ pr_err("%s: No voltage domain specified for %s. Cannot initialize\n",
+ __func__, oh->name);
+ return 0;
+ }
+
+ return sr_init_by_name(oh->name, sr_dev_attr->sensor_voltdm_name);
+}
+
/*
* API to be called from board files to enable smartreflex
* autocompensation at init.
sr_enable_on_init = true;
}
+static const char * const omap4_sr_instances[] = {
+ "mpu",
+ "iva",
+ "core",
+};
+
+static const char * const dra7_sr_instances[] = {
+ "mpu",
+ "core",
+};
+
int __init omap_devinit_smartreflex(void)
{
+ const char * const *sr_inst;
+ int i, nr_sr = 0;
+
+ if (soc_is_omap44xx()) {
+ sr_inst = omap4_sr_instances;
+ nr_sr = ARRAY_SIZE(omap4_sr_instances);
+
+ } else if (soc_is_dra7xx()) {
+ sr_inst = dra7_sr_instances;
+ nr_sr = ARRAY_SIZE(dra7_sr_instances);
+ }
+
+ if (nr_sr) {
+ const char *name, *voltdm;
+
+ for (i = 0; i < nr_sr; i++) {
+ name = kasprintf(GFP_KERNEL, "smartreflex_%s", sr_inst[i]);
+ voltdm = sr_inst[i];
+ sr_init_by_name(name, voltdm);
+ }
+
+ return 0;
+ }
+
return omap_hwmod_for_each_by_class("smartreflex", sr_dev_init, NULL);
}
#endif
static int mst_pcmcia0_irqs[11] = {
- [0 ... 10] = -1,
+ [0 ... 4] = -1,
[5] = MAINSTONE_S0_CD_IRQ,
+ [6 ... 7] = -1,
[8] = MAINSTONE_S0_STSCHG_IRQ,
+ [9] = -1,
[10] = MAINSTONE_S0_IRQ,
};
static int mst_pcmcia1_irqs[11] = {
- [0 ... 10] = -1,
+ [0 ... 4] = -1,
[5] = MAINSTONE_S1_CD_IRQ,
+ [6 ... 7] = -1,
[8] = MAINSTONE_S1_STSCHG_IRQ,
+ [9] = -1,
[10] = MAINSTONE_S1_IRQ,
};
If unsure, say Y.
+config NVIDIA_CARMEL_CNP_ERRATUM
+ bool "NVIDIA Carmel CNP: CNP on Carmel semantically different than ARM cores"
+ default y
+ help
+ If CNP is enabled on Carmel cores, non-sharable TLBIs on a core will not
+ invalidate shared TLB entries installed by a different core, as it would
+ on standard ARM cores.
+
+ If unsure, say Y.
+
config SOCIONEXT_SYNQUACER_PREITS
bool "Socionext Synquacer: Workaround for GICv3 pre-ITS"
default y
ranges = <0x0 0x00 0x1700000 0x100000>;
reg = <0x00 0x1700000 0x0 0x100000>;
interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
+ dma-coherent;
sec_jr0: jr@10000 {
compatible = "fsl,sec-v5.4-job-ring",
ranges = <0x0 0x00 0x1700000 0x100000>;
reg = <0x00 0x1700000 0x0 0x100000>;
interrupts = <0 75 0x4>;
+ dma-coherent;
sec_jr0: jr@10000 {
compatible = "fsl,sec-v5.4-job-ring",
ranges = <0x0 0x00 0x1700000 0x100000>;
reg = <0x00 0x1700000 0x0 0x100000>;
interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
+ dma-coherent;
sec_jr0: jr@10000 {
compatible = "fsl,sec-v5.4-job-ring",
#define MX8MM_IOMUXC_SD1_CMD_USDHC1_CMD 0x0A4 0x30C 0x000 0x0 0x0
#define MX8MM_IOMUXC_SD1_CMD_GPIO2_IO1 0x0A4 0x30C 0x000 0x5 0x0
#define MX8MM_IOMUXC_SD1_DATA0_USDHC1_DATA0 0x0A8 0x310 0x000 0x0 0x0
-#define MX8MM_IOMUXC_SD1_DATA0_GPIO2_IO2 0x0A8 0x31 0x000 0x5 0x0
+#define MX8MM_IOMUXC_SD1_DATA0_GPIO2_IO2 0x0A8 0x310 0x000 0x5 0x0
#define MX8MM_IOMUXC_SD1_DATA1_USDHC1_DATA1 0x0AC 0x314 0x000 0x0 0x0
#define MX8MM_IOMUXC_SD1_DATA1_GPIO2_IO3 0x0AC 0x314 0x000 0x5 0x0
#define MX8MM_IOMUXC_SD1_DATA2_USDHC1_DATA2 0x0B0 0x318 0x000 0x0 0x0
&i2c2 {
clock-frequency = <400000>;
- pinctrl-names = "default";
+ pinctrl-names = "default", "gpio";
pinctrl-0 = <&pinctrl_i2c2>;
pinctrl-1 = <&pinctrl_i2c2_gpio>;
sda-gpios = <&gpio5 17 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
&i2c1 {
clock-frequency = <400000>;
- pinctrl-names = "default";
+ pinctrl-names = "default", "gpio";
pinctrl-0 = <&pinctrl_i2c1>;
pinctrl-1 = <&pinctrl_i2c1_gpio>;
sda-gpios = <&gpio5 15 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
#define MX8MQ_IOMUXC_SD1_CMD_USDHC1_CMD 0x0A4 0x30C 0x000 0x0 0x0
#define MX8MQ_IOMUXC_SD1_CMD_GPIO2_IO1 0x0A4 0x30C 0x000 0x5 0x0
#define MX8MQ_IOMUXC_SD1_DATA0_USDHC1_DATA0 0x0A8 0x310 0x000 0x0 0x0
-#define MX8MQ_IOMUXC_SD1_DATA0_GPIO2_IO2 0x0A8 0x31 0x000 0x5 0x0
+#define MX8MQ_IOMUXC_SD1_DATA0_GPIO2_IO2 0x0A8 0x310 0x000 0x5 0x0
#define MX8MQ_IOMUXC_SD1_DATA1_USDHC1_DATA1 0x0AC 0x314 0x000 0x0 0x0
#define MX8MQ_IOMUXC_SD1_DATA1_GPIO2_IO3 0x0AC 0x314 0x000 0x5 0x0
#define MX8MQ_IOMUXC_SD1_DATA2_USDHC1_DATA2 0x0B0 0x318 0x000 0x0 0x0
// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
/*
* Device Tree file for CZ.NIC Turris Mox Board
- * 2019 by Marek Behun <marek.behun@nic.cz>
+ * 2019 by Marek Behún <kabel@kernel.org>
*/
/dts-v1/;
};
CP11X_LABEL(sata0): sata@540000 {
- compatible = "marvell,armada-8k-ahci";
+ compatible = "marvell,armada-8k-ahci",
+ "generic-ahci";
reg = <0x540000 0x30000>;
dma-coherent;
+ interrupts = <107 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&CP11X_LABEL(clk) 1 15>,
<&CP11X_LABEL(clk) 1 16>;
#address-cells = <1>;
status = "disabled";
sata-port@0 {
- interrupts = <109 IRQ_TYPE_LEVEL_HIGH>;
reg = <0>;
};
sata-port@1 {
- interrupts = <107 IRQ_TYPE_LEVEL_HIGH>;
reg = <1>;
};
};
} while (--n > 0);
sum += ((sum >> 32) | (sum << 32));
- return csum_fold((__force u32)(sum >> 32));
+ return csum_fold((__force __wsum)(sum >> 32));
}
#define ip_fast_csum ip_fast_csum
#define ARM64_WORKAROUND_1508412 58
#define ARM64_HAS_LDAPR 59
#define ARM64_KVM_PROTECTED_MODE 60
+#define ARM64_WORKAROUND_NVIDIA_CARMEL_CNP 61
-#define ARM64_NCAPS 61
+#define ARM64_NCAPS 62
#endif /* __ASM_CPUCAPS_H */
#define CPTR_EL2_DEFAULT CPTR_EL2_RES1
/* Hyp Debug Configuration Register bits */
+#define MDCR_EL2_TTRF (1 << 19)
#define MDCR_EL2_TPMS (1 << 14)
#define MDCR_EL2_E2PB_MASK (UL(0x3))
#define MDCR_EL2_E2PB_SHIFT (UL(12))
extern struct task_struct *cpu_switch_to(struct task_struct *prev,
struct task_struct *next);
+asmlinkage void arm64_preempt_schedule_irq(void);
+
#define task_pt_regs(p) \
((struct pt_regs *)(THREAD_SIZE + task_stack_page(p)) - 1)
#define arch_setup_new_exec arch_setup_new_exec
void arch_release_task_struct(struct task_struct *tsk);
+int arch_dup_task_struct(struct task_struct *dst,
+ struct task_struct *src);
#endif
0, 0,
1, 0),
},
+#endif
+#ifdef CONFIG_NVIDIA_CARMEL_CNP_ERRATUM
+ {
+ /* NVIDIA Carmel */
+ .desc = "NVIDIA Carmel CNP erratum",
+ .capability = ARM64_WORKAROUND_NVIDIA_CARMEL_CNP,
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_NVIDIA_CARMEL),
+ },
#endif
{
}
* of support.
*/
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_PMUVER_SHIFT, 4, 0),
- ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_TRACEVER_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_DEBUGVER_SHIFT, 4, 0x6),
ARM64_FTR_END,
};
* may share TLB entries with a CPU stuck in the crashed
* kernel.
*/
- if (is_kdump_kernel())
+ if (is_kdump_kernel())
+ return false;
+
+ if (cpus_have_const_cap(ARM64_WORKAROUND_NVIDIA_CARMEL_CNP))
return false;
return has_cpuid_feature(entry, scope);
* with the CLIDR_EL1 fields to avoid triggering false warnings
* when there is a mismatch across the CPUs. Keep track of the
* effective value of the CTR_EL0 in our internal records for
- * acurate sanity check and feature enablement.
+ * accurate sanity check and feature enablement.
*/
info->reg_ctr = read_cpuid_effective_cachetype();
info->reg_dczid = read_cpuid(DCZID_EL0);
ssize_t elfcorehdr_read(char *buf, size_t count, u64 *ppos)
{
memcpy(buf, phys_to_virt((phys_addr_t)*ppos), count);
+ *ppos += count;
+
return count;
}
#include <asm/processor.h>
#include <asm/pointer_auth.h>
#include <asm/stacktrace.h>
+#include <asm/switch_to.h>
+#include <asm/system_misc.h>
#if defined(CONFIG_STACKPROTECTOR) && !defined(CONFIG_STACKPROTECTOR_PER_TASK)
#include <linux/stackprotector.h>
#ifdef CONFIG_STACKTRACE
-void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie,
- struct task_struct *task, struct pt_regs *regs)
+noinline void arch_stack_walk(stack_trace_consume_fn consume_entry,
+ void *cookie, struct task_struct *task,
+ struct pt_regs *regs)
{
struct stackframe frame;
start_backtrace(&frame, regs->regs[29], regs->pc);
else if (task == current)
start_backtrace(&frame,
- (unsigned long)__builtin_frame_address(0),
- (unsigned long)arch_stack_walk);
+ (unsigned long)__builtin_frame_address(1),
+ (unsigned long)__builtin_return_address(0));
else
start_backtrace(&frame, thread_saved_fp(task),
thread_saved_pc(task));
* - Debug ROM Address (MDCR_EL2_TDRA)
* - OS related registers (MDCR_EL2_TDOSA)
* - Statistical profiler (MDCR_EL2_TPMS/MDCR_EL2_E2PB)
+ * - Self-hosted Trace Filter controls (MDCR_EL2_TTRF)
*
* Additionally, KVM only traps guest accesses to the debug registers if
* the guest is not actively using them (see the KVM_ARM64_DEBUG_DIRTY
vcpu->arch.mdcr_el2 = __this_cpu_read(mdcr_el2) & MDCR_EL2_HPMN_MASK;
vcpu->arch.mdcr_el2 |= (MDCR_EL2_TPM |
MDCR_EL2_TPMS |
+ MDCR_EL2_TTRF |
MDCR_EL2_TPMCR |
MDCR_EL2_TDRA |
MDCR_EL2_TDOSA);
if (has_vhe())
flags = local_daif_save();
+ /*
+ * Table 11-2 "Permitted ICC_SRE_ELx.SRE settings" indicates
+ * that to be able to set ICC_SRE_EL1.SRE to 0, all the
+ * interrupt overrides must be set. You've got to love this.
+ */
+ sysreg_clear_set(hcr_el2, 0, HCR_AMO | HCR_FMO | HCR_IMO);
+ isb();
write_gicreg(0, ICC_SRE_EL1);
isb();
write_gicreg(sre, ICC_SRE_EL1);
isb();
+ sysreg_clear_set(hcr_el2, HCR_AMO | HCR_FMO | HCR_IMO, 0);
+ isb();
if (has_vhe())
local_daif_restore(flags);
struct range arch_get_mappable_range(void)
{
struct range mhp_range;
+ u64 start_linear_pa = __pa(_PAGE_OFFSET(vabits_actual));
+ u64 end_linear_pa = __pa(PAGE_END - 1);
+
+ if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) {
+ /*
+ * Check for a wrap, it is possible because of randomized linear
+ * mapping the start physical address is actually bigger than
+ * the end physical address. In this case set start to zero
+ * because [0, end_linear_pa] range must still be able to cover
+ * all addressable physical addresses.
+ */
+ if (start_linear_pa > end_linear_pa)
+ start_linear_pa = 0;
+ }
+
+ WARN_ON(start_linear_pa > end_linear_pa);
/*
* Linear mapping region is the range [PAGE_OFFSET..(PAGE_END - 1)]
* range which can be mapped inside this linear mapping range, must
* also be derived from its end points.
*/
- mhp_range.start = __pa(_PAGE_OFFSET(vabits_actual));
- mhp_range.end = __pa(PAGE_END - 1);
+ mhp_range.start = start_linear_pa;
+ mhp_range.end = end_linear_pa;
+
return mhp_range;
}
return 0;
}
-/* Ftrace callback handler for kprobes -- called under preepmt disabed */
+/* Ftrace callback handler for kprobes -- called under preepmt disabled */
void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *ops, struct ftrace_regs *fregs)
{
static inline unsigned long user_stack_pointer(struct pt_regs *regs)
{
- /* FIXME: should this be bspstore + nr_dirty regs? */
- return regs->ar_bspstore;
+ return regs->r12;
}
static inline int is_syscall_success(struct pt_regs *regs)
unsigned long __ip = instruction_pointer(regs); \
(__ip & ~3UL) + ((__ip & 3UL) << 2); \
})
-/*
- * Why not default? Because user_stack_pointer() on ia64 gives register
- * stack backing store instead...
- */
-#define current_user_stack_pointer() (current_pt_regs()->r12)
/* given a pointer to a task_struct, return the user's pt_regs */
# define task_pt_regs(t) (((struct pt_regs *) ((char *) (t) + IA64_STK_OFFSET)) - 1)
char *buf) \
{ \
u32 cpu=dev->id; \
- return sprintf(buf, "%lx\n", name[cpu]); \
+ return sprintf(buf, "%llx\n", name[cpu]); \
}
#define store(name) \
#ifdef ERR_INJ_DEBUG
printk(KERN_DEBUG "pal_mc_err_inject for cpu%d:\n", cpu);
- printk(KERN_DEBUG "err_type_info=%lx,\n", err_type_info[cpu]);
- printk(KERN_DEBUG "err_struct_info=%lx,\n", err_struct_info[cpu]);
- printk(KERN_DEBUG "err_data_buffer=%lx, %lx, %lx.\n",
+ printk(KERN_DEBUG "err_type_info=%llx,\n", err_type_info[cpu]);
+ printk(KERN_DEBUG "err_struct_info=%llx,\n", err_struct_info[cpu]);
+ printk(KERN_DEBUG "err_data_buffer=%llx, %llx, %llx.\n",
err_data_buffer[cpu].data1,
err_data_buffer[cpu].data2,
err_data_buffer[cpu].data3);
#ifdef ERR_INJ_DEBUG
printk(KERN_DEBUG "Returns: status=%d,\n", (int)status[cpu]);
- printk(KERN_DEBUG "capabilities=%lx,\n", capabilities[cpu]);
- printk(KERN_DEBUG "resources=%lx\n", resources[cpu]);
+ printk(KERN_DEBUG "capabilities=%llx,\n", capabilities[cpu]);
+ printk(KERN_DEBUG "resources=%llx\n", resources[cpu]);
#endif
return size;
}
char *buf)
{
unsigned int cpu=dev->id;
- return sprintf(buf, "%lx\n", phys_addr[cpu]);
+ return sprintf(buf, "%llx\n", phys_addr[cpu]);
}
static ssize_t
ret = get_user_pages_fast(virt_addr, 1, FOLL_WRITE, NULL);
if (ret<=0) {
#ifdef ERR_INJ_DEBUG
- printk("Virtual address %lx is not existing.\n",virt_addr);
+ printk("Virtual address %llx is not existing.\n", virt_addr);
#endif
return -EINVAL;
}
{
unsigned int cpu=dev->id;
- return sprintf(buf, "%lx, %lx, %lx\n",
+ return sprintf(buf, "%llx, %llx, %llx\n",
err_data_buffer[cpu].data1,
err_data_buffer[cpu].data2,
err_data_buffer[cpu].data3);
int ret;
#ifdef ERR_INJ_DEBUG
- printk("write err_data_buffer=[%lx,%lx,%lx] on cpu%d\n",
+ printk("write err_data_buffer=[%llx,%llx,%llx] on cpu%d\n",
err_data_buffer[cpu].data1,
err_data_buffer[cpu].data2,
err_data_buffer[cpu].data3,
cpu);
#endif
- ret=sscanf(buf, "%lx, %lx, %lx",
+ ret = sscanf(buf, "%llx, %llx, %llx",
&err_data_buffer[cpu].data1,
&err_data_buffer[cpu].data2,
&err_data_buffer[cpu].data3);
data = mca_bootmem();
first_time = 0;
} else
- data = (void *)__get_free_pages(GFP_KERNEL,
+ data = (void *)__get_free_pages(GFP_ATOMIC,
get_order(sz));
if (!data)
panic("Could not allocate MCA memory for cpu %d\n",
#include <asm/prom.h>
#ifdef CONFIG_MIPS_ELF_APPENDED_DTB
-const char __section(".appended_dtb") __appended_dtb[0x100000];
+char __section(".appended_dtb") __appended_dtb[0x100000];
#endif /* CONFIG_MIPS_ELF_APPENDED_DTB */
struct cpuinfo_mips cpu_data[NR_CPUS] __read_mostly;
.fill : {
FILL(0);
BYTE(0);
- . = ALIGN(8);
+ STRUCT_ALIGN();
}
__appended_dtb = .;
/* leave space for appended DTB */
{
struct address_space *mapping;
- mapping = page_mapping(page);
+ mapping = page_mapping_file(page);
if (mapping && !mapping_mapped(mapping))
set_bit(PG_dcache_dirty, &page->flags);
else {
#endif
case 4: return __cmpxchg_u32((unsigned int *)ptr,
(unsigned int)old, (unsigned int)new_);
- case 1: return __cmpxchg_u8((u8 *)ptr, (u8)old, (u8)new_);
+ case 1: return __cmpxchg_u8((u8 *)ptr, old & 0xff, new_ & 0xff);
}
__cmpxchg_called_with_bad_pointer();
return old;
regs->gr[23] = 0; \
} while(0)
-struct task_struct;
struct mm_struct;
/* Free all resources held by a thread. */
* Floating-point emulation code
* Copyright (C) 2001 Hewlett-Packard (Paul Bame) <bame@debian.org>
*/
-/*
- * BEGIN_DESC
- *
- * File:
- * @(#) pa/fp/fpu.h $Revision: 1.1 $
- *
- * Purpose:
- * <<please update with a synopis of the functionality provided by this file>>
- *
- *
- * END_DESC
-*/
-
-#ifdef __NO_PA_HDRS
- PA header file -- do not include this header file for non-PA builds.
-#endif
-
#ifndef _MACHINE_FPU_INCLUDED /* allows multiple inclusion */
#define _MACHINE_FPU_INCLUDED
-#if 0
-#ifndef _SYS_STDSYMS_INCLUDED
-# include <sys/stdsyms.h>
-#endif /* _SYS_STDSYMS_INCLUDED */
-#include <machine/pdc/pdc_rqsts.h>
-#endif
-
#define PA83_FPU_FLAG 0x00000001
#define PA89_FPU_FLAG 0x00000002
#define PA2_0_FPU_FLAG 0x00000010
#define COPR_FP 0x00000080 /* Floating point -- Coprocessor 0 */
#define SFU_MPY_DIVIDE 0x00008000 /* Multiply/Divide __ SFU 0 */
-
#define EM_FPU_TYPE_OFFSET 272
/* version of EMULATION software for COPR,0,0 instruction */
#define EMULATION_VERSION 4
/*
- * The only was to differeniate between TIMEX and ROLEX (or PCX-S and PCX-T)
- * is thorough the potential type field from the PDC_MODEL call. The
- * following flags are used at assist this differeniation.
+ * The only way to differentiate between TIMEX and ROLEX (or PCX-S and PCX-T)
+ * is through the potential type field from the PDC_MODEL call.
+ * The following flags are used to assist this differentiation.
*/
#define ROLEX_POTENTIAL_KEY_FLAGS PDC_MODEL_CPU_KEY_WORD_TO_IO
#define TIMEX_POTENTIAL_KEY_FLAGS (PDC_MODEL_CPU_KEY_QUAD_STORE | \
PDC_MODEL_CPU_KEY_RECIP_SQRT)
-
#endif /* ! _MACHINE_FPU_INCLUDED */
#include <linux/bug.h>
#include <asm/cputable.h>
-static inline bool early_cpu_has_feature(unsigned long feature)
+static __always_inline bool early_cpu_has_feature(unsigned long feature)
{
return !!((CPU_FTRS_ALWAYS & feature) ||
(CPU_FTRS_POSSIBLE & cur_cpu_spec->cpu_features & feature));
return static_branch_likely(&cpu_feature_keys[i]);
}
#else
-static inline bool cpu_has_feature(unsigned long feature)
+static __always_inline bool cpu_has_feature(unsigned long feature)
{
return early_cpu_has_feature(feature);
}
targets += prom_init_check
clean-files := vmlinux.lds
+
+# Force dependency (incbin is bad)
+$(obj)/vdso32_wrapper.o : $(obj)/vdso32/vdso32.so.dbg
+$(obj)/vdso64_wrapper.o : $(obj)/vdso64/vdso64.so.dbg
CFLAGS_ptrace-view.o += -DUTS_MACHINE='"$(UTS_MACHINE)"'
obj-y += ptrace.o ptrace-view.o
-obj-$(CONFIG_PPC_FPU_REGS) += ptrace-fpu.o
+obj-y += ptrace-fpu.o
obj-$(CONFIG_COMPAT) += ptrace32.o
obj-$(CONFIG_VSX) += ptrace-vsx.o
ifneq ($(CONFIG_VSX),y)
-obj-$(CONFIG_PPC_FPU_REGS) += ptrace-novsx.o
+obj-y += ptrace-novsx.o
endif
obj-$(CONFIG_ALTIVEC) += ptrace-altivec.o
obj-$(CONFIG_SPE) += ptrace-spe.o
extern const struct user_regset_view user_ppc_native_view;
/* ptrace-fpu */
-#ifdef CONFIG_PPC_FPU_REGS
int ptrace_get_fpr(struct task_struct *child, int index, unsigned long *data);
int ptrace_put_fpr(struct task_struct *child, int index, unsigned long data);
-#else
-static inline int
-ptrace_get_fpr(struct task_struct *child, int index, unsigned long *data)
-{
- return -EIO;
-}
-
-static inline int
-ptrace_put_fpr(struct task_struct *child, int index, unsigned long data)
-{
- return -EIO;
-}
-#endif
/* ptrace-(no)adv */
void ppc_gethwdinfo(struct ppc_debug_info *dbginfo);
int ptrace_get_fpr(struct task_struct *child, int index, unsigned long *data)
{
+#ifdef CONFIG_PPC_FPU_REGS
unsigned int fpidx = index - PT_FPR0;
+#endif
if (index > PT_FPSCR)
return -EIO;
+#ifdef CONFIG_PPC_FPU_REGS
flush_fp_to_thread(child);
if (fpidx < (PT_FPSCR - PT_FPR0))
memcpy(data, &child->thread.TS_FPR(fpidx), sizeof(long));
else
*data = child->thread.fp_state.fpscr;
+#else
+ *data = 0;
+#endif
return 0;
}
int ptrace_put_fpr(struct task_struct *child, int index, unsigned long data)
{
+#ifdef CONFIG_PPC_FPU_REGS
unsigned int fpidx = index - PT_FPR0;
+#endif
if (index > PT_FPSCR)
return -EIO;
+#ifdef CONFIG_PPC_FPU_REGS
flush_fp_to_thread(child);
if (fpidx < (PT_FPSCR - PT_FPR0))
memcpy(&child->thread.TS_FPR(fpidx), &data, sizeof(long));
else
child->thread.fp_state.fpscr = data;
+#endif
return 0;
}
int fpr_get(struct task_struct *target, const struct user_regset *regset,
struct membuf to)
{
+#ifdef CONFIG_PPC_FPU_REGS
BUILD_BUG_ON(offsetof(struct thread_fp_state, fpscr) !=
offsetof(struct thread_fp_state, fpr[32]));
flush_fp_to_thread(target);
return membuf_write(&to, &target->thread.fp_state, 33 * sizeof(u64));
+#else
+ return membuf_write(&to, &empty_zero_page, 33 * sizeof(u64));
+#endif
}
/*
unsigned int pos, unsigned int count,
const void *kbuf, const void __user *ubuf)
{
+#ifdef CONFIG_PPC_FPU_REGS
BUILD_BUG_ON(offsetof(struct thread_fp_state, fpscr) !=
offsetof(struct thread_fp_state, fpr[32]));
return user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&target->thread.fp_state, 0, -1);
+#else
+ return 0;
+#endif
}
.size = sizeof(long), .align = sizeof(long),
.regset_get = gpr_get, .set = gpr_set
},
-#ifdef CONFIG_PPC_FPU_REGS
[REGSET_FPR] = {
.core_note_type = NT_PRFPREG, .n = ELF_NFPREG,
.size = sizeof(double), .align = sizeof(double),
.regset_get = fpr_get, .set = fpr_set
},
-#endif
#ifdef CONFIG_ALTIVEC
[REGSET_VMX] = {
.core_note_type = NT_PPC_VMX, .n = 34,
else
prepare_save_user_regs(1);
- if (!user_write_access_begin(frame, sizeof(*frame)))
+ if (!user_access_begin(frame, sizeof(*frame)))
goto badframe;
/* Put the siginfo & fill in most of the ucontext */
unsafe_put_user(PPC_INST_ADDI + __NR_rt_sigreturn, &mctx->mc_pad[0],
failed);
unsafe_put_user(PPC_INST_SC, &mctx->mc_pad[1], failed);
+ asm("dcbst %y0; sync; icbi %y0; sync" :: "Z" (mctx->mc_pad[0]));
}
unsafe_put_sigset_t(&frame->uc.uc_sigmask, oldset, failed);
- user_write_access_end();
+ user_access_end();
if (copy_siginfo_to_user(&frame->info, &ksig->info))
goto badframe;
- if (tramp == (unsigned long)mctx->mc_pad)
- flush_icache_range(tramp, tramp + 2 * sizeof(unsigned long));
-
regs->link = tramp;
#ifdef CONFIG_PPC_FPU_REGS
return 0;
failed:
- user_write_access_end();
+ user_access_end();
badframe:
signal_fault(tsk, regs, "handle_rt_signal32", frame);
else
prepare_save_user_regs(1);
- if (!user_write_access_begin(frame, sizeof(*frame)))
+ if (!user_access_begin(frame, sizeof(*frame)))
goto badframe;
sc = (struct sigcontext __user *) &frame->sctx;
/* Set up the sigreturn trampoline: li r0,sigret; sc */
unsafe_put_user(PPC_INST_ADDI + __NR_sigreturn, &mctx->mc_pad[0], failed);
unsafe_put_user(PPC_INST_SC, &mctx->mc_pad[1], failed);
+ asm("dcbst %y0; sync; icbi %y0; sync" :: "Z" (mctx->mc_pad[0]));
}
- user_write_access_end();
-
- if (tramp == (unsigned long)mctx->mc_pad)
- flush_icache_range(tramp, tramp + 2 * sizeof(unsigned long));
+ user_access_end();
regs->link = tramp;
return 0;
failed:
- user_write_access_end();
+ user_access_end();
badframe:
signal_fault(tsk, regs, "handle_signal32", frame);
V_FUNCTION_BEGIN(__kernel_time)
cvdso_call_time __c_kernel_time
V_FUNCTION_END(__kernel_time)
+
+/* Routines for restoring integer registers, called by the compiler. */
+/* Called with r11 pointing to the stack header word of the caller of the */
+/* function, just beyond the end of the integer restore area. */
+_GLOBAL(_restgpr_31_x)
+_GLOBAL(_rest32gpr_31_x)
+ lwz r0,4(r11)
+ lwz r31,-4(r11)
+ mtlr r0
+ mr r1,r11
+ blr
want_v = hpte_encode_avpn(vpn, psize, ssize);
- flags = (newpp & 7) | H_AVPN;
+ flags = (newpp & (HPTE_R_PP | HPTE_R_N | HPTE_R_KEY_LO)) | H_AVPN;
+ flags |= (newpp & HPTE_R_KEY_HI) >> 48;
if (mmu_has_feature(MMU_FTR_KERNEL_RO))
/* Move pp0 into bit 8 (IBM 55) */
flags |= (newpp & HPTE_R_PP0) >> 55;
return ret;
}
+/**
+ * struct pseries_suspend_info - State shared between CPUs for join/suspend.
+ * @counter: Threads are to increment this upon resuming from suspend
+ * or if an error is received from H_JOIN. The thread which performs
+ * the first increment (i.e. sets it to 1) is responsible for
+ * waking the other threads.
+ * @done: False if join/suspend is in progress. True if the operation is
+ * complete (successful or not).
+ */
+struct pseries_suspend_info {
+ atomic_t counter;
+ bool done;
+};
+
static int do_join(void *arg)
{
- atomic_t *counter = arg;
+ struct pseries_suspend_info *info = arg;
+ atomic_t *counter = &info->counter;
long hvrc;
int ret;
+retry:
/* Must ensure MSR.EE off for H_JOIN. */
hard_irq_disable();
hvrc = plpar_hcall_norets(H_JOIN);
case H_SUCCESS:
/*
* The suspend is complete and this cpu has received a
- * prod.
+ * prod, or we've received a stray prod from unrelated
+ * code (e.g. paravirt spinlocks) and we need to join
+ * again.
+ *
+ * This barrier orders the return from H_JOIN above vs
+ * the load of info->done. It pairs with the barrier
+ * in the wakeup/prod path below.
*/
+ smp_mb();
+ if (READ_ONCE(info->done) == false) {
+ pr_info_ratelimited("premature return from H_JOIN on CPU %i, retrying",
+ smp_processor_id());
+ goto retry;
+ }
ret = 0;
break;
case H_BAD_MODE:
if (atomic_inc_return(counter) == 1) {
pr_info("CPU %u waking all threads\n", smp_processor_id());
+ WRITE_ONCE(info->done, true);
+ /*
+ * This barrier orders the store to info->done vs subsequent
+ * H_PRODs to wake the other CPUs. It pairs with the barrier
+ * in the H_SUCCESS case above.
+ */
+ smp_mb();
prod_others();
}
/*
int ret;
while (true) {
- atomic_t counter = ATOMIC_INIT(0);
+ struct pseries_suspend_info info;
unsigned long vasi_state;
int vasi_err;
- ret = stop_machine(do_join, &counter, cpu_online_mask);
+ info = (struct pseries_suspend_info) {
+ .counter = ATOMIC_INIT(0),
+ .done = false,
+ };
+
+ ret = stop_machine(do_join, &info, cpu_online_mask);
if (ret == 0)
break;
/*
select PCI_MSI if PCI
select RISCV_INTC
select RISCV_TIMER if RISCV_SBI
- select SPARSEMEM_STATIC if 32BIT
select SPARSE_IRQ
select SYSCTL_EXCEPTION_TRACE
select THREAD_INFO_IN_TASK
config ARCH_SPARSEMEM_ENABLE
def_bool y
depends on MMU
- select SPARSEMEM_VMEMMAP_ENABLE
+ select SPARSEMEM_STATIC if 32BIT && SPARSMEM
+ select SPARSEMEM_VMEMMAP_ENABLE if 64BIT
config ARCH_SELECT_MEMORY_MODEL
def_bool ARCH_SPARSEMEM_ENABLE
# Common NUMA Features
config NUMA
bool "NUMA Memory Allocation and Scheduler Support"
- depends on SMP
+ depends on SMP && MMU
select GENERIC_ARCH_NUMA
select OF_NUMA
select ARCH_SUPPORTS_NUMA_BALANCING
select SIFIVE_PLIC
select ARCH_HAS_RESET_CONTROLLER
select PINCTRL
+ select COMMON_CLK
+ select COMMON_CLK_K210
help
This enables support for Canaan Kendryte K210 SoC platform hardware.
long long __ashrti3(long long a, int b);
long long __ashlti3(long long a, int b);
+
+#define DECLARE_DO_ERROR_INFO(name) asmlinkage void name(struct pt_regs *regs)
+
+DECLARE_DO_ERROR_INFO(do_trap_unknown);
+DECLARE_DO_ERROR_INFO(do_trap_insn_misaligned);
+DECLARE_DO_ERROR_INFO(do_trap_insn_fault);
+DECLARE_DO_ERROR_INFO(do_trap_insn_illegal);
+DECLARE_DO_ERROR_INFO(do_trap_load_fault);
+DECLARE_DO_ERROR_INFO(do_trap_load_misaligned);
+DECLARE_DO_ERROR_INFO(do_trap_store_misaligned);
+DECLARE_DO_ERROR_INFO(do_trap_store_fault);
+DECLARE_DO_ERROR_INFO(do_trap_ecall_u);
+DECLARE_DO_ERROR_INFO(do_trap_ecall_s);
+DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
+DECLARE_DO_ERROR_INFO(do_trap_break);
+
#endif /* _ASM_RISCV_PROTOTYPES_H */
#include <asm-generic/irq.h>
+extern void __init init_IRQ(void);
+
#endif /* _ASM_RISCV_IRQ_H */
int riscv_of_parent_hartid(struct device_node *node);
extern void riscv_fill_hwcap(void);
+extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
#endif /* __ASSEMBLY__ */
extern unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs,
unsigned int n);
+void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
+ unsigned long frame_pointer);
+int do_syscall_trace_enter(struct pt_regs *regs);
+void do_syscall_trace_exit(struct pt_regs *regs);
+
/**
* regs_get_register() - get register value from its offset
* @regs: pt_regs from which register value is gotten
SBI_EXT_RFENCE_REMOTE_FENCE_I = 0,
SBI_EXT_RFENCE_REMOTE_SFENCE_VMA,
SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID,
- SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA,
SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID,
- SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA,
+ SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA,
SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID,
+ SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA,
};
enum sbi_ext_hsm_fid {
return 0;
}
+extern void time_init(void);
+
#endif /* _ASM_RISCV_TIMEX_H */
* data types like structures or arrays.
*
* @ptr must have pointer-to-simple-variable type, and @x must be assignable
- * to the result of dereferencing @ptr.
+ * to the result of dereferencing @ptr. The value of @x is copied to avoid
+ * re-ordering where @x is evaluated inside the block that enables user-space
+ * access (thus bypassing user space protection if @x is a function).
*
* Caller must check the pointer with access_ok() before calling this
* function.
#define __put_user(x, ptr) \
({ \
__typeof__(*(ptr)) __user *__gu_ptr = (ptr); \
+ __typeof__(*__gu_ptr) __val = (x); \
long __pu_err = 0; \
\
__chk_user_ptr(__gu_ptr); \
\
__enable_user_access(); \
- __put_user_nocheck(x, __gu_ptr, __pu_err); \
+ __put_user_nocheck(__val, __gu_ptr, __pu_err); \
__disable_user_access(); \
\
__pu_err; \
CFLAGS_REMOVE_patch.o = $(CC_FLAGS_FTRACE)
CFLAGS_REMOVE_sbi.o = $(CC_FLAGS_FTRACE)
endif
+CFLAGS_syscall_table.o += $(call cc-option,-Wno-override-init,)
extra-y += head.o
extra-y += vmlinux.lds
#endif
.section ".rodata"
+ .align LGREG
/* Exception vector table */
ENTRY(excp_vect_table)
RISCV_PTR do_trap_insn_misaligned
#include <linux/kprobes.h>
-/* Ftrace callback handler for kprobes -- called under preepmt disabed */
+/* Ftrace callback handler for kprobes -- called under preepmt disabled */
void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
- struct ftrace_ops *ops, struct ftrace_regs *regs)
+ struct ftrace_ops *ops, struct ftrace_regs *fregs)
{
struct kprobe *p;
+ struct pt_regs *regs;
struct kprobe_ctlblk *kcb;
p = get_kprobe((kprobe_opcode_t *)ip);
if (unlikely(!p) || kprobe_disabled(p))
return;
+ regs = ftrace_get_regs(fregs);
kcb = get_kprobe_ctlblk();
if (kprobe_running()) {
kprobes_inc_nmissed_count(p);
} else {
- unsigned long orig_ip = instruction_pointer(&(regs->regs));
+ unsigned long orig_ip = instruction_pointer(regs);
- instruction_pointer_set(&(regs->regs), ip);
+ instruction_pointer_set(regs, ip);
__this_cpu_write(current_kprobe, p);
kcb->kprobe_status = KPROBE_HIT_ACTIVE;
- if (!p->pre_handler || !p->pre_handler(p, &(regs->regs))) {
+ if (!p->pre_handler || !p->pre_handler(p, regs)) {
/*
* Emulate singlestep (and also recover regs->pc)
* as if there is a nop
*/
- instruction_pointer_set(&(regs->regs),
+ instruction_pointer_set(regs,
(unsigned long)p->addr + MCOUNT_INSN_SIZE);
if (unlikely(p->post_handler)) {
kcb->kprobe_status = KPROBE_HIT_SSDONE;
- p->post_handler(p, &(regs->regs), 0);
+ p->post_handler(p, regs, 0);
}
- instruction_pointer_set(&(regs->regs), orig_ip);
+ instruction_pointer_set(regs, orig_ip);
}
/*
* normal page fault.
*/
regs->epc = (unsigned long) cur->addr;
- if (!instruction_pointer(regs))
- BUG();
+ BUG_ON(!instruction_pointer(regs));
if (kcb->kprobe_status == KPROBE_REENTER)
restore_previous_kprobe(kcb);
#include <linux/cpu.h>
#include <linux/kernel.h>
#include <linux/sched.h>
+#include <linux/sched/debug.h>
#include <linux/sched/task_stack.h>
#include <linux/tick.h>
#include <linux/ptrace.h>
EXPORT_SYMBOL(sbi_clear_ipi);
/**
- * sbi_set_timer_v01() - Program the timer for next timer event.
+ * __sbi_set_timer_v01() - Program the timer for next timer event.
* @stime_value: The value after which next timer event should fire.
*
* Return: None
bss_res.end = __pa_symbol(__bss_stop) - 1;
bss_res.flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
- mem_res_sz = (memblock.memory.cnt + memblock.reserved.cnt) * sizeof(*mem_res);
+ /* + 1 as memblock_alloc() might increase memblock.reserved.cnt */
+ mem_res_sz = (memblock.memory.cnt + memblock.reserved.cnt + 1) * sizeof(*mem_res);
mem_res = memblock_alloc(mem_res_sz, SMP_CACHE_BYTES);
if (!mem_res)
panic("%s: Failed to allocate %zu bytes\n", __func__, mem_res_sz);
#include <asm/stacktrace.h>
-register const unsigned long sp_in_global __asm__("sp");
+register unsigned long sp_in_global __asm__("sp");
#ifdef CONFIG_FRAME_POINTER
#include <linux/delay.h>
#include <asm/sbi.h>
#include <asm/processor.h>
+#include <asm/timex.h>
unsigned long riscv_timebase;
EXPORT_SYMBOL_GPL(riscv_timebase);
#include <linux/module.h>
#include <linux/irq.h>
+#include <asm/asm-prototypes.h>
#include <asm/bug.h>
#include <asm/processor.h>
#include <asm/ptrace.h>
memset(start, KASAN_SHADOW_INIT, end - start);
}
-void __init kasan_shallow_populate(void *start, void *end)
+static void __init kasan_shallow_populate(void *start, void *end)
{
unsigned long vaddr = (unsigned long)start & PAGE_MASK;
unsigned long vend = PAGE_ALIGN((unsigned long)end);
}
vaddr += PAGE_SIZE;
}
+
+ local_flush_tlb_all();
}
void __init kasan_init(void)
break;
kasan_populate(kasan_mem_to_shadow(start), kasan_mem_to_shadow(end));
- };
+ }
for (i = 0; i < PTRS_PER_PTE; i++)
set_pte(&kasan_early_shadow_pte[i],
----------------------------------------------------------------------------- */
/* Base stuff */
int zpci_create_device(u32 fid, u32 fh, enum zpci_state state);
-void zpci_remove_device(struct zpci_dev *zdev);
+void zpci_remove_device(struct zpci_dev *zdev, bool set_error);
int zpci_enable_device(struct zpci_dev *);
int zpci_disable_device(struct zpci_dev *);
int zpci_register_ioat(struct zpci_dev *, u8, u64, u64, u64);
STACK_TYPE_IRQ,
STACK_TYPE_NODAT,
STACK_TYPE_RESTART,
+ STACK_TYPE_MCCK,
};
struct stack_info {
#include <vdso/datapage.h>
struct arch_vdso_data {
- __u64 tod_steering_delta;
+ __s64 tod_steering_delta;
__u64 tod_steering_end;
};
static int diag8_response(int cmdlen, char *response, int *rlen)
{
+ unsigned long _cmdlen = cmdlen | 0x40000000L;
+ unsigned long _rlen = *rlen;
register unsigned long reg2 asm ("2") = (addr_t) cpcmd_buf;
register unsigned long reg3 asm ("3") = (addr_t) response;
- register unsigned long reg4 asm ("4") = cmdlen | 0x40000000L;
- register unsigned long reg5 asm ("5") = *rlen;
+ register unsigned long reg4 asm ("4") = _cmdlen;
+ register unsigned long reg5 asm ("5") = _rlen;
asm volatile(
" diag %2,%0,0x8\n"
return in_stack(sp, info, STACK_TYPE_NODAT, top - THREAD_SIZE, top);
}
+static bool in_mcck_stack(unsigned long sp, struct stack_info *info)
+{
+ unsigned long frame_size, top;
+
+ frame_size = STACK_FRAME_OVERHEAD + sizeof(struct pt_regs);
+ top = S390_lowcore.mcck_stack + frame_size;
+ return in_stack(sp, info, STACK_TYPE_MCCK, top - THREAD_SIZE, top);
+}
+
static bool in_restart_stack(unsigned long sp, struct stack_info *info)
{
unsigned long frame_size, top;
/* Check per-cpu stacks */
if (!in_irq_stack(sp, info) &&
!in_nodat_stack(sp, info) &&
- !in_restart_stack(sp, info))
+ !in_restart_stack(sp, info) &&
+ !in_mcck_stack(sp, info))
goto unknown;
recursion_check:
memcpy(®s->int_code, &S390_lowcore.ext_cpu_addr, 4);
regs->int_parm = S390_lowcore.ext_params;
- regs->int_parm_long = *(unsigned long *)S390_lowcore.ext_params2;
+ regs->int_parm_long = S390_lowcore.ext_params2;
from_idle = !user_mode(regs) && regs->psw.addr == (unsigned long)psw_idle_exit;
if (from_idle)
*/
static size_t cf_diag_needspace(unsigned int sets)
{
- struct cpu_cf_events *cpuhw = this_cpu_ptr(&cpu_cf_events);
+ struct cpu_cf_events *cpuhw = get_cpu_ptr(&cpu_cf_events);
size_t bytes = 0;
int i;
sizeof(((struct s390_ctrset_cpudata *)0)->no_sets));
debug_sprintf_event(cf_diag_dbg, 5, "%s bytes %ld\n", __func__,
bytes);
+ put_cpu_ptr(&cpu_cf_events);
return bytes;
}
if (!new)
panic("Couldn't allocate machine check stack");
WRITE_ONCE(S390_lowcore.mcck_stack, new + STACK_INIT_OFFSET);
- memblock_free(old, THREAD_SIZE);
+ memblock_free_late(old, THREAD_SIZE);
return 0;
}
early_initcall(stack_realloc);
{
struct ptff_qto qto;
struct ptff_qui qui;
+ int cs;
/* Initialize TOD steering parameters */
tod_steering_end = tod_clock_base.tod;
- vdso_data->arch_data.tod_steering_end = tod_steering_end;
+ for (cs = 0; cs < CS_BASES; cs++)
+ vdso_data[cs].arch_data.tod_steering_end = tod_steering_end;
if (!test_facility(28))
return;
{
unsigned long now, adj;
struct ptff_qto qto;
+ int cs;
/* Fixup the monotonic sched clock. */
tod_clock_base.eitod += delta;
panic("TOD clock sync offset %li is too large to drift\n",
tod_steering_delta);
tod_steering_end = now + (abs(tod_steering_delta) << 15);
- vdso_data->arch_data.tod_steering_end = tod_steering_end;
+ for (cs = 0; cs < CS_BASES; cs++) {
+ vdso_data[cs].arch_data.tod_steering_end = tod_steering_end;
+ vdso_data[cs].arch_data.tod_steering_delta = tod_steering_delta;
+ }
/* Update LPAR offset. */
if (ptff_query(PTFF_QTO) && ptff(&qto, sizeof(qto), PTFF_QTO) == 0)
avg_steal = S390_lowcore.avg_steal_timer / 2;
if ((s64) steal > 0) {
S390_lowcore.steal_timer = 0;
- account_steal_time(steal);
+ account_steal_time(cputime_to_nsecs(steal));
avg_steal += steal;
}
S390_lowcore.avg_steal_timer = avg_steal;
}
EXPORT_SYMBOL_GPL(zpci_disable_device);
-void zpci_remove_device(struct zpci_dev *zdev)
+/* zpci_remove_device - Removes the given zdev from the PCI core
+ * @zdev: the zdev to be removed from the PCI core
+ * @set_error: if true the device's error state is set to permanent failure
+ *
+ * Sets a zPCI device to a configured but offline state; the zPCI
+ * device is still accessible through its hotplug slot and the zPCI
+ * API but is removed from the common code PCI bus, making it
+ * no longer available to drivers.
+ */
+void zpci_remove_device(struct zpci_dev *zdev, bool set_error)
{
struct zpci_bus *zbus = zdev->zbus;
struct pci_dev *pdev;
+ if (!zdev->zbus->bus)
+ return;
+
pdev = pci_get_slot(zbus->bus, zdev->devfn);
if (pdev) {
- if (pdev->is_virtfn)
- return zpci_iov_remove_virtfn(pdev, zdev->vfn);
+ if (set_error)
+ pdev->error_state = pci_channel_io_perm_failure;
+ if (pdev->is_virtfn) {
+ zpci_iov_remove_virtfn(pdev, zdev->vfn);
+ /* balance pci_get_slot */
+ pci_dev_put(pdev);
+ return;
+ }
pci_stop_and_remove_bus_device_locked(pdev);
+ /* balance pci_get_slot */
+ pci_dev_put(pdev);
}
}
struct zpci_dev *zdev = container_of(kref, struct zpci_dev, kref);
if (zdev->zbus->bus)
- zpci_remove_device(zdev);
+ zpci_remove_device(zdev, false);
switch (zdev->state) {
case ZPCI_FN_STATE_ONLINE:
static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf)
{
struct zpci_dev *zdev = get_zdev_by_fid(ccdf->fid);
- struct pci_dev *pdev = NULL;
enum zpci_state state;
+ struct pci_dev *pdev;
int ret;
- if (zdev && zdev->zbus->bus)
- pdev = pci_get_slot(zdev->zbus->bus, zdev->devfn);
-
zpci_err("avail CCDF:\n");
zpci_err_hex(ccdf, sizeof(*ccdf));
case 0x0303: /* Deconfiguration requested */
if (!zdev)
break;
- if (pdev)
- zpci_remove_device(zdev);
+ zpci_remove_device(zdev, false);
ret = zpci_disable_device(zdev);
if (ret)
case 0x0304: /* Configured -> Standby|Reserved */
if (!zdev)
break;
- if (pdev) {
- /* Give the driver a hint that the function is
- * already unusable. */
- pdev->error_state = pci_channel_io_perm_failure;
- zpci_remove_device(zdev);
- }
+ /* Give the driver a hint that the function is
+ * already unusable.
+ */
+ zpci_remove_device(zdev, true);
zdev->fh = ccdf->fh;
zpci_disable_device(zdev);
REALMODE_CFLAGS := -m16 -g -Os -DDISABLE_BRANCH_PROFILING \
-Wall -Wstrict-prototypes -march=i386 -mregparm=3 \
-fno-strict-aliasing -fomit-frame-pointer -fno-pic \
- -mno-mmx -mno-sse
+ -mno-mmx -mno-sse $(call cc-option,-fcf-protection=none)
REALMODE_CFLAGS += -ffreestanding
REALMODE_CFLAGS += -fno-stack-protector
return ret;
if (event->attr.precise_ip) {
+ if ((event->attr.config & INTEL_ARCH_EVENT_MASK) == INTEL_FIXED_VLBR_EVENT)
+ return -EINVAL;
+
if (!(event->attr.freq || (event->attr.wakeup_events && !event->attr.watermark))) {
event->hw.flags |= PERF_X86_EVENT_AUTO_RELOAD;
if (!(event->attr.sample_type &
*/
if (!pebs_status && cpuc->pebs_enabled &&
!(cpuc->pebs_enabled & (cpuc->pebs_enabled-1)))
- pebs_status = cpuc->pebs_enabled;
+ pebs_status = p->status = cpuc->pebs_enabled;
bit = find_first_bit((unsigned long *)&pebs_status,
x86_pmu.max_pebs_events);
else
set_pte(pte, __pte(pte_val(*pte) | _PAGE_PRESENT));
- /* Flush this CPU's TLB. */
+ /*
+ * Flush this CPU's TLB, assuming whoever did the allocation/free is
+ * likely to continue running on this CPU.
+ */
+ preempt_disable();
flush_tlb_one_kernel(addr);
+ preempt_enable();
return true;
}
u64 options;
};
+/* Current state of Hyper-V TSC page clocksource */
+enum hv_tsc_page_status {
+ /* TSC page was not set up or disabled */
+ HV_TSC_PAGE_UNSET = 0,
+ /* TSC page MSR was written by the guest, update pending */
+ HV_TSC_PAGE_GUEST_CHANGED,
+ /* TSC page MSR was written by KVM userspace, update pending */
+ HV_TSC_PAGE_HOST_CHANGED,
+ /* TSC page was properly set up and is currently active */
+ HV_TSC_PAGE_SET,
+ /* TSC page is currently being updated and therefore is inactive */
+ HV_TSC_PAGE_UPDATING,
+ /* TSC page was set up with an inaccessible GPA */
+ HV_TSC_PAGE_BROKEN,
+};
+
/* Hyper-V emulation context */
struct kvm_hv {
struct mutex hv_lock;
u64 hv_guest_os_id;
u64 hv_hypercall;
u64 hv_tsc_page;
+ enum hv_tsc_page_status hv_tsc_page_status;
/* Hyper-v based guest crash (NT kernel bugcheck) parameters */
u64 hv_crash_param[HV_X64_MSR_CRASH_PARAMS];
KVM_IRQCHIP_SPLIT, /* created with KVM_CAP_SPLIT_IRQCHIP */
};
+struct kvm_x86_msr_filter {
+ u8 count;
+ bool default_allow:1;
+ struct msr_bitmap_range ranges[16];
+};
+
#define APICV_INHIBIT_REASON_DISABLE 0
#define APICV_INHIBIT_REASON_HYPERV 1
#define APICV_INHIBIT_REASON_NESTED 2
bool guest_can_read_msr_platform_info;
bool exception_payload_enabled;
+ bool bus_lock_detection_enabled;
+
/* Deflect RDMSR and WRMSR to user space when they trigger a #GP */
u32 user_space_msr_mask;
-
- struct {
- u8 count;
- bool default_allow:1;
- struct msr_bitmap_range ranges[16];
- } msr_filter;
-
- bool bus_lock_detection_enabled;
+ struct kvm_x86_msr_filter __rcu *msr_filter;
struct kvm_pmu_event_filter __rcu *pmu_event_filter;
struct task_struct *nx_lpage_recovery_thread;
*size = fpu_kernel_xstate_size;
}
-/*
- * Thread-synchronous status.
- *
- * This is different from the flags in that nobody else
- * ever touches our thread-synchronous status, so we don't
- * have to worry about atomic accesses.
- */
-#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
-
static inline void
native_load_sp0(unsigned long sp0)
{
void play_dead_common(void);
void wbinvd_on_cpu(int cpu);
int wbinvd_on_all_cpus(void);
+void cond_wakeup_cpu0(void);
void native_smp_send_reschedule(int cpu);
void native_send_call_func_ipi(const struct cpumask *mask);
#endif
+/*
+ * Thread-synchronous status.
+ *
+ * This is different from the flags in that nobody else
+ * ever touches our thread-synchronous status, so we don't
+ * have to worry about atomic accesses.
+ */
+#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
+
+#ifndef __ASSEMBLY__
#ifdef CONFIG_COMPAT
#define TS_I386_REGS_POKED 0x0004 /* regs poked by 32-bit ptracer */
+
+#define arch_set_restart_data(restart) \
+ do { restart->arch_data = current_thread_info()->status; } while (0)
+
#endif
-#ifndef __ASSEMBLY__
#ifdef CONFIG_X86_32
#define in_ia32_syscall() true
}
#endif
-/*
- * The maximum amount of extra memory compared to the base size. The
- * main scaling factor is the size of struct page. At extreme ratios
- * of base:extra, all the base memory can be filled with page
- * structures for the extra memory, leaving no space for anything
- * else.
- *
- * 10x seems like a reasonable balance between scaling flexibility and
- * leaving a practically usable system.
- */
-#define XEN_EXTRA_MEM_RATIO (10)
-
/*
* Helper functions to write or read unsigned long values to/from
* memory, when the access may fault.
/*
* Initialize the ACPI boot-time table parser.
*/
- if (acpi_table_init()) {
+ if (acpi_locate_initial_tables())
disable_acpi();
- return;
- }
+ else
+ acpi_reserve_initial_tables();
+}
+
+int __init early_acpi_boot_init(void)
+{
+ if (acpi_disabled)
+ return 1;
+
+ acpi_table_init_complete();
acpi_table_parse(ACPI_SIG_BOOT, acpi_parse_sbf);
} else {
printk(KERN_WARNING PREFIX "Disabling ACPI support\n");
disable_acpi();
- return;
+ return 1;
}
}
-}
-
-int __init early_acpi_boot_init(void)
-{
- /*
- * If acpi_disabled, bail out
- */
- if (acpi_disabled)
- return 1;
/*
* Process the Multiple APIC Description Table (MADT), if present
[0 ... NR_CPUS - 1] = -1,
};
+bool arch_match_cpu_phys_id(int cpu, u64 phys_id)
+{
+ return phys_id == cpuid_to_apicid[cpu];
+}
+
#ifdef CONFIG_SMP
/**
* apic_id_is_primary_thread - Check whether APIC ID belongs to a primary thread
if (idx >= 0 && test_bit(mp_irqs[idx].srcbus, mp_bus_not_pci)) {
irq = mp_irqs[idx].srcbusirq;
legacy = mp_is_legacy_irq(irq);
+ /*
+ * IRQ2 is unusable for historical reasons on systems which
+ * have a legacy PIC. See the comment vs. IRQ2 further down.
+ *
+ * If this gets removed at some point then the related code
+ * in lapic_assign_system_vectors() needs to be adjusted as
+ * well.
+ */
+ if (legacy && irq == PIC_CASCADE_IR)
+ return -EINVAL;
}
mutex_lock(&ioapic_mutex);
#include "common.h"
-/* Ftrace callback handler for kprobes -- called under preepmt disabed */
+/* Ftrace callback handler for kprobes -- called under preepmt disabled */
void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *ops, struct ftrace_regs *fregs)
{
static void kvm_wait(u8 *ptr, u8 val)
{
- unsigned long flags;
-
if (in_nmi())
return;
- local_irq_save(flags);
-
- if (READ_ONCE(*ptr) != val)
- goto out;
-
/*
* halt until it's our turn and kicked. Note that we do safe halt
* for irq enabled case to avoid hang when lock info is overwritten
* in irq spinlock slowpath and no spurious interrupt occur to save us.
*/
- if (arch_irqs_disabled_flags(flags))
- halt();
- else
- safe_halt();
+ if (irqs_disabled()) {
+ if (READ_ONCE(*ptr) == val)
+ halt();
+ } else {
+ local_irq_disable();
-out:
- local_irq_restore(flags);
+ if (READ_ONCE(*ptr) == val)
+ safe_halt();
+
+ local_irq_enable();
+ }
}
#ifdef CONFIG_X86_32
cleanup_highmap();
+ /* Look for ACPI tables and reserve memory occupied by them. */
+ acpi_boot_table_init();
+
memblock_set_current_limit(ISA_END_ADDRESS);
e820__memblock_setup();
early_platform_quirks();
- /*
- * Parse the ACPI tables for possible boot-time SMP configuration.
- */
- acpi_boot_table_init();
-
early_acpi_boot_init();
initmem_init();
static inline unsigned long get_nr_restart_syscall(const struct pt_regs *regs)
{
- /*
- * This function is fundamentally broken as currently
- * implemented.
- *
- * The idea is that we want to trigger a call to the
- * restart_block() syscall and that we want in_ia32_syscall(),
- * in_x32_syscall(), etc. to match whatever they were in the
- * syscall being restarted. We assume that the syscall
- * instruction at (regs->ip - 2) matches whatever syscall
- * instruction we used to enter in the first place.
- *
- * The problem is that we can get here when ptrace pokes
- * syscall-like values into regs even if we're not in a syscall
- * at all.
- *
- * For now, we maintain historical behavior and guess based on
- * stored state. We could do better by saving the actual
- * syscall arch in restart_block or (with caveats on x32) by
- * checking if regs->ip points to 'int $0x80'. The current
- * behavior is incorrect if a tracer has a different bitness
- * than the tracee.
- */
#ifdef CONFIG_IA32_EMULATION
- if (current_thread_info()->status & (TS_COMPAT|TS_I386_REGS_POKED))
+ if (current->restart_block.arch_data & TS_COMPAT)
return __NR_ia32_restart_syscall;
#endif
#ifdef CONFIG_X86_X32_ABI
local_irq_disable();
}
-static bool wakeup_cpu0(void)
+/**
+ * cond_wakeup_cpu0 - Wake up CPU0 if needed.
+ *
+ * If NMI wants to wake up CPU0, start CPU0.
+ */
+void cond_wakeup_cpu0(void)
{
if (smp_processor_id() == 0 && enable_start_cpu0)
- return true;
-
- return false;
+ start_cpu0();
}
+EXPORT_SYMBOL_GPL(cond_wakeup_cpu0);
/*
* We need to flush the caches before going to sleep, lest we have
__monitor(mwait_ptr, 0, 0);
mb();
__mwait(eax, 0);
- /*
- * If NMI wants to wake up CPU0, start CPU0.
- */
- if (wakeup_cpu0())
- start_cpu0();
+
+ cond_wakeup_cpu0();
}
}
while (1) {
native_halt();
- /*
- * If NMI wants to wake up CPU0, start CPU0.
- */
- if (wakeup_cpu0())
- start_cpu0();
+
+ cond_wakeup_cpu0();
}
}
tsk->thread.trap_nr = X86_TRAP_GP;
if (fixup_vdso_exception(regs, X86_TRAP_GP, error_code, 0))
- return;
+ goto exit;
show_signal(tsk, SIGSEGV, "", desc, regs, error_code);
force_sig(SIGSEGV);
goto exit;
if (fixup_vdso_exception(regs, trapnr, 0, 0))
- return;
+ goto exit;
force_sig_fault(SIGFPE, si_code,
(void __user *)uprobe_get_trap_addr(regs));
# SPDX-License-Identifier: GPL-2.0
-ccflags-y += -Iarch/x86/kvm
+ccflags-y += -I $(srctree)/arch/x86/kvm
ccflags-$(CONFIG_KVM_WERROR) += -Werror
ifeq ($(CONFIG_FRAME_POINTER),y)
u64 tsc;
/*
- * The guest has not set up the TSC page or the clock isn't
- * stable, fall back to get_kvmclock_ns.
+ * Fall back to get_kvmclock_ns() when TSC page hasn't been set up,
+ * is broken, disabled or being updated.
*/
- if (!hv->tsc_ref.tsc_sequence)
+ if (hv->hv_tsc_page_status != HV_TSC_PAGE_SET)
return div_u64(get_kvmclock_ns(kvm), 100);
vcpu = kvm_get_vcpu(kvm, 0);
return true;
}
+/*
+ * Don't touch TSC page values if the guest has opted for TSC emulation after
+ * migration. KVM doesn't fully support reenlightenment notifications and TSC
+ * access emulation and Hyper-V is known to expect the values in TSC page to
+ * stay constant before TSC access emulation is disabled from guest side
+ * (HV_X64_MSR_TSC_EMULATION_STATUS). KVM userspace is expected to preserve TSC
+ * frequency and guest visible TSC value across migration (and prevent it when
+ * TSC scaling is unsupported).
+ */
+static inline bool tsc_page_update_unsafe(struct kvm_hv *hv)
+{
+ return (hv->hv_tsc_page_status != HV_TSC_PAGE_GUEST_CHANGED) &&
+ hv->hv_tsc_emulation_control;
+}
+
void kvm_hv_setup_tsc_page(struct kvm *kvm,
struct pvclock_vcpu_time_info *hv_clock)
{
BUILD_BUG_ON(sizeof(tsc_seq) != sizeof(hv->tsc_ref.tsc_sequence));
BUILD_BUG_ON(offsetof(struct ms_hyperv_tsc_page, tsc_sequence) != 0);
- if (!(hv->hv_tsc_page & HV_X64_MSR_TSC_REFERENCE_ENABLE))
+ if (hv->hv_tsc_page_status == HV_TSC_PAGE_BROKEN ||
+ hv->hv_tsc_page_status == HV_TSC_PAGE_UNSET)
return;
mutex_lock(&hv->hv_lock);
*/
if (unlikely(kvm_read_guest(kvm, gfn_to_gpa(gfn),
&tsc_seq, sizeof(tsc_seq))))
+ goto out_err;
+
+ if (tsc_seq && tsc_page_update_unsafe(hv)) {
+ if (kvm_read_guest(kvm, gfn_to_gpa(gfn), &hv->tsc_ref, sizeof(hv->tsc_ref)))
+ goto out_err;
+
+ hv->hv_tsc_page_status = HV_TSC_PAGE_SET;
goto out_unlock;
+ }
/*
* While we're computing and writing the parameters, force the
hv->tsc_ref.tsc_sequence = 0;
if (kvm_write_guest(kvm, gfn_to_gpa(gfn),
&hv->tsc_ref, sizeof(hv->tsc_ref.tsc_sequence)))
- goto out_unlock;
+ goto out_err;
if (!compute_tsc_page_parameters(hv_clock, &hv->tsc_ref))
- goto out_unlock;
+ goto out_err;
/* Ensure sequence is zero before writing the rest of the struct. */
smp_wmb();
if (kvm_write_guest(kvm, gfn_to_gpa(gfn), &hv->tsc_ref, sizeof(hv->tsc_ref)))
- goto out_unlock;
+ goto out_err;
/*
* Now switch to the TSC page mechanism by writing the sequence.
smp_wmb();
hv->tsc_ref.tsc_sequence = tsc_seq;
- kvm_write_guest(kvm, gfn_to_gpa(gfn),
- &hv->tsc_ref, sizeof(hv->tsc_ref.tsc_sequence));
+ if (kvm_write_guest(kvm, gfn_to_gpa(gfn),
+ &hv->tsc_ref, sizeof(hv->tsc_ref.tsc_sequence)))
+ goto out_err;
+
+ hv->hv_tsc_page_status = HV_TSC_PAGE_SET;
+ goto out_unlock;
+
+out_err:
+ hv->hv_tsc_page_status = HV_TSC_PAGE_BROKEN;
+out_unlock:
+ mutex_unlock(&hv->hv_lock);
+}
+
+void kvm_hv_invalidate_tsc_page(struct kvm *kvm)
+{
+ struct kvm_hv *hv = to_kvm_hv(kvm);
+ u64 gfn;
+
+ if (hv->hv_tsc_page_status == HV_TSC_PAGE_BROKEN ||
+ hv->hv_tsc_page_status == HV_TSC_PAGE_UNSET ||
+ tsc_page_update_unsafe(hv))
+ return;
+
+ mutex_lock(&hv->hv_lock);
+
+ if (!(hv->hv_tsc_page & HV_X64_MSR_TSC_REFERENCE_ENABLE))
+ goto out_unlock;
+
+ /* Preserve HV_TSC_PAGE_GUEST_CHANGED/HV_TSC_PAGE_HOST_CHANGED states */
+ if (hv->hv_tsc_page_status == HV_TSC_PAGE_SET)
+ hv->hv_tsc_page_status = HV_TSC_PAGE_UPDATING;
+
+ gfn = hv->hv_tsc_page >> HV_X64_MSR_TSC_REFERENCE_ADDRESS_SHIFT;
+
+ hv->tsc_ref.tsc_sequence = 0;
+ if (kvm_write_guest(kvm, gfn_to_gpa(gfn),
+ &hv->tsc_ref, sizeof(hv->tsc_ref.tsc_sequence)))
+ hv->hv_tsc_page_status = HV_TSC_PAGE_BROKEN;
+
out_unlock:
mutex_unlock(&hv->hv_lock);
}
}
case HV_X64_MSR_REFERENCE_TSC:
hv->hv_tsc_page = data;
- if (hv->hv_tsc_page & HV_X64_MSR_TSC_REFERENCE_ENABLE)
+ if (hv->hv_tsc_page & HV_X64_MSR_TSC_REFERENCE_ENABLE) {
+ if (!host)
+ hv->hv_tsc_page_status = HV_TSC_PAGE_GUEST_CHANGED;
+ else
+ hv->hv_tsc_page_status = HV_TSC_PAGE_HOST_CHANGED;
kvm_make_request(KVM_REQ_MASTERCLOCK_UPDATE, vcpu);
+ } else {
+ hv->hv_tsc_page_status = HV_TSC_PAGE_UNSET;
+ }
break;
case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
return kvm_hv_msr_set_crash_data(kvm,
hv->hv_tsc_emulation_control = data;
break;
case HV_X64_MSR_TSC_EMULATION_STATUS:
+ if (data && !host)
+ return 1;
+
hv->hv_tsc_emulation_status = data;
break;
case HV_X64_MSR_TIME_REF_COUNT:
void kvm_hv_setup_tsc_page(struct kvm *kvm,
struct pvclock_vcpu_time_info *hv_clock);
+void kvm_hv_invalidate_tsc_page(struct kvm *kvm);
void kvm_hv_init_vm(struct kvm *kvm);
void kvm_hv_destroy_vm(struct kvm *kvm);
struct kvm_mmu_page *sp;
unsigned int ratio;
LIST_HEAD(invalid_list);
+ bool flush = false;
ulong to_zap;
rcu_idx = srcu_read_lock(&kvm->srcu);
lpage_disallowed_link);
WARN_ON_ONCE(!sp->lpage_disallowed);
if (is_tdp_mmu_page(sp)) {
- kvm_tdp_mmu_zap_gfn_range(kvm, sp->gfn,
- sp->gfn + KVM_PAGES_PER_HPAGE(sp->role.level));
+ flush |= kvm_tdp_mmu_zap_sp(kvm, sp);
} else {
kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list);
WARN_ON_ONCE(sp->lpage_disallowed);
}
if (need_resched() || rwlock_needbreak(&kvm->mmu_lock)) {
- kvm_mmu_commit_zap_page(kvm, &invalid_list);
+ kvm_mmu_remote_flush_or_zap(kvm, &invalid_list, flush);
cond_resched_rwlock_write(&kvm->mmu_lock);
+ flush = false;
}
}
- kvm_mmu_commit_zap_page(kvm, &invalid_list);
+ kvm_mmu_remote_flush_or_zap(kvm, &invalid_list, flush);
write_unlock(&kvm->mmu_lock);
srcu_read_unlock(&kvm->srcu, rcu_idx);
return to_shadow_page(__pa(sptep));
}
+static inline int kvm_mmu_page_as_id(struct kvm_mmu_page *sp)
+{
+ return sp->role.smm ? 1 : 0;
+}
+
static inline bool kvm_vcpu_ad_need_write_protect(struct kvm_vcpu *vcpu)
{
/*
return gfn & -KVM_PAGES_PER_HPAGE(level);
}
+/*
+ * Return the TDP iterator to the root PT and allow it to continue its
+ * traversal over the paging structure from there.
+ */
+void tdp_iter_restart(struct tdp_iter *iter)
+{
+ iter->yielded_gfn = iter->next_last_level_gfn;
+ iter->level = iter->root_level;
+
+ iter->gfn = round_gfn_for_level(iter->next_last_level_gfn, iter->level);
+ tdp_iter_refresh_sptep(iter);
+
+ iter->valid = true;
+}
+
/*
* Sets a TDP iterator to walk a pre-order traversal of the paging structure
* rooted at root_pt, starting with the walk to translate next_last_level_gfn.
WARN_ON(root_level > PT64_ROOT_MAX_LEVEL);
iter->next_last_level_gfn = next_last_level_gfn;
- iter->yielded_gfn = iter->next_last_level_gfn;
iter->root_level = root_level;
iter->min_level = min_level;
- iter->level = root_level;
- iter->pt_path[iter->level - 1] = (tdp_ptep_t)root_pt;
-
- iter->gfn = round_gfn_for_level(iter->next_last_level_gfn, iter->level);
- tdp_iter_refresh_sptep(iter);
+ iter->pt_path[iter->root_level - 1] = (tdp_ptep_t)root_pt;
+ iter->as_id = kvm_mmu_page_as_id(sptep_to_sp(root_pt));
- iter->valid = true;
+ tdp_iter_restart(iter);
}
/*
iter->valid = false;
}
-tdp_ptep_t tdp_iter_root_pt(struct tdp_iter *iter)
-{
- return iter->pt_path[iter->root_level - 1];
-}
-
int min_level;
/* The iterator's current level within the paging structure */
int level;
+ /* The address space ID, i.e. SMM vs. regular. */
+ int as_id;
/* A snapshot of the value at sptep */
u64 old_spte;
/*
void tdp_iter_start(struct tdp_iter *iter, u64 *root_pt, int root_level,
int min_level, gfn_t next_last_level_gfn);
void tdp_iter_next(struct tdp_iter *iter);
-tdp_ptep_t tdp_iter_root_pt(struct tdp_iter *iter);
+void tdp_iter_restart(struct tdp_iter *iter);
#endif /* __KVM_X86_MMU_TDP_ITER_H */
list_for_each_entry(_root, &_kvm->arch.tdp_mmu_roots, link)
static bool zap_gfn_range(struct kvm *kvm, struct kvm_mmu_page *root,
- gfn_t start, gfn_t end, bool can_yield);
+ gfn_t start, gfn_t end, bool can_yield, bool flush);
void kvm_tdp_mmu_free_root(struct kvm *kvm, struct kvm_mmu_page *root)
{
list_del(&root->link);
- zap_gfn_range(kvm, root, 0, max_gfn, false);
+ zap_gfn_range(kvm, root, 0, max_gfn, false, false);
free_page((unsigned long)root->spt);
kmem_cache_free(mmu_page_header_cache, root);
u64 old_spte, u64 new_spte, int level,
bool shared);
-static int kvm_mmu_page_as_id(struct kvm_mmu_page *sp)
-{
- return sp->role.smm ? 1 : 0;
-}
-
static void handle_changed_spte_acc_track(u64 old_spte, u64 new_spte, int level)
{
bool pfn_changed = spte_to_pfn(old_spte) != spte_to_pfn(new_spte);
*
* Given a page table that has been removed from the TDP paging structure,
* iterates through the page table to clear SPTEs and free child page tables.
+ *
+ * Note that pt is passed in as a tdp_ptep_t, but it does not need RCU
+ * protection. Since this thread removed it from the paging structure,
+ * this thread will be responsible for ensuring the page is freed. Hence the
+ * early rcu_dereferences in the function.
*/
-static void handle_removed_tdp_mmu_page(struct kvm *kvm, u64 *pt,
+static void handle_removed_tdp_mmu_page(struct kvm *kvm, tdp_ptep_t pt,
bool shared)
{
- struct kvm_mmu_page *sp = sptep_to_sp(pt);
+ struct kvm_mmu_page *sp = sptep_to_sp(rcu_dereference(pt));
int level = sp->role.level;
gfn_t base_gfn = sp->gfn;
u64 old_child_spte;
tdp_mmu_unlink_page(kvm, sp, shared);
for (i = 0; i < PT64_ENT_PER_PAGE; i++) {
- sptep = pt + i;
+ sptep = rcu_dereference(pt) + i;
gfn = base_gfn + (i * KVM_PAGES_PER_HPAGE(level - 1));
if (shared) {
struct tdp_iter *iter,
u64 new_spte)
{
- u64 *root_pt = tdp_iter_root_pt(iter);
- struct kvm_mmu_page *root = sptep_to_sp(root_pt);
- int as_id = kvm_mmu_page_as_id(root);
-
lockdep_assert_held_read(&kvm->mmu_lock);
/*
new_spte) != iter->old_spte)
return false;
- handle_changed_spte(kvm, as_id, iter->gfn, iter->old_spte, new_spte,
- iter->level, true);
+ handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte,
+ new_spte, iter->level, true);
return true;
}
* here since the SPTE is going from non-present
* to non-present.
*/
- WRITE_ONCE(*iter->sptep, 0);
+ WRITE_ONCE(*rcu_dereference(iter->sptep), 0);
return true;
}
u64 new_spte, bool record_acc_track,
bool record_dirty_log)
{
- tdp_ptep_t root_pt = tdp_iter_root_pt(iter);
- struct kvm_mmu_page *root = sptep_to_sp(root_pt);
- int as_id = kvm_mmu_page_as_id(root);
-
lockdep_assert_held_write(&kvm->mmu_lock);
/*
WRITE_ONCE(*rcu_dereference(iter->sptep), new_spte);
- __handle_changed_spte(kvm, as_id, iter->gfn, iter->old_spte, new_spte,
- iter->level, false);
+ __handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte,
+ new_spte, iter->level, false);
if (record_acc_track)
handle_changed_spte_acc_track(iter->old_spte, new_spte,
iter->level);
if (record_dirty_log)
- handle_changed_spte_dirty_log(kvm, as_id, iter->gfn,
+ handle_changed_spte_dirty_log(kvm, iter->as_id, iter->gfn,
iter->old_spte, new_spte,
iter->level);
}
WARN_ON(iter->gfn > iter->next_last_level_gfn);
- tdp_iter_start(iter, iter->pt_path[iter->root_level - 1],
- iter->root_level, iter->min_level,
- iter->next_last_level_gfn);
+ tdp_iter_restart(iter);
return true;
}
* scheduler needs the CPU or there is contention on the MMU lock. If this
* function cannot yield, it will not release the MMU lock or reschedule and
* the caller must ensure it does not supply too large a GFN range, or the
- * operation can cause a soft lockup.
+ * operation can cause a soft lockup. Note, in some use cases a flush may be
+ * required by prior actions. Ensure the pending flush is performed prior to
+ * yielding.
*/
static bool zap_gfn_range(struct kvm *kvm, struct kvm_mmu_page *root,
- gfn_t start, gfn_t end, bool can_yield)
+ gfn_t start, gfn_t end, bool can_yield, bool flush)
{
struct tdp_iter iter;
- bool flush_needed = false;
rcu_read_lock();
tdp_root_for_each_pte(iter, root, start, end) {
if (can_yield &&
- tdp_mmu_iter_cond_resched(kvm, &iter, flush_needed)) {
- flush_needed = false;
+ tdp_mmu_iter_cond_resched(kvm, &iter, flush)) {
+ flush = false;
continue;
}
continue;
tdp_mmu_set_spte(kvm, &iter, 0);
- flush_needed = true;
+ flush = true;
}
rcu_read_unlock();
- return flush_needed;
+ return flush;
}
/*
* SPTEs have been cleared and a TLB flush is needed before releasing the
* MMU lock.
*/
-bool kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, gfn_t start, gfn_t end)
+bool __kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, gfn_t start, gfn_t end,
+ bool can_yield)
{
struct kvm_mmu_page *root;
bool flush = false;
for_each_tdp_mmu_root_yield_safe(kvm, root)
- flush |= zap_gfn_range(kvm, root, start, end, true);
+ flush = zap_gfn_range(kvm, root, start, end, can_yield, flush);
return flush;
}
struct kvm_mmu_page *root, gfn_t start,
gfn_t end, unsigned long unused)
{
- return zap_gfn_range(kvm, root, start, end, false);
+ return zap_gfn_range(kvm, root, start, end, false, false);
}
int kvm_tdp_mmu_zap_hva_range(struct kvm *kvm, unsigned long start,
hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu);
void kvm_tdp_mmu_free_root(struct kvm *kvm, struct kvm_mmu_page *root);
-bool kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, gfn_t start, gfn_t end);
+bool __kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, gfn_t start, gfn_t end,
+ bool can_yield);
+static inline bool kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, gfn_t start,
+ gfn_t end)
+{
+ return __kvm_tdp_mmu_zap_gfn_range(kvm, start, end, true);
+}
+static inline bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp)
+{
+ gfn_t end = sp->gfn + KVM_PAGES_PER_HPAGE(sp->role.level);
+
+ /*
+ * Don't allow yielding, as the caller may have a flush pending. Note,
+ * if mmu_lock is held for write, zapping will never yield in this case,
+ * but explicitly disallow it for safety. The TDP MMU does not yield
+ * until it has made forward progress (steps sideways), and when zapping
+ * a single shadow page that it's guaranteed to see (thus the mmu_lock
+ * requirement), its "step sideways" will always step beyond the bounds
+ * of the shadow page's gfn range and stop iterating before yielding.
+ */
+ lockdep_assert_held_write(&kvm->mmu_lock);
+ return __kvm_tdp_mmu_zap_gfn_range(kvm, sp->gfn, end, false);
+}
void kvm_tdp_mmu_zap_all(struct kvm *kvm);
int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
return true;
}
-static bool nested_vmcb_checks(struct vcpu_svm *svm, struct vmcb *vmcb12)
+static bool nested_vmcb_check_save(struct vcpu_svm *svm, struct vmcb *vmcb12)
{
struct kvm_vcpu *vcpu = &svm->vcpu;
bool vmcb12_lma;
+ /*
+ * FIXME: these should be done after copying the fields,
+ * to avoid TOC/TOU races. For these save area checks
+ * the possible damage is limited since kvm_set_cr0 and
+ * kvm_set_cr4 handle failure; EFER_SVME is an exception
+ * so it is force-set later in nested_prepare_vmcb_save.
+ */
if ((vmcb12->save.efer & EFER_SVME) == 0)
return false;
if (!kvm_is_valid_cr4(&svm->vcpu, vmcb12->save.cr4))
return false;
- return nested_vmcb_check_controls(&vmcb12->control);
+ return true;
}
static void load_nested_vmcb_control(struct vcpu_svm *svm,
svm->vmcb->save.gdtr = vmcb12->save.gdtr;
svm->vmcb->save.idtr = vmcb12->save.idtr;
kvm_set_rflags(&svm->vcpu, vmcb12->save.rflags | X86_EFLAGS_FIXED);
- svm_set_efer(&svm->vcpu, vmcb12->save.efer);
+
+ /*
+ * Force-set EFER_SVME even though it is checked earlier on the
+ * VMCB12, because the guest can flip the bit between the check
+ * and now. Clearing EFER_SVME would call svm_free_nested.
+ */
+ svm_set_efer(&svm->vcpu, vmcb12->save.efer | EFER_SVME);
+
svm_set_cr0(&svm->vcpu, vmcb12->save.cr0);
svm_set_cr4(&svm->vcpu, vmcb12->save.cr4);
svm->vmcb->save.cr2 = svm->vcpu.arch.cr2 = vmcb12->save.cr2;
svm->nested.vmcb12_gpa = vmcb12_gpa;
- load_nested_vmcb_control(svm, &vmcb12->control);
nested_prepare_vmcb_control(svm);
nested_prepare_vmcb_save(svm, vmcb12);
if (WARN_ON_ONCE(!svm->nested.initialized))
return -EINVAL;
- if (!nested_vmcb_checks(svm, vmcb12)) {
+ load_nested_vmcb_control(svm, &vmcb12->control);
+
+ if (!nested_vmcb_check_save(svm, vmcb12) ||
+ !nested_vmcb_check_controls(&svm->nested.ctl)) {
vmcb12->control.exit_code = SVM_EXIT_ERR;
vmcb12->control.exit_code_hi = 0;
vmcb12->control.exit_info_1 = 0;
*/
if (!(save->cr0 & X86_CR0_PG))
goto out_free;
+ if (!(save->efer & EFER_SVME))
+ goto out_free;
/*
* All checks done, we can enter guest mode. L1 control fields
static inline struct kvm_pmc *get_gp_pmc_amd(struct kvm_pmu *pmu, u32 msr,
enum pmu_type type)
{
+ struct kvm_vcpu *vcpu = pmu_to_vcpu(pmu);
+
switch (msr) {
case MSR_F15H_PERF_CTL0:
case MSR_F15H_PERF_CTL1:
case MSR_F15H_PERF_CTL3:
case MSR_F15H_PERF_CTL4:
case MSR_F15H_PERF_CTL5:
+ if (!guest_cpuid_has(vcpu, X86_FEATURE_PERFCTR_CORE))
+ return NULL;
+ fallthrough;
case MSR_K7_EVNTSEL0 ... MSR_K7_EVNTSEL3:
if (type != PMU_TYPE_EVNTSEL)
return NULL;
case MSR_F15H_PERF_CTR3:
case MSR_F15H_PERF_CTR4:
case MSR_F15H_PERF_CTR5:
+ if (!guest_cpuid_has(vcpu, X86_FEATURE_PERFCTR_CORE))
+ return NULL;
+ fallthrough;
case MSR_K7_PERFCTR0 ... MSR_K7_PERFCTR3:
if (type != PMU_TYPE_COUNTER)
return NULL;
* When called, it means the previous get/set msr reached an invalid msr.
* Return true if we want to ignore/silent this failed msr access.
*/
-static bool kvm_msr_ignored_check(struct kvm_vcpu *vcpu, u32 msr,
- u64 data, bool write)
+static bool kvm_msr_ignored_check(u32 msr, u64 data, bool write)
{
const char *op = write ? "wrmsr" : "rdmsr";
if (r == KVM_MSR_RET_INVALID) {
/* Unconditionally clear the output for simplicity */
*data = 0;
- if (kvm_msr_ignored_check(vcpu, index, 0, false))
+ if (kvm_msr_ignored_check(index, 0, false))
r = 0;
}
bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type)
{
+ struct kvm_x86_msr_filter *msr_filter;
+ struct msr_bitmap_range *ranges;
struct kvm *kvm = vcpu->kvm;
- struct msr_bitmap_range *ranges = kvm->arch.msr_filter.ranges;
- u32 count = kvm->arch.msr_filter.count;
- u32 i;
- bool r = kvm->arch.msr_filter.default_allow;
+ bool allowed;
int idx;
+ u32 i;
- /* MSR filtering not set up or x2APIC enabled, allow everything */
- if (!count || (index >= 0x800 && index <= 0x8ff))
+ /* x2APIC MSRs do not support filtering. */
+ if (index >= 0x800 && index <= 0x8ff)
return true;
- /* Prevent collision with set_msr_filter */
idx = srcu_read_lock(&kvm->srcu);
- for (i = 0; i < count; i++) {
+ msr_filter = srcu_dereference(kvm->arch.msr_filter, &kvm->srcu);
+ if (!msr_filter) {
+ allowed = true;
+ goto out;
+ }
+
+ allowed = msr_filter->default_allow;
+ ranges = msr_filter->ranges;
+
+ for (i = 0; i < msr_filter->count; i++) {
u32 start = ranges[i].base;
u32 end = start + ranges[i].nmsrs;
u32 flags = ranges[i].flags;
unsigned long *bitmap = ranges[i].bitmap;
if ((index >= start) && (index < end) && (flags & type)) {
- r = !!test_bit(index - start, bitmap);
+ allowed = !!test_bit(index - start, bitmap);
break;
}
}
+out:
srcu_read_unlock(&kvm->srcu, idx);
- return r;
+ return allowed;
}
EXPORT_SYMBOL_GPL(kvm_msr_allowed);
int ret = __kvm_set_msr(vcpu, index, data, host_initiated);
if (ret == KVM_MSR_RET_INVALID)
- if (kvm_msr_ignored_check(vcpu, index, data, true))
+ if (kvm_msr_ignored_check(index, data, true))
ret = 0;
return ret;
if (ret == KVM_MSR_RET_INVALID) {
/* Unconditionally clear *data for simplicity */
*data = 0;
- if (kvm_msr_ignored_check(vcpu, index, 0, false))
+ if (kvm_msr_ignored_check(index, 0, false))
ret = 0;
}
kvm_vcpu_write_tsc_offset(vcpu, offset);
raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
- spin_lock(&kvm->arch.pvclock_gtod_sync_lock);
+ spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);
if (!matched) {
kvm->arch.nr_vcpus_matched_tsc = 0;
} else if (!already_matched) {
}
kvm_track_tsc_matching(vcpu);
- spin_unlock(&kvm->arch.pvclock_gtod_sync_lock);
+ spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags);
}
static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu,
int i;
struct kvm_vcpu *vcpu;
struct kvm_arch *ka = &kvm->arch;
+ unsigned long flags;
+
+ kvm_hv_invalidate_tsc_page(kvm);
- spin_lock(&ka->pvclock_gtod_sync_lock);
kvm_make_mclock_inprogress_request(kvm);
+
/* no guest entries from this point */
+ spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
pvclock_update_vm_gtod_copy(kvm);
+ spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
kvm_for_each_vcpu(i, vcpu, kvm)
kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
/* guest entries allowed */
kvm_for_each_vcpu(i, vcpu, kvm)
kvm_clear_request(KVM_REQ_MCLOCK_INPROGRESS, vcpu);
-
- spin_unlock(&ka->pvclock_gtod_sync_lock);
#endif
}
{
struct kvm_arch *ka = &kvm->arch;
struct pvclock_vcpu_time_info hv_clock;
+ unsigned long flags;
u64 ret;
- spin_lock(&ka->pvclock_gtod_sync_lock);
+ spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
if (!ka->use_master_clock) {
- spin_unlock(&ka->pvclock_gtod_sync_lock);
+ spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
return get_kvmclock_base_ns() + ka->kvmclock_offset;
}
hv_clock.tsc_timestamp = ka->master_cycle_now;
hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
- spin_unlock(&ka->pvclock_gtod_sync_lock);
+ spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
/* both __this_cpu_read() and rdtsc() should be on the same cpu */
get_cpu();
* If the host uses TSC clock, then passthrough TSC as stable
* to the guest.
*/
- spin_lock(&ka->pvclock_gtod_sync_lock);
+ spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
use_master_clock = ka->use_master_clock;
if (use_master_clock) {
host_tsc = ka->master_cycle_now;
kernel_ns = ka->master_kernel_ns;
}
- spin_unlock(&ka->pvclock_gtod_sync_lock);
+ spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
/* Keep irq disabled to prevent changes to the clock */
local_irq_save(flags);
return r;
}
-static void kvm_clear_msr_filter(struct kvm *kvm)
+static struct kvm_x86_msr_filter *kvm_alloc_msr_filter(bool default_allow)
+{
+ struct kvm_x86_msr_filter *msr_filter;
+
+ msr_filter = kzalloc(sizeof(*msr_filter), GFP_KERNEL_ACCOUNT);
+ if (!msr_filter)
+ return NULL;
+
+ msr_filter->default_allow = default_allow;
+ return msr_filter;
+}
+
+static void kvm_free_msr_filter(struct kvm_x86_msr_filter *msr_filter)
{
u32 i;
- u32 count = kvm->arch.msr_filter.count;
- struct msr_bitmap_range ranges[16];
- mutex_lock(&kvm->lock);
- kvm->arch.msr_filter.count = 0;
- memcpy(ranges, kvm->arch.msr_filter.ranges, count * sizeof(ranges[0]));
- mutex_unlock(&kvm->lock);
- synchronize_srcu(&kvm->srcu);
+ if (!msr_filter)
+ return;
- for (i = 0; i < count; i++)
- kfree(ranges[i].bitmap);
+ for (i = 0; i < msr_filter->count; i++)
+ kfree(msr_filter->ranges[i].bitmap);
+
+ kfree(msr_filter);
}
-static int kvm_add_msr_filter(struct kvm *kvm, struct kvm_msr_filter_range *user_range)
+static int kvm_add_msr_filter(struct kvm_x86_msr_filter *msr_filter,
+ struct kvm_msr_filter_range *user_range)
{
- struct msr_bitmap_range *ranges = kvm->arch.msr_filter.ranges;
struct msr_bitmap_range range;
unsigned long *bitmap = NULL;
size_t bitmap_size;
goto err;
}
- /* Everything ok, add this range identifier to our global pool */
- ranges[kvm->arch.msr_filter.count] = range;
- /* Make sure we filled the array before we tell anyone to walk it */
- smp_wmb();
- kvm->arch.msr_filter.count++;
+ /* Everything ok, add this range identifier. */
+ msr_filter->ranges[msr_filter->count] = range;
+ msr_filter->count++;
return 0;
err:
static int kvm_vm_ioctl_set_msr_filter(struct kvm *kvm, void __user *argp)
{
struct kvm_msr_filter __user *user_msr_filter = argp;
+ struct kvm_x86_msr_filter *new_filter, *old_filter;
struct kvm_msr_filter filter;
bool default_allow;
- int r = 0;
bool empty = true;
+ int r = 0;
u32 i;
if (copy_from_user(&filter, user_msr_filter, sizeof(filter)))
if (empty && !default_allow)
return -EINVAL;
- kvm_clear_msr_filter(kvm);
-
- kvm->arch.msr_filter.default_allow = default_allow;
+ new_filter = kvm_alloc_msr_filter(default_allow);
+ if (!new_filter)
+ return -ENOMEM;
- /*
- * Protect from concurrent calls to this function that could trigger
- * a TOCTOU violation on kvm->arch.msr_filter.count.
- */
- mutex_lock(&kvm->lock);
for (i = 0; i < ARRAY_SIZE(filter.ranges); i++) {
- r = kvm_add_msr_filter(kvm, &filter.ranges[i]);
- if (r)
- break;
+ r = kvm_add_msr_filter(new_filter, &filter.ranges[i]);
+ if (r) {
+ kvm_free_msr_filter(new_filter);
+ return r;
+ }
}
+ mutex_lock(&kvm->lock);
+
+ /* The per-VM filter is protected by kvm->lock... */
+ old_filter = srcu_dereference_check(kvm->arch.msr_filter, &kvm->srcu, 1);
+
+ rcu_assign_pointer(kvm->arch.msr_filter, new_filter);
+ synchronize_srcu(&kvm->srcu);
+
+ kvm_free_msr_filter(old_filter);
+
kvm_make_all_cpus_request(kvm, KVM_REQ_MSR_FILTER_CHANGED);
mutex_unlock(&kvm->lock);
- return r;
+ return 0;
}
long kvm_arch_vm_ioctl(struct file *filp,
}
#endif
case KVM_SET_CLOCK: {
+ struct kvm_arch *ka = &kvm->arch;
struct kvm_clock_data user_ns;
u64 now_ns;
* pvclock_update_vm_gtod_copy().
*/
kvm_gen_update_masterclock(kvm);
- now_ns = get_kvmclock_ns(kvm);
- kvm->arch.kvmclock_offset += user_ns.clock - now_ns;
+
+ /*
+ * This pairs with kvm_guest_time_update(): when masterclock is
+ * in use, we use master_kernel_ns + kvmclock_offset to set
+ * unsigned 'system_time' so if we use get_kvmclock_ns() (which
+ * is slightly ahead) here we risk going negative on unsigned
+ * 'system_time' when 'user_ns.clock' is very small.
+ */
+ spin_lock_irq(&ka->pvclock_gtod_sync_lock);
+ if (kvm->arch.use_master_clock)
+ now_ns = ka->master_kernel_ns;
+ else
+ now_ns = get_kvmclock_base_ns();
+ ka->kvmclock_offset = user_ns.clock - now_ns;
+ spin_unlock_irq(&ka->pvclock_gtod_sync_lock);
+
kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE);
break;
}
int cpu = get_cpu();
cpumask_set_cpu(cpu, vcpu->arch.wbinvd_dirty_mask);
- smp_call_function_many(vcpu->arch.wbinvd_dirty_mask,
+ on_each_cpu_mask(vcpu->arch.wbinvd_dirty_mask,
wbinvd_ipi, NULL, 1);
put_cpu();
cpumask_clear(vcpu->arch.wbinvd_dirty_mask);
struct kvm *kvm;
struct kvm_vcpu *vcpu;
int cpu;
+ unsigned long flags;
mutex_lock(&kvm_lock);
list_for_each_entry(kvm, &vm_list, vm_list)
list_for_each_entry(kvm, &vm_list, vm_list) {
struct kvm_arch *ka = &kvm->arch;
- spin_lock(&ka->pvclock_gtod_sync_lock);
-
+ spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags);
pvclock_update_vm_gtod_copy(kvm);
+ spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags);
kvm_for_each_vcpu(cpu, vcpu, kvm)
kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
kvm_for_each_vcpu(cpu, vcpu, kvm)
kvm_clear_request(KVM_REQ_MCLOCK_INPROGRESS, vcpu);
-
- spin_unlock(&ka->pvclock_gtod_sync_lock);
}
mutex_unlock(&kvm_lock);
}
void kvm_arch_destroy_vm(struct kvm *kvm)
{
- u32 i;
-
if (current->mm == kvm->mm) {
/*
* Free memory regions allocated on behalf of userspace,
mutex_unlock(&kvm->slots_lock);
}
static_call_cond(kvm_x86_vm_destroy)(kvm);
- for (i = 0; i < kvm->arch.msr_filter.count; i++)
- kfree(kvm->arch.msr_filter.ranges[i].bitmap);
+ kvm_free_msr_filter(srcu_dereference_check(kvm->arch.msr_filter, &kvm->srcu, 1));
kvm_pic_destroy(kvm);
kvm_ioapic_destroy(kvm);
kvm_free_vcpus(kvm);
void kvm_write_wall_clock(struct kvm *kvm, gpa_t wall_clock, int sec_hi_ofs);
void kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip);
-void kvm_write_tsc(struct kvm_vcpu *vcpu, struct msr_data *msr);
u64 get_kvmclock_ns(struct kvm *kvm);
int kvm_read_guest_virt(struct kvm_vcpu *vcpu,
if (pgprot_val(old_prot) == pgprot_val(new_prot))
return;
- pa = pfn << page_level_shift(level);
+ pa = pfn << PAGE_SHIFT;
size = page_level_size(level);
/*
}
if (image) {
- if (unlikely(proglen + ilen > oldproglen)) {
+ /*
+ * When populating the image, assert that:
+ *
+ * i) We do not write beyond the allocated space, and
+ * ii) addrs[i] did not change from the prior run, in order
+ * to validate assumptions made for computing branch
+ * displacements.
+ */
+ if (unlikely(proglen + ilen > oldproglen ||
+ proglen + ilen != addrs[i])) {
pr_err("bpf_jit: fatal error\n");
return -EFAULT;
}
* add rsp, 8 // skip eth_type_trans's frame
* ret // return to its caller
*/
-int arch_prepare_bpf_trampoline(void *image, void *image_end,
+int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *image_end,
const struct btf_func_model *m, u32 flags,
struct bpf_tramp_progs *tprogs,
void *orig_call)
save_regs(m, &prog, nr_args, stack_size);
+ if (flags & BPF_TRAMP_F_CALL_ORIG) {
+ /* arg1: mov rdi, im */
+ emit_mov_imm64(&prog, BPF_REG_1, (long) im >> 32, (u32) (long) im);
+ if (emit_call(&prog, __bpf_tramp_enter, prog)) {
+ ret = -EINVAL;
+ goto cleanup;
+ }
+ }
+
if (fentry->nr_progs)
if (invoke_bpf(m, &prog, fentry, stack_size))
return -EINVAL;
}
if (flags & BPF_TRAMP_F_CALL_ORIG) {
- if (fentry->nr_progs || fmod_ret->nr_progs)
- restore_regs(m, &prog, nr_args, stack_size);
+ restore_regs(m, &prog, nr_args, stack_size);
/* call original function */
if (emit_call(&prog, orig_call, prog)) {
}
/* remember return value in a stack for bpf prog to access */
emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -8);
+ im->ip_after_call = prog;
+ memcpy(prog, ideal_nops[NOP_ATOMIC5], X86_PATCH_SIZE);
+ prog += X86_PATCH_SIZE;
}
if (fmod_ret->nr_progs) {
* the return value is only updated on the stack and still needs to be
* restored to R0.
*/
- if (flags & BPF_TRAMP_F_CALL_ORIG)
+ if (flags & BPF_TRAMP_F_CALL_ORIG) {
+ im->ip_epilogue = prog;
+ /* arg1: mov rdi, im */
+ emit_mov_imm64(&prog, BPF_REG_1, (long) im >> 32, (u32) (long) im);
+ if (emit_call(&prog, __bpf_tramp_exit, prog)) {
+ ret = -EINVAL;
+ goto cleanup;
+ }
/* restore original return value back into RAX */
emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, -8);
+ }
EMIT1(0x5B); /* pop rbx */
EMIT1(0xC9); /* leave */
padding = true;
goto skip_init_addrs;
}
- addrs = kmalloc_array(prog->len + 1, sizeof(*addrs), GFP_KERNEL);
+ addrs = kvmalloc_array(prog->len + 1, sizeof(*addrs), GFP_KERNEL);
if (!addrs) {
prog = orig_prog;
goto out_addrs;
if (image)
bpf_prog_fill_jited_linfo(prog, addrs + 1);
out_addrs:
- kfree(addrs);
+ kvfree(addrs);
kfree(jit_data);
prog->aux->jit_data = NULL;
}
}
if (image) {
- if (unlikely(proglen + ilen > oldproglen)) {
+ /*
+ * When populating the image, assert that:
+ *
+ * i) We do not write beyond the allocated space, and
+ * ii) addrs[i] did not change from the prior run, in order
+ * to validate assumptions made for computing branch
+ * displacements.
+ */
+ if (unlikely(proglen + ilen > oldproglen ||
+ proglen + ilen != addrs[i])) {
pr_err("bpf_jit: fatal error\n");
return -EFAULT;
}
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Sébastien Hinderer <Sebastien.Hinderer@ens-lyon.org>");
MODULE_DESCRIPTION("A power_off handler for Iris devices from EuroBraille");
-MODULE_SUPPORTED_DEVICE("Eurobraille/Iris");
static bool force;
unsigned long xen_max_p2m_pfn __read_mostly;
EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
-#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
-#define P2M_LIMIT CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
+#ifdef CONFIG_XEN_MEMORY_HOTPLUG_LIMIT
+#define P2M_LIMIT CONFIG_XEN_MEMORY_HOTPLUG_LIMIT
#else
#define P2M_LIMIT 0
#endif
xen_p2m_last_pfn = xen_max_p2m_pfn;
p2m_limit = (phys_addr_t)P2M_LIMIT * 1024 * 1024 * 1024 / PAGE_SIZE;
- if (!p2m_limit && IS_ENABLED(CONFIG_XEN_UNPOPULATED_ALLOC))
- p2m_limit = xen_start_info->nr_pages * XEN_EXTRA_MEM_RATIO;
-
vm.flags = VM_ALLOC;
vm.size = ALIGN(sizeof(unsigned long) * max(xen_max_p2m_pfn, p2m_limit),
PMD_SIZE * PMDS_PER_MID_PAGE);
} xen_remap_buf __initdata __aligned(PAGE_SIZE);
static unsigned long xen_remap_mfn __initdata = INVALID_P2M_ENTRY;
+/*
+ * The maximum amount of extra memory compared to the base size. The
+ * main scaling factor is the size of struct page. At extreme ratios
+ * of base:extra, all the base memory can be filled with page
+ * structures for the extra memory, leaving no space for anything
+ * else.
+ *
+ * 10x seems like a reasonable balance between scaling flexibility and
+ * leaving a practically usable system.
+ */
+#define EXTRA_MEM_RATIO (10)
+
static bool xen_512gb_limit __initdata = IS_ENABLED(CONFIG_XEN_512GB);
static void __init xen_parse_512gb(void)
extra_pages += max_pages - max_pfn;
/*
- * Clamp the amount of extra memory to a XEN_EXTRA_MEM_RATIO
+ * Clamp the amount of extra memory to a EXTRA_MEM_RATIO
* factor the base size.
*
* Make sure we have no memory above max_pages, as this area
* isn't handled by the p2m management.
*/
- extra_pages = min3(XEN_EXTRA_MEM_RATIO * min(max_pfn, PFN_DOWN(MAXMEM)),
+ extra_pages = min3(EXTRA_MEM_RATIO * min(max_pfn, PFN_DOWN(MAXMEM)),
extra_pages, max_pages - max_pfn);
i = 0;
addr = xen_e820_table.entries[0].addr;
LOAD_CP_REGS_TAB(6)
LOAD_CP_REGS_TAB(7)
-/*
- * coprocessor_flush(struct thread_info*, index)
- * a2 a3
- *
- * Save coprocessor registers for coprocessor 'index'.
- * The register values are saved to or loaded from the coprocessor area
- * inside the task_info structure.
- *
- * Note that this function doesn't update the coprocessor_owner information!
- *
- */
-
-ENTRY(coprocessor_flush)
-
- /* reserve 4 bytes on stack to save a0 */
- abi_entry(4)
-
- s32i a0, a1, 0
- movi a0, .Lsave_cp_regs_jump_table
- addx8 a3, a3, a0
- l32i a4, a3, 4
- l32i a3, a3, 0
- add a2, a2, a4
- beqz a3, 1f
- callx0 a3
-1: l32i a0, a1, 0
-
- abi_ret(4)
-
-ENDPROC(coprocessor_flush)
-
/*
* Entry condition:
*
ENDPROC(fast_coprocessor)
+ .text
+
+/*
+ * coprocessor_flush(struct thread_info*, index)
+ * a2 a3
+ *
+ * Save coprocessor registers for coprocessor 'index'.
+ * The register values are saved to or loaded from the coprocessor area
+ * inside the task_info structure.
+ *
+ * Note that this function doesn't update the coprocessor_owner information!
+ *
+ */
+
+ENTRY(coprocessor_flush)
+
+ /* reserve 4 bytes on stack to save a0 */
+ abi_entry(4)
+
+ s32i a0, a1, 0
+ movi a0, .Lsave_cp_regs_jump_table
+ addx8 a3, a3, a0
+ l32i a4, a3, 4
+ l32i a3, a3, 0
+ add a2, a2, a4
+ beqz a3, 1f
+ callx0 a3
+1: l32i a0, a1, 0
+
+ abi_ret(4)
+
+ENDPROC(coprocessor_flush)
+
.data
ENTRY(coprocessor_owner)
*/
fault = handle_mm_fault(vma, address, flags, regs);
- if (fault_signal_pending(fault, regs))
+ if (fault_signal_pending(fault, regs)) {
+ if (!user_mode(regs))
+ goto bad_page_fault;
return;
+ }
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
{
struct bio *parent = bio->bi_private;
- if (!parent->bi_status)
+ if (bio->bi_status && !parent->bi_status)
parent->bi_status = bio->bi_status;
bio_put(bio);
return parent;
}
EXPORT_SYMBOL_GPL(bio_release_pages);
-static int bio_iov_bvec_set(struct bio *bio, struct iov_iter *iter)
+static void __bio_iov_bvec_set(struct bio *bio, struct iov_iter *iter)
{
WARN_ON_ONCE(bio->bi_max_vecs);
bio->bi_iter.bi_size = iter->count;
bio_set_flag(bio, BIO_NO_PAGE_REF);
bio_set_flag(bio, BIO_CLONED);
+}
+static int bio_iov_bvec_set(struct bio *bio, struct iov_iter *iter)
+{
+ __bio_iov_bvec_set(bio, iter);
iov_iter_advance(iter, iter->count);
return 0;
}
+static int bio_iov_bvec_set_append(struct bio *bio, struct iov_iter *iter)
+{
+ struct request_queue *q = bio->bi_bdev->bd_disk->queue;
+ struct iov_iter i = *iter;
+
+ iov_iter_truncate(&i, queue_max_zone_append_sectors(q) << 9);
+ __bio_iov_bvec_set(bio, &i);
+ iov_iter_advance(iter, i.count);
+ return 0;
+}
+
#define PAGE_PTRS_PER_BVEC (sizeof(struct bio_vec) / sizeof(struct page *))
/**
int ret = 0;
if (iov_iter_is_bvec(iter)) {
- if (WARN_ON_ONCE(bio_op(bio) == REQ_OP_ZONE_APPEND))
- return -EINVAL;
+ if (bio_op(bio) == REQ_OP_ZONE_APPEND)
+ return bio_iov_bvec_set_append(bio, iter);
return bio_iov_bvec_set(bio, iter);
}
switch (bio_op(rq->bio)) {
case REQ_OP_DISCARD:
case REQ_OP_SECURE_ERASE:
+ if (queue_max_discard_segments(rq->q) > 1) {
+ struct bio *bio = rq->bio;
+
+ for_each_bio(bio)
+ nr_phys_segs++;
+ return nr_phys_segs;
+ }
+ return 1;
case REQ_OP_WRITE_ZEROES:
return 0;
case REQ_OP_WRITE_SAME:
RQF_NAME(QUIET),
RQF_NAME(ELVPRIV),
RQF_NAME(IO_STAT),
- RQF_NAME(ALLOCED),
RQF_NAME(PM),
RQF_NAME(HASHED),
RQF_NAME(STATS),
const char *dname;
int err;
+ /*
+ * disk_max_parts() won't be zero, either GENHD_FL_EXT_DEVT is set
+ * or 'minors' is passed to alloc_disk().
+ */
+ if (partno >= disk_max_parts(disk))
+ return ERR_PTR(-EINVAL);
+
/*
* Partitions are not supported on zoned block devices that are used as
* such.
* just create and link the new node(s) here.
*/
new_node =
- ACPI_ALLOCATE_ZEROED(sizeof(struct acpi_namespace_node));
+ acpi_ns_create_node(*ACPI_CAST_PTR(u32, init_val->name));
if (!new_node) {
status = AE_NO_MEMORY;
goto unlock_and_exit;
}
- ACPI_COPY_NAMESEG(new_node->name.ascii, init_val->name);
new_node->descriptor_type = ACPI_DESC_TYPE_NAMED;
new_node->type = init_val->type;
#ifndef _ACPI_INTERNAL_H_
#define _ACPI_INTERNAL_H_
+#include <linux/idr.h>
+
#define PREFIX "ACPI: "
int early_acpi_osi_init(void);
extern struct list_head acpi_bus_id_list;
+#define ACPI_MAX_DEVICE_INSTANCES 4096
+
struct acpi_device_bus_id {
const char *bus_id;
- unsigned int instance_no;
+ struct ida instance_ida;
struct list_head node;
};
*/
#ifdef CONFIG_X86
#include <asm/apic.h>
+#include <asm/cpu.h>
#endif
#define _COMPONENT ACPI_PROCESSOR_COMPONENT
wait_for_freeze();
} else
return -ENODEV;
+
+#if defined(CONFIG_X86) && defined(CONFIG_HOTPLUG_CPU)
+ cond_wakeup_cpu0();
+#endif
}
/* Never reached */
list_for_each_entry(acpi_device_bus_id, &acpi_bus_id_list, node)
if (!strcmp(acpi_device_bus_id->bus_id,
acpi_device_hid(device))) {
- if (acpi_device_bus_id->instance_no > 0)
- acpi_device_bus_id->instance_no--;
- else {
+ ida_simple_remove(&acpi_device_bus_id->instance_ida, device->pnp.instance_no);
+ if (ida_is_empty(&acpi_device_bus_id->instance_ida)) {
list_del(&acpi_device_bus_id->node);
kfree_const(acpi_device_bus_id->bus_id);
kfree(acpi_device_bus_id);
return NULL;
}
+static int acpi_device_set_name(struct acpi_device *device,
+ struct acpi_device_bus_id *acpi_device_bus_id)
+{
+ struct ida *instance_ida = &acpi_device_bus_id->instance_ida;
+ int result;
+
+ result = ida_simple_get(instance_ida, 0, ACPI_MAX_DEVICE_INSTANCES, GFP_KERNEL);
+ if (result < 0)
+ return result;
+
+ device->pnp.instance_no = result;
+ dev_set_name(&device->dev, "%s:%02x", acpi_device_bus_id->bus_id, result);
+ return 0;
+}
+
int acpi_device_add(struct acpi_device *device,
void (*release)(struct device *))
{
acpi_device_bus_id = acpi_device_bus_id_match(acpi_device_hid(device));
if (acpi_device_bus_id) {
- acpi_device_bus_id->instance_no++;
+ result = acpi_device_set_name(device, acpi_device_bus_id);
+ if (result)
+ goto err_unlock;
} else {
acpi_device_bus_id = kzalloc(sizeof(*acpi_device_bus_id),
GFP_KERNEL);
goto err_unlock;
}
+ ida_init(&acpi_device_bus_id->instance_ida);
+
+ result = acpi_device_set_name(device, acpi_device_bus_id);
+ if (result) {
+ kfree(acpi_device_bus_id);
+ goto err_unlock;
+ }
+
list_add_tail(&acpi_device_bus_id->node, &acpi_bus_id_list);
}
- dev_set_name(&device->dev, "%s:%02x", acpi_device_bus_id->bus_id, acpi_device_bus_id->instance_no);
if (device->parent)
list_add_tail(&device->node, &device->parent->children);
device_initialize(&device->dev);
dev_set_uevent_suppress(&device->dev, true);
acpi_init_coherency(device);
+ /* Assume there are unmet deps to start with. */
+ device->dep_unmet = 1;
}
void acpi_device_add_finalize(struct acpi_device *device)
{
struct acpi_dep_data *dep;
+ adev->dep_unmet = 0;
+
mutex_lock(&acpi_dep_list_lock);
list_for_each_entry(dep, &acpi_dep_list, node) {
return AE_CTRL_DEPTH;
acpi_scan_init_hotplug(device);
- if (!check_dep)
+ /*
+ * If check_dep is true at this point, the device has no dependencies,
+ * or the creation of the device object would have been postponed above.
+ */
+ if (check_dep)
+ device->dep_unmet = 0;
+ else
acpi_scan_dep_init(device);
out:
}
/*
- * acpi_table_init()
+ * acpi_locate_initial_tables()
*
* find RSDP, find and checksum SDT/XSDT.
* checksum all tables, print SDT/XSDT
* result: sdt_entry[] is initialized
*/
-int __init acpi_table_init(void)
+int __init acpi_locate_initial_tables(void)
{
acpi_status status;
status = acpi_initialize_tables(initial_tables, ACPI_MAX_TABLES, 0);
if (ACPI_FAILURE(status))
return -EINVAL;
- acpi_table_initrd_scan();
+ return 0;
+}
+
+void __init acpi_reserve_initial_tables(void)
+{
+ int i;
+
+ for (i = 0; i < ACPI_MAX_TABLES; i++) {
+ struct acpi_table_desc *table_desc = &initial_tables[i];
+ u64 start = table_desc->address;
+ u64 size = table_desc->length;
+
+ if (!start || !size)
+ break;
+
+ pr_info("Reserving %4s table memory at [mem 0x%llx-0x%llx]\n",
+ table_desc->signature.ascii, start, start + size - 1);
+
+ memblock_reserve(start, size);
+ }
+}
+
+void __init acpi_table_init_complete(void)
+{
+ acpi_table_initrd_scan();
check_multiple_madt();
+}
+
+int __init acpi_table_init(void)
+{
+ int ret;
+
+ ret = acpi_locate_initial_tables();
+ if (ret)
+ return ret;
+
+ acpi_table_init_complete();
+
return 0;
}
},
},
{
+ .callback = video_detect_force_vendor,
.ident = "Sony VPCEH3U1E",
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Sony Corporation"),
MODULE_AUTHOR("Christophe Lizzi - credits to Uwe Dannowski and Heikki Vatiainen");
MODULE_DESCRIPTION("FORE Systems 200E-series ATM driver - version " FORE200E_VERSION);
-MODULE_SUPPORTED_DEVICE("PCA-200E, SBA-200E");
-
static const int fore200e_rx_buf_nbr[ BUFFER_SCHEME_NBR ][ BUFFER_MAGN_NBR ] = {
{ BUFFER_S1_NBR, BUFFER_L1_NBR },
char c;
for (; count-- > 0; (*ppos)++, tmp++) {
- if (!in_interrupt() && (((count + 1) & 0x1f) == 0))
+ if (((count + 1) & 0x1f) == 0) {
/*
- * let's be a little nice with other processes
- * that need some CPU
+ * charlcd_write() is invoked as a VFS->write() callback
+ * and as such it is always invoked from preemptible
+ * context and may sleep.
*/
- schedule();
+ cond_resched();
+ }
if (get_user(c, tmp))
return -EFAULT;
int count = strlen(s);
for (; count-- > 0; tmp++) {
- if (!in_interrupt() && (((count + 1) & 0x1f) == 0))
- /*
- * let's be a little nice with other processes
- * that need some CPU
- */
- schedule();
+ if (((count + 1) & 0x1f) == 0)
+ cond_resched();
charlcd_write_char(lcd, *tmp);
}
get_device(dev);
+ kfree(dev->p->deferred_probe_reason);
+ dev->p->deferred_probe_reason = NULL;
+
/*
* Drop the mutex while probing each device; the probe path may
* manipulate the deferred list
static void deferred_probe_timeout_work_func(struct work_struct *work)
{
- struct device_private *private, *p;
+ struct device_private *p;
driver_deferred_probe_timeout = 0;
driver_deferred_probe_trigger();
flush_work(&deferred_probe_work);
- list_for_each_entry_safe(private, p, &deferred_probe_pending_list, deferred_probe)
- dev_info(private->device, "deferred probe pending\n");
+ mutex_lock(&deferred_probe_mutex);
+ list_for_each_entry(p, &deferred_probe_pending_list, deferred_probe)
+ dev_info(p->device, "deferred probe pending\n");
+ mutex_unlock(&deferred_probe_mutex);
wake_up_all(&probe_timeout_waitqueue);
}
static DECLARE_DELAYED_WORK(deferred_probe_timeout_work, deferred_probe_timeout_work_func);
return 0;
}
-static void rpm_put_suppliers(struct device *dev)
+static void __rpm_put_suppliers(struct device *dev, bool try_to_suspend)
{
struct device_link *link;
device_links_read_lock_held()) {
while (refcount_dec_not_one(&link->rpm_active))
- pm_runtime_put(link->supplier);
+ pm_runtime_put_noidle(link->supplier);
+
+ if (try_to_suspend)
+ pm_request_idle(link->supplier);
}
}
+static void rpm_put_suppliers(struct device *dev)
+{
+ __rpm_put_suppliers(dev, true);
+}
+
+static void rpm_suspend_suppliers(struct device *dev)
+{
+ struct device_link *link;
+ int idx = device_links_read_lock();
+
+ list_for_each_entry_rcu(link, &dev->links.suppliers, c_node,
+ device_links_read_lock_held())
+ pm_request_idle(link->supplier);
+
+ device_links_read_unlock(idx);
+}
+
/**
* __rpm_callback - Run a given runtime PM callback for a given device.
* @cb: Runtime PM callback to run.
static int __rpm_callback(int (*cb)(struct device *), struct device *dev)
__releases(&dev->power.lock) __acquires(&dev->power.lock)
{
- bool use_links = dev->power.links_count > 0;
- bool get = false;
int retval, idx;
- bool put;
+ bool use_links = dev->power.links_count > 0;
if (dev->power.irq_safe) {
spin_unlock(&dev->power.lock);
- } else if (!use_links) {
- spin_unlock_irq(&dev->power.lock);
} else {
- get = dev->power.runtime_status == RPM_RESUMING;
-
spin_unlock_irq(&dev->power.lock);
- /* Resume suppliers if necessary. */
- if (get) {
+ /*
+ * Resume suppliers if necessary.
+ *
+ * The device's runtime PM status cannot change until this
+ * routine returns, so it is safe to read the status outside of
+ * the lock.
+ */
+ if (use_links && dev->power.runtime_status == RPM_RESUMING) {
idx = device_links_read_lock();
retval = rpm_get_suppliers(dev);
- if (retval)
+ if (retval) {
+ rpm_put_suppliers(dev);
goto fail;
+ }
device_links_read_unlock(idx);
}
if (dev->power.irq_safe) {
spin_lock(&dev->power.lock);
- return retval;
- }
-
- spin_lock_irq(&dev->power.lock);
-
- if (!use_links)
- return retval;
-
- /*
- * If the device is suspending and the callback has returned success,
- * drop the usage counters of the suppliers that have been reference
- * counted on its resume.
- *
- * Do that if the resume fails too.
- */
- put = dev->power.runtime_status == RPM_SUSPENDING && !retval;
- if (put)
- __update_runtime_status(dev, RPM_SUSPENDED);
- else
- put = get && retval;
-
- if (put) {
- spin_unlock_irq(&dev->power.lock);
+ } else {
+ /*
+ * If the device is suspending and the callback has returned
+ * success, drop the usage counters of the suppliers that have
+ * been reference counted on its resume.
+ *
+ * Do that if resume fails too.
+ */
+ if (use_links
+ && ((dev->power.runtime_status == RPM_SUSPENDING && !retval)
+ || (dev->power.runtime_status == RPM_RESUMING && retval))) {
+ idx = device_links_read_lock();
- idx = device_links_read_lock();
+ __rpm_put_suppliers(dev, false);
fail:
- rpm_put_suppliers(dev);
-
- device_links_read_unlock(idx);
+ device_links_read_unlock(idx);
+ }
spin_lock_irq(&dev->power.lock);
}
goto out;
}
+ if (dev->power.irq_safe)
+ goto out;
+
/* Maybe the parent is now able to suspend. */
- if (parent && !parent->power.ignore_children && !dev->power.irq_safe) {
+ if (parent && !parent->power.ignore_children) {
spin_unlock(&dev->power.lock);
spin_lock(&parent->power.lock);
spin_lock(&dev->power.lock);
}
+ /* Maybe the suppliers are now able to suspend. */
+ if (dev->power.links_count > 0) {
+ spin_unlock_irq(&dev->power.lock);
+
+ rpm_suspend_suppliers(dev);
+
+ spin_lock_irq(&dev->power.lock);
+ }
out:
trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
device_links_read_lock_held())
if (link->flags & DL_FLAG_PM_RUNTIME) {
link->supplier_preactivated = true;
- refcount_inc(&link->rpm_active);
pm_runtime_get_sync(link->supplier);
+ refcount_inc(&link->rpm_active);
}
device_links_read_unlock(idx);
void pm_runtime_put_suppliers(struct device *dev)
{
struct device_link *link;
+ unsigned long flags;
+ bool put;
int idx;
idx = device_links_read_lock();
device_links_read_lock_held())
if (link->supplier_preactivated) {
link->supplier_preactivated = false;
- if (refcount_dec_not_one(&link->rpm_active))
+ spin_lock_irqsave(&dev->power.lock, flags);
+ put = pm_runtime_status_suspended(dev) &&
+ refcount_dec_not_one(&link->rpm_active);
+ spin_unlock_irqrestore(&dev->power.lock, flags);
+ if (put)
pm_runtime_put(link->supplier);
}
module_param(FLOPPY_IRQ, int, 0);
module_param(FLOPPY_DMA, int, 0);
MODULE_AUTHOR("Alain L. Knaff");
-MODULE_SUPPORTED_DEVICE("fd");
MODULE_LICENSE("GPL");
/* This doesn't actually get used other than for module information */
}
if (dev->zoned)
- cmd->error = null_process_zoned_cmd(cmd, op,
- sector, nr_sectors);
+ sts = null_process_zoned_cmd(cmd, op, sector, nr_sectors);
else
- cmd->error = null_process_cmd(cmd, op, sector, nr_sectors);
+ sts = null_process_cmd(cmd, op, sector, nr_sectors);
+
+ /* Do not overwrite errors (e.g. timeout errors) */
+ if (cmd->error == BLK_STS_OK)
+ cmd->error = sts;
out:
nullb_complete_cmd(cmd);
static enum blk_eh_timer_return null_timeout_rq(struct request *rq, bool res)
{
+ struct nullb_cmd *cmd = blk_mq_rq_to_pdu(rq);
+
pr_info("rq %p timed out\n", rq);
- blk_mq_complete_request(rq);
+
+ /*
+ * If the device is marked as blocking (i.e. memory backed or zoned
+ * device), the submission path may be blocked waiting for resources
+ * and cause real timeouts. For these real timeouts, the submission
+ * path will complete the request using blk_mq_complete_request().
+ * Only fake timeouts need to execute blk_mq_complete_request() here.
+ */
+ cmd->error = BLK_STS_TIMEOUT;
+ if (cmd->fake_timeout)
+ blk_mq_complete_request(rq);
return BLK_EH_DONE;
}
cmd->rq = bd->rq;
cmd->error = BLK_STS_OK;
cmd->nq = nq;
+ cmd->fake_timeout = should_timeout_request(bd->rq);
blk_mq_start_request(bd->rq);
return BLK_STS_OK;
}
}
- if (should_timeout_request(bd->rq))
+ if (cmd->fake_timeout)
return BLK_STS_OK;
return null_handle_cmd(cmd, sector, nr_sectors, req_op(bd->rq));
blk_status_t error;
struct nullb_queue *nq;
struct hrtimer timer;
+ bool fake_timeout;
};
struct nullb_queue {
out:
for (i = last_map; i < num; i++) {
/* Don't zap current batch's valid persistent grants. */
- if(i >= last_map + segs_to_map)
+ if(i >= map_until)
pages[i]->persistent_gnt = NULL;
pages[i]->handle = BLKBACK_INVALID_HANDLE;
}
module_exit(rsi_91x_bt_module_exit);
MODULE_AUTHOR("Redpine Signals Inc");
MODULE_DESCRIPTION("RSI BT driver");
-MODULE_SUPPORTED_DEVICE("RSI-BT");
MODULE_LICENSE("Dual BSD/GPL");
data->diag = NULL;
}
- if (!enable_autosuspend)
- usb_disable_autosuspend(data->udev);
+ if (enable_autosuspend)
+ usb_enable_autosuspend(data->udev);
err = hci_register_dev(hdev);
if (err < 0)
gpiod_put(data->reset_gpio);
hci_free_dev(hdev);
-
- if (!enable_autosuspend)
- usb_enable_autosuspend(data->udev);
}
#ifdef CONFIG_PM
/*
* Turris Mox module configuration bus driver
*
- * Copyright (C) 2019 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2019 Marek Behún <kabel@kernel.org>
*/
#include <dt-bindings/bus/moxtet.h>
}
module_exit(moxtet_exit);
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
MODULE_DESCRIPTION("CZ.NIC's Turris Mox module configuration bus");
MODULE_LICENSE("GPL v2");
* This part of the memory is above 4 GB, so we don't
* care for the MBus bridge hole.
*/
- if (reg_start >= 0x100000000ULL)
+ if ((u64)reg_start >= 0x100000000ULL)
continue;
/*
*/
l3->debug_irq = platform_get_irq(pdev, 0);
ret = devm_request_irq(l3->dev, l3->debug_irq, l3_interrupt_handler,
- 0x0, "l3-dbg-irq", l3);
+ IRQF_NO_THREAD, "l3-dbg-irq", l3);
if (ret) {
dev_err(l3->dev, "request_irq failed for %d\n",
l3->debug_irq);
l3->app_irq = platform_get_irq(pdev, 1);
ret = devm_request_irq(l3->dev, l3->app_irq, l3_interrupt_handler,
- 0x0, "l3-app-irq", l3);
+ IRQF_NO_THREAD, "l3-app-irq", l3);
if (ret)
dev_err(l3->dev, "request_irq failed for %d\n", l3->app_irq);
pm_runtime_put_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
- reset_control_assert(ddata->rsts);
+
+ if (!reset_control_status(ddata->rsts))
+ reset_control_assert(ddata->rsts);
unprepare:
sysc_unprepare(ddata);
config AGP_PARISC
tristate "HP Quicksilver AGP support"
- depends on AGP && PARISC && 64BIT
+ depends on AGP && PARISC && 64BIT && IOMMU_SBA
help
This option gives you AGP GART support for the HP Quicksilver
AGP bus adapter on HP PA-RISC machines (Ok, just on the C8000
MODULE_LICENSE("GPL");
MODULE_ALIAS_MISCDEV(AC_MINOR);
-MODULE_SUPPORTED_DEVICE("ac");
-
-
static struct applicom_board {
unsigned long PhysIO;
void __iomem *RamIO;
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Jonathan Buzzard <jonathan@buzzard.org.uk>");
MODULE_DESCRIPTION("Toshiba laptop SMM driver");
-MODULE_SUPPORTED_DEVICE("toshiba");
static DEFINE_MUTEX(tosh_mutex);
static int tosh_fn;
static void devm_clk_hw_register_fixed_factor_release(struct device *dev, void *res)
{
- clk_hw_unregister_fixed_factor(&((struct clk_fixed_factor *)res)->hw);
+ struct clk_fixed_factor *fix = res;
+
+ /*
+ * We can not use clk_hw_unregister_fixed_factor, since it will kfree()
+ * the hw, resulting in double free. Just unregister the hw and let
+ * devres code kfree() it.
+ */
+ clk_hw_unregister(&fix->hw);
}
static struct clk_hw *
*/
#include <linux/clk-provider.h>
+#include <linux/device.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/io.h>
}
EXPORT_SYMBOL_GPL(__clk_hw_register_mux);
+static void devm_clk_hw_release_mux(struct device *dev, void *res)
+{
+ clk_hw_unregister_mux(*(struct clk_hw **)res);
+}
+
+struct clk_hw *__devm_clk_hw_register_mux(struct device *dev, struct device_node *np,
+ const char *name, u8 num_parents,
+ const char * const *parent_names,
+ const struct clk_hw **parent_hws,
+ const struct clk_parent_data *parent_data,
+ unsigned long flags, void __iomem *reg, u8 shift, u32 mask,
+ u8 clk_mux_flags, u32 *table, spinlock_t *lock)
+{
+ struct clk_hw **ptr, *hw;
+
+ ptr = devres_alloc(devm_clk_hw_release_mux, sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+
+ hw = __clk_hw_register_mux(dev, np, name, num_parents, parent_names, parent_hws,
+ parent_data, flags, reg, shift, mask,
+ clk_mux_flags, table, lock);
+
+ if (!IS_ERR(hw)) {
+ *ptr = hw;
+ devres_add(dev, ptr);
+ } else {
+ devres_free(ptr);
+ }
+
+ return hw;
+}
+EXPORT_SYMBOL_GPL(__devm_clk_hw_register_mux);
+
struct clk *clk_register_mux_table(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents,
unsigned long flags, void __iomem *reg, u8 shift, u32 mask,
/* search the list of notifiers for this clk */
list_for_each_entry(cn, &clk_notifier_list, node)
if (cn->clk == clk)
- break;
+ goto found;
/* if clk wasn't in the notifier list, allocate new clk_notifier */
- if (cn->clk != clk) {
- cn = kzalloc(sizeof(*cn), GFP_KERNEL);
- if (!cn)
- goto out;
+ cn = kzalloc(sizeof(*cn), GFP_KERNEL);
+ if (!cn)
+ goto out;
- cn->clk = clk;
- srcu_init_notifier_head(&cn->notifier_head);
+ cn->clk = clk;
+ srcu_init_notifier_head(&cn->notifier_head);
- list_add(&cn->node, &clk_notifier_list);
- }
+ list_add(&cn->node, &clk_notifier_list);
+found:
ret = srcu_notifier_chain_register(&cn->notifier_head, nb);
clk->core->notifier_count++;
*/
int clk_notifier_unregister(struct clk *clk, struct notifier_block *nb)
{
- struct clk_notifier *cn = NULL;
- int ret = -EINVAL;
+ struct clk_notifier *cn;
+ int ret = -ENOENT;
if (!clk || !nb)
return -EINVAL;
clk_prepare_lock();
- list_for_each_entry(cn, &clk_notifier_list, node)
- if (cn->clk == clk)
- break;
-
- if (cn->clk == clk) {
- ret = srcu_notifier_chain_unregister(&cn->notifier_head, nb);
+ list_for_each_entry(cn, &clk_notifier_list, node) {
+ if (cn->clk == clk) {
+ ret = srcu_notifier_chain_unregister(&cn->notifier_head, nb);
- clk->core->notifier_count--;
+ clk->core->notifier_count--;
- /* XXX the notifier code should handle this better */
- if (!cn->notifier_head.head) {
- srcu_cleanup_notifier_head(&cn->notifier_head);
- list_del(&cn->node);
- kfree(cn);
+ /* XXX the notifier code should handle this better */
+ if (!cn->notifier_head.head) {
+ srcu_cleanup_notifier_head(&cn->notifier_head);
+ list_del(&cn->node);
+ kfree(cn);
+ }
+ break;
}
-
- } else {
- ret = -ENOENT;
}
clk_prepare_unlock();
.name = "cam_cc_bps_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_cci_0_clk_src",
.parent_data = cam_cc_parent_data_5,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_cci_1_clk_src",
.parent_data = cam_cc_parent_data_5,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_cphy_rx_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_csi0phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_csi1phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_csi2phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_csi3phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_fast_ahb_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_icp_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_ife_0_clk_src",
.parent_data = cam_cc_parent_data_4,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_ife_0_csid_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_ife_1_clk_src",
.parent_data = cam_cc_parent_data_4,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_ife_1_csid_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.parent_data = cam_cc_parent_data_4,
.num_parents = 4,
.flags = CLK_SET_RATE_PARENT,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_ife_lite_csid_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_ipe_0_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_jpeg_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_lrme_clk_src",
.parent_data = cam_cc_parent_data_6,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_mclk0_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_mclk1_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_mclk2_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_mclk3_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.name = "cam_cc_mclk4_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
.flags = CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
struct clk_rate_request parent_req = { };
struct clk_rcg2_gfx3d *cgfx = to_clk_rcg2_gfx3d(hw);
struct clk_hw *xo, *p0, *p1, *p2;
- unsigned long request, p0_rate;
+ unsigned long p0_rate;
+ u8 mux_div = cgfx->div;
int ret;
p0 = cgfx->hws[0];
return 0;
}
- request = req->rate;
- if (cgfx->div > 1)
- parent_req.rate = request = request * cgfx->div;
+ if (mux_div == 0)
+ mux_div = 1;
+
+ parent_req.rate = req->rate * mux_div;
/* This has to be a fixed rate PLL */
p0_rate = clk_hw_get_rate(p0);
- if (request == p0_rate) {
+ if (parent_req.rate == p0_rate) {
req->rate = req->best_parent_rate = p0_rate;
req->best_parent_hw = p0;
return 0;
if (req->best_parent_hw == p0) {
/* Are we going back to a previously used rate? */
- if (clk_hw_get_rate(p2) == request)
+ if (clk_hw_get_rate(p2) == parent_req.rate)
req->best_parent_hw = p2;
else
req->best_parent_hw = p1;
return ret;
req->rate = req->best_parent_rate = parent_req.rate;
- if (cgfx->div > 1)
- req->rate /= cgfx->div;
+ req->rate /= mux_div;
return 0;
}
.num_clks = ARRAY_SIZE(sm8350_rpmh_clocks),
};
+/* Resource name must match resource id present in cmd-db */
+DEFINE_CLK_RPMH_ARC(sc7280, bi_tcxo, bi_tcxo_ao, "xo.lvl", 0x3, 4);
+
static struct clk_hw *sc7280_rpmh_clocks[] = {
- [RPMH_CXO_CLK] = &sdm845_bi_tcxo.hw,
- [RPMH_CXO_CLK_A] = &sdm845_bi_tcxo_ao.hw,
+ [RPMH_CXO_CLK] = &sc7280_bi_tcxo.hw,
+ [RPMH_CXO_CLK_A] = &sc7280_bi_tcxo_ao.hw,
[RPMH_LN_BB_CLK2] = &sdm845_ln_bb_clk2.hw,
[RPMH_LN_BB_CLK2_A] = &sdm845_ln_bb_clk2_ao.hw,
[RPMH_RF_CLK1] = &sdm845_rf_clk1.hw,
.name = "gcc_sdcc1_apps_clk_src",
.parent_data = gcc_parent_data_1,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_floor_ops,
},
};
.name = "gcc_sdcc1_ice_core_clk_src",
.parent_data = gcc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_floor_ops,
+ .ops = &clk_rcg2_ops,
},
};
val = readl(socfpgaclk->div_reg) >> socfpgaclk->shift;
val &= GENMASK(socfpgaclk->width - 1, 0);
/* Check for GPIO_DB_CLK by its offset */
- if ((int) socfpgaclk->div_reg & SOCFPGA_GPIO_DB_CLK_OFFSET)
+ if ((uintptr_t) socfpgaclk->div_reg & SOCFPGA_GPIO_DB_CLK_OFFSET)
div = val + 1;
else
div = (1 << val);
struct counter_device counter;
struct regmap *regmap;
struct clk *clk;
- u32 ceiling;
+ u32 max_arr;
bool enabled;
struct stm32_timer_regs bak;
};
* @STM32_COUNT_ENCODER_MODE_3: counts on both TI1FP1 and TI2FP2 edges
*/
enum stm32_count_function {
- STM32_COUNT_SLAVE_MODE_DISABLED = -1,
+ STM32_COUNT_SLAVE_MODE_DISABLED,
STM32_COUNT_ENCODER_MODE_1,
STM32_COUNT_ENCODER_MODE_2,
STM32_COUNT_ENCODER_MODE_3,
};
static enum counter_count_function stm32_count_functions[] = {
+ [STM32_COUNT_SLAVE_MODE_DISABLED] = COUNTER_COUNT_FUNCTION_INCREASE,
[STM32_COUNT_ENCODER_MODE_1] = COUNTER_COUNT_FUNCTION_QUADRATURE_X2_A,
[STM32_COUNT_ENCODER_MODE_2] = COUNTER_COUNT_FUNCTION_QUADRATURE_X2_B,
[STM32_COUNT_ENCODER_MODE_3] = COUNTER_COUNT_FUNCTION_QUADRATURE_X4,
const unsigned long val)
{
struct stm32_timer_cnt *const priv = counter->priv;
+ u32 ceiling;
- if (val > priv->ceiling)
+ regmap_read(priv->regmap, TIM_ARR, &ceiling);
+ if (val > ceiling)
return -EINVAL;
return regmap_write(priv->regmap, TIM_CNT, val);
regmap_read(priv->regmap, TIM_SMCR, &smcr);
switch (smcr & TIM_SMCR_SMS) {
+ case 0:
+ *function = STM32_COUNT_SLAVE_MODE_DISABLED;
+ return 0;
case 1:
*function = STM32_COUNT_ENCODER_MODE_1;
return 0;
case 3:
*function = STM32_COUNT_ENCODER_MODE_3;
return 0;
+ default:
+ return -EINVAL;
}
-
- return -EINVAL;
}
static int stm32_count_function_set(struct counter_device *counter,
u32 cr1, sms;
switch (function) {
+ case STM32_COUNT_SLAVE_MODE_DISABLED:
+ sms = 0;
+ break;
case STM32_COUNT_ENCODER_MODE_1:
sms = 1;
break;
sms = 3;
break;
default:
- sms = 0;
- break;
+ return -EINVAL;
}
/* Store enable status */
regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN, 0);
- /* TIMx_ARR register shouldn't be buffered (ARPE=0) */
- regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_ARPE, 0);
- regmap_write(priv->regmap, TIM_ARR, priv->ceiling);
-
regmap_update_bits(priv->regmap, TIM_SMCR, TIM_SMCR_SMS, sms);
/* Make sure that registers are updated */
if (ret)
return ret;
+ if (ceiling > priv->max_arr)
+ return -ERANGE;
+
/* TIMx_ARR register shouldn't be buffered (ARPE=0) */
regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_ARPE, 0);
regmap_write(priv->regmap, TIM_ARR, ceiling);
- priv->ceiling = ceiling;
return len;
}
size_t function;
int err;
- /* Default action mode (e.g. STM32_COUNT_SLAVE_MODE_DISABLED) */
- *action = STM32_SYNAPSE_ACTION_NONE;
-
err = stm32_count_function_get(counter, count, &function);
if (err)
- return 0;
+ return err;
switch (function) {
+ case STM32_COUNT_SLAVE_MODE_DISABLED:
+ /* counts on internal clock when CEN=1 */
+ *action = STM32_SYNAPSE_ACTION_NONE;
+ return 0;
case STM32_COUNT_ENCODER_MODE_1:
/* counts up/down on TI1FP1 edge depending on TI2FP2 level */
if (synapse->signal->id == count->synapses[0].signal->id)
*action = STM32_SYNAPSE_ACTION_BOTH_EDGES;
- break;
+ else
+ *action = STM32_SYNAPSE_ACTION_NONE;
+ return 0;
case STM32_COUNT_ENCODER_MODE_2:
/* counts up/down on TI2FP2 edge depending on TI1FP1 level */
if (synapse->signal->id == count->synapses[1].signal->id)
*action = STM32_SYNAPSE_ACTION_BOTH_EDGES;
- break;
+ else
+ *action = STM32_SYNAPSE_ACTION_NONE;
+ return 0;
case STM32_COUNT_ENCODER_MODE_3:
/* counts up/down on both TI1FP1 and TI2FP2 edges */
*action = STM32_SYNAPSE_ACTION_BOTH_EDGES;
- break;
+ return 0;
+ default:
+ return -EINVAL;
}
-
- return 0;
}
static const struct counter_ops stm32_timer_cnt_ops = {
priv->regmap = ddata->regmap;
priv->clk = ddata->clk;
- priv->ceiling = ddata->max_arr;
+ priv->max_arr = ddata->max_arr;
priv->counter.name = dev_name(dev);
priv->counter.parent = dev;
__ATTR_RO(_name##_frequencies)
/*
- * show_scaling_available_frequencies - show available normal frequencies for
+ * scaling_available_frequencies_show - show available normal frequencies for
* the specified CPU
*/
static ssize_t scaling_available_frequencies_show(struct cpufreq_policy *policy,
EXPORT_SYMBOL_GPL(cpufreq_freq_attr_scaling_available_freqs);
/*
- * show_available_boost_freqs - show available boost frequencies for
+ * scaling_boost_frequencies_show - show available boost frequencies for
* the specified CPU
*/
static ssize_t scaling_boost_frequencies_show(struct cpufreq_policy *policy,
/**
* dma_fence_get_stub - return a signaled fence
*
- * Return a stub fence which is already signaled.
+ * Return a stub fence which is already signaled. The fence's
+ * timestamp corresponds to the first time after boot this
+ * function is called.
*/
struct dma_fence *dma_fence_get_stub(void)
{
}
EXPORT_SYMBOL(dma_fence_get_stub);
+/**
+ * dma_fence_allocate_private_stub - return a private, signaled fence
+ *
+ * Return a newly allocated and signaled stub fence.
+ */
+struct dma_fence *dma_fence_allocate_private_stub(void)
+{
+ struct dma_fence *fence;
+
+ fence = kzalloc(sizeof(*fence), GFP_KERNEL);
+ if (fence == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ dma_fence_init(fence,
+ &dma_fence_stub_ops,
+ &dma_fence_stub_lock,
+ 0, 0);
+ dma_fence_signal(fence);
+
+ return fence;
+}
+EXPORT_SYMBOL(dma_fence_allocate_private_stub);
+
/**
* dma_fence_context_alloc - allocate an array of fence contexts
* @num: amount of contexts to allocate
sizeof(*edev->nh), GFP_KERNEL);
if (!edev->nh) {
ret = -ENOMEM;
+ device_unregister(&edev->dev);
goto err_dev;
}
struct client *client = file->private_data;
spinlock_t *client_list_lock = &client->lynx->client_list_lock;
struct nosy_stats stats;
+ int ret;
switch (cmd) {
case NOSY_IOC_GET_STATS:
return 0;
case NOSY_IOC_START:
+ ret = -EBUSY;
spin_lock_irq(client_list_lock);
- list_add_tail(&client->link, &client->lynx->client_list);
+ if (list_empty(&client->link)) {
+ list_add_tail(&client->link, &client->lynx->client_list);
+ ret = 0;
+ }
spin_unlock_irq(client_list_lock);
- return 0;
+ return ret;
case NOSY_IOC_STOP:
spin_lock_irq(client_list_lock);
}
/* first try to find a slot in an existing linked list entry */
- for (prsv = efi_memreserve_root->next; prsv; prsv = rsv->next) {
+ for (prsv = efi_memreserve_root->next; prsv; ) {
rsv = memremap(prsv, sizeof(*rsv), MEMREMAP_WB);
index = atomic_fetch_add_unless(&rsv->count, 1, rsv->size);
if (index < rsv->size) {
memunmap(rsv);
return efi_mem_reserve_iomem(addr, size);
}
+ prsv = rsv->next;
memunmap(rsv);
}
}
}
+ break;
+ case EFI_UNSUPPORTED:
+ err = -EOPNOTSUPP;
+ status = EFI_NOT_FOUND;
break;
case EFI_NOT_FOUND:
break;
/*
* Turris Mox rWTM firmware driver
*
- * Copyright (C) 2019 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2019 Marek Behún <kabel@kernel.org>
*/
#include <linux/armada-37xx-rwtm-mailbox.h>
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("Turris Mox rWTM firmware driver");
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
/*
* Turris Mox Moxtet GPIO expander
*
- * Copyright (C) 2018 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2018 Marek Behún <kabel@kernel.org>
*/
#include <linux/bitops.h>
};
module_moxtet_driver(moxtet_gpio_driver);
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
MODULE_DESCRIPTION("Turris Mox Moxtet GPIO expander");
MODULE_LICENSE("GPL v2");
struct lock_class_key *lock_key,
struct lock_class_key *request_key)
{
+ struct fwnode_handle *fwnode = gc->parent ? dev_fwnode(gc->parent) : NULL;
unsigned long flags;
int ret = 0;
unsigned i;
of_gpio_dev_init(gc, gdev);
+ /*
+ * Assign fwnode depending on the result of the previous calls,
+ * if none of them succeed, assign it to the parent's one.
+ */
+ gdev->dev.fwnode = dev_fwnode(&gdev->dev) ?: fwnode;
+
gdev->id = ida_alloc(&gpio_ida, GFP_KERNEL);
if (gdev->id < 0) {
ret = gdev->id;
return ret;
}
- if (driver_register(&gpio_stub_drv) < 0) {
+ ret = driver_register(&gpio_stub_drv);
+ if (ret < 0) {
pr_err("gpiolib: could not register GPIO stub driver\n");
bus_unregister(&gpio_bus_type);
return ret;
vi.o mxgpu_vi.o nbio_v6_1.o soc15.o emu_soc.o mxgpu_ai.o nbio_v7_0.o vega10_reg_init.o \
vega20_reg_init.o nbio_v7_4.o nbio_v2_3.o nv.o navi10_reg_init.o navi14_reg_init.o \
arct_reg_init.o navi12_reg_init.o mxgpu_nv.o sienna_cichlid_reg_init.o vangogh_reg_init.o \
- nbio_v7_2.o dimgrey_cavefish_reg_init.o hdp_v4_0.o hdp_v5_0.o aldebaran_reg_init.o
+ nbio_v7_2.o dimgrey_cavefish_reg_init.o hdp_v4_0.o hdp_v5_0.o aldebaran_reg_init.o aldebaran.o
# add DF block
amdgpu-y += \
# add UMC block
amdgpu-y += \
- umc_v6_1.o umc_v6_0.o umc_v8_7.o
+ umc_v6_0.o umc_v6_1.o umc_v6_7.o umc_v8_7.o
# add IH block
amdgpu-y += \
smuio_v11_0_6.o \
smuio_v13_0.o
+# add reset block
+amdgpu-y += \
+ amdgpu_reset.o
+
# add amdkfd interfaces
amdgpu-y += amdgpu_amdkfd.o
+
ifneq ($(CONFIG_HSA_AMD),)
AMDKFD_PATH := ../amdkfd
include $(FULL_AMD_PATH)/amdkfd/Makefile
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "aldebaran.h"
+#include "amdgpu_reset.h"
+#include "amdgpu_amdkfd.h"
+#include "amdgpu_dpm.h"
+#include "amdgpu_job.h"
+#include "amdgpu_ring.h"
+#include "amdgpu_ras.h"
+#include "amdgpu_psp.h"
+#include "amdgpu_xgmi.h"
+
+static struct amdgpu_reset_handler *
+aldebaran_get_reset_handler(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ struct amdgpu_reset_handler *handler;
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+
+ if (reset_context->method != AMD_RESET_METHOD_NONE) {
+ dev_dbg(adev->dev, "Getting reset handler for method %d\n",
+ reset_context->method);
+ list_for_each_entry(handler, &reset_ctl->reset_handlers,
+ handler_list) {
+ if (handler->reset_method == reset_context->method)
+ return handler;
+ }
+ }
+
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ list_for_each_entry(handler, &reset_ctl->reset_handlers,
+ handler_list) {
+ if (handler->reset_method == AMD_RESET_METHOD_MODE2) {
+ reset_context->method = AMD_RESET_METHOD_MODE2;
+ return handler;
+ }
+ }
+ }
+
+ dev_dbg(adev->dev, "Reset handler not found!\n");
+
+ return NULL;
+}
+
+static int aldebaran_mode2_suspend_ip(struct amdgpu_device *adev)
+{
+ int r, i;
+
+ amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
+ amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
+
+ for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
+ if (!(adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_GFX ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_SDMA))
+ continue;
+
+ r = adev->ip_blocks[i].version->funcs->suspend(adev);
+
+ if (r) {
+ dev_err(adev->dev,
+ "suspend of IP block <%s> failed %d\n",
+ adev->ip_blocks[i].version->funcs->name, r);
+ return r;
+ }
+
+ adev->ip_blocks[i].status.hw = false;
+ }
+
+ return r;
+}
+
+static int
+aldebaran_mode2_prepare_hwcontext(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ int r = 0;
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+
+ dev_dbg(adev->dev, "Aldebaran prepare hw context\n");
+ /* Don't suspend on bare metal if we are not going to HW reset the ASIC */
+ if (!amdgpu_sriov_vf(adev))
+ r = aldebaran_mode2_suspend_ip(adev);
+
+ return r;
+}
+
+static void aldebaran_async_reset(struct work_struct *work)
+{
+ struct amdgpu_reset_handler *handler;
+ struct amdgpu_reset_control *reset_ctl =
+ container_of(work, struct amdgpu_reset_control, reset_work);
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+
+ list_for_each_entry(handler, &reset_ctl->reset_handlers,
+ handler_list) {
+ if (handler->reset_method == reset_ctl->active_reset) {
+ dev_dbg(adev->dev, "Resetting device\n");
+ handler->do_reset(adev);
+ break;
+ }
+ }
+}
+
+static int aldebaran_mode2_reset(struct amdgpu_device *adev)
+{
+ /* disable BM */
+ pci_clear_master(adev->pdev);
+ adev->asic_reset_res = amdgpu_dpm_mode2_reset(adev);
+ return adev->asic_reset_res;
+}
+
+static int
+aldebaran_mode2_perform_reset(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ struct amdgpu_device *tmp_adev = NULL;
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+ int r = 0;
+
+ dev_dbg(adev->dev, "aldebaran perform hw reset\n");
+ if (reset_context->hive == NULL) {
+ /* Wrong context, return error */
+ return -EINVAL;
+ }
+
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ mutex_lock(&tmp_adev->reset_cntl->reset_lock);
+ tmp_adev->reset_cntl->active_reset = AMD_RESET_METHOD_MODE2;
+ }
+ /*
+ * Mode2 reset doesn't need any sync between nodes in XGMI hive, instead launch
+ * them together so that they can be completed asynchronously on multiple nodes
+ */
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ /* For XGMI run all resets in parallel to speed up the process */
+ if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
+ if (!queue_work(system_unbound_wq,
+ &tmp_adev->reset_cntl->reset_work))
+ r = -EALREADY;
+ } else
+ r = aldebaran_mode2_reset(tmp_adev);
+ if (r) {
+ dev_err(tmp_adev->dev,
+ "ASIC reset failed with error, %d for drm dev, %s",
+ r, adev_to_drm(tmp_adev)->unique);
+ break;
+ }
+ }
+
+ /* For XGMI wait for all resets to complete before proceed */
+ if (!r) {
+ list_for_each_entry(tmp_adev,
+ &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
+ flush_work(&tmp_adev->reset_cntl->reset_work);
+ r = tmp_adev->asic_reset_res;
+ if (r)
+ break;
+ }
+ }
+ }
+
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ mutex_unlock(&tmp_adev->reset_cntl->reset_lock);
+ tmp_adev->reset_cntl->active_reset = AMD_RESET_METHOD_NONE;
+ }
+
+ return r;
+}
+
+static int aldebaran_mode2_restore_ip(struct amdgpu_device *adev)
+{
+ struct amdgpu_firmware_info *ucode_list[AMDGPU_UCODE_ID_MAXIMUM];
+ struct amdgpu_firmware_info *ucode;
+ struct amdgpu_ip_block *cmn_block;
+ int ucode_count = 0;
+ int i, r;
+
+ dev_dbg(adev->dev, "Reloading ucodes after reset\n");
+ for (i = 0; i < adev->firmware.max_ucodes; i++) {
+ ucode = &adev->firmware.ucode[i];
+ if (!ucode->fw)
+ continue;
+ switch (ucode->ucode_id) {
+ case AMDGPU_UCODE_ID_SDMA0:
+ case AMDGPU_UCODE_ID_SDMA1:
+ case AMDGPU_UCODE_ID_SDMA2:
+ case AMDGPU_UCODE_ID_SDMA3:
+ case AMDGPU_UCODE_ID_SDMA4:
+ case AMDGPU_UCODE_ID_SDMA5:
+ case AMDGPU_UCODE_ID_SDMA6:
+ case AMDGPU_UCODE_ID_SDMA7:
+ case AMDGPU_UCODE_ID_CP_MEC1:
+ case AMDGPU_UCODE_ID_CP_MEC1_JT:
+ case AMDGPU_UCODE_ID_RLC_RESTORE_LIST_CNTL:
+ case AMDGPU_UCODE_ID_RLC_RESTORE_LIST_GPM_MEM:
+ case AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM:
+ case AMDGPU_UCODE_ID_RLC_G:
+ ucode_list[ucode_count++] = ucode;
+ break;
+ default:
+ break;
+ };
+ }
+
+ /* Reinit NBIF block */
+ cmn_block =
+ amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_COMMON);
+ if (unlikely(!cmn_block)) {
+ dev_err(adev->dev, "Failed to get BIF handle\n");
+ return -EINVAL;
+ }
+ r = cmn_block->version->funcs->resume(adev);
+ if (r)
+ return r;
+
+ /* Reinit GFXHUB */
+ adev->gfxhub.funcs->init(adev);
+ r = adev->gfxhub.funcs->gart_enable(adev);
+ if (r) {
+ dev_err(adev->dev, "GFXHUB gart reenable failed after reset\n");
+ return r;
+ }
+
+ /* Reload GFX firmware */
+ r = psp_load_fw_list(&adev->psp, ucode_list, ucode_count);
+ if (r) {
+ dev_err(adev->dev, "GFX ucode load failed after reset\n");
+ return r;
+ }
+
+ /* Resume RLC, FW needs RLC alive to complete reset process */
+ adev->gfx.rlc.funcs->resume(adev);
+
+ /* Wait for FW reset event complete */
+ r = smu_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0);
+ if (r) {
+ dev_err(adev->dev,
+ "Failed to get response from firmware after reset\n");
+ return r;
+ }
+
+ for (i = 0; i < adev->num_ip_blocks; i++) {
+ if (!(adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_GFX ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_SDMA))
+ continue;
+ r = adev->ip_blocks[i].version->funcs->resume(adev);
+ if (r) {
+ dev_err(adev->dev,
+ "resume of IP block <%s> failed %d\n",
+ adev->ip_blocks[i].version->funcs->name, r);
+ return r;
+ }
+
+ adev->ip_blocks[i].status.hw = true;
+ }
+
+ for (i = 0; i < adev->num_ip_blocks; i++) {
+ if (!(adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_GFX ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_SDMA ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_COMMON))
+ continue;
+
+ if (adev->ip_blocks[i].version->funcs->late_init) {
+ r = adev->ip_blocks[i].version->funcs->late_init(
+ (void *)adev);
+ if (r) {
+ dev_err(adev->dev,
+ "late_init of IP block <%s> failed %d after reset\n",
+ adev->ip_blocks[i].version->funcs->name,
+ r);
+ return r;
+ }
+ }
+ adev->ip_blocks[i].status.late_initialized = true;
+ }
+
+ amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
+ amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE);
+
+ return r;
+}
+
+static int
+aldebaran_mode2_restore_hwcontext(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ int r;
+ struct amdgpu_device *tmp_adev = NULL;
+
+ if (reset_context->hive == NULL) {
+ /* Wrong context, return error */
+ return -EINVAL;
+ }
+
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ dev_info(tmp_adev->dev,
+ "GPU reset succeeded, trying to resume\n");
+ r = aldebaran_mode2_restore_ip(tmp_adev);
+ if (r)
+ goto end;
+
+ /*
+ * Add this ASIC as tracked as reset was already
+ * complete successfully.
+ */
+ amdgpu_register_gpu_instance(tmp_adev);
+
+ /* Resume RAS */
+ amdgpu_ras_resume(tmp_adev);
+
+ /* Update PSP FW topology after reset */
+ if (reset_context->hive &&
+ tmp_adev->gmc.xgmi.num_physical_nodes > 1)
+ r = amdgpu_xgmi_update_topology(reset_context->hive,
+ tmp_adev);
+
+ if (!r) {
+ amdgpu_irq_gpu_reset_resume_helper(tmp_adev);
+
+ r = amdgpu_ib_ring_tests(tmp_adev);
+ if (r) {
+ dev_err(tmp_adev->dev,
+ "ib ring test failed (%d).\n", r);
+ r = -EAGAIN;
+ tmp_adev->asic_reset_res = r;
+ goto end;
+ }
+ }
+ }
+
+end:
+ return r;
+}
+
+static struct amdgpu_reset_handler aldebaran_mode2_handler = {
+ .reset_method = AMD_RESET_METHOD_MODE2,
+ .prepare_env = NULL,
+ .prepare_hwcontext = aldebaran_mode2_prepare_hwcontext,
+ .perform_reset = aldebaran_mode2_perform_reset,
+ .restore_hwcontext = aldebaran_mode2_restore_hwcontext,
+ .restore_env = NULL,
+ .do_reset = aldebaran_mode2_reset,
+};
+
+int aldebaran_reset_init(struct amdgpu_device *adev)
+{
+ struct amdgpu_reset_control *reset_ctl;
+
+ reset_ctl = kzalloc(sizeof(*reset_ctl), GFP_KERNEL);
+ if (!reset_ctl)
+ return -ENOMEM;
+
+ reset_ctl->handle = adev;
+ reset_ctl->async_reset = aldebaran_async_reset;
+ reset_ctl->active_reset = AMD_RESET_METHOD_NONE;
+ reset_ctl->get_reset_handler = aldebaran_get_reset_handler;
+
+ INIT_LIST_HEAD(&reset_ctl->reset_handlers);
+ INIT_WORK(&reset_ctl->reset_work, reset_ctl->async_reset);
+ /* Only mode2 is handled through reset control now */
+ amdgpu_reset_add_handler(reset_ctl, &aldebaran_mode2_handler);
+
+ adev->reset_cntl = reset_ctl;
+
+ return 0;
+}
+
+int aldebaran_reset_fini(struct amdgpu_device *adev)
+{
+ kfree(adev->reset_cntl);
+ adev->reset_cntl = NULL;
+ return 0;
+}
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __ALDEBARAN_H__
+#define __ALDEBARAN_H__
+
+#include "amdgpu.h"
+
+int aldebaran_reset_init(struct amdgpu_device *adev);
+int aldebaran_reset_fini(struct amdgpu_device *adev);
+
+#endif
#include "amdgpu_gfxhub.h"
#include "amdgpu_df.h"
#include "amdgpu_smuio.h"
-#include "amdgpu_hdp.h"
#define MAX_GPU_INSTANCE 16
struct amdgpu_atif;
struct kfd_vm_fault_info;
struct amdgpu_hive_info;
+struct amdgpu_reset_context;
+struct amdgpu_reset_control;
enum amdgpu_cp_irq {
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP = 0,
};
enum amd_reset_method {
+ AMD_RESET_METHOD_NONE = -1,
AMD_RESET_METHOD_LEGACY = 0,
AMD_RESET_METHOD_MODE0,
AMD_RESET_METHOD_MODE1,
struct amdgpu_irq_src pageflip_irq;
struct amdgpu_irq_src hpd_irq;
struct amdgpu_irq_src dmub_trace_irq;
+ struct amdgpu_irq_src dmub_outbox_irq;
/* rings */
u64 fence_context;
/* s3/s4 mask */
bool in_suspend;
- bool in_hibernate;
-
- /*
- * The combination flag in_poweroff_reboot_com used to identify the poweroff
- * and reboot opt in the s0i3 system-wide suspend.
- */
- bool in_poweroff_reboot_com;
+ bool in_s3;
+ bool in_s4;
+ bool in_s0ix;
atomic_t in_gpu_reset;
enum pp_mp1_state mp1_state;
bool in_pci_err_recovery;
struct pci_saved_state *pci_state;
+
+ struct amdgpu_reset_control *reset_cntl;
};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
bool amdgpu_device_has_dc_support(struct amdgpu_device *adev);
int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
- struct amdgpu_job *job,
- bool *need_full_reset_arg);
+ struct amdgpu_reset_context *reset_context);
-int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
- struct list_head *device_list_handle,
- bool *need_full_reset_arg,
- bool skip_hw_reset);
+int amdgpu_do_asic_reset(struct list_head *device_list_handle,
+ struct amdgpu_reset_context *reset_context);
int emu_soc_asic_init(struct amdgpu_device *adev);
const u32 *registers,
const u32 array_size);
-bool amdgpu_device_supports_atpx(struct drm_device *dev);
int amdgpu_device_mode1_reset(struct amdgpu_device *adev);
+bool amdgpu_device_supports_atpx(struct drm_device *dev);
+bool amdgpu_device_supports_px(struct drm_device *dev);
bool amdgpu_device_supports_boco(struct drm_device *dev);
bool amdgpu_device_supports_baco(struct drm_device *dev);
bool amdgpu_device_is_peer_accessible(struct amdgpu_device *adev,
bool amdgpu_device_cache_pci_state(struct pci_dev *pdev);
bool amdgpu_device_load_pci_state(struct pci_dev *pdev);
+bool amdgpu_device_skip_hw_access(struct amdgpu_device *adev);
+
+int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
+ enum amd_clockgating_state state);
+int amdgpu_device_set_pg_state(struct amdgpu_device *adev,
+ enum amd_powergating_state state);
+
#include "amdgpu_object.h"
static inline bool amdgpu_is_tmz(struct amdgpu_device *adev)
bp.flags = AMDGPU_GEM_CREATE_CPU_GTT_USWC;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
if (cp_mqd_gfx9)
bp.flags |= AMDGPU_GEM_CREATE_CP_MQD_GFX9;
{
struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct amdgpu_bo *bo = NULL;
+ struct amdgpu_bo_user *ubo;
struct amdgpu_bo_param bp;
int r;
bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS;
bp.type = ttm_bo_type_device;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
- r = amdgpu_bo_create(adev, &bp, &bo);
+ r = amdgpu_bo_create_user(adev, &bp, &ubo);
if (r) {
dev_err(adev->dev,
"failed to allocate gws BO for amdkfd (%d)\n", r);
return r;
}
+ bo = &ubo->bo;
*mem_obj = bo;
return 0;
}
*dma_buf_kgd = (struct kgd_dev *)adev;
if (bo_size)
*bo_size = amdgpu_bo_size(bo);
- if (metadata_size)
- *metadata_size = bo->metadata_size;
if (metadata_buffer)
r = amdgpu_bo_get_metadata(bo, metadata_buffer, buffer_size,
metadata_size, &metadata_flags);
return amdgpu_atombios_get_max_vddc(adev, VOLTAGE_TYPE_VDDC, leakage_idx, voltage);
}
-int amdgpu_atombios_get_leakage_id_from_vbios(struct amdgpu_device *adev,
- u16 *leakage_id)
-{
- union set_voltage args;
- int index = GetIndexIntoMasterTable(COMMAND, SetVoltage);
- u8 frev, crev;
-
- if (!amdgpu_atom_parse_cmd_header(adev->mode_info.atom_context, index, &frev, &crev))
- return -EINVAL;
-
- switch (crev) {
- case 3:
- case 4:
- args.v3.ucVoltageType = 0;
- args.v3.ucVoltageMode = ATOM_GET_LEAKAGE_ID;
- args.v3.usVoltageLevel = 0;
-
- amdgpu_atom_execute_table(adev->mode_info.atom_context, index, (uint32_t *)&args);
-
- *leakage_id = le16_to_cpu(args.v3.usVoltageLevel);
- break;
- default:
- DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
- return -EINVAL;
- }
-
- return 0;
-}
-
-int amdgpu_atombios_get_leakage_vddc_based_on_leakage_params(struct amdgpu_device *adev,
- u16 *vddc, u16 *vddci,
- u16 virtual_voltage_id,
- u16 vbios_voltage_id)
-{
- int index = GetIndexIntoMasterTable(DATA, ASIC_ProfilingInfo);
- u8 frev, crev;
- u16 data_offset, size;
- int i, j;
- ATOM_ASIC_PROFILING_INFO_V2_1 *profile;
- u16 *leakage_bin, *vddc_id_buf, *vddc_buf, *vddci_id_buf, *vddci_buf;
-
- *vddc = 0;
- *vddci = 0;
-
- if (!amdgpu_atom_parse_data_header(adev->mode_info.atom_context, index, &size,
- &frev, &crev, &data_offset))
- return -EINVAL;
-
- profile = (ATOM_ASIC_PROFILING_INFO_V2_1 *)
- (adev->mode_info.atom_context->bios + data_offset);
-
- switch (frev) {
- case 1:
- return -EINVAL;
- case 2:
- switch (crev) {
- case 1:
- if (size < sizeof(ATOM_ASIC_PROFILING_INFO_V2_1))
- return -EINVAL;
- leakage_bin = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usLeakageBinArrayOffset));
- vddc_id_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDC_IdArrayOffset));
- vddc_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDC_LevelArrayOffset));
- vddci_id_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDCI_IdArrayOffset));
- vddci_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDCI_LevelArrayOffset));
-
- if (profile->ucElbVDDC_Num > 0) {
- for (i = 0; i < profile->ucElbVDDC_Num; i++) {
- if (vddc_id_buf[i] == virtual_voltage_id) {
- for (j = 0; j < profile->ucLeakageBinNum; j++) {
- if (vbios_voltage_id <= leakage_bin[j]) {
- *vddc = vddc_buf[j * profile->ucElbVDDC_Num + i];
- break;
- }
- }
- break;
- }
- }
- }
- if (profile->ucElbVDDCI_Num > 0) {
- for (i = 0; i < profile->ucElbVDDCI_Num; i++) {
- if (vddci_id_buf[i] == virtual_voltage_id) {
- for (j = 0; j < profile->ucLeakageBinNum; j++) {
- if (vbios_voltage_id <= leakage_bin[j]) {
- *vddci = vddci_buf[j * profile->ucElbVDDCI_Num + i];
- break;
- }
- }
- break;
- }
- }
- }
- break;
- default:
- DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
- return -EINVAL;
- }
- break;
- default:
- DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
- return -EINVAL;
- }
-
- return 0;
-}
-
-union get_voltage_info {
- struct _GET_VOLTAGE_INFO_INPUT_PARAMETER_V1_2 in;
- struct _GET_EVV_VOLTAGE_INFO_OUTPUT_PARAMETER_V1_2 evv_out;
-};
-
-int amdgpu_atombios_get_voltage_evv(struct amdgpu_device *adev,
- u16 virtual_voltage_id,
- u16 *voltage)
-{
- int index = GetIndexIntoMasterTable(COMMAND, GetVoltageInfo);
- u32 entry_id;
- u32 count = adev->pm.dpm.dyn_state.vddc_dependency_on_sclk.count;
- union get_voltage_info args;
-
- for (entry_id = 0; entry_id < count; entry_id++) {
- if (adev->pm.dpm.dyn_state.vddc_dependency_on_sclk.entries[entry_id].v ==
- virtual_voltage_id)
- break;
- }
-
- if (entry_id >= count)
- return -EINVAL;
-
- args.in.ucVoltageType = VOLTAGE_TYPE_VDDC;
- args.in.ucVoltageMode = ATOM_GET_VOLTAGE_EVV_VOLTAGE;
- args.in.usVoltageLevel = cpu_to_le16(virtual_voltage_id);
- args.in.ulSCLKFreq =
- cpu_to_le32(adev->pm.dpm.dyn_state.vddc_dependency_on_sclk.entries[entry_id].clk);
-
- amdgpu_atom_execute_table(adev->mode_info.atom_context, index, (uint32_t *)&args);
-
- *voltage = le16_to_cpu(args.evv_out.usVoltageLevel);
-
- return 0;
-}
-
union voltage_object_info {
struct _ATOM_VOLTAGE_OBJECT_INFO v1;
struct _ATOM_VOLTAGE_OBJECT_INFO_V2 v2;
struct amdgpu_device *adev = drm_to_adev(ddev);
struct atom_context *ctx = adev->mode_info.atom_context;
- return snprintf(buf, PAGE_SIZE, "%s\n", ctx->vbios_version);
+ return sysfs_emit(buf, "%s\n", ctx->vbios_version);
}
static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version,
void amdgpu_atombios_set_engine_dram_timings(struct amdgpu_device *adev,
u32 eng_clock, u32 mem_clock);
-int amdgpu_atombios_get_leakage_id_from_vbios(struct amdgpu_device *adev,
- u16 *leakage_id);
-
-int amdgpu_atombios_get_leakage_vddc_based_on_leakage_params(struct amdgpu_device *adev,
- u16 *vddc, u16 *vddci,
- u16 virtual_voltage_id,
- u16 vbios_voltage_id);
-
-int amdgpu_atombios_get_voltage_evv(struct amdgpu_device *adev,
- u16 virtual_voltage_id,
- u16 *voltage);
-
bool
amdgpu_atombios_is_voltage_gpio(struct amdgpu_device *adev,
u8 voltage_type, u8 voltage_mode);
bp.flags = 0;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
n = AMDGPU_BENCHMARK_ITERATIONS;
r = amdgpu_bo_create(adev, &bp, &sobj);
if (r) {
#include "amdgpu_ras.h"
#include "amdgpu_pmu.h"
#include "amdgpu_fru_eeprom.h"
+#include "amdgpu_reset.h"
#include <linux/suspend.h>
#include <drm/task_barrier.h>
struct amdgpu_device *adev = drm_to_adev(ddev);
uint64_t cnt = amdgpu_asic_get_pcie_replay_count(adev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", cnt);
+ return sysfs_emit(buf, "%llu\n", cnt);
}
static DEVICE_ATTR(pcie_replay_count, S_IRUGO,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%s\n", adev->product_name);
+ return sysfs_emit(buf, "%s\n", adev->product_name);
}
static DEVICE_ATTR(product_name, S_IRUGO,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%s\n", adev->product_number);
+ return sysfs_emit(buf, "%s\n", adev->product_number);
}
static DEVICE_ATTR(product_number, S_IRUGO,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%s\n", adev->serial);
+ return sysfs_emit(buf, "%s\n", adev->serial);
}
static DEVICE_ATTR(serial_number, S_IRUGO,
amdgpu_device_get_serial_number, NULL);
/**
- * amdgpu_device_supports_atpx - Is the device a dGPU with HG/PX power control
+ * amdgpu_device_supports_px - Is the device a dGPU with ATPX power control
*
* @dev: drm_device pointer
*
- * Returns true if the device is a dGPU with HG/PX power control,
+ * Returns true if the device is a dGPU with ATPX power control,
* otherwise return false.
*/
-bool amdgpu_device_supports_atpx(struct drm_device *dev)
+bool amdgpu_device_supports_px(struct drm_device *dev)
{
struct amdgpu_device *adev = drm_to_adev(dev);
- if (adev->flags & AMD_IS_PX)
+ if ((adev->flags & AMD_IS_PX) && !amdgpu_is_atpx_hybrid())
return true;
return false;
}
*
* @dev: drm_device pointer
*
- * Returns true if the device is a dGPU with HG/PX power control,
+ * Returns true if the device is a dGPU with ACPI power control,
* otherwise return false.
*/
bool amdgpu_device_supports_boco(struct drm_device *dev)
{
struct amdgpu_device *adev = drm_to_adev(dev);
- if (adev->has_pr3)
+ if (adev->has_pr3 ||
+ ((adev->flags & AMD_IS_PX) && amdgpu_is_atpx_hybrid()))
return true;
return false;
}
/*
* register access helper functions.
*/
+
+/* Check if hw access should be skipped because of hotplug or device error */
+bool amdgpu_device_skip_hw_access(struct amdgpu_device *adev)
+{
+ if (adev->in_pci_err_recovery)
+ return true;
+
+#ifdef CONFIG_LOCKDEP
+ /*
+ * This is a bit complicated to understand, so worth a comment. What we assert
+ * here is that the GPU reset is not running on another thread in parallel.
+ *
+ * For this we trylock the read side of the reset semaphore, if that succeeds
+ * we know that the reset is not running in paralell.
+ *
+ * If the trylock fails we assert that we are either already holding the read
+ * side of the lock or are the reset thread itself and hold the write side of
+ * the lock.
+ */
+ if (in_task()) {
+ if (down_read_trylock(&adev->reset_sem))
+ up_read(&adev->reset_sem);
+ else
+ lockdep_assert_held(&adev->reset_sem);
+ }
+#endif
+ return false;
+}
+
/**
* amdgpu_device_rreg - read a memory mapped IO or indirect register
*
{
uint32_t ret;
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if ((reg * 4) < adev->rmmio_size) {
*/
uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if (offset < adev->rmmio_size)
*/
void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (offset < adev->rmmio_size)
uint32_t reg, uint32_t v,
uint32_t acc_flags)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if ((reg * 4) < adev->rmmio_size) {
void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
uint32_t reg, uint32_t v)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (amdgpu_sriov_fullaccess(adev) &&
adev->gfx.rlc.funcs &&
adev->gfx.rlc.funcs->is_rlcg_access_range) {
if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
- return adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, v);
+ return adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, v, 0);
} else {
writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
}
*/
u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if (index < adev->doorbell.num_doorbells) {
*/
void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (index < adev->doorbell.num_doorbells) {
*/
u64 amdgpu_mm_rdoorbell64(struct amdgpu_device *adev, u32 index)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if (index < adev->doorbell.num_doorbells) {
*/
void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (index < adev->doorbell.num_doorbells) {
struct drm_device *dev = pci_get_drvdata(pdev);
int r;
- if (amdgpu_device_supports_atpx(dev) && state == VGA_SWITCHEROO_OFF)
+ if (amdgpu_device_supports_px(dev) && state == VGA_SWITCHEROO_OFF)
return;
if (state == VGA_SWITCHEROO_ON) {
amdgpu_vf_error_put(adev, AMDGIM_ERROR_VF_ATOMBIOS_INIT_FAIL, 0, 0);
return r;
}
+
+ /*get pf2vf msg info at it's earliest time*/
+ if (amdgpu_sriov_vf(adev))
+ amdgpu_virt_init_data_exchange(adev);
+
}
}
* Returns 0 on success, negative error code on failure.
*/
-static int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
- enum amd_clockgating_state state)
+int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
+ enum amd_clockgating_state state)
{
int i, j, r;
i = state == AMD_CG_STATE_GATE ? j : adev->num_ip_blocks - j - 1;
if (!adev->ip_blocks[i].status.late_initialized)
continue;
+ /* skip CG for GFX on S0ix */
+ if (adev->in_s0ix &&
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX)
+ continue;
/* skip CG for VCE/UVD, it's handled specially */
if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_UVD &&
adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_VCE &&
return 0;
}
-static int amdgpu_device_set_pg_state(struct amdgpu_device *adev, enum amd_powergating_state state)
+int amdgpu_device_set_pg_state(struct amdgpu_device *adev,
+ enum amd_powergating_state state)
{
int i, j, r;
i = state == AMD_PG_STATE_GATE ? j : adev->num_ip_blocks - j - 1;
if (!adev->ip_blocks[i].status.late_initialized)
continue;
+ /* skip PG for GFX on S0ix */
+ if (adev->in_s0ix &&
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX)
+ continue;
/* skip CG for VCE/UVD, it's handled specially */
if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_UVD &&
adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_VCE &&
{
int i, r;
- if (adev->in_poweroff_reboot_com ||
- !amdgpu_acpi_is_s0ix_supported(adev) || amdgpu_in_reset(adev)) {
- amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
- amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
- }
+ amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
+ amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
if (!adev->ip_blocks[i].status.valid)
{
int i, r;
+ if (adev->in_s0ix)
+ amdgpu_gfx_state_change_set(adev, sGpuChangeState_D3Entry);
+
for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
if (!adev->ip_blocks[i].status.valid)
continue;
adev->ip_blocks[i].status.hw = false;
continue;
}
+
+ /* skip suspend of gfx and psp for S0ix
+ * gfx is in gfxoff state, so on resume it will exit gfxoff just
+ * like at runtime. PSP is also part of the always on hardware
+ * so no need to suspend it.
+ */
+ if (adev->in_s0ix &&
+ (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX))
+ continue;
+
/* XXX handle errors */
r = adev->ip_blocks[i].version->funcs->suspend(adev);
/* XXX handle errors */
if (adev->asic_reset_res)
goto fail;
- if (adev->mmhub.funcs && adev->mmhub.funcs->reset_ras_error_count)
- adev->mmhub.funcs->reset_ras_error_count(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->reset_ras_error_count)
+ adev->mmhub.ras_funcs->reset_ras_error_count(adev);
} else {
task_barrier_full(&hive->tb);
struct drm_device *ddev = adev_to_drm(adev);
struct pci_dev *pdev = adev->pdev;
int r, i;
- bool atpx = false;
+ bool px = false;
u32 max_MBps;
adev->shutdown = false;
if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA)
vga_client_register(adev->pdev, adev, NULL, amdgpu_device_vga_set_decode);
- if (amdgpu_device_supports_atpx(ddev))
- atpx = true;
- if (amdgpu_has_atpx() &&
- (amdgpu_is_atpx_hybrid() ||
- amdgpu_has_atpx_dgpu_power_cntl()) &&
- !pci_is_thunderbolt_attached(adev->pdev))
+ if (amdgpu_device_supports_px(ddev)) {
+ px = true;
vga_switcheroo_register_client(adev->pdev,
- &amdgpu_switcheroo_ops, atpx);
- if (atpx)
+ &amdgpu_switcheroo_ops, px);
vga_switcheroo_init_domain_pm_ops(adev->dev, &adev->vga_pm_domain);
+ }
if (amdgpu_emu_mode == 1) {
/* post the asic on emulation mode */
goto fence_driver_init;
}
+ amdgpu_reset_init(adev);
+
/* detect if we are with an SRIOV vbios */
amdgpu_device_detect_sriov_bios(adev);
failed:
amdgpu_vf_error_trans_all(adev);
- if (atpx)
+ if (px)
vga_switcheroo_fini_domain_pm_ops(adev->dev);
failed_unmap:
release_firmware(adev->firmware.gpu_info_fw);
adev->firmware.gpu_info_fw = NULL;
adev->accel_working = false;
+
+ amdgpu_reset_fini(adev);
+
/* free i2c buses */
if (!amdgpu_device_has_dc_support(adev))
amdgpu_i2c_fini(adev);
kfree(adev->bios);
adev->bios = NULL;
- if (amdgpu_has_atpx() &&
- (amdgpu_is_atpx_hybrid() ||
- amdgpu_has_atpx_dgpu_power_cntl()) &&
- !pci_is_thunderbolt_attached(adev->pdev))
+ if (amdgpu_device_supports_px(adev_to_drm(adev))) {
vga_switcheroo_unregister_client(adev->pdev);
- if (amdgpu_device_supports_atpx(adev_to_drm(adev)))
vga_switcheroo_fini_domain_pm_ops(adev->dev);
+ }
if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA)
vga_client_register(adev->pdev, NULL, NULL, NULL);
iounmap(adev->rmmio);
*/
int amdgpu_device_suspend(struct drm_device *dev, bool fbcon)
{
- struct amdgpu_device *adev;
- struct drm_crtc *crtc;
- struct drm_connector *connector;
- struct drm_connector_list_iter iter;
+ struct amdgpu_device *adev = drm_to_adev(dev);
int r;
- adev = drm_to_adev(dev);
-
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
cancel_delayed_work_sync(&adev->delayed_init_work);
- if (!amdgpu_device_has_dc_support(adev)) {
- /* turn off display hw */
- drm_modeset_lock_all(dev);
- drm_connector_list_iter_begin(dev, &iter);
- drm_for_each_connector_iter(connector, &iter)
- drm_helper_connector_dpms(connector,
- DRM_MODE_DPMS_OFF);
- drm_connector_list_iter_end(&iter);
- drm_modeset_unlock_all(dev);
- /* unpin the front buffers and cursors */
- list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
- struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct drm_framebuffer *fb = crtc->primary->fb;
- struct amdgpu_bo *robj;
-
- if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
- struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
- r = amdgpu_bo_reserve(aobj, true);
- if (r == 0) {
- amdgpu_bo_unpin(aobj);
- amdgpu_bo_unreserve(aobj);
- }
- }
-
- if (fb == NULL || fb->obj[0] == NULL) {
- continue;
- }
- robj = gem_to_amdgpu_bo(fb->obj[0]);
- /* don't unpin kernel fb objects */
- if (!amdgpu_fbdev_robj_is_fb(adev, robj)) {
- r = amdgpu_bo_reserve(robj, true);
- if (r == 0) {
- amdgpu_bo_unpin(robj);
- amdgpu_bo_unreserve(robj);
- }
- }
- }
- }
-
amdgpu_ras_suspend(adev);
r = amdgpu_device_ip_suspend_phase1(adev);
- amdgpu_amdkfd_suspend(adev, adev->in_runpm);
+ if (!adev->in_s0ix)
+ amdgpu_amdkfd_suspend(adev, adev->in_runpm);
/* evict vram memory */
amdgpu_bo_evict_vram(adev);
amdgpu_fence_driver_suspend(adev);
- if (adev->in_poweroff_reboot_com ||
- !amdgpu_acpi_is_s0ix_supported(adev) || amdgpu_in_reset(adev))
- r = amdgpu_device_ip_suspend_phase2(adev);
- else
- amdgpu_gfx_state_change_set(adev, sGpuChangeState_D3Entry);
+ r = amdgpu_device_ip_suspend_phase2(adev);
/* evict remaining vram memory
* This second call to evict vram is to evict the gart page table
* using the CPU.
*/
int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
{
- struct drm_connector *connector;
- struct drm_connector_list_iter iter;
struct amdgpu_device *adev = drm_to_adev(dev);
- struct drm_crtc *crtc;
int r = 0;
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
- if (amdgpu_acpi_is_s0ix_supported(adev))
+ if (adev->in_s0ix)
amdgpu_gfx_state_change_set(adev, sGpuChangeState_D0Entry);
/* post card */
queue_delayed_work(system_wq, &adev->delayed_init_work,
msecs_to_jiffies(AMDGPU_RESUME_MS));
- if (!amdgpu_device_has_dc_support(adev)) {
- /* pin cursors */
- list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
- struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
-
- if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
- struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
- r = amdgpu_bo_reserve(aobj, true);
- if (r == 0) {
- r = amdgpu_bo_pin(aobj, AMDGPU_GEM_DOMAIN_VRAM);
- if (r != 0)
- dev_err(adev->dev, "Failed to pin cursor BO (%d)\n", r);
- amdgpu_crtc->cursor_addr = amdgpu_bo_gpu_offset(aobj);
- amdgpu_bo_unreserve(aobj);
- }
- }
- }
+ if (!adev->in_s0ix) {
+ r = amdgpu_amdkfd_resume(adev, adev->in_runpm);
+ if (r)
+ return r;
}
- r = amdgpu_amdkfd_resume(adev, adev->in_runpm);
- if (r)
- return r;
/* Make sure IB tests flushed */
flush_delayed_work(&adev->delayed_init_work);
- /* blat the mode back in */
- if (fbcon) {
- if (!amdgpu_device_has_dc_support(adev)) {
- /* pre DCE11 */
- drm_helper_resume_force_mode(dev);
-
- /* turn on display hw */
- drm_modeset_lock_all(dev);
-
- drm_connector_list_iter_begin(dev, &iter);
- drm_for_each_connector_iter(connector, &iter)
- drm_helper_connector_dpms(connector,
- DRM_MODE_DPMS_ON);
- drm_connector_list_iter_end(&iter);
-
- drm_modeset_unlock_all(dev);
- }
+ if (fbcon)
amdgpu_fbdev_set_suspend(adev, 0);
- }
drm_kms_helper_poll_enable(dev);
amdgpu_amdkfd_post_reset(adev);
error:
- amdgpu_virt_release_full_gpu(adev, true);
if (!r && adev->virt.gim_feature & AMDGIM_FEATURE_GIM_FLR_VRAMLOST) {
amdgpu_inc_vram_lost(adev);
r = amdgpu_device_recover_vram(adev);
}
+ amdgpu_virt_release_full_gpu(adev, true);
return r;
}
case CHIP_SIENNA_CICHLID:
case CHIP_NAVY_FLOUNDER:
case CHIP_DIMGREY_CAVEFISH:
+ case CHIP_VANGOGH:
+ case CHIP_ALDEBARAN:
break;
default:
goto disabled;
}
int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
- struct amdgpu_job *job,
- bool *need_full_reset_arg)
+ struct amdgpu_reset_context *reset_context)
{
int i, r = 0;
- bool need_full_reset = *need_full_reset_arg;
+ struct amdgpu_job *job = NULL;
+ bool need_full_reset =
+ test_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+
+ if (reset_context->reset_req_dev == adev)
+ job = reset_context->job;
/* no need to dump if device is not in good state during probe period */
if (!adev->gmc.xgmi.pending_reset)
if(job)
drm_sched_increase_karma(&job->base);
+ r = amdgpu_reset_prepare_hwcontext(adev, reset_context);
+ /* If reset handler not implemented, continue; otherwise return */
+ if (r == -ENOSYS)
+ r = 0;
+ else
+ return r;
+
/* Don't suspend on bare metal if we are not going to HW reset the ASIC */
if (!amdgpu_sriov_vf(adev)) {
if (need_full_reset)
r = amdgpu_device_ip_suspend(adev);
-
- *need_full_reset_arg = need_full_reset;
+ if (need_full_reset)
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+ else
+ clear_bit(AMDGPU_NEED_FULL_RESET,
+ &reset_context->flags);
}
return r;
}
-int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
- struct list_head *device_list_handle,
- bool *need_full_reset_arg,
- bool skip_hw_reset)
+int amdgpu_do_asic_reset(struct list_head *device_list_handle,
+ struct amdgpu_reset_context *reset_context)
{
struct amdgpu_device *tmp_adev = NULL;
- bool need_full_reset = *need_full_reset_arg, vram_lost = false;
+ bool need_full_reset, skip_hw_reset, vram_lost = false;
int r = 0;
+ /* Try reset handler method first */
+ tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device,
+ reset_list);
+ r = amdgpu_reset_perform_reset(tmp_adev, reset_context);
+ /* If reset handler not implemented, continue; otherwise return */
+ if (r == -ENOSYS)
+ r = 0;
+ else
+ return r;
+
+ /* Reset handler not implemented, use the default method */
+ need_full_reset =
+ test_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+ skip_hw_reset = test_bit(AMDGPU_SKIP_HW_RESET, &reset_context->flags);
+
/*
* ASIC reset has to be done on all XGMI hive nodes ASAP
* to allow proper links negotiation in FW (within 1 sec)
if (!r && amdgpu_ras_intr_triggered()) {
list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
- if (tmp_adev->mmhub.funcs &&
- tmp_adev->mmhub.funcs->reset_ras_error_count)
- tmp_adev->mmhub.funcs->reset_ras_error_count(tmp_adev);
+ if (tmp_adev->mmhub.ras_funcs &&
+ tmp_adev->mmhub.ras_funcs->reset_ras_error_count)
+ tmp_adev->mmhub.ras_funcs->reset_ras_error_count(tmp_adev);
}
amdgpu_ras_intr_cleared();
*/
amdgpu_register_gpu_instance(tmp_adev);
- if (!hive && tmp_adev->gmc.xgmi.num_physical_nodes > 1)
+ if (!reset_context->hive &&
+ tmp_adev->gmc.xgmi.num_physical_nodes > 1)
amdgpu_xgmi_add_device(tmp_adev);
r = amdgpu_device_ip_late_init(tmp_adev);
}
/* Update PSP FW topology after reset */
- if (hive && tmp_adev->gmc.xgmi.num_physical_nodes > 1)
- r = amdgpu_xgmi_update_topology(hive, tmp_adev);
+ if (reset_context->hive &&
+ tmp_adev->gmc.xgmi.num_physical_nodes > 1)
+ r = amdgpu_xgmi_update_topology(
+ reset_context->hive, tmp_adev);
}
}
}
end:
- *need_full_reset_arg = need_full_reset;
+ if (need_full_reset)
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+ else
+ clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
return r;
}
return 0;
}
+void amdgpu_device_recheck_guilty_jobs(
+ struct amdgpu_device *adev, struct list_head *device_list_handle,
+ struct amdgpu_reset_context *reset_context)
+{
+ int i, r = 0;
+
+ for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
+ struct amdgpu_ring *ring = adev->rings[i];
+ int ret = 0;
+ struct drm_sched_job *s_job;
+
+ if (!ring || !ring->sched.thread)
+ continue;
+
+ s_job = list_first_entry_or_null(&ring->sched.pending_list,
+ struct drm_sched_job, list);
+ if (s_job == NULL)
+ continue;
+
+ /* clear job's guilty and depend the folowing step to decide the real one */
+ drm_sched_reset_karma(s_job);
+ drm_sched_resubmit_jobs_ext(&ring->sched, 1);
+
+ ret = dma_fence_wait_timeout(s_job->s_fence->parent, false, ring->sched.timeout);
+ if (ret == 0) { /* timeout */
+ DRM_ERROR("Found the real bad job! ring:%s, job_id:%llx\n",
+ ring->sched.name, s_job->id);
+
+ /* set guilty */
+ drm_sched_increase_karma(s_job);
+retry:
+ /* do hw reset */
+ if (amdgpu_sriov_vf(adev)) {
+ amdgpu_virt_fini_data_exchange(adev);
+ r = amdgpu_device_reset_sriov(adev, false);
+ if (r)
+ adev->asic_reset_res = r;
+ } else {
+ clear_bit(AMDGPU_SKIP_HW_RESET,
+ &reset_context->flags);
+ r = amdgpu_do_asic_reset(device_list_handle,
+ reset_context);
+ if (r && r == -EAGAIN)
+ goto retry;
+ }
+
+ /*
+ * add reset counter so that the following
+ * resubmitted job could flush vmid
+ */
+ atomic_inc(&adev->gpu_reset_counter);
+ continue;
+ }
+
+ /* got the hw fence, signal finished fence */
+ atomic_dec(ring->sched.score);
+ dma_fence_get(&s_job->s_fence->finished);
+ dma_fence_signal(&s_job->s_fence->finished);
+ dma_fence_put(&s_job->s_fence->finished);
+
+ /* remove node from list and free the job */
+ spin_lock(&ring->sched.job_list_lock);
+ list_del_init(&s_job->list);
+ spin_unlock(&ring->sched.job_list_lock);
+ ring->sched.ops->free_job(s_job);
+ }
+}
+
/**
* amdgpu_device_gpu_recover - reset the asic and recover scheduler
*
struct amdgpu_job *job)
{
struct list_head device_list, *device_list_handle = NULL;
- bool need_full_reset = false;
bool job_signaled = false;
struct amdgpu_hive_info *hive = NULL;
struct amdgpu_device *tmp_adev = NULL;
int i, r = 0;
bool need_emergency_restart = false;
bool audio_suspended = false;
+ int tmp_vram_lost_counter;
+ struct amdgpu_reset_context reset_context;
+
+ memset(&reset_context, 0, sizeof(reset_context));
/*
* Special case: RAS triggered and full reset isn't supported
mutex_lock(&hive->hive_lock);
}
+ reset_context.method = AMD_RESET_METHOD_NONE;
+ reset_context.reset_req_dev = adev;
+ reset_context.job = job;
+ reset_context.hive = hive;
+ clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+
/*
* lock the device before we try to operate the linked list
* if didn't get the device lock, don't touch the linked list since
retry: /* Rest of adevs pre asic reset from XGMI hive. */
list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
- r = amdgpu_device_pre_asic_reset(tmp_adev,
- (tmp_adev == adev) ? job : NULL,
- &need_full_reset);
+ r = amdgpu_device_pre_asic_reset(tmp_adev, &reset_context);
/*TODO Should we stop ?*/
if (r) {
dev_err(tmp_adev->dev, "GPU pre asic reset failed with err, %d for drm dev, %s ",
}
}
+ tmp_vram_lost_counter = atomic_read(&((adev)->vram_lost_counter));
/* Actual ASIC resets if needed.*/
/* TODO Implement XGMI hive reset logic for SRIOV */
if (amdgpu_sriov_vf(adev)) {
if (r)
adev->asic_reset_res = r;
} else {
- r = amdgpu_do_asic_reset(hive, device_list_handle, &need_full_reset, false);
+ r = amdgpu_do_asic_reset(device_list_handle, &reset_context);
if (r && r == -EAGAIN)
goto retry;
}
/* Post ASIC reset for all devs .*/
list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
+ /*
+ * Sometimes a later bad compute job can block a good gfx job as gfx
+ * and compute ring share internal GC HW mutually. We add an additional
+ * guilty jobs recheck step to find the real guilty job, it synchronously
+ * submits and pends for the first job being signaled. If it gets timeout,
+ * we identify it as a real guilty job.
+ */
+ if (amdgpu_gpu_recovery == 2 &&
+ !(tmp_vram_lost_counter < atomic_read(&adev->vram_lost_counter)))
+ amdgpu_device_recheck_guilty_jobs(
+ tmp_adev, device_list_handle, &reset_context);
+
for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
struct amdgpu_ring *ring = tmp_adev->rings[i];
struct drm_device *dev = pci_get_drvdata(pdev);
struct amdgpu_device *adev = drm_to_adev(dev);
int r, i;
- bool need_full_reset = true;
+ struct amdgpu_reset_context reset_context;
u32 memsize;
struct list_head device_list;
DRM_INFO("PCI error: slot reset callback!!\n");
+ memset(&reset_context, 0, sizeof(reset_context));
+
INIT_LIST_HEAD(&device_list);
list_add_tail(&adev->reset_list, &device_list);
goto out;
}
+ reset_context.method = AMD_RESET_METHOD_NONE;
+ reset_context.reset_req_dev = adev;
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+ set_bit(AMDGPU_SKIP_HW_RESET, &reset_context.flags);
+
adev->in_pci_err_recovery = true;
- r = amdgpu_device_pre_asic_reset(adev, NULL, &need_full_reset);
+ r = amdgpu_device_pre_asic_reset(adev, &reset_context);
adev->in_pci_err_recovery = false;
if (r)
goto out;
- r = amdgpu_do_asic_reset(NULL, &device_list, &need_full_reset, true);
+ r = amdgpu_do_asic_reset(&device_list, &reset_context);
out:
if (!r) {
return amdgpu_display_get_crtc_scanoutpos(dev, pipe, 0, vpos, hpos,
stime, etime, mode);
}
+
+int amdgpu_display_suspend_helper(struct amdgpu_device *adev)
+{
+ struct drm_device *dev = adev_to_drm(adev);
+ struct drm_crtc *crtc;
+ struct drm_connector *connector;
+ struct drm_connector_list_iter iter;
+ int r;
+
+ /* turn off display hw */
+ drm_modeset_lock_all(dev);
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter)
+ drm_helper_connector_dpms(connector,
+ DRM_MODE_DPMS_OFF);
+ drm_connector_list_iter_end(&iter);
+ drm_modeset_unlock_all(dev);
+ /* unpin the front buffers and cursors */
+ list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
+ struct drm_framebuffer *fb = crtc->primary->fb;
+ struct amdgpu_bo *robj;
+
+ if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
+ struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
+ r = amdgpu_bo_reserve(aobj, true);
+ if (r == 0) {
+ amdgpu_bo_unpin(aobj);
+ amdgpu_bo_unreserve(aobj);
+ }
+ }
+
+ if (fb == NULL || fb->obj[0] == NULL) {
+ continue;
+ }
+ robj = gem_to_amdgpu_bo(fb->obj[0]);
+ /* don't unpin kernel fb objects */
+ if (!amdgpu_fbdev_robj_is_fb(adev, robj)) {
+ r = amdgpu_bo_reserve(robj, true);
+ if (r == 0) {
+ amdgpu_bo_unpin(robj);
+ amdgpu_bo_unreserve(robj);
+ }
+ }
+ }
+ return r;
+}
+
+int amdgpu_display_resume_helper(struct amdgpu_device *adev)
+{
+ struct drm_device *dev = adev_to_drm(adev);
+ struct drm_connector *connector;
+ struct drm_connector_list_iter iter;
+ struct drm_crtc *crtc;
+ int r;
+
+ /* pin cursors */
+ list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
+
+ if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
+ struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
+ r = amdgpu_bo_reserve(aobj, true);
+ if (r == 0) {
+ r = amdgpu_bo_pin(aobj, AMDGPU_GEM_DOMAIN_VRAM);
+ if (r != 0)
+ dev_err(adev->dev, "Failed to pin cursor BO (%d)\n", r);
+ amdgpu_crtc->cursor_addr = amdgpu_bo_gpu_offset(aobj);
+ amdgpu_bo_unreserve(aobj);
+ }
+ }
+ }
+
+ drm_helper_resume_force_mode(dev);
+
+ /* turn on display hw */
+ drm_modeset_lock_all(dev);
+
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter)
+ drm_helper_connector_dpms(connector,
+ DRM_MODE_DPMS_ON);
+ drm_connector_list_iter_end(&iter);
+
+ drm_modeset_unlock_all(dev);
+
+ return 0;
+}
+
const struct drm_format_info *
amdgpu_lookup_format_info(u32 format, uint64_t modifier);
+int amdgpu_display_suspend_helper(struct amdgpu_device *adev);
+int amdgpu_display_resume_helper(struct amdgpu_device *adev);
+
#endif
#include <linux/vga_switcheroo.h>
#include <drm/drm_probe_helper.h>
#include <linux/mmu_notifier.h>
+#include <linux/suspend.h>
#include "amdgpu.h"
#include "amdgpu_irq.h"
#include "amdgpu_ras.h"
#include "amdgpu_xgmi.h"
+#include "amdgpu_reset.h"
/*
* KMS wrapper.
* DOC: gpu_recovery (int)
* Set to enable GPU recovery mechanism (1 = enable, 0 = disable). The default is -1 (auto, disabled except SRIOV).
*/
-MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism, (1 = enable, 0 = disable, -1 = auto)");
+MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism, (2 = advanced tdr mode, 1 = enable, 0 = disable, -1 = auto)");
module_param_named(gpu_recovery, amdgpu_gpu_recovery, int, 0444);
/**
{0x1002, 0x73A3, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
{0x1002, 0x73AB, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
{0x1002, 0x73AE, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
+ {0x1002, 0x73AF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
{0x1002, 0x73BF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
/* Van Gogh */
*/
if (!amdgpu_passthrough(adev))
adev->mp1_state = PP_MP1_STATE_UNLOAD;
- adev->in_poweroff_reboot_com = true;
amdgpu_device_ip_suspend(adev);
- adev->in_poweroff_reboot_com = false;
adev->mp1_state = PP_MP1_STATE_NONE;
}
struct list_head device_list;
struct amdgpu_device *adev;
int i, r;
- bool need_full_reset = true;
+ struct amdgpu_reset_context reset_context;
+
+ memset(&reset_context, 0, sizeof(reset_context));
mutex_lock(&mgpu_info.mutex);
if (mgpu_info.pending_reset == true) {
mgpu_info.pending_reset = true;
mutex_unlock(&mgpu_info.mutex);
+ /* Use a common context, just need to make sure full reset is done */
+ reset_context.method = AMD_RESET_METHOD_NONE;
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+
for (i = 0; i < mgpu_info.num_dgpu; i++) {
adev = mgpu_info.gpu_ins[i].adev;
- r = amdgpu_device_pre_asic_reset(adev, NULL, &need_full_reset);
+ reset_context.reset_req_dev = adev;
+ r = amdgpu_device_pre_asic_reset(adev, &reset_context);
if (r) {
dev_err(adev->dev, "GPU pre asic reset failed with err, %d for drm dev, %s ",
r, adev_to_drm(adev)->unique);
list_for_each_entry(adev, &device_list, reset_list)
amdgpu_unregister_gpu_instance(adev);
- r = amdgpu_do_asic_reset(NULL, &device_list, &need_full_reset, true);
+ /* Use a common context, just need to make sure full reset is done */
+ set_bit(AMDGPU_SKIP_HW_RESET, &reset_context.flags);
+ r = amdgpu_do_asic_reset(&device_list, &reset_context);
+
if (r) {
DRM_ERROR("reinit gpus failure");
return;
return;
}
+static int amdgpu_pmops_prepare(struct device *dev)
+{
+ struct drm_device *drm_dev = dev_get_drvdata(dev);
+
+ /* Return a positive number here so
+ * DPM_FLAG_SMART_SUSPEND works properly
+ */
+ if (amdgpu_device_supports_boco(drm_dev))
+ return pm_runtime_suspended(dev) &&
+ pm_suspend_via_firmware();
+
+ return 0;
+}
+
+static void amdgpu_pmops_complete(struct device *dev)
+{
+ /* nothing to do */
+}
+
static int amdgpu_pmops_suspend(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
+ struct amdgpu_device *adev = drm_to_adev(drm_dev);
+ int r;
- return amdgpu_device_suspend(drm_dev, true);
+ if (amdgpu_acpi_is_s0ix_supported(adev))
+ adev->in_s0ix = true;
+ adev->in_s3 = true;
+ r = amdgpu_device_suspend(drm_dev, true);
+ adev->in_s3 = false;
+
+ return r;
}
static int amdgpu_pmops_resume(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
+ struct amdgpu_device *adev = drm_to_adev(drm_dev);
+ int r;
- return amdgpu_device_resume(drm_dev, true);
+ r = amdgpu_device_resume(drm_dev, true);
+ if (amdgpu_acpi_is_s0ix_supported(adev))
+ adev->in_s0ix = false;
+ return r;
}
static int amdgpu_pmops_freeze(struct device *dev)
struct amdgpu_device *adev = drm_to_adev(drm_dev);
int r;
- adev->in_hibernate = true;
+ adev->in_s4 = true;
r = amdgpu_device_suspend(drm_dev, true);
- adev->in_hibernate = false;
+ adev->in_s4 = false;
if (r)
return r;
return amdgpu_asic_reset(adev);
static int amdgpu_pmops_poweroff(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
- struct amdgpu_device *adev = drm_to_adev(drm_dev);
- int r;
- adev->in_poweroff_reboot_com = true;
- r = amdgpu_device_suspend(drm_dev, true);
- adev->in_poweroff_reboot_com = false;
- return r;
+ return amdgpu_device_suspend(drm_dev, true);
}
static int amdgpu_pmops_restore(struct device *dev)
}
adev->in_runpm = true;
- if (amdgpu_device_supports_atpx(drm_dev))
+ if (amdgpu_device_supports_px(drm_dev))
drm_dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
ret = amdgpu_device_suspend(drm_dev, false);
return ret;
}
- if (amdgpu_device_supports_atpx(drm_dev)) {
+ if (amdgpu_device_supports_px(drm_dev)) {
/* Only need to handle PCI state in the driver for ATPX
* PCI core handles it for _PR3.
*/
- if (!amdgpu_is_atpx_hybrid()) {
- amdgpu_device_cache_pci_state(pdev);
- pci_disable_device(pdev);
- pci_ignore_hotplug(pdev);
- pci_set_power_state(pdev, PCI_D3cold);
- }
+ amdgpu_device_cache_pci_state(pdev);
+ pci_disable_device(pdev);
+ pci_ignore_hotplug(pdev);
+ pci_set_power_state(pdev, PCI_D3cold);
drm_dev->switch_power_state = DRM_SWITCH_POWER_DYNAMIC_OFF;
} else if (amdgpu_device_supports_baco(drm_dev)) {
amdgpu_device_baco_enter(drm_dev);
if (!adev->runpm)
return -EINVAL;
- if (amdgpu_device_supports_atpx(drm_dev)) {
+ if (amdgpu_device_supports_px(drm_dev)) {
drm_dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
/* Only need to handle PCI state in the driver for ATPX
* PCI core handles it for _PR3.
*/
- if (!amdgpu_is_atpx_hybrid()) {
- pci_set_power_state(pdev, PCI_D0);
- amdgpu_device_load_pci_state(pdev);
- ret = pci_enable_device(pdev);
- if (ret)
- return ret;
- }
+ pci_set_power_state(pdev, PCI_D0);
+ amdgpu_device_load_pci_state(pdev);
+ ret = pci_enable_device(pdev);
+ if (ret)
+ return ret;
pci_set_master(pdev);
} else if (amdgpu_device_supports_boco(drm_dev)) {
/* Only need to handle PCI state in the driver for ATPX
amdgpu_device_baco_exit(drm_dev);
}
ret = amdgpu_device_resume(drm_dev, false);
- if (amdgpu_device_supports_atpx(drm_dev))
+ if (amdgpu_device_supports_px(drm_dev))
drm_dev->switch_power_state = DRM_SWITCH_POWER_ON;
adev->in_runpm = false;
return 0;
}
static const struct dev_pm_ops amdgpu_pm_ops = {
+ .prepare = amdgpu_pmops_prepare,
+ .complete = amdgpu_pmops_complete,
.suspend = amdgpu_pmops_suspend,
.resume = amdgpu_pmops_resume,
.freeze = amdgpu_pmops_freeze,
* Helper function for amdgpu_fence_driver_init().
*/
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
- unsigned num_hw_submission)
+ unsigned num_hw_submission,
+ atomic_t *sched_score)
{
struct amdgpu_device *adev = ring->adev;
long timeout;
return -ENOMEM;
/* No need to setup the GPU scheduler for rings that don't need it */
- if (!ring->no_scheduler) {
- switch (ring->funcs->type) {
- case AMDGPU_RING_TYPE_GFX:
- timeout = adev->gfx_timeout;
- break;
- case AMDGPU_RING_TYPE_COMPUTE:
- timeout = adev->compute_timeout;
- break;
- case AMDGPU_RING_TYPE_SDMA:
- timeout = adev->sdma_timeout;
- break;
- default:
- timeout = adev->video_timeout;
- break;
- }
+ if (ring->no_scheduler)
+ return 0;
- r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
- num_hw_submission, amdgpu_job_hang_limit,
- timeout, NULL, ring->name);
- if (r) {
- DRM_ERROR("Failed to create scheduler on ring %s.\n",
- ring->name);
- return r;
- }
+ switch (ring->funcs->type) {
+ case AMDGPU_RING_TYPE_GFX:
+ timeout = adev->gfx_timeout;
+ break;
+ case AMDGPU_RING_TYPE_COMPUTE:
+ timeout = adev->compute_timeout;
+ break;
+ case AMDGPU_RING_TYPE_SDMA:
+ timeout = adev->sdma_timeout;
+ break;
+ default:
+ timeout = adev->video_timeout;
+ break;
+ }
+
+ r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
+ num_hw_submission, amdgpu_job_hang_limit,
+ timeout, sched_score, ring->name);
+ if (r) {
+ DRM_ERROR("Failed to create scheduler on ring %s.\n",
+ ring->name);
+ return r;
}
return 0;
AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
r = amdgpu_bo_create(adev, &bp, &adev->gart.bo);
if (r) {
return r;
struct drm_gem_object **obj)
{
struct amdgpu_bo *bo;
+ struct amdgpu_bo_user *ubo;
struct amdgpu_bo_param bp;
int r;
bp.preferred_domain = initial_domain;
bp.flags = flags;
bp.domain = initial_domain;
- r = amdgpu_bo_create(adev, &bp, &bo);
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
+ r = amdgpu_bo_create_user(adev, &bp, &ubo);
if (r)
return r;
+ bo = &ubo->bo;
*obj = &bo->tbo.base;
(*obj)->funcs = &amdgpu_gem_object_funcs;
ring->eop_gpu_addr = kiq->eop_gpu_addr;
ring->no_scheduler = true;
sprintf(ring->name, "kiq_%d.%d.%d", ring->me, ring->pipe, ring->queue);
- r = amdgpu_ring_init(adev, ring, 1024,
- irq, AMDGPU_CP_KIQ_IRQ_DRIVER0,
- AMDGPU_RING_PRIO_DEFAULT);
+ r = amdgpu_ring_init(adev, ring, 1024, irq, AMDGPU_CP_KIQ_IRQ_DRIVER0,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
dev_warn(adev->dev, "(%d) failed to init kiq ring\n", r);
{
struct amdgpu_kiq *kiq = &adev->gfx.kiq;
struct amdgpu_ring *kiq_ring = &kiq->ring;
- int i;
+ int i, r;
if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues)
return -EINVAL;
+ spin_lock(&adev->gfx.kiq.ring_lock);
if (amdgpu_ring_alloc(kiq_ring, kiq->pmf->unmap_queues_size *
- adev->gfx.num_compute_rings))
+ adev->gfx.num_compute_rings)) {
+ spin_unlock(&adev->gfx.kiq.ring_lock);
return -ENOMEM;
+ }
for (i = 0; i < adev->gfx.num_compute_rings; i++)
kiq->pmf->kiq_unmap_queues(kiq_ring, &adev->gfx.compute_ring[i],
RESET_QUEUES, 0, 0);
+ r = amdgpu_ring_test_helper(kiq_ring);
+ spin_unlock(&adev->gfx.kiq.ring_lock);
- return amdgpu_ring_test_helper(kiq_ring);
+ return r;
}
int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev,
DRM_INFO("kiq ring mec %d pipe %d q %d\n", kiq_ring->me, kiq_ring->pipe,
kiq_ring->queue);
-
+ spin_lock(&adev->gfx.kiq.ring_lock);
r = amdgpu_ring_alloc(kiq_ring, kiq->pmf->map_queues_size *
adev->gfx.num_compute_rings +
kiq->pmf->set_resources_size);
if (r) {
DRM_ERROR("Failed to lock KIQ (%d).\n", r);
+ spin_unlock(&adev->gfx.kiq.ring_lock);
return r;
}
kiq->pmf->kiq_map_queues(kiq_ring, &adev->gfx.compute_ring[i]);
r = amdgpu_ring_test_helper(kiq_ring);
+ spin_unlock(&adev->gfx.kiq.ring_lock);
if (r)
DRM_ERROR("KCQ enable failed\n");
*/
if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) {
kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
- if (adev->gfx.funcs->query_ras_error_count)
- adev->gfx.funcs->query_ras_error_count(adev, err_data);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_count)
+ adev->gfx.ras_funcs->query_ras_error_count(adev, err_data);
amdgpu_ras_reset_gpu(adev);
}
return AMDGPU_RAS_SUCCESS;
struct amdgpu_kiq *kiq = &adev->gfx.kiq;
struct amdgpu_ring *ring = &kiq->ring;
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
BUG_ON(!ring->funcs->emit_rreg);
BUG_ON(!ring->funcs->emit_wreg);
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
spin_lock_irqsave(&kiq->ring_lock, flags);
void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state)
{
- if (is_support_sw_smu(adev)) {
- smu_gfx_state_change_set(&adev->smu, state);
- } else {
- mutex_lock(&adev->pm.mutex);
- if (adev->powerplay.pp_funcs &&
- adev->powerplay.pp_funcs->gfx_state_change_set)
- ((adev)->powerplay.pp_funcs->gfx_state_change_set(
- (adev)->powerplay.pp_handle, state));
- mutex_unlock(&adev->pm.mutex);
- }
+ mutex_lock(&adev->pm.mutex);
+ if (adev->powerplay.pp_funcs &&
+ adev->powerplay.pp_funcs->gfx_state_change_set)
+ ((adev)->powerplay.pp_funcs->gfx_state_change_set(
+ (adev)->powerplay.pp_handle, state));
+ mutex_unlock(&adev->pm.mutex);
}
uint32_t bitmap[4][4];
};
+struct amdgpu_gfx_ras_funcs {
+ int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
+ int (*ras_error_inject)(struct amdgpu_device *adev,
+ void *inject_if);
+ int (*query_ras_error_count)(struct amdgpu_device *adev,
+ void *ras_error_status);
+ void (*reset_ras_error_count)(struct amdgpu_device *adev);
+ void (*query_ras_error_status)(struct amdgpu_device *adev);
+ void (*reset_ras_error_status)(struct amdgpu_device *adev);
+ void (*enable_watchdog_timer)(struct amdgpu_device *adev);
+};
+
struct amdgpu_gfx_funcs {
/* get the gpu clock counter */
uint64_t (*get_gpu_clock_counter)(struct amdgpu_device *adev);
uint32_t *dst);
void (*select_me_pipe_q)(struct amdgpu_device *adev, u32 me, u32 pipe,
u32 queue, u32 vmid);
- int (*ras_error_inject)(struct amdgpu_device *adev, void *inject_if);
- int (*query_ras_error_count) (struct amdgpu_device *adev, void *ras_error_status);
- void (*reset_ras_error_count) (struct amdgpu_device *adev);
void (*init_spm_golden)(struct amdgpu_device *adev);
- void (*query_ras_error_status) (struct amdgpu_device *adev);
- void (*reset_ras_error_status) (struct amdgpu_device *adev);
void (*update_perfmon_mgcg)(struct amdgpu_device *adev, bool enable);
- void (*enable_watchdog_timer)(struct amdgpu_device *adev);
};
struct sq_work {
DECLARE_BITMAP (pipe_reserve_bitmap, AMDGPU_MAX_COMPUTE_QUEUES);
/*ras */
- struct ras_common_if *ras_if;
+ struct ras_common_if *ras_if;
+ const struct amdgpu_gfx_ras_funcs *ras_funcs;
};
#define amdgpu_gfx_get_gpu_clock_counter(adev) (adev)->gfx.funcs->get_gpu_clock_counter((adev))
AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
r = amdgpu_bo_create(adev, &bp, &adev->gmc.pdb0_bo);
if (r)
return r;
{
int r;
- if (adev->umc.funcs && adev->umc.funcs->ras_late_init) {
- r = adev->umc.funcs->ras_late_init(adev);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ras_late_init) {
+ r = adev->umc.ras_funcs->ras_late_init(adev);
if (r)
return r;
}
- if (adev->mmhub.funcs && adev->mmhub.funcs->ras_late_init) {
- r = adev->mmhub.funcs->ras_late_init(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->ras_late_init) {
+ r = adev->mmhub.ras_funcs->ras_late_init(adev);
if (r)
return r;
}
- return amdgpu_xgmi_ras_late_init(adev);
+ if (!adev->gmc.xgmi.connected_to_cpu)
+ adev->gmc.xgmi.ras_funcs = &xgmi_ras_funcs;
+
+ if (adev->gmc.xgmi.ras_funcs &&
+ adev->gmc.xgmi.ras_funcs->ras_late_init) {
+ r = adev->gmc.xgmi.ras_funcs->ras_late_init(adev);
+ if (r)
+ return r;
+ }
+
+ return 0;
}
void amdgpu_gmc_ras_fini(struct amdgpu_device *adev)
{
- amdgpu_umc_ras_fini(adev);
- amdgpu_mmhub_ras_fini(adev);
- amdgpu_xgmi_ras_fini(adev);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ras_fini)
+ adev->umc.ras_funcs->ras_fini(adev);
+
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->ras_fini)
+ amdgpu_mmhub_ras_fini(adev);
+
+ if (adev->gmc.xgmi.ras_funcs &&
+ adev->gmc.xgmi.ras_funcs->ras_fini)
+ adev->gmc.xgmi.ras_funcs->ras_fini(adev);
}
/*
switch (adev->asic_type) {
case CHIP_VEGA10:
case CHIP_VEGA20:
+ case CHIP_ARCTURUS:
case CHIP_ALDEBARAN:
/*
* noretry = 0 will cause kfd page fault tests fail
unsigned int (*get_vbios_fb_size)(struct amdgpu_device *adev);
};
+struct amdgpu_xgmi_ras_funcs {
+ int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
+ int (*query_ras_error_count)(struct amdgpu_device *adev,
+ void *ras_error_status);
+ void (*reset_ras_error_count)(struct amdgpu_device *adev);
+};
+
struct amdgpu_xgmi {
/* from psp */
u64 node_id;
struct ras_common_if *ras_if;
bool connected_to_cpu;
bool pending_reset;
+ const struct amdgpu_xgmi_ras_funcs *ras_funcs;
};
struct amdgpu_gmc {
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- man->size * PAGE_SIZE);
+ return sysfs_emit(buf, "%llu\n", man->size * PAGE_SIZE);
}
/**
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- amdgpu_gtt_mgr_usage(man));
+ return sysfs_emit(buf, "%llu\n", amdgpu_gtt_mgr_usage(man));
}
static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO,
* ack the interrupt if it is there
*/
if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF)) {
- if (adev->nbio.funcs &&
- adev->nbio.funcs->handle_ras_controller_intr_no_bifring)
- adev->nbio.funcs->handle_ras_controller_intr_no_bifring(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring)
+ adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring(adev);
- if (adev->nbio.funcs &&
- adev->nbio.funcs->handle_ras_err_event_athub_intr_no_bifring)
- adev->nbio.funcs->handle_ras_err_event_athub_intr_no_bifring(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring)
+ adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring(adev);
}
return ret;
kfree(src->enabled_types);
src->enabled_types = NULL;
- if (src->data) {
- kfree(src->data);
- kfree(src);
- adev->irq.client[i].sources[j] = NULL;
- }
}
kfree(adev->irq.client[i].sources);
adev->irq.client[i].sources = NULL;
unsigned num_types;
atomic_t *enabled_types;
const struct amdgpu_irq_src_funcs *funcs;
- void *data;
};
struct amdgpu_irq_client {
goto out;
}
- if (amdgpu_device_supports_atpx(dev) &&
+ if (amdgpu_device_supports_px(dev) &&
(amdgpu_runtime_pm != 0)) { /* enable runpm by default for atpx */
adev->runpm = true;
dev_info(adev->dev, "Using ATPX for runtime pm\n");
if (adev->runpm) {
/* only need to skip on ATPX */
- if (amdgpu_device_supports_atpx(dev) &&
- !amdgpu_is_atpx_hybrid())
+ if (amdgpu_device_supports_px(dev))
dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NO_DIRECT_COMPLETE);
+ /* we want direct complete for BOCO */
+ if (amdgpu_device_supports_boco(dev))
+ dev_pm_set_driver_flags(dev->dev, DPM_FLAG_SMART_PREPARE |
+ DPM_FLAG_SMART_SUSPEND |
+ DPM_FLAG_MAY_SKIP_RESUME);
pm_runtime_use_autosuspend(dev->dev);
pm_runtime_set_autosuspend_delay(dev->dev, 5000);
pm_runtime_allow(dev->dev);
dev_info->high_va_offset = AMDGPU_GMC_HOLE_END;
dev_info->high_va_max = AMDGPU_GMC_HOLE_END | vm_size;
}
- dev_info->virtual_address_alignment = max((int)PAGE_SIZE, AMDGPU_GPU_PAGE_SIZE);
+ dev_info->virtual_address_alignment = max_t(u32, PAGE_SIZE, AMDGPU_GPU_PAGE_SIZE);
dev_info->pte_fragment_size = (1 << adev->vm_manager.fragment_size) * AMDGPU_GPU_PAGE_SIZE;
- dev_info->gart_page_size = AMDGPU_GPU_PAGE_SIZE;
+ dev_info->gart_page_size = max_t(u32, PAGE_SIZE, AMDGPU_GPU_PAGE_SIZE);
dev_info->cu_active_number = adev->gfx.cu_info.number;
dev_info->cu_ao_mask = adev->gfx.cu_info.ao_cu_mask;
dev_info->ce_ram_size = adev->gfx.ce_ram_size;
#ifndef __AMDGPU_MMHUB_H__
#define __AMDGPU_MMHUB_H__
-struct amdgpu_mmhub_funcs {
- void (*ras_init)(struct amdgpu_device *adev);
+struct amdgpu_mmhub_ras_funcs {
int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
void (*query_ras_error_count)(struct amdgpu_device *adev,
- void *ras_error_status);
+ void *ras_error_status);
+ void (*query_ras_error_status)(struct amdgpu_device *adev);
void (*reset_ras_error_count)(struct amdgpu_device *adev);
+};
+
+struct amdgpu_mmhub_funcs {
u64 (*get_fb_location)(struct amdgpu_device *adev);
void (*init)(struct amdgpu_device *adev);
int (*gart_enable)(struct amdgpu_device *adev);
uint64_t page_table_base);
void (*update_power_gating)(struct amdgpu_device *adev,
bool enable);
- void (*query_ras_error_status)(struct amdgpu_device *adev);
};
struct amdgpu_mmhub {
struct ras_common_if *ras_if;
const struct amdgpu_mmhub_funcs *funcs;
+ const struct amdgpu_mmhub_ras_funcs *ras_funcs;
};
int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev);
u32 ref_and_mask_sdma7;
};
+struct amdgpu_nbio_ras_funcs {
+ void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev);
+ void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev);
+ int (*init_ras_controller_interrupt)(struct amdgpu_device *adev);
+ int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev);
+ void (*query_ras_error_count)(struct amdgpu_device *adev,
+ void *ras_error_status);
+ int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
+};
+
struct amdgpu_nbio_funcs {
const struct nbio_hdp_flush_reg *hdp_flush_reg;
u32 (*get_hdp_flush_req_offset)(struct amdgpu_device *adev);
void (*ih_control)(struct amdgpu_device *adev);
void (*init_registers)(struct amdgpu_device *adev);
void (*remap_hdp_registers)(struct amdgpu_device *adev);
- void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev);
- void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev);
- int (*init_ras_controller_interrupt)(struct amdgpu_device *adev);
- int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev);
- void (*query_ras_error_count)(struct amdgpu_device *adev,
- void *ras_error_status);
- int (*ras_late_init)(struct amdgpu_device *adev);
void (*enable_aspm)(struct amdgpu_device *adev,
bool enable);
void (*program_aspm)(struct amdgpu_device *adev);
struct amdgpu_irq_src ras_err_event_athub_irq;
struct ras_common_if *ras_if;
const struct amdgpu_nbio_funcs *funcs;
+ const struct amdgpu_nbio_ras_funcs *ras_funcs;
};
int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev);
{
struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev);
struct amdgpu_bo *bo = ttm_to_amdgpu_bo(tbo);
+ struct amdgpu_bo_user *ubo;
if (bo->tbo.pin_count > 0)
amdgpu_bo_subtract_pin_size(bo);
}
amdgpu_bo_unref(&bo->parent);
- kfree(bo->metadata);
+ if (bo->tbo.type == ttm_bo_type_device) {
+ ubo = to_amdgpu_bo_user(bo);
+ kfree(ubo->metadata);
+ }
+
kfree(bo);
}
bp.flags |= AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
if (!*bo_ptr) {
r = amdgpu_bo_create(adev, &bp, bo_ptr);
if (!amdgpu_bo_validate_size(adev, size, bp->domain))
return -ENOMEM;
- *bo_ptr = NULL;
+ BUG_ON(bp->bo_ptr_size < sizeof(struct amdgpu_bo));
- bo = kzalloc(sizeof(struct amdgpu_bo), GFP_KERNEL);
+ *bo_ptr = NULL;
+ bo = kzalloc(bp->bo_ptr_size, GFP_KERNEL);
if (bo == NULL)
return -ENOMEM;
drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size);
AMDGPU_GEM_CREATE_SHADOW;
bp.type = ttm_bo_type_kernel;
bp.resv = bo->tbo.base.resv;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
r = amdgpu_bo_do_create(adev, &bp, &bo->shadow);
if (!r) {
int r;
bp->flags = bp->flags & ~AMDGPU_GEM_CREATE_SHADOW;
+
r = amdgpu_bo_do_create(adev, bp, bo_ptr);
if (r)
return r;
return r;
}
+/**
+ * amdgpu_bo_create_user - create an &amdgpu_bo_user buffer object
+ * @adev: amdgpu device object
+ * @bp: parameters to be used for the buffer object
+ * @ubo_ptr: pointer to the buffer object pointer
+ *
+ * Create a BO to be used by user application;
+ *
+ * Returns:
+ * 0 for success or a negative error code on failure.
+ */
+
+int amdgpu_bo_create_user(struct amdgpu_device *adev,
+ struct amdgpu_bo_param *bp,
+ struct amdgpu_bo_user **ubo_ptr)
+{
+ struct amdgpu_bo *bo_ptr;
+ int r;
+
+ bp->flags = bp->flags & ~AMDGPU_GEM_CREATE_SHADOW;
+ bp->bo_ptr_size = sizeof(struct amdgpu_bo_user);
+ r = amdgpu_bo_do_create(adev, bp, &bo_ptr);
+ if (r)
+ return r;
+
+ *ubo_ptr = to_amdgpu_bo_user(bo_ptr);
+ return r;
+}
/**
* amdgpu_bo_validate - validate an &amdgpu_bo buffer object
* @bo: pointer to the buffer object
{
struct ttm_resource_manager *man;
- /* late 2.6.33 fix IGP hibernate - we need pm ops to do this correct */
-#ifndef CONFIG_HIBERNATION
- if (adev->flags & AMD_IS_APU) {
- /* Useless to evict on IGP chips */
+ if (adev->in_s3 && (adev->flags & AMD_IS_APU)) {
+ /* No need to evict vram on APUs for suspend to ram */
return 0;
}
-#endif
man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
return ttm_resource_manager_evict_all(&adev->mman.bdev, man);
}
}
-/**
- * amdgpu_bo_fbdev_mmap - mmap fbdev memory
- * @bo: &amdgpu_bo buffer object
- * @vma: vma as input from the fbdev mmap method
- *
- * Calls ttm_fbdev_mmap() to mmap fbdev memory if it is backed by a bo.
- *
- * Returns:
- * 0 for success or a negative error code on failure.
- */
-int amdgpu_bo_fbdev_mmap(struct amdgpu_bo *bo,
- struct vm_area_struct *vma)
-{
- if (vma->vm_pgoff != 0)
- return -EACCES;
-
- return ttm_bo_mmap_obj(vma, &bo->tbo);
-}
-
/**
* amdgpu_bo_set_tiling_flags - set tiling flags
* @bo: &amdgpu_bo buffer object
int amdgpu_bo_set_tiling_flags(struct amdgpu_bo *bo, u64 tiling_flags)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+ struct amdgpu_bo_user *ubo;
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
if (adev->family <= AMDGPU_FAMILY_CZ &&
AMDGPU_TILING_GET(tiling_flags, TILE_SPLIT) > 6)
return -EINVAL;
- bo->tiling_flags = tiling_flags;
+ ubo = to_amdgpu_bo_user(bo);
+ ubo->tiling_flags = tiling_flags;
return 0;
}
*/
void amdgpu_bo_get_tiling_flags(struct amdgpu_bo *bo, u64 *tiling_flags)
{
+ struct amdgpu_bo_user *ubo;
+
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
dma_resv_assert_held(bo->tbo.base.resv);
+ ubo = to_amdgpu_bo_user(bo);
if (tiling_flags)
- *tiling_flags = bo->tiling_flags;
+ *tiling_flags = ubo->tiling_flags;
}
/**
int amdgpu_bo_set_metadata (struct amdgpu_bo *bo, void *metadata,
uint32_t metadata_size, uint64_t flags)
{
+ struct amdgpu_bo_user *ubo;
void *buffer;
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
+ ubo = to_amdgpu_bo_user(bo);
if (!metadata_size) {
- if (bo->metadata_size) {
- kfree(bo->metadata);
- bo->metadata = NULL;
- bo->metadata_size = 0;
+ if (ubo->metadata_size) {
+ kfree(ubo->metadata);
+ ubo->metadata = NULL;
+ ubo->metadata_size = 0;
}
return 0;
}
if (buffer == NULL)
return -ENOMEM;
- kfree(bo->metadata);
- bo->metadata_flags = flags;
- bo->metadata = buffer;
- bo->metadata_size = metadata_size;
+ kfree(ubo->metadata);
+ ubo->metadata_flags = flags;
+ ubo->metadata = buffer;
+ ubo->metadata_size = metadata_size;
return 0;
}
size_t buffer_size, uint32_t *metadata_size,
uint64_t *flags)
{
+ struct amdgpu_bo_user *ubo;
+
if (!buffer && !metadata_size)
return -EINVAL;
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
+ ubo = to_amdgpu_bo_user(bo);
if (buffer) {
- if (buffer_size < bo->metadata_size)
+ if (buffer_size < ubo->metadata_size)
return -EINVAL;
- if (bo->metadata_size)
- memcpy(buffer, bo->metadata, bo->metadata_size);
+ if (ubo->metadata_size)
+ memcpy(buffer, ubo->metadata, ubo->metadata_size);
}
if (metadata_size)
- *metadata_size = bo->metadata_size;
+ *metadata_size = ubo->metadata_size;
if (flags)
- *flags = bo->metadata_flags;
+ *flags = ubo->metadata_flags;
return 0;
}
#define AMDGPU_BO_INVALID_OFFSET LONG_MAX
#define AMDGPU_BO_MAX_PLACEMENTS 3
+#define to_amdgpu_bo_user(abo) container_of((abo), struct amdgpu_bo_user, bo)
+
struct amdgpu_bo_param {
unsigned long size;
int byte_align;
+ u32 bo_ptr_size;
u32 domain;
u32 preferred_domain;
u64 flags;
struct ttm_buffer_object tbo;
struct ttm_bo_kmap_obj kmap;
u64 flags;
- u64 tiling_flags;
- u64 metadata_flags;
- void *metadata;
- u32 metadata_size;
unsigned prime_shared_count;
/* per VM structure for page tables and with virtual addresses */
struct amdgpu_vm_bo_base *vm_bo;
struct kgd_mem *kfd_bo;
};
+struct amdgpu_bo_user {
+ struct amdgpu_bo bo;
+ u64 tiling_flags;
+ u64 metadata_flags;
+ void *metadata;
+ u32 metadata_size;
+
+};
+
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
{
return container_of(tbo, struct amdgpu_bo, tbo);
int amdgpu_bo_create_kernel_at(struct amdgpu_device *adev,
uint64_t offset, uint64_t size, uint32_t domain,
struct amdgpu_bo **bo_ptr, void **cpu_addr);
+int amdgpu_bo_create_user(struct amdgpu_device *adev,
+ struct amdgpu_bo_param *bp,
+ struct amdgpu_bo_user **ubo_ptr);
void amdgpu_bo_free_kernel(struct amdgpu_bo **bo, u64 *gpu_addr,
void **cpu_addr);
int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr);
int amdgpu_bo_evict_vram(struct amdgpu_device *adev);
int amdgpu_bo_init(struct amdgpu_device *adev);
void amdgpu_bo_fini(struct amdgpu_device *adev);
-int amdgpu_bo_fbdev_mmap(struct amdgpu_bo *bo,
- struct vm_area_struct *vma);
int amdgpu_bo_set_tiling_flags(struct amdgpu_bo *bo, u64 tiling_flags);
void amdgpu_bo_get_tiling_flags(struct amdgpu_bo *bo, u64 *tiling_flags);
int amdgpu_bo_set_metadata (struct amdgpu_bo *bo, void *metadata,
return ret;
}
+static int psp_boot_config_set(struct amdgpu_device *adev)
+{
+ struct psp_context *psp = &adev->psp;
+ struct psp_gfx_cmd_resp *cmd = psp->cmd;
+
+ if (adev->asic_type != CHIP_SIENNA_CICHLID)
+ return 0;
+
+ memset(cmd, 0, sizeof(struct psp_gfx_cmd_resp));
+
+ cmd->cmd_id = GFX_CMD_ID_BOOT_CFG;
+ cmd->cmd.boot_cfg.sub_cmd = BOOTCFG_CMD_SET;
+ cmd->cmd.boot_cfg.boot_config = BOOT_CONFIG_GECC;
+ cmd->cmd.boot_cfg.boot_config_valid = BOOT_CONFIG_GECC;
+
+ return psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr);
+}
+
static int psp_rl_load(struct amdgpu_device *adev)
{
struct psp_context *psp = &adev->psp;
return ret;
}
+ ret = psp_boot_config_set(adev);
+ if (ret) {
+ DRM_WARN("PSP set boot config@\n");
+ }
+
ret = psp_tmr_init(psp);
if (ret) {
DRM_ERROR("PSP tmr init failed!\n");
if (!ucode->fw || amdgpu_sriov_vf(psp->adev))
return 0;
-
- if (amdgpu_in_reset(adev) && ras && ras->supported &&
- adev->asic_type == CHIP_ARCTURUS) {
+ if ((amdgpu_in_reset(adev) &&
+ ras && ras->supported &&
+ (adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_VEGA20)) ||
+ (adev->in_runpm &&
+ adev->asic_type >= CHIP_NAVI10 &&
+ adev->asic_type <= CHIP_NAVI12)) {
ret = amdgpu_dpm_set_mp1_state(adev, PP_MP1_STATE_UNLOAD);
if (ret) {
DRM_WARN("Failed to set MP1 state prepare for reload\n");
return false;
}
+int psp_load_fw_list(struct psp_context *psp,
+ struct amdgpu_firmware_info **ucode_list, int ucode_count)
+{
+ int ret = 0, i;
+ struct amdgpu_firmware_info *ucode;
+
+ for (i = 0; i < ucode_count; ++i) {
+ ucode = ucode_list[i];
+ psp_print_fw_hdr(psp, ucode);
+ ret = psp_execute_np_fw_load(psp, ucode);
+ if (ret)
+ return ret;
+ }
+ return ret;
+}
+
static int psp_np_fw_load(struct psp_context *psp)
{
int i, ret;
return ret;
}
- return snprintf(buf, PAGE_SIZE, "%x\n", fw_ver);
+ return sysfs_emit(buf, "%x\n", fw_ver);
}
static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev,
const char *chip_name);
int psp_get_fw_attestation_records_addr(struct psp_context *psp,
uint64_t *output_ptr);
+
+int psp_load_fw_list(struct psp_context *psp,
+ struct amdgpu_firmware_info **ucode_list, int ucode_count);
#endif
return false;
}
+static int amdgpu_reserve_page_direct(struct amdgpu_device *adev, uint64_t address)
+{
+ struct ras_err_data err_data = {0, 0, 0, NULL};
+ struct eeprom_table_record err_rec;
+
+ if ((address >= adev->gmc.mc_vram_size) ||
+ (address >= RAS_UMC_INJECT_ADDR_LIMIT)) {
+ dev_warn(adev->dev,
+ "RAS WARN: input address 0x%llx is invalid.\n",
+ address);
+ return -EINVAL;
+ }
+
+ if (amdgpu_ras_check_bad_page(adev, address)) {
+ dev_warn(adev->dev,
+ "RAS WARN: 0x%llx has been marked as bad page!\n",
+ address);
+ return 0;
+ }
+
+ memset(&err_rec, 0x0, sizeof(struct eeprom_table_record));
+
+ err_rec.address = address;
+ err_rec.retired_page = address >> AMDGPU_GPU_PAGE_SHIFT;
+ err_rec.ts = (uint64_t)ktime_get_real_seconds();
+ err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
+
+ err_data.err_addr = &err_rec;
+ err_data.err_addr_cnt = 1;
+
+ if (amdgpu_bad_page_threshold != 0) {
+ amdgpu_ras_add_bad_pages(adev, err_data.err_addr,
+ err_data.err_addr_cnt);
+ amdgpu_ras_save_bad_pages(adev);
+ }
+
+ dev_warn(adev->dev, "WARNING: THIS IS ONLY FOR TEST PURPOSES AND WILL CORRUPT RAS EEPROM\n");
+ dev_warn(adev->dev, "Clear EEPROM:\n");
+ dev_warn(adev->dev, " echo 1 > /sys/kernel/debug/dri/0/ras/ras_eeprom_reset\n");
+
+ return 0;
+}
+
static ssize_t amdgpu_ras_debugfs_read(struct file *f, char __user *buf,
size_t size, loff_t *pos)
{
op = 1;
else if (sscanf(str, "inject %32s %8s", block_name, err) == 2)
op = 2;
+ else if (sscanf(str, "retire_page") == 0)
+ op = 3;
else if (str[0] && str[1] && str[2] && str[3])
/* ascii string, but commands are not matched. */
return -EINVAL;
if (op != -1) {
+
+ if (op == 3) {
+ if (sscanf(str, "%*s %llu", &address) != 1)
+ if (sscanf(str, "%*s 0x%llx", &address) != 1)
+ return -EINVAL;
+
+ data->op = op;
+ data->inject.address = address;
+
+ return 0;
+ }
+
if (amdgpu_ras_find_block_id_by_name(block_name, &block_id))
return -EINVAL;
if (ret)
return -EINVAL;
+ if (data.op == 3)
+ {
+ ret = amdgpu_reserve_page_direct(adev, data.inject.address);
+
+ if (ret)
+ return size;
+ else
+ return ret;
+ }
+
if (!amdgpu_ras_is_supported(adev, data.head.block))
return -EINVAL;
};
if (!amdgpu_ras_get_error_query_ready(obj->adev))
- return snprintf(buf, PAGE_SIZE,
- "Query currently inaccessible\n");
+ return sysfs_emit(buf, "Query currently inaccessible\n");
if (amdgpu_ras_query_error_status(obj->adev, &info))
return -EINVAL;
- return snprintf(buf, PAGE_SIZE, "%s: %lu\n%s: %lu\n",
- "ue", info.ue_count,
- "ce", info.ce_count);
+ return sysfs_emit(buf, "%s: %lu\n%s: %lu\n", "ue", info.ue_count,
+ "ce", info.ce_count);
}
/* obj begin */
static inline void put_obj(struct ras_manager *obj)
{
- if (obj && --obj->use == 0)
+ if (obj && (--obj->use == 0))
list_del(&obj->node);
- if (obj && obj->use < 0) {
- DRM_ERROR("RAS ERROR: Unbalance obj(%s) use\n", obj->head.name);
- }
+ if (obj && (obj->use < 0))
+ DRM_ERROR("RAS ERROR: Unbalance obj(%s) use\n", obj->head.name);
}
/* make one obj and return it. */
switch (info->head.block) {
case AMDGPU_RAS_BLOCK__UMC:
- if (adev->umc.funcs->query_ras_error_count)
- adev->umc.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_count)
+ adev->umc.ras_funcs->query_ras_error_count(adev, &err_data);
/* umc query_ras_error_address is also responsible for clearing
* error status
*/
- if (adev->umc.funcs->query_ras_error_address)
- adev->umc.funcs->query_ras_error_address(adev, &err_data);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_address)
+ adev->umc.ras_funcs->query_ras_error_address(adev, &err_data);
break;
case AMDGPU_RAS_BLOCK__SDMA:
if (adev->sdma.funcs->query_ras_error_count) {
}
break;
case AMDGPU_RAS_BLOCK__GFX:
- if (adev->gfx.funcs->query_ras_error_count)
- adev->gfx.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_count)
+ adev->gfx.ras_funcs->query_ras_error_count(adev, &err_data);
- if (adev->gfx.funcs->query_ras_error_status)
- adev->gfx.funcs->query_ras_error_status(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_status)
+ adev->gfx.ras_funcs->query_ras_error_status(adev);
break;
case AMDGPU_RAS_BLOCK__MMHUB:
- if (adev->mmhub.funcs->query_ras_error_count)
- adev->mmhub.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->query_ras_error_count)
+ adev->mmhub.ras_funcs->query_ras_error_count(adev, &err_data);
- if (adev->mmhub.funcs->query_ras_error_status)
- adev->mmhub.funcs->query_ras_error_status(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->query_ras_error_status)
+ adev->mmhub.ras_funcs->query_ras_error_status(adev);
break;
case AMDGPU_RAS_BLOCK__PCIE_BIF:
- if (adev->nbio.funcs->query_ras_error_count)
- adev->nbio.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->query_ras_error_count)
+ adev->nbio.ras_funcs->query_ras_error_count(adev, &err_data);
break;
case AMDGPU_RAS_BLOCK__XGMI_WAFL:
- amdgpu_xgmi_query_ras_error_count(adev, &err_data);
+ if (adev->gmc.xgmi.ras_funcs &&
+ adev->gmc.xgmi.ras_funcs->query_ras_error_count)
+ adev->gmc.xgmi.ras_funcs->query_ras_error_count(adev, &err_data);
break;
default:
break;
switch (block) {
case AMDGPU_RAS_BLOCK__GFX:
- if (adev->gfx.funcs->reset_ras_error_count)
- adev->gfx.funcs->reset_ras_error_count(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->reset_ras_error_count)
+ adev->gfx.ras_funcs->reset_ras_error_count(adev);
- if (adev->gfx.funcs->reset_ras_error_status)
- adev->gfx.funcs->reset_ras_error_status(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->reset_ras_error_status)
+ adev->gfx.ras_funcs->reset_ras_error_status(adev);
break;
case AMDGPU_RAS_BLOCK__MMHUB:
- if (adev->mmhub.funcs->reset_ras_error_count)
- adev->mmhub.funcs->reset_ras_error_count(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->reset_ras_error_count)
+ adev->mmhub.ras_funcs->reset_ras_error_count(adev);
break;
case AMDGPU_RAS_BLOCK__SDMA:
if (adev->sdma.funcs->reset_ras_error_count)
switch (info->head.block) {
case AMDGPU_RAS_BLOCK__GFX:
- if (adev->gfx.funcs->ras_error_inject)
- ret = adev->gfx.funcs->ras_error_inject(adev, info);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->ras_error_inject)
+ ret = adev->gfx.ras_funcs->ras_error_inject(adev, info);
else
ret = -EINVAL;
break;
case AMDGPU_RAS_BLOCK__UMC:
+ case AMDGPU_RAS_BLOCK__SDMA:
case AMDGPU_RAS_BLOCK__MMHUB:
case AMDGPU_RAS_BLOCK__PCIE_BIF:
ret = psp_ras_trigger_error(&adev->psp, &block_info);
*/
switch (info->head.block) {
case AMDGPU_RAS_BLOCK__GFX:
- if (adev->gfx.funcs->query_ras_error_status)
- adev->gfx.funcs->query_ras_error_status(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_status)
+ adev->gfx.ras_funcs->query_ras_error_status(adev);
break;
case AMDGPU_RAS_BLOCK__MMHUB:
- if (adev->mmhub.funcs->query_ras_error_status)
- adev->mmhub.funcs->query_ras_error_status(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->query_ras_error_status)
+ adev->mmhub.ras_funcs->query_ras_error_status(adev);
break;
default:
break;
return 0;
}
-static int amdgpu_ras_check_asic_type(struct amdgpu_device *adev)
+static bool amdgpu_ras_asic_supported(struct amdgpu_device *adev)
{
- if (adev->asic_type != CHIP_VEGA10 &&
- adev->asic_type != CHIP_VEGA20 &&
- adev->asic_type != CHIP_ARCTURUS &&
- adev->asic_type != CHIP_SIENNA_CICHLID)
- return 1;
- else
- return 0;
+ return adev->asic_type == CHIP_VEGA10 ||
+ adev->asic_type == CHIP_VEGA20 ||
+ adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN ||
+ adev->asic_type == CHIP_SIENNA_CICHLID;
}
/*
*supported = 0;
if (amdgpu_sriov_vf(adev) || !adev->is_atom_fw ||
- amdgpu_ras_check_asic_type(adev))
+ !amdgpu_ras_asic_supported(adev))
return;
- if (amdgpu_atomfirmware_mem_ecc_supported(adev)) {
- dev_info(adev->dev, "MEM ECC is active.\n");
- *hw_supported |= (1 << AMDGPU_RAS_BLOCK__UMC |
- 1 << AMDGPU_RAS_BLOCK__DF);
- } else
- dev_info(adev->dev, "MEM ECC is not presented.\n");
+ if (!adev->gmc.xgmi.connected_to_cpu) {
+ if (amdgpu_atomfirmware_mem_ecc_supported(adev)) {
+ dev_info(adev->dev, "MEM ECC is active.\n");
+ *hw_supported |= (1 << AMDGPU_RAS_BLOCK__UMC |
+ 1 << AMDGPU_RAS_BLOCK__DF);
+ } else {
+ dev_info(adev->dev, "MEM ECC is not presented.\n");
+ }
- if (amdgpu_atomfirmware_sram_ecc_supported(adev)) {
- dev_info(adev->dev, "SRAM ECC is active.\n");
- *hw_supported |= ~(1 << AMDGPU_RAS_BLOCK__UMC |
- 1 << AMDGPU_RAS_BLOCK__DF);
- } else
- dev_info(adev->dev, "SRAM ECC is not presented.\n");
+ if (amdgpu_atomfirmware_sram_ecc_supported(adev)) {
+ dev_info(adev->dev, "SRAM ECC is active.\n");
+ *hw_supported |= ~(1 << AMDGPU_RAS_BLOCK__UMC |
+ 1 << AMDGPU_RAS_BLOCK__DF);
+ } else {
+ dev_info(adev->dev, "SRAM ECC is not presented.\n");
+ }
+ } else {
+ /* driver only manages a few IP blocks RAS feature
+ * when GPU is connected cpu through XGMI */
+ *hw_supported |= (1 << AMDGPU_RAS_BLOCK__GFX |
+ 1 << AMDGPU_RAS_BLOCK__SDMA |
+ 1 << AMDGPU_RAS_BLOCK__MMHUB);
+ }
/* hw_supported needs to be aligned with RAS block mask. */
*hw_supported &= AMDGPU_RAS_BLOCK_MASK;
/* Might need get this flag from vbios. */
con->flags = RAS_DEFAULT_FLAGS;
- if (adev->nbio.funcs->init_ras_controller_interrupt) {
- r = adev->nbio.funcs->init_ras_controller_interrupt(adev);
+ /* initialize nbio ras function ahead of any other
+ * ras functions so hardware fatal error interrupt
+ * can be enabled as early as possible */
+ switch (adev->asic_type) {
+ case CHIP_VEGA20:
+ case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
+ if (!adev->gmc.xgmi.connected_to_cpu)
+ adev->nbio.ras_funcs = &nbio_v7_4_ras_funcs;
+ break;
+ default:
+ /* nbio ras is not available */
+ break;
+ }
+
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_controller_interrupt) {
+ r = adev->nbio.ras_funcs->init_ras_controller_interrupt(adev);
if (r)
goto release_con;
}
- if (adev->nbio.funcs->init_ras_err_event_athub_interrupt) {
- r = adev->nbio.funcs->init_ras_err_event_athub_interrupt(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt) {
+ r = adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt(adev);
if (r)
goto release_con;
}
return r;
}
+static int amdgpu_persistent_edc_harvesting_supported(struct amdgpu_device *adev)
+{
+ if (adev->gmc.xgmi.connected_to_cpu)
+ return 1;
+ return 0;
+}
+
+static int amdgpu_persistent_edc_harvesting(struct amdgpu_device *adev,
+ struct ras_common_if *ras_block)
+{
+ struct ras_query_if info = {
+ .head = *ras_block,
+ };
+
+ if (!amdgpu_persistent_edc_harvesting_supported(adev))
+ return 0;
+
+ if (amdgpu_ras_query_error_status(adev, &info) != 0)
+ DRM_WARN("RAS init harvest failure");
+
+ if (amdgpu_ras_reset_error_status(adev, ras_block->block) != 0)
+ DRM_WARN("RAS init harvest reset failure");
+
+ return 0;
+}
+
/* helper function to handle common stuff in ip late init phase */
int amdgpu_ras_late_init(struct amdgpu_device *adev,
struct ras_common_if *ras_block,
return r;
}
+ /* check for errors on warm reset edc persisant supported ASIC */
+ amdgpu_persistent_edc_harvesting(adev, ras_block);
+
/* in resume phase, no need to create ras fs node */
if (adev->in_suspend || amdgpu_in_reset(adev))
return 0;
#define EEPROM_I2C_TARGET_ADDR_ARCTURUS 0xA8
#define EEPROM_I2C_TARGET_ADDR_ARCTURUS_D342 0xA0
#define EEPROM_I2C_TARGET_ADDR_SIENNA_CICHLID 0xA0
+#define EEPROM_I2C_TARGET_ADDR_ALDEBARAN 0xA0
/*
* The 2 macros bellow represent the actual size in bytes that
{
if ((adev->asic_type == CHIP_VEGA20) ||
(adev->asic_type == CHIP_ARCTURUS) ||
- (adev->asic_type == CHIP_SIENNA_CICHLID))
+ (adev->asic_type == CHIP_SIENNA_CICHLID) ||
+ (adev->asic_type == CHIP_ALDEBARAN))
return true;
return false;
*i2c_addr = EEPROM_I2C_TARGET_ADDR_SIENNA_CICHLID;
break;
+ case CHIP_ALDEBARAN:
+ *i2c_addr = EEPROM_I2C_TARGET_ADDR_ALDEBARAN;
+ break;
+
default:
return false;
}
BUG_ON(start + size > res->num_pages << PAGE_SHIFT);
node = res->mm_node;
- while (start > node->size << PAGE_SHIFT)
+ while (start >= node->size << PAGE_SHIFT)
start -= node++->size << PAGE_SHIFT;
cur->start = (node->start << PAGE_SHIFT) + start;
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "amdgpu_reset.h"
+#include "aldebaran.h"
+
+int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_handler *handler)
+{
+ /* TODO: Check if handler exists? */
+ list_add_tail(&handler->handler_list, &reset_ctl->reset_handlers);
+ return 0;
+}
+
+int amdgpu_reset_init(struct amdgpu_device *adev)
+{
+ int ret = 0;
+
+ switch (adev->asic_type) {
+ case CHIP_ALDEBARAN:
+ ret = aldebaran_reset_init(adev);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+int amdgpu_reset_fini(struct amdgpu_device *adev)
+{
+ int ret = 0;
+
+ switch (adev->asic_type) {
+ case CHIP_ALDEBARAN:
+ ret = aldebaran_reset_fini(adev);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+int amdgpu_reset_prepare_hwcontext(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context)
+{
+ struct amdgpu_reset_handler *reset_handler = NULL;
+
+ if (adev->reset_cntl && adev->reset_cntl->get_reset_handler)
+ reset_handler = adev->reset_cntl->get_reset_handler(
+ adev->reset_cntl, reset_context);
+ if (!reset_handler)
+ return -ENOSYS;
+
+ return reset_handler->prepare_hwcontext(adev->reset_cntl,
+ reset_context);
+}
+
+int amdgpu_reset_perform_reset(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context)
+{
+ int ret;
+ struct amdgpu_reset_handler *reset_handler = NULL;
+
+ if (adev->reset_cntl)
+ reset_handler = adev->reset_cntl->get_reset_handler(
+ adev->reset_cntl, reset_context);
+ if (!reset_handler)
+ return -ENOSYS;
+
+ ret = reset_handler->perform_reset(adev->reset_cntl, reset_context);
+ if (ret)
+ return ret;
+
+ return reset_handler->restore_hwcontext(adev->reset_cntl,
+ reset_context);
+}
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_RESET_H__
+#define __AMDGPU_RESET_H__
+
+#include "amdgpu.h"
+
+enum AMDGPU_RESET_FLAGS {
+
+ AMDGPU_NEED_FULL_RESET = 0,
+ AMDGPU_SKIP_HW_RESET = 1,
+};
+
+struct amdgpu_reset_context {
+ enum amd_reset_method method;
+ struct amdgpu_device *reset_req_dev;
+ struct amdgpu_job *job;
+ struct amdgpu_hive_info *hive;
+ unsigned long flags;
+};
+
+struct amdgpu_reset_handler {
+ enum amd_reset_method reset_method;
+ struct list_head handler_list;
+ int (*prepare_env)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*prepare_hwcontext)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*perform_reset)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*restore_hwcontext)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*restore_env)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+
+ int (*do_reset)(struct amdgpu_device *adev);
+};
+
+struct amdgpu_reset_control {
+ void *handle;
+ struct work_struct reset_work;
+ struct mutex reset_lock;
+ struct list_head reset_handlers;
+ atomic_t in_reset;
+ enum amd_reset_method active_reset;
+ struct amdgpu_reset_handler *(*get_reset_handler)(
+ struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ void (*async_reset)(struct work_struct *work);
+};
+
+int amdgpu_reset_init(struct amdgpu_device *adev);
+int amdgpu_reset_fini(struct amdgpu_device *adev);
+
+int amdgpu_reset_prepare_hwcontext(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context);
+
+int amdgpu_reset_perform_reset(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context);
+
+int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_handler *handler);
+
+#endif
*/
int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
unsigned int max_dw, struct amdgpu_irq_src *irq_src,
- unsigned int irq_type, unsigned int hw_prio)
+ unsigned int irq_type, unsigned int hw_prio,
+ atomic_t *sched_score)
{
int r;
int sched_hw_submission = amdgpu_sched_hw_submission;
ring->adev = adev;
ring->idx = adev->num_rings++;
adev->rings[ring->idx] = ring;
- r = amdgpu_fence_driver_init_ring(ring, sched_hw_submission);
+ r = amdgpu_fence_driver_init_ring(ring, sched_hw_submission,
+ sched_score);
if (r)
return r;
}
void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
- unsigned num_hw_submission);
+ unsigned num_hw_submission,
+ atomic_t *sched_score);
int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring,
struct amdgpu_irq_src *irq_src,
unsigned irq_type);
void amdgpu_ring_undo(struct amdgpu_ring *ring);
int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
unsigned int ring_size, struct amdgpu_irq_src *irq_src,
- unsigned int irq_type, unsigned int prio);
+ unsigned int irq_type, unsigned int prio,
+ atomic_t *sched_score);
void amdgpu_ring_fini(struct amdgpu_ring *ring);
void amdgpu_ring_emit_reg_write_reg_wait_helper(struct amdgpu_ring *ring,
uint32_t reg0, uint32_t val0,
void (*reset)(struct amdgpu_device *adev);
void (*start)(struct amdgpu_device *adev);
void (*update_spm_vmid)(struct amdgpu_device *adev, unsigned vmid);
- void (*rlcg_wreg)(struct amdgpu_device *adev, u32 offset, u32 v);
+ void (*rlcg_wreg)(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag);
+ u32 (*rlcg_rreg)(struct amdgpu_device *adev, u32 offset, u32 flag);
bool (*is_rlcg_access_range)(struct amdgpu_device *adev, uint32_t reg);
};
struct drm_device *dev = adev_to_drm(adev);
uint32_t phy_id;
uint32_t op;
- int i;
char str[64];
- char i2c_output[256];
int ret;
if (*pos || size > sizeof(str) - 1)
ret = psp_securedisplay_invoke(psp, TA_SECUREDISPLAY_COMMAND__SEND_ROI_CRC);
if (!ret) {
if (securedisplay_cmd->status == TA_SECUREDISPLAY_STATUS__SUCCESS) {
- memset(i2c_output, 0, sizeof(i2c_output));
- for (i = 0; i < TA_SECUREDISPLAY_I2C_BUFFER_SIZE; i++)
- sprintf(i2c_output, "%s 0x%X", i2c_output,
- securedisplay_cmd->securedisplay_out_message.send_roi_crc.i2c_buf[i]);
- dev_info(adev->dev, "SECUREDISPLAY: I2C buffer out put is :%s\n", i2c_output);
+ dev_info(adev->dev, "SECUREDISPLAY: I2C buffer out put is: %*ph\n",
+ TA_SECUREDISPLAY_I2C_BUFFER_SIZE,
+ securedisplay_cmd->securedisplay_out_message.send_roi_crc.i2c_buf);
} else {
psp_securedisplay_parse_resp_status(psp, securedisplay_cmd->status);
}
bp.flags = 0;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
r = amdgpu_bo_create(adev, &bp, &vram_obj);
if (r) {
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
struct amdgpu_ttm_tt *gtt = (void *)ttm;
- int r;
-
int write = !(gtt->userflags & AMDGPU_GEM_USERPTR_READONLY);
enum dma_data_direction direction = write ?
DMA_BIDIRECTIONAL : DMA_TO_DEVICE;
+ int r;
/* Allocate an SG array and squash pages into it */
r = sg_alloc_table_from_pages(ttm->sg, ttm->pages, ttm->num_pages, 0,
- ttm->num_pages << PAGE_SHIFT,
+ (u64)ttm->num_pages << PAGE_SHIFT,
GFP_KERNEL);
if (r)
goto release_sg;
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
struct amdgpu_ttm_tt *gtt = (void *)ttm;
-
int write = !(gtt->userflags & AMDGPU_GEM_USERPTR_READONLY);
enum dma_data_direction direction = write ?
DMA_BIDIRECTIONAL : DMA_TO_DEVICE;
/* double check that we don't free the table twice */
- if (!ttm->sg->sgl)
+ if (!ttm->sg || !ttm->sg->sgl)
return;
/* unmap the pages mapped to the device */
struct amdgpu_ttm_tt *gtt = (void *)ttm;
int r;
- if (!gtt->bound)
- return;
-
/* if the pages have userptr pinning then clear that first */
if (gtt->userptr)
amdgpu_ttm_tt_unpin_userptr(bdev, ttm);
+ if (!gtt->bound)
+ return;
+
if (gtt->offset == AMDGPU_BO_INVALID_OFFSET)
return;
memcpy(buf, &value, bytes);
}
} else {
- bytes = cursor.size & 0x3ull;
+ bytes = cursor.size & ~0x3ULL;
amdgpu_device_vram_access(adev, cursor.start,
(uint32_t *)buf, bytes,
write);
}
/* ras init of specific umc version */
- if (adev->umc.funcs && adev->umc.funcs->err_cnt_init)
- adev->umc.funcs->err_cnt_init(adev);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->err_cnt_init)
+ adev->umc.ras_funcs->err_cnt_init(adev);
return 0;
struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
- if (adev->umc.funcs &&
- adev->umc.funcs->query_ras_error_count)
- adev->umc.funcs->query_ras_error_count(adev, ras_error_status);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_count)
+ adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status);
- if (adev->umc.funcs &&
- adev->umc.funcs->query_ras_error_address &&
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_address &&
adev->umc.max_ras_err_cnt_per_query) {
err_data->err_addr =
kcalloc(adev->umc.max_ras_err_cnt_per_query,
/* umc query_ras_error_address is also responsible for clearing
* error status
*/
- adev->umc.funcs->query_ras_error_address(adev, ras_error_status);
+ adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status);
}
/* only uncorrectable error needs gpu reset */
#define LOOP_UMC_CH_INST(ch_inst) for ((ch_inst) = 0; (ch_inst) < adev->umc.channel_inst_num; (ch_inst)++)
#define LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) LOOP_UMC_INST((umc_inst)) LOOP_UMC_CH_INST((ch_inst))
-struct amdgpu_umc_funcs {
+struct amdgpu_umc_ras_funcs {
void (*err_cnt_init)(struct amdgpu_device *adev);
int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
void (*query_ras_error_count)(struct amdgpu_device *adev,
- void *ras_error_status);
+ void *ras_error_status);
void (*query_ras_error_address)(struct amdgpu_device *adev,
void *ras_error_status);
+};
+
+struct amdgpu_umc_funcs {
void (*init_registers)(struct amdgpu_device *adev);
};
struct ras_common_if *ras_if;
const struct amdgpu_umc_funcs *funcs;
+ const struct amdgpu_umc_ras_funcs *ras_funcs;
};
int amdgpu_umc_ras_late_init(struct amdgpu_device *adev);
if ((adev->asic_type == CHIP_POLARIS10 ||
adev->asic_type == CHIP_POLARIS11) &&
(adev->uvd.fw_version < FW_1_66_16))
- DRM_ERROR("POLARIS10/11 UVD firmware version %hu.%hu is too old.\n",
+ DRM_ERROR("POLARIS10/11 UVD firmware version %u.%u is too old.\n",
version_major, version_minor);
} else {
unsigned int enc_major, enc_minor, dec_minor;
void *saved_bo;
struct amdgpu_ring ring_dec;
struct amdgpu_ring ring_enc[AMDGPU_VCN_MAX_ENC_RINGS];
+ atomic_t sched_score;
struct amdgpu_irq_src irq;
struct amdgpu_vcn_reg external;
struct amdgpu_bo *dpg_sram_bo;
((struct amd_sriov_msg_pf2vf_info *)pf2vf_info)->vf2pf_update_interval_ms;
adev->virt.gim_feature =
((struct amd_sriov_msg_pf2vf_info *)pf2vf_info)->feature_flags.all;
+ adev->virt.reg_access =
+ ((struct amd_sriov_msg_pf2vf_info *)pf2vf_info)->reg_access_flags.all;
break;
default:
if (adev->virt.ras_init_done)
amdgpu_virt_add_bad_page(adev, bp_block_offset, bp_block_size);
}
+ } else if (adev->bios != NULL) {
+ adev->virt.fw_reserve.p_pf2vf =
+ (struct amd_sriov_msg_pf2vf_info_header *)
+ (adev->bios + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
+
+ amdgpu_virt_read_pf2vf_data(adev);
+
+ return;
}
if (adev->virt.vf2pf_update_interval_ms != 0) {
AMDGIM_FEATURE_GIM_MM_BW_MGR = 0x8,
/* PP ONE VF MODE in GIM */
AMDGIM_FEATURE_PP_ONE_VF = (1 << 4),
+ /* Indirect Reg Access enabled */
+ AMDGIM_FEATURE_INDIRECT_REG_ACCESS = (1 << 5),
+};
+
+enum AMDGIM_REG_ACCESS_FLAG {
+ /* Use PSP to program IH_RB_CNTL */
+ AMDGIM_FEATURE_IH_REG_PSP_EN = (1 << 0),
+ /* Use RLC to program MMHUB regs */
+ AMDGIM_FEATURE_MMHUB_REG_RLC_EN = (1 << 1),
+ /* Use RLC to program GC regs */
+ AMDGIM_FEATURE_GC_REG_RLC_EN = (1 << 2),
};
struct amdgim_pf2vf_info_v1 {
bool tdr_debug;
struct amdgpu_virt_ras_err_handler_data *virt_eh_data;
bool ras_init_done;
+ uint32_t reg_access;
/* vf2pf message */
struct delayed_work vf2pf_work;
#define amdgpu_sriov_fullaccess(adev) \
(amdgpu_sriov_vf((adev)) && !amdgpu_sriov_runtime((adev)))
+#define amdgpu_sriov_reg_indirect_en(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.gim_feature & (AMDGIM_FEATURE_INDIRECT_REG_ACCESS)))
+
+#define amdgpu_sriov_reg_indirect_ih(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.reg_access & (AMDGIM_FEATURE_IH_REG_PSP_EN)))
+
+#define amdgpu_sriov_reg_indirect_mmhub(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.reg_access & (AMDGIM_FEATURE_MMHUB_REG_RLC_EN)))
+
+#define amdgpu_sriov_reg_indirect_gc(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.reg_access & (AMDGIM_FEATURE_GC_REG_RLC_EN)))
+
#define amdgpu_passthrough(adev) \
((adev)->virt.caps & AMDGPU_PASSTHROUGH_MODE)
bp->domain = amdgpu_bo_get_preferred_pin_domain(adev, bp->domain);
bp->flags = AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
AMDGPU_GEM_CREATE_CPU_GTT_USWC;
+ bp->bo_ptr_size = sizeof(struct amdgpu_bo);
if (vm->use_cpu_for_update)
bp->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
else if (!vm->root.base.bo || vm->root.base.bo->shadow)
uint64_t eaddr;
/* validate the parameters */
- if (saddr & AMDGPU_GPU_PAGE_MASK || offset & AMDGPU_GPU_PAGE_MASK ||
- size == 0 || size & AMDGPU_GPU_PAGE_MASK)
+ if (saddr & ~PAGE_MASK || offset & ~PAGE_MASK ||
+ size == 0 || size & ~PAGE_MASK)
return -EINVAL;
/* make sure object fit at this offset */
int r;
/* validate the parameters */
- if (saddr & AMDGPU_GPU_PAGE_MASK || offset & AMDGPU_GPU_PAGE_MASK ||
- size == 0 || size & AMDGPU_GPU_PAGE_MASK)
+ if (saddr & ~PAGE_MASK || offset & ~PAGE_MASK ||
+ size == 0 || size & ~PAGE_MASK)
return -EINVAL;
/* make sure object fit at this offset */
after->start = eaddr + 1;
after->last = tmp->last;
after->offset = tmp->offset;
- after->offset += after->start - tmp->start;
+ after->offset += (after->start - tmp->start) << PAGE_SHIFT;
after->flags = tmp->flags;
after->bo_va = tmp->bo_va;
list_add(&after->list, &tmp->bo_va->invalids);
struct amdgpu_bo *root;
uint64_t value, flags;
struct amdgpu_vm *vm;
- long r;
+ int r;
spin_lock(&adev->vm_manager.pasid_lock);
vm = idr_find(&adev->vm_manager.pasid_idr, pasid);
value = 0;
}
+ r = dma_resv_reserve_shared(root->tbo.base.resv, 1);
+ if (r) {
+ pr_debug("failed %d to reserve fence slot\n", r);
+ goto error_unlock;
+ }
+
r = amdgpu_vm_bo_update_mapping(adev, adev, vm, true, false, NULL, addr,
addr, flags, value, NULL, NULL,
NULL);
error_unlock:
amdgpu_bo_unreserve(root);
if (r < 0)
- DRM_ERROR("Can't handle page fault (%ld)\n", r);
+ DRM_ERROR("Can't handle page fault (%d)\n", r);
error_unref:
amdgpu_bo_unref(&root);
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", adev->gmc.real_vram_size);
+ return sysfs_emit(buf, "%llu\n", adev->gmc.real_vram_size);
}
/**
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", adev->gmc.visible_vram_size);
+ return sysfs_emit(buf, "%llu\n", adev->gmc.visible_vram_size);
}
/**
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- amdgpu_vram_mgr_usage(man));
+ return sysfs_emit(buf, "%llu\n", amdgpu_vram_mgr_usage(man));
}
/**
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- amdgpu_vram_mgr_vis_usage(man));
+ return sysfs_emit(buf, "%llu\n", amdgpu_vram_mgr_vis_usage(man));
}
static ssize_t amdgpu_mem_info_vram_vendor(struct device *dev,
switch (adev->gmc.vram_vendor) {
case SAMSUNG:
- return snprintf(buf, PAGE_SIZE, "samsung\n");
+ return sysfs_emit(buf, "samsung\n");
case INFINEON:
- return snprintf(buf, PAGE_SIZE, "infineon\n");
+ return sysfs_emit(buf, "infineon\n");
case ELPIDA:
- return snprintf(buf, PAGE_SIZE, "elpida\n");
+ return sysfs_emit(buf, "elpida\n");
case ETRON:
- return snprintf(buf, PAGE_SIZE, "etron\n");
+ return sysfs_emit(buf, "etron\n");
case NANYA:
- return snprintf(buf, PAGE_SIZE, "nanya\n");
+ return sysfs_emit(buf, "nanya\n");
case HYNIX:
- return snprintf(buf, PAGE_SIZE, "hynix\n");
+ return sysfs_emit(buf, "hynix\n");
case MOSEL:
- return snprintf(buf, PAGE_SIZE, "mosel\n");
+ return sysfs_emit(buf, "mosel\n");
case WINBOND:
- return snprintf(buf, PAGE_SIZE, "winbond\n");
+ return sysfs_emit(buf, "winbond\n");
case ESMT:
- return snprintf(buf, PAGE_SIZE, "esmt\n");
+ return sysfs_emit(buf, "esmt\n");
case MICRON:
- return snprintf(buf, PAGE_SIZE, "micron\n");
+ return sysfs_emit(buf, "micron\n");
default:
- return snprintf(buf, PAGE_SIZE, "unknown\n");
+ return sysfs_emit(buf, "unknown\n");
}
}
#include <linux/list.h>
#include "amdgpu.h"
#include "amdgpu_xgmi.h"
-#include "amdgpu_smu.h"
#include "amdgpu_ras.h"
#include "soc15.h"
#include "df/df_3_6_offset.h"
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", adev->gmc.xgmi.node_id);
+ return sysfs_emit(buf, "%llu\n", adev->gmc.xgmi.node_id);
}
adev->df.funcs->set_fica(adev, ficaa_pie_status_in, 0, 0);
- return snprintf(buf, PAGE_SIZE, "%u\n", error_count);
+ return sysfs_emit(buf, "%u\n", error_count);
}
return psp_xgmi_terminate(&adev->psp);
}
-int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev)
+static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev)
{
int r;
struct ras_ih_if ih_info = {
adev->gmc.xgmi.num_physical_nodes == 0)
return 0;
- amdgpu_xgmi_reset_ras_error_count(adev);
+ adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev);
if (!adev->gmc.xgmi.ras_if) {
adev->gmc.xgmi.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
return r;
}
-void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev)
+static void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev)
{
if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL) &&
adev->gmc.xgmi.ras_if) {
WREG32_PCIE(pcs_status_reg, 0);
}
-void amdgpu_xgmi_reset_ras_error_count(struct amdgpu_device *adev)
+static void amdgpu_xgmi_reset_ras_error_count(struct amdgpu_device *adev)
{
uint32_t i;
return 0;
}
-int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status)
+static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
{
struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
int i;
break;
}
- amdgpu_xgmi_reset_ras_error_count(adev);
+ adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev);
err_data->ue_count += ue_cnt;
err_data->ce_count += ce_cnt;
return 0;
}
+
+const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs = {
+ .ras_late_init = amdgpu_xgmi_ras_late_init,
+ .ras_fini = amdgpu_xgmi_ras_fini,
+ .query_ras_error_count = amdgpu_xgmi_query_ras_error_count,
+ .reset_ras_error_count = amdgpu_xgmi_reset_ras_error_count,
+};
uint32_t pcs_err_shift;
};
+extern const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs;
struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev);
void amdgpu_put_xgmi_hive(struct amdgpu_hive_info *hive);
int amdgpu_xgmi_update_topology(struct amdgpu_hive_info *hive, struct amdgpu_device *adev);
int amdgpu_xgmi_set_pstate(struct amdgpu_device *adev, int pstate);
int amdgpu_xgmi_get_hops_count(struct amdgpu_device *adev,
struct amdgpu_device *peer_adev);
-int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev);
-void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev);
uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev,
uint64_t addr);
-int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status);
-void amdgpu_xgmi_reset_ras_error_count(struct amdgpu_device *adev);
-
static inline bool amdgpu_xgmi_same_hive(struct amdgpu_device *adev,
struct amdgpu_device *bo_adev)
{
uint32_t host_flr_vramlost : 1;
uint32_t mm_bw_management : 1;
uint32_t pp_one_vf_mode : 1;
- uint32_t reserved : 27;
+ uint32_t reg_indirect_acc : 1;
+ uint32_t reserved : 26;
} flags;
uint32_t all;
};
+union amd_sriov_reg_access_flags {
+ struct {
+ uint32_t vf_reg_access_ih : 1;
+ uint32_t vf_reg_access_mmhub : 1;
+ uint32_t vf_reg_access_gc : 1;
+ uint32_t reserved : 29;
+ } flags;
+ uint32_t all;
+};
+
union amd_sriov_msg_os_info {
struct {
uint32_t windows : 1;
/* identification in ROCm SMI */
uint64_t uuid;
uint32_t fcn_idx;
+ /* flags which indicate the register access method VF should use */
+ union amd_sriov_reg_access_flags reg_access_flags;
/* reserved */
- uint32_t reserved[256-26];
+ uint32_t reserved[256-27];
};
struct amd_sriov_msg_vf2pf_info_header {
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
&adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
static int dce_v10_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v10_0_is_idle(void *handle)
static int dce_v11_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v11_0_is_idle(void *handle)
static int dce_v6_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v6_0_is_idle(void *handle)
static int dce_v8_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v8_0_is_idle(void *handle)
#include "dce_v11_0.h"
#include "dce_virtual.h"
#include "ivsrcid/ivsrcid_vislands30.h"
+#include "amdgpu_display.h"
#define DCE_VIRTUAL_VBLANK_PERIOD 16666666
static int dce_virtual_suspend(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
return dce_virtual_hw_fini(handle);
}
static int dce_virtual_resume(void *handle)
{
- return dce_virtual_hw_init(handle);
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = dce_virtual_hw_init(handle);
+ if (r)
+ return r;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_virtual_is_idle(void *handle)
count++;
}
- return snprintf(buf, PAGE_SIZE, "%i\n", count);
+ return sysfs_emit(buf, "%i\n", count);
}
/* device attr for available perfmon counters */
#include "amdgpu.h"
#include "amdgpu_gfx.h"
#include "amdgpu_psp.h"
-#include "amdgpu_smu.h"
#include "nv.h"
#include "nvd.h"
#define mmGC_THROTTLE_CTRL_Sienna_Cichlid 0x2030
#define mmGC_THROTTLE_CTRL_Sienna_Cichlid_BASE_IDX 0
+#define GFX_RLCG_GC_WRITE_OLD (0x8 << 28)
+#define GFX_RLCG_GC_WRITE (0x0 << 28)
+#define GFX_RLCG_GC_READ (0x1 << 28)
+#define GFX_RLCG_MMHUB_WRITE (0x2 << 28)
+
MODULE_FIRMWARE("amdgpu/navi10_ce.bin");
MODULE_FIRMWARE("amdgpu/navi10_pfp.bin");
MODULE_FIRMWARE("amdgpu/navi10_me.bin");
SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00800000)
};
-static void gfx_v10_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 v)
+static bool gfx_v10_is_rlcg_rw(struct amdgpu_device *adev, u32 offset, uint32_t *flag, bool write)
+{
+ /* always programed by rlcg, only for gc */
+ if (offset == SOC15_REG_OFFSET(GC, 0, mmRLC_CSIB_ADDR_HI) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmRLC_CSIB_ADDR_LO) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmRLC_CSIB_LENGTH) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmCP_ME_CNTL)) {
+ if (!amdgpu_sriov_reg_indirect_gc(adev))
+ *flag = GFX_RLCG_GC_WRITE_OLD;
+ else
+ *flag = write ? GFX_RLCG_GC_WRITE : GFX_RLCG_GC_READ;
+
+ return true;
+ }
+
+ /* currently support gc read/write, mmhub write */
+ if (offset >= SOC15_REG_OFFSET(GC, 0, mmSDMA0_DEC_START) &&
+ offset <= SOC15_REG_OFFSET(GC, 0, mmRLC_GTS_OFFSET_MSB)) {
+ if (amdgpu_sriov_reg_indirect_gc(adev))
+ *flag = write ? GFX_RLCG_GC_WRITE : GFX_RLCG_GC_READ;
+ else
+ return false;
+ } else {
+ if (amdgpu_sriov_reg_indirect_mmhub(adev))
+ *flag = GFX_RLCG_MMHUB_WRITE;
+ else
+ return false;
+ }
+
+ return true;
+}
+
+static u32 gfx_v10_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag)
{
static void *scratch_reg0;
static void *scratch_reg1;
+ static void *scratch_reg2;
+ static void *scratch_reg3;
static void *spare_int;
+ static uint32_t grbm_cntl;
+ static uint32_t grbm_idx;
uint32_t i = 0;
uint32_t retries = 50000;
+ u32 ret = 0;
+
+ scratch_reg0 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0) * 4;
+ scratch_reg1 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1) * 4;
+ scratch_reg2 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG2) * 4;
+ scratch_reg3 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3) * 4;
+ spare_int = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT) * 4;
+
+ grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL;
+ grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX;
+
+ if (offset == grbm_cntl || offset == grbm_idx) {
+ if (offset == grbm_cntl)
+ writel(v, scratch_reg2);
+ else if (offset == grbm_idx)
+ writel(v, scratch_reg3);
+
+ writel(v, ((void __iomem *)adev->rmmio) + (offset * 4));
+ } else {
+ writel(v, scratch_reg0);
+ writel(offset | flag, scratch_reg1);
+ writel(1, spare_int);
+ for (i = 0; i < retries; i++) {
+ u32 tmp;
+
+ tmp = readl(scratch_reg1);
+ if (!(tmp & flag))
+ break;
- scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4;
- scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4;
- spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4;
+ udelay(10);
+ }
- if (amdgpu_sriov_runtime(adev)) {
- pr_err("shouldn't call rlcg write register during runtime\n");
- return;
+ if (i >= retries)
+ pr_err("timeout: rlcg program reg:0x%05x failed !\n", offset);
}
- writel(v, scratch_reg0);
- writel(offset | 0x80000000, scratch_reg1);
- writel(1, spare_int);
- for (i = 0; i < retries; i++) {
- u32 tmp;
+ ret = readl(scratch_reg0);
- tmp = readl(scratch_reg1);
- if (!(tmp & 0x80000000))
- break;
+ return ret;
+}
+
+static void gfx_v10_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 value, u32 flag)
+{
+ uint32_t rlcg_flag;
- udelay(10);
+ if (amdgpu_sriov_fullaccess(adev) &&
+ gfx_v10_is_rlcg_rw(adev, offset, &rlcg_flag, 1)) {
+ gfx_v10_rlcg_rw(adev, offset, value, rlcg_flag);
+
+ return;
}
+ if (flag & AMDGPU_REGS_NO_KIQ)
+ WREG32_NO_KIQ(offset, value);
+ else
+ WREG32(offset, value);
+}
- if (i >= retries)
- pr_err("timeout: rlcg program reg:0x%05x failed !\n", offset);
+static u32 gfx_v10_rlcg_rreg(struct amdgpu_device *adev, u32 offset, u32 flag)
+{
+ uint32_t rlcg_flag;
+
+ if (amdgpu_sriov_fullaccess(adev) &&
+ gfx_v10_is_rlcg_rw(adev, offset, &rlcg_flag, 0))
+ return gfx_v10_rlcg_rw(adev, offset, 0, rlcg_flag);
+
+ if (flag & AMDGPU_REGS_NO_KIQ)
+ return RREG32_NO_KIQ(offset);
+ else
+ return RREG32(offset);
+
+ return 0;
}
static const struct soc15_reg_golden golden_settings_gc_10_1_nv14[] =
sprintf(ring->name, "gfx_%d.%d.%d", ring->me, ring->pipe, ring->queue);
irq_type = AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP + ring->pipe;
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type,
- AMDGPU_RING_PRIO_DEFAULT);
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
return 0;
hw_prio = amdgpu_gfx_is_high_priority_compute_queue(adev, ring) ?
AMDGPU_GFX_PIPE_PRIO_HIGH : AMDGPU_GFX_PIPE_PRIO_NORMAL;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type, hw_prio);
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ hw_prio, NULL);
if (r)
return r;
* loaded firstly, so in direct type, it has to load smc ucode
* here before rlc.
*/
- if (adev->smu.ppt_funcs != NULL && !(adev->flags & AMD_IS_APU)) {
- r = smu_load_microcode(&adev->smu);
+ if (!(adev->flags & AMD_IS_APU)) {
+ r = amdgpu_pm_load_smu_firmware(adev, NULL);
if (r)
return r;
-
- r = smu_check_fw_status(&adev->smu);
- if (r) {
- pr_err("SMC firmware status is not correct\n");
- return r;
- }
}
gfx_v10_0_disable_gpa_mode(adev);
}
.start = gfx_v10_0_rlc_start,
.update_spm_vmid = gfx_v10_0_update_spm_vmid,
.rlcg_wreg = gfx_v10_rlcg_wreg,
+ .rlcg_rreg = gfx_v10_rlcg_rreg,
.is_rlcg_access_range = gfx_v10_0_is_rlcg_access_range,
};
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
irq_type = AMDGPU_CP_IRQ_COMPUTE_MEC1_PIPE0_EOP + ring->pipe;
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq, irq_type,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
mutex_unlock(&adev->srbm_mutex);
/* Initialize all compute VMIDs to have no GDS, GWS, or OA
- acccess. These should be enabled by FW for target VMIDs. */
+ access. These should be enabled by FW for target VMIDs. */
for (i = adev->vm_manager.first_kfd_vmid; i < AMDGPU_NUM_VMID; i++) {
WREG32(amdgpu_gds_reg_offset[i].mem_base, 0);
WREG32(amdgpu_gds_reg_offset[i].mem_size, 0);
* @adev: amdgpu_device pointer
*
* Set up the number and offset of the CP scratch registers.
- * NOTE: use of CP scratch registers is a legacy inferface and
+ * NOTE: use of CP scratch registers is a legacy interface and
* is not used by default on newer asics (r6xx+). On newer asics,
* memory buffers are used for fences rather than scratch regs.
*/
* @seq: sequence number
* @flags: fence related flags
*
- * Emits a fence sequnce number on the gfx ring and flushes
+ * Emits a fence sequence number on the gfx ring and flushes
* GPU caches.
*/
static void gfx_v7_0_ring_emit_fence_gfx(struct amdgpu_ring *ring, u64 addr,
* @seq: sequence number
* @flags: fence related flags
*
- * Emits a fence sequnce number on the compute ring and flushes
+ * Emits a fence sequence number on the compute ring and flushes
* GPU caches.
*/
static void gfx_v7_0_ring_emit_fence_compute(struct amdgpu_ring *ring,
* gfx_v7_0_ring_emit_ib - emit an IB (Indirect Buffer) on the ring
*
* @ring: amdgpu_ring structure holding ring information
- * @job: job to retrive vmid from
+ * @job: job to retrieve vmid from
* @ib: amdgpu indirect buffer object
* @flags: options (AMDGPU_HAVE_CTX_SWITCH)
*
* Emits an DE (drawing engine) or CE (constant engine) IB
* on the gfx ring. IBs are usually generated by userspace
* acceleration drivers and submitted to the kernel for
- * sheduling on the ring. This function schedules the IB
+ * scheduling on the ring. This function schedules the IB
* on the gfx ring for execution by the GPU.
*/
static void gfx_v7_0_ring_emit_ib_gfx(struct amdgpu_ring *ring,
/*
* CP.
- * On CIK, gfx and compute now have independant command processors.
+ * On CIK, gfx and compute now have independent command processors.
*
* GFX
* Gfx consists of a single ring and can process both gfx jobs and
ring->wptr = 0;
WREG32(mmCP_RB0_WPTR, lower_32_bits(ring->wptr));
- /* set the wb address wether it's enabled or not */
+ /* set the wb address whether it's enabled or not */
rptr_addr = adev->wb.gpu_addr + (ring->rptr_offs * 4);
WREG32(mmCP_RB0_RPTR_ADDR, lower_32_bits(rptr_addr));
WREG32(mmCP_RB0_RPTR_ADDR_HI, upper_32_bits(rptr_addr) & 0xFF);
mqd->cp_hqd_pq_wptr_poll_addr_lo = wb_gpu_addr & 0xfffffffc;
mqd->cp_hqd_pq_wptr_poll_addr_hi = upper_32_bits(wb_gpu_addr) & 0xffff;
- /* set the wb address wether it's enabled or not */
+ /* set the wb address whether it's enabled or not */
wb_gpu_addr = adev->wb.gpu_addr + (ring->rptr_offs * 4);
mqd->cp_hqd_pq_rptr_report_addr_lo = wb_gpu_addr & 0xfffffffc;
mqd->cp_hqd_pq_rptr_report_addr_hi =
/**
* gfx_v7_0_ring_emit_vm_flush - cik vm flush using the CP
*
- * @ring: the ring to emmit the commands to
+ * @ring: the ring to emit the commands to
*
* Sync the command pipeline with the PFP. E.g. wait for everything
* to be completed.
amdgpu_ring_write(ring, 4); /* poll interval */
if (usepfp) {
- /* synce CE with ME to prevent CE fetch CEIB before context switch done */
+ /* sync CE with ME to prevent CE fetch CEIB before context switch done */
amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
amdgpu_ring_write(ring, 0);
amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
/* type-2 packets are deprecated on MEC, use type-3 instead */
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq, irq_type,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
hw_prio = amdgpu_gfx_is_high_priority_compute_queue(adev, ring) ?
AMDGPU_GFX_PIPE_PRIO_HIGH : AMDGPU_RING_PRIO_DEFAULT;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type, hw_prio);
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ hw_prio, NULL);
if (r)
return r;
r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0,
};
-static void gfx_v9_0_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 v)
+static void gfx_v9_0_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag)
{
static void *scratch_reg0;
static void *scratch_reg1;
}
+static void gfx_v9_0_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag)
+{
+ if (amdgpu_sriov_fullaccess(adev)) {
+ gfx_v9_0_rlcg_rw(adev, offset, v, flag);
+
+ return;
+ }
+
+ if (flag & AMDGPU_REGS_NO_KIQ)
+ WREG32_NO_KIQ(offset, v);
+ else
+ WREG32(offset, v);
+}
+
#define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042
#define VEGA12_GB_ADDR_CONFIG_GOLDEN 0x24104041
#define RAVEN_GB_ADDR_CONFIG_GOLDEN 0x24000042
}
static const struct amdgpu_gfx_funcs gfx_v9_0_gfx_funcs = {
- .get_gpu_clock_counter = &gfx_v9_0_get_gpu_clock_counter,
- .select_se_sh = &gfx_v9_0_select_se_sh,
- .read_wave_data = &gfx_v9_0_read_wave_data,
- .read_wave_sgprs = &gfx_v9_0_read_wave_sgprs,
- .read_wave_vgprs = &gfx_v9_0_read_wave_vgprs,
- .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
+ .get_gpu_clock_counter = &gfx_v9_0_get_gpu_clock_counter,
+ .select_se_sh = &gfx_v9_0_select_se_sh,
+ .read_wave_data = &gfx_v9_0_read_wave_data,
+ .read_wave_sgprs = &gfx_v9_0_read_wave_sgprs,
+ .read_wave_vgprs = &gfx_v9_0_read_wave_vgprs,
+ .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
+};
+
+static const struct amdgpu_gfx_ras_funcs gfx_v9_0_ras_funcs = {
+ .ras_late_init = amdgpu_gfx_ras_late_init,
+ .ras_fini = amdgpu_gfx_ras_fini,
.ras_error_inject = &gfx_v9_0_ras_error_inject,
.query_ras_error_count = &gfx_v9_0_query_ras_error_count,
.reset_ras_error_count = &gfx_v9_0_reset_ras_error_count,
};
-static const struct amdgpu_gfx_funcs gfx_v9_4_gfx_funcs = {
- .get_gpu_clock_counter = &gfx_v9_0_get_gpu_clock_counter,
- .select_se_sh = &gfx_v9_0_select_se_sh,
- .read_wave_data = &gfx_v9_0_read_wave_data,
- .read_wave_sgprs = &gfx_v9_0_read_wave_sgprs,
- .read_wave_vgprs = &gfx_v9_0_read_wave_vgprs,
- .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
- .ras_error_inject = &gfx_v9_4_ras_error_inject,
- .query_ras_error_count = &gfx_v9_4_query_ras_error_count,
- .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count,
- .query_ras_error_status = &gfx_v9_4_query_ras_error_status,
-};
-
-static const struct amdgpu_gfx_funcs gfx_v9_4_2_gfx_funcs = {
- .get_gpu_clock_counter = &gfx_v9_0_get_gpu_clock_counter,
- .select_se_sh = &gfx_v9_0_select_se_sh,
- .read_wave_data = &gfx_v9_0_read_wave_data,
- .read_wave_sgprs = &gfx_v9_0_read_wave_sgprs,
- .read_wave_vgprs = &gfx_v9_0_read_wave_vgprs,
- .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
- .ras_error_inject = &gfx_v9_4_2_ras_error_inject,
- .query_ras_error_count = &gfx_v9_4_2_query_ras_error_count,
- .reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count,
- .query_ras_error_status = &gfx_v9_4_2_query_ras_error_status,
- .reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status,
- .enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer,
-};
-
static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)
{
u32 gb_addr_config;
DRM_INFO("fix gfx.config for vega12\n");
break;
case CHIP_VEGA20:
+ adev->gfx.ras_funcs = &gfx_v9_0_ras_funcs;
adev->gfx.config.max_hw_contexts = 8;
adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
gb_addr_config = RAVEN_GB_ADDR_CONFIG_GOLDEN;
break;
case CHIP_ARCTURUS:
- adev->gfx.funcs = &gfx_v9_4_gfx_funcs;
+ adev->gfx.ras_funcs = &gfx_v9_4_ras_funcs;
adev->gfx.config.max_hw_contexts = 8;
adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
gb_addr_config |= 0x22010042;
break;
case CHIP_ALDEBARAN:
- adev->gfx.funcs = &gfx_v9_4_2_gfx_funcs;
+ adev->gfx.ras_funcs = &gfx_v9_4_2_ras_funcs;
adev->gfx.config.max_hw_contexts = 8;
adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
hw_prio = amdgpu_gfx_is_high_priority_compute_queue(adev, ring) ?
AMDGPU_GFX_PIPE_PRIO_HIGH : AMDGPU_GFX_PIPE_PRIO_NORMAL;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- return amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type, hw_prio);
+ return amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ hw_prio, NULL);
}
static int gfx_v9_0_sw_init(void *handle)
sprintf(ring->name, "gfx_%d", i);
ring->use_doorbell = true;
ring->doorbell_index = adev->doorbell_index.gfx_ring0 << 1;
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq,
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
int i;
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_gfx_ras_fini(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->ras_fini)
+ adev->gfx.ras_funcs->ras_fini(adev);
for (i = 0; i < adev->gfx.num_gfx_rings; i++)
amdgpu_ring_fini(&adev->gfx.gfx_ring[i]);
}
gfx_v9_0_cp_enable(adev, false);
- adev->gfx.rlc.funcs->stop(adev);
+ /* Skip suspend with A+A reset */
+ if (adev->gmc.xgmi.connected_to_cpu && amdgpu_in_reset(adev)) {
+ dev_dbg(adev->dev, "Device in reset. Skipping RLC halt\n");
+ return 0;
+ }
+
+ adev->gfx.rlc.funcs->stop(adev);
return 0;
}
if (r)
return r;
- r = amdgpu_gfx_ras_late_init(adev);
- if (r)
- return r;
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->ras_late_init) {
+ r = adev->gfx.ras_funcs->ras_late_init(adev);
+ if (r)
+ return r;
+ }
- if (adev->gfx.funcs->enable_watchdog_timer)
- adev->gfx.funcs->enable_watchdog_timer(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->enable_watchdog_timer)
+ adev->gfx.ras_funcs->enable_watchdog_timer(adev);
return 0;
}
return 0;
}
-int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status)
+static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
{
struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
uint32_t sec_count = 0, ded_count = 0;
return 0;
}
-void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev)
+static void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev)
{
int i, j, k;
WREG32_SOC15(GC, 0, mmATC_L2_CACHE_4K_DSM_INDEX, 255);
}
-int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev, void *inject_if)
+static int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev,
+ void *inject_if)
{
struct ras_inject_if *info = (struct ras_inject_if *)inject_if;
int ret;
static const struct soc15_reg_entry gfx_v9_4_rdrsp_status_regs =
{ SOC15_REG_ENTRY(GC, 0, mmGCEA_ERR_STATUS), 0, 1, 32 };
-void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev)
+static void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev)
{
uint32_t i, j;
uint32_t reg_value;
gfx_v9_4_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
}
+
+const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs = {
+ .ras_late_init = amdgpu_gfx_ras_late_init,
+ .ras_fini = amdgpu_gfx_ras_fini,
+ .ras_error_inject = &gfx_v9_4_ras_error_inject,
+ .query_ras_error_count = &gfx_v9_4_query_ras_error_count,
+ .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count,
+ .query_ras_error_status = &gfx_v9_4_query_ras_error_status,
+};
#ifndef __GFX_V9_4_H__
#define __GFX_V9_4_H__
-void gfx_v9_4_clear_ras_edc_counter(struct amdgpu_device *adev);
-
-int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status);
-
-int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev,
- void *inject_if);
-
-void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev);
-
-void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev);
+extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs;
#endif /* __GFX_V9_4_H__ */
}
gfx_v9_4_2_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
-}
\ No newline at end of file
+}
+
+const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs = {
+ .ras_late_init = amdgpu_gfx_ras_late_init,
+ .ras_fini = amdgpu_gfx_ras_fini,
+ .ras_error_inject = &gfx_v9_4_2_ras_error_inject,
+ .query_ras_error_count = &gfx_v9_4_2_query_ras_error_count,
+ .reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count,
+ .query_ras_error_status = &gfx_v9_4_2_query_ras_error_status,
+ .reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status,
+ .enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer,
+};
uint32_t die_id);
void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev);
-void gfx_v9_4_2_reset_ras_error_count(struct amdgpu_device *adev);
-int gfx_v9_4_2_ras_error_inject(struct amdgpu_device *adev, void *inject_if);
-void gfx_v9_4_2_query_ras_error_status(struct amdgpu_device *adev);
-int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status);
-void gfx_v9_4_2_reset_ras_error_status(struct amdgpu_device *adev);
-void gfx_v9_4_2_enable_watchdog_timer(struct amdgpu_device *adev);
+extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs;
+
#endif /* __GFX_V9_4_2_H__ */
adev->umc.umc_inst_num = UMC_V8_7_UMC_INSTANCE_NUM;
adev->umc.channel_offs = UMC_V8_7_PER_CHANNEL_OFFSET_SIENNA;
adev->umc.channel_idx_tbl = &umc_v8_7_channel_idx_tbl[0][0];
- adev->umc.funcs = &umc_v8_7_funcs;
+ adev->umc.ras_funcs = &umc_v8_7_ras_funcs;
break;
default:
break;
adev->gmc.vm_fault.num_types = 1;
adev->gmc.vm_fault.funcs = &gmc_v9_0_irq_funcs;
- if (!amdgpu_sriov_vf(adev)) {
+ if (!amdgpu_sriov_vf(adev) &&
+ !adev->gmc.xgmi.connected_to_cpu) {
adev->gmc.ecc_irq.num_types = 1;
adev->gmc.ecc_irq.funcs = &gmc_v9_0_ecc_funcs;
}
adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM;
adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_VG20;
adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0];
- adev->umc.funcs = &umc_v6_1_funcs;
+ adev->umc.ras_funcs = &umc_v6_1_ras_funcs;
break;
case CHIP_ARCTURUS:
adev->umc.max_ras_err_cnt_per_query = UMC_V6_1_TOTAL_CHANNEL_NUM;
adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM;
adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_ARCT;
adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0];
- adev->umc.funcs = &umc_v6_1_funcs;
+ adev->umc.ras_funcs = &umc_v6_1_ras_funcs;
break;
default:
break;
}
}
+static void gmc_v9_0_set_mmhub_ras_funcs(struct amdgpu_device *adev)
+{
+ switch (adev->asic_type) {
+ case CHIP_VEGA20:
+ adev->mmhub.ras_funcs = &mmhub_v1_0_ras_funcs;
+ break;
+ case CHIP_ARCTURUS:
+ adev->mmhub.ras_funcs = &mmhub_v9_4_ras_funcs;
+ break;
+ case CHIP_ALDEBARAN:
+ adev->mmhub.ras_funcs = &mmhub_v1_7_ras_funcs;
+ break;
+ default:
+ /* mmhub ras is not available */
+ break;
+ }
+}
+
static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev)
{
adev->gfxhub.funcs = &gfxhub_v1_0_funcs;
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- gmc_v9_0_set_gmc_funcs(adev);
- gmc_v9_0_set_irq_funcs(adev);
- gmc_v9_0_set_umc_funcs(adev);
- gmc_v9_0_set_mmhub_funcs(adev);
- gmc_v9_0_set_gfxhub_funcs(adev);
-
if (adev->asic_type == CHIP_VEGA20 ||
adev->asic_type == CHIP_ARCTURUS)
adev->gmc.xgmi.supported = true;
adev->gmc.xgmi.supported = true;
adev->gmc.xgmi.connected_to_cpu =
adev->smuio.funcs->is_host_gpu_xgmi_supported(adev);
- }
+ }
+
+ gmc_v9_0_set_gmc_funcs(adev);
+ gmc_v9_0_set_irq_funcs(adev);
+ gmc_v9_0_set_umc_funcs(adev);
+ gmc_v9_0_set_mmhub_funcs(adev);
+ gmc_v9_0_set_mmhub_ras_funcs(adev);
+ gmc_v9_0_set_gfxhub_funcs(adev);
adev->gmc.shared_aperture_start = 0x2000000000000000ULL;
adev->gmc.shared_aperture_end =
}
}
- if (adev->mmhub.funcs && adev->mmhub.funcs->reset_ras_error_count)
- adev->mmhub.funcs->reset_ras_error_count(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->reset_ras_error_count)
+ adev->mmhub.ras_funcs->reset_ras_error_count(adev);
r = amdgpu_gmc_ras_late_init(adev);
if (r)
if (r)
return r;
- if (!amdgpu_sriov_vf(adev)) {
+ if (!amdgpu_sriov_vf(adev) &&
+ !adev->gmc.xgmi.connected_to_cpu) {
/* interrupt sent to DF. */
r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DF, 0,
&adev->gmc.ecc_irq);
ring = &adev->jpeg.inst->ring_dec;
sprintf(ring->name, "jpeg_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst->irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1;
sprintf(ring->name, "jpeg_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst->irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1 + 8 * i;
sprintf(ring->name, "jpeg_dec_%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst[i].irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1;
sprintf(ring->name, "jpeg_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring->no_scheduler = true;
sprintf(ring->name, "mes_%d.%d.%d", ring->me, ring->pipe, ring->queue);
- return amdgpu_ring_init(adev, ring, 1024, NULL, 0, AMDGPU_RING_PRIO_DEFAULT);
+ return amdgpu_ring_init(adev, ring, 1024, NULL, 0,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
}
static int mes_v10_1_mqd_sw_init(struct amdgpu_device *adev)
}
}
-const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = {
+const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs = {
.ras_late_init = amdgpu_mmhub_ras_late_init,
+ .ras_fini = amdgpu_mmhub_ras_fini,
.query_ras_error_count = mmhub_v1_0_query_ras_error_count,
.reset_ras_error_count = mmhub_v1_0_reset_ras_error_count,
+};
+
+const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = {
.get_fb_location = mmhub_v1_0_get_fb_location,
.init = mmhub_v1_0_init,
.gart_enable = mmhub_v1_0_gart_enable,
#define __MMHUB_V1_0_H__
extern const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs;
+extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs;
#endif
}
}
-const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs = {
+const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs = {
.ras_late_init = amdgpu_mmhub_ras_late_init,
+ .ras_fini = amdgpu_mmhub_ras_fini,
.query_ras_error_count = mmhub_v1_7_query_ras_error_count,
.reset_ras_error_count = mmhub_v1_7_reset_ras_error_count,
+ .query_ras_error_status = mmhub_v1_7_query_ras_error_status,
+};
+
+const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs = {
.get_fb_location = mmhub_v1_7_get_fb_location,
.init = mmhub_v1_7_init,
.gart_enable = mmhub_v1_7_gart_enable,
.set_clockgating = mmhub_v1_7_set_clockgating,
.get_clockgating = mmhub_v1_7_get_clockgating,
.setup_vm_pt_regs = mmhub_v1_7_setup_vm_pt_regs,
- .query_ras_error_status = mmhub_v1_7_query_ras_error_status,
};
#define __MMHUB_V1_7_H__
extern const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs;
+extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs;
#endif
}
const struct amdgpu_mmhub_funcs mmhub_v2_0_funcs = {
- .ras_late_init = amdgpu_mmhub_ras_late_init,
.init = mmhub_v2_0_init,
.gart_enable = mmhub_v2_0_gart_enable,
.set_fault_enable_default = mmhub_v2_0_set_fault_enable_default,
}
const struct amdgpu_mmhub_funcs mmhub_v2_3_funcs = {
- .ras_late_init = amdgpu_mmhub_ras_late_init,
.init = mmhub_v2_3_init,
.gart_enable = mmhub_v2_3_gart_enable,
.set_fault_enable_default = mmhub_v2_3_set_fault_enable_default,
}
}
-const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = {
+const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs = {
.ras_late_init = amdgpu_mmhub_ras_late_init,
+ .ras_fini = amdgpu_mmhub_ras_fini,
.query_ras_error_count = mmhub_v9_4_query_ras_error_count,
.reset_ras_error_count = mmhub_v9_4_reset_ras_error_count,
+ .query_ras_error_status = mmhub_v9_4_query_ras_error_status,
+};
+
+const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = {
.get_fb_location = mmhub_v9_4_get_fb_location,
.init = mmhub_v9_4_init,
.gart_enable = mmhub_v9_4_gart_enable,
.set_clockgating = mmhub_v9_4_set_clockgating,
.get_clockgating = mmhub_v9_4_get_clockgating,
.setup_vm_pt_regs = mmhub_v9_4_setup_vm_pt_regs,
- .query_ras_error_status = mmhub_v9_4_query_ras_error_status,
};
#define __MMHUB_V9_4_H__
extern const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs;
+extern const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs;
#endif
DOORBELL_INTERRUPT_DISABLE, enable ? 0 : 1);
}
+const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs = {
+ .handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring,
+ .handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring,
+ .init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt,
+ .init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt,
+ .query_ras_error_count = nbio_v7_4_query_ras_error_count,
+ .ras_late_init = amdgpu_nbio_ras_late_init,
+ .ras_fini = amdgpu_nbio_ras_fini,
+};
+
const struct amdgpu_nbio_funcs nbio_v7_4_funcs = {
.get_hdp_flush_req_offset = nbio_v7_4_get_hdp_flush_req_offset,
.get_hdp_flush_done_offset = nbio_v7_4_get_hdp_flush_done_offset,
.ih_control = nbio_v7_4_ih_control,
.init_registers = nbio_v7_4_init_registers,
.remap_hdp_registers = nbio_v7_4_remap_hdp_registers,
- .handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring,
- .handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring,
- .init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt,
- .init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt,
- .query_ras_error_count = nbio_v7_4_query_ras_error_count,
- .ras_late_init = amdgpu_nbio_ras_late_init,
};
extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg;
extern const struct amdgpu_nbio_funcs nbio_v7_4_funcs;
+extern const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs;
#endif
#include "amdgpu_vce.h"
#include "amdgpu_ucode.h"
#include "amdgpu_psp.h"
-#include "amdgpu_smu.h"
#include "atom.h"
#include "amd_pcie.h"
return ret;
}
-static bool nv_asic_supports_baco(struct amdgpu_device *adev)
-{
- struct smu_context *smu = &adev->smu;
-
- if (smu_baco_is_support(smu))
- return true;
- else
- return false;
-}
-
static enum amd_reset_method
nv_asic_reset_method(struct amdgpu_device *adev)
{
- struct smu_context *smu = &adev->smu;
-
if (amdgpu_reset_method == AMD_RESET_METHOD_MODE1 ||
amdgpu_reset_method == AMD_RESET_METHOD_MODE2 ||
amdgpu_reset_method == AMD_RESET_METHOD_BACO ||
case CHIP_DIMGREY_CAVEFISH:
return AMD_RESET_METHOD_MODE1;
default:
- if (smu_baco_is_support(smu))
+ if (amdgpu_dpm_is_baco_supported(adev))
return AMD_RESET_METHOD_BACO;
else
return AMD_RESET_METHOD_MODE1;
static int nv_asic_reset(struct amdgpu_device *adev)
{
int ret = 0;
- struct smu_context *smu = &adev->smu;
-
- /* skip reset on vangogh for now */
- if (adev->asic_type == CHIP_VANGOGH)
- return 0;
switch (nv_asic_reset_method(adev)) {
case AMD_RESET_METHOD_PCI:
break;
case AMD_RESET_METHOD_BACO:
dev_info(adev->dev, "BACO reset\n");
-
- ret = smu_baco_enter(smu);
- if (ret)
- return ret;
- ret = smu_baco_exit(smu);
- if (ret)
- return ret;
+ ret = amdgpu_dpm_baco_reset(adev);
break;
case AMD_RESET_METHOD_MODE2:
dev_info(adev->dev, "MODE2 reset\n");
.need_full_reset = &nv_need_full_reset,
.need_reset_on_init = &nv_need_reset_on_init,
.get_pcie_replay_count = &nv_get_pcie_replay_count,
- .supports_baco = &nv_asic_supports_baco,
+ .supports_baco = &amdgpu_dpm_is_baco_supported,
.pre_asic_init = &nv_pre_asic_init,
.update_umd_stable_pstate = &nv_update_umd_stable_pstate,
.query_video_codecs = &nv_query_video_codecs,
/* IDs upto 0x1F are reserved for older programs (Raven, Vega 10/12/20) */
GFX_CMD_ID_LOAD_TOC = 0x00000020, /* Load TOC and obtain TMR size */
GFX_CMD_ID_AUTOLOAD_RLC = 0x00000021, /* Indicates all graphics fw loaded, start RLC autoload */
+ GFX_CMD_ID_BOOT_CFG = 0x00000022, /* Boot Config */
+};
+
+/* PSP boot config sub-commands */
+enum psp_gfx_boot_config_cmd
+{
+ BOOTCFG_CMD_SET = 1, /* Set boot configuration settings */
+ BOOTCFG_CMD_GET = 2, /* Get boot configuration settings */
+ BOOTCFG_CMD_INVALIDATE = 3 /* Reset current boot configuration settings to VBIOS defaults */
+};
+
+/* PSP boot config bitmask values */
+enum psp_gfx_boot_config
+{
+ BOOT_CONFIG_GECC = 0x1,
};
/* Command to load Trusted Application binary into PSP OS. */
uint32_t toc_size; /* FW buffer size in bytes */
};
+/* Dynamic boot configuration */
+struct psp_gfx_cmd_boot_cfg
+{
+ uint32_t timestamp; /* calendar time as number of seconds */
+ enum psp_gfx_boot_config_cmd sub_cmd; /* sub-command indicating how to process command data */
+ uint32_t boot_config; /* dynamic boot configuration bitmask */
+ uint32_t boot_config_valid; /* dynamic boot configuration valid bits bitmask */
+};
+
/* All GFX ring buffer commands. */
union psp_gfx_commands
{
struct psp_gfx_cmd_reg_prog cmd_setup_reg_prog;
struct psp_gfx_cmd_setup_tmr cmd_setup_vmr;
struct psp_gfx_cmd_load_toc cmd_load_toc;
+ struct psp_gfx_cmd_boot_cfg boot_cfg;
};
struct psp_gfx_uresp_reserved
}
/*
- * save and restore proces
+ * save and restore process
*/
static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops)
{
if (ops & PSP_MEM_TRAIN_SEND_LONG_MSG) {
/*
- * Long traing will encroach certain mount of bottom VRAM,
- * saving the content of this bottom VRAM to system memory
- * before training, and restoring it after training to avoid
+ * Long training will encroach a certain amount on the bottom of VRAM;
+ * save the content from the bottom of VRAM to system memory
+ * before training, and restore it after training to avoid
* VRAM corruption.
*/
sz = GDDR6_MEM_TRAINING_ENCROACHED_SIZE;
ring->ring_obj = NULL;
ring->use_doorbell = false;
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
}
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
AMDGPU_SDMA_IRQ_INSTANCE0 + i,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
r = amdgpu_ring_init(adev, ring, 1024,
&adev->sdma.trap_irq,
AMDGPU_SDMA_IRQ_INSTANCE0 + i,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
: (adev->doorbell_index.sdma_engine[1] << 1); // get DWORD offset
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
(adev->doorbell_index.sdma_engine[i] << 1); //get DWORD offset
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
AMDGPU_SDMA_IRQ_INSTANCE0 + i,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
&adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
#include "smuio_v13_0.h"
#include "dce_virtual.h"
#include "mxgpu_ai.h"
-#include "amdgpu_smu.h"
#include "amdgpu_ras.h"
#include "amdgpu_xgmi.h"
#include <uapi/linux/kfd_ioctl.h>
AMD_CG_SUPPORT_HDP_LS |
AMD_CG_SUPPORT_SDMA_MGCG |
AMD_CG_SUPPORT_SDMA_LS |
- AMD_CG_SUPPORT_IH_CG;
- /*AMD_CG_SUPPORT_VCN_MGCG |AMD_CG_SUPPORT_JPEG_MGCG;*/
+ AMD_CG_SUPPORT_IH_CG |
+ AMD_CG_SUPPORT_VCN_MGCG | AMD_CG_SUPPORT_JPEG_MGCG;
adev->pg_flags = AMD_PG_SUPPORT_VCN_DPG;
adev->external_rev_id = adev->rev_id + 0x3c;
break;
if (adev->hdp.funcs->reset_ras_error_count)
adev->hdp.funcs->reset_ras_error_count(adev);
- if (adev->nbio.funcs->ras_late_init)
- r = adev->nbio.funcs->ras_late_init(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->ras_late_init)
+ r = adev->nbio.ras_funcs->ras_late_init(adev);
return r;
}
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_nbio_ras_fini(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->ras_fini)
+ adev->nbio.ras_funcs->ras_fini(adev);
adev->df.funcs->sw_fini(adev);
return 0;
}
if (adev->nbio.ras_if &&
amdgpu_ras_is_supported(adev, adev->nbio.ras_if->block)) {
- if (adev->nbio.funcs->init_ras_controller_interrupt)
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_controller_interrupt)
amdgpu_irq_put(adev, &adev->nbio.ras_controller_irq, 0);
- if (adev->nbio.funcs->init_ras_err_event_athub_interrupt)
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt)
amdgpu_irq_put(adev, &adev->nbio.ras_err_event_athub_irq, 0);
}
})
#define WREG32_RLC(reg, value) \
- do { \
- if (amdgpu_sriov_fullaccess(adev)) { \
- uint32_t i = 0; \
- uint32_t retries = 50000; \
- uint32_t r0 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0; \
- uint32_t r1 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1; \
- uint32_t spare_int = adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT; \
- WREG32(r0, value); \
- WREG32(r1, (reg | 0x80000000)); \
- WREG32(spare_int, 0x1); \
- for (i = 0; i < retries; i++) { \
- u32 tmp = RREG32(r1); \
- if (!(tmp & 0x80000000)) \
- break; \
- udelay(10); \
- } \
- if (i >= retries) \
- pr_err("timeout: rlcg program reg:0x%05x failed !\n", reg); \
- } else { \
- WREG32(reg, value); \
- } \
+ do { \
+ if (adev->gfx.rlc.funcs->rlcg_wreg) \
+ adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, value, 0); \
+ else \
+ WREG32(reg, value); \
} while (0)
#define WREG32_RLC_EX(prefix, reg, value) \
} while (0)
#define WREG32_SOC15_RLC_SHADOW(ip, inst, reg, value) \
- do { \
- uint32_t target_reg = adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg;\
- if (amdgpu_sriov_fullaccess(adev)) { \
- uint32_t r2 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG2; \
- uint32_t r3 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3; \
- uint32_t grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; \
- uint32_t grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; \
- if (target_reg == grbm_cntl) \
- WREG32(r2, value); \
- else if (target_reg == grbm_idx) \
- WREG32(r3, value); \
- WREG32(target_reg, value); \
- } else { \
- WREG32(target_reg, value); \
- } \
+ WREG32_RLC((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg), value)
+
+#define RREG32_RLC(reg) \
+ (adev->gfx.rlc.funcs->rlcg_rreg ? \
+ adev->gfx.rlc.funcs->rlcg_rreg(adev, reg, 0) : RREG32(reg))
+
+#define WREG32_RLC_NO_KIQ(reg, value) \
+ do { \
+ if (adev->gfx.rlc.funcs->rlcg_wreg) \
+ adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, value, AMDGPU_REGS_NO_KIQ); \
+ else \
+ WREG32_NO_KIQ(reg, value); \
} while (0)
+#define RREG32_RLC_NO_KIQ(reg) \
+ (adev->gfx.rlc.funcs->rlcg_rreg ? \
+ adev->gfx.rlc.funcs->rlcg_rreg(adev, reg, AMDGPU_REGS_NO_KIQ) : RREG32_NO_KIQ(reg))
+
#define WREG32_SOC15_RLC_SHADOW_EX(prefix, ip, inst, reg, value) \
do { \
uint32_t target_reg = adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg;\
} \
} while (0)
+#define RREG32_SOC15_RLC(ip, inst, reg) \
+ RREG32_RLC(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg)
+
#define WREG32_SOC15_RLC(ip, inst, reg, value) \
do { \
- uint32_t target_reg = adev->reg_offset[GC_HWIP][0][reg##_BASE_IDX] + reg;\
- WREG32_RLC(target_reg, value); \
+ uint32_t target_reg = adev->reg_offset[ip##_HWIP][0][reg##_BASE_IDX] + reg;\
+ WREG32_RLC(target_reg, value); \
} while (0)
#define WREG32_SOC15_RLC_EX(prefix, ip, inst, reg, value) \
} while (0)
#define WREG32_FIELD15_RLC(ip, idx, reg, field, val) \
- WREG32_RLC((adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg), \
- (RREG32(adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg) \
- & ~REG_FIELD_MASK(reg, field)) | (val) << REG_FIELD_SHIFT(reg, field))
+ WREG32_RLC((adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg), \
+ (RREG32_RLC(adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg) \
+ & ~REG_FIELD_MASK(reg, field)) | (val) << REG_FIELD_SHIFT(reg, field))
#define WREG32_SOC15_OFFSET_RLC(ip, inst, reg, offset, value) \
- WREG32_RLC(((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) + offset), value)
+ WREG32_RLC(((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) + offset), value)
+
+#define RREG32_SOC15_OFFSET_RLC(ip, inst, reg, offset) \
+ RREG32_RLC(((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) + offset))
#endif
*/
#include "umc_v6_1.h"
#include "amdgpu_ras.h"
+#include "amdgpu_umc.h"
#include "amdgpu.h"
#include "rsmu/rsmu_0_0_2_offset.h"
umc_v6_1_enable_umc_index_mode(adev);
}
-const struct amdgpu_umc_funcs umc_v6_1_funcs = {
+const struct amdgpu_umc_ras_funcs umc_v6_1_ras_funcs = {
.err_cnt_init = umc_v6_1_err_cnt_init,
.ras_late_init = amdgpu_umc_ras_late_init,
+ .ras_fini = amdgpu_umc_ras_fini,
.query_ras_error_count = umc_v6_1_query_ras_error_count,
.query_ras_error_address = umc_v6_1_query_ras_error_address,
};
/* umc ce count initial value */
#define UMC_V6_1_CE_CNT_INIT (UMC_V6_1_CE_CNT_MAX - UMC_V6_1_CE_INT_THRESHOLD)
-extern const struct amdgpu_umc_funcs umc_v6_1_funcs;
+extern const struct amdgpu_umc_ras_funcs umc_v6_1_ras_funcs;
extern const uint32_t
umc_v6_1_channel_idx_tbl[UMC_V6_1_UMC_INSTANCE_NUM][UMC_V6_1_CHANNEL_INSTANCE_NUM];
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "umc_v6_7.h"
+#include "amdgpu_ras.h"
+#include "amdgpu_umc.h"
+#include "amdgpu.h"
+
+#include "umc/umc_6_7_0_offset.h"
+#include "umc/umc_6_7_0_sh_mask.h"
+
+static inline uint32_t get_umc_v6_7_reg_offset(struct amdgpu_device *adev,
+ uint32_t umc_inst,
+ uint32_t ch_inst)
+{
+ return adev->umc.channel_offs * ch_inst + UMC_V6_7_INST_DIST * umc_inst;
+}
+
+static void umc_v6_7_query_correctable_error_count(struct amdgpu_device *adev,
+ uint32_t umc_reg_offset,
+ unsigned long *error_count)
+{
+ uint32_t ecc_err_cnt_sel, ecc_err_cnt_sel_addr;
+ uint32_t ecc_err_cnt, ecc_err_cnt_addr;
+ uint64_t mc_umc_status;
+ uint32_t mc_umc_status_addr;
+
+ /* UMC 6_1_1 registers */
+ ecc_err_cnt_sel_addr =
+ SOC15_REG_OFFSET(UMC, 0, regUMCCH0_0_EccErrCntSel);
+ ecc_err_cnt_addr =
+ SOC15_REG_OFFSET(UMC, 0, regUMCCH0_0_EccErrCnt);
+ mc_umc_status_addr =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
+
+ /* select the lower chip and check the error count */
+ ecc_err_cnt_sel = RREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4);
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel, UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 0);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4, ecc_err_cnt_sel);
+
+ ecc_err_cnt = RREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4);
+ *error_count +=
+ (REG_GET_FIELD(ecc_err_cnt, UMCCH0_0_EccErrCnt, EccErrCnt) -
+ UMC_V6_7_CE_CNT_INIT);
+
+ /* select the higher chip and check the err counter */
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel, UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 1);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4, ecc_err_cnt_sel);
+
+ ecc_err_cnt = RREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4);
+ *error_count +=
+ (REG_GET_FIELD(ecc_err_cnt, UMCCH0_0_EccErrCnt, EccErrCnt) -
+ UMC_V6_7_CE_CNT_INIT);
+
+ /* check for SRAM correctable error
+ MCUMC_STATUS is a 64 bit register */
+ mc_umc_status = RREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4);
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)
+ *error_count += 1;
+}
+
+static void umc_v6_7_querry_uncorrectable_error_count(struct amdgpu_device *adev,
+ uint32_t umc_reg_offset,
+ unsigned long *error_count)
+{
+ uint64_t mc_umc_status;
+ uint32_t mc_umc_status_addr;
+
+ mc_umc_status_addr =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
+
+ /* check the MCUMC_STATUS */
+ mc_umc_status = RREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4);
+ if ((REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1) &&
+ (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1))
+ *error_count += 1;
+}
+
+static void umc_v6_7_reset_error_count_per_channel(struct amdgpu_device *adev,
+ uint32_t umc_reg_offset)
+{
+ uint32_t ecc_err_cnt_addr;
+ uint32_t ecc_err_cnt_sel, ecc_err_cnt_sel_addr;
+
+ ecc_err_cnt_sel_addr =
+ SOC15_REG_OFFSET(UMC, 0,
+ regUMCCH0_0_EccErrCntSel);
+ ecc_err_cnt_addr =
+ SOC15_REG_OFFSET(UMC, 0,
+ regUMCCH0_0_EccErrCnt);
+
+ /* select the lower chip */
+ ecc_err_cnt_sel = RREG32_PCIE((ecc_err_cnt_sel_addr +
+ umc_reg_offset) * 4);
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel,
+ UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 0);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4,
+ ecc_err_cnt_sel);
+
+ /* clear lower chip error count */
+ WREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4,
+ UMC_V6_7_CE_CNT_INIT);
+
+ /* select the higher chip */
+ ecc_err_cnt_sel = RREG32_PCIE((ecc_err_cnt_sel_addr +
+ umc_reg_offset) * 4);
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel,
+ UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 1);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4,
+ ecc_err_cnt_sel);
+
+ /* clear higher chip error count */
+ WREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4,
+ UMC_V6_7_CE_CNT_INIT);
+}
+
+static void umc_v6_7_reset_error_count(struct amdgpu_device *adev)
+{
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+
+ umc_v6_7_reset_error_count_per_channel(adev,
+ umc_reg_offset);
+ }
+}
+
+static void umc_v6_7_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+
+ /*TODO: driver needs to toggle DF Cstate to ensure
+ * safe access of UMC registers. Will add the protection */
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+ umc_v6_7_query_correctable_error_count(adev,
+ umc_reg_offset,
+ &(err_data->ce_count));
+ umc_v6_7_querry_uncorrectable_error_count(adev,
+ umc_reg_offset,
+ &(err_data->ue_count));
+ }
+
+ umc_v6_7_reset_error_count(adev);
+}
+
+static void umc_v6_7_query_error_address(struct amdgpu_device *adev,
+ struct ras_err_data *err_data,
+ uint32_t umc_reg_offset,
+ uint32_t ch_inst,
+ uint32_t umc_inst)
+{
+ uint32_t mc_umc_status_addr;
+ uint64_t mc_umc_status, err_addr, retired_page, mc_umc_addrt0;
+ struct eeprom_table_record *err_rec;
+ uint32_t channel_index;
+
+ mc_umc_status_addr =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
+ mc_umc_addrt0 =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_ADDRT0);
+
+ mc_umc_status = RREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4);
+
+ if (mc_umc_status == 0)
+ return;
+
+ if (!err_data->err_addr) {
+ /* clear umc status */
+ WREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4, 0x0ULL);
+ return;
+ }
+
+ err_rec = &err_data->err_addr[err_data->err_addr_cnt];
+
+ channel_index =
+ adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
+
+ /* calculate error address if ue/ce error is detected */
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
+ (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)) {
+
+ err_addr = RREG64_PCIE((mc_umc_addrt0 + umc_reg_offset) * 4);
+ err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);
+
+ /* translate umc channel address to soc pa, 3 parts are included */
+ retired_page = ADDR_OF_8KB_BLOCK(err_addr) |
+ ADDR_OF_256B_BLOCK(channel_index) |
+ OFFSET_IN_256B_BLOCK(err_addr);
+
+ /* we only save ue error information currently, ce is skipped */
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
+ == 1) {
+ err_rec->address = err_addr;
+ /* page frame address is saved */
+ err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;
+ err_rec->ts = (uint64_t)ktime_get_real_seconds();
+ err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
+ err_rec->cu = 0;
+ err_rec->mem_channel = channel_index;
+ err_rec->mcumc_id = umc_inst;
+
+ err_data->err_addr_cnt++;
+ }
+ }
+
+ /* clear umc status */
+ WREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4, 0x0ULL);
+}
+
+static void umc_v6_7_query_ras_error_address(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+
+ /*TODO: driver needs to toggle DF Cstate to ensure
+ * safe access of UMC resgisters. Will add the protection
+ * when firmware interface is ready */
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+ umc_v6_7_query_error_address(adev,
+ err_data,
+ umc_reg_offset,
+ ch_inst,
+ umc_inst);
+ }
+}
+
+const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs = {
+ .ras_late_init = amdgpu_umc_ras_late_init,
+ .ras_fini = amdgpu_umc_ras_fini,
+ .query_ras_error_count = umc_v6_7_query_ras_error_count,
+ .query_ras_error_address = umc_v6_7_query_ras_error_address,
+};
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef __UMC_V6_7_H__
+#define __UMC_V6_7_H__
+
+/* EccErrCnt max value */
+#define UMC_V6_7_CE_CNT_MAX 0xffff
+/* umc ce interrupt threshold */
+#define UMC_V6_7_CE_INT_THRESHOLD 0xffff
+/* umc ce count initial value */
+#define UMC_V6_7_CE_CNT_INIT (UMC_V6_7_CE_CNT_MAX - UMC_V6_7_CE_INT_THRESHOLD)
+
+#define UMC_V6_7_INST_DIST 0x40000
+
+extern const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs;
+
+#endif
*/
#include "umc_v8_7.h"
#include "amdgpu_ras.h"
+#include "amdgpu_umc.h"
#include "amdgpu.h"
#include "rsmu/rsmu_0_0_2_offset.h"
}
}
-const struct amdgpu_umc_funcs umc_v8_7_funcs = {
+const struct amdgpu_umc_ras_funcs umc_v8_7_ras_funcs = {
.err_cnt_init = umc_v8_7_err_cnt_init,
.ras_late_init = amdgpu_umc_ras_late_init,
+ .ras_fini = amdgpu_umc_ras_fini,
.query_ras_error_count = umc_v8_7_query_ras_error_count,
.query_ras_error_address = umc_v8_7_query_ras_error_address,
};
/* umc ce count initial value */
#define UMC_V8_7_CE_CNT_INIT (UMC_V8_7_CE_CNT_MAX - UMC_V8_7_CE_INT_THRESHOLD)
-extern const struct amdgpu_umc_funcs umc_v8_7_funcs;
+extern const struct amdgpu_umc_ras_funcs umc_v8_7_ras_funcs;
extern const uint32_t
umc_v8_7_channel_idx_tbl[UMC_V8_7_UMC_INSTANCE_NUM][UMC_V8_7_CHANNEL_INSTANCE_NUM];
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
sprintf(ring->name, "uvd_enc%d", i);
r = amdgpu_ring_init(adev, ring, 512,
&adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
sprintf(ring->name, "uvd_%d", ring->me);
r = amdgpu_ring_init(adev, ring, 512,
&adev->uvd.inst[j].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
}
r = amdgpu_ring_init(adev, ring, 512,
&adev->uvd.inst[j].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
for (i = 0; i < adev->vce.num_rings; i++) {
ring = &adev->vce.ring[i];
sprintf(ring->name, "vce%d", i);
- r = amdgpu_ring_init(adev, ring, 512,
- &adev->vce.irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ r = amdgpu_ring_init(adev, ring, 512, &adev->vce.irq, 0,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
ring = &adev->vce.ring[i];
sprintf(ring->name, "vce%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vce.irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
ring->doorbell_index = adev->doorbell_index.uvd_vce.vce_ring2_3 * 2 + 1;
}
r = amdgpu_ring_init(adev, ring, 512, &adev->vce.irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
ring = &adev->vcn.inst->ring_dec;
sprintf(ring->name, "vcn_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring = &adev->vcn.inst->ring_enc[i];
sprintf(ring->name, "vcn_enc%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
sprintf(ring->name, "vcn_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1 + i;
sprintf(ring->name, "vcn_enc%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
(amdgpu_sriov_vf(adev) ? 2*j : 8*j);
sprintf(ring->name, "vcn_dec_%d", j);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst[j].irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
sprintf(ring->name, "vcn_enc_%d.%d", j, i);
r = amdgpu_ring_init(adev, ring, 512,
&adev->vcn.inst[j].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
#define VCN_INSTANCES_SIENNA_CICHLID 2
#define DEC_SW_RING_ENABLED FALSE
+#define RDECODE_MSG_CREATE 0x00000000
+#define RDECODE_MESSAGE_CREATE 0x00000001
+
static int amdgpu_ih_clientid_vcns[] = {
SOC15_IH_CLIENTID_VCN,
SOC15_IH_CLIENTID_VCN1
for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
volatile struct amdgpu_fw_shared *fw_shared;
+
if (adev->vcn.harvest_config & (1 << i))
continue;
if (r)
return r;
+ atomic_set(&adev->vcn.inst[i].sched_score, 0);
+
ring = &adev->vcn.inst[i].ring_dec;
ring->use_doorbell = true;
if (amdgpu_sriov_vf(adev)) {
} else {
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 8 * i;
}
- if (adev->asic_type == CHIP_SIENNA_CICHLID && i != 0)
- ring->no_scheduler = true;
sprintf(ring->name, "vcn_dec_%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst[i].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT,
+ &adev->vcn.inst[i].sched_score);
if (r)
return r;
} else {
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 2 + j + 8 * i;
}
- if (adev->asic_type == CHIP_SIENNA_CICHLID && i != 1)
- ring->no_scheduler = true;
sprintf(ring->name, "vcn_enc_%d.%d", i, j);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst[i].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT,
+ &adev->vcn.inst[i].sched_score);
if (r)
return r;
}
.emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
};
+static int vcn_v3_0_limit_sched(struct amdgpu_cs_parser *p)
+{
+ struct drm_gpu_scheduler **scheds;
+
+ /* The create msg must be in the first IB submitted */
+ if (atomic_read(&p->entity->fence_seq))
+ return -EINVAL;
+
+ scheds = p->adev->gpu_sched[AMDGPU_HW_IP_VCN_DEC]
+ [AMDGPU_RING_PRIO_DEFAULT].sched;
+ drm_sched_entity_modify_sched(p->entity, scheds, 1);
+ return 0;
+}
+
+static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, uint64_t addr)
+{
+ struct ttm_operation_ctx ctx = { false, false };
+ struct amdgpu_bo_va_mapping *map;
+ uint32_t *msg, num_buffers;
+ struct amdgpu_bo *bo;
+ uint64_t start, end;
+ unsigned int i;
+ void * ptr;
+ int r;
+
+ addr &= AMDGPU_GMC_HOLE_MASK;
+ r = amdgpu_cs_find_mapping(p, addr, &bo, &map);
+ if (r) {
+ DRM_ERROR("Can't find BO for addr 0x%08Lx\n", addr);
+ return r;
+ }
+
+ start = map->start * AMDGPU_GPU_PAGE_SIZE;
+ end = (map->last + 1) * AMDGPU_GPU_PAGE_SIZE;
+ if (addr & 0x7) {
+ DRM_ERROR("VCN messages must be 8 byte aligned!\n");
+ return -EINVAL;
+ }
+
+ bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+ amdgpu_bo_placement_from_domain(bo, bo->allowed_domains);
+ r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
+ if (r) {
+ DRM_ERROR("Failed validating the VCN message BO (%d)!\n", r);
+ return r;
+ }
+
+ r = amdgpu_bo_kmap(bo, &ptr);
+ if (r) {
+ DRM_ERROR("Failed mapping the VCN message (%d)!\n", r);
+ return r;
+ }
+
+ msg = ptr + addr - start;
+
+ /* Check length */
+ if (msg[1] > end - addr) {
+ r = -EINVAL;
+ goto out;
+ }
+
+ if (msg[3] != RDECODE_MSG_CREATE)
+ goto out;
+
+ num_buffers = msg[2];
+ for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) {
+ uint32_t offset, size, *create;
+
+ if (msg[0] != RDECODE_MESSAGE_CREATE)
+ continue;
+
+ offset = msg[1];
+ size = msg[2];
+
+ if (offset + size > end) {
+ r = -EINVAL;
+ goto out;
+ }
+
+ create = ptr + addr + offset - start;
+
+ /* H246, HEVC and VP9 can run on any instance */
+ if (create[0] == 0x7 || create[0] == 0x10 || create[0] == 0x11)
+ continue;
+
+ r = vcn_v3_0_limit_sched(p);
+ if (r)
+ goto out;
+ }
+
+out:
+ amdgpu_bo_kunmap(bo);
+ return r;
+}
+
+static int vcn_v3_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p,
+ uint32_t ib_idx)
+{
+ struct amdgpu_ring *ring = to_amdgpu_ring(p->entity->rq->sched);
+ struct amdgpu_ib *ib = &p->job->ibs[ib_idx];
+ uint32_t msg_lo = 0, msg_hi = 0;
+ unsigned i;
+ int r;
+
+ /* The first instance can decode anything */
+ if (!ring->me)
+ return 0;
+
+ for (i = 0; i < ib->length_dw; i += 2) {
+ uint32_t reg = amdgpu_get_ib_value(p, ib_idx, i);
+ uint32_t val = amdgpu_get_ib_value(p, ib_idx, i + 1);
+
+ if (reg == PACKET0(p->adev->vcn.internal.data0, 0)) {
+ msg_lo = val;
+ } else if (reg == PACKET0(p->adev->vcn.internal.data1, 0)) {
+ msg_hi = val;
+ } else if (reg == PACKET0(p->adev->vcn.internal.cmd, 0) &&
+ val == 0) {
+ r = vcn_v3_0_dec_msg(p, ((u64)msg_hi) << 32 | msg_lo);
+ if (r)
+ return r;
+ }
+ }
+ return 0;
+}
+
static const struct amdgpu_ring_funcs vcn_v3_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_DEC,
.align_mask = 0xf,
.get_rptr = vcn_v3_0_dec_ring_get_rptr,
.get_wptr = vcn_v3_0_dec_ring_get_wptr,
.set_wptr = vcn_v3_0_dec_ring_set_wptr,
+ .patch_cs_in_place = vcn_v3_0_ring_patch_cs_in_place,
.emit_frame_size =
SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 +
SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 +
{
uint32_t tmp;
- /* vega20 ih reroute will go through psp
- * this function is only used for arcturus
+ /* vega20 ih reroute will go through psp this
+ * function is used for newer asics starting arcturus
*/
- if (adev->asic_type == CHIP_ARCTURUS) {
+ if (adev->asic_type >= CHIP_ARCTURUS) {
/* Reroute to IH ring 1 for VMC */
WREG32_SOC15(OSSSYS, 0, mmIH_CLIENT_CFG_INDEX, 0x12);
tmp = RREG32_SOC15(OSSSYS, 0, mmIH_CLIENT_CFG_DATA);
{
struct kfd_ioctl_get_process_apertures_args *args = data;
struct kfd_process_device_apertures *pAperture;
- struct kfd_process_device *pdd;
+ int i;
dev_dbg(kfd_device, "get apertures for PASID 0x%x", p->pasid);
args->num_of_nodes = 0;
mutex_lock(&p->mutex);
+ /* Run over all pdd of the process */
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
+ pAperture =
+ &args->process_apertures[args->num_of_nodes];
+ pAperture->gpu_id = pdd->dev->id;
+ pAperture->lds_base = pdd->lds_base;
+ pAperture->lds_limit = pdd->lds_limit;
+ pAperture->gpuvm_base = pdd->gpuvm_base;
+ pAperture->gpuvm_limit = pdd->gpuvm_limit;
+ pAperture->scratch_base = pdd->scratch_base;
+ pAperture->scratch_limit = pdd->scratch_limit;
- /*if the process-device list isn't empty*/
- if (kfd_has_process_device_data(p)) {
- /* Run over all pdd of the process */
- pdd = kfd_get_first_process_device_data(p);
- do {
- pAperture =
- &args->process_apertures[args->num_of_nodes];
- pAperture->gpu_id = pdd->dev->id;
- pAperture->lds_base = pdd->lds_base;
- pAperture->lds_limit = pdd->lds_limit;
- pAperture->gpuvm_base = pdd->gpuvm_base;
- pAperture->gpuvm_limit = pdd->gpuvm_limit;
- pAperture->scratch_base = pdd->scratch_base;
- pAperture->scratch_limit = pdd->scratch_limit;
-
- dev_dbg(kfd_device,
- "node id %u\n", args->num_of_nodes);
- dev_dbg(kfd_device,
- "gpu id %u\n", pdd->dev->id);
- dev_dbg(kfd_device,
- "lds_base %llX\n", pdd->lds_base);
- dev_dbg(kfd_device,
- "lds_limit %llX\n", pdd->lds_limit);
- dev_dbg(kfd_device,
- "gpuvm_base %llX\n", pdd->gpuvm_base);
- dev_dbg(kfd_device,
- "gpuvm_limit %llX\n", pdd->gpuvm_limit);
- dev_dbg(kfd_device,
- "scratch_base %llX\n", pdd->scratch_base);
- dev_dbg(kfd_device,
- "scratch_limit %llX\n", pdd->scratch_limit);
-
- args->num_of_nodes++;
-
- pdd = kfd_get_next_process_device_data(p, pdd);
- } while (pdd && (args->num_of_nodes < NUM_OF_SUPPORTED_GPUS));
- }
+ dev_dbg(kfd_device,
+ "node id %u\n", args->num_of_nodes);
+ dev_dbg(kfd_device,
+ "gpu id %u\n", pdd->dev->id);
+ dev_dbg(kfd_device,
+ "lds_base %llX\n", pdd->lds_base);
+ dev_dbg(kfd_device,
+ "lds_limit %llX\n", pdd->lds_limit);
+ dev_dbg(kfd_device,
+ "gpuvm_base %llX\n", pdd->gpuvm_base);
+ dev_dbg(kfd_device,
+ "gpuvm_limit %llX\n", pdd->gpuvm_limit);
+ dev_dbg(kfd_device,
+ "scratch_base %llX\n", pdd->scratch_base);
+ dev_dbg(kfd_device,
+ "scratch_limit %llX\n", pdd->scratch_limit);
+ if (++args->num_of_nodes >= NUM_OF_SUPPORTED_GPUS)
+ break;
+ }
mutex_unlock(&p->mutex);
return 0;
{
struct kfd_ioctl_get_process_apertures_new_args *args = data;
struct kfd_process_device_apertures *pa;
- struct kfd_process_device *pdd;
- uint32_t nodes = 0;
int ret;
+ int i;
dev_dbg(kfd_device, "get apertures for PASID 0x%x", p->pasid);
* sufficient memory
*/
mutex_lock(&p->mutex);
-
- if (!kfd_has_process_device_data(p))
- goto out_unlock;
-
- /* Run over all pdd of the process */
- pdd = kfd_get_first_process_device_data(p);
- do {
- args->num_of_nodes++;
- pdd = kfd_get_next_process_device_data(p, pdd);
- } while (pdd);
-
+ args->num_of_nodes = p->n_pdds;
goto out_unlock;
}
mutex_lock(&p->mutex);
- if (!kfd_has_process_device_data(p)) {
+ if (!p->n_pdds) {
args->num_of_nodes = 0;
kfree(pa);
goto out_unlock;
}
/* Run over all pdd of the process */
- pdd = kfd_get_first_process_device_data(p);
- do {
- pa[nodes].gpu_id = pdd->dev->id;
- pa[nodes].lds_base = pdd->lds_base;
- pa[nodes].lds_limit = pdd->lds_limit;
- pa[nodes].gpuvm_base = pdd->gpuvm_base;
- pa[nodes].gpuvm_limit = pdd->gpuvm_limit;
- pa[nodes].scratch_base = pdd->scratch_base;
- pa[nodes].scratch_limit = pdd->scratch_limit;
+ for (i = 0; i < min(p->n_pdds, args->num_of_nodes); i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
+ pa[i].gpu_id = pdd->dev->id;
+ pa[i].lds_base = pdd->lds_base;
+ pa[i].lds_limit = pdd->lds_limit;
+ pa[i].gpuvm_base = pdd->gpuvm_base;
+ pa[i].gpuvm_limit = pdd->gpuvm_limit;
+ pa[i].scratch_base = pdd->scratch_base;
+ pa[i].scratch_limit = pdd->scratch_limit;
dev_dbg(kfd_device,
"gpu id %u\n", pdd->dev->id);
"scratch_base %llX\n", pdd->scratch_base);
dev_dbg(kfd_device,
"scratch_limit %llX\n", pdd->scratch_limit);
- nodes++;
-
- pdd = kfd_get_next_process_device_data(p, pdd);
- } while (pdd && (nodes < args->num_of_nodes));
+ }
mutex_unlock(&p->mutex);
- args->num_of_nodes = nodes;
+ args->num_of_nodes = i;
ret = copy_to_user(
(void __user *)args->kfd_process_device_apertures_ptr,
pa,
- (nodes * sizeof(struct kfd_process_device_apertures)));
+ (i * sizeof(struct kfd_process_device_apertures)));
kfree(pa);
return ret ? -EFAULT : 0;
/* Wait till CP writes sync code: */
status = amdkfd_fence_wait_timeout(
- (unsigned int *) rm_state,
+ rm_state,
QUEUESTATE__ACTIVE, 1500);
kfd_gtt_sa_free(dbgdev->dev, mem_obj);
return single_open(file, show, NULL);
}
+static int kfd_debugfs_hang_hws_read(struct seq_file *m, void *data)
+{
+ seq_printf(m, "echo gpu_id > hang_hws\n");
+ return 0;
+}
static ssize_t kfd_debugfs_hang_hws_write(struct file *file,
const char __user *user_buf, size_t size, loff_t *ppos)
debugfs_create_file("rls", S_IFREG | 0444, debugfs_root,
kfd_debugfs_rls_by_device, &kfd_debugfs_fops);
debugfs_create_file("hang_hws", S_IFREG | 0200, debugfs_root,
- NULL, &kfd_debugfs_hang_hws_fops);
+ kfd_debugfs_hang_hws_read, &kfd_debugfs_hang_hws_fops);
}
void kfd_debugfs_fini(void)
void kgd2kfd_smi_event_throttle(struct kfd_dev *kfd, uint32_t throttle_bitmask)
{
- if (kfd)
+ if (kfd && kfd->init_complete)
kfd_smi_event_update_thermal_throttling(kfd, throttle_bitmask);
}
if (retval)
goto fail_allocate_vidmem;
- dqm->fence_addr = dqm->fence_mem->cpu_ptr;
+ dqm->fence_addr = (uint64_t *)dqm->fence_mem->cpu_ptr;
dqm->fence_gpu_addr = dqm->fence_mem->gpu_addr;
init_interrupts(dqm);
return retval;
}
-int amdkfd_fence_wait_timeout(unsigned int *fence_addr,
- unsigned int fence_value,
+int amdkfd_fence_wait_timeout(uint64_t *fence_addr,
+ uint64_t fence_value,
unsigned int timeout_ms)
{
unsigned long end_jiffies = msecs_to_jiffies(timeout_ms) + jiffies;
uint16_t vmid_pasid[VMID_NUM];
uint64_t pipelines_addr;
uint64_t fence_gpu_addr;
- unsigned int *fence_addr;
+ uint64_t *fence_addr;
struct kfd_mem_obj *fence_mem;
bool active_runlist;
int sched_policy;
*/
void kfd_iommu_unbind_process(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- if (pdd->bound == PDD_BOUND)
- amd_iommu_unbind_pasid(pdd->dev->pdev, p->pasid);
+ for (i = 0; i < p->n_pdds; i++)
+ if (p->pdds[i]->bound == PDD_BOUND)
+ amd_iommu_unbind_pasid(p->pdds[i]->dev->pdev, p->pasid);
}
/* Callback for process shutdown invoked by the IOMMU driver */
}
int pm_send_query_status(struct packet_manager *pm, uint64_t fence_address,
- uint32_t fence_value)
+ uint64_t fence_value)
{
uint32_t *buffer, size;
int retval = 0;
}
static int pm_query_status_v9(struct packet_manager *pm, uint32_t *buffer,
- uint64_t fence_address, uint32_t fence_value)
+ uint64_t fence_address, uint64_t fence_value)
{
struct pm4_mes_query_status *packet;
}
static int pm_query_status_vi(struct packet_manager *pm, uint32_t *buffer,
- uint64_t fence_address, uint32_t fence_value)
+ uint64_t fence_address, uint64_t fence_value)
{
struct pm4_mes_query_status *packet;
#include <linux/swap.h>
#include "amd_shared.h"
+#include "amdgpu.h"
#define KFD_MAX_RING_ENTRY_SIZE 8
/* Data that is per-process-per device. */
struct kfd_process_device {
- /*
- * List of all per-device data for a process.
- * Starts from kfd_process.per_device_data.
- */
- struct list_head per_device_list;
-
/* The device that owns this data. */
struct kfd_dev *dev;
u32 pasid;
/*
- * List of kfd_process_device structures,
+ * Array of kfd_process_device pointers,
* one for each device the process is using.
*/
- struct list_head per_device_data;
+ struct kfd_process_device *pdds[MAX_GPU_INSTANCE];
+ uint32_t n_pdds;
struct process_queue_manager pqm;
void kfd_process_device_remove_obj_handle(struct kfd_process_device *pdd,
int handle);
-/* Process device data iterator */
-struct kfd_process_device *kfd_get_first_process_device_data(
- struct kfd_process *p);
-struct kfd_process_device *kfd_get_next_process_device_data(
- struct kfd_process *p,
- struct kfd_process_device *pdd);
-bool kfd_has_process_device_data(struct kfd_process *p);
-
/* PASIDs */
int kfd_pasid_init(void);
void kfd_pasid_exit(void);
u32 *ctl_stack_used_size,
u32 *save_area_used_size);
-int amdkfd_fence_wait_timeout(unsigned int *fence_addr,
- unsigned int fence_value,
+int amdkfd_fence_wait_timeout(uint64_t *fence_addr,
+ uint64_t fence_value,
unsigned int timeout_ms);
/* Packet Manager */
uint32_t filter_param, bool reset,
unsigned int sdma_engine);
int (*query_status)(struct packet_manager *pm, uint32_t *buffer,
- uint64_t fence_address, uint32_t fence_value);
+ uint64_t fence_address, uint64_t fence_value);
int (*release_mem)(uint64_t gpu_addr, uint32_t *buffer);
/* Packet sizes */
struct scheduling_resources *res);
int pm_send_runlist(struct packet_manager *pm, struct list_head *dqm_queues);
int pm_send_query_status(struct packet_manager *pm, uint64_t fence_address,
- uint32_t fence_value);
+ uint64_t fence_value);
int pm_send_unmap_queue(struct packet_manager *pm, enum kfd_queue_type type,
enum kfd_unmap_queues_filter mode,
static int kfd_procfs_add_sysfs_stats(struct kfd_process *p)
{
int ret = 0;
- struct kfd_process_device *pdd;
+ int i;
char stats_dir_filename[MAX_SYSFS_FILENAME_LEN];
if (!p)
* - proc/<pid>/stats_<gpuid>/evicted_ms
* - proc/<pid>/stats_<gpuid>/cu_occupancy
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
struct kobject *kobj_stats;
snprintf(stats_dir_filename, MAX_SYSFS_FILENAME_LEN,
static int kfd_procfs_add_sysfs_files(struct kfd_process *p)
{
int ret = 0;
- struct kfd_process_device *pdd;
+ int i;
if (!p)
return -EINVAL;
* - proc/<pid>/vram_<gpuid>
* - proc/<pid>/sdma_<gpuid>
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
snprintf(pdd->vram_filename, MAX_SYSFS_FILENAME_LEN, "vram_%u",
pdd->dev->id);
ret = kfd_sysfs_create_file(p, &pdd->attr_vram, pdd->vram_filename);
kref_put(&p->ref, kfd_process_ref_release);
}
+
static void kfd_process_device_free_bos(struct kfd_process_device *pdd)
{
struct kfd_process *p = pdd->process;
void *mem;
int id;
+ int i;
/*
* Remove all handles from idr and release appropriate
* local memory object
*/
idr_for_each_entry(&pdd->alloc_idr, mem, id) {
- struct kfd_process_device *peer_pdd;
- list_for_each_entry(peer_pdd, &p->per_device_data,
- per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *peer_pdd = p->pdds[i];
+
if (!peer_pdd->vm)
continue;
amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
static void kfd_process_free_outstanding_kfd_bos(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- kfd_process_device_free_bos(pdd);
+ for (i = 0; i < p->n_pdds; i++)
+ kfd_process_device_free_bos(p->pdds[i]);
}
static void kfd_process_destroy_pdds(struct kfd_process *p)
{
- struct kfd_process_device *pdd, *temp;
+ int i;
+
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
- list_for_each_entry_safe(pdd, temp, &p->per_device_data,
- per_device_list) {
pr_debug("Releasing pdd (topology id %d) for process (pasid 0x%x)\n",
pdd->dev->id, p->pasid);
amdgpu_amdkfd_gpuvm_destroy_process_vm(
pdd->dev->kgd, pdd->vm);
- list_del(&pdd->per_device_list);
-
if (pdd->qpd.cwsr_kaddr && !pdd->qpd.cwsr_base)
free_pages((unsigned long)pdd->qpd.cwsr_kaddr,
get_order(KFD_CWSR_TBA_TMA_SIZE));
}
kfree(pdd);
+ p->pdds[i] = NULL;
}
+ p->n_pdds = 0;
}
/* No process locking is needed in this function, because the process
{
struct kfd_process *p = container_of(work, struct kfd_process,
release_work);
- struct kfd_process_device *pdd;
+ int i;
/* Remove the procfs files */
if (p->kobj) {
kobject_put(p->kobj_queues);
p->kobj_queues = NULL;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
sysfs_remove_file(p->kobj, &pdd->attr_vram);
sysfs_remove_file(p->kobj, &pdd->attr_sdma);
sysfs_remove_file(p->kobj, &pdd->attr_evict);
struct mm_struct *mm)
{
struct kfd_process *p;
- struct kfd_process_device *pdd = NULL;
+ int i;
/*
* The kfd_process structure can not be free because the
* pdd is in debug mode, we should first force unregistration,
* then we will be able to destroy the queues
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
- struct kfd_dev *dev = pdd->dev;
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_dev *dev = p->pdds[i]->dev;
mutex_lock(kfd_get_dbgmgr_mutex());
if (dev && dev->dbgmgr && dev->dbgmgr->pasid == p->pasid) {
static int kfd_process_init_cwsr_apu(struct kfd_process *p, struct file *filep)
{
unsigned long offset;
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
- struct kfd_dev *dev = pdd->dev;
- struct qcm_process_device *qpd = &pdd->qpd;
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_dev *dev = p->pdds[i]->dev;
+ struct qcm_process_device *qpd = &p->pdds[i]->qpd;
if (!dev->cwsr_enabled || qpd->cwsr_kaddr || qpd->cwsr_base)
continue;
mutex_init(&process->mutex);
process->mm = thread->mm;
process->lead_thread = thread->group_leader;
- INIT_LIST_HEAD(&process->per_device_data);
+ process->n_pdds = 0;
INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
process->last_restore_timestamp = get_jiffies_64();
struct kfd_process_device *kfd_get_process_device_data(struct kfd_dev *dev,
struct kfd_process *p)
{
- struct kfd_process_device *pdd = NULL;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- if (pdd->dev == dev)
- return pdd;
+ for (i = 0; i < p->n_pdds; i++)
+ if (p->pdds[i]->dev == dev)
+ return p->pdds[i];
return NULL;
}
{
struct kfd_process_device *pdd = NULL;
+ if (WARN_ON_ONCE(p->n_pdds >= MAX_GPU_INSTANCE))
+ return NULL;
pdd = kzalloc(sizeof(*pdd), GFP_KERNEL);
if (!pdd)
return NULL;
pdd->vram_usage = 0;
pdd->sdma_past_activity_counter = 0;
atomic64_set(&pdd->evict_duration_counter, 0);
- list_add(&pdd->per_device_list, &p->per_device_data);
+ p->pdds[p->n_pdds++] = pdd;
/* Init idr used for memory handle translation */
idr_init(&pdd->alloc_idr);
return ERR_PTR(err);
}
-struct kfd_process_device *kfd_get_first_process_device_data(
- struct kfd_process *p)
-{
- return list_first_entry(&p->per_device_data,
- struct kfd_process_device,
- per_device_list);
-}
-
-struct kfd_process_device *kfd_get_next_process_device_data(
- struct kfd_process *p,
- struct kfd_process_device *pdd)
-{
- if (list_is_last(&pdd->per_device_list, &p->per_device_data))
- return NULL;
- return list_next_entry(pdd, per_device_list);
-}
-
-bool kfd_has_process_device_data(struct kfd_process *p)
-{
- return !(list_empty(&p->per_device_data));
-}
-
/* Create specific handle mapped to mem from process local memory idr
* Assumes that the process lock is held.
*/
*/
int kfd_process_evict_queues(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
int r = 0;
+ int i;
unsigned int n_evicted = 0;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
r = pdd->dev->dqm->ops.evict_process_queues(pdd->dev->dqm,
&pdd->qpd);
if (r) {
/* To keep state consistent, roll back partial eviction by
* restoring queues
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
if (n_evicted == 0)
break;
if (pdd->dev->dqm->ops.restore_process_queues(pdd->dev->dqm,
/* kfd_process_restore_queues - Restore all user queues of a process */
int kfd_process_restore_queues(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
int r, ret = 0;
+ int i;
+
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
r = pdd->dev->dqm->ops.restore_process_queues(pdd->dev->dqm,
&pdd->qpd);
if (r) {
void kfd_process_dequeue_from_all_devices(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- kfd_process_dequeue_from_device(pdd);
+ for (i = 0; i < p->n_pdds; i++)
+ kfd_process_dequeue_from_device(p->pdds[i]);
}
int pqm_init(struct process_queue_manager *pqm, struct kfd_process *p)
struct kfd_smi_client *client = filep->private_data;
unsigned char *buf;
- buf = kmalloc(MAX_KFIFO_SIZE * sizeof(*buf), GFP_KERNEL);
+ buf = kmalloc_array(MAX_KFIFO_SIZE, sizeof(*buf), GFP_KERNEL);
if (!buf)
return -ENOMEM;
* DOC: overview
*
* The AMDgpu display manager, **amdgpu_dm** (or even simpler,
- * **dm**) sits between DRM and DC. It acts as a liason, converting DRM
+ * **dm**) sits between DRM and DC. It acts as a liaison, converting DRM
* requests into DC requests, and DC responses into DRM responses.
*
* The root control structure is &struct amdgpu_display_manager.
/* basic init/fini API */
static int amdgpu_dm_init(struct amdgpu_device *adev);
static void amdgpu_dm_fini(struct amdgpu_device *adev);
+static bool is_freesync_video_mode(const struct drm_display_mode *mode, struct amdgpu_dm_connector *aconnector);
static enum drm_mode_subconnector get_subconnector_type(struct dc_link *link)
{
/* IRQ could occur when in initial stage */
/* TODO work and BO cleanup */
if (amdgpu_crtc == NULL) {
- DRM_DEBUG_DRIVER("CRTC is null, returning.\n");
+ DC_LOG_PFLIP("CRTC is null, returning.\n");
return;
}
spin_lock_irqsave(&adev_to_drm(adev)->event_lock, flags);
if (amdgpu_crtc->pflip_status != AMDGPU_FLIP_SUBMITTED){
- DRM_DEBUG_DRIVER("amdgpu_crtc->pflip_status = %d !=AMDGPU_FLIP_SUBMITTED(%d) on crtc:%d[%p] \n",
+ DC_LOG_PFLIP("amdgpu_crtc->pflip_status = %d !=AMDGPU_FLIP_SUBMITTED(%d) on crtc:%d[%p] \n",
amdgpu_crtc->pflip_status,
AMDGPU_FLIP_SUBMITTED,
amdgpu_crtc->crtc_id,
amdgpu_crtc->pflip_status = AMDGPU_FLIP_NONE;
spin_unlock_irqrestore(&adev_to_drm(adev)->event_lock, flags);
- DRM_DEBUG_DRIVER("crtc:%d[%p], pflip_stat:AMDGPU_FLIP_NONE, vrr[%d]-fp %d\n",
- amdgpu_crtc->crtc_id, amdgpu_crtc,
- vrr_active, (int) !e);
+ DC_LOG_PFLIP("crtc:%d[%p], pflip_stat:AMDGPU_FLIP_NONE, vrr[%d]-fp %d\n",
+ amdgpu_crtc->crtc_id, amdgpu_crtc,
+ vrr_active, (int) !e);
}
static void dm_vupdate_high_irq(void *interrupt_params)
struct common_irq_params *irq_params = interrupt_params;
struct amdgpu_device *adev = irq_params->adev;
struct amdgpu_crtc *acrtc;
+ struct drm_device *drm_dev;
+ struct drm_vblank_crtc *vblank;
+ ktime_t frame_duration_ns, previous_timestamp;
unsigned long flags;
int vrr_active;
if (acrtc) {
vrr_active = amdgpu_dm_vrr_active_irq(acrtc);
+ drm_dev = acrtc->base.dev;
+ vblank = &drm_dev->vblank[acrtc->base.index];
+ previous_timestamp = atomic64_read(&irq_params->previous_timestamp);
+ frame_duration_ns = vblank->time - previous_timestamp;
- DRM_DEBUG_VBL("crtc:%d, vupdate-vrr:%d\n",
+ if (frame_duration_ns > 0) {
+ trace_amdgpu_refresh_rate_track(acrtc->base.index,
+ frame_duration_ns,
+ ktime_divns(NSEC_PER_SEC, frame_duration_ns));
+ atomic64_set(&irq_params->previous_timestamp, vblank->time);
+ }
+
+ DC_LOG_VBLANK("crtc:%d, vupdate-vrr:%d\n",
acrtc->crtc_id,
vrr_active);
vrr_active = amdgpu_dm_vrr_active_irq(acrtc);
- DRM_DEBUG_VBL("crtc:%d, vupdate-vrr:%d, planes:%d\n", acrtc->crtc_id,
+ DC_LOG_VBLANK("crtc:%d, vupdate-vrr:%d, planes:%d\n", acrtc->crtc_id,
vrr_active, acrtc->dm_irq_params.active_planes);
/**
}
#if defined(CONFIG_DRM_AMD_DC_DCN)
+#define DMUB_TRACE_MAX_READ 64
+static void dm_dmub_trace_high_irq(void *interrupt_params)
+{
+ struct common_irq_params *irq_params = interrupt_params;
+ struct amdgpu_device *adev = irq_params->adev;
+ struct amdgpu_display_manager *dm = &adev->dm;
+ struct dmcub_trace_buf_entry entry = { 0 };
+ uint32_t count = 0;
+
+ do {
+ if (dc_dmub_srv_get_dmub_outbox0_msg(dm->dc, &entry)) {
+ trace_amdgpu_dmub_trace_high_irq(entry.trace_code, entry.tick_count,
+ entry.param0, entry.param1);
+
+ DRM_DEBUG_DRIVER("trace_code:%u, tick_count:%u, param0:%u, param1:%u\n",
+ entry.trace_code, entry.tick_count, entry.param0, entry.param1);
+ } else
+ break;
+
+ count++;
+
+ } while (count <= DMUB_TRACE_MAX_READ);
+
+ ASSERT(count <= DMUB_TRACE_MAX_READ);
+}
+
static void mmhub_read_system_context(struct amdgpu_device *adev, struct dc_phy_addr_space_config *pa_config)
{
uint64_t pt_base;
if (vblank_work->enable)
dm->active_vblank_irq_count++;
- else
+ else if(dm->active_vblank_irq_count)
dm->active_vblank_irq_count--;
dc_allow_idle_optimizations(dm->dc, dm->active_vblank_irq_count == 0);
- DRM_DEBUG_DRIVER("Allow idle optimizations (MALL): %d\n", dm->active_vblank_irq_count == 0);
-
+ DRM_DEBUG_KMS("Allow idle optimizations (MALL): %d\n", dm->active_vblank_irq_count == 0);
mutex_unlock(&dm->dc_lock);
}
if (acrtc && state->stream_status[i].plane_count != 0) {
irq_source = IRQ_TYPE_PFLIP + acrtc->otg_inst;
rc = dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY;
- DRM_DEBUG("crtc %d - vupdate irq %sabling: r=%d\n",
- acrtc->crtc_id, enable ? "en" : "dis", rc);
+ DRM_DEBUG_VBL("crtc %d - vupdate irq %sabling: r=%d\n",
+ acrtc->crtc_id, enable ? "en" : "dis", rc);
if (rc)
DRM_WARN("Failed to %s pflip interrupts\n",
enable ? "enable" : "disable");
}
+ if (dc->ctx->dmub_srv) {
+ i = DCN_1_0__SRCID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT;
+ r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE, i, &adev->dmub_trace_irq);
+
+ if (r) {
+ DRM_ERROR("Failed to add dmub trace irq id!\n");
+ return r;
+ }
+
+ int_params.int_context = INTERRUPT_HIGH_IRQ_CONTEXT;
+ int_params.irq_source =
+ dc_interrupt_to_irq_source(dc, i, 0);
+
+ c_irq_params = &adev->dm.dmub_trace_params[0];
+
+ c_irq_params->adev = adev;
+ c_irq_params->irq_src = int_params.irq_source;
+
+ amdgpu_dm_irq_register_interrupt(adev, &int_params,
+ dm_dmub_trace_high_irq, c_irq_params);
+ }
+
/* HPD */
r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE, DCN_1_0__SRCID__DC_HPD1_INT,
&adev->hpd_irq);
stream->src = src;
stream->dst = dst;
- DRM_DEBUG_DRIVER("Destination Rectangle x:%d y:%d width:%d height:%d\n",
- dst.x, dst.y, dst.width, dst.height);
+ DRM_DEBUG_KMS("Destination Rectangle x:%d y:%d width:%d height:%d\n",
+ dst.x, dst.y, dst.width, dst.height);
}
timing_out->hdmi_vic = hv_frame.vic;
}
- timing_out->h_addressable = mode_in->hdisplay;
- timing_out->h_total = mode_in->htotal;
- timing_out->h_sync_width = mode_in->hsync_end - mode_in->hsync_start;
- timing_out->h_front_porch = mode_in->hsync_start - mode_in->hdisplay;
- timing_out->v_total = mode_in->vtotal;
- timing_out->v_addressable = mode_in->vdisplay;
- timing_out->v_front_porch = mode_in->vsync_start - mode_in->vdisplay;
- timing_out->v_sync_width = mode_in->vsync_end - mode_in->vsync_start;
- timing_out->pix_clk_100hz = mode_in->clock * 10;
+ if (is_freesync_video_mode(mode_in, aconnector)) {
+ timing_out->h_addressable = mode_in->hdisplay;
+ timing_out->h_total = mode_in->htotal;
+ timing_out->h_sync_width = mode_in->hsync_end - mode_in->hsync_start;
+ timing_out->h_front_porch = mode_in->hsync_start - mode_in->hdisplay;
+ timing_out->v_total = mode_in->vtotal;
+ timing_out->v_addressable = mode_in->vdisplay;
+ timing_out->v_front_porch = mode_in->vsync_start - mode_in->vdisplay;
+ timing_out->v_sync_width = mode_in->vsync_end - mode_in->vsync_start;
+ timing_out->pix_clk_100hz = mode_in->clock * 10;
+ } else {
+ timing_out->h_addressable = mode_in->crtc_hdisplay;
+ timing_out->h_total = mode_in->crtc_htotal;
+ timing_out->h_sync_width = mode_in->crtc_hsync_end - mode_in->crtc_hsync_start;
+ timing_out->h_front_porch = mode_in->crtc_hsync_start - mode_in->crtc_hdisplay;
+ timing_out->v_total = mode_in->crtc_vtotal;
+ timing_out->v_addressable = mode_in->crtc_vdisplay;
+ timing_out->v_front_porch = mode_in->crtc_vsync_start - mode_in->crtc_vdisplay;
+ timing_out->v_sync_width = mode_in->crtc_vsync_end - mode_in->crtc_vsync_start;
+ timing_out->pix_clk_100hz = mode_in->crtc_clock * 10;
+ }
timing_out->aspect_ratio = get_aspect_ratio(mode_in);
static void set_multisync_trigger_params(
struct dc_stream_state *stream)
{
+ struct dc_stream_state *master = NULL;
+
if (stream->triggered_crtc_reset.enabled) {
- stream->triggered_crtc_reset.event = CRTC_EVENT_VSYNC_RISING;
- stream->triggered_crtc_reset.delay = TRIGGER_DELAY_NEXT_LINE;
+ master = stream->triggered_crtc_reset.event_source;
+ stream->triggered_crtc_reset.event =
+ master->timing.flags.VSYNC_POSITIVE_POLARITY ?
+ CRTC_EVENT_VSYNC_RISING : CRTC_EVENT_VSYNC_FALLING;
+ stream->triggered_crtc_reset.delay = TRIGGER_DELAY_NEXT_PIXEL;
}
}
static void dm_enable_per_frame_crtc_master_sync(struct dc_state *context)
{
int i = 0;
+ struct dc_stream_state *stream;
if (context->stream_count < 2)
return;
* crtc_sync_master.multi_sync_enabled flag
* For now it's set to false
*/
- set_multisync_trigger_params(context->streams[i]);
}
+
set_master_stream(context->streams, context->stream_count);
+
+ for (i = 0; i < context->stream_count ; i++) {
+ stream = context->streams[i];
+
+ if (!stream)
+ continue;
+
+ set_multisync_trigger_params(stream);
+ }
}
static struct drm_display_mode *
return m_pref;
}
-static bool is_freesync_video_mode(struct drm_display_mode *mode,
+static bool is_freesync_video_mode(const struct drm_display_mode *mode,
struct amdgpu_dm_connector *aconnector)
{
struct drm_display_mode *high_mode;
if (recalculate_timing)
drm_mode_set_crtcinfo(&saved_mode, 0);
- else
+ else if (!dm_state)
drm_mode_set_crtcinfo(&mode, 0);
/*
rc = dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY;
- DRM_DEBUG_DRIVER("crtc %d - vupdate irq %sabling: r=%d\n",
- acrtc->crtc_id, enable ? "en" : "dis", rc);
+ DRM_DEBUG_VBL("crtc %d - vupdate irq %sabling: r=%d\n",
+ acrtc->crtc_id, enable ? "en" : "dis", rc);
return rc;
}
} while (stream == NULL && requested_bpc >= 6);
+ if (dc_result == DC_FAIL_ENC_VALIDATE && !aconnector->force_yuv420_output) {
+ DRM_DEBUG_KMS("Retry forcing YCbCr420 encoding\n");
+
+ aconnector->force_yuv420_output = true;
+ stream = create_validate_stream_for_sink(aconnector, drm_mode,
+ dm_state, old_stream);
+ aconnector->force_yuv420_output = false;
+ }
+
return stream;
}
int r;
if (!new_state->fb) {
- DRM_DEBUG_DRIVER("No FB bound\n");
+ DRM_DEBUG_KMS("No FB bound\n");
return 0;
}
if (!(amdgpu_freesync_vid_mode && edid))
return;
-
+
if (amdgpu_dm_connector->max_vfreq - amdgpu_dm_connector->min_vfreq > 10)
amdgpu_dm_connector->num_modes +=
add_fs_modes(amdgpu_dm_connector);
if (!plane->state->fb && !old_plane_state->fb)
return;
- DRM_DEBUG_DRIVER("%s: crtc_id=%d with size %d to %d\n",
- __func__,
- amdgpu_crtc->crtc_id,
- plane->state->crtc_w,
- plane->state->crtc_h);
+ DC_LOG_CURSOR("%s: crtc_id=%d with size %d to %d\n",
+ __func__,
+ amdgpu_crtc->crtc_id,
+ plane->state->crtc_w,
+ plane->state->crtc_h);
ret = get_cursor_position(plane, crtc, &position);
if (ret)
/* Mark this event as consumed */
acrtc->base.state->event = NULL;
- DRM_DEBUG_DRIVER("crtc:%d, pflip_stat:AMDGPU_FLIP_SUBMITTED\n",
- acrtc->crtc_id);
+ DC_LOG_PFLIP("crtc:%d, pflip_stat:AMDGPU_FLIP_SUBMITTED\n",
+ acrtc->crtc_id);
}
static void update_freesync_state_on_stream(
&bundle->flip_addrs[planes_count].address,
afb->tmz_surface, false);
- DRM_DEBUG_DRIVER("plane: id=%d dcc_en=%d\n",
+ DRM_DEBUG_ATOMIC("plane: id=%d dcc_en=%d\n",
new_plane_state->plane->index,
bundle->plane_infos[planes_count].dcc.enable);
dc_plane,
bundle->flip_addrs[planes_count].flip_timestamp_in_us);
- DRM_DEBUG_DRIVER("%s Flipping to hi: 0x%x, low: 0x%x\n",
+ DRM_DEBUG_ATOMIC("%s Flipping to hi: 0x%x, low: 0x%x\n",
__func__,
bundle->flip_addrs[planes_count].address.grph.addr.high_part,
bundle->flip_addrs[planes_count].address.grph.addr.low_part);
dm_new_crtc_state = to_dm_crtc_state(new_crtc_state);
dm_old_crtc_state = to_dm_crtc_state(old_crtc_state);
- DRM_DEBUG_DRIVER(
+ DRM_DEBUG_ATOMIC(
"amdgpu_crtc id:%d crtc_state_flags: enable:%d, active:%d, "
"planes_changed:%d, mode_changed:%d,active_changed:%d,"
"connectors_changed:%d\n",
if (modeset_required(new_crtc_state, dm_new_crtc_state->stream, dm_old_crtc_state->stream)) {
- DRM_DEBUG_DRIVER("Atomic commit: SET crtc id %d: [%p]\n", acrtc->crtc_id, acrtc);
+ DRM_DEBUG_ATOMIC("Atomic commit: SET crtc id %d: [%p]\n", acrtc->crtc_id, acrtc);
if (!dm_new_crtc_state->stream) {
/*
crtc->hwmode = new_crtc_state->mode;
mode_set_reset_required = true;
} else if (modereset_required(new_crtc_state)) {
- DRM_DEBUG_DRIVER("Atomic commit: RESET. crtc id %d:[%p]\n", acrtc->crtc_id, acrtc);
+ DRM_DEBUG_ATOMIC("Atomic commit: RESET. crtc id %d:[%p]\n", acrtc->crtc_id, acrtc);
/* i.e. reset mode */
if (dm_old_crtc_state->stream)
remove_stream(adev, acrtc, dm_old_crtc_state->stream);
dm_enable_per_frame_crtc_master_sync(dc_state);
mutex_lock(&dm->dc_lock);
WARN_ON(!dc_commit_state(dm->dc, dc_state));
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ /* Allow idle optimization when vblank count is 0 for display off */
+ if (dm->active_vblank_irq_count == 0)
+ dc_allow_idle_optimizations(dm->dc,true);
+#endif
mutex_unlock(&dm->dc_lock);
}
if (!drm_atomic_crtc_needs_modeset(new_crtc_state))
goto skip_modeset;
- DRM_DEBUG_DRIVER(
+ DRM_DEBUG_ATOMIC(
"amdgpu_crtc id:%d crtc_state_flags: enable:%d, active:%d, "
"planes_changed:%d, mode_changed:%d,active_changed:%d,"
"connectors_changed:%d\n",
dc_stream_retain(new_stream);
- DRM_DEBUG_DRIVER("Enabling DRM crtc: %d\n",
- crtc->base.id);
+ DRM_DEBUG_ATOMIC("Enabling DRM crtc: %d\n",
+ crtc->base.id);
if (dc_add_stream_to_ctx(
dm->dc,
if (!dc_new_plane_state)
return -ENOMEM;
- DRM_DEBUG_DRIVER("Enabling DRM plane: %d on DRM crtc %d\n",
- plane->base.id, new_plane_crtc->base.id);
+ DRM_DEBUG_ATOMIC("Enabling DRM plane: %d on DRM crtc %d\n",
+ plane->base.id, new_plane_crtc->base.id);
ret = fill_dc_plane_attributes(
drm_to_adev(new_plane_crtc->dev),
new_cursor_state = drm_atomic_get_new_plane_state(state, crtc->cursor);
new_primary_state = drm_atomic_get_new_plane_state(state, crtc->primary);
- if (!new_cursor_state || !new_primary_state || !new_cursor_state->fb) {
+ if (!new_cursor_state || !new_primary_state ||
+ !new_cursor_state->fb || !new_primary_state->fb) {
return 0;
}
struct common_irq_params {
struct amdgpu_device *adev;
enum dc_irq_source irq_src;
+ atomic64_t previous_timestamp;
};
/**
struct common_irq_params
vupdate_params[DC_IRQ_SOURCE_VUPDATE6 - DC_IRQ_SOURCE_VUPDATE1 + 1];
+ /**
+ * @dmub_trace_params:
+ *
+ * DMUB trace event IRQ parameters, passed to registered handlers when
+ * triggered.
+ */
+ struct common_irq_params
+ dmub_trace_params[1];
+
spinlock_t irq_handler_list_table_lock;
struct backlight_device *backlight_dev;
#endif
#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ /**
+ * @crc_rd_wrk:
+ *
+ * Work to be executed in a separate thread to communicate with PSP.
+ */
struct crc_rd_work *crc_rd_wrk;
#endif
goto cleanup;
}
- aux = &aconn->dm_dp_aux.aux;
+ aux = (aconn->port) ? &aconn->port->aux : &aconn->dm_dp_aux.aux;
if (!aux) {
DRM_DEBUG_DRIVER("No dp aux for amd connector\n");
#include "resource.h"
#include "dsc.h"
#include "dc_link_dp.h"
+#include "link_hwss.h"
#include "dc/dc_dmub_srv.h"
struct dmub_debugfs_trace_header {
*
* --- to get dp configuration
*
- * cat link_settings
+ * cat /sys/kernel/debug/dri/0/DP-x/link_settings
*
* It will list current, verified, reported, preferred dp configuration.
* current -- for current video mode
* echo <lane_count> <link_rate> > link_settings
*
* for example, to force to 2 lane, 2.7GHz,
- * echo 4 0xa > link_settings
+ * echo 4 0xa > /sys/kernel/debug/dri/0/DP-x/link_settings
*
* spread_spectrum could not be changed dynamically.
*
* done. please check link settings after force operation to see if HW get
* programming.
*
- * cat link_settings
+ * cat /sys/kernel/debug/dri/0/DP-x/link_settings
*
* check current and preferred settings.
*
{
struct amdgpu_dm_connector *connector = file_inode(f)->i_private;
struct dc_link *link = connector->dc_link;
- struct dc *dc = (struct dc *)link->dc;
struct dc_link_settings prefer_link_settings;
char *wr_buf = NULL;
const uint32_t wr_buf_size = 40;
int max_param_num = 2;
uint8_t param_nums = 0;
long param[2];
- bool valid_input = false;
+ bool valid_input = true;
if (size == 0)
return -EINVAL;
case LANE_COUNT_ONE:
case LANE_COUNT_TWO:
case LANE_COUNT_FOUR:
- valid_input = true;
break;
default:
+ valid_input = false;
break;
}
case LINK_RATE_RBR2:
case LINK_RATE_HIGH2:
case LINK_RATE_HIGH3:
- valid_input = true;
break;
default:
+ valid_input = false;
break;
}
* spread spectrum will not be changed
*/
prefer_link_settings.link_spread = link->cur_link_settings.link_spread;
+ prefer_link_settings.use_link_rate_set = false;
prefer_link_settings.lane_count = param[0];
prefer_link_settings.link_rate = param[1];
- dc_link_set_preferred_link_settings(dc, &prefer_link_settings, link);
+ dp_retrain_link_dp_test(link, &prefer_link_settings, false);
kfree(wr_buf);
return size;
return result;
}
+static int dp_lttpr_status_show(struct seq_file *m, void *d)
+{
+ char *data;
+ struct amdgpu_dm_connector *connector = file_inode(m->file)->i_private;
+ struct dc_link *link = connector->dc_link;
+ uint32_t read_size = 1;
+ uint8_t repeater_count = 0;
+
+ data = kzalloc(read_size, GFP_KERNEL);
+ if (!data)
+ return 0;
+
+ dm_helpers_dp_read_dpcd(link->ctx, link, 0xF0002, data, read_size);
+
+ switch ((uint8_t)*data) {
+ case 0x80:
+ repeater_count = 1;
+ break;
+ case 0x40:
+ repeater_count = 2;
+ break;
+ case 0x20:
+ repeater_count = 3;
+ break;
+ case 0x10:
+ repeater_count = 4;
+ break;
+ case 0x8:
+ repeater_count = 5;
+ break;
+ case 0x4:
+ repeater_count = 6;
+ break;
+ case 0x2:
+ repeater_count = 7;
+ break;
+ case 0x1:
+ repeater_count = 8;
+ break;
+ case 0x0:
+ repeater_count = 0;
+ break;
+ default:
+ repeater_count = (uint8_t)*data;
+ break;
+ }
+
+ seq_printf(m, "phy repeater count: %d\n", repeater_count);
+
+ dm_helpers_dp_read_dpcd(link->ctx, link, 0xF0003, data, read_size);
+
+ if ((uint8_t)*data == 0x55)
+ seq_printf(m, "phy repeater mode: transparent\n");
+ else if ((uint8_t)*data == 0xAA)
+ seq_printf(m, "phy repeater mode: non-transparent\n");
+ else if ((uint8_t)*data == 0x00)
+ seq_printf(m, "phy repeater mode: non lttpr\n");
+ else
+ seq_printf(m, "phy repeater mode: read error\n");
+
+ kfree(data);
+ return 0;
+}
+
static ssize_t dp_phy_settings_write(struct file *f, const char __user *buf,
size_t size, loff_t *pos)
{
DEFINE_SHOW_ATTRIBUTE(dmub_fw_state);
DEFINE_SHOW_ATTRIBUTE(dmub_tracebuffer);
DEFINE_SHOW_ATTRIBUTE(output_bpc);
+DEFINE_SHOW_ATTRIBUTE(dp_lttpr_status);
#ifdef CONFIG_DRM_AMD_DC_HDCP
DEFINE_SHOW_ATTRIBUTE(hdcp_sink_capability);
#endif
} dp_debugfs_entries[] = {
{"link_settings", &dp_link_settings_debugfs_fops},
{"phy_settings", &dp_phy_settings_debugfs_fop},
+ {"lttpr_status", &dp_lttpr_status_fops},
{"test_pattern", &dp_phy_test_pattern_fops},
#ifdef CONFIG_DRM_AMD_DC_HDCP
{"hdcp_sink_capability", &hdcp_sink_capability_fops},
aconnector = to_amdgpu_dm_connector(connector);
+ /* Ensure we're only dumping the topology of a root mst node */
+ if (!aconnector->mst_mgr.mst_state)
+ continue;
+
seq_printf(m, "\nMST topology for connector %d\n", aconnector->connector_id);
drm_dp_mst_dump_topology(m, &aconnector->mst_mgr);
}
}
/*
- * Sets the force_timing_sync debug optino from the given string.
+ * Sets trigger hpd for MST topologies.
+ * All connected connectors will be rediscovered and re started as needed if val of 1 is sent.
+ * All topologies will be disconnected if val of 0 is set .
+ * Usage to enable topologies: echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_trigger_hpd_mst
+ * Usage to disable topologies: echo 0 > /sys/kernel/debug/dri/0/amdgpu_dm_trigger_hpd_mst
+ */
+static int trigger_hpd_mst_set(void *data, u64 val)
+{
+ struct amdgpu_device *adev = data;
+ struct drm_device *dev = adev_to_drm(adev);
+ struct drm_connector_list_iter iter;
+ struct amdgpu_dm_connector *aconnector;
+ struct drm_connector *connector;
+ struct dc_link *link = NULL;
+
+ if (val == 1) {
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter) {
+ aconnector = to_amdgpu_dm_connector(connector);
+ if (aconnector->dc_link->type == dc_connection_mst_branch &&
+ aconnector->mst_mgr.aux) {
+ dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD);
+ drm_dp_mst_topology_mgr_set_mst(&aconnector->mst_mgr, true);
+ }
+ }
+ } else if (val == 0) {
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter) {
+ aconnector = to_amdgpu_dm_connector(connector);
+ if (!aconnector->dc_link)
+ continue;
+
+ if (!(aconnector->port && &aconnector->mst_port->mst_mgr))
+ continue;
+
+ link = aconnector->dc_link;
+ dp_receiver_power_ctrl(link, false);
+ drm_dp_mst_topology_mgr_set_mst(&aconnector->mst_port->mst_mgr, false);
+ link->mst_stream_alloc_table.stream_count = 0;
+ memset(link->mst_stream_alloc_table.stream_allocations, 0,
+ sizeof(link->mst_stream_alloc_table.stream_allocations));
+ }
+ } else {
+ return 0;
+ }
+ drm_kms_helper_hotplug_event(dev);
+
+ return 0;
+}
+
+/*
+ * The interface doesn't need get function, so it will return the
+ * value of zero
+ * Usage: cat /sys/kernel/debug/dri/0/amdgpu_dm_trigger_hpd_mst
+ */
+static int trigger_hpd_mst_get(void *data, u64 *val)
+{
+ *val = 0;
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(trigger_hpd_mst_ops, trigger_hpd_mst_get,
+ trigger_hpd_mst_set, "%llu\n");
+
+
+/*
+ * Sets the force_timing_sync debug option from the given string.
* All connected displays will be force synchronized immediately.
* Usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_force_timing_sync
*/
DEFINE_DEBUGFS_ATTRIBUTE(visual_confirm_fops, visual_confirm_get,
visual_confirm_set, "%llu\n");
+/*
+ * Dumps the DCC_EN bit for each pipe.
+ * Example usage: cat /sys/kernel/debug/dri/0/amdgpu_dm_dcc_en
+ */
+static ssize_t dcc_en_bits_read(
+ struct file *f,
+ char __user *buf,
+ size_t size,
+ loff_t *pos)
+{
+ struct amdgpu_device *adev = file_inode(f)->i_private;
+ struct dc *dc = adev->dm.dc;
+ char *rd_buf = NULL;
+ const uint32_t rd_buf_size = 32;
+ uint32_t result = 0;
+ int offset = 0;
+ int num_pipes = dc->res_pool->pipe_count;
+ int *dcc_en_bits;
+ int i, r;
+
+ dcc_en_bits = kcalloc(num_pipes, sizeof(int), GFP_KERNEL);
+ if (!dcc_en_bits)
+ return -ENOMEM;
+
+ if (!dc->hwss.get_dcc_en_bits) {
+ kfree(dcc_en_bits);
+ return 0;
+ }
+
+ dc->hwss.get_dcc_en_bits(dc, dcc_en_bits);
+
+ rd_buf = kcalloc(rd_buf_size, sizeof(char), GFP_KERNEL);
+ if (!rd_buf)
+ return -ENOMEM;
+
+ for (i = 0; i < num_pipes; i++)
+ offset += snprintf(rd_buf + offset, rd_buf_size - offset,
+ "%d ", dcc_en_bits[i]);
+ rd_buf[strlen(rd_buf)] = '\n';
+
+ kfree(dcc_en_bits);
+
+ while (size) {
+ if (*pos >= rd_buf_size)
+ break;
+ r = put_user(*(rd_buf + result), buf);
+ if (r)
+ return r; /* r = -EFAULT */
+ buf += 1;
+ size -= 1;
+ *pos += 1;
+ result += 1;
+ }
+
+ kfree(rd_buf);
+ return result;
+}
+
void dtn_debugfs_init(struct amdgpu_device *adev)
{
static const struct file_operations dtn_log_fops = {
.write = dtn_log_write,
.llseek = default_llseek
};
+ static const struct file_operations dcc_en_bits_fops = {
+ .owner = THIS_MODULE,
+ .read = dcc_en_bits_read,
+ .llseek = default_llseek
+ };
struct drm_minor *minor = adev_to_drm(adev)->primary;
struct dentry *root = minor->debugfs_root;
debugfs_create_file_unsafe("amdgpu_dm_dmcub_trace_event_en", 0644, root,
adev, &dmcub_trace_event_state_fops);
+
+ debugfs_create_file_unsafe("amdgpu_dm_trigger_hpd_mst", 0644, root,
+ adev, &trigger_hpd_mst_ops);
+
+ debugfs_create_file_unsafe("amdgpu_dm_dcc_en", 0644, root, adev,
+ &dcc_en_bits_fops);
}
psp_set_srm(hdcp_work->hdcp.config.psp.handle, hdcp_work->srm, hdcp_work->srm_size,
&hdcp_work->srm_version);
- display->adjust.disable = 0;
+ display->adjust.disable = MOD_HDCP_DISPLAY_NOT_DISABLE;
if (content_type == DRM_MODE_HDCP_CONTENT_TYPE0) {
hdcp_w->link.adjust.hdcp1.disable = 0;
hdcp_w->link.adjust.hdcp2.force_type = MOD_HDCP_FORCE_TYPE_0;
schedule_delayed_work(&hdcp_w->property_validate_dwork,
msecs_to_jiffies(DRM_HDCP_CHECK_PERIOD_MS));
} else {
- display->adjust.disable = 1;
+ display->adjust.disable = MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION;
hdcp_w->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF;
cancel_delayed_work(&hdcp_w->property_validate_dwork);
}
link->dp.rev = aconnector->dc_link->dpcd_caps.dpcd_rev.raw;
link->dp.assr_enabled = config->assr_enabled;
link->dp.mst_enabled = config->mst_enabled;
- display->adjust.disable = 1;
+ display->adjust.disable = MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION;
link->adjust.auth_delay = 3;
link->adjust.hdcp1.disable = 0;
bool dm_helpers_dmub_outbox0_interrupt_control(struct dc_context *ctx, bool enable)
{
- // TODO
- return true;
+ enum dc_irq_source irq_source;
+ bool ret;
+
+ irq_source = DC_IRQ_SOURCE_DMCUB_OUTBOX0;
+
+ ret = dc_interrupt_set(ctx->dc, irq_source, enable);
+
+ DRM_DEBUG_DRIVER("Dmub trace irq %sabling: r=%d\n",
+ enable ? "en" : "dis", ret);
+ return ret;
}
* @handler_arg: Argument passed to the handler when triggered
* @dm: DM which this handler belongs to
* @irq_source: DC interrupt source that this handler is registered for
+ * @work: work struct
*/
struct amdgpu_dm_irq_handler_data {
struct list_head list;
return hnd_list;
}
+/**
+ * unregister_all_irq_handlers() - Cleans up handlers from the DM IRQ table
+ * @adev: The base driver device containing the DM device
+ *
+ * Go through low and high context IRQ tables and deallocate handlers.
+ */
+static void unregister_all_irq_handlers(struct amdgpu_device *adev)
+{
+ struct list_head *hnd_list_low;
+ struct list_head *hnd_list_high;
+ struct list_head *entry, *tmp;
+ struct amdgpu_dm_irq_handler_data *handler;
+ unsigned long irq_table_flags;
+ int i;
+
+ DM_IRQ_TABLE_LOCK(adev, irq_table_flags);
+
+ for (i = 0; i < DAL_IRQ_SOURCES_NUMBER; i++) {
+ hnd_list_low = &adev->dm.irq_handler_list_low_tab[i];
+ hnd_list_high = &adev->dm.irq_handler_list_high_tab[i];
+
+ list_for_each_safe(entry, tmp, hnd_list_low) {
+
+ handler = list_entry(entry, struct amdgpu_dm_irq_handler_data,
+ list);
+
+ if (handler == NULL || handler->handler == NULL)
+ continue;
+
+ list_del(&handler->list);
+ kfree(handler);
+ }
+
+ list_for_each_safe(entry, tmp, hnd_list_high) {
+
+ handler = list_entry(entry, struct amdgpu_dm_irq_handler_data,
+ list);
+
+ if (handler == NULL || handler->handler == NULL)
+ continue;
+
+ list_del(&handler->list);
+ kfree(handler);
+ }
+ }
+
+ DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
+}
+
static bool
validate_irq_registration_params(struct dc_interrupt_params *int_params,
void (*ih)(void *))
}
}
}
+ /* Deallocate handlers from the table. */
+ unregister_all_irq_handlers(adev);
}
int amdgpu_dm_irq_suspend(struct amdgpu_device *adev)
__func__);
}
+static int amdgpu_dm_set_dmub_trace_irq_state(struct amdgpu_device *adev,
+ struct amdgpu_irq_src *source,
+ unsigned int type,
+ enum amdgpu_interrupt_state state)
+{
+ enum dc_irq_source irq_source = DC_IRQ_SOURCE_DMCUB_OUTBOX0;
+ bool st = (state == AMDGPU_IRQ_STATE_ENABLE);
+
+ dc_interrupt_set(adev->dm.dc, irq_source, st);
+ return 0;
+}
+
static const struct amdgpu_irq_src_funcs dm_crtc_irq_funcs = {
.set = amdgpu_dm_set_crtc_irq_state,
.process = amdgpu_dm_irq_handler,
.process = amdgpu_dm_irq_handler,
};
+static const struct amdgpu_irq_src_funcs dm_dmub_trace_irq_funcs = {
+ .set = amdgpu_dm_set_dmub_trace_irq_state,
+ .process = amdgpu_dm_irq_handler,
+};
+
static const struct amdgpu_irq_src_funcs dm_pageflip_irq_funcs = {
.set = amdgpu_dm_set_pflip_irq_state,
.process = amdgpu_dm_irq_handler,
adev->vupdate_irq.num_types = adev->mode_info.num_crtc;
adev->vupdate_irq.funcs = &dm_vupdate_irq_funcs;
+ adev->dmub_trace_irq.num_types = 1;
+ adev->dmub_trace_irq.funcs = &dm_dmub_trace_irq_funcs;
+
adev->pageflip_irq.num_types = adev->mode_info.num_crtc;
adev->pageflip_irq.funcs = &dm_pageflip_irq_funcs;
#include "amdgpu_dm_irq.h"
#include "amdgpu_pm.h"
#include "dm_pp_smu.h"
-#include "amdgpu_smu.h"
-
bool dm_pp_apply_display_requirements(
const struct dc_context *ctx,
const struct dm_pp_display_configuration *pp_display_cfg)
{
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
int i;
if (adev->pm.dpm_enabled) {
adev->powerplay.pp_funcs->display_configuration_change(
adev->powerplay.pp_handle,
&adev->pm.pm_display_cfg);
- else if (adev->smu.ppt_funcs)
- smu_display_configuration_change(smu,
- &adev->pm.pm_display_cfg);
amdgpu_pm_compute_clocks(adev);
}
}
}
-static enum smu_clk_type dc_to_smu_clock_type(
- enum dm_pp_clock_type dm_pp_clk_type)
-{
- enum smu_clk_type smu_clk_type = SMU_CLK_COUNT;
-
- switch (dm_pp_clk_type) {
- case DM_PP_CLOCK_TYPE_DISPLAY_CLK:
- smu_clk_type = SMU_DISPCLK;
- break;
- case DM_PP_CLOCK_TYPE_ENGINE_CLK:
- smu_clk_type = SMU_GFXCLK;
- break;
- case DM_PP_CLOCK_TYPE_MEMORY_CLK:
- smu_clk_type = SMU_MCLK;
- break;
- case DM_PP_CLOCK_TYPE_DCEFCLK:
- smu_clk_type = SMU_DCEFCLK;
- break;
- case DM_PP_CLOCK_TYPE_SOCCLK:
- smu_clk_type = SMU_SOCCLK;
- break;
- default:
- DRM_ERROR("DM_PPLIB: invalid clock type: %d!\n",
- dm_pp_clk_type);
- break;
- }
-
- return smu_clk_type;
-}
-
static enum amd_pp_clock_type dc_to_pp_clock_type(
enum dm_pp_clock_type dm_pp_clk_type)
{
&pp_clks);
if (ret)
return false;
- } else if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->get_clock_by_type_with_latency) {
- if (smu_get_clock_by_type_with_latency(&adev->smu,
- dc_to_smu_clock_type(clk_type),
- &pp_clks))
- return false;
}
-
pp_to_dc_clock_levels_with_latency(&pp_clks, clk_level_info, clk_type);
return true;
ret = adev->powerplay.pp_funcs->display_clock_voltage_request(
adev->powerplay.pp_handle,
&pp_clock_request);
- else if (adev->smu.ppt_funcs &&
- adev->smu.ppt_funcs->display_clock_voltage_request)
- ret = smu_display_clock_voltage_request(&adev->smu,
- &pp_clock_request);
if (ret)
return false;
return true;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- smu_set_watermarks_for_clock_ranges(&adev->smu, ranges);
+ if (pp_funcs && pp_funcs->set_watermarks_for_clocks_ranges)
+ pp_funcs->set_watermarks_for_clocks_ranges(pp_handle, ranges);
return PP_SMU_RESULT_OK;
}
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->set_active_display_count)
return PP_SMU_RESULT_UNSUPPORTED;
/* 0: successful or smu.ppt_funcs->set_display_count = NULL; 1: fail */
- if (smu_set_display_count(smu, count))
+ if (pp_funcs->set_active_display_count(pp_handle, count))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->set_min_deep_sleep_dcefclk)
return PP_SMU_RESULT_UNSUPPORTED;
/* 0: successful or smu.ppt_funcs->set_deep_sleep_dcefclk = NULL;1: fail */
- if (smu_set_deep_sleep_dcefclk(smu, mhz))
+ if (pp_funcs->set_min_deep_sleep_dcefclk(pp_handle, mhz))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
struct pp_display_clock_request clock_req;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->display_clock_voltage_request)
return PP_SMU_RESULT_UNSUPPORTED;
clock_req.clock_type = amd_pp_dcef_clock;
/* 0: successful or smu.ppt_funcs->display_clock_voltage_request = NULL
* 1: fail
*/
- if (smu_display_clock_voltage_request(smu, &clock_req))
+ if (pp_funcs->display_clock_voltage_request(pp_handle, &clock_req))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
struct pp_display_clock_request clock_req;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->display_clock_voltage_request)
return PP_SMU_RESULT_UNSUPPORTED;
clock_req.clock_type = amd_pp_mem_clock;
/* 0: successful or smu.ppt_funcs->display_clock_voltage_request = NULL
* 1: fail
*/
- if (smu_display_clock_voltage_request(smu, &clock_req))
+ if (pp_funcs->display_clock_voltage_request(pp_handle, &clock_req))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (smu_display_disable_memory_clock_switch(smu, !pstate_handshake_supported))
- return PP_SMU_RESULT_FAIL;
+ if (pp_funcs && pp_funcs->display_disable_memory_clock_switch) {
+ if (pp_funcs->display_disable_memory_clock_switch(pp_handle,
+ !pstate_handshake_supported))
+ return PP_SMU_RESULT_FAIL;
+ }
return PP_SMU_RESULT_OK;
}
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
struct pp_display_clock_request clock_req;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->display_clock_voltage_request)
return PP_SMU_RESULT_UNSUPPORTED;
switch (clock_id) {
/* 0: successful or smu.ppt_funcs->display_clock_voltage_request = NULL
* 1: fail
*/
- if (smu_display_clock_voltage_request(smu, &clock_req))
+ if (pp_funcs->display_clock_voltage_request(pp_handle, &clock_req))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
-
- if (!smu->ppt_funcs)
- return PP_SMU_RESULT_UNSUPPORTED;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs->get_max_sustainable_clocks_by_dc)
+ if (!pp_funcs || !pp_funcs->get_max_sustainable_clocks_by_dc)
return PP_SMU_RESULT_UNSUPPORTED;
- if (!smu_get_max_sustainable_clocks_by_dc(smu, max_clocks))
+ if (!pp_funcs->get_max_sustainable_clocks_by_dc(pp_handle, max_clocks))
return PP_SMU_RESULT_OK;
return PP_SMU_RESULT_FAIL;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
-
- if (!smu->ppt_funcs)
- return PP_SMU_RESULT_UNSUPPORTED;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs->get_uclk_dpm_states)
+ if (!pp_funcs || !pp_funcs->get_uclk_dpm_states)
return PP_SMU_RESULT_UNSUPPORTED;
- if (!smu_get_uclk_dpm_states(smu,
- clock_values_in_khz, num_states))
+ if (!pp_funcs->get_uclk_dpm_states(pp_handle,
+ clock_values_in_khz,
+ num_states))
return PP_SMU_RESULT_OK;
return PP_SMU_RESULT_FAIL;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
-
- if (!smu->ppt_funcs)
- return PP_SMU_RESULT_UNSUPPORTED;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs->get_dpm_clock_table)
+ if (!pp_funcs || !pp_funcs->get_dpm_clock_table)
return PP_SMU_RESULT_UNSUPPORTED;
- if (!smu_get_dpm_clock_table(smu, clock_table))
+ if (!pp_funcs->get_dpm_clock_table(pp_handle, clock_table))
return PP_SMU_RESULT_OK;
return PP_SMU_RESULT_FAIL;
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- smu_set_watermarks_for_clock_ranges(&adev->smu, ranges);
+ if (pp_funcs && pp_funcs->set_watermarks_for_clocks_ranges)
+ pp_funcs->set_watermarks_for_clocks_ranges(pp_handle, ranges);
return PP_SMU_RESULT_OK;
}
)
);
+TRACE_EVENT(amdgpu_dmub_trace_high_irq,
+ TP_PROTO(uint32_t trace_code, uint32_t tick_count, uint32_t param0,
+ uint32_t param1),
+ TP_ARGS(trace_code, tick_count, param0, param1),
+ TP_STRUCT__entry(
+ __field(uint32_t, trace_code)
+ __field(uint32_t, tick_count)
+ __field(uint32_t, param0)
+ __field(uint32_t, param1)
+ ),
+ TP_fast_assign(
+ __entry->trace_code = trace_code;
+ __entry->tick_count = tick_count;
+ __entry->param0 = param0;
+ __entry->param1 = param1;
+ ),
+ TP_printk("trace_code=%u tick_count=%u param0=%u param1=%u",
+ __entry->trace_code, __entry->tick_count,
+ __entry->param0, __entry->param1)
+);
+
+TRACE_EVENT(amdgpu_refresh_rate_track,
+ TP_PROTO(int crtc_index, ktime_t refresh_rate_ns, uint32_t refresh_rate_hz),
+ TP_ARGS(crtc_index, refresh_rate_ns, refresh_rate_hz),
+ TP_STRUCT__entry(
+ __field(int, crtc_index)
+ __field(ktime_t, refresh_rate_ns)
+ __field(uint32_t, refresh_rate_hz)
+ ),
+ TP_fast_assign(
+ __entry->crtc_index = crtc_index;
+ __entry->refresh_rate_ns = refresh_rate_ns;
+ __entry->refresh_rate_hz = refresh_rate_hz;
+ ),
+ TP_printk("crtc_index=%d refresh_rate=%dHz (%lld)",
+ __entry->crtc_index,
+ __entry->refresh_rate_hz,
+ __entry->refresh_rate_ns)
+);
+
#endif /* _AMDGPU_DM_TRACE_H_ */
#undef TRACE_INCLUDE_PATH
include $(AMD_DC)
-DISPLAY_CORE = dc.o dc_stat.o dc_link.o dc_resource.o dc_hw_sequencer.o dc_sink.o \
-dc_surface.o dc_link_hwss.o dc_link_dp.o dc_link_ddc.o dc_debug.o dc_stream.o
+DISPLAY_CORE = dc.o dc_link.o dc_resource.o dc_hw_sequencer.o dc_sink.o \
+dc_surface.o dc_link_hwss.o dc_link_dp.o dc_link_ddc.o dc_debug.o dc_stream.o \
+dc_link_enc_cfg.o
ifdef CONFIG_DRM_AMD_DC_DCN
DISPLAY_CORE += dc_vm_helper.o
return result;
}
+static enum bp_result get_disp_caps_v4_1(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_1 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_1,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result get_disp_caps_v4_2(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_2 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_2,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result get_disp_caps_v4_3(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_3 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_3,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result get_disp_caps_v4_4(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_4 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_4,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result bios_parser_get_lttpr_interop(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps)
+{
+ struct bios_parser *bp = BP_FROM_DCB(dcb);
+ enum bp_result result = BP_RESULT_UNSUPPORTED;
+ struct atom_common_table_header *header;
+ struct atom_data_revision tbl_revision;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_UNSUPPORTED;
+
+ header = GET_IMAGE(struct atom_common_table_header,
+ DATA_TABLES(dce_info));
+ get_atom_data_table_revision(header, &tbl_revision);
+ switch (tbl_revision.major) {
+ case 4:
+ switch (tbl_revision.minor) {
+ case 1:
+ result = get_disp_caps_v4_1(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ case 2:
+ result = get_disp_caps_v4_2(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ case 3:
+ result = get_disp_caps_v4_3(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ case 4:
+ result = get_disp_caps_v4_4(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ default:
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+
+ return result;
+}
+
+static enum bp_result bios_parser_get_lttpr_caps(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps)
+{
+ struct bios_parser *bp = BP_FROM_DCB(dcb);
+ enum bp_result result = BP_RESULT_UNSUPPORTED;
+ struct atom_common_table_header *header;
+ struct atom_data_revision tbl_revision;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_UNSUPPORTED;
+
+ header = GET_IMAGE(struct atom_common_table_header,
+ DATA_TABLES(dce_info));
+ get_atom_data_table_revision(header, &tbl_revision);
+ switch (tbl_revision.major) {
+ case 4:
+ switch (tbl_revision.minor) {
+ case 1:
+ result = get_disp_caps_v4_1(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ case 2:
+ result = get_disp_caps_v4_2(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ case 3:
+ result = get_disp_caps_v4_3(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ case 4:
+ result = get_disp_caps_v4_4(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ default:
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+
+ return result;
+}
+
static enum bp_result get_embedded_panel_info_v2_1(
struct bios_parser *bp,
struct embedded_panel_info *info)
.get_soc_bb_info = bios_parser_get_soc_bb_info,
.get_disp_connector_caps_info = bios_parser_get_disp_connector_caps_info,
+
+ .get_lttpr_caps = bios_parser_get_lttpr_caps,
+
+ .get_lttpr_interop = bios_parser_get_lttpr_interop,
};
static bool bios_parser2_construct(
int32_t num_cursor_lines;
int32_t i, j, k;
- struct bw_fixed yclk[3];
- struct bw_fixed sclk[8];
+ struct bw_fixed *yclk;
+ struct bw_fixed *sclk;
bool d0_underlay_enable;
bool d1_underlay_enable;
bool fbc_enabled;
bool lpt_enabled;
enum bw_defines sclk_message;
enum bw_defines yclk_message;
- enum bw_defines tiling_mode[maximum_number_of_surfaces];
- enum bw_defines surface_type[maximum_number_of_surfaces];
+ enum bw_defines *tiling_mode;
+ enum bw_defines *surface_type;
enum bw_defines voltage;
enum bw_defines pipe_check;
enum bw_defines hsr_check;
int32_t number_of_displays_enabled_with_margin = 0;
int32_t number_of_aligned_displays_with_no_margin = 0;
+ yclk = kcalloc(3, sizeof(*yclk), GFP_KERNEL);
+ if (!yclk)
+ return;
+
+ sclk = kcalloc(8, sizeof(*sclk), GFP_KERNEL);
+ if (!sclk)
+ goto free_yclk;
+
+ tiling_mode = kcalloc(maximum_number_of_surfaces, sizeof(*tiling_mode), GFP_KERNEL);
+ if (!tiling_mode)
+ goto free_sclk;
+
+ surface_type = kcalloc(maximum_number_of_surfaces, sizeof(*surface_type), GFP_KERNEL);
+ if (!surface_type)
+ goto free_tiling_mode;
+
yclk[low] = vbios->low_yclk;
yclk[mid] = vbios->mid_yclk;
yclk[high] = vbios->high_yclk;
}
}
}
+
+ kfree(surface_type);
+free_tiling_mode:
+ kfree(tiling_mode);
+free_yclk:
+ kfree(yclk);
+free_sclk:
+ kfree(sclk);
}
/*******************************************************************************
struct bw_calcs_vbios *bw_vbios,
struct hw_asic_id asic_id)
{
- struct bw_calcs_dceip dceip = { 0 };
- struct bw_calcs_vbios vbios = { 0 };
+ struct bw_calcs_dceip *dceip;
+ struct bw_calcs_vbios *vbios;
enum bw_calcs_version version = bw_calcs_version_from_asic_id(asic_id);
- dceip.version = version;
+ dceip = kzalloc(sizeof(*dceip), GFP_KERNEL);
+ if (!dceip)
+ return;
+
+ vbios = kzalloc(sizeof(*vbios), GFP_KERNEL);
+ if (!vbios) {
+ kfree(dceip);
+ return;
+ }
+
+ dceip->version = version;
switch (version) {
case BW_CALCS_VERSION_CARRIZO:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 64;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(1600);
- vbios.mid_yclk = bw_int_to_fixed(1600);
- vbios.low_yclk = bw_frc_to_fixed(66666, 100);
- vbios.low_sclk = bw_int_to_fixed(200);
- vbios.mid1_sclk = bw_int_to_fixed(300);
- vbios.mid2_sclk = bw_int_to_fixed(300);
- vbios.mid3_sclk = bw_int_to_fixed(300);
- vbios.mid4_sclk = bw_int_to_fixed(300);
- vbios.mid5_sclk = bw_int_to_fixed(300);
- vbios.mid6_sclk = bw_int_to_fixed(300);
- vbios.high_sclk = bw_frc_to_fixed(62609, 100);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(352);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(467);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(643);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(50);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
- vbios.stutter_self_refresh_exit_latency = bw_frc_to_fixed(153, 10);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_frc_to_fixed(19649, 1000);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 3;
- dceip.number_of_underlay_pipes = 1;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = false;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 64;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(1600);
+ vbios->mid_yclk = bw_int_to_fixed(1600);
+ vbios->low_yclk = bw_frc_to_fixed(66666, 100);
+ vbios->low_sclk = bw_int_to_fixed(200);
+ vbios->mid1_sclk = bw_int_to_fixed(300);
+ vbios->mid2_sclk = bw_int_to_fixed(300);
+ vbios->mid3_sclk = bw_int_to_fixed(300);
+ vbios->mid4_sclk = bw_int_to_fixed(300);
+ vbios->mid5_sclk = bw_int_to_fixed(300);
+ vbios->mid6_sclk = bw_int_to_fixed(300);
+ vbios->high_sclk = bw_frc_to_fixed(62609, 100);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(352);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(467);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(643);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(50);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->stutter_self_refresh_exit_latency = bw_frc_to_fixed(153, 10);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_frc_to_fixed(19649, 1000);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 3;
+ dceip->number_of_underlay_pipes = 1;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = false;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 2;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(82176);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = false;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 2;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(82176);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = false;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(0);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0); /* todo: this is a bug*/
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0); /* todo: this is a bug*/
break;
case BW_CALCS_VERSION_POLARIS10:
/* TODO: Treat VEGAM the same as P10 for now
* Need to tune the para for VEGAM if needed */
case BW_CALCS_VERSION_VEGAM:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 32;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(6000);
- vbios.mid_yclk = bw_int_to_fixed(3200);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(300);
- vbios.mid1_sclk = bw_int_to_fixed(400);
- vbios.mid2_sclk = bw_int_to_fixed(500);
- vbios.mid3_sclk = bw_int_to_fixed(600);
- vbios.mid4_sclk = bw_int_to_fixed(700);
- vbios.mid5_sclk = bw_int_to_fixed(800);
- vbios.mid6_sclk = bw_int_to_fixed(974);
- vbios.high_sclk = bw_int_to_fixed(1154);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(459);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(654);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1108);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_int_to_fixed(45);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 6;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 32;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(6000);
+ vbios->mid_yclk = bw_int_to_fixed(3200);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(300);
+ vbios->mid1_sclk = bw_int_to_fixed(400);
+ vbios->mid2_sclk = bw_int_to_fixed(500);
+ vbios->mid3_sclk = bw_int_to_fixed(600);
+ vbios->mid4_sclk = bw_int_to_fixed(700);
+ vbios->mid5_sclk = bw_int_to_fixed(800);
+ vbios->mid6_sclk = bw_int_to_fixed(974);
+ vbios->high_sclk = bw_int_to_fixed(1154);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(459);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(654);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1108);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(45);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 6;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_POLARIS11:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 32;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(6000);
- vbios.mid_yclk = bw_int_to_fixed(3200);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(300);
- vbios.mid1_sclk = bw_int_to_fixed(400);
- vbios.mid2_sclk = bw_int_to_fixed(500);
- vbios.mid3_sclk = bw_int_to_fixed(600);
- vbios.mid4_sclk = bw_int_to_fixed(700);
- vbios.mid5_sclk = bw_int_to_fixed(800);
- vbios.mid6_sclk = bw_int_to_fixed(974);
- vbios.high_sclk = bw_int_to_fixed(1154);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(459);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(654);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1108);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- if (vbios.number_of_dram_channels == 2) // 64-bit
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 32;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(6000);
+ vbios->mid_yclk = bw_int_to_fixed(3200);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(300);
+ vbios->mid1_sclk = bw_int_to_fixed(400);
+ vbios->mid2_sclk = bw_int_to_fixed(500);
+ vbios->mid3_sclk = bw_int_to_fixed(600);
+ vbios->mid4_sclk = bw_int_to_fixed(700);
+ vbios->mid5_sclk = bw_int_to_fixed(800);
+ vbios->mid6_sclk = bw_int_to_fixed(974);
+ vbios->high_sclk = bw_int_to_fixed(1154);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(459);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(654);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1108);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ if (vbios->number_of_dram_channels == 2) // 64-bit
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
else
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_int_to_fixed(45);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 5;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(45);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 5;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_POLARIS12:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 32;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(6000);
- vbios.mid_yclk = bw_int_to_fixed(3200);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(678);
- vbios.mid1_sclk = bw_int_to_fixed(864);
- vbios.mid2_sclk = bw_int_to_fixed(900);
- vbios.mid3_sclk = bw_int_to_fixed(920);
- vbios.mid4_sclk = bw_int_to_fixed(940);
- vbios.mid5_sclk = bw_int_to_fixed(960);
- vbios.mid6_sclk = bw_int_to_fixed(980);
- vbios.high_sclk = bw_int_to_fixed(1049);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(459);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(654);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1108);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- if (vbios.number_of_dram_channels == 2) // 64-bit
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 32;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(6000);
+ vbios->mid_yclk = bw_int_to_fixed(3200);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(678);
+ vbios->mid1_sclk = bw_int_to_fixed(864);
+ vbios->mid2_sclk = bw_int_to_fixed(900);
+ vbios->mid3_sclk = bw_int_to_fixed(920);
+ vbios->mid4_sclk = bw_int_to_fixed(940);
+ vbios->mid5_sclk = bw_int_to_fixed(960);
+ vbios->mid6_sclk = bw_int_to_fixed(980);
+ vbios->high_sclk = bw_int_to_fixed(1049);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(459);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(654);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1108);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ if (vbios->number_of_dram_channels == 2) // 64-bit
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
else
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_int_to_fixed(250);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = false;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 5;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = true;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(250);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = false;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 5;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = true;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_STONEY:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 64;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(1866);
- vbios.mid_yclk = bw_int_to_fixed(1866);
- vbios.low_yclk = bw_int_to_fixed(1333);
- vbios.low_sclk = bw_int_to_fixed(200);
- vbios.mid1_sclk = bw_int_to_fixed(600);
- vbios.mid2_sclk = bw_int_to_fixed(600);
- vbios.mid3_sclk = bw_int_to_fixed(600);
- vbios.mid4_sclk = bw_int_to_fixed(600);
- vbios.mid5_sclk = bw_int_to_fixed(600);
- vbios.mid6_sclk = bw_int_to_fixed(600);
- vbios.high_sclk = bw_int_to_fixed(800);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(352);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(467);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(643);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(50);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
- vbios.stutter_self_refresh_exit_latency = bw_frc_to_fixed(158, 10);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_frc_to_fixed(2008, 100);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 2;
- dceip.number_of_underlay_pipes = 1;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 64;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(1866);
+ vbios->mid_yclk = bw_int_to_fixed(1866);
+ vbios->low_yclk = bw_int_to_fixed(1333);
+ vbios->low_sclk = bw_int_to_fixed(200);
+ vbios->mid1_sclk = bw_int_to_fixed(600);
+ vbios->mid2_sclk = bw_int_to_fixed(600);
+ vbios->mid3_sclk = bw_int_to_fixed(600);
+ vbios->mid4_sclk = bw_int_to_fixed(600);
+ vbios->mid5_sclk = bw_int_to_fixed(600);
+ vbios->mid6_sclk = bw_int_to_fixed(600);
+ vbios->high_sclk = bw_int_to_fixed(800);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(352);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(467);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(643);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(50);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->stutter_self_refresh_exit_latency = bw_frc_to_fixed(158, 10);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_frc_to_fixed(2008, 100);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 2;
+ dceip->number_of_underlay_pipes = 1;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 2;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(82176);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = false;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 2;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(82176);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = false;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(0);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_VEGA10:
- vbios.memory_type = bw_def_hbm;
- vbios.dram_channel_width_in_bits = 128;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 16;
- vbios.high_yclk = bw_int_to_fixed(2400);
- vbios.mid_yclk = bw_int_to_fixed(1700);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(300);
- vbios.mid1_sclk = bw_int_to_fixed(350);
- vbios.mid2_sclk = bw_int_to_fixed(400);
- vbios.mid3_sclk = bw_int_to_fixed(500);
- vbios.mid4_sclk = bw_int_to_fixed(600);
- vbios.mid5_sclk = bw_int_to_fixed(700);
- vbios.mid6_sclk = bw_int_to_fixed(760);
- vbios.high_sclk = bw_int_to_fixed(776);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(460);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(670);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1133);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_frc_to_fixed(75, 10);
- vbios.stutter_self_refresh_entry_latency = bw_frc_to_fixed(19, 10);
- vbios.nbp_state_change_latency = bw_int_to_fixed(39);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = false;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 8;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(2304);
- dceip.dmif_pipe_en_fbc_chunk_tracker = true;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 6;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = true;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_hbm;
+ vbios->dram_channel_width_in_bits = 128;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 16;
+ vbios->high_yclk = bw_int_to_fixed(2400);
+ vbios->mid_yclk = bw_int_to_fixed(1700);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(300);
+ vbios->mid1_sclk = bw_int_to_fixed(350);
+ vbios->mid2_sclk = bw_int_to_fixed(400);
+ vbios->mid3_sclk = bw_int_to_fixed(500);
+ vbios->mid4_sclk = bw_int_to_fixed(600);
+ vbios->mid5_sclk = bw_int_to_fixed(700);
+ vbios->mid6_sclk = bw_int_to_fixed(760);
+ vbios->high_sclk = bw_int_to_fixed(776);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(460);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(670);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1133);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_frc_to_fixed(75, 10);
+ vbios->stutter_self_refresh_entry_latency = bw_frc_to_fixed(19, 10);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(39);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = false;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 8;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(2304);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = true;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 6;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = true;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 24576;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = false;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 24576;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = false;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
default:
break;
}
- *bw_dceip = dceip;
- *bw_vbios = vbios;
+ *bw_dceip = *dceip;
+ *bw_vbios = *vbios;
+ kfree(dceip);
+ kfree(vbios);
}
/*
{
struct hw_asic_id asic_id = ctx->asic_id;
- struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
-
- if (clk_mgr == NULL) {
- BREAK_TO_DEBUGGER();
- return NULL;
- }
-
switch (asic_id.chip_family) {
#if defined(CONFIG_DRM_AMD_DC_SI)
- case FAMILY_SI:
+ case FAMILY_SI: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
dce60_clk_mgr_construct(ctx, clk_mgr);
- break;
+ dce_clk_mgr_construct(ctx, clk_mgr);
+ return &clk_mgr->base;
+ }
#endif
case FAMILY_CI:
- case FAMILY_KV:
+ case FAMILY_KV: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
dce_clk_mgr_construct(ctx, clk_mgr);
- break;
- case FAMILY_CZ:
+ return &clk_mgr->base;
+ }
+ case FAMILY_CZ: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
dce110_clk_mgr_construct(ctx, clk_mgr);
- break;
- case FAMILY_VI:
+ return &clk_mgr->base;
+ }
+ case FAMILY_VI: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
if (ASIC_REV_IS_TONGA_P(asic_id.hw_internal_rev) ||
ASIC_REV_IS_FIJI_P(asic_id.hw_internal_rev)) {
dce_clk_mgr_construct(ctx, clk_mgr);
- break;
+ return &clk_mgr->base;
}
if (ASIC_REV_IS_POLARIS10_P(asic_id.hw_internal_rev) ||
ASIC_REV_IS_POLARIS11_M(asic_id.hw_internal_rev) ||
ASIC_REV_IS_POLARIS12_V(asic_id.hw_internal_rev)) {
dce112_clk_mgr_construct(ctx, clk_mgr);
- break;
+ return &clk_mgr->base;
}
if (ASIC_REV_IS_VEGAM(asic_id.hw_internal_rev)) {
dce112_clk_mgr_construct(ctx, clk_mgr);
- break;
+ return &clk_mgr->base;
+ }
+ return &clk_mgr->base;
+ }
+ case FAMILY_AI: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
}
- break;
- case FAMILY_AI:
if (ASICREV_IS_VEGA20_P(asic_id.hw_internal_rev))
dce121_clk_mgr_construct(ctx, clk_mgr);
else
dce120_clk_mgr_construct(ctx, clk_mgr);
- break;
-
+ return &clk_mgr->base;
+ }
#if defined(CONFIG_DRM_AMD_DC_DCN)
- case FAMILY_RV:
+ case FAMILY_RV: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
+
if (ASICREV_IS_RENOIR(asic_id.hw_internal_rev)) {
rn_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_GREEN_SARDINE(asic_id.hw_internal_rev)) {
rn_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_RAVEN2(asic_id.hw_internal_rev)) {
rv2_clk_mgr_construct(ctx, clk_mgr, pp_smu);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_RAVEN(asic_id.hw_internal_rev) ||
ASICREV_IS_PICASSO(asic_id.hw_internal_rev)) {
rv1_clk_mgr_construct(ctx, clk_mgr, pp_smu);
- break;
+ return &clk_mgr->base;
}
- break;
+ return &clk_mgr->base;
+ }
+ case FAMILY_NV: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
- case FAMILY_NV:
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
if (ASICREV_IS_SIENNA_CICHLID_P(asic_id.hw_internal_rev)) {
dcn3_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_DIMGREY_CAVEFISH_P(asic_id.hw_internal_rev)) {
dcn3_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
dcn20_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
-
+ return &clk_mgr->base;
+ }
case FAMILY_VGH:
- if (ASICREV_IS_VANGOGH(asic_id.hw_internal_rev))
+ if (ASICREV_IS_VANGOGH(asic_id.hw_internal_rev)) {
+ struct clk_mgr_vgh *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
vg_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
+ return &clk_mgr->base.base;
+ }
break;
#endif
default:
break;
}
- return &clk_mgr->base;
+ return NULL;
}
void dc_destroy_clk_mgr(struct clk_mgr *clk_mgr_base)
if (ASICREV_IS_SIENNA_CICHLID_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
dcn3_clk_mgr_destroy(clk_mgr);
}
+ if (ASICREV_IS_DIMGREY_CAVEFISH_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
+ dcn3_clk_mgr_destroy(clk_mgr);
+ }
break;
case FAMILY_VGH:
},
}
};
+static unsigned int find_socclk_for_voltage(struct dpm_clocks *clock_table, unsigned int voltage)
+{
+ int i;
+
+ for (i = 0; i < PP_SMU_NUM_SOCCLK_DPM_LEVELS; i++) {
+ if (clock_table->SocClocks[i].Vol == voltage)
+ return clock_table->SocClocks[i].Freq;
+ }
+ ASSERT(0);
+ return 0;
+}
static unsigned int find_dcfclk_for_voltage(struct dpm_clocks *clock_table, unsigned int voltage)
{
int i;
bw_params->clk_table.entries[i].memclk_mhz = clock_table->MemClocks[j].Freq;
bw_params->clk_table.entries[i].voltage = clock_table->FClocks[j].Vol;
bw_params->clk_table.entries[i].dcfclk_mhz = find_dcfclk_for_voltage(clock_table, clock_table->FClocks[j].Vol);
+ bw_params->clk_table.entries[i].socclk_mhz = find_socclk_for_voltage(clock_table,
+ bw_params->clk_table.entries[i].voltage);
}
bw_params->vram_type = bios_info->memory_type;
clk_mgr->base.ctx->dc, clk_mgr_base->bw_params);
}
+static bool dcn3_is_smu_prsent(struct clk_mgr *clk_mgr_base)
+{
+ struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
+ return clk_mgr->smu_present;
+}
+
static bool dcn3_are_clock_states_equal(struct dc_clocks *a,
struct dc_clocks *b)
{
.are_clock_states_equal = dcn3_are_clock_states_equal,
.enable_pme_wa = dcn3_enable_pme_wa,
.notify_link_rate_change = dcn30_notify_link_rate_change,
+ .is_smu_present = dcn3_is_smu_prsent
};
static void dcn3_init_clocks_fpga(struct clk_mgr *clk_mgr)
int dcn301_smu_get_smu_version(struct clk_mgr_internal *clk_mgr)
{
- return dcn301_smu_send_msg_with_param(
- clk_mgr,
- VBIOSSMC_MSG_GetSmuVersion,
- 0);
+ int smu_version = dcn301_smu_send_msg_with_param(clk_mgr,
+ VBIOSSMC_MSG_GetSmuVersion,
+ 0);
+
+ DC_LOG_DEBUG("%s %x\n", __func__, smu_version);
+
+ return smu_version;
}
{
int actual_dispclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_dispclk_khz);
+
/* Unit of SMU msg parameter is Mhz */
actual_dispclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
{
int actual_dprefclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s %d\n", __func__, clk_mgr->base.dprefclk_khz / 1000);
+
actual_dprefclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDprefclkFreq,
{
int actual_dcfclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_dcfclk_khz);
+
actual_dcfclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetHardMinDcfclkByFreq,
{
int actual_min_ds_dcfclk_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_min_ds_dcfclk_khz);
+
actual_min_ds_dcfclk_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetMinDeepSleepDcfclk,
{
int actual_dppclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_dpp_khz);
+
actual_dppclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDppclkFreq,
{
//TODO: Work with smu team to define optimization options.
+ DC_LOG_DEBUG("%s(%x)\n", __func__, idle_info);
+
dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDisplayIdleOptimizations,
idle_info.idle_info.phy_ref_clk_off = 1;
}
+ DC_LOG_DEBUG("%s(%d)\n", __func__, enable);
+
dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDisplayIdleOptimizations,
void dcn301_smu_set_dram_addr_high(struct clk_mgr_internal *clk_mgr, uint32_t addr_high)
{
+ DC_LOG_DEBUG("%s(%x)\n", __func__, addr_high);
+
dcn301_smu_send_msg_with_param(clk_mgr,
VBIOSSMC_MSG_SetVbiosDramAddrHigh, addr_high);
}
void dcn301_smu_set_dram_addr_low(struct clk_mgr_internal *clk_mgr, uint32_t addr_low)
{
+ DC_LOG_DEBUG("%s(%x)\n", __func__, addr_low);
+
dcn301_smu_send_msg_with_param(clk_mgr,
VBIOSSMC_MSG_SetVbiosDramAddrLow, addr_low);
}
// For dcn20_update_clocks_update_dpp_dto
#include "dcn20/dcn20_clk_mgr.h"
-
-
#include "vg_clk_mgr.h"
+#include "dcn301_smu.h"
#include "reg_helper.h"
#include "core_types.h"
#include "dm_helpers.h"
/* Macros */
+#define TO_CLK_MGR_VGH(clk_mgr)\
+ container_of(clk_mgr, struct clk_mgr_vgh, base)
+
#define REG(reg_name) \
(CLK_BASE.instance[0].segment[mm ## reg_name ## _BASE_IDX] + mm ## reg_name)
/* TODO: evaluate how to lower or disable all dcn clocks in screen off case */
-int vg_get_active_display_cnt_wa(
+static int vg_get_active_display_cnt_wa(
struct dc *dc,
struct dc_state *context)
{
}
}
- if (should_set_clock(safe_to_lower, new_clocks->dcfclk_khz, clk_mgr_base->clks.dcfclk_khz)) {
+ if (should_set_clock(safe_to_lower, new_clocks->dcfclk_khz, clk_mgr_base->clks.dcfclk_khz) && !dc->debug.disable_min_fclk) {
clk_mgr_base->clks.dcfclk_khz = new_clocks->dcfclk_khz;
dcn301_smu_set_hard_min_dcfclk(clk_mgr, clk_mgr_base->clks.dcfclk_khz);
}
if (should_set_clock(safe_to_lower,
- new_clocks->dcfclk_deep_sleep_khz, clk_mgr_base->clks.dcfclk_deep_sleep_khz)) {
+ new_clocks->dcfclk_deep_sleep_khz, clk_mgr_base->clks.dcfclk_deep_sleep_khz) && !dc->debug.disable_min_fclk) {
clk_mgr_base->clks.dcfclk_deep_sleep_khz = new_clocks->dcfclk_deep_sleep_khz;
dcn301_smu_set_min_deep_sleep_dcfclk(clk_mgr, clk_mgr_base->clks.dcfclk_deep_sleep_khz);
}
s->dprefclk_khz = sb.dprefclk * 1000;
}
-void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base)
+static void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
}
-void vg_notify_wm_ranges(struct clk_mgr *clk_mgr_base)
+static void vg_notify_wm_ranges(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
- struct watermarks *table = clk_mgr_base->smu_wm_set.wm_set;
+ struct clk_mgr_vgh *clk_mgr_vgh = TO_CLK_MGR_VGH(clk_mgr);
+ struct watermarks *table = clk_mgr_vgh->smu_wm_set.wm_set;
if (!clk_mgr->smu_ver)
return;
- if (!table || clk_mgr_base->smu_wm_set.mc_address.quad_part == 0)
+ if (!table || clk_mgr_vgh->smu_wm_set.mc_address.quad_part == 0)
return;
memset(table, 0, sizeof(*table));
vg_build_watermark_ranges(clk_mgr_base->bw_params, table);
dcn301_smu_set_dram_addr_high(clk_mgr,
- clk_mgr_base->smu_wm_set.mc_address.high_part);
+ clk_mgr_vgh->smu_wm_set.mc_address.high_part);
dcn301_smu_set_dram_addr_low(clk_mgr,
- clk_mgr_base->smu_wm_set.mc_address.low_part);
+ clk_mgr_vgh->smu_wm_set.mc_address.low_part);
dcn301_smu_transfer_wm_table_dram_2_smu(clk_mgr);
}
return 0;
}
-void vg_clk_mgr_helper_populate_bw_params(
+static void vg_clk_mgr_helper_populate_bw_params(
struct clk_mgr_internal *clk_mgr,
struct integrated_info *bios_info,
const struct vg_dpm_clocks *clock_table)
static struct watermarks dummy_wms = { 0 };
-void vg_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
+static void vg_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
struct smu_dpm_clks *smu_dpm_clks)
{
struct vg_dpm_clocks *table = smu_dpm_clks->dpm_clks;
void vg_clk_mgr_construct(
struct dc_context *ctx,
- struct clk_mgr_internal *clk_mgr,
+ struct clk_mgr_vgh *clk_mgr,
struct pp_smu_funcs *pp_smu,
struct dccg *dccg)
{
struct smu_dpm_clks smu_dpm_clks = { 0 };
- clk_mgr->base.ctx = ctx;
- clk_mgr->base.funcs = &vg_funcs;
+ clk_mgr->base.base.ctx = ctx;
+ clk_mgr->base.base.funcs = &vg_funcs;
- clk_mgr->pp_smu = pp_smu;
+ clk_mgr->base.pp_smu = pp_smu;
- clk_mgr->dccg = dccg;
- clk_mgr->dfs_bypass_disp_clk = 0;
+ clk_mgr->base.dccg = dccg;
+ clk_mgr->base.dfs_bypass_disp_clk = 0;
- clk_mgr->dprefclk_ss_percentage = 0;
- clk_mgr->dprefclk_ss_divider = 1000;
- clk_mgr->ss_on_dprefclk = false;
- clk_mgr->dfs_ref_freq_khz = 48000;
+ clk_mgr->base.dprefclk_ss_percentage = 0;
+ clk_mgr->base.dprefclk_ss_divider = 1000;
+ clk_mgr->base.ss_on_dprefclk = false;
+ clk_mgr->base.dfs_ref_freq_khz = 48000;
- clk_mgr->base.smu_wm_set.wm_set = (struct watermarks *)dm_helpers_allocate_gpu_mem(
- clk_mgr->base.ctx,
+ clk_mgr->smu_wm_set.wm_set = (struct watermarks *)dm_helpers_allocate_gpu_mem(
+ clk_mgr->base.base.ctx,
DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
sizeof(struct watermarks),
- &clk_mgr->base.smu_wm_set.mc_address.quad_part);
+ &clk_mgr->smu_wm_set.mc_address.quad_part);
- if (clk_mgr->base.smu_wm_set.wm_set == 0) {
- clk_mgr->base.smu_wm_set.wm_set = &dummy_wms;
- clk_mgr->base.smu_wm_set.mc_address.quad_part = 0;
+ if (clk_mgr->smu_wm_set.wm_set == 0) {
+ clk_mgr->smu_wm_set.wm_set = &dummy_wms;
+ clk_mgr->smu_wm_set.mc_address.quad_part = 0;
}
- ASSERT(clk_mgr->base.smu_wm_set.wm_set);
+ ASSERT(clk_mgr->smu_wm_set.wm_set);
smu_dpm_clks.dpm_clks = (struct vg_dpm_clocks *)dm_helpers_allocate_gpu_mem(
- clk_mgr->base.ctx,
+ clk_mgr->base.base.ctx,
DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
sizeof(struct vg_dpm_clocks),
&smu_dpm_clks.mc_address.quad_part);
if (IS_FPGA_MAXIMUS_DC(ctx->dce_environment)) {
vg_funcs.update_clocks = dcn2_update_clocks_fpga;
- clk_mgr->base.dentist_vco_freq_khz = 3600000;
+ clk_mgr->base.base.dentist_vco_freq_khz = 3600000;
} else {
struct clk_log_info log_info = {0};
- clk_mgr->smu_ver = dcn301_smu_get_smu_version(clk_mgr);
+ clk_mgr->base.smu_ver = dcn301_smu_get_smu_version(&clk_mgr->base);
- if (clk_mgr->smu_ver)
- clk_mgr->smu_present = true;
+ if (clk_mgr->base.smu_ver)
+ clk_mgr->base.smu_present = true;
/* TODO: Check we get what we expect during bringup */
- clk_mgr->base.dentist_vco_freq_khz = get_vco_frequency_from_reg(clk_mgr);
+ clk_mgr->base.base.dentist_vco_freq_khz = get_vco_frequency_from_reg(&clk_mgr->base);
/* in case we don't get a value from the register, use default */
- if (clk_mgr->base.dentist_vco_freq_khz == 0)
- clk_mgr->base.dentist_vco_freq_khz = 3600000;
+ if (clk_mgr->base.base.dentist_vco_freq_khz == 0)
+ clk_mgr->base.base.dentist_vco_freq_khz = 3600000;
if (ctx->dc_bios->integrated_info->memory_type == LpDdr5MemType) {
vg_bw_params.wm_table = lpddr5_wm_table;
vg_bw_params.wm_table = ddr4_wm_table;
}
/* Saved clocks configured at boot for debug purposes */
- vg_dump_clk_registers(&clk_mgr->base.boot_snapshot, &clk_mgr->base, &log_info);
+ vg_dump_clk_registers(&clk_mgr->base.base.boot_snapshot, &clk_mgr->base.base, &log_info);
}
- clk_mgr->base.dprefclk_khz = 600000;
- dce_clock_read_ss_info(clk_mgr);
+ clk_mgr->base.base.dprefclk_khz = 600000;
+ dce_clock_read_ss_info(&clk_mgr->base);
- clk_mgr->base.bw_params = &vg_bw_params;
+ clk_mgr->base.base.bw_params = &vg_bw_params;
- vg_get_dpm_table_from_smu(clk_mgr, &smu_dpm_clks);
+ vg_get_dpm_table_from_smu(&clk_mgr->base, &smu_dpm_clks);
if (ctx->dc_bios && ctx->dc_bios->integrated_info) {
vg_clk_mgr_helper_populate_bw_params(
- clk_mgr,
+ &clk_mgr->base,
ctx->dc_bios->integrated_info,
smu_dpm_clks.dpm_clks);
}
if (smu_dpm_clks.dpm_clks && smu_dpm_clks.mc_address.quad_part != 0)
- dm_helpers_free_gpu_mem(clk_mgr->base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
+ dm_helpers_free_gpu_mem(clk_mgr->base.base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
smu_dpm_clks.dpm_clks);
/*
- if (!IS_FPGA_MAXIMUS_DC(ctx->dce_environment) && clk_mgr->smu_ver) {
+ if (!IS_FPGA_MAXIMUS_DC(ctx->dce_environment) && clk_mgr->base.smu_ver) {
enable powerfeatures when displaycount goes to 0
dcn301_smu_enable_phy_refclk_pwrdwn(clk_mgr, !debug->disable_48mhz_pwrdwn);
}
*/
}
-void vg_clk_mgr_destroy(struct clk_mgr_internal *clk_mgr)
+void vg_clk_mgr_destroy(struct clk_mgr_internal *clk_mgr_int)
{
- if (clk_mgr->base.smu_wm_set.wm_set && clk_mgr->base.smu_wm_set.mc_address.quad_part != 0)
- dm_helpers_free_gpu_mem(clk_mgr->base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
- clk_mgr->base.smu_wm_set.wm_set);
+ struct clk_mgr_vgh *clk_mgr = TO_CLK_MGR_VGH(clk_mgr_int);
+
+ if (clk_mgr->smu_wm_set.wm_set && clk_mgr->smu_wm_set.mc_address.quad_part != 0)
+ dm_helpers_free_gpu_mem(clk_mgr_int->base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
+ clk_mgr->smu_wm_set.wm_set);
}
#ifndef __VG_CLK_MGR_H__
#define __VG_CLK_MGR_H__
+#include "clk_mgr_internal.h"
-int vg_get_active_display_cnt_wa(
- struct dc *dc,
- struct dc_state *context);
+struct watermarks;
-void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base);
+struct smu_watermark_set {
+ struct watermarks *wm_set;
+ union large_integer mc_address;
+};
+
+struct clk_mgr_vgh {
+ struct clk_mgr_internal base;
+ struct smu_watermark_set smu_wm_set;
+};
void vg_clk_mgr_construct(struct dc_context *ctx,
- struct clk_mgr_internal *clk_mgr,
+ struct clk_mgr_vgh *clk_mgr,
struct pp_smu_funcs *pp_smu,
struct dccg *dccg);
void vg_clk_mgr_destroy(struct clk_mgr_internal *clk_mgr);
-#include "dcn301_smu.h"
-void vg_notify_wm_ranges(struct clk_mgr *clk_mgr_base);
-
-void vg_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
- struct smu_dpm_clks *smu_dpm_clks);
-
-void vg_clk_mgr_helper_populate_bw_params(
- struct clk_mgr_internal *clk_mgr,
- struct integrated_info *bios_info,
- const struct vg_dpm_clocks *clock_table);
-
#endif //__VG_CLK_MGR_H__
#include "timing_generator.h"
#include "abm.h"
#include "virtual/virtual_link_encoder.h"
+#include "hubp.h"
#include "link_hwss.h"
#include "link_encoder.h"
+#include "link_enc_cfg.h"
#include "dc_link_ddc.h"
#include "dm_helpers.h"
int i = 0;
bool ret = false;
- stream->adjust = *adjust;
+ stream->adjust.v_total_max = adjust->v_total_max;
+ stream->adjust.v_total_mid = adjust->v_total_mid;
+ stream->adjust.v_total_mid_frame_num = adjust->v_total_mid_frame_num;
+ stream->adjust.v_total_min = adjust->v_total_min;
for (i = 0; i < MAX_PIPES; i++) {
struct pipe_ctx *pipe = &dc->current_state->res_ctx.pipe_ctx[i];
if (pipe->stream == stream && pipe->stream_res.tg) {
dc->hwss.set_drr(&pipe,
1,
- adjust->v_total_min,
- adjust->v_total_max,
- adjust->v_total_mid,
- adjust->v_total_mid_frame_num);
+ *adjust);
ret = true;
}
if (!create_links(dc, init_params->num_virtual_links))
goto fail;
+ /* Initialise DIG link encoder resource tracking variables. */
+ link_enc_cfg_init(dc, dc->current_state);
+
return true;
fail:
if (stream_status == NULL || stream_status->plane_count != surface_count)
overall_type = UPDATE_TYPE_FULL;
+ if (stream_update && stream_update->pending_test_pattern) {
+ overall_type = UPDATE_TYPE_FULL;
+ }
+
/* some stream updates require passive update */
if (stream_update) {
union stream_update_flags *su_flags = &stream_update->stream->update_flags;
if (update->dither_option)
stream->dither_option = *update->dither_option;
+ if (update->pending_test_pattern)
+ stream->test_pattern = *update->pending_test_pattern;
/* update current stream with writeback info */
if (update->wb_update) {
int i;
}
}
+
/* Full fe update*/
if (update_type == UPDATE_TYPE_FAST)
continue;
if (stream_update->dsc_config)
dp_update_dsc_config(pipe_ctx);
+ if (stream_update->pending_test_pattern) {
+ dc_link_dp_set_test_pattern(stream->link,
+ stream->test_pattern.type,
+ stream->test_pattern.color_space,
+ stream->test_pattern.p_link_settings,
+ stream->test_pattern.p_custom_pattern,
+ stream->test_pattern.cust_pattern_size);
+ }
+
if (stream_update->dpms_off) {
if (*stream_update->dpms_off) {
core_link_disable_stream(pipe_ctx);
}
}
+#ifdef CONFIG_DRM_AMD_DC_DCN
+ if (stream->test_pattern.type != DP_TEST_PATTERN_VIDEO_MODE) {
+ struct pipe_ctx *mpcc_pipe;
+ struct pipe_ctx *odm_pipe;
+
+ for (mpcc_pipe = top_pipe_to_program; mpcc_pipe; mpcc_pipe = mpcc_pipe->bottom_pipe)
+ for (odm_pipe = mpcc_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe)
+ odm_pipe->ttu_regs.min_ttu_vblank = MAX_TTU;
+ }
+#endif
+
if ((update_type != UPDATE_TYPE_FAST) && stream->update_flags.bits.dsc_changed)
if (top_pipe_to_program->stream_res.tg->funcs->lock_doublebuffer_enable) {
if (should_use_dmub_lock(stream->link)) {
for (j = 0; j < dc->res_pool->pipe_count; j++) {
struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[j];
+ if (!pipe_ctx->plane_state)
+ continue;
+
if (pipe_ctx->bottom_pipe || pipe_ctx->next_odm_pipe ||
!pipe_ctx->stream || pipe_ctx->stream != stream ||
!pipe_ctx->plane_state->update_flags.bits.addr_update)
if (dc->debug.disable_idle_power_optimizations)
return;
+ if (dc->clk_mgr->funcs->is_smu_present)
+ if (!dc->clk_mgr->funcs->is_smu_present(dc->clk_mgr))
+ return;
+
if (allow == dc->idle_optimizations_allowed)
return;
link->panel_cntl->funcs->destroy(&link->panel_cntl);
if (link->link_enc) {
- /* Update link encoder tracking variables. These are used for the dynamic
- * assignment of link encoders to streams.
+ /* Update link encoder resource tracking variables. These are used for
+ * the dynamic assignment of link encoders to streams. Virtual links
+ * are not assigned encoder resources on creation.
*/
- link->dc->res_pool->link_encoders[link->link_enc->preferred_engine] = NULL;
- link->dc->res_pool->dig_link_enc_count--;
+ if (link->link_id.id != CONNECTOR_ID_VIRTUAL) {
+ link->dc->res_pool->link_encoders[link->eng_id - ENGINE_ID_DIGA] = NULL;
+ link->dc->res_pool->dig_link_enc_count--;
+ }
link->link_enc->funcs->destroy(&link->link_enc);
}
link->link_id =
bios->funcs->get_connector_id(bios, init_params->connector_index);
+ link->ep_type = DISPLAY_ENDPOINT_PHY;
+
DC_LOG_DC("BIOS object table - link_id: %d", link->link_id.id);
if (bios->funcs->get_disp_connector_caps_info) {
(link->link_id.id == CONNECTOR_ID_EDP ||
link->link_id.id == CONNECTOR_ID_LVDS)) {
panel_cntl_init_data.ctx = dc_ctx;
- panel_cntl_init_data.inst = link->link_index;
+ panel_cntl_init_data.inst =
+ panel_cntl_init_data.ctx->dc_edp_id_count;
link->panel_cntl =
link->dc->res_pool->funcs->panel_cntl_create(
&panel_cntl_init_data);
+ panel_cntl_init_data.ctx->dc_edp_id_count++;
if (link->panel_cntl == NULL) {
DC_ERROR("Failed to create link panel_cntl!\n");
/* Update link encoder tracking variables. These are used for the dynamic
* assignment of link encoders to streams.
*/
- link->dc->res_pool->link_encoders[link->link_enc->preferred_engine] = link->link_enc;
+ link->eng_id = link->link_enc->preferred_engine;
+ link->dc->res_pool->link_encoders[link->eng_id - ENGINE_ID_DIGA] = link->link_enc;
link->dc->res_pool->dig_link_enc_count++;
link->link_enc_hw_inst = link->link_enc->transmitter;
static struct fixed31_32 get_pbn_from_bw_in_kbps(uint64_t kbps)
{
struct fixed31_32 peak_kbps;
- uint32_t numerator;
- uint32_t denominator;
+ uint32_t numerator = 0;
+ uint32_t denominator = 1;
/*
* margin 5300ppm + 300ppm ~ 0.6% as per spec, factor is 1.006
static inline bool is_repeater(struct dc_link *link, uint32_t offset)
{
- return (link->lttpr_non_transparent_mode && offset != 0);
+ return (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) && (offset != 0);
}
static void dpcd_set_lt_pattern_and_lane_settings(
/* 3. wait receiver to lock-on*/
wait_time_microsec = lt_settings->cr_pattern_time;
- if (link->lttpr_non_transparent_mode)
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT)
wait_time_microsec = TRAINING_AUX_RD_INTERVAL;
wait_for_training_aux_rd_interval(
if (is_max_vs_reached(lt_settings))
break;
- /* 7. same voltage*/
- /* Note: VS same for all lanes,
- * so comparing first lane is sufficient*/
- if (lt_settings->lane_settings[0].VOLTAGE_SWING ==
+ /* 7. same lane settings*/
+ /* Note: settings are the same for all lanes,
+ * so comparing first lane is sufficient*/
+ if ((lt_settings->lane_settings[0].VOLTAGE_SWING ==
req_settings.lane_settings[0].VOLTAGE_SWING)
+ && (lt_settings->lane_settings[0].PRE_EMPHASIS ==
+ req_settings.lane_settings[0].PRE_EMPHASIS))
retries_cr++;
else
retries_cr = 0;
return 0; // invalid value
}
-static void configure_lttpr_mode(struct dc_link *link)
+static void configure_lttpr_mode_transparent(struct dc_link *link)
+{
+ uint8_t repeater_mode = DP_PHY_REPEATER_MODE_TRANSPARENT;
+
+ core_link_write_dpcd(link,
+ DP_PHY_REPEATER_MODE,
+ (uint8_t *)&repeater_mode,
+ sizeof(repeater_mode));
+}
+
+static void configure_lttpr_mode_non_transparent(struct dc_link *link)
{
/* aux timeout is already set to extended */
/* RESET/SET lttpr mode to enable non transparent mode */
link->dpcd_caps.lttpr_caps.mode = repeater_mode;
}
- if (link->lttpr_non_transparent_mode) {
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) {
DC_LOG_HW_LINK_TRAINING("%s\n Set LTTPR to Non Transparent Mode\n", __func__);
<_settings);
/* Configure lttpr mode */
- if (link->lttpr_non_transparent_mode)
- configure_lttpr_mode(link);
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT)
+ configure_lttpr_mode_non_transparent(link);
+ else if (link->lttpr_mode == LTTPR_MODE_TRANSPARENT)
+ configure_lttpr_mode_transparent(link);
if (link->ctx->dc->work_arounds.lt_early_cr_pattern)
start_clock_recovery_pattern_early(link, <_settings, DPRX);
dp_set_fec_ready(link, fec_enable);
- if (link->lttpr_non_transparent_mode) {
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) {
/* 2. perform link training (set link training done
* to false is done as well)
return status;
}
+static enum dp_panel_mode try_enable_assr(struct dc_stream_state *stream)
+{
+ struct dc_link *link = stream->link;
+ enum dp_panel_mode panel_mode = dp_get_panel_mode(link);
+#ifdef CONFIG_DRM_AMD_DC_HDCP
+ struct cp_psp *cp_psp = &stream->ctx->cp_psp;
+#endif
+
+ /* ASSR must be supported on the panel */
+ if (panel_mode == DP_PANEL_MODE_DEFAULT)
+ return panel_mode;
+
+ /* eDP or internal DP only */
+ if (link->connector_signal != SIGNAL_TYPE_EDP &&
+ !(link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT &&
+ link->is_internal_display))
+ return DP_PANEL_MODE_DEFAULT;
+
+#ifdef CONFIG_DRM_AMD_DC_HDCP
+ if (cp_psp && cp_psp->funcs.enable_assr) {
+ if (!cp_psp->funcs.enable_assr(cp_psp->handle, link)) {
+ /* since eDP implies ASSR on, change panel
+ * mode to disable ASSR
+ */
+ panel_mode = DP_PANEL_MODE_DEFAULT;
+ }
+ } else
+ panel_mode = DP_PANEL_MODE_DEFAULT;
+
+#else
+ /* turn off ASSR if the implementation is not compiled in */
+ panel_mode = DP_PANEL_MODE_DEFAULT;
+#endif
+ return panel_mode;
+}
+
bool perform_link_training_with_retries(
const struct dc_link_settings *link_setting,
bool skip_video_pattern,
uint8_t delay_between_attempts = LINK_TRAINING_RETRY_DELAY;
struct dc_stream_state *stream = pipe_ctx->stream;
struct dc_link *link = stream->link;
- enum dp_panel_mode panel_mode = dp_get_panel_mode(link);
+ enum dp_panel_mode panel_mode;
/* We need to do this before the link training to ensure the idle pattern in SST
* mode will be sent right after the link training
msleep(delay_dp_power_up_in_ms);
}
-#ifdef CONFIG_DRM_AMD_DC_HDCP
- if (panel_mode == DP_PANEL_MODE_EDP) {
- struct cp_psp *cp_psp = &stream->ctx->cp_psp;
-
- if (cp_psp && cp_psp->funcs.enable_assr) {
- if (!cp_psp->funcs.enable_assr(cp_psp->handle, link)) {
- /* since eDP implies ASSR on, change panel
- * mode to disable ASSR
- */
- panel_mode = DP_PANEL_MODE_DEFAULT;
- }
- } else
- panel_mode = DP_PANEL_MODE_DEFAULT;
- }
-#endif
-
+ panel_mode = try_enable_assr(stream);
dp_set_panel_mode(link, panel_mode);
+ DC_LOG_DETECTION_DP_CAPS("Link: %d ASSR enabled: %d\n",
+ link->link_index,
+ panel_mode != DP_PANEL_MODE_DEFAULT);
if (link->aux_access_disabled) {
dc_link_dp_perform_link_training_skip_aux(link, link_setting);
return true;
- } else if (dc_link_dp_perform_link_training(
- link,
- link_setting,
- skip_video_pattern) == LINK_TRAINING_SUCCESS)
- return true;
+ } else {
+ enum link_training_result status = LINK_TRAINING_CR_FAIL_LANE0;
+
+ status = dc_link_dp_perform_link_training(
+ link,
+ link_setting,
+ skip_video_pattern);
+ if (status == LINK_TRAINING_SUCCESS)
+ return true;
+ }
/* latest link training still fail, skip delay and keep PHY on
*/
* account for lttpr repeaters cap
* notes: repeaters do not snoop in the DPRX Capabilities addresses (3.6.3).
*/
- if (link->lttpr_non_transparent_mode) {
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) {
if (link->dpcd_caps.lttpr_caps.max_lane_count < max_link_cap.lane_count)
max_link_cap.lane_count = link->dpcd_caps.lttpr_caps.max_lane_count;
max_link_cap = get_max_link_cap(link);
/* Grant extended timeout request */
- if (link->lttpr_non_transparent_mode && link->dpcd_caps.lttpr_caps.max_ext_timeout > 0) {
+ if ((link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) && (link->dpcd_caps.lttpr_caps.max_ext_timeout > 0)) {
uint8_t grant = link->dpcd_caps.lttpr_caps.max_ext_timeout & 0x80;
core_link_write_dpcd(link, DP_PHY_REPEATER_EXTENDED_WAIT_TIMEOUT, &grant, sizeof(grant));
enum dp_test_pattern test_pattern;
enum dp_test_pattern_color_space test_pattern_color_space =
DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED;
+ enum dc_color_depth requestColorDepth = COLOR_DEPTH_UNDEFINED;
+ struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx;
+ struct pipe_ctx *pipe_ctx = NULL;
+ int i;
memset(&dpcd_test_pattern, 0, sizeof(dpcd_test_pattern));
memset(&dpcd_test_params, 0, sizeof(dpcd_test_params));
+ for (i = 0; i < MAX_PIPES; i++) {
+ if (pipes[i].stream == NULL)
+ continue;
+
+ if (pipes[i].stream->link == link && !pipes[i].top_pipe && !pipes[i].prev_odm_pipe) {
+ pipe_ctx = &pipes[i];
+ break;
+ }
+ }
+
+ if (pipe_ctx == NULL)
+ return;
+
/* get link test pattern and pattern parameters */
core_link_read_dpcd(
link,
DP_TEST_PATTERN_COLOR_SPACE_YCBCR709 :
DP_TEST_PATTERN_COLOR_SPACE_YCBCR601;
+ switch (dpcd_test_params.bits.BPC) {
+ case 0: // 6 bits
+ requestColorDepth = COLOR_DEPTH_666;
+ break;
+ case 1: // 8 bits
+ requestColorDepth = COLOR_DEPTH_888;
+ break;
+ case 2: // 10 bits
+ requestColorDepth = COLOR_DEPTH_101010;
+ break;
+ case 3: // 12 bits
+ requestColorDepth = COLOR_DEPTH_121212;
+ break;
+ default:
+ break;
+ }
+
+ if (requestColorDepth != COLOR_DEPTH_UNDEFINED
+ && pipe_ctx->stream->timing.display_color_depth != requestColorDepth) {
+ DC_LOG_DEBUG("%s: original bpc %d, changing to %d\n",
+ __func__,
+ pipe_ctx->stream->timing.display_color_depth,
+ requestColorDepth);
+ pipe_ctx->stream->timing.display_color_depth = requestColorDepth;
+ dp_update_dsc_config(pipe_ctx);
+ }
+
dc_link_dp_set_test_pattern(
link,
test_pattern,
struct dp_sink_hw_fw_revision dp_hw_fw_revision;
bool is_lttpr_present = false;
const uint32_t post_oui_delay = 30; // 30ms
+ bool vbios_lttpr_enable = false;
+ bool vbios_lttpr_interop = false;
+ struct dc_bios *bios = link->dc->ctx->dc_bios;
memset(dpcd_data, '\0', sizeof(dpcd_data));
memset(lttpr_dpcd_data, '\0', sizeof(lttpr_dpcd_data));
return false;
}
- if (link->dc->caps.extended_aux_timeout_support &&
- link->dc->config.allow_lttpr_non_transparent_mode) {
+ /* Query BIOS to determine if LTTPR functionality is forced on by system */
+ if (bios->funcs->get_lttpr_caps) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_lttpr_enable = 0;
+
+ bp_query_result = bios->funcs->get_lttpr_caps(bios, &is_vbios_lttpr_enable);
+ vbios_lttpr_enable = (bp_query_result == BP_RESULT_OK) && !!is_vbios_lttpr_enable;
+ }
+
+ if (bios->funcs->get_lttpr_interop) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_interop_enabled = 0;
+
+ bp_query_result = bios->funcs->get_lttpr_interop(bios, &is_vbios_interop_enabled);
+ vbios_lttpr_interop = (bp_query_result == BP_RESULT_OK) && !!is_vbios_interop_enabled;
+ }
+
+ /*
+ * Logic to determine LTTPR mode
+ */
+ link->lttpr_mode = LTTPR_MODE_NON_LTTPR;
+ if (vbios_lttpr_enable && vbios_lttpr_interop)
+ link->lttpr_mode = LTTPR_MODE_NON_TRANSPARENT;
+ else if (!vbios_lttpr_enable && vbios_lttpr_interop) {
+ if (link->dc->config.allow_lttpr_non_transparent_mode)
+ link->lttpr_mode = LTTPR_MODE_NON_TRANSPARENT;
+ else
+ link->lttpr_mode = LTTPR_MODE_TRANSPARENT;
+ } else if (!vbios_lttpr_enable && !vbios_lttpr_interop) {
+ if (!link->dc->config.allow_lttpr_non_transparent_mode
+ || !link->dc->caps.extended_aux_timeout_support)
+ link->lttpr_mode = LTTPR_MODE_NON_LTTPR;
+ else
+ link->lttpr_mode = LTTPR_MODE_NON_TRANSPARENT;
+ }
+
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT || link->lttpr_mode == LTTPR_MODE_TRANSPARENT) {
/* By reading LTTPR capability, RX assumes that we will enable
- * LTTPR non transparent if LTTPR is present.
- * Therefore, only query LTTPR capability when both LTTPR
- * extended aux timeout and
- * non transparent mode is supported by hardware
+ * LTTPR extended aux timeout if LTTPR is present.
*/
status = core_link_read_dpcd(
link,
CONN_DATA_DETECT(link, lttpr_dpcd_data, sizeof(lttpr_dpcd_data), "LTTPR Caps: ");
}
- /* decide lttpr non transparent mode */
- link->lttpr_non_transparent_mode = is_lttpr_present;
-
if (!is_lttpr_present)
dc_link_aux_try_to_configure_timeout(link->ddc, LINK_AUX_DEFAULT_TIMEOUT_PERIOD);
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#include "link_enc_cfg.h"
+#include "resource.h"
+#include "dc_link_dp.h"
+
+/* Check whether stream is supported by DIG link encoders. */
+static bool is_dig_link_enc_stream(struct dc_stream_state *stream)
+{
+ bool is_dig_stream = false;
+ struct link_encoder *link_enc = NULL;
+ int i;
+
+ /* Loop over created link encoder objects. */
+ for (i = 0; i < stream->ctx->dc->res_pool->res_cap->num_dig_link_enc; i++) {
+ link_enc = stream->ctx->dc->res_pool->link_encoders[i];
+
+ if (link_enc &&
+ ((uint32_t)stream->signal & link_enc->output_signals)) {
+ if (dc_is_dp_signal(stream->signal)) {
+ /* DIGs do not support DP2.0 streams with 128b/132b encoding. */
+ struct dc_link_settings link_settings = {0};
+
+ decide_link_settings(stream, &link_settings);
+ if ((link_settings.link_rate >= LINK_RATE_LOW) &&
+ link_settings.link_rate <= LINK_RATE_HIGH3) {
+ is_dig_stream = true;
+ break;
+ }
+ } else {
+ is_dig_stream = true;
+ break;
+ }
+ }
+ }
+
+ return is_dig_stream;
+}
+
+/* Update DIG link encoder resource tracking variables in dc_state. */
+static void update_link_enc_assignment(
+ struct dc_state *state,
+ struct dc_stream_state *stream,
+ enum engine_id eng_id,
+ bool add_enc)
+{
+ int eng_idx;
+ int stream_idx;
+ int i;
+
+ if (eng_id != ENGINE_ID_UNKNOWN) {
+ eng_idx = eng_id - ENGINE_ID_DIGA;
+ stream_idx = -1;
+
+ /* Index of stream in dc_state used to update correct entry in
+ * link_enc_assignments table.
+ */
+ for (i = 0; i < state->stream_count; i++) {
+ if (stream == state->streams[i]) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ /* Update link encoder assignments table, link encoder availability
+ * pool and link encoder assigned to stream in state.
+ * Add/remove encoder resource to/from stream.
+ */
+ if (stream_idx != -1) {
+ if (add_enc) {
+ state->res_ctx.link_enc_assignments[stream_idx] = (struct link_enc_assignment){
+ .valid = true,
+ .ep_id = (struct display_endpoint_id) {
+ .link_id = stream->link->link_id,
+ .ep_type = stream->link->ep_type},
+ .eng_id = eng_id};
+ state->res_ctx.link_enc_avail[eng_idx] = ENGINE_ID_UNKNOWN;
+ stream->link_enc = stream->ctx->dc->res_pool->link_encoders[eng_idx];
+ } else {
+ state->res_ctx.link_enc_assignments[stream_idx].valid = false;
+ state->res_ctx.link_enc_avail[eng_idx] = eng_id;
+ stream->link_enc = NULL;
+ }
+ } else {
+ dm_output_to_console("%s: Stream not found in dc_state.\n", __func__);
+ }
+ }
+}
+
+/* Return first available DIG link encoder. */
+static enum engine_id find_first_avail_link_enc(
+ struct dc_context *ctx,
+ struct dc_state *state)
+{
+ enum engine_id eng_id = ENGINE_ID_UNKNOWN;
+ int i;
+
+ for (i = 0; i < ctx->dc->res_pool->res_cap->num_dig_link_enc; i++) {
+ eng_id = state->res_ctx.link_enc_avail[i];
+ if (eng_id != ENGINE_ID_UNKNOWN)
+ break;
+ }
+
+ return eng_id;
+}
+
+/* Return stream using DIG link encoder resource. NULL if unused. */
+static struct dc_stream_state *get_stream_using_link_enc(
+ struct dc_state *state,
+ enum engine_id eng_id)
+{
+ struct dc_stream_state *stream = NULL;
+ int stream_idx = -1;
+ int i;
+
+ for (i = 0; i < state->stream_count; i++) {
+ struct link_enc_assignment assignment = state->res_ctx.link_enc_assignments[i];
+
+ if (assignment.valid && (assignment.eng_id == eng_id)) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ if (stream_idx != -1)
+ stream = state->streams[stream_idx];
+ else
+ dm_output_to_console("%s: No stream using DIG(%d).\n", __func__, eng_id);
+
+ return stream;
+}
+
+void link_enc_cfg_init(
+ struct dc *dc,
+ struct dc_state *state)
+{
+ int i;
+
+ for (i = 0; i < dc->res_pool->res_cap->num_dig_link_enc; i++) {
+ if (dc->res_pool->link_encoders[i])
+ state->res_ctx.link_enc_avail[i] = (enum engine_id) i;
+ else
+ state->res_ctx.link_enc_avail[i] = ENGINE_ID_UNKNOWN;
+ }
+}
+
+void link_enc_cfg_link_encs_assign(
+ struct dc *dc,
+ struct dc_state *state,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count)
+{
+ enum engine_id eng_id = ENGINE_ID_UNKNOWN;
+ int i;
+
+ /* Release DIG link encoder resources before running assignment algorithm. */
+ for (i = 0; i < stream_count; i++)
+ dc->res_pool->funcs->link_enc_unassign(state, streams[i]);
+
+ /* (a) Assign DIG link encoders to physical (unmappable) endpoints first. */
+ for (i = 0; i < stream_count; i++) {
+ struct dc_stream_state *stream = streams[i];
+
+ /* Skip stream if not supported by DIG link encoder. */
+ if (!is_dig_link_enc_stream(stream))
+ continue;
+
+ /* Physical endpoints have a fixed mapping to DIG link encoders. */
+ if (!stream->link->is_dig_mapping_flexible) {
+ eng_id = stream->link->eng_id;
+ update_link_enc_assignment(state, stream, eng_id, true);
+ }
+ }
+
+ /* (b) Then assign encoders to mappable endpoints. */
+ eng_id = ENGINE_ID_UNKNOWN;
+
+ for (i = 0; i < stream_count; i++) {
+ struct dc_stream_state *stream = streams[i];
+
+ /* Skip stream if not supported by DIG link encoder. */
+ if (!is_dig_link_enc_stream(stream))
+ continue;
+
+ /* Mappable endpoints have a flexible mapping to DIG link encoders. */
+ if (stream->link->is_dig_mapping_flexible) {
+ eng_id = find_first_avail_link_enc(stream->ctx, state);
+ update_link_enc_assignment(state, stream, eng_id, true);
+ }
+ }
+}
+
+void link_enc_cfg_link_enc_unassign(
+ struct dc_state *state,
+ struct dc_stream_state *stream)
+{
+ enum engine_id eng_id = ENGINE_ID_UNKNOWN;
+
+ /* Only DIG link encoders. */
+ if (!is_dig_link_enc_stream(stream))
+ return;
+
+ if (stream->link_enc)
+ eng_id = stream->link_enc->preferred_engine;
+
+ update_link_enc_assignment(state, stream, eng_id, false);
+}
+
+bool link_enc_cfg_is_transmitter_mappable(
+ struct dc_state *state,
+ struct link_encoder *link_enc)
+{
+ bool is_mappable = false;
+ enum engine_id eng_id = link_enc->preferred_engine;
+ struct dc_stream_state *stream = get_stream_using_link_enc(state, eng_id);
+
+ if (stream)
+ is_mappable = stream->link->is_dig_mapping_flexible;
+
+ return is_mappable;
+}
+
+struct dc_link *link_enc_cfg_get_link_using_link_enc(
+ struct dc_state *state,
+ enum engine_id eng_id)
+{
+ struct dc_link *link = NULL;
+ int stream_idx = -1;
+ int i;
+
+ for (i = 0; i < state->stream_count; i++) {
+ struct link_enc_assignment assignment = state->res_ctx.link_enc_assignments[i];
+
+ if (assignment.valid && (assignment.eng_id == eng_id)) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ if (stream_idx != -1)
+ link = state->streams[stream_idx]->link;
+ else
+ dm_output_to_console("%s: No link using DIG(%d).\n", __func__, eng_id);
+
+ return link;
+}
+
+struct link_encoder *link_enc_cfg_get_link_enc_used_by_link(
+ struct dc_state *state,
+ struct dc_link *link)
+{
+ struct link_encoder *link_enc = NULL;
+ struct display_endpoint_id ep_id;
+ int stream_idx = -1;
+ int i;
+
+ ep_id = (struct display_endpoint_id) {
+ .link_id = link->link_id,
+ .ep_type = link->ep_type};
+
+ for (i = 0; i < state->stream_count; i++) {
+ struct link_enc_assignment assignment = state->res_ctx.link_enc_assignments[i];
+
+ if (assignment.valid &&
+ assignment.ep_id.link_id.id == ep_id.link_id.id &&
+ assignment.ep_id.link_id.enum_id == ep_id.link_id.enum_id &&
+ assignment.ep_id.link_id.type == ep_id.link_id.type &&
+ assignment.ep_id.ep_type == ep_id.ep_type) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ if (stream_idx != -1)
+ link_enc = state->streams[stream_idx]->link_enc;
+ else
+ dm_output_to_console("%s: No link encoder used by link(%d).\n", __func__, link->link_index);
+
+ return link_enc;
+}
#include "dpcd_defs.h"
#include "dsc.h"
#include "resource.h"
+#include "link_enc_cfg.h"
#include "clk_mgr.h"
static uint8_t convert_to_count(uint8_t lttpr_repeater_count)
enum clock_source_id clock_source,
const struct dc_link_settings *link_settings)
{
- struct link_encoder *link_enc = link->link_enc;
+ struct link_encoder *link_enc;
struct dc *dc = link->ctx->dc;
struct dmcu *dmcu = dc->res_pool->dmcu;
link->dc->res_pool->dp_clock_source;
unsigned int i;
+ /* Link should always be assigned encoder when en-/disabling. */
+ if (link->is_dig_mapping_flexible && dc->res_pool->funcs->link_encs_assign)
+ link_enc = link_enc_cfg_get_link_enc_used_by_link(link->dc->current_state, link);
+ else
+ link_enc = link->link_enc;
+ ASSERT(link_enc);
+
if (link->connector_signal == SIGNAL_TYPE_EDP) {
link->dc->hwss.edp_power_control(link, true);
link->dc->hwss.edp_wait_for_hpd_ready(link, true);
{
struct dc *dc = link->ctx->dc;
struct dmcu *dmcu = dc->res_pool->dmcu;
+ struct link_encoder *link_enc;
+
+ /* Link should always be assigned encoder when en-/disabling. */
+ if (link->is_dig_mapping_flexible && dc->res_pool->funcs->link_encs_assign)
+ link_enc = link_enc_cfg_get_link_enc_used_by_link(link->dc->current_state, link);
+ else
+ link_enc = link->link_enc;
+ ASSERT(link_enc);
if (!link->wa_flags.dp_keep_receiver_powered)
dp_receiver_power_ctrl(link, false);
if (signal == SIGNAL_TYPE_EDP) {
if (link->dc->hwss.edp_backlight_control)
link->dc->hwss.edp_backlight_control(link, false);
- link->link_enc->funcs->disable_output(link->link_enc, signal);
+ link_enc->funcs->disable_output(link_enc, signal);
link->dc->hwss.edp_power_control(link, false);
} else {
if (dmcu != NULL && dmcu->funcs->lock_phy)
dmcu->funcs->lock_phy(dmcu);
- link->link_enc->funcs->disable_output(link->link_enc, signal);
+ link_enc->funcs->disable_output(link_enc, signal);
if (dmcu != NULL && dmcu->funcs->unlock_phy)
dmcu->funcs->unlock_phy(dmcu);
{
struct link_encoder *encoder = link->link_enc;
- if (link->lttpr_non_transparent_mode && !is_immediate_downstream(link, offset))
+ if ((link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) && !is_immediate_downstream(link, offset))
return;
/* call Encoder to set lane settings */
dc->res_pool,
del_pipe->stream_res.stream_enc,
false);
+ /* Release link encoder from stream in new dc_state. */
+ if (dc->res_pool->funcs->link_enc_unassign)
+ dc->res_pool->funcs->link_enc_unassign(new_ctx, del_pipe->stream);
if (del_pipe->stream_res.audio)
update_audio_usage(
if (pipe_ctx_old->stream_res.dsc != pipe_ctx->stream_res.dsc)
return true;
+ /* DIG link encoder resource assignment for stream changed. */
+ if (pipe_ctx_old->stream->link_enc != pipe_ctx->stream->link_enc)
+ return true;
+
return false;
}
/* forward declaration */
struct aux_payload;
-#define DC_VER "3.2.127"
+#define DC_VER "3.2.130"
#define MAX_SURFACES 3
#define MAX_PLANES 6
enum pipe_split_policy pipe_split_policy;
bool force_single_disp_pipe_split;
bool voltage_align_fclk;
+ bool disable_min_fclk;
bool disable_dfs_bypass;
bool disable_dpp_power_gate;
struct dc_bios *dcb,
struct graphics_object_id object_id,
struct bp_disp_connector_caps_info *info);
+ enum bp_result (*get_lttpr_caps)(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps);
+ enum bp_result (*get_lttpr_interop)(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps);
};
struct bios_registers {
dc_link_fec_ready,
dc_link_fec_enabled
};
+
+enum lttpr_mode {
+ LTTPR_MODE_NON_LTTPR,
+ LTTPR_MODE_TRANSPARENT,
+ LTTPR_MODE_NON_TRANSPARENT,
+};
+
struct dc_link_status {
bool link_active;
struct dpcd_caps *dpcd_caps;
bool link_state_valid;
bool aux_access_disabled;
bool sync_lt_in_progress;
- bool lttpr_non_transparent_mode;
+ enum lttpr_mode lttpr_mode;
bool is_internal_display;
/* TODO: Rename. Flag an endpoint as having a programmable mapping to a
uint8_t hpd_src;
uint8_t link_enc_hw_inst;
+ /* DIG link encoder ID. Used as index in link encoder resource pool.
+ * For links with fixed mapping to DIG, this is not changed after dc_link
+ * object creation.
+ */
+ enum engine_id eng_id;
bool test_pattern_enabled;
union compliance_test_state compliance_test_state;
struct panel_cntl *panel_cntl;
struct link_encoder *link_enc;
struct graphics_object_id link_id;
+ /* Endpoint type distinguishes display endpoints which do not have entries
+ * in the BIOS connector table from those that do. Helps when tracking link
+ * encoder to display endpoint assignments.
+ */
+ enum display_endpoint_type ep_type;
union ddi_channel_mapping ddi_channel_mapping;
struct connector_device_tag_info device_tag;
struct dpcd_caps dpcd_caps;
uint32_t raw;
};
+struct test_pattern {
+ enum dp_test_pattern type;
+ enum dp_test_pattern_color_space color_space;
+ struct link_training_settings const *p_link_settings;
+ unsigned char const *p_custom_pattern;
+ unsigned int cust_pattern_size;
+};
+
struct dc_stream_state {
// sink is deprecated, new code should not reference
// this pointer
struct dc_sink *sink;
struct dc_link *link;
+ /* For dynamic link encoder assignment, update the link encoder assigned to
+ * a stream via the volatile dc_state rather than the static dc_link.
+ */
+ struct link_encoder *link_enc;
struct dc_panel_patch sink_patches;
union display_content_support content_support;
struct dc_crtc_timing timing;
uint32_t stream_id;
bool is_dsc_enabled;
+
+ struct test_pattern test_pattern;
union stream_update_flags update_flags;
bool has_non_synchronizable_pclk;
struct dc_dsc_config *dsc_config;
struct dc_transfer_func *func_shaper;
struct dc_3dlut *lut3d_func;
+
+ struct test_pattern *pending_test_pattern;
};
bool dc_is_stream_unchanged(
struct gpio_service *gpio_service;
uint32_t dc_sink_id_count;
uint32_t dc_stream_id_count;
+ uint32_t dc_edp_id_count;
uint64_t fbc_gpu_addr;
struct dc_dmub_srv *dmub_srv;
PSR_STATE5,
PSR_STATE5a,
PSR_STATE5b,
- PSR_STATE5c
+ PSR_STATE5c,
+ PSR_STATE_INVALID = 0xFF
};
struct psr_config {
DC_PSR_VERSION_UNSUPPORTED = 0xFFFFFFFF,
};
+/* Possible values of display_endpoint_id.endpoint */
+enum display_endpoint_type {
+ DISPLAY_ENDPOINT_PHY = 0, /* Physical connector. */
+ DISPLAY_ENDPOINT_UNKNOWN = -1
+};
+
+/* Extends graphics_object_id with an additional member 'ep_type' for
+ * distinguishing between physical endpoints (with entries in BIOS connector table) and
+ * logical endpoints.
+ */
+struct display_endpoint_id {
+ struct graphics_object_id link_id;
+ enum display_endpoint_type ep_type;
+};
+
#endif /* DC_TYPES_H_ */
const struct dce_abm_shift *abm_shift,
const struct dce_abm_mask *abm_mask)
{
- struct dce_abm *abm_dce = kzalloc(sizeof(*abm_dce), GFP_KERNEL);
+ struct dce_abm *abm_dce = kzalloc(sizeof(*abm_dce), GFP_ATOMIC);
if (abm_dce == NULL) {
BREAK_TO_DEBUGGER();
const struct dce_dmcu_shift *dmcu_shift,
const struct dce_dmcu_mask *dmcu_mask)
{
- struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_KERNEL);
+ struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_ATOMIC);
if (dmcu_dce == NULL) {
BREAK_TO_DEBUGGER();
const struct dce_dmcu_shift *dmcu_shift,
const struct dce_dmcu_mask *dmcu_mask)
{
- struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_KERNEL);
+ struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_ATOMIC);
if (dmcu_dce == NULL) {
BREAK_TO_DEBUGGER();
const struct dce_dmcu_shift *dmcu_shift,
const struct dce_dmcu_mask *dmcu_mask)
{
- struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_KERNEL);
+ struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_ATOMIC);
if (dmcu_dce == NULL) {
BREAK_TO_DEBUGGER();
static void dmub_psr_get_state(struct dmub_psr *dmub, enum dc_psr_state *state)
{
struct dmub_srv *srv = dmub->ctx->dmub_srv->dmub;
- uint32_t raw_state;
+ uint32_t raw_state = 0;
+ uint32_t retry_count = 0;
enum dmub_status status;
- // Send gpint command and wait for ack
- status = dmub_srv_send_gpint_command(srv, DMUB_GPINT__GET_PSR_STATE, 0, 30);
-
- if (status == DMUB_STATUS_OK) {
- // GPINT was executed, get response
- dmub_srv_get_gpint_response(srv, &raw_state);
- *state = convert_psr_state(raw_state);
- } else
- // Return invalid state when GPINT times out
- *state = 0xFF;
+ do {
+ // Send gpint command and wait for ack
+ status = dmub_srv_send_gpint_command(srv, DMUB_GPINT__GET_PSR_STATE, 0, 30);
+
+ if (status == DMUB_STATUS_OK) {
+ // GPINT was executed, get response
+ dmub_srv_get_gpint_response(srv, &raw_state);
+ *state = convert_psr_state(raw_state);
+ } else
+ // Return invalid state when GPINT times out
+ *state = PSR_STATE_INVALID;
+
+ // Assert if max retry hit
+ if (retry_count >= 1000)
+ ASSERT(0);
+ } while (++retry_count <= 1000 && *state == PSR_STATE_INVALID);
}
/*
******************************************************************************/
static void set_drr(struct pipe_ctx **pipe_ctx,
- int num_pipes, unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number)
+ int num_pipes, struct dc_crtc_timing_adjust adjust)
{
int i = 0;
struct drr_params params = {0};
// Note DRR trigger events are generated regardless of whether num frames met.
unsigned int num_frames = 2;
- params.vertical_total_max = vmax;
- params.vertical_total_min = vmin;
+ params.vertical_total_max = adjust.v_total_max;
+ params.vertical_total_min = adjust.v_total_min;
/* TODO: If multiple pipes are to be supported, you need
* some GSL stuff. Static screen triggers may be programmed differently
pipe_ctx[i]->stream_res.tg->funcs->set_drr(
pipe_ctx[i]->stream_res.tg, ¶ms);
- if (vmax != 0 && vmin != 0)
+ if (adjust.v_total_max != 0 && adjust.v_total_min != 0)
pipe_ctx[i]->stream_res.tg->funcs->set_static_screen_control(
pipe_ctx[i]->stream_res.tg,
event_triggers, num_frames);
return &opp->base;
}
-struct dce_aux *dce80_aux_engine_create(
+static struct dce_aux *dce80_aux_engine_create(
struct dc_context *ctx,
uint32_t inst)
{
I2C_COMMON_MASK_SH_LIST_DCE_COMMON_BASE(_MASK)
};
-struct dce_i2c_hw *dce80_i2c_hw_create(
+static struct dce_i2c_hw *dce80_i2c_hw_create(
struct dc_context *ctx,
uint32_t inst)
{
return dce_i2c_hw;
}
-struct dce_i2c_sw *dce80_i2c_sw_create(
+static struct dce_i2c_sw *dce80_i2c_sw_create(
struct dc_context *ctx)
{
struct dce_i2c_sw *dce_i2c_sw =
.flags.bits.IS_TPS3_CAPABLE = true
};
-struct link_encoder *dce80_link_encoder_create(
+static struct link_encoder *dce80_link_encoder_create(
const struct encoder_init_data *enc_init_data)
{
struct dce110_link_encoder *enc110 =
return &panel_cntl->base;
}
-struct clock_source *dce80_clock_source_create(
+static struct clock_source *dce80_clock_source_create(
struct dc_context *ctx,
struct dc_bios *bios,
enum clock_source_id id,
return NULL;
}
-void dce80_clock_source_destroy(struct clock_source **clk_src)
+static void dce80_clock_source_destroy(struct clock_source **clk_src)
{
kfree(TO_DCE110_CLK_SRC(*clk_src));
*clk_src = NULL;
}
}
-bool dce80_validate_bandwidth(
+static bool dce80_validate_bandwidth(
struct dc *dc,
struct dc_state *context,
bool fast_validate)
return true;
}
-enum dc_status dce80_validate_global(
+static enum dc_status dce80_validate_global(
struct dc *dc,
struct dc_state *context)
{
num = *numerator;
denom = *denominator;
for (i = 0; i < count; i++) {
- uint32_t num_reminder, denom_reminder;
+ uint32_t num_remainder, denom_remainder;
uint64_t num_result, denom_result;
if (checkUint32Bounary &&
num <= max_int32 && denom <= max_int32) {
break;
}
do {
- num_result = div_u64_rem(num, prime_numbers[i], &num_reminder);
- denom_result = div_u64_rem(denom, prime_numbers[i], &denom_reminder);
- if (num_reminder == 0 && denom_reminder == 0) {
+ num_result = div_u64_rem(num, prime_numbers[i], &num_remainder);
+ denom_result = div_u64_rem(denom, prime_numbers[i], &denom_remainder);
+ if (num_remainder == 0 && denom_remainder == 0) {
num = num_result;
denom = denom_result;
}
- } while (num_reminder == 0 && denom_reminder == 0);
+ } while (num_remainder == 0 && denom_remainder == 0);
}
*numerator = num;
*denominator = denom;
}
void dcn10_set_drr(struct pipe_ctx **pipe_ctx,
- int num_pipes, unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number)
+ int num_pipes, struct dc_crtc_timing_adjust adjust)
{
int i = 0;
struct drr_params params = {0};
// Note DRR trigger events are generated regardless of whether num frames met.
unsigned int num_frames = 2;
- params.vertical_total_max = vmax;
- params.vertical_total_min = vmin;
- params.vertical_total_mid = vmid;
- params.vertical_total_mid_frame_num = vmid_frame_number;
-
+ params.vertical_total_max = adjust.v_total_max;
+ params.vertical_total_min = adjust.v_total_min;
+ params.vertical_total_mid = adjust.v_total_mid;
+ params.vertical_total_mid_frame_num = adjust.v_total_mid_frame_num;
/* TODO: If multiple pipes are to be supported, you need
* some GSL stuff. Static screen triggers may be programmed differently
* as well.
for (i = 0; i < num_pipes; i++) {
pipe_ctx[i]->stream_res.tg->funcs->set_drr(
pipe_ctx[i]->stream_res.tg, ¶ms);
- if (vmax != 0 && vmin != 0)
+ if (adjust.v_total_max != 0 && adjust.v_total_min != 0)
pipe_ctx[i]->stream_res.tg->funcs->set_static_screen_control(
pipe_ctx[i]->stream_res.tg,
event_triggers, num_frames);
dc->clk_mgr->funcs->get_clock(dc->clk_mgr, context, clock_type, clock_cfg);
}
+
+void dcn10_get_dcc_en_bits(struct dc *dc, int *dcc_en_bits)
+{
+ struct resource_pool *pool = dc->res_pool;
+ int i;
+
+ for (i = 0; i < pool->pipe_count; i++) {
+ struct hubp *hubp = pool->hubps[i];
+ struct dcn_hubp_state *s = &(TO_DCN10_HUBP(hubp)->state);
+
+ hubp->funcs->hubp_read_state(hubp);
+
+ if (!s->blank_en)
+ dcc_en_bits[i] = s->dcc_en ? 1 : 0;
+ }
+}
/*
-* Copyright 2016 Advanced Micro Devices, Inc.
+* Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
struct dc_bios *dcb,
enum pipe_gating_control power_gating);
void dcn10_set_drr(struct pipe_ctx **pipe_ctx,
- int num_pipes, unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number);
+ int num_pipes, struct dc_crtc_timing_adjust adjust);
void dcn10_get_position(struct pipe_ctx **pipe_ctx,
int num_pipes,
struct crtc_position *position);
void dcn10_set_hdr_multiplier(struct pipe_ctx *pipe_ctx);
void dcn10_verify_allow_pstate_change_high(struct dc *dc);
+void dcn10_get_dcc_en_bits(struct dc *dc, int *dcc_en_bits);
+
#endif /* __DC_HWSS_DCN10_H__ */
/*
- * Copyright 2016 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
.set_backlight_level = dce110_set_backlight_level,
.set_abm_immediate_disable = dce110_set_abm_immediate_disable,
.set_pipe = dce110_set_pipe,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn10_private_funcs = {
OTG_VERTICAL_INTERRUPT2_LINE_START, start_line);
}
+/**
+ * Vupdate keepout can be set to a window to block the update lock for that pipe from changing.
+ * Start offset begins with vstartup and goes for x number of clocks,
+ * end offset starts from end of vupdate to x number of clocks.
+ */
+void optc1_set_vupdate_keepout(struct timing_generator *optc,
+ struct vupdate_keepout_params *params)
+{
+ struct optc *optc1 = DCN10TG_FROM_TG(optc);
+
+ REG_SET_3(OTG_VUPDATE_KEEPOUT, 0,
+ MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, params->start_offset,
+ MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, params->end_offset,
+ OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, params->enable);
+}
+
/**
* program_timing_generator used by mode timing set
* Program CRTC Timing Registers - OTG_H_*, OTG_V_*, Pixel repetition.
SF(OTG0_OTG_DOUBLE_BUFFER_CONTROL, OTG_UPDATE_PENDING, mask_sh),\
SF(OTG0_OTG_DOUBLE_BUFFER_CONTROL, OTG_BLANK_DATA_DOUBLE_BUFFER_EN, mask_sh),\
SF(OTG0_OTG_DOUBLE_BUFFER_CONTROL, OTG_RANGE_TIMING_DBUF_UPDATE_MODE, mask_sh),\
+ SF(OTG0_OTG_VUPDATE_KEEPOUT, OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, mask_sh), \
+ SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, mask_sh), \
+ SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, mask_sh), \
SF(OTG0_OTG_H_TOTAL, OTG_H_TOTAL, mask_sh),\
SF(OTG0_OTG_H_BLANK_START_END, OTG_H_BLANK_START, mask_sh),\
SF(OTG0_OTG_H_BLANK_START_END, OTG_H_BLANK_END, mask_sh),\
.meta_chunk_size_kbytes = 2,
.writeback_chunk_size_kbytes = 2,
.line_buffer_size_bits = 589824,
- .max_line_buffer_lines = 32,
+ .max_line_buffer_lines = 12,
.IsLineBufferBppFixed = 0,
.LineBufferFixedBpp = -1,
.writeback_luma_buffer_size_kbytes = 12,
.recovery_enabled = false, /*enable this by default after testing.*/
.max_downscale_src_width = 3840,
.underflow_assert_delay_us = 0xFFFFFFFF,
- .use_max_lb = true
};
static const struct dc_debug_options debug_defaults_diags = {
.disable_pplib_clock_request = true,
.disable_pplib_wm_range = true,
.underflow_assert_delay_us = 0xFFFFFFFF,
- .use_max_lb = true
};
static void dcn10_dpp_destroy(struct dpp **dpp)
const struct dccg_shift *dccg_shift,
const struct dccg_mask *dccg_mask)
{
- struct dcn_dccg *dccg_dcn = kzalloc(sizeof(*dccg_dcn), GFP_KERNEL);
+ struct dcn_dccg *dccg_dcn = kzalloc(sizeof(*dccg_dcn), GFP_ATOMIC);
struct dccg *base;
if (dccg_dcn == NULL) {
.optimize_timing_for_fsft = dcn20_optimize_timing_for_fsft,
#endif
.set_disp_pattern_generator = dcn20_set_disp_pattern_generator,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn20_private_funcs = {
} else {
AUX_REG_WRITE(AUX_DPHY_RX_CONTROL0, 0x103d1110);
- AUX_REG_WRITE(AUX_DPHY_TX_CONTROL, 0x21c4d);
-
+ AUX_REG_WRITE(AUX_DPHY_TX_CONTROL, 0x21c7a);
}
//AUX_DPHY_TX_REF_CONTROL'AUX_TX_REF_DIV HW default is 0x32;
OTG_GSL_WINDOW_END_Y, params->gsl_window_end_y);
}
-/**
- * Vupdate keepout can be set to a window to block the update lock for that pipe from changing.
- * Start offset begins with vstartup and goes for x number of clocks,
- * end offset starts from end of vupdate to x number of clocks.
- */
-void optc2_set_vupdate_keepout(struct timing_generator *optc,
- const struct vupdate_keepout_params *params)
-{
- struct optc *optc1 = DCN10TG_FROM_TG(optc);
-
- REG_SET_3(OTG_VUPDATE_KEEPOUT, 0,
- MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, params->start_offset,
- MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, params->end_offset,
- OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, params->enable);
-}
-
void optc2_set_gsl_source_select(
struct timing_generator *optc,
int group_idx,
SF(OTG0_OTG_GSL_WINDOW_X, OTG_GSL_WINDOW_END_X, mask_sh), \
SF(OTG0_OTG_GSL_WINDOW_Y, OTG_GSL_WINDOW_START_Y, mask_sh),\
SF(OTG0_OTG_GSL_WINDOW_Y, OTG_GSL_WINDOW_END_Y, mask_sh),\
- SF(OTG0_OTG_VUPDATE_KEEPOUT, OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, mask_sh), \
- SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, mask_sh), \
- SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, mask_sh), \
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_MASTER_MODE, mask_sh), \
SF(OTG0_OTG_GSL_CONTROL, OTG_MASTER_UPDATE_LOCK_GSL_EN, mask_sh), \
SF(OTG0_OTG_DSC_START_POSITION, OTG_DSC_START_POSITION_X, mask_sh), \
.is_line_buffer_bpp_fixed = 0,
.line_buffer_fixed_bpp = 0,
.dcc_supported = true,
- .max_line_buffer_lines = 32,
+ .max_line_buffer_lines = 12,
.writeback_luma_buffer_size_kbytes = 12,
.writeback_chroma_buffer_size_kbytes = 8,
.writeback_chroma_line_buffer_width_pixels = 4,
.is_line_buffer_bpp_fixed = 0,
.line_buffer_fixed_bpp = 0,
.dcc_supported = true,
- .max_line_buffer_lines = 32,
+ .max_line_buffer_lines = 12,
.writeback_luma_buffer_size_kbytes = 12,
.writeback_chroma_buffer_size_kbytes = 8,
.writeback_chroma_line_buffer_width_pixels = 4,
.scl_reset_length10 = true,
.sanity_checks = false,
.underflow_assert_delay_us = 0xFFFFFFFF,
- .use_max_lb = true
};
static const struct dc_debug_options debug_defaults_diags = {
.scl_reset_length10 = true,
.underflow_assert_delay_us = 0xFFFFFFFF,
.enable_tri_buf = true,
- .use_max_lb = true
};
void dcn20_dpp_destroy(struct dpp **dpp)
uint32_t inst)
{
struct dcn20_dpp *dpp =
- kzalloc(sizeof(struct dcn20_dpp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_dpp), GFP_ATOMIC);
if (!dpp)
return NULL;
struct dc_context *ctx, uint32_t inst)
{
struct dcn10_ipp *ipp =
- kzalloc(sizeof(struct dcn10_ipp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn10_ipp), GFP_ATOMIC);
if (!ipp) {
BREAK_TO_DEBUGGER();
struct dc_context *ctx, uint32_t inst)
{
struct dcn20_opp *opp =
- kzalloc(sizeof(struct dcn20_opp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_opp), GFP_ATOMIC);
if (!opp) {
BREAK_TO_DEBUGGER();
uint32_t inst)
{
struct aux_engine_dce110 *aux_engine =
- kzalloc(sizeof(struct aux_engine_dce110), GFP_KERNEL);
+ kzalloc(sizeof(struct aux_engine_dce110), GFP_ATOMIC);
if (!aux_engine)
return NULL;
uint32_t inst)
{
struct dce_i2c_hw *dce_i2c_hw =
- kzalloc(sizeof(struct dce_i2c_hw), GFP_KERNEL);
+ kzalloc(sizeof(struct dce_i2c_hw), GFP_ATOMIC);
if (!dce_i2c_hw)
return NULL;
struct mpc *dcn20_mpc_create(struct dc_context *ctx)
{
struct dcn20_mpc *mpc20 = kzalloc(sizeof(struct dcn20_mpc),
- GFP_KERNEL);
+ GFP_ATOMIC);
if (!mpc20)
return NULL;
{
int i;
struct dcn20_hubbub *hubbub = kzalloc(sizeof(struct dcn20_hubbub),
- GFP_KERNEL);
+ GFP_ATOMIC);
if (!hubbub)
return NULL;
uint32_t instance)
{
struct optc *tgn10 =
- kzalloc(sizeof(struct optc), GFP_KERNEL);
+ kzalloc(sizeof(struct optc), GFP_ATOMIC);
if (!tgn10)
return NULL;
bool dp_clk_src)
{
struct dce110_clk_src *clk_src =
- kzalloc(sizeof(struct dce110_clk_src), GFP_KERNEL);
+ kzalloc(sizeof(struct dce110_clk_src), GFP_ATOMIC);
if (!clk_src)
return NULL;
struct dc_context *ctx, uint32_t inst)
{
struct dcn20_dsc *dsc =
- kzalloc(sizeof(struct dcn20_dsc), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_dsc), GFP_ATOMIC);
if (!dsc) {
BREAK_TO_DEBUGGER();
uint32_t inst)
{
struct dcn20_hubp *hubp2 =
- kzalloc(sizeof(struct dcn20_hubp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_hubp), GFP_ATOMIC);
if (!hubp2)
return NULL;
pipes[pipe_cnt].dout.output_bpp = res_ctx->pipe_ctx[i].stream->timing.dsc_cfg.bits_per_pixel / 16.0;
/* todo: default max for now, until there is logic reflecting this in dc*/
- pipes[pipe_cnt].dout.output_bpc = 12;
+ pipes[pipe_cnt].dout.dsc_input_bpc = 12;
/*fill up the audio sample rate (unit in kHz)*/
get_audio_check(&res_ctx->pipe_ctx[i].stream->audio_info, &aud_check);
pipes[pipe_cnt].dout.max_audio_sample_rate = aud_check.max_audiosample_rate / 1000;
static struct pp_smu_funcs *dcn20_pp_smu_create(struct dc_context *ctx)
{
- struct pp_smu_funcs *pp_smu = kzalloc(sizeof(*pp_smu), GFP_KERNEL);
+ struct pp_smu_funcs *pp_smu = kzalloc(sizeof(*pp_smu), GFP_ATOMIC);
if (!pp_smu)
return pp_smu;
struct dc *dc)
{
struct dcn20_resource_pool *pool =
- kzalloc(sizeof(struct dcn20_resource_pool), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_resource_pool), GFP_ATOMIC);
if (!pool)
return NULL;
/*
- * Copyright 2016 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
#endif
.is_abm_supported = dcn21_is_abm_supported,
.set_disp_pattern_generator = dcn20_set_disp_pattern_generator,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn21_private_funcs = {
#include "dce/dce_audio.h"
#include "dce/dce_hwseq.h"
#include "virtual/virtual_stream_encoder.h"
-#include "dce110/dce110_resource.h"
#include "dml/display_mode_vba.h"
#include "dcn20/dcn20_dccg.h"
#include "dcn21/dcn21_dccg.h"
.is_line_buffer_bpp_fixed = 0,
.line_buffer_fixed_bpp = 0,
.dcc_supported = true,
- .max_line_buffer_lines = 32,
+ .max_line_buffer_lines = 12,
.writeback_luma_buffer_size_kbytes = 12,
.writeback_chroma_buffer_size_kbytes = 8,
.writeback_chroma_line_buffer_width_pixels = 4,
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_EN, mask_sh),\
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_IND_BLK, mask_sh),\
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_IND_BLK_C, mask_sh),\
+ HUBP_SF(HUBPREQ0_DCSURF_SURFACE_FLIP_INTERRUPT, SURFACE_FLIP_INT_MASK, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, DET_BUF_PLANE1_BASE_ADDRESS, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CB_B, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CR_R, mask_sh),\
/*
- * Copyright 2020 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
.hardware_release = dcn30_hardware_release,
.set_pipe = dcn21_set_pipe,
.set_disp_pattern_generator = dcn30_set_disp_pattern_generator,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn30_private_funcs = {
.dcc_supported = true,
.writeback_interface_buffer_size_kbytes = 90,
.writeback_line_buffer_buffer_size = 0,
- .max_line_buffer_lines = 32,
+ .max_line_buffer_lines = 12,
.writeback_luma_buffer_size_kbytes = 12, // writeback_line_buffer_buffer_size = 656640
.writeback_chroma_buffer_size_kbytes = 8,
.writeback_chroma_line_buffer_width_pixels = 4,
/*
- * Copyright 2020 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
.set_abm_immediate_disable = dcn21_set_abm_immediate_disable,
.set_pipe = dcn21_set_pipe,
.set_disp_pattern_generator = dcn30_set_disp_pattern_generator,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn301_private_funcs = {
/*
- * Copyright 2020 Advanced Micro Devices, Inc.
+ * Copyright 2019-2021 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
.dcc_supported = true,
.writeback_interface_buffer_size_kbytes = 90,
.writeback_line_buffer_buffer_size = 656640,
- .max_line_buffer_lines = 32,
+ .max_line_buffer_lines = 12,
.writeback_luma_buffer_size_kbytes = 12, // writeback_line_buffer_buffer_size = 656640
.writeback_chroma_buffer_size_kbytes = 8,
.writeback_chroma_line_buffer_width_pixels = 4,
.dcc_supported = true,
.writeback_interface_buffer_size_kbytes = 90,
.writeback_line_buffer_buffer_size = 0,
- .max_line_buffer_lines = 32,
+ .max_line_buffer_lines = 12,
.writeback_luma_buffer_size_kbytes = 12, // writeback_line_buffer_buffer_size = 656640
.writeback_chroma_buffer_size_kbytes = 8,
.writeback_chroma_line_buffer_width_pixels = 4,
.min_dcfclk = 500.0, /* TODO: set this to actual min DCFCLK */
.num_states = 1,
- .sr_exit_time_us = 12,
+ .sr_exit_time_us = 15.5,
.sr_enter_plus_exit_time_us = 20,
.urgent_latency_us = 4.0,
.urgent_latency_pixel_data_only_us = 4.0,
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
dml_print("DML PARAMS: PIPE [%d] DISPLAY OUTPUT PARAMS:\n", i);
dml_print("DML PARAMS: output_type = %d\n", dout->output_type);
dml_print("DML PARAMS: output_format = %d\n", dout->output_format);
- dml_print("DML PARAMS: output_bpc = %d\n", dout->output_bpc);
+ dml_print("DML PARAMS: dsc_input_bpc = %d\n", dout->dsc_input_bpc);
dml_print("DML PARAMS: output_bpp = %3.4f\n", dout->output_bpp);
dml_print("DML PARAMS: dp_lanes = %d\n", dout->dp_lanes);
dml_print("DML PARAMS: dsc_enable = %d\n", dout->dsc_enable);
double writeback_max_vscl_ratio;
double writeback_min_hscl_ratio;
double writeback_min_vscl_ratio;
- double maximum_dsc_bits_per_component;
+ unsigned int maximum_dsc_bits_per_component;
unsigned int writeback_max_hscl_taps;
unsigned int writeback_max_vscl_taps;
unsigned int writeback_line_buffer_luma_buffer_size;
struct _vcs_dpi_display_output_params_st {
int dp_lanes;
double output_bpp;
+ unsigned int dsc_input_bpc;
int dsc_enable;
int wb_enable;
int num_active_wb;
- int output_bpc;
int output_type;
int is_virtual;
int output_format;
mode_lib->vba.DSCEnable[mode_lib->vba.NumberOfActivePlanes] = dout->dsc_enable;
mode_lib->vba.NumberOfDSCSlices[mode_lib->vba.NumberOfActivePlanes] =
dout->dsc_slices;
- mode_lib->vba.DSCInputBitPerComponent[mode_lib->vba.NumberOfActivePlanes] = dout->output_bpc;
+ if (!dout->dsc_input_bpc) {
+ mode_lib->vba.DSCInputBitPerComponent[mode_lib->vba.NumberOfActivePlanes] =
+ ip->maximum_dsc_bits_per_component;
+ } else {
+ mode_lib->vba.DSCInputBitPerComponent[mode_lib->vba.NumberOfActivePlanes] =
+ dout->dsc_input_bpc;
+ }
mode_lib->vba.WritebackEnable[mode_lib->vba.NumberOfActivePlanes] = dout->wb_enable;
mode_lib->vba.ActiveWritebacksPerPlane[mode_lib->vba.NumberOfActivePlanes] =
dout->num_active_wb;
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
display_e2e_pipe_params_st *pipes,
bool fast_validate);
+ /*
+ * Algorithm for assigning available link encoders to links.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ */
+ void (*link_encs_assign)(
+ struct dc *dc,
+ struct dc_state *state,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count);
+ /*
+ * Unassign a link encoder from a stream.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ */
+ void (*link_enc_unassign)(
+ struct dc_state *state,
+ struct dc_stream_state *stream);
+
enum dc_status (*validate_global)(
struct dc *dc,
struct dc_state *context);
uint8_t clock_source_ref_count[MAX_CLOCK_SOURCES];
uint8_t dp_clock_source_ref_count;
bool is_dsc_acquired[MAX_PIPES];
+ /* A table/array of encoder-to-link assignments. One entry per stream.
+ * Indexed by stream index in dc_state.
+ */
+ struct link_enc_assignment link_enc_assignments[MAX_PIPES];
+ /* List of available link encoders. Uses engine ID as encoder identifier. */
+ enum engine_id link_enc_avail[MAX_DIG_LINK_ENCODERS];
#if defined(CONFIG_DRM_AMD_DC_DCN)
bool is_mpc_3dlut_acquired[MAX_PIPES];
#endif
/* Get current memclk states from PMFW, update relevant structures */
void (*get_memclk_states_from_smu)(struct clk_mgr *clk_mgr);
-};
-
-struct dpm_clocks;
-struct wartermarks;
-struct smu_watermark_set {
- struct watermarks *wm_set;
- union large_integer mc_address;
+ /* Get SMU present */
+ bool (*is_smu_present)(struct clk_mgr *clk_mgr);
};
struct clk_mgr {
struct clk_state_registers_and_bypass boot_snapshot;
struct clk_bw_params *bw_params;
struct pp_smu_wm_range_sets ranges;
- struct smu_watermark_set smu_wm_set;
};
/* forward declarations */
#include "mem_input.h"
#define OPP_ID_INVALID 0xf
+#define MAX_TTU 0xffffff
enum cursor_pitch {
struct link_encoder *enc);
};
+/*
+ * Used to track assignments of links (display endpoints) to link encoders.
+ *
+ * Entry in link_enc_assignments table in struct resource_context.
+ * Entries only marked valid once encoder assigned to a link and invalidated once unassigned.
+ * Uses engine ID as identifier since PHY ID not relevant for USB4 DPIA endpoint.
+ */
+struct link_enc_assignment {
+ bool valid;
+ struct display_endpoint_id ep_id;
+ enum engine_id eng_id;
+};
+
#endif /* LINK_ENCODER_H_ */
struct pipe_ctx *pipe_ctx,
enum vline_select vline);
void (*set_drr)(struct pipe_ctx **pipe_ctx, int num_pipes,
- unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number);
+ struct dc_crtc_timing_adjust adjust);
void (*set_static_screen_control)(struct pipe_ctx **pipe_ctx,
int num_pipes,
const struct dc_static_screen_params *events);
void (*set_pipe)(struct pipe_ctx *pipe_ctx);
+ void (*get_dcc_en_bits)(struct dc *dc, int *dcc_en_bits);
+
/* Idle Optimization Related */
bool (*apply_idle_power_optimizations)(struct dc *dc, bool enable);
--- /dev/null
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef DC_INC_LINK_ENC_CFG_H_
+#define DC_INC_LINK_ENC_CFG_H_
+
+/* This module implements functionality for dynamically assigning DIG link
+ * encoder resources to display endpoints (links).
+ */
+
+#include "core_types.h"
+
+/*
+ * Initialise link encoder resource tracking.
+ */
+void link_enc_cfg_init(
+ struct dc *dc,
+ struct dc_state *state);
+
+/*
+ * Algorithm for assigning available DIG link encoders to streams.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ *
+ * Loop over all streams twice:
+ * a) First assign encoders to unmappable endpoints.
+ * b) Then assign encoders to mappable endpoints.
+ */
+void link_enc_cfg_link_encs_assign(
+ struct dc *dc,
+ struct dc_state *state,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count);
+
+/*
+ * Unassign a link encoder from a stream.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ */
+void link_enc_cfg_link_enc_unassign(
+ struct dc_state *state,
+ struct dc_stream_state *stream);
+
+/*
+ * Check whether the transmitter driven by a link encoder is a mappable
+ * endpoint.
+ */
+bool link_enc_cfg_is_transmitter_mappable(
+ struct dc_state *state,
+ struct link_encoder *link_enc);
+
+/* Return link using DIG link encoder resource. NULL if unused. */
+struct dc_link *link_enc_cfg_get_link_using_link_enc(
+ struct dc_state *state,
+ enum engine_id eng_id);
+
+/* Return DIG link encoder used by link. NULL if unused. */
+struct link_encoder *link_enc_cfg_get_link_enc_used_by_link(
+ struct dc_state *state,
+ struct dc_link *link);
+
+#endif /* DC_INC_LINK_ENC_CFG_H_ */
.ack = NULL
};
+static const struct irq_source_info_funcs dmub_trace_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
static const struct irq_source_info_funcs vline0_irq_info_funcs = {
.set = NULL,
BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
+#define SRI_DMUB(reg_name)\
+ BASE(mm ## reg_name ## _BASE_IDX) + \
+ mm ## reg_name
#define IRQ_REG_ENTRY(block, reg_num, reg1, mask1, reg2, mask2)\
.enable_reg = SRI(reg1, block, reg_num),\
.ack_value = \
block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK \
-
+#define IRQ_REG_ENTRY_DMUB(reg1, mask1, reg2, mask2)\
+ .enable_reg = SRI_DMUB(reg1),\
+ .enable_mask = \
+ reg1 ## __ ## mask1 ## _MASK,\
+ .enable_value = {\
+ reg1 ## __ ## mask1 ## _MASK,\
+ ~reg1 ## __ ## mask1 ## _MASK \
+ },\
+ .ack_reg = SRI_DMUB(reg2),\
+ .ack_mask = \
+ reg2 ## __ ## mask2 ## _MASK,\
+ .ack_value = \
+ reg2 ## __ ## mask2 ## _MASK \
#define hpd_int_entry(reg_num)\
[DC_IRQ_SOURCE_HPD1 + reg_num] = {\
.funcs = &vline0_irq_info_funcs\
}
+#define dmub_trace_int_entry()\
+ [DC_IRQ_SOURCE_DMCUB_OUTBOX0] = {\
+ IRQ_REG_ENTRY_DMUB(DMCUB_INTERRUPT_ENABLE, DMCUB_OUTBOX0_READY_INT_EN,\
+ DMCUB_INTERRUPT_ACK, DMCUB_OUTBOX0_READY_INT_ACK),\
+ .funcs = &dmub_trace_irq_info_funcs\
+ }
+
#define dummy_irq_entry() \
{\
.funcs = &dummy_irq_info_funcs\
vline0_int_entry(3),
vline0_int_entry(4),
vline0_int_entry(5),
+ dmub_trace_int_entry(),
};
static const struct irq_service_funcs irq_service_funcs_dcn21 = {
BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
+#define SRI_DMUB(reg_name)\
+ BASE(mm ## reg_name ## _BASE_IDX) + \
+ mm ## reg_name
#define IRQ_REG_ENTRY(block, reg_num, reg1, mask1, reg2, mask2)\
.enable_reg = SRI(reg1, block, reg_num),\
.ack_value = \
block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK \
-
+#define IRQ_REG_ENTRY_DMUB(reg1, mask1, reg2, mask2)\
+ .enable_reg = SRI_DMUB(reg1),\
+ .enable_mask = \
+ reg1 ## __ ## mask1 ## _MASK,\
+ .enable_value = {\
+ reg1 ## __ ## mask1 ## _MASK,\
+ ~reg1 ## __ ## mask1 ## _MASK \
+ },\
+ .ack_reg = SRI_DMUB(reg2),\
+ .ack_mask = \
+ reg2 ## __ ## mask2 ## _MASK,\
+ .ack_value = \
+ reg2 ## __ ## mask2 ## _MASK \
#define hpd_int_entry(reg_num)\
[DC_IRQ_SOURCE_HPD1 + reg_num] = {\
.funcs = &vline0_irq_info_funcs\
}
+#define dmub_trace_int_entry()\
+ [DC_IRQ_SOURCE_DMCUB_OUTBOX0] = {\
+ IRQ_REG_ENTRY_DMUB(DMCUB_INTERRUPT_ENABLE, DMCUB_OUTBOX0_READY_INT_EN,\
+ DMCUB_INTERRUPT_ACK, DMCUB_OUTBOX0_READY_INT_ACK),\
+ .funcs = &dmub_trace_irq_info_funcs\
+ }
+
#define dummy_irq_entry() \
{\
.funcs = &dummy_irq_info_funcs\
vline0_int_entry(3),
vline0_int_entry(4),
vline0_int_entry(5),
+ dmub_trace_int_entry(),
};
static const struct irq_service_funcs irq_service_funcs_dcn30 = {
return DC_IRQ_SOURCE_VBLANK5;
case DCN_1_0__SRCID__DC_D6_OTG_VSTARTUP:
return DC_IRQ_SOURCE_VBLANK6;
+ case DCN_1_0__SRCID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT:
+ return DC_IRQ_SOURCE_DMCUB_OUTBOX0;
case DCN_1_0__SRCID__OTG1_VERTICAL_INTERRUPT0_CONTROL:
return DC_IRQ_SOURCE_DC1_VLINE0;
case DCN_1_0__SRCID__OTG2_VERTICAL_INTERRUPT0_CONTROL:
.ack = NULL
};
+static const struct irq_source_info_funcs dmub_trace_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
static const struct irq_source_info_funcs vline0_irq_info_funcs = {
.set = NULL,
.ack = NULL
BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
+#define SRI_DMUB(reg_name)\
+ BASE(mm ## reg_name ## _BASE_IDX) + \
+ mm ## reg_name
#define IRQ_REG_ENTRY(block, reg_num, reg1, mask1, reg2, mask2)\
.enable_reg = SRI(reg1, block, reg_num),\
.ack_mask = block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK,\
.ack_value = block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK \
+#define dmub_trace_int_entry()\
+ [DC_IRQ_SOURCE_DMCUB_OUTBOX0] = {\
+ IRQ_REG_ENTRY_DMUB(DMCUB_INTERRUPT_ENABLE, DMCUB_OUTBOX0_READY_INT_EN,\
+ DMCUB_INTERRUPT_ACK, DMCUB_OUTBOX0_READY_INT_ACK),\
+ .funcs = &dmub_trace_irq_info_funcs\
+ }
+#define IRQ_REG_ENTRY_DMUB(reg1, mask1, reg2, mask2)\
+ .enable_reg = SRI_DMUB(reg1),\
+ .enable_mask = \
+ reg1 ## __ ## mask1 ## _MASK,\
+ .enable_value = {\
+ reg1 ## __ ## mask1 ## _MASK,\
+ ~reg1 ## __ ## mask1 ## _MASK \
+ },\
+ .ack_reg = SRI_DMUB(reg2),\
+ .ack_mask = \
+ reg2 ## __ ## mask2 ## _MASK,\
+ .ack_value = \
+ reg2 ## __ ## mask2 ## _MASK \
#define hpd_int_entry(reg_num)\
[DC_IRQ_SOURCE_HPD1 + reg_num] = {\
vline0_int_entry(2),
vline0_int_entry(3),
vline0_int_entry(4),
+ dmub_trace_int_entry(),
};
static const struct irq_service_funcs irq_service_funcs_dcn302 = {
/* Firmware versioning. */
#ifdef DMUB_EXPOSE_VERSION
-#define DMUB_FW_VERSION_GIT_HASH 0xc29b1734b
+#define DMUB_FW_VERSION_GIT_HASH 0x7f2db1846
#define DMUB_FW_VERSION_MAJOR 0
#define DMUB_FW_VERSION_MINOR 0
-#define DMUB_FW_VERSION_REVISION 56
+#define DMUB_FW_VERSION_REVISION 59
#define DMUB_FW_VERSION_TEST 0
#define DMUB_FW_VERSION_VBIOS 0
#define DMUB_FW_VERSION_HOTFIX 0
* Max PSR version supported by FW.
*/
uint8_t psr;
-#ifndef TRIM_FAMS
- uint8_t fw_assisted_mclk_switch;
- uint8_t reserved[6];
-#else
uint8_t reserved[7];
-#endif
};
#if defined(__cplusplus)
* Command type used for OUTBOX1 notification enable
*/
DMUB_CMD__OUTBOX1_ENABLE = 71,
-#ifndef TRIM_FAMS
- DMUB_CMD__FW_ASSISTED_MCLK_SWITCH = 76,
-#endif
-
/**
* Command type used for all VBIOS interface commands.
*/
DMUB_CMD__PSR_FORCE_STATIC = 5,
};
-#ifndef TRIM_FAMS
-enum dmub_cmd_fams_type {
- DMUB_CMD__FAMS_SETUP_FW_CTRL = 0,
- DMUB_CMD__FAMS_DRR_UPDATE = 1,
-};
-#endif
-
/**
* PSR versions.
*/
* Length of each horizontal line in us.
*/
uint32_t line_time_in_us;
+ /**
+ * FEC enable status in driver
+ */
+ uint8_t fec_enable_status;
+ /**
+ * FEC re-enable delay when PSR exit.
+ * unit is 100us, range form 0~255(0xFF).
+ */
+ uint8_t fec_enable_delay_in100us;
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
+ uint8_t pad3[2];
};
/**
struct dmub_optc_state dmub_optc_state_req;
};
-#ifndef TRIM_FAMS
-struct dmub_cmd_fw_assisted_mclk_switch_pipe_data {
- uint32_t pix_clk_100hz;
- uint32_t min_refresh_in_uhz;
- uint32_t max_ramp_step;
-};
-
-struct dmub_cmd_fw_assisted_mclk_switch_config {
- uint32_t fams_enabled;
- struct dmub_cmd_fw_assisted_mclk_switch_pipe_data pipe_data[DMUB_MAX_STREAMS];
-};
-
-struct dmub_rb_cmd_fw_assisted_mclk_switch {
- struct dmub_cmd_header header;
- struct dmub_cmd_fw_assisted_mclk_switch_config config_data;
-};
-#endif
-
/**
* Data passed from driver to FW in a DMUB_CMD__VBIOS_LVTMA_CONTROL command.
*/
*/
struct dmub_rb_cmd_query_feature_caps query_feature_caps;
struct dmub_rb_cmd_drr_update drr_update;
-#ifndef TRIM_FAMS
- struct dmub_rb_cmd_fw_assisted_mclk_switch fw_assisted_mclk_switch;
-#endif
/**
* Definition of a DMUB_CMD__VBIOS_LVTMA_CONTROL command.
*/
if (!dmub->sw_init)
return DMUB_STATUS_INVALID;
+ if (!inst_fb || !stack_fb || !data_fb || !bios_fb || !mail_fb ||
+ !tracebuff_fb || !fw_state_fb || !scratch_mem_fb) {
+ ASSERT(0);
+ return DMUB_STATUS_INVALID;
+ }
+
dmub->fb_base = params->fb_base;
dmub->fb_offset = params->fb_offset;
dmub->psp_version = params->psp_version;
if (dmub->hw_funcs.reset)
dmub->hw_funcs.reset(dmub);
- if (inst_fb && data_fb) {
- cw0.offset.quad_part = inst_fb->gpu_addr;
- cw0.region.base = DMUB_CW0_BASE;
- cw0.region.top = cw0.region.base + inst_fb->size - 1;
-
- cw1.offset.quad_part = stack_fb->gpu_addr;
- cw1.region.base = DMUB_CW1_BASE;
- cw1.region.top = cw1.region.base + stack_fb->size - 1;
-
- if (params->load_inst_const && dmub->hw_funcs.backdoor_load) {
- /**
- * Read back all the instruction memory so we don't hang the
- * DMCUB when backdoor loading if the write from x86 hasn't been
- * flushed yet. This only occurs in backdoor loading.
- */
- dmub_flush_buffer_mem(inst_fb);
- dmub->hw_funcs.backdoor_load(dmub, &cw0, &cw1);
- }
-
- }
-
- if (inst_fb && data_fb && bios_fb && mail_fb && tracebuff_fb &&
- fw_state_fb && scratch_mem_fb) {
- cw2.offset.quad_part = data_fb->gpu_addr;
- cw2.region.base = DMUB_CW0_BASE + inst_fb->size;
- cw2.region.top = cw2.region.base + data_fb->size;
-
- cw3.offset.quad_part = bios_fb->gpu_addr;
- cw3.region.base = DMUB_CW3_BASE;
- cw3.region.top = cw3.region.base + bios_fb->size;
+ cw0.offset.quad_part = inst_fb->gpu_addr;
+ cw0.region.base = DMUB_CW0_BASE;
+ cw0.region.top = cw0.region.base + inst_fb->size - 1;
- cw4.offset.quad_part = mail_fb->gpu_addr;
- cw4.region.base = DMUB_CW4_BASE;
- cw4.region.top = cw4.region.base + mail_fb->size;
+ cw1.offset.quad_part = stack_fb->gpu_addr;
+ cw1.region.base = DMUB_CW1_BASE;
+ cw1.region.top = cw1.region.base + stack_fb->size - 1;
+ if (params->load_inst_const && dmub->hw_funcs.backdoor_load) {
/**
- * Doubled the mailbox region to accomodate inbox and outbox.
- * Note: Currently, currently total mailbox size is 16KB. It is split
- * equally into 8KB between inbox and outbox. If this config is
- * changed, then uncached base address configuration of outbox1
- * has to be updated in funcs->setup_out_mailbox.
+ * Read back all the instruction memory so we don't hang the
+ * DMCUB when backdoor loading if the write from x86 hasn't been
+ * flushed yet. This only occurs in backdoor loading.
*/
- inbox1.base = cw4.region.base;
- inbox1.top = cw4.region.base + DMUB_RB_SIZE;
- outbox1.base = inbox1.top;
- outbox1.top = cw4.region.top;
+ dmub_flush_buffer_mem(inst_fb);
+ dmub->hw_funcs.backdoor_load(dmub, &cw0, &cw1);
+ }
- cw5.offset.quad_part = tracebuff_fb->gpu_addr;
- cw5.region.base = DMUB_CW5_BASE;
- cw5.region.top = cw5.region.base + tracebuff_fb->size;
+ cw2.offset.quad_part = data_fb->gpu_addr;
+ cw2.region.base = DMUB_CW0_BASE + inst_fb->size;
+ cw2.region.top = cw2.region.base + data_fb->size;
- outbox0.base = DMUB_REGION5_BASE + TRACE_BUFFER_ENTRY_OFFSET;
- outbox0.top = outbox0.base + tracebuff_fb->size - TRACE_BUFFER_ENTRY_OFFSET;
+ cw3.offset.quad_part = bios_fb->gpu_addr;
+ cw3.region.base = DMUB_CW3_BASE;
+ cw3.region.top = cw3.region.base + bios_fb->size;
+ cw4.offset.quad_part = mail_fb->gpu_addr;
+ cw4.region.base = DMUB_CW4_BASE;
+ cw4.region.top = cw4.region.base + mail_fb->size;
- cw6.offset.quad_part = fw_state_fb->gpu_addr;
- cw6.region.base = DMUB_CW6_BASE;
- cw6.region.top = cw6.region.base + fw_state_fb->size;
+ /**
+ * Doubled the mailbox region to accomodate inbox and outbox.
+ * Note: Currently, currently total mailbox size is 16KB. It is split
+ * equally into 8KB between inbox and outbox. If this config is
+ * changed, then uncached base address configuration of outbox1
+ * has to be updated in funcs->setup_out_mailbox.
+ */
+ inbox1.base = cw4.region.base;
+ inbox1.top = cw4.region.base + DMUB_RB_SIZE;
+ outbox1.base = inbox1.top;
+ outbox1.top = cw4.region.top;
- dmub->fw_state = fw_state_fb->cpu_addr;
+ cw5.offset.quad_part = tracebuff_fb->gpu_addr;
+ cw5.region.base = DMUB_CW5_BASE;
+ cw5.region.top = cw5.region.base + tracebuff_fb->size;
- dmub->scratch_mem_fb = *scratch_mem_fb;
+ outbox0.base = DMUB_REGION5_BASE + TRACE_BUFFER_ENTRY_OFFSET;
+ outbox0.top = outbox0.base + tracebuff_fb->size - TRACE_BUFFER_ENTRY_OFFSET;
- if (dmub->hw_funcs.setup_windows)
- dmub->hw_funcs.setup_windows(dmub, &cw2, &cw3, &cw4,
- &cw5, &cw6);
+ cw6.offset.quad_part = fw_state_fb->gpu_addr;
+ cw6.region.base = DMUB_CW6_BASE;
+ cw6.region.top = cw6.region.base + fw_state_fb->size;
- if (dmub->hw_funcs.setup_outbox0)
- dmub->hw_funcs.setup_outbox0(dmub, &outbox0);
+ dmub->fw_state = fw_state_fb->cpu_addr;
- if (dmub->hw_funcs.setup_mailbox)
- dmub->hw_funcs.setup_mailbox(dmub, &inbox1);
- if (dmub->hw_funcs.setup_out_mailbox)
- dmub->hw_funcs.setup_out_mailbox(dmub, &outbox1);
- }
+ dmub->scratch_mem_fb = *scratch_mem_fb;
- if (mail_fb) {
- dmub_memset(&rb_params, 0, sizeof(rb_params));
- rb_params.ctx = dmub;
- rb_params.base_address = mail_fb->cpu_addr;
- rb_params.capacity = DMUB_RB_SIZE;
+ if (dmub->hw_funcs.setup_windows)
+ dmub->hw_funcs.setup_windows(dmub, &cw2, &cw3, &cw4, &cw5, &cw6);
- dmub_rb_init(&dmub->inbox1_rb, &rb_params);
+ if (dmub->hw_funcs.setup_outbox0)
+ dmub->hw_funcs.setup_outbox0(dmub, &outbox0);
- // Initialize outbox1 ring buffer
- rb_params.ctx = dmub;
- rb_params.base_address = (void *) ((uint8_t *) (mail_fb->cpu_addr) + DMUB_RB_SIZE);
- rb_params.capacity = DMUB_RB_SIZE;
- dmub_rb_init(&dmub->outbox1_rb, &rb_params);
+ if (dmub->hw_funcs.setup_mailbox)
+ dmub->hw_funcs.setup_mailbox(dmub, &inbox1);
+ if (dmub->hw_funcs.setup_out_mailbox)
+ dmub->hw_funcs.setup_out_mailbox(dmub, &outbox1);
- }
+ dmub_memset(&rb_params, 0, sizeof(rb_params));
+ rb_params.ctx = dmub;
+ rb_params.base_address = mail_fb->cpu_addr;
+ rb_params.capacity = DMUB_RB_SIZE;
+ dmub_rb_init(&dmub->inbox1_rb, &rb_params);
+
+ // Initialize outbox1 ring buffer
+ rb_params.ctx = dmub;
+ rb_params.base_address = (void *) ((uint8_t *) (mail_fb->cpu_addr) + DMUB_RB_SIZE);
+ rb_params.capacity = DMUB_RB_SIZE;
+ dmub_rb_init(&dmub->outbox1_rb, &rb_params);
dmub_memset(&outbox0_rb_params, 0, sizeof(outbox0_rb_params));
outbox0_rb_params.ctx = dmub;
- outbox0_rb_params.base_address = (void *)((uint64_t)(tracebuff_fb->cpu_addr) + TRACE_BUFFER_ENTRY_OFFSET);
+ outbox0_rb_params.base_address = (void *)((uintptr_t)(tracebuff_fb->cpu_addr) + TRACE_BUFFER_ENTRY_OFFSET);
outbox0_rb_params.capacity = tracebuff_fb->size - dmub_align(TRACE_BUFFER_ENTRY_OFFSET, 64);
dmub_rb_init(&dmub->outbox0_rb, &outbox0_rb_params);
dmub->hw_funcs.set_gpint(dmub, reg);
for (i = 0; i < timeout_us; ++i) {
+ udelay(1);
+
if (dmub->hw_funcs.is_gpint_acked(dmub, reg))
return DMUB_STATUS_OK;
}
#define DC_LOG_DC(...) DRM_DEBUG_KMS(__VA_ARGS__)
#define DC_LOG_DTN(...) DRM_DEBUG_KMS(__VA_ARGS__)
#define DC_LOG_SURFACE(...) pr_debug("[SURFACE]:"__VA_ARGS__)
+#define DC_LOG_CURSOR(...) pr_debug("[CURSOR]:"__VA_ARGS__)
+#define DC_LOG_PFLIP(...) pr_debug("[PFLIP]:"__VA_ARGS__)
+#define DC_LOG_VBLANK(...) pr_debug("[VBLANK]:"__VA_ARGS__)
#define DC_LOG_HW_HOTPLUG(...) DRM_DEBUG_KMS(__VA_ARGS__)
#define DC_LOG_HW_LINK_TRAINING(...) pr_debug("[HW_LINK_TRAINING]:"__VA_ARGS__)
#define DC_LOG_HW_SET_MODE(...) DRM_DEBUG_KMS(__VA_ARGS__)
static bool build_freesync_hdr(struct pwl_float_data_ex *rgb_regamma,
uint32_t hw_points_num,
const struct hw_x_point *coordinate_x,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
struct calculate_buffer *cal_buffer)
{
uint32_t i;
static bool calculate_curve(enum dc_transfer_func_predefined trans,
struct dc_transfer_func_distributed_points *points,
struct pwl_float_data_ex *rgb_regamma,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
uint32_t sdr_ref_white_level,
struct calculate_buffer *cal_buffer)
{
bool mod_color_calculate_regamma_params(struct dc_transfer_func *output_tf,
const struct dc_gamma *ramp, bool mapUserRamp, bool canRomBeUsed,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
struct calculate_buffer *cal_buffer)
{
struct dc_transfer_func_distributed_points *tf_pts = &output_tf->tf_pts;
};
};
-struct freesync_hdr_tf_params {
+struct hdr_tm_params {
unsigned int sdr_white_level;
unsigned int min_content; // luminance in 1/10000 nits
unsigned int max_content; // luminance in nits
bool mod_color_calculate_regamma_params(struct dc_transfer_func *output_tf,
const struct dc_gamma *ramp, bool mapUserRamp, bool canRomBeUsed,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
struct calculate_buffer *cal_buffer);
bool mod_color_calculate_degamma_params(struct dc_color_caps *dc_caps,
return duration_in_us;
}
-static unsigned int calc_v_total_from_refresh(
+unsigned int mod_freesync_calc_v_total_from_refresh(
const struct dc_stream_state *stream,
unsigned int refresh_in_uhz)
{
/* Restore FreeSync */
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
/* BTR set to "active" so engage */
} else {
if (update) {
if (in_out_vrr->fixed.fixed_active) {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(
+ mod_freesync_calc_v_total_from_refresh(
stream, in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
in_out_vrr->adjust.v_total_min;
} else {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
}
}
infopacket->sb[6] |= 0x02;
/* PB6 = [Bit 2 = FreeSync Active] */
- if (vrr->state == VRR_STATE_ACTIVE_VARIABLE ||
- vrr->state == VRR_STATE_ACTIVE_FIXED)
+ if (vrr->state != VRR_STATE_DISABLED &&
+ vrr->state != VRR_STATE_UNSUPPORTED)
infopacket->sb[6] |= 0x04;
// For v1 & 2 infoframes program nominal if non-fs mode, otherwise full range
refresh_range >= MIN_REFRESH_RANGE) {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
} else if (in_out_vrr->state == VRR_STATE_ACTIVE_FIXED) {
in_out_vrr->fixed.target_refresh_in_uhz =
} else {
in_out_vrr->fixed.fixed_active = true;
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->fixed.target_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
in_out_vrr->adjust.v_total_min;
/* Restore FreeSync */
if (in_out_vrr->btr.frame_counter == 0) {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
}
}
return nominal_field_rate_in_uhz;
}
+unsigned long long mod_freesync_calc_field_rate_from_timing(
+ unsigned int vtotal, unsigned int htotal, unsigned int pix_clk)
+{
+ unsigned long long field_rate_in_uhz = 0;
+ unsigned int total = htotal * vtotal;
+
+ /* Calculate nominal field rate for stream, rounded up to nearest integer */
+ field_rate_in_uhz = pix_clk;
+ field_rate_in_uhz *= 1000000ULL;
+
+ field_rate_in_uhz = div_u64(field_rate_in_uhz, total);
+
+ return field_rate_in_uhz;
+}
+
bool mod_freesync_is_valid_range(uint32_t min_refresh_cap_in_uhz,
uint32_t max_refresh_cap_in_uhz,
uint32_t nominal_field_rate_in_uhz)
*/
for (i = 0; i < MAX_NUM_OF_DISPLAYS; i++) {
if (hdcp->displays[i].state != MOD_HDCP_DISPLAY_INACTIVE &&
- !hdcp->displays[i].adjust.disable) {
+ hdcp->displays[i].adjust.disable != MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION) {
is_auth_needed = 1;
break;
}
*/
for (i = 0; i < MAX_NUM_OF_DISPLAYS; i++) {
if (hdcp->displays[i].state != MOD_HDCP_DISPLAY_INACTIVE &&
- !hdcp->displays[i].adjust.disable) {
+ hdcp->displays[i].adjust.disable != MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION) {
is_auth_needed = 1;
break;
}
goto out;
}
+ /* save current encryption states to restore after next authentication */
+ mod_hdcp_save_current_encryption_states(hdcp);
+
/* reset existing authentication status */
status = reset_authentication(hdcp, output);
if (status != MOD_HDCP_STATUS_SUCCESS)
goto out;
}
+ /* save current encryption states to restore after next authentication */
+ mod_hdcp_save_current_encryption_states(hdcp);
+
/* stop current authentication */
status = reset_authentication(hdcp, output);
if (status != MOD_HDCP_STATUS_SUCCESS)
struct mod_hdcp *hdcp, struct mod_hdcp_display *display);
enum mod_hdcp_status mod_hdcp_remove_display_from_topology(
struct mod_hdcp *hdcp, uint8_t index);
+bool mod_hdcp_is_link_encryption_enabled(struct mod_hdcp *hdcp);
+void mod_hdcp_save_current_encryption_states(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_create_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_destroy_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_validate_rx(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_enable_dp_stream_encryption(
struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_link_maintenance(struct mod_hdcp *hdcp);
-enum mod_hdcp_status mod_hdcp_hdcp1_get_link_encryption_status(struct mod_hdcp *hdcp,
- enum mod_hdcp_encryption_status *encryption_status);
enum mod_hdcp_status mod_hdcp_hdcp2_create_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp2_destroy_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp2_prepare_ake_init(struct mod_hdcp *hdcp);
goto out;
}
- if (!mod_hdcp_execute_and_set(mod_hdcp_hdcp1_link_maintenance,
+ mod_hdcp_execute_and_set(mod_hdcp_hdcp1_link_maintenance,
&input->link_maintenance, &status,
- hdcp, "link_maintenance"))
- goto out;
+ hdcp, "link_maintenance");
+
+ if (status != MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_save_current_encryption_states(hdcp);
out:
return status;
}
event_ctx->unexpected_event = 1;
goto out;
}
-
- if (!mod_hdcp_execute_and_set(mod_hdcp_read_bstatus,
- &input->bstatus_read, &status,
- hdcp, "bstatus_read"))
- goto out;
- if (!mod_hdcp_execute_and_set(check_link_integrity_dp,
- &input->link_integrity_check, &status,
- hdcp, "link_integrity_check"))
- goto out;
- if (!mod_hdcp_execute_and_set(check_no_reauthentication_request_dp,
- &input->reauth_request_check, &status,
- hdcp, "reauth_request_check"))
+ if (!mod_hdcp_is_link_encryption_enabled(hdcp))
goto out;
+
+ if (status == MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_execute_and_set(mod_hdcp_read_bstatus,
+ &input->bstatus_read, &status,
+ hdcp, "bstatus_read");
+ if (status == MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_execute_and_set(check_link_integrity_dp,
+ &input->link_integrity_check, &status,
+ hdcp, "link_integrity_check");
+ if (status == MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_execute_and_set(check_no_reauthentication_request_dp,
+ &input->reauth_request_check, &status,
+ hdcp, "reauth_request_check");
+
+ if (status != MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_save_current_encryption_states(hdcp);
out:
return status;
}
}
break;
case H1_A45_AUTHENTICATED:
- if (input->link_maintenance != PASS) {
+ if (input->link_maintenance == FAIL) {
/* 1A-07: consider invalid ri' a failure */
/* 1A-07a: consider read ri' not returned a failure */
fail_and_restart_in_ms(0, &status, output);
}
break;
case D1_A4_AUTHENTICATED:
- if (input->link_integrity_check != PASS ||
- input->reauth_request_check != PASS) {
+ if (input->link_integrity_check == FAIL ||
+ input->reauth_request_check == FAIL) {
/* 1A-07: restart hdcp on a link integrity failure */
fail_and_restart_in_ms(0, &status, output);
break;
event_ctx->unexpected_event = 1;
goto out;
}
-
- if (!process_rxstatus(hdcp, event_ctx, input, &status))
- goto out;
- if (event_ctx->rx_id_list_ready)
+ if (!mod_hdcp_is_link_encryption_enabled(hdcp))
goto out;
+
+ process_rxstatus(hdcp, event_ctx, input, &status);
+
+ if (status != MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_save_current_encryption_states(hdcp);
out:
return status;
}
HDCP_FULL_DDC_TRACE(hdcp);
break;
case H2_A5_AUTHENTICATED:
- if (input->rxstatus_read != PASS ||
- input->reauth_request_check != PASS) {
+ if (input->rxstatus_read == FAIL ||
+ input->reauth_request_check == FAIL) {
fail_and_restart_in_ms(0, &status, output);
break;
} else if (event_ctx->rx_id_list_ready && conn->is_repeater) {
HDCP_FULL_DDC_TRACE(hdcp);
break;
case D2_A5_AUTHENTICATED:
- if (input->rxstatus_read != PASS ||
- input->reauth_request_check != PASS) {
+ if (input->rxstatus_read == FAIL ||
+ input->reauth_request_check == FAIL) {
fail_and_restart_in_ms(0, &status, output);
break;
- } else if (input->link_integrity_check_dp != PASS) {
+ } else if (input->link_integrity_check_dp == FAIL) {
if (hdcp->connection.hdcp2_retry_count >= 1)
adjust->hdcp2.force_type = MOD_HDCP_FORCE_TYPE_0;
fail_and_restart_in_ms(0, &status, output);
return status;
}
+bool mod_hdcp_is_link_encryption_enabled(struct mod_hdcp *hdcp)
+{
+ /* unsupported */
+ return true;
+}
+
+void mod_hdcp_save_current_encryption_states(struct mod_hdcp *hdcp)
+{
+ /* unsupported */
+}
unsigned long long mod_freesync_calc_nominal_field_rate(
const struct dc_stream_state *stream);
+unsigned long long mod_freesync_calc_field_rate_from_timing(
+ unsigned int vtotal, unsigned int htotal, unsigned int pix_clk);
+
bool mod_freesync_is_valid_range(uint32_t min_refresh_cap_in_uhz,
uint32_t max_refresh_cap_in_uhz,
uint32_t nominal_field_rate_in_uhz);
-
+unsigned int mod_freesync_calc_v_total_from_refresh(
+ const struct dc_stream_state *stream,
+ unsigned int refresh_in_uhz);
#endif
MOD_HDCP_DISPLAY_ENCRYPTION_ENABLED
};
+enum mod_hdcp_display_disable_option {
+ MOD_HDCP_DISPLAY_NOT_DISABLE = 0,
+ MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION,
+ MOD_HDCP_DISPLAY_DISABLE_ENCRYPTION,
+};
+
struct mod_hdcp_ddc {
void *handle;
struct {
};
struct mod_hdcp_display_adjustment {
- uint8_t disable : 1;
- uint8_t reserved : 7;
+ uint8_t disable : 2;
+ uint8_t reserved : 6;
};
struct mod_hdcp_link_adjustment_hdcp1 {
uint8_t index;
};
-struct mod_hdcp;
-
/* dm allocates memory of mod_hdcp per dc_link on dm init based on memory size*/
size_t mod_hdcp_get_memory_size(void);
#define mmCP_CE_IB2_BASE_HI_BASE_IDX 1
#define mmCP_CE_IB2_BUFSZ 0x20cb
#define mmCP_CE_IB2_BUFSZ_BASE_IDX 1
+#define mmCP_IB1_BASE_LO 0x20cc
+#define mmCP_IB1_BASE_LO_BASE_IDX 1
+#define mmCP_IB1_BASE_HI 0x20cd
+#define mmCP_IB1_BASE_HI_BASE_IDX 1
+#define mmCP_IB1_BUFSZ 0x20ce
+#define mmCP_IB1_BUFSZ_BASE_IDX 1
#define mmCP_IB2_BASE_LO 0x20cf
#define mmCP_IB2_BASE_LO_BASE_IDX 1
#define mmCP_IB2_BASE_HI 0x20d0
//CP_CE_IB2_BUFSZ
#define CP_CE_IB2_BUFSZ__IB2_BUFSZ__SHIFT 0x0
#define CP_CE_IB2_BUFSZ__IB2_BUFSZ_MASK 0x000FFFFFL
+//CP_IB1_BASE_LO
+#define CP_IB1_BASE_LO__IB1_BASE_LO__SHIFT 0x2
+#define CP_IB1_BASE_LO__IB1_BASE_LO_MASK 0xFFFFFFFCL
+//CP_IB1_BASE_HI
+#define CP_IB1_BASE_HI__IB1_BASE_HI__SHIFT 0x0
+#define CP_IB1_BASE_HI__IB1_BASE_HI_MASK 0x0000FFFFL
+//CP_IB1_BUFSZ
+#define CP_IB1_BUFSZ__IB1_BUFSZ__SHIFT 0x0
+#define CP_IB1_BUFSZ__IB1_BUFSZ_MASK 0x000FFFFFL
//CP_IB2_BASE_LO
#define CP_IB2_BASE_LO__IB2_BASE_LO__SHIFT 0x2
#define CP_IB2_BASE_LO__IB2_BASE_LO_MASK 0xFFFFFFFCL
#define PIXEL_CLOCK_V6_MISC_HDMI_BPP_MASK 0x0c
#define PIXEL_CLOCK_V6_MISC_HDMI_24BPP 0x00
#define PIXEL_CLOCK_V6_MISC_HDMI_36BPP 0x04
-#define PIXEL_CLOCK_V6_MISC_HDMI_36BPP_V6 0x08 //for V6, the correct defintion for 36bpp should be 2 for 36bpp(2:1)
+#define PIXEL_CLOCK_V6_MISC_HDMI_36BPP_V6 0x08 //for V6, the correct definition for 36bpp should be 2 for 36bpp(2:1)
#define PIXEL_CLOCK_V6_MISC_HDMI_30BPP 0x08
-#define PIXEL_CLOCK_V6_MISC_HDMI_30BPP_V6 0x04 //for V6, the correct defintion for 30bpp should be 1 for 36bpp(5:4)
+#define PIXEL_CLOCK_V6_MISC_HDMI_30BPP_V6 0x04 //for V6, the correct definition for 30bpp should be 1 for 36bpp(5:4)
#define PIXEL_CLOCK_V6_MISC_HDMI_48BPP 0x0c
#define PIXEL_CLOCK_V6_MISC_REF_DIV_SRC 0x10
#define PIXEL_CLOCK_V6_MISC_GEN_DPREFCLK 0x40
uint8_t reserved3[8];
};
+struct atom_display_controller_info_v4_3
+{
+ struct atom_common_table_header table_header;
+ uint32_t display_caps;
+ uint32_t bootup_dispclk_10khz;
+ uint16_t dce_refclk_10khz;
+ uint16_t i2c_engine_refclk_10khz;
+ uint16_t dvi_ss_percentage; // in unit of 0.001%
+ uint16_t dvi_ss_rate_10hz;
+ uint16_t hdmi_ss_percentage; // in unit of 0.001%
+ uint16_t hdmi_ss_rate_10hz;
+ uint16_t dp_ss_percentage; // in unit of 0.001%
+ uint16_t dp_ss_rate_10hz;
+ uint8_t dvi_ss_mode; // enum of atom_spread_spectrum_mode
+ uint8_t hdmi_ss_mode; // enum of atom_spread_spectrum_mode
+ uint8_t dp_ss_mode; // enum of atom_spread_spectrum_mode
+ uint8_t ss_reserved;
+ uint8_t dfp_hardcode_mode_num; // DFP hardcode mode number defined in StandardVESA_TimingTable when EDID is not available
+ uint8_t dfp_hardcode_refreshrate;// DFP hardcode mode refreshrate defined in StandardVESA_TimingTable when EDID is not available
+ uint8_t vga_hardcode_mode_num; // VGA hardcode mode number defined in StandardVESA_TimingTable when EDID is not avablable
+ uint8_t vga_hardcode_refreshrate;// VGA hardcode mode number defined in StandardVESA_TimingTable when EDID is not avablable
+ uint16_t dpphy_refclk_10khz;
+ uint16_t reserved2;
+ uint8_t dcnip_min_ver;
+ uint8_t dcnip_max_ver;
+ uint8_t max_disp_pipe_num;
+ uint8_t max_vbios_active_disp_pipe_num;
+ uint8_t max_ppll_num;
+ uint8_t max_disp_phy_num;
+ uint8_t max_aux_pairs;
+ uint8_t remotedisplayconfig;
+ uint8_t reserved3[8];
+};
+
struct atom_display_controller_info_v4_4 {
struct atom_common_table_header table_header;
uint32_t display_caps;
DCE_INFO_CAPS_DISABLE_DFP_DP_HBR2 =0x04,
// only for VBIOS
DCE_INFO_CAPS_ENABLE_INTERLAC_TIMING =0x08,
-
+ // only for VBIOS
+ DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE =0x20,
+ DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE = 0x40,
};
/*
#define DCN_1_0__SRCID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT 0x68
#define DCN_1_0__CTXID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT 6
+#define DCN_1_0__SRCID__DMCUB_OUTBOX_LOW_PRIORITY_READY_INT 0x68 // DMCUB_IHC_outbox1_ready_int IHC_DMCUB_outbox1_ready_int_ack DMCUB_OUTBOX_LOW_PRIORITY_READY_INTERRUPT DISP_INTERRUPT_STATUS_CONTINUE24 Level/Pulse
+#define DCN_1_0__CTXID__DMCUB_OUTBOX_LOW_PRIORITY_READY_INT 8
#endif // __IRQSRCS_DCN_1_0_H__
struct pp_clock_levels_with_voltage;
struct pp_clock_levels_with_latency;
struct amd_pp_clocks;
+struct pp_smu_wm_range_sets;
+struct pp_smu_nv_clock_table;
+struct dpm_clocks;
struct amd_pm_funcs {
/* export for dpm on ci and si */
int (*set_df_cstate)(void *handle, enum pp_df_cstate state);
int (*set_xgmi_pstate)(void *handle, uint32_t pstate);
ssize_t (*get_gpu_metrics)(void *handle, void **table);
+ int (*set_watermarks_for_clock_ranges)(void *handle,
+ struct pp_smu_wm_range_sets *ranges);
+ int (*display_disable_memory_clock_switch)(void *handle,
+ bool disable_memory_clock_switch);
+ int (*get_max_sustainable_clocks_by_dc)(void *handle,
+ struct pp_smu_nv_clock_table *max_clocks);
+ int (*get_uclk_dpm_states)(void *handle,
+ unsigned int *clock_values_in_khz,
+ unsigned int *num_states);
+ int (*get_dpm_clock_table)(void *handle,
+ struct dpm_clocks *clock_table);
};
struct metrics_table_header {
{
int ret = 0;
const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- bool swsmu = is_support_sw_smu(adev);
switch (block_type) {
case AMD_IP_BLOCK_TYPE_UVD:
case AMD_IP_BLOCK_TYPE_GFX:
case AMD_IP_BLOCK_TYPE_VCN:
case AMD_IP_BLOCK_TYPE_SDMA:
- if (pp_funcs && pp_funcs->set_powergating_by_smu) {
- ret = (pp_funcs->set_powergating_by_smu(
- (adev)->powerplay.pp_handle, block_type, gate));
- }
- break;
case AMD_IP_BLOCK_TYPE_JPEG:
- if (swsmu)
- ret = smu_dpm_set_power_gate(&adev->smu, block_type, gate);
- break;
case AMD_IP_BLOCK_TYPE_GMC:
case AMD_IP_BLOCK_TYPE_ACP:
if (pp_funcs && pp_funcs->set_powergating_by_smu) {
pr_err("smu firmware loading failed\n");
return r;
}
- *smu_version = adev->pm.fw_version;
+
+ if (smu_version)
+ *smu_version = adev->pm.fw_version;
}
+
return 0;
}
#include "amdgpu_drv.h"
#include "amdgpu_pm.h"
#include "amdgpu_dpm.h"
-#include "amdgpu_smu.h"
#include "atom.h"
#include <linux/pci.h>
#include <linux/hwmon.h>
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
pm_runtime_mark_last_busy(ddev->dev);
pm_runtime_put_autosuspend(ddev->dev);
- return snprintf(buf, PAGE_SIZE, "%s\n",
- (pm == POWER_STATE_TYPE_BATTERY) ? "battery" :
- (pm == POWER_STATE_TYPE_BALANCED) ? "balanced" : "performance");
+ return sysfs_emit(buf, "%s\n",
+ (pm == POWER_STATE_TYPE_BATTERY) ? "battery" :
+ (pm == POWER_STATE_TYPE_BALANCED) ? "balanced" : "performance");
}
static ssize_t amdgpu_set_power_dpm_state(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (strncmp("battery", buf, strlen("battery")) == 0)
state = POWER_STATE_TYPE_BATTERY;
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
pm_runtime_mark_last_busy(ddev->dev);
pm_runtime_put_autosuspend(ddev->dev);
- return snprintf(buf, PAGE_SIZE, "%s\n",
- (level == AMD_DPM_FORCED_LEVEL_AUTO) ? "auto" :
- (level == AMD_DPM_FORCED_LEVEL_LOW) ? "low" :
- (level == AMD_DPM_FORCED_LEVEL_HIGH) ? "high" :
- (level == AMD_DPM_FORCED_LEVEL_MANUAL) ? "manual" :
- (level == AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD) ? "profile_standard" :
- (level == AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK) ? "profile_min_sclk" :
- (level == AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK) ? "profile_min_mclk" :
- (level == AMD_DPM_FORCED_LEVEL_PROFILE_PEAK) ? "profile_peak" :
- (level == AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM) ? "perf_determinism" :
- "unknown");
+ return sysfs_emit(buf, "%s\n",
+ (level == AMD_DPM_FORCED_LEVEL_AUTO) ? "auto" :
+ (level == AMD_DPM_FORCED_LEVEL_LOW) ? "low" :
+ (level == AMD_DPM_FORCED_LEVEL_HIGH) ? "high" :
+ (level == AMD_DPM_FORCED_LEVEL_MANUAL) ? "manual" :
+ (level == AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD) ? "profile_standard" :
+ (level == AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK) ? "profile_min_sclk" :
+ (level == AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK) ? "profile_min_mclk" :
+ (level == AMD_DPM_FORCED_LEVEL_PROFILE_PEAK) ? "profile_peak" :
+ (level == AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM) ? "perf_determinism" :
+ "unknown");
}
static ssize_t amdgpu_set_power_dpm_force_performance_level(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (strncmp("low", buf, strlen("low")) == 0) {
level = AMD_DPM_FORCED_LEVEL_LOW;
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (i == data.nums)
i = -EINVAL;
- return snprintf(buf, PAGE_SIZE, "%d\n", i);
+ return sysfs_emit(buf, "%d\n", i);
}
static ssize_t amdgpu_get_pp_force_state(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (adev->pp_force_state_enabled)
return amdgpu_get_pp_cur_state(dev, attr, buf);
else
- return snprintf(buf, PAGE_SIZE, "\n");
+ return sysfs_emit(buf, "\n");
}
static ssize_t amdgpu_set_pp_force_state(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (strlen(buf) == 1)
adev->pp_force_state_enabled = false;
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (count > 127)
return -EINVAL;
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = kstrtou64(buf, 0, &featuremask);
if (ret)
return ret;
}
- if (is_support_sw_smu(adev)) {
- ret = smu_sys_set_pp_feature_mask(&adev->smu, featuremask);
- if (ret) {
- pm_runtime_mark_last_busy(ddev->dev);
- pm_runtime_put_autosuspend(ddev->dev);
- return -EINVAL;
- }
- } else if (adev->powerplay.pp_funcs->set_ppfeature_status) {
+ if (adev->powerplay.pp_funcs->set_ppfeature_status) {
ret = amdgpu_dpm_set_ppfeature_status(adev, featuremask);
if (ret) {
pm_runtime_mark_last_busy(ddev->dev);
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = amdgpu_read_mask(buf, count, &mask);
if (ret)
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
pm_runtime_mark_last_busy(ddev->dev);
pm_runtime_put_autosuspend(ddev->dev);
- return snprintf(buf, PAGE_SIZE, "%d\n", value);
+ return sysfs_emit(buf, "%d\n", value);
}
static ssize_t amdgpu_set_pp_sclk_od(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = kstrtol(buf, 0, &value);
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
pm_runtime_mark_last_busy(ddev->dev);
pm_runtime_put_autosuspend(ddev->dev);
- return snprintf(buf, PAGE_SIZE, "%d\n", value);
+ return sysfs_emit(buf, "%d\n", value);
}
static ssize_t amdgpu_set_pp_mclk_od(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = kstrtol(buf, 0, &value);
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
tmp[0] = *(buf);
tmp[1] = '\0';
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(ddev->dev);
if (r < 0) {
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", value);
+ return sysfs_emit(buf, "%d\n", value);
}
/**
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(ddev->dev);
if (r < 0) {
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", value);
+ return sysfs_emit(buf, "%d\n", value);
}
/**
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (adev->flags & AMD_IS_APU)
return -ENODATA;
pm_runtime_mark_last_busy(ddev->dev);
pm_runtime_put_autosuspend(ddev->dev);
- return snprintf(buf, PAGE_SIZE, "%llu %llu %i\n",
- count0, count1, pcie_get_mps(adev->pdev));
+ return sysfs_emit(buf, "%llu %llu %i\n",
+ count0, count1, pcie_get_mps(adev->pdev));
}
/**
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (adev->unique_id)
- return snprintf(buf, PAGE_SIZE, "%016llx\n", adev->unique_id);
+ return sysfs_emit(buf, "%016llx\n", adev->unique_id);
return 0;
}
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%s: thermal throttling logging %s, with interval %d seconds\n",
- adev_to_drm(adev)->unique,
- atomic_read(&adev->throttling_logging_enabled) ? "enabled" : "disabled",
- adev->throttling_logging_rs.interval / HZ + 1);
+ return sysfs_emit(buf, "%s: thermal throttling logging %s, with interval %d seconds\n",
+ adev_to_drm(adev)->unique,
+ atomic_read(&adev->throttling_logging_enabled) ? "enabled" : "disabled",
+ adev->throttling_logging_rs.interval / HZ + 1);
}
static ssize_t amdgpu_set_thermal_throttling_logging(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(ddev->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (channel >= PP_TEMP_MAX)
return -EINVAL;
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", temp);
+ return sysfs_emit(buf, "%d\n", temp);
}
static ssize_t amdgpu_hwmon_show_temp_thresh(struct device *dev,
else
temp = adev->pm.dpm.thermal.max_temp;
- return snprintf(buf, PAGE_SIZE, "%d\n", temp);
+ return sysfs_emit(buf, "%d\n", temp);
}
static ssize_t amdgpu_hwmon_show_hotspot_temp_thresh(struct device *dev,
else
temp = adev->pm.dpm.thermal.max_hotspot_crit_temp;
- return snprintf(buf, PAGE_SIZE, "%d\n", temp);
+ return sysfs_emit(buf, "%d\n", temp);
}
static ssize_t amdgpu_hwmon_show_mem_temp_thresh(struct device *dev,
else
temp = adev->pm.dpm.thermal.max_mem_crit_temp;
- return snprintf(buf, PAGE_SIZE, "%d\n", temp);
+ return sysfs_emit(buf, "%d\n", temp);
}
static ssize_t amdgpu_hwmon_show_temp_label(struct device *dev,
if (channel >= PP_TEMP_MAX)
return -EINVAL;
- return snprintf(buf, PAGE_SIZE, "%s\n", temp_label[channel].label);
+ return sysfs_emit(buf, "%s\n", temp_label[channel].label);
}
static ssize_t amdgpu_hwmon_show_temp_emergency(struct device *dev,
break;
}
- return snprintf(buf, PAGE_SIZE, "%d\n", temp);
+ return sysfs_emit(buf, "%d\n", temp);
}
static ssize_t amdgpu_hwmon_get_pwm1_enable(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
err = kstrtoint(buf, 10, &value);
if (err)
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
err = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (err < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
err = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (err < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
err = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (err < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", min_rpm);
+ return sysfs_emit(buf, "%d\n", min_rpm);
}
static ssize_t amdgpu_hwmon_get_fan1_max(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", max_rpm);
+ return sysfs_emit(buf, "%d\n", max_rpm);
}
static ssize_t amdgpu_hwmon_get_fan1_target(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
err = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (err < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
err = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (err < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
ret = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (ret < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
err = kstrtoint(buf, 10, &value);
if (err)
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", vddgfx);
+ return sysfs_emit(buf, "%d\n", vddgfx);
}
static ssize_t amdgpu_hwmon_show_vddgfx_label(struct device *dev,
struct device_attribute *attr,
char *buf)
{
- return snprintf(buf, PAGE_SIZE, "vddgfx\n");
+ return sysfs_emit(buf, "vddgfx\n");
}
static ssize_t amdgpu_hwmon_show_vddnb(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
/* only APUs have vddnb */
if (!(adev->flags & AMD_IS_APU))
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%d\n", vddnb);
+ return sysfs_emit(buf, "%d\n", vddnb);
}
static ssize_t amdgpu_hwmon_show_vddnb_label(struct device *dev,
struct device_attribute *attr,
char *buf)
{
- return snprintf(buf, PAGE_SIZE, "vddnb\n");
+ return sysfs_emit(buf, "vddnb\n");
}
static ssize_t amdgpu_hwmon_show_power_avg(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
/* convert to microwatts */
uw = (query >> 8) * 1000000 + (query & 0xff) * 1000;
- return snprintf(buf, PAGE_SIZE, "%u\n", uw);
+ return sysfs_emit(buf, "%u\n", uw);
}
static ssize_t amdgpu_hwmon_show_power_cap_min(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
{
int limit_type = to_sensor_dev_attr(attr)->index;
- return snprintf(buf, PAGE_SIZE, "%s\n",
+ return sysfs_emit(buf, "%s\n",
limit_type == SMU_FAST_PPT_LIMIT ? "fastPPT" : "slowPPT");
}
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
if (amdgpu_sriov_vf(adev))
return -EINVAL;
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%u\n", sclk * 10 * 1000);
+ return sysfs_emit(buf, "%u\n", sclk * 10 * 1000);
}
static ssize_t amdgpu_hwmon_show_sclk_label(struct device *dev,
struct device_attribute *attr,
char *buf)
{
- return snprintf(buf, PAGE_SIZE, "sclk\n");
+ return sysfs_emit(buf, "sclk\n");
}
static ssize_t amdgpu_hwmon_show_mclk(struct device *dev,
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
if (r < 0) {
if (r)
return r;
- return snprintf(buf, PAGE_SIZE, "%u\n", mclk * 10 * 1000);
+ return sysfs_emit(buf, "%u\n", mclk * 10 * 1000);
}
static ssize_t amdgpu_hwmon_show_mclk_label(struct device *dev,
struct device_attribute *attr,
char *buf)
{
- return snprintf(buf, PAGE_SIZE, "mclk\n");
+ return sysfs_emit(buf, "mclk\n");
}
/**
if (amdgpu_in_reset(adev))
return -EPERM;
+ if (adev->in_suspend && !adev->in_runpm)
+ return -EPERM;
r = pm_runtime_get_sync(dev->dev);
if (r < 0) {
#define PPSMC_MSG_SetSystemVirtualSTBtoDramAddrHigh 0x40
#define PPSMC_MSG_SetSystemVirtualSTBtoDramAddrLow 0x41
-#define PPSMC_Message_Count 0x42
+#define PPSMC_MSG_GfxDriverResetRecovery 0x42
+#define PPSMC_Message_Count 0x43
//PPSMC Reset Types
#define PPSMC_RESET_TYPE_WARM_RESET 0x00
uint32_t clk_dependency;
};
+enum smu_event_type {
+
+ SMU_EVENT_RESET_COMPLETE = 0,
+};
+
#define SMU_TABLE_INIT(tables, table_id, s, a, d) \
do { \
tables[table_id].size = s; \
struct smu_power_gate power_gate;
};
-
#define SMU_FEATURE_MAX (64)
struct smu_feature
{
*/
int (*check_fw_status)(struct smu_context *smu);
+ /**
+ * @set_mp1_state: put SMU into a correct state for comming
+ * resume from runpm or gpu reset.
+ */
+ int (*set_mp1_state)(struct smu_context *smu,
+ enum pp_mp1_state mp1_state);
+
/**
* @setup_pptable: Initialize the power play table and populate it with
* default values.
* @set_light_sbr: Set light sbr mode for the SMU.
*/
int (*set_light_sbr)(struct smu_context *smu, bool enable);
+
+ /**
+ * @wait_for_event: Wait for events from SMU.
+ */
+ int (*wait_for_event)(struct smu_context *smu,
+ enum smu_event_type event, uint64_t event_arg);
};
typedef enum {
[profile] = {1, (workload)}
#if !defined(SWSMU_CODE_LAYER_L2) && !defined(SWSMU_CODE_LAYER_L3) && !defined(SWSMU_CODE_LAYER_L4)
-int smu_load_microcode(struct smu_context *smu);
-
-int smu_check_fw_status(struct smu_context *smu);
-
-int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled);
-
-int smu_set_fan_speed_rpm(void *handle, uint32_t speed);
-
int smu_get_power_limit(struct smu_context *smu,
uint32_t *limit,
enum smu_ppt_limit_level limit_level);
-int smu_set_power_limit(void *handle, uint32_t limit);
-int smu_print_ppclk_levels(void *handle, enum pp_clock_type type, char *buf);
-
-int smu_od_edit_dpm_table(void *handle,
- enum PP_OD_DPM_TABLE_COMMAND type,
- long *input, uint32_t size);
-
-int smu_read_sensor(void *handle, int sensor, void *data, int *size);
-int smu_get_power_profile_mode(void *handle, char *buf);
-int smu_set_power_profile_mode(void *handle, long *param, uint32_t param_size);
-u32 smu_get_fan_control_mode(void *handle);
-int smu_set_fan_control_mode(struct smu_context *smu, int value);
-void smu_pp_set_fan_control_mode(void *handle, u32 value);
-int smu_get_fan_speed_percent(void *handle, u32 *speed);
-int smu_set_fan_speed_percent(void *handle, u32 speed);
-int smu_get_fan_speed_rpm(void *handle, uint32_t *speed);
-
-int smu_set_deep_sleep_dcefclk(struct smu_context *smu, int clk);
-
-int smu_get_clock_by_type_with_latency(struct smu_context *smu,
- enum smu_clk_type clk_type,
- struct pp_clock_levels_with_latency *clocks);
-
-int smu_display_clock_voltage_request(struct smu_context *smu,
- struct pp_display_clock_request *clock_req);
-int smu_display_disable_memory_clock_switch(struct smu_context *smu, bool disable_memory_clock_switch);
-
-int smu_set_xgmi_pstate(void *handle,
- uint32_t pstate);
-
-int smu_set_azalia_d3_pme(struct smu_context *smu);
-
-bool smu_baco_is_support(struct smu_context *smu);
-int smu_get_baco_capability(void *handle, bool *cap);
-
-int smu_baco_get_state(struct smu_context *smu, enum smu_baco_state *state);
-
-int smu_baco_enter(struct smu_context *smu);
-int smu_baco_exit(struct smu_context *smu);
-int smu_baco_set_state(void *handle, int state);
-
-
bool smu_mode1_reset_is_support(struct smu_context *smu);
bool smu_mode2_reset_is_support(struct smu_context *smu);
int smu_mode1_reset(struct smu_context *smu);
-int smu_mode2_reset(void *handle);
extern const struct amd_ip_funcs smu_ip_funcs;
bool is_support_sw_smu(struct amdgpu_device *adev);
bool is_support_cclk_dpm(struct amdgpu_device *adev);
-int smu_reset(struct smu_context *smu);
-int smu_sys_get_pp_table(void *handle, char **table);
-int smu_sys_set_pp_table(void *handle, const char *buf, size_t size);
-int smu_get_power_num_states(void *handle, struct pp_states_info *state_info);
-enum amd_pm_state_type smu_get_current_power_state(void *handle);
int smu_write_watermarks_table(struct smu_context *smu);
-int smu_set_watermarks_for_clock_ranges(
- struct smu_context *smu,
- struct pp_smu_wm_range_sets *clock_ranges);
-
-/* smu to display interface */
-extern int smu_display_configuration_change(struct smu_context *smu, const
- struct amd_pp_display_configuration
- *display_config);
-extern int smu_dpm_set_power_gate(void *handle, uint32_t block_type, bool gate);
-extern int smu_handle_task(struct smu_context *smu,
- enum amd_dpm_forced_level level,
- enum amd_pp_task task_id,
- bool lock_needed);
-extern int smu_handle_dpm_task(void *handle,
- enum amd_pp_task task_id,
- enum amd_pm_state_type *user_state);
-int smu_switch_power_profile(void *handle,
- enum PP_SMC_POWER_PROFILE type,
- bool en);
+
int smu_get_dpm_freq_range(struct smu_context *smu, enum smu_clk_type clk_type,
uint32_t *min, uint32_t *max);
-u32 smu_get_mclk(void *handle, bool low);
-u32 smu_get_sclk(void *handle, bool low);
+
int smu_set_soft_freq_range(struct smu_context *smu, enum smu_clk_type clk_type,
uint32_t min, uint32_t max);
-enum amd_dpm_forced_level smu_get_performance_level(void *handle);
-int smu_force_performance_level(void *handle, enum amd_dpm_forced_level level);
-int smu_set_display_count(struct smu_context *smu, uint32_t count);
-int smu_set_ac_dc(struct smu_context *smu);
-int smu_sys_get_pp_feature_mask(void *handle, char *buf);
-int smu_sys_set_pp_feature_mask(void *handle, uint64_t new_mask);
-int smu_force_ppclk_levels(void *handle, enum pp_clock_type type, uint32_t mask);
-int smu_set_mp1_state(void *handle,
- enum pp_mp1_state mp1_state);
-int smu_set_df_cstate(void *handle,
- enum pp_df_cstate state);
-int smu_allow_xgmi_power_down(struct smu_context *smu, bool en);
-int smu_get_max_sustainable_clocks_by_dc(struct smu_context *smu,
- struct pp_smu_nv_clock_table *max_clocks);
-
-int smu_get_uclk_dpm_states(struct smu_context *smu,
- unsigned int *clock_values_in_khz,
- unsigned int *num_states);
+int smu_set_ac_dc(struct smu_context *smu);
-int smu_get_dpm_clock_table(struct smu_context *smu,
- struct dpm_clocks *clock_table);
+int smu_allow_xgmi_power_down(struct smu_context *smu, bool en);
int smu_get_status_gfxoff(struct amdgpu_device *adev, uint32_t *value);
-ssize_t smu_sys_get_gpu_metrics(void *handle, void **table);
-
-int smu_enable_mgpu_fan_boost(void *handle);
-int smu_gfx_state_change_set(struct smu_context *smu, uint32_t state);
-
int smu_set_light_sbr(struct smu_context *smu, bool enable);
+int smu_wait_for_event(struct amdgpu_device *adev, enum smu_event_type event,
+ uint64_t event_arg);
+
#endif
#endif
uint8_t GpioI2cSda; // Serial Data
uint16_t spare5;
+ uint16_t XgmiMaxCurrent; // in Amps
+ int8_t XgmiOffset; // in Amps
+ uint8_t Padding_TelemetryXgmi;
+
//reserved
- uint32_t reserved[16];
+ uint32_t reserved[15];
} PPTable_t;
uint16_t TemperatureAllHBM[4] ;
uint32_t GfxBusyAcc ;
uint32_t DramBusyAcc ;
- uint32_t Spare[4];
+ uint32_t EnergyAcc64bitLow ; //15.259uJ resolution
+ uint32_t EnergyAcc64bitHigh ;
+ uint32_t TimeStampLow ; //10ns resolution
+ uint32_t TimeStampHigh ;
// Padding - ignore
uint32_t MmHubPadding[8]; // SMU internal use
__SMU_DUMMY_MAP(DisableDeterminism), \
__SMU_DUMMY_MAP(SetUclkDpmMode), \
__SMU_DUMMY_MAP(LightSBR), \
+ __SMU_DUMMY_MAP(GfxDriverResetRecovery),
#undef __SMU_DUMMY_MAP
#define __SMU_DUMMY_MAP(type) SMU_MSG_##type
#define LINK_WIDTH_MAX 6
#define LINK_SPEED_MAX 3
-static __maybe_unused uint16_t link_width[] = {0, 1, 2, 4, 8, 12, 16};
-static __maybe_unused uint16_t link_speed[] = {25, 50, 80, 160};
+static const __maybe_unused uint16_t link_width[] = {0, 1, 2, 4, 8, 12, 16};
+static const __maybe_unused uint16_t link_speed[] = {25, 50, 80, 160};
static const
struct smu_temperature_range __maybe_unused smu11_thermal_policy[] =
#include "amdgpu_smu.h"
#define SMU13_DRIVER_IF_VERSION_INV 0xFFFFFFFF
-#define SMU13_DRIVER_IF_VERSION_ALDE 0x5
+#define SMU13_DRIVER_IF_VERSION_ALDE 0x6
/* MP Apertures */
#define MP0_Public 0x03800000
int smu_v13_0_gfx_ulv_control(struct smu_context *smu,
bool enablement);
+int smu_v13_0_wait_for_event(struct smu_context *smu, enum smu_event_type event,
+ uint64_t event_arg);
+
#endif
#endif
*size = 4;
break;
case AMDGPU_PP_SENSOR_GPU_LOAD:
- if (has_gfx_busy) {
+ if (!has_gfx_busy)
+ ret = -EOPNOTSUPP;
+ else {
ret = smum_send_msg_to_smc(hwmgr,
PPSMC_MSG_GetGfxBusy,
&activity_percent);
if (!ret)
- activity_percent = activity_percent > 100 ? 100 : activity_percent;
+ *((uint32_t *)value) = min(activity_percent, (u32)100);
else
- return -EIO;
- *((uint32_t *)value) = activity_percent;
- return 0;
- } else {
- return -EOPNOTSUPP;
+ ret = -EIO;
}
+ break;
default:
ret = -EOPNOTSUPP;
break;
(hwmgr->chip_id == CHIP_POLARIS10) ||
(hwmgr->chip_id == CHIP_POLARIS11) ||
(hwmgr->chip_id == CHIP_POLARIS12) ||
- (hwmgr->chip_id == CHIP_TONGA))
+ (hwmgr->chip_id == CHIP_TONGA) ||
+ (hwmgr->chip_id == CHIP_TOPAZ))
PHM_WRITE_FIELD(hwmgr->device, MC_SEQ_CNTL_3, CAC_EN, 0x1);
disable_mclk_switching_for_display = ((1 < hwmgr->display_config->num_display) &&
!hwmgr->display_config->multi_monitor_in_sync) ||
- smu7_vblank_too_short(hwmgr, hwmgr->display_config->min_vblank_time);
+ (hwmgr->display_config->num_display &&
+ smu7_vblank_too_short(hwmgr, hwmgr->display_config->min_vblank_time));
disable_mclk_switching = disable_mclk_switching_for_frame_lock ||
disable_mclk_switching_for_display;
#define LINK_WIDTH_MAX 6
#define LINK_SPEED_MAX 3
-static int link_width[] = {0, 1, 2, 4, 8, 12, 16};
-static int link_speed[] = {25, 50, 80, 160};
+static const int link_width[] = {0, 1, 2, 4, 8, 12, 16};
+static const int link_speed[] = {25, 50, 80, 160};
static int vega12_force_clock_level(struct pp_hwmgr *hwmgr,
enum pp_clock_type type, uint32_t mask);
#define LINK_WIDTH_MAX 6
#define LINK_SPEED_MAX 3
-static int link_width[] = {0, 1, 2, 4, 8, 12, 16};
-static int link_speed[] = {25, 50, 80, 160};
+static const int link_width[] = {0, 1, 2, 4, 8, 12, 16};
+static const int link_speed[] = {25, 50, 80, 160};
static void vega20_set_default_registry_data(struct pp_hwmgr *hwmgr)
{
static int smu_force_smuclk_levels(struct smu_context *smu,
enum smu_clk_type clk_type,
uint32_t mask);
-
-int smu_sys_get_pp_feature_mask(void *handle, char *buf)
+static int smu_handle_task(struct smu_context *smu,
+ enum amd_dpm_forced_level level,
+ enum amd_pp_task task_id,
+ bool lock_needed);
+static int smu_reset(struct smu_context *smu);
+static int smu_set_fan_speed_percent(void *handle, u32 speed);
+static int smu_set_fan_control_mode(struct smu_context *smu, int value);
+static int smu_set_power_limit(void *handle, uint32_t limit);
+static int smu_set_fan_speed_rpm(void *handle, uint32_t speed);
+static int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled);
+
+static int smu_sys_get_pp_feature_mask(void *handle,
+ char *buf)
{
struct smu_context *smu = handle;
int size = 0;
return size;
}
-int smu_sys_set_pp_feature_mask(void *handle, uint64_t new_mask)
+static int smu_sys_set_pp_feature_mask(void *handle,
+ uint64_t new_mask)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-u32 smu_get_mclk(void *handle, bool low)
+static u32 smu_get_mclk(void *handle, bool low)
{
struct smu_context *smu = handle;
uint32_t clk_freq;
return clk_freq * 100;
}
-u32 smu_get_sclk(void *handle, bool low)
+static u32 smu_get_sclk(void *handle, bool low)
{
struct smu_context *smu = handle;
uint32_t clk_freq;
* Under this case, the smu->mutex lock protection is already enforced on
* the parent API smu_force_performance_level of the call path.
*/
-int smu_dpm_set_power_gate(void *handle, uint32_t block_type,
- bool gate)
+static int smu_dpm_set_power_gate(void *handle,
+ uint32_t block_type,
+ bool gate)
{
struct smu_context *smu = handle;
int ret = 0;
smu->user_dpm_profile.flags &= ~SMU_DPM_USER_PROFILE_RESTORE;
}
-int smu_get_power_num_states(void *handle,
- struct pp_states_info *state_info)
+static int smu_get_power_num_states(void *handle,
+ struct pp_states_info *state_info)
{
if (!state_info)
return -EINVAL;
}
-int smu_sys_get_pp_table(void *handle, char **table)
+static int smu_sys_get_pp_table(void *handle,
+ char **table)
{
struct smu_context *smu = handle;
struct smu_table_context *smu_table = &smu->smu_table;
return powerplay_table_size;
}
-int smu_sys_set_pp_table(void *handle, const char *buf, size_t size)
+static int smu_sys_set_pp_table(void *handle,
+ const char *buf,
+ size_t size)
{
struct smu_context *smu = handle;
struct smu_table_context *smu_table = &smu->smu_table;
return ret;
}
+
static int smu_late_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
bool use_baco = !smu->is_apu &&
((amdgpu_in_reset(adev) &&
(amdgpu_asic_reset_method(adev) == AMD_RESET_METHOD_BACO)) ||
- ((adev->in_runpm || adev->in_hibernate) && amdgpu_asic_supports_baco(adev)));
+ ((adev->in_runpm || adev->in_s4) && amdgpu_asic_supports_baco(adev)));
/*
* For custom pptable uploading, skip the DPM features
return smu_smc_hw_cleanup(smu);
}
-int smu_reset(struct smu_context *smu)
+static int smu_reset(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
int ret;
smu->watermarks_bitmap &= ~(WATERMARKS_LOADED);
- if (smu->is_apu)
+ /* skip CGPG when in S0ix */
+ if (smu->is_apu && !adev->in_s0ix)
smu_set_gfx_cgpg(&adev->smu, false);
return 0;
return 0;
}
-int smu_display_configuration_change(struct smu_context *smu,
- const struct amd_pp_display_configuration *display_config)
+static int smu_display_configuration_change(void *handle,
+ const struct amd_pp_display_configuration *display_config)
{
+ struct smu_context *smu = handle;
int index = 0;
int num_of_active_display = 0;
return ret;
}
-int smu_handle_task(struct smu_context *smu,
- enum amd_dpm_forced_level level,
- enum amd_pp_task task_id,
- bool lock_needed)
+static int smu_handle_task(struct smu_context *smu,
+ enum amd_dpm_forced_level level,
+ enum amd_pp_task task_id,
+ bool lock_needed)
{
int ret = 0;
return ret;
}
-int smu_handle_dpm_task(void *handle,
- enum amd_pp_task task_id,
- enum amd_pm_state_type *user_state)
+static int smu_handle_dpm_task(void *handle,
+ enum amd_pp_task task_id,
+ enum amd_pm_state_type *user_state)
{
struct smu_context *smu = handle;
struct smu_dpm_context *smu_dpm = &smu->smu_dpm;
}
-
-int smu_switch_power_profile(void *handle,
- enum PP_SMC_POWER_PROFILE type,
- bool en)
+static int smu_switch_power_profile(void *handle,
+ enum PP_SMC_POWER_PROFILE type,
+ bool en)
{
struct smu_context *smu = handle;
struct smu_dpm_context *smu_dpm_ctx = &(smu->smu_dpm);
return 0;
}
-enum amd_dpm_forced_level smu_get_performance_level(void *handle)
+static enum amd_dpm_forced_level smu_get_performance_level(void *handle)
{
struct smu_context *smu = handle;
struct smu_dpm_context *smu_dpm_ctx = &(smu->smu_dpm);
return level;
}
-int smu_force_performance_level(void *handle, enum amd_dpm_forced_level level)
+static int smu_force_performance_level(void *handle,
+ enum amd_dpm_forced_level level)
{
struct smu_context *smu = handle;
struct smu_dpm_context *smu_dpm_ctx = &(smu->smu_dpm);
return ret;
}
-int smu_set_display_count(struct smu_context *smu, uint32_t count)
+static int smu_set_display_count(void *handle, uint32_t count)
{
+ struct smu_context *smu = handle;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
return ret;
}
-int smu_force_ppclk_levels(void *handle, enum pp_clock_type type, uint32_t mask)
+static int smu_force_ppclk_levels(void *handle,
+ enum pp_clock_type type,
+ uint32_t mask)
{
struct smu_context *smu = handle;
enum smu_clk_type clk_type;
* However, the mp1 state setting should still be granted
* even if the dpm_enabled cleared.
*/
-int smu_set_mp1_state(void *handle,
- enum pp_mp1_state mp1_state)
+static int smu_set_mp1_state(void *handle,
+ enum pp_mp1_state mp1_state)
{
struct smu_context *smu = handle;
- uint16_t msg;
- int ret;
+ int ret = 0;
if (!smu->pm_enabled)
return -EOPNOTSUPP;
mutex_lock(&smu->mutex);
- switch (mp1_state) {
- case PP_MP1_STATE_SHUTDOWN:
- msg = SMU_MSG_PrepareMp1ForShutdown;
- break;
- case PP_MP1_STATE_UNLOAD:
- msg = SMU_MSG_PrepareMp1ForUnload;
- break;
- case PP_MP1_STATE_RESET:
- msg = SMU_MSG_PrepareMp1ForReset;
- break;
- case PP_MP1_STATE_NONE:
- default:
- mutex_unlock(&smu->mutex);
- return 0;
- }
-
- ret = smu_send_smc_msg(smu, msg, NULL);
- /* some asics may not support those messages */
- if (ret == -EINVAL)
- ret = 0;
- if (ret)
- dev_err(smu->adev->dev, "[PrepareMp1] Failed!\n");
+ if (smu->ppt_funcs &&
+ smu->ppt_funcs->set_mp1_state)
+ ret = smu->ppt_funcs->set_mp1_state(smu, mp1_state);
mutex_unlock(&smu->mutex);
return ret;
}
-int smu_set_df_cstate(void *handle,
- enum pp_df_cstate state)
+static int smu_set_df_cstate(void *handle,
+ enum pp_df_cstate state)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_set_watermarks_for_clock_ranges(struct smu_context *smu,
- struct pp_smu_wm_range_sets *clock_ranges)
+static int smu_set_watermarks_for_clock_ranges(void *handle,
+ struct pp_smu_wm_range_sets *clock_ranges)
{
+ struct smu_context *smu = handle;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
.funcs = &smu_ip_funcs,
};
-int smu_load_microcode(struct smu_context *smu)
+static int smu_load_microcode(void *handle)
{
+ struct smu_context *smu = handle;
+ struct amdgpu_device *adev = smu->adev;
int ret = 0;
- if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
+ if (!smu->pm_enabled)
return -EOPNOTSUPP;
- mutex_lock(&smu->mutex);
+ /* This should be used for non PSP loading */
+ if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP)
+ return 0;
- if (smu->ppt_funcs->load_microcode)
+ if (smu->ppt_funcs->load_microcode) {
ret = smu->ppt_funcs->load_microcode(smu);
+ if (ret) {
+ dev_err(adev->dev, "Load microcode failed\n");
+ return ret;
+ }
+ }
- mutex_unlock(&smu->mutex);
-
- return ret;
-}
-
-int smu_check_fw_status(struct smu_context *smu)
-{
- int ret = 0;
-
- if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
- return -EOPNOTSUPP;
-
- mutex_lock(&smu->mutex);
-
- if (smu->ppt_funcs->check_fw_status)
+ if (smu->ppt_funcs->check_fw_status) {
ret = smu->ppt_funcs->check_fw_status(smu);
-
- mutex_unlock(&smu->mutex);
+ if (ret) {
+ dev_err(adev->dev, "SMC is not ready\n");
+ return ret;
+ }
+ }
return ret;
}
-int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled)
+static int smu_set_gfx_cgpg(struct smu_context *smu, bool enabled)
{
int ret = 0;
return ret;
}
-int smu_set_fan_speed_rpm(void *handle, uint32_t speed)
+static int smu_set_fan_speed_rpm(void *handle, uint32_t speed)
{
struct smu_context *smu = handle;
u32 percent;
return ret;
}
-int smu_set_power_limit(void *handle, uint32_t limit)
+static int smu_set_power_limit(void *handle, uint32_t limit)
{
struct smu_context *smu = handle;
uint32_t limit_type = limit >> 24;
return ret;
}
-int smu_print_ppclk_levels(void *handle, enum pp_clock_type type, char *buf)
+static int smu_print_ppclk_levels(void *handle,
+ enum pp_clock_type type,
+ char *buf)
{
struct smu_context *smu = handle;
enum smu_clk_type clk_type;
return smu_print_smuclk_levels(smu, clk_type, buf);
}
-int smu_od_edit_dpm_table(void *handle,
- enum PP_OD_DPM_TABLE_COMMAND type,
- long *input, uint32_t size)
+static int smu_od_edit_dpm_table(void *handle,
+ enum PP_OD_DPM_TABLE_COMMAND type,
+ long *input, uint32_t size)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_read_sensor(void *handle, int sensor, void *data, int *size_arg)
+static int smu_read_sensor(void *handle,
+ int sensor,
+ void *data,
+ int *size_arg)
{
struct smu_context *smu = handle;
struct smu_umd_pstate_table *pstate_table =
return ret;
}
-int smu_get_power_profile_mode(void *handle, char *buf)
+static int smu_get_power_profile_mode(void *handle, char *buf)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_set_power_profile_mode(void *handle, long *param, uint32_t param_size)
+static int smu_set_power_profile_mode(void *handle,
+ long *param,
+ uint32_t param_size)
{
struct smu_context *smu = handle;
int ret = 0;
}
-u32 smu_get_fan_control_mode(void *handle)
+static u32 smu_get_fan_control_mode(void *handle)
{
struct smu_context *smu = handle;
u32 ret = 0;
return ret;
}
-int smu_set_fan_control_mode(struct smu_context *smu, int value)
+static int smu_set_fan_control_mode(struct smu_context *smu, int value)
{
int ret = 0;
return ret;
}
-void smu_pp_set_fan_control_mode(void *handle, u32 value) {
+static void smu_pp_set_fan_control_mode(void *handle, u32 value)
+{
struct smu_context *smu = handle;
smu_set_fan_control_mode(smu, value);
}
-int smu_get_fan_speed_percent(void *handle, u32 *speed)
+static int smu_get_fan_speed_percent(void *handle, u32 *speed)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_set_fan_speed_percent(void *handle, u32 speed)
+static int smu_set_fan_speed_percent(void *handle, u32 speed)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_get_fan_speed_rpm(void *handle, uint32_t *speed)
+static int smu_get_fan_speed_rpm(void *handle, uint32_t *speed)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_set_deep_sleep_dcefclk(struct smu_context *smu, int clk)
+static int smu_set_deep_sleep_dcefclk(void *handle, uint32_t clk)
{
+ struct smu_context *smu = handle;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
return ret;
}
-int smu_get_clock_by_type_with_latency(struct smu_context *smu,
- enum smu_clk_type clk_type,
- struct pp_clock_levels_with_latency *clocks)
+static int smu_get_clock_by_type_with_latency(void *handle,
+ enum amd_pp_clock_type type,
+ struct pp_clock_levels_with_latency *clocks)
{
+ struct smu_context *smu = handle;
+ enum smu_clk_type clk_type;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
mutex_lock(&smu->mutex);
- if (smu->ppt_funcs->get_clock_by_type_with_latency)
+ if (smu->ppt_funcs->get_clock_by_type_with_latency) {
+ switch (type) {
+ case amd_pp_sys_clock:
+ clk_type = SMU_GFXCLK;
+ break;
+ case amd_pp_mem_clock:
+ clk_type = SMU_MCLK;
+ break;
+ case amd_pp_dcef_clock:
+ clk_type = SMU_DCEFCLK;
+ break;
+ case amd_pp_disp_clock:
+ clk_type = SMU_DISPCLK;
+ break;
+ default:
+ dev_err(smu->adev->dev, "Invalid clock type!\n");
+ mutex_unlock(&smu->mutex);
+ return -EINVAL;
+ }
+
ret = smu->ppt_funcs->get_clock_by_type_with_latency(smu, clk_type, clocks);
+ }
mutex_unlock(&smu->mutex);
return ret;
}
-int smu_display_clock_voltage_request(struct smu_context *smu,
- struct pp_display_clock_request *clock_req)
+static int smu_display_clock_voltage_request(void *handle,
+ struct pp_display_clock_request *clock_req)
{
+ struct smu_context *smu = handle;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
}
-int smu_display_disable_memory_clock_switch(struct smu_context *smu, bool disable_memory_clock_switch)
+static int smu_display_disable_memory_clock_switch(void *handle,
+ bool disable_memory_clock_switch)
{
+ struct smu_context *smu = handle;
int ret = -EINVAL;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
return ret;
}
-int smu_set_xgmi_pstate(void *handle,
- uint32_t pstate)
+static int smu_set_xgmi_pstate(void *handle,
+ uint32_t pstate)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_set_azalia_d3_pme(struct smu_context *smu)
-{
- int ret = 0;
-
- if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
- return -EOPNOTSUPP;
-
- mutex_lock(&smu->mutex);
-
- if (smu->ppt_funcs->set_azalia_d3_pme)
- ret = smu->ppt_funcs->set_azalia_d3_pme(smu);
-
- mutex_unlock(&smu->mutex);
-
- return ret;
-}
-
-/*
- * On system suspending or resetting, the dpm_enabled
- * flag will be cleared. So that those SMU services which
- * are not supported will be gated.
- *
- * However, the baco/mode1 reset should still be granted
- * as they are still supported and necessary.
- */
-bool smu_baco_is_support(struct smu_context *smu)
-{
- bool ret = false;
-
- if (!smu->pm_enabled)
- return false;
-
- mutex_lock(&smu->mutex);
-
- if (smu->ppt_funcs && smu->ppt_funcs->baco_is_support)
- ret = smu->ppt_funcs->baco_is_support(smu);
-
- mutex_unlock(&smu->mutex);
-
- return ret;
-}
-
-int smu_get_baco_capability(void *handle, bool *cap)
+static int smu_get_baco_capability(void *handle, bool *cap)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-
-int smu_baco_get_state(struct smu_context *smu, enum smu_baco_state *state)
-{
- if (smu->ppt_funcs->baco_get_state)
- return -EINVAL;
-
- mutex_lock(&smu->mutex);
- *state = smu->ppt_funcs->baco_get_state(smu);
- mutex_unlock(&smu->mutex);
-
- return 0;
-}
-
-int smu_baco_enter(struct smu_context *smu)
-{
- int ret = 0;
-
- if (!smu->pm_enabled)
- return -EOPNOTSUPP;
-
- mutex_lock(&smu->mutex);
-
- if (smu->ppt_funcs->baco_enter)
- ret = smu->ppt_funcs->baco_enter(smu);
-
- mutex_unlock(&smu->mutex);
-
- if (ret)
- dev_err(smu->adev->dev, "Failed to enter BACO state!\n");
-
- return ret;
-}
-
-int smu_baco_exit(struct smu_context *smu)
-{
- int ret = 0;
-
- if (!smu->pm_enabled)
- return -EOPNOTSUPP;
-
- mutex_lock(&smu->mutex);
-
- if (smu->ppt_funcs->baco_exit)
- ret = smu->ppt_funcs->baco_exit(smu);
-
- mutex_unlock(&smu->mutex);
-
- if (ret)
- dev_err(smu->adev->dev, "Failed to exit BACO state!\n");
-
- return ret;
-}
-
-int smu_baco_set_state(void *handle, int state)
+static int smu_baco_set_state(void *handle, int state)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_mode2_reset(void *handle)
+static int smu_mode2_reset(void *handle)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_get_max_sustainable_clocks_by_dc(struct smu_context *smu,
- struct pp_smu_nv_clock_table *max_clocks)
+static int smu_get_max_sustainable_clocks_by_dc(void *handle,
+ struct pp_smu_nv_clock_table *max_clocks)
{
+ struct smu_context *smu = handle;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
return ret;
}
-int smu_get_uclk_dpm_states(struct smu_context *smu,
- unsigned int *clock_values_in_khz,
- unsigned int *num_states)
+static int smu_get_uclk_dpm_states(void *handle,
+ unsigned int *clock_values_in_khz,
+ unsigned int *num_states)
{
+ struct smu_context *smu = handle;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
return ret;
}
-enum amd_pm_state_type smu_get_current_power_state(void *handle)
+static enum amd_pm_state_type smu_get_current_power_state(void *handle)
{
struct smu_context *smu = handle;
enum amd_pm_state_type pm_state = POWER_STATE_TYPE_DEFAULT;
return pm_state;
}
-int smu_get_dpm_clock_table(struct smu_context *smu,
- struct dpm_clocks *clock_table)
+static int smu_get_dpm_clock_table(void *handle,
+ struct dpm_clocks *clock_table)
{
+ struct smu_context *smu = handle;
int ret = 0;
if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
return ret;
}
-ssize_t smu_sys_get_gpu_metrics(void *handle, void **table)
+static ssize_t smu_sys_get_gpu_metrics(void *handle, void **table)
{
struct smu_context *smu = handle;
ssize_t size;
return size;
}
-int smu_enable_mgpu_fan_boost(void *handle)
+static int smu_enable_mgpu_fan_boost(void *handle)
{
struct smu_context *smu = handle;
int ret = 0;
return ret;
}
-int smu_gfx_state_change_set(struct smu_context *smu, uint32_t state)
+static int smu_gfx_state_change_set(void *handle,
+ uint32_t state)
{
+ struct smu_context *smu = handle;
int ret = 0;
mutex_lock(&smu->mutex);
.get_power_profile_mode = smu_get_power_profile_mode,
.force_clock_level = smu_force_ppclk_levels,
.print_clock_levels = smu_print_ppclk_levels,
+ .get_uclk_dpm_states = smu_get_uclk_dpm_states,
+ .get_dpm_clock_table = smu_get_dpm_clock_table,
+ .display_configuration_change = smu_display_configuration_change,
+ .get_clock_by_type_with_latency = smu_get_clock_by_type_with_latency,
+ .display_clock_voltage_request = smu_display_clock_voltage_request,
+ .set_active_display_count = smu_set_display_count,
+ .set_min_deep_sleep_dcefclk = smu_set_deep_sleep_dcefclk,
+ .set_watermarks_for_clock_ranges = smu_set_watermarks_for_clock_ranges,
+ .display_disable_memory_clock_switch = smu_display_disable_memory_clock_switch,
+ .get_max_sustainable_clocks_by_dc = smu_get_max_sustainable_clocks_by_dc,
+ .load_firmware = smu_load_microcode,
+ .gfx_state_change_set = smu_gfx_state_change_set,
};
+
+int smu_wait_for_event(struct amdgpu_device *adev, enum smu_event_type event,
+ uint64_t event_arg)
+{
+ int ret = -EINVAL;
+ struct smu_context *smu = &adev->smu;
+
+ if (smu->ppt_funcs->wait_for_event) {
+ mutex_lock(&smu->mutex);
+ ret = smu->ppt_funcs->wait_for_event(smu, event, event_arg);
+ mutex_unlock(&smu->mutex);
+ }
+
+ return ret;
+}
.get_fan_parameters = arcturus_get_fan_parameters,
.interrupt_work = smu_v11_0_interrupt_work,
.set_light_sbr = smu_v11_0_set_light_sbr,
+ .set_mp1_state = smu_cmn_set_mp1_state,
};
void arcturus_set_ppt_funcs(struct smu_context *smu)
return 0;
}
+static int navi10_set_mp1_state(struct smu_context *smu,
+ enum pp_mp1_state mp1_state)
+{
+ struct amdgpu_device *adev = smu->adev;
+ uint32_t mp1_fw_flags;
+ int ret = 0;
+
+ ret = smu_cmn_set_mp1_state(smu, mp1_state);
+ if (ret)
+ return ret;
+
+ if (mp1_state == PP_MP1_STATE_UNLOAD) {
+ mp1_fw_flags = RREG32_PCIE(MP1_Public |
+ (smnMP1_FIRMWARE_FLAGS & 0xffffffff));
+
+ mp1_fw_flags &= ~MP1_FIRMWARE_FLAGS__INTERRUPTS_ENABLED_MASK;
+
+ WREG32_PCIE(MP1_Public |
+ (smnMP1_FIRMWARE_FLAGS & 0xffffffff), mp1_fw_flags);
+ }
+
+ return 0;
+}
+
static int navi10_setup_pptable(struct smu_context *smu)
{
int ret = 0;
.get_fan_parameters = navi10_get_fan_parameters,
.post_init = navi10_post_smu_init,
.interrupt_work = smu_v11_0_interrupt_work,
+ .set_mp1_state = navi10_set_mp1_state,
};
void navi10_set_ppt_funcs(struct smu_context *smu)
return smu_v11_0_system_features_control(smu, en);
}
+static int sienna_cichlid_set_mp1_state(struct smu_context *smu,
+ enum pp_mp1_state mp1_state)
+{
+ int ret;
+
+ switch (mp1_state) {
+ case PP_MP1_STATE_UNLOAD:
+ ret = smu_cmn_set_mp1_state(smu, mp1_state);
+ break;
+ default:
+ /* Ignore others */
+ ret = 0;
+ }
+
+ return ret;
+}
+
static const struct pptable_funcs sienna_cichlid_ppt_funcs = {
.get_allowed_feature_mask = sienna_cichlid_get_allowed_feature_mask,
.set_default_dpm_table = sienna_cichlid_set_default_dpm_table,
.get_fan_parameters = sienna_cichlid_get_fan_parameters,
.interrupt_work = smu_v11_0_interrupt_work,
.gpo_control = sienna_cichlid_gpo_control,
+ .set_mp1_state = sienna_cichlid_set_mp1_state,
};
void sienna_cichlid_set_ppt_funcs(struct smu_context *smu)
smu->smu_table.boot_values.firmware_caps = v_3_1->firmware_capability;
break;
case 3:
+ case 4:
default:
v_3_3 = (struct atom_firmware_info_v3_3 *)header;
smu->smu_table.boot_values.revision = v_3_3->firmware_revision;
static bool vangogh_is_dpm_running(struct smu_context *smu)
{
+ struct amdgpu_device *adev = smu->adev;
int ret = 0;
uint32_t feature_mask[2];
uint64_t feature_enabled;
+ /* we need to re-init after suspend so return false */
+ if (adev->in_suspend)
+ return false;
+
ret = smu_cmn_get_enabled_32_bits_mask(smu, feature_mask, 2);
if (ret)
.get_ppt_limit = vangogh_get_ppt_limit,
.get_power_limit = vangogh_get_power_limit,
.set_power_limit = vangogh_set_power_limit,
+ .get_vbios_bootup_values = smu_v11_0_get_vbios_bootup_values,
};
void vangogh_set_ppt_funcs(struct smu_context *smu)
MSG_MAP(SetExecuteDMATest, PPSMC_MSG_SetExecuteDMATest, 0),
MSG_MAP(EnableDeterminism, PPSMC_MSG_EnableDeterminism, 0),
MSG_MAP(DisableDeterminism, PPSMC_MSG_DisableDeterminism, 0),
- MSG_MAP(SetUclkDpmMode, PPSMC_MSG_SetUclkDpmMode, 0),
+ MSG_MAP(SetUclkDpmMode, PPSMC_MSG_SetUclkDpmMode, 0),
+ MSG_MAP(GfxDriverResetRecovery, PPSMC_MSG_GfxDriverResetRecovery, 0),
};
static const struct cmn2asic_mapping aldebaran_clk_map[SMU_CLK_COUNT] = {
return !!(feature_enabled & SMC_DPM_FEATURE);
}
+static void aldebaran_fill_i2c_req(SwI2cRequest_t *req, bool write,
+ uint8_t address, uint32_t numbytes,
+ uint8_t *data)
+{
+ int i;
+
+ req->I2CcontrollerPort = 0;
+ req->I2CSpeed = 2;
+ req->SlaveAddress = address;
+ req->NumCmds = numbytes;
+
+ for (i = 0; i < numbytes; i++) {
+ SwI2cCmd_t *cmd = &req->SwI2cCmds[i];
+
+ /* First 2 bytes are always write for lower 2b EEPROM address */
+ if (i < 2)
+ cmd->CmdConfig = CMDCONFIG_READWRITE_MASK;
+ else
+ cmd->CmdConfig = write ? CMDCONFIG_READWRITE_MASK : 0;
+
+
+ /* Add RESTART for read after address filled */
+ cmd->CmdConfig |= (i == 2 && !write) ? CMDCONFIG_RESTART_MASK : 0;
+
+ /* Add STOP in the end */
+ cmd->CmdConfig |= (i == (numbytes - 1)) ? CMDCONFIG_STOP_MASK : 0;
+
+ /* Fill with data regardless if read or write to simplify code */
+ cmd->ReadWriteData = data[i];
+ }
+}
+
+static int aldebaran_i2c_read_data(struct i2c_adapter *control,
+ uint8_t address,
+ uint8_t *data,
+ uint32_t numbytes)
+{
+ uint32_t i, ret = 0;
+ SwI2cRequest_t req;
+ struct amdgpu_device *adev = to_amdgpu_device(control);
+ struct smu_table_context *smu_table = &adev->smu.smu_table;
+ struct smu_table *table = &smu_table->driver_table;
+
+ if (numbytes > MAX_SW_I2C_COMMANDS) {
+ dev_err(adev->dev, "numbytes requested %d is over max allowed %d\n",
+ numbytes, MAX_SW_I2C_COMMANDS);
+ return -EINVAL;
+ }
+
+ memset(&req, 0, sizeof(req));
+ aldebaran_fill_i2c_req(&req, false, address, numbytes, data);
+
+ mutex_lock(&adev->smu.mutex);
+ /* Now read data starting with that address */
+ ret = smu_cmn_update_table(&adev->smu, SMU_TABLE_I2C_COMMANDS, 0, &req,
+ true);
+ mutex_unlock(&adev->smu.mutex);
+
+ if (!ret) {
+ SwI2cRequest_t *res = (SwI2cRequest_t *)table->cpu_addr;
+
+ /* Assume SMU fills res.SwI2cCmds[i].Data with read bytes */
+ for (i = 0; i < numbytes; i++)
+ data[i] = res->SwI2cCmds[i].ReadWriteData;
+
+ dev_dbg(adev->dev, "aldebaran_i2c_read_data, address = %x, bytes = %d, data :",
+ (uint16_t)address, numbytes);
+
+ print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_NONE,
+ 8, 1, data, numbytes, false);
+ } else
+ dev_err(adev->dev, "aldebaran_i2c_read_data - error occurred :%x", ret);
+
+ return ret;
+}
+
+static int aldebaran_i2c_write_data(struct i2c_adapter *control,
+ uint8_t address,
+ uint8_t *data,
+ uint32_t numbytes)
+{
+ uint32_t ret;
+ SwI2cRequest_t req;
+ struct amdgpu_device *adev = to_amdgpu_device(control);
+
+ if (numbytes > MAX_SW_I2C_COMMANDS) {
+ dev_err(adev->dev, "numbytes requested %d is over max allowed %d\n",
+ numbytes, MAX_SW_I2C_COMMANDS);
+ return -EINVAL;
+ }
+
+ memset(&req, 0, sizeof(req));
+ aldebaran_fill_i2c_req(&req, true, address, numbytes, data);
+
+ mutex_lock(&adev->smu.mutex);
+ ret = smu_cmn_update_table(&adev->smu, SMU_TABLE_I2C_COMMANDS, 0, &req, true);
+ mutex_unlock(&adev->smu.mutex);
+
+ if (!ret) {
+ dev_dbg(adev->dev, "aldebaran_i2c_write(), address = %x, bytes = %d , data: ",
+ (uint16_t)address, numbytes);
+
+ print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_NONE,
+ 8, 1, data, numbytes, false);
+ /*
+ * According to EEPROM spec there is a MAX of 10 ms required for
+ * EEPROM to flush internal RX buffer after STOP was issued at the
+ * end of write transaction. During this time the EEPROM will not be
+ * responsive to any more commands - so wait a bit more.
+ */
+ msleep(10);
+
+ } else
+ dev_err(adev->dev, "aldebaran_i2c_write- error occurred :%x", ret);
+
+ return ret;
+}
+
+static int aldebaran_i2c_xfer(struct i2c_adapter *i2c_adap,
+ struct i2c_msg *msgs, int num)
+{
+ uint32_t i, j, ret, data_size, data_chunk_size, next_eeprom_addr = 0;
+ uint8_t *data_ptr, data_chunk[MAX_SW_I2C_COMMANDS] = { 0 };
+
+ for (i = 0; i < num; i++) {
+ /*
+ * SMU interface allows at most MAX_SW_I2C_COMMANDS bytes of data at
+ * once and hence the data needs to be spliced into chunks and sent each
+ * chunk separately
+ */
+ data_size = msgs[i].len - 2;
+ data_chunk_size = MAX_SW_I2C_COMMANDS - 2;
+ next_eeprom_addr = (msgs[i].buf[0] << 8 & 0xff00) | (msgs[i].buf[1] & 0xff);
+ data_ptr = msgs[i].buf + 2;
+
+ for (j = 0; j < data_size / data_chunk_size; j++) {
+ /* Insert the EEPROM dest addess, bits 0-15 */
+ data_chunk[0] = ((next_eeprom_addr >> 8) & 0xff);
+ data_chunk[1] = (next_eeprom_addr & 0xff);
+
+ if (msgs[i].flags & I2C_M_RD) {
+ ret = aldebaran_i2c_read_data(i2c_adap,
+ (uint8_t)msgs[i].addr,
+ data_chunk, MAX_SW_I2C_COMMANDS);
+
+ memcpy(data_ptr, data_chunk + 2, data_chunk_size);
+ } else {
+
+ memcpy(data_chunk + 2, data_ptr, data_chunk_size);
+
+ ret = aldebaran_i2c_write_data(i2c_adap,
+ (uint8_t)msgs[i].addr,
+ data_chunk, MAX_SW_I2C_COMMANDS);
+ }
+
+ if (ret) {
+ num = -EIO;
+ goto fail;
+ }
+
+ next_eeprom_addr += data_chunk_size;
+ data_ptr += data_chunk_size;
+ }
+
+ if (data_size % data_chunk_size) {
+ data_chunk[0] = ((next_eeprom_addr >> 8) & 0xff);
+ data_chunk[1] = (next_eeprom_addr & 0xff);
+
+ if (msgs[i].flags & I2C_M_RD) {
+ ret = aldebaran_i2c_read_data(i2c_adap,
+ (uint8_t)msgs[i].addr,
+ data_chunk, (data_size % data_chunk_size) + 2);
+
+ memcpy(data_ptr, data_chunk + 2, data_size % data_chunk_size);
+ } else {
+ memcpy(data_chunk + 2, data_ptr, data_size % data_chunk_size);
+
+ ret = aldebaran_i2c_write_data(i2c_adap,
+ (uint8_t)msgs[i].addr,
+ data_chunk, (data_size % data_chunk_size) + 2);
+ }
+
+ if (ret) {
+ num = -EIO;
+ goto fail;
+ }
+ }
+ }
+
+fail:
+ return num;
+}
+
+static u32 aldebaran_i2c_func(struct i2c_adapter *adap)
+{
+ return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL;
+}
+
+
+static const struct i2c_algorithm aldebaran_i2c_algo = {
+ .master_xfer = aldebaran_i2c_xfer,
+ .functionality = aldebaran_i2c_func,
+};
+
+static int aldebaran_i2c_control_init(struct smu_context *smu, struct i2c_adapter *control)
+{
+ struct amdgpu_device *adev = to_amdgpu_device(control);
+ int res;
+
+ control->owner = THIS_MODULE;
+ control->class = I2C_CLASS_SPD;
+ control->dev.parent = &adev->pdev->dev;
+ control->algo = &aldebaran_i2c_algo;
+ snprintf(control->name, sizeof(control->name), "AMDGPU SMU");
+
+ res = i2c_add_adapter(control);
+ if (res)
+ DRM_ERROR("Failed to register hw i2c, err: %d\n", res);
+
+ return res;
+}
+
+static void aldebaran_i2c_control_fini(struct smu_context *smu, struct i2c_adapter *control)
+{
+ i2c_del_adapter(control);
+}
+
static void aldebaran_get_unique_id(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
return sizeof(struct gpu_metrics_v1_1);
}
+static int aldebaran_mode2_reset(struct smu_context *smu)
+{
+ u32 smu_version;
+ int ret = 0, index;
+ struct amdgpu_device *adev = smu->adev;
+ int timeout = 10;
+
+ smu_cmn_get_smc_version(smu, NULL, &smu_version);
+
+ index = smu_cmn_to_asic_specific_index(smu, CMN2ASIC_MAPPING_MSG,
+ SMU_MSG_GfxDeviceDriverReset);
+
+ mutex_lock(&smu->message_lock);
+ if (smu_version >= 0x00441400) {
+ ret = smu_cmn_send_msg_without_waiting(smu, (uint16_t)index, SMU_RESET_MODE_2);
+ /* This is similar to FLR, wait till max FLR timeout */
+ msleep(100);
+ dev_dbg(smu->adev->dev, "restore config space...\n");
+ /* Restore the config space saved during init */
+ amdgpu_device_load_pci_state(adev->pdev);
+
+ dev_dbg(smu->adev->dev, "wait for reset ack\n");
+ while (ret == -ETIME && timeout) {
+ ret = smu_cmn_wait_for_response(smu);
+ /* Wait a bit more time for getting ACK */
+ if (ret == -ETIME) {
+ --timeout;
+ usleep_range(500, 1000);
+ continue;
+ }
+
+ if (ret != 1) {
+ dev_err(adev->dev, "failed to send mode2 message \tparam: 0x%08x response %#x\n",
+ SMU_RESET_MODE_2, ret);
+ goto out;
+ }
+ }
+
+ } else {
+ dev_err(adev->dev, "smu fw 0x%x does not support MSG_GfxDeviceDriverReset MSG\n",
+ smu_version);
+ }
+
+ if (ret == 1)
+ ret = 0;
+out:
+ mutex_unlock(&smu->message_lock);
+
+ return ret;
+}
+
static bool aldebaran_is_mode1_reset_supported(struct smu_context *smu)
{
#if 0
return true;
}
+static int aldebaran_set_mp1_state(struct smu_context *smu,
+ enum pp_mp1_state mp1_state)
+{
+ switch (mp1_state) {
+ case PP_MP1_STATE_UNLOAD:
+ return smu_cmn_set_mp1_state(smu, mp1_state);
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static const struct pptable_funcs aldebaran_ppt_funcs = {
/* init dpm */
.get_allowed_feature_mask = aldebaran_get_allowed_feature_mask,
.mode1_reset_is_support = aldebaran_is_mode1_reset_supported,
.mode2_reset_is_support = aldebaran_is_mode2_reset_supported,
.mode1_reset = smu_v13_0_mode1_reset,
- .mode2_reset = smu_v13_0_mode2_reset,
+ .set_mp1_state = aldebaran_set_mp1_state,
+ .mode2_reset = aldebaran_mode2_reset,
+ .wait_for_event = smu_v13_0_wait_for_event,
+ .i2c_init = aldebaran_i2c_control_init,
+ .i2c_fini = aldebaran_i2c_control_fini,
};
void aldebaran_set_ppt_funcs(struct smu_context *smu)
#define PCIE_LC_SPEED_CNTL__LC_CURRENT_DATA_RATE_MASK 0xC000
#define PCIE_LC_SPEED_CNTL__LC_CURRENT_DATA_RATE__SHIFT 0xE
-static int link_width[] = {0, 1, 2, 4, 8, 12, 16};
-static int link_speed[] = {25, 50, 80, 160};
+static const int link_width[] = {0, 1, 2, 4, 8, 12, 16};
+static const int link_speed[] = {25, 50, 80, 160};
int smu_v13_0_init_microcode(struct smu_context *smu)
{
return ret;
}
-int smu_v13_0_mode2_reset(struct smu_context *smu)
+static int smu_v13_0_wait_for_reset_complete(struct smu_context *smu,
+ uint64_t event_arg)
{
- u32 smu_version;
int ret = 0;
- struct amdgpu_device *adev = smu->adev;
- smu_cmn_get_smc_version(smu, NULL, &smu_version);
- if (smu_version >= 0x00440700)
- ret = smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_GfxDeviceDriverReset, SMU_RESET_MODE_2, NULL);
- else
- dev_err(adev->dev, "smu fw 0x%x does not support MSG_GfxDeviceDriverReset MSG\n", smu_version);
- /*TODO: mode2 reset wait time should be shorter, will modify it later*/
+
+ dev_dbg(smu->adev->dev, "waiting for smu reset complete\n");
+ ret = smu_cmn_send_smc_msg(smu, SMU_MSG_GfxDriverResetRecovery, NULL);
+
+ return ret;
+}
+
+int smu_v13_0_wait_for_event(struct smu_context *smu, enum smu_event_type event,
+ uint64_t event_arg)
+{
+ int ret = -EINVAL;
+
+ switch (event) {
+ case SMU_EVENT_RESET_COMPLETE:
+ ret = smu_v13_0_wait_for_reset_complete(smu, event_arg);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+int smu_v13_0_mode2_reset(struct smu_context *smu)
+{
+ int ret;
+
+ ret = smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_GfxDeviceDriverReset,
+ SMU_RESET_MODE_2, NULL);
+ /*TODO: mode2 reset wait time should be shorter, add ASIC specific func if required */
if (!ret)
msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS);
+
return ret;
}
enum smu_clk_type clk_type,
uint32_t *value)
{
- return smu_v13_0_get_dpm_freq_by_index(smu,
- clk_type,
- 0xff,
- value);
+ int ret;
+
+ ret = smu_v13_0_get_dpm_freq_by_index(smu, clk_type, 0xff, value);
+ /* FW returns 0 based max level, increment by one */
+ if (!ret && value)
+ ++(*value);
+
+ return ret;
}
int smu_v13_0_set_single_dpm_table(struct smu_context *smu,
*arg = RREG32_SOC15(MP1, 0, mmMP1_SMN_C2PMSG_82);
}
-static int smu_cmn_wait_for_response(struct smu_context *smu)
+int smu_cmn_wait_for_response(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
- uint32_t cur_value, i, timeout = adev->usec_timeout * 10;
+ uint32_t cur_value, i, timeout = adev->usec_timeout * 20;
for (i = 0; i < timeout; i++) {
cur_value = RREG32_SOC15(MP1, 0, mmMP1_SMN_C2PMSG_90);
header->structure_size = structure_size;
}
+
+int smu_cmn_set_mp1_state(struct smu_context *smu,
+ enum pp_mp1_state mp1_state)
+{
+ enum smu_message_type msg;
+ int ret;
+
+ switch (mp1_state) {
+ case PP_MP1_STATE_SHUTDOWN:
+ msg = SMU_MSG_PrepareMp1ForShutdown;
+ break;
+ case PP_MP1_STATE_UNLOAD:
+ msg = SMU_MSG_PrepareMp1ForUnload;
+ break;
+ case PP_MP1_STATE_RESET:
+ msg = SMU_MSG_PrepareMp1ForReset;
+ break;
+ case PP_MP1_STATE_NONE:
+ default:
+ return 0;
+ }
+
+ ret = smu_cmn_send_smc_msg(smu, msg, NULL);
+ if (ret)
+ dev_err(smu->adev->dev, "[PrepareMp1] Failed!\n");
+
+ return ret;
+}
enum smu_message_type msg,
uint32_t *read_arg);
+int smu_cmn_wait_for_response(struct smu_context *smu);
+
int smu_cmn_to_asic_specific_index(struct smu_context *smu,
enum smu_cmn2asic_mapping_type type,
uint32_t index);
void smu_cmn_init_soft_gpu_metrics(void *table, uint8_t frev, uint8_t crev);
+int smu_cmn_set_mp1_state(struct smu_context *smu,
+ enum pp_mp1_state mp1_state);
+
#endif
#endif
#define has_bit(nr, mask) (BIT(nr) & (mask))
#define has_bits(bits, mask) (((bits) & (mask)) == (bits))
-#define dp_for_each_set_bit(bit, mask) \
- for_each_set_bit((bit), ((unsigned long *)&(mask)), sizeof(mask) * 8)
-
#define dp_wait_cond(__cond, __tries, __min_range, __max_range) \
({ \
int num_tries = __tries; \
{
struct komeda_dev *mdev = dev_to_mdev(dev);
- return snprintf(buf, PAGE_SIZE, "0x%08x\n", mdev->chip.core_id);
+ return sysfs_emit(buf, "0x%08x\n", mdev->chip.core_id);
}
static DEVICE_ATTR_RO(core_id);
if (pipe->layers[i]->layer_type == KOMEDA_FMT_RICH_LAYER)
config_id.n_richs++;
}
- return snprintf(buf, PAGE_SIZE, "0x%08x\n", config_id.value);
+ return sysfs_emit(buf, "0x%08x\n", config_id.value);
}
static DEVICE_ATTR_RO(config_id);
{
struct komeda_dev *mdev = dev_to_mdev(dev);
- return snprintf(buf, PAGE_SIZE, "%lu\n", clk_get_rate(mdev->aclk));
+ return sysfs_emit(buf, "%lu\n", clk_get_rate(mdev->aclk));
}
static DEVICE_ATTR_RO(aclk_hz);
{
struct komeda_component *c;
int i;
+ unsigned long avail_comps = pipe->avail_comps;
- dp_for_each_set_bit(i, pipe->avail_comps) {
+ for_each_set_bit(i, &avail_comps, 32) {
c = komeda_pipeline_get_component(pipe, i);
komeda_component_destroy(mdev, c);
}
{
struct komeda_component *c;
int id;
+ unsigned long avail_comps = pipe->avail_comps;
DRM_INFO("Pipeline-%d: n_layers: %d, n_scalers: %d, output: %s.\n",
pipe->id, pipe->n_layers, pipe->n_scalers,
pipe->of_output_links[1] ?
pipe->of_output_links[1]->full_name : "none");
- dp_for_each_set_bit(id, pipe->avail_comps) {
+ for_each_set_bit(id, &avail_comps, 32) {
c = komeda_pipeline_get_component(pipe, id);
komeda_component_dump(c);
struct komeda_pipeline *pipe = c->pipeline;
struct komeda_component *input;
int id;
+ unsigned long supported_inputs = c->supported_inputs;
- dp_for_each_set_bit(id, c->supported_inputs) {
+ for_each_set_bit(id, &supported_inputs, 32) {
input = komeda_pipeline_get_component(pipe, id);
if (!input) {
c->supported_inputs &= ~(BIT(id));
struct komeda_component *c;
struct komeda_layer *layer;
int i, id;
+ unsigned long avail_comps = pipe->avail_comps;
- dp_for_each_set_bit(id, pipe->avail_comps) {
+ for_each_set_bit(id, &avail_comps, 32) {
c = komeda_pipeline_get_component(pipe, id);
komeda_component_verify_inputs(c);
}
{
struct komeda_component *c;
u32 id;
+ unsigned long avail_comps;
seq_printf(sf, "\n======== Pipeline-%d ==========\n", pipe->id);
if (pipe->funcs && pipe->funcs->dump_register)
pipe->funcs->dump_register(pipe, sf);
- dp_for_each_set_bit(id, pipe->avail_comps) {
+ avail_comps = pipe->avail_comps;
+ for_each_set_bit(id, &avail_comps, 32) {
c = komeda_pipeline_get_component(pipe, id);
seq_printf(sf, "\n------%s------\n", c->name);
struct komeda_pipeline_state *old = priv_to_pipe_st(pipe->obj.state);
struct komeda_component_state *c_st;
struct komeda_component *c;
- u32 disabling_comps, id;
+ u32 id;
+ unsigned long disabling_comps;
WARN_ON(!old);
disabling_comps = (~new->active_comps) & old->active_comps;
/* unbound all disabling component */
- dp_for_each_set_bit(id, disabling_comps) {
+ for_each_set_bit(id, &disabling_comps, 32) {
c = komeda_pipeline_get_component(pipe, id);
c_st = komeda_component_get_state_and_set_user(c,
drm_st, NULL, new->crtc);
struct komeda_pipeline_state *old;
struct komeda_component *c;
struct komeda_component_state *c_st;
- u32 id, disabling_comps = 0;
+ u32 id;
+ unsigned long disabling_comps;
old = komeda_pipeline_get_old_state(pipe, old_state);
disabling_comps = old->active_comps &
pipe->standalone_disabled_comps;
- DRM_DEBUG_ATOMIC("PIPE%d: active_comps: 0x%x, disabling_comps: 0x%x.\n",
+ DRM_DEBUG_ATOMIC("PIPE%d: active_comps: 0x%x, disabling_comps: 0x%lx.\n",
pipe->id, old->active_comps, disabling_comps);
- dp_for_each_set_bit(id, disabling_comps) {
+ for_each_set_bit(id, &disabling_comps, 32) {
c = komeda_pipeline_get_component(pipe, id);
c_st = priv_to_comp_st(c->obj.state);
struct komeda_pipeline_state *new = priv_to_pipe_st(pipe->obj.state);
struct komeda_pipeline_state *old;
struct komeda_component *c;
- u32 id, changed_comps = 0;
+ u32 id;
+ unsigned long changed_comps;
old = komeda_pipeline_get_old_state(pipe, old_state);
changed_comps = new->active_comps | old->active_comps;
- DRM_DEBUG_ATOMIC("PIPE%d: active_comps: 0x%x, changed: 0x%x.\n",
+ DRM_DEBUG_ATOMIC("PIPE%d: active_comps: 0x%x, changed: 0x%lx.\n",
pipe->id, new->active_comps, changed_comps);
- dp_for_each_set_bit(id, changed_comps) {
+ for_each_set_bit(id, &changed_comps, 32) {
c = komeda_pipeline_get_component(pipe, id);
if (new->active_comps & BIT(c->id))
depends on OF
help
Driver for display connectors with support for DDC and hot-plug
- detection. Most display controller handle display connectors
+ detection. Most display controllers handle display connectors
internally and don't need this driver, but the DRM subsystem is
moving towards separating connector handling from display controllers
on ARM-based platforms. Saying Y here when this driver is not needed
tristate "TC358762 DSI/DPI bridge"
depends on OF
select DRM_MIPI_DSI
+ select DRM_KMS_HELPER
select DRM_PANEL_BRIDGE
help
Toshiba TC358762 DSI/DPI bridge driver.
#include <linux/device.h>
#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
#include <linux/i2c.h>
#include <linux/gpio.h>
#include <linux/of_gpio.h>
* This function walks the current configuration and adds all connectors
* currently using @crtc to the atomic configuration @state. Note that this
* function must acquire the connection mutex. This can potentially cause
- * unneeded seralization if the update is just for the planes on one CRTC. Hence
+ * unneeded serialization if the update is just for the planes on one CRTC. Hence
* drivers and helpers should only call this when really needed (e.g. when a
* full modeset needs to happen due to some change).
*
*
* Since acquiring a plane state will always also acquire the w/w mutex of the
* current CRTC for that plane (if there is any) adding all the plane states for
- * a CRTC will not reduce parallism of atomic updates.
+ * a CRTC will not reduce parallelism of atomic updates.
*
* Returns:
* 0 on success or can fail with -EDEADLK or -ENOMEM. When the error is EDEADLK
return 0;
up_req = kzalloc(sizeof(*up_req), GFP_KERNEL);
- if (!up_req) {
- DRM_ERROR("Not enough memory to process MST up req\n");
+ if (!up_req)
return -ENOMEM;
- }
+
INIT_LIST_HEAD(&up_req->next);
drm_dp_sideband_parse_req(&mgr->up_req_recv, &up_req->msg);
void drm_sysfs_lease_event(struct drm_device *dev);
/* drm_gem.c */
-struct drm_gem_object;
int drm_gem_init(struct drm_device *dev);
int drm_gem_handle_create_tail(struct drm_file *file_priv,
struct drm_gem_object *obj,
{
struct drm_display_mode *mode;
+ if (cmd->xres == 0 || cmd->yres == 0)
+ return NULL;
+
if (cmd->cvt)
mode = drm_cvt_mode(dev,
cmd->xres, cmd->yres,
*
* Assign a already signaled stub fence to the sync object.
*/
-static void drm_syncobj_assign_null_handle(struct drm_syncobj *syncobj)
+static int drm_syncobj_assign_null_handle(struct drm_syncobj *syncobj)
{
- struct dma_fence *fence = dma_fence_get_stub();
+ struct dma_fence *fence = dma_fence_allocate_private_stub();
+
+ if (IS_ERR(fence))
+ return PTR_ERR(fence);
drm_syncobj_replace_fence(syncobj, fence);
dma_fence_put(fence);
+ return 0;
}
/* 5s default for wait submission */
int drm_syncobj_create(struct drm_syncobj **out_syncobj, uint32_t flags,
struct dma_fence *fence)
{
+ int ret;
struct drm_syncobj *syncobj;
syncobj = kzalloc(sizeof(struct drm_syncobj), GFP_KERNEL);
INIT_LIST_HEAD(&syncobj->cb_list);
spin_lock_init(&syncobj->lock);
- if (flags & DRM_SYNCOBJ_CREATE_SIGNALED)
- drm_syncobj_assign_null_handle(syncobj);
+ if (flags & DRM_SYNCOBJ_CREATE_SIGNALED) {
+ ret = drm_syncobj_assign_null_handle(syncobj);
+ if (ret < 0) {
+ drm_syncobj_put(syncobj);
+ return ret;
+ }
+ }
if (fence)
drm_syncobj_replace_fence(syncobj, fence);
if (ret < 0)
return ret;
- for (i = 0; i < args->count_handles; i++)
- drm_syncobj_assign_null_handle(syncobjs[i]);
+ for (i = 0; i < args->count_handles; i++) {
+ ret = drm_syncobj_assign_null_handle(syncobjs[i]);
+ if (ret < 0)
+ break;
+ }
drm_syncobj_array_free(syncobjs, args->count_handles);
u64 diff_ns;
u32 cur_vblank, diff = 1;
int count = DRM_TIMESTAMP_MAXRETRIES;
+ u32 max_vblank_count = drm_max_vblank_count(dev, pipe);
if (drm_WARN_ON(dev, pipe >= dev->num_crtcs))
return;
drm_dbg_vbl(dev,
"missed %d vblanks in %lld ns, frame duration=%d ns, hw_diff=%d\n",
diff, diff_ns, framedur_ns, cur_vblank - vblank->last);
- store_vblank(dev, pipe, diff, t_vblank, cur_vblank);
+ vblank->last = (cur_vblank - diff) & max_vblank_count;
}
/**
struct page **pages = pvec + pinned;
ret = pin_user_pages_fast(ptr, num_pages,
- !userptr->ro ? FOLL_WRITE : 0, pages);
+ FOLL_WRITE | FOLL_FORCE | FOLL_LONGTERM,
+ pages);
if (ret < 0) {
unpin_user_pages(pvec, pinned);
kvfree(pvec);
#include <linux/irq.h>
#include <linux/mfd/syscon.h>
#include <linux/of_device.h>
-#include <linux/of_gpio.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/regmap.h>
#include <linux/pm_runtime.h>
static struct mutex power_mutex; /* Serialize power ops */
-static spinlock_t power_ctrl_lock; /* Serialize power claim */
+static DEFINE_SPINLOCK(power_ctrl_lock); /* Serialize power claim */
/**
* gma_power_init - initialise power manager
dev_priv->display_power = true; /* We start active */
dev_priv->display_count = 0; /* Currently no users */
dev_priv->suspended = false; /* And not suspended */
- spin_lock_init(&power_ctrl_lock);
mutex_init(&power_mutex);
if (dev_priv->ops->init_pm)
return;
}
+ if (!pkg->package.count) {
+ DRM_DEBUG_DRIVER("no connection in _DSM\n");
+ return;
+ }
+
connector_count = &pkg->package.elements[0];
DRM_DEBUG_DRIVER("MUX info connectors: %lld\n",
(unsigned long long)connector_count->integer.value);
for (i = 1; i < pkg->package.count; i++) {
union acpi_object *obj = &pkg->package.elements[i];
- union acpi_object *connector_id = &obj->package.elements[0];
- union acpi_object *info = &obj->package.elements[1];
+ union acpi_object *connector_id;
+ union acpi_object *info;
+
+ if (obj->type != ACPI_TYPE_PACKAGE || obj->package.count < 2) {
+ DRM_DEBUG_DRIVER("Invalid object for MUX #%d\n", i);
+ continue;
+ }
+
+ connector_id = &obj->package.elements[0];
+ info = &obj->package.elements[1];
+ if (info->type != ACPI_TYPE_BUFFER || info->buffer.length < 4) {
+ DRM_DEBUG_DRIVER("Invalid info for MUX obj #%d\n", i);
+ continue;
+ }
+
DRM_DEBUG_DRIVER("Connector id: 0x%016llx\n",
(unsigned long long)connector_id->integer.value);
DRM_DEBUG_DRIVER(" port id: %s\n",
ret = drmm_mode_config_init(drm);
if (ret)
- return ret;
+ goto err_kms;
ret = drm_vblank_init(drm, MAX_CRTC);
if (ret)
int dual = ldb->ldb_ctrl & LDB_SPLIT_MODE_EN;
int mux = drm_of_encoder_active_port_id(imx_ldb_ch->child, encoder);
+ if (mux < 0 || mux >= ARRAY_SIZE(ldb->clk_sel)) {
+ dev_warn(ldb->dev, "%s: invalid mux %d\n", __func__, mux);
+ return;
+ }
+
drm_panel_prepare(imx_ldb_ch->panel);
if (dual) {
int mux = drm_of_encoder_active_port_id(imx_ldb_ch->child, encoder);
u32 bus_format = imx_ldb_ch->bus_format;
+ if (mux < 0 || mux >= ARRAY_SIZE(ldb->clk_sel)) {
+ dev_warn(ldb->dev, "%s: invalid mux %d\n", __func__, mux);
+ return;
+ }
+
if (mode->clock > 170000) {
dev_warn(ldb->dev,
"%s: mode exceeds 170 MHz pixel clock\n", __func__);
struct imx_ldb_channel *channel = &imx_ldb->channel[i];
if (!channel->ldb)
- break;
+ continue;
ret = imx_ldb_register(drm, channel);
if (ret)
select SND_SOC_HDMI_CODEC if SND_SOC
select SYNC_FILE
select PM_OPP
+ select NVMEM
help
DRM/KMS driver for MSM/snapdragon.
Choose this option if you have a need for MIPI DSI connector
support.
-config DRM_MSM_DSI_PLL
- bool "Enable DSI PLL driver in MSM DRM"
- depends on DRM_MSM_DSI && COMMON_CLK
- default y
- help
- Choose this option to enable DSI PLL driver which provides DSI
- source clocks under common clock framework.
-
config DRM_MSM_DSI_28NM_PHY
bool "Enable DSI 28nm PHY driver in MSM DRM"
depends on DRM_MSM_DSI
msm-$(CONFIG_DRM_MSM_DSI_10NM_PHY) += dsi/phy/dsi_phy_10nm.o
msm-$(CONFIG_DRM_MSM_DSI_7NM_PHY) += dsi/phy/dsi_phy_7nm.o
-ifeq ($(CONFIG_DRM_MSM_DSI_PLL),y)
-msm-y += dsi/pll/dsi_pll.o
-msm-$(CONFIG_DRM_MSM_DSI_28NM_PHY) += dsi/pll/dsi_pll_28nm.o
-msm-$(CONFIG_DRM_MSM_DSI_28NM_8960_PHY) += dsi/pll/dsi_pll_28nm_8960.o
-msm-$(CONFIG_DRM_MSM_DSI_14NM_PHY) += dsi/pll/dsi_pll_14nm.o
-msm-$(CONFIG_DRM_MSM_DSI_10NM_PHY) += dsi/pll/dsi_pll_10nm.o
-msm-$(CONFIG_DRM_MSM_DSI_7NM_PHY) += dsi/pll/dsi_pll_7nm.o
-endif
-
obj-$(CONFIG_DRM_MSM) += msm.o
static int a5xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value)
{
- *value = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_CP_0_LO,
- REG_A5XX_RBBM_PERFCTR_CP_0_HI);
+ *value = gpu_read64(gpu, REG_A5XX_RBBM_ALWAYSON_COUNTER_LO,
+ REG_A5XX_RBBM_ALWAYSON_COUNTER_HI);
return 0;
}
/* Set up the limits management */
if (adreno_is_a530(adreno_gpu))
a530_lm_setup(gpu);
- else
+ else if (adreno_is_a540(adreno_gpu))
a540_lm_setup(gpu);
/* Set up SP/TP power collpase */
}
struct a6xx_gmu_oob_bits {
- int set, ack, set_new, ack_new;
+ int set, ack, set_new, ack_new, clear, clear_new;
const char *name;
};
.ack = 24,
.set_new = 30,
.ack_new = 31,
+ .clear = 24,
+ .clear_new = 31,
},
[GMU_OOB_PERFCOUNTER_SET] = {
.ack = 25,
.set_new = 28,
.ack_new = 30,
+ .clear = 25,
+ .clear_new = 29,
},
[GMU_OOB_BOOT_SLUMBER] = {
.name = "BOOT_SLUMBER",
.set = 22,
.ack = 30,
+ .clear = 30,
},
[GMU_OOB_DCVS_SET] = {
.name = "GPU_DCVS",
.set = 23,
.ack = 31,
+ .clear = 31,
},
};
return;
if (gmu->legacy)
- bit = a6xx_gmu_oob_bits[state].ack;
+ bit = a6xx_gmu_oob_bits[state].clear;
else
- bit = a6xx_gmu_oob_bits[state].ack_new;
+ bit = a6xx_gmu_oob_bits[state].clear_new;
- gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, bit);
+ gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 1 << bit);
}
/* Enable CPU control of SPTP power power collapse */
return a6xx_idle(gpu, ring) ? 0 : -EINVAL;
}
-static void a6xx_ucode_check_version(struct a6xx_gpu *a6xx_gpu,
+/*
+ * Check that the microcode version is new enough to include several key
+ * security fixes. Return true if the ucode is safe.
+ */
+static bool a6xx_ucode_check_version(struct a6xx_gpu *a6xx_gpu,
struct drm_gem_object *obj)
{
+ struct adreno_gpu *adreno_gpu = &a6xx_gpu->base;
+ struct msm_gpu *gpu = &adreno_gpu->base;
u32 *buf = msm_gem_get_vaddr(obj);
+ bool ret = false;
if (IS_ERR(buf))
- return;
+ return false;
/*
- * If the lowest nibble is 0xa that is an indication that this microcode
- * has been patched. The actual version is in dword [3] but we only care
- * about the patchlevel which is the lowest nibble of dword [3]
- *
- * Otherwise check that the firmware is greater than or equal to 1.90
- * which was the first version that had this fix built in
+ * Targets up to a640 (a618, a630 and a640) need to check for a
+ * microcode version that is patched to support the whereami opcode or
+ * one that is new enough to include it by default.
*/
- if (((buf[0] & 0xf) == 0xa) && (buf[2] & 0xf) >= 1)
- a6xx_gpu->has_whereami = true;
- else if ((buf[0] & 0xfff) > 0x190)
- a6xx_gpu->has_whereami = true;
+ if (adreno_is_a618(adreno_gpu) || adreno_is_a630(adreno_gpu) ||
+ adreno_is_a640(adreno_gpu)) {
+ /*
+ * If the lowest nibble is 0xa that is an indication that this
+ * microcode has been patched. The actual version is in dword
+ * [3] but we only care about the patchlevel which is the lowest
+ * nibble of dword [3]
+ *
+ * Otherwise check that the firmware is greater than or equal
+ * to 1.90 which was the first version that had this fix built
+ * in
+ */
+ if ((((buf[0] & 0xf) == 0xa) && (buf[2] & 0xf) >= 1) ||
+ (buf[0] & 0xfff) >= 0x190) {
+ a6xx_gpu->has_whereami = true;
+ ret = true;
+ goto out;
+ }
+ DRM_DEV_ERROR(&gpu->pdev->dev,
+ "a630 SQE ucode is too old. Have version %x need at least %x\n",
+ buf[0] & 0xfff, 0x190);
+ } else {
+ /*
+ * a650 tier targets don't need whereami but still need to be
+ * equal to or newer than 0.95 for other security fixes
+ */
+ if (adreno_is_a650(adreno_gpu)) {
+ if ((buf[0] & 0xfff) >= 0x095) {
+ ret = true;
+ goto out;
+ }
+
+ DRM_DEV_ERROR(&gpu->pdev->dev,
+ "a650 SQE ucode is too old. Have version %x need at least %x\n",
+ buf[0] & 0xfff, 0x095);
+ }
+
+ /*
+ * When a660 is added those targets should return true here
+ * since those have all the critical security fixes built in
+ * from the start
+ */
+ }
+out:
msm_gem_put_vaddr(obj);
+ return ret;
}
static int a6xx_ucode_init(struct msm_gpu *gpu)
}
msm_gem_object_set_name(a6xx_gpu->sqe_bo, "sqefw");
- a6xx_ucode_check_version(a6xx_gpu, a6xx_gpu->sqe_bo);
+ if (!a6xx_ucode_check_version(a6xx_gpu, a6xx_gpu->sqe_bo)) {
+ msm_gem_unpin_iova(a6xx_gpu->sqe_bo, gpu->aspace);
+ drm_gem_object_put(a6xx_gpu->sqe_bo);
+
+ a6xx_gpu->sqe_bo = NULL;
+ return -EPERM;
+ }
}
gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE_LO,
/* Force the GPU power on so we can read this register */
a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_PERFCOUNTER_SET);
- *value = gpu_read64(gpu, REG_A6XX_RBBM_PERFCTR_CP_0_LO,
- REG_A6XX_RBBM_PERFCTR_CP_0_HI);
+ *value = gpu_read64(gpu, REG_A6XX_CP_ALWAYS_ON_COUNTER_LO,
+ REG_A6XX_CP_ALWAYS_ON_COUNTER_HI);
a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_PERFCOUNTER_SET);
mutex_unlock(&perfcounter_oob);
u32 revn)
{
struct opp_table *opp_table;
- struct nvmem_cell *cell;
u32 supp_hw = UINT_MAX;
- void *buf;
+ u16 speedbin;
+ int ret;
- cell = nvmem_cell_get(dev, "speed_bin");
+ ret = nvmem_cell_read_u16(dev, "speed_bin", &speedbin);
/*
* -ENOENT means that the platform doesn't support speedbin which is
* fine
*/
- if (PTR_ERR(cell) == -ENOENT)
+ if (ret == -ENOENT) {
return 0;
- else if (IS_ERR(cell)) {
- DRM_DEV_ERROR(dev,
- "failed to read speed-bin. Some OPPs may not be supported by hardware");
- goto done;
- }
-
- buf = nvmem_cell_read(cell, NULL);
- if (IS_ERR(buf)) {
- nvmem_cell_put(cell);
+ } else if (ret) {
DRM_DEV_ERROR(dev,
- "failed to read speed-bin. Some OPPs may not be supported by hardware");
+ "failed to read speed-bin (%d). Some OPPs may not be supported by hardware",
+ ret);
goto done;
}
+ speedbin = le16_to_cpu(speedbin);
- supp_hw = fuse_to_supp_hw(dev, revn, *((u32 *) buf));
-
- kfree(buf);
- nvmem_cell_put(cell);
+ supp_hw = fuse_to_supp_hw(dev, revn, speedbin);
done:
opp_table = dev_pm_opp_set_supported_hw(dev, &supp_hw, 1);
case MSM_PARAM_FAULTS:
*value = gpu->global_faults;
return 0;
+ case MSM_PARAM_SUSPENDS:
+ *value = gpu->suspend_count;
+ return 0;
default:
DBG("%s: invalid param: %u", gpu->name, param);
return -EINVAL;
if (!dpu_kms->hw_intr || !dpu_kms->hw_intr->ops.irq_idx_lookup)
return -EINVAL;
- return dpu_kms->hw_intr->ops.irq_idx_lookup(intr_type,
- instance_idx);
+ return dpu_kms->hw_intr->ops.irq_idx_lookup(dpu_kms->hw_intr,
+ intr_type, instance_idx);
}
/**
} else {
DPU_DEBUG("crtc=%d disable\n", crtc->base.id);
memset(old, 0, sizeof(*old));
- memset(new, 0, sizeof(*new));
update_bus = true;
update_clk = true;
}
kfree(dpu_crtc);
}
+static struct drm_encoder *get_encoder_from_crtc(struct drm_crtc *crtc)
+{
+ struct drm_device *dev = crtc->dev;
+ struct drm_encoder *encoder;
+
+ drm_for_each_encoder(encoder, dev)
+ if (encoder->crtc == crtc)
+ return encoder;
+
+ return NULL;
+}
+
+static u32 dpu_crtc_get_vblank_counter(struct drm_crtc *crtc)
+{
+ struct drm_encoder *encoder;
+
+ encoder = get_encoder_from_crtc(crtc);
+ if (!encoder) {
+ DRM_ERROR("no encoder found for crtc %d\n", crtc->index);
+ return false;
+ }
+
+ return dpu_encoder_get_frame_count(encoder);
+}
+
+static bool dpu_crtc_get_scanout_position(struct drm_crtc *crtc,
+ bool in_vblank_irq,
+ int *vpos, int *hpos,
+ ktime_t *stime, ktime_t *etime,
+ const struct drm_display_mode *mode)
+{
+ unsigned int pipe = crtc->index;
+ struct drm_encoder *encoder;
+ int line, vsw, vbp, vactive_start, vactive_end, vfp_end;
+
+ encoder = get_encoder_from_crtc(crtc);
+ if (!encoder) {
+ DRM_ERROR("no encoder found for crtc %d\n", pipe);
+ return false;
+ }
+
+ vsw = mode->crtc_vsync_end - mode->crtc_vsync_start;
+ vbp = mode->crtc_vtotal - mode->crtc_vsync_end;
+
+ /*
+ * the line counter is 1 at the start of the VSYNC pulse and VTOTAL at
+ * the end of VFP. Translate the porch values relative to the line
+ * counter positions.
+ */
+
+ vactive_start = vsw + vbp + 1;
+ vactive_end = vactive_start + mode->crtc_vdisplay;
+
+ /* last scan line before VSYNC */
+ vfp_end = mode->crtc_vtotal;
+
+ if (stime)
+ *stime = ktime_get();
+
+ line = dpu_encoder_get_linecount(encoder);
+
+ if (line < vactive_start)
+ line -= vactive_start;
+ else if (line > vactive_end)
+ line = line - vfp_end - vactive_start;
+ else
+ line -= vactive_start;
+
+ *vpos = line;
+ *hpos = 0;
+
+ if (etime)
+ *etime = ktime_get();
+
+ return true;
+}
+
static void _dpu_crtc_setup_blend_cfg(struct dpu_crtc_mixer *mixer,
struct dpu_plane_state *pstate, struct dpu_format *format)
{
uint32_t stage_idx, lm_idx;
int zpos_cnt[DPU_STAGE_MAX + 1] = { 0 };
bool bg_alpha_enable = false;
+ DECLARE_BITMAP(fetch_active, SSPP_MAX);
+ memset(fetch_active, 0, sizeof(fetch_active));
drm_atomic_crtc_for_each_plane(plane, crtc) {
state = plane->state;
if (!state)
fb = state->fb;
dpu_plane_get_ctl_flush(plane, ctl, &flush_mask);
-
+ set_bit(dpu_plane_pipe(plane), fetch_active);
DPU_DEBUG("crtc %d stage:%d - plane %d sspp %d fb %d\n",
crtc->base.id,
pstate->stage,
}
}
+ if (ctl->ops.set_active_pipes)
+ ctl->ops.set_active_pipes(ctl, fetch_active);
+
_dpu_crtc_program_lm_output_roi(crtc);
}
DPU_DEBUG("crtc%d -> enable %d, active %d, skip atomic_check\n",
crtc->base.id, crtc_state->enable,
crtc_state->active);
+ memset(&cstate->new_perf, 0, sizeof(cstate->new_perf));
goto end;
}
.early_unregister = dpu_crtc_early_unregister,
.enable_vblank = msm_crtc_enable_vblank,
.disable_vblank = msm_crtc_disable_vblank,
+ .get_vblank_timestamp = drm_crtc_vblank_helper_get_vblank_timestamp,
+ .get_vblank_counter = dpu_crtc_get_vblank_counter,
};
static const struct drm_crtc_helper_funcs dpu_crtc_helper_funcs = {
.atomic_check = dpu_crtc_atomic_check,
.atomic_begin = dpu_crtc_atomic_begin,
.atomic_flush = dpu_crtc_atomic_flush,
+ .get_scanout_position = dpu_crtc_get_scanout_position,
};
/* initialize crtc */
return 0;
}
+int dpu_encoder_get_frame_count(struct drm_encoder *drm_enc)
+{
+ struct dpu_encoder_virt *dpu_enc;
+ struct dpu_encoder_phys *phys;
+ int framecount = 0;
+
+ dpu_enc = to_dpu_encoder_virt(drm_enc);
+ phys = dpu_enc ? dpu_enc->cur_master : NULL;
+
+ if (phys && phys->ops.get_frame_count)
+ framecount = phys->ops.get_frame_count(phys);
+
+ return framecount;
+}
+
+int dpu_encoder_get_linecount(struct drm_encoder *drm_enc)
+{
+ struct dpu_encoder_virt *dpu_enc;
+ struct dpu_encoder_phys *phys;
+ int linecount = 0;
+
+ dpu_enc = to_dpu_encoder_virt(drm_enc);
+ phys = dpu_enc ? dpu_enc->cur_master : NULL;
+
+ if (phys && phys->ops.get_line_count)
+ linecount = phys->ops.get_line_count(phys);
+
+ return linecount;
+}
+
void dpu_encoder_get_hw_resources(struct drm_encoder *drm_enc,
struct dpu_encoder_hw_resources *hw_res)
{
*/
void dpu_encoder_set_idle_timeout(struct drm_encoder *drm_enc,
u32 idle_timeout);
+/**
+ * dpu_encoder_get_linecount - get interface line count for the encoder.
+ * @drm_enc: Pointer to previously created drm encoder structure
+ */
+int dpu_encoder_get_linecount(struct drm_encoder *drm_enc);
+
+/**
+ * dpu_encoder_get_frame_count - get interface frame count for the encoder.
+ * @drm_enc: Pointer to previously created drm encoder structure
+ */
+int dpu_encoder_get_frame_count(struct drm_encoder *drm_enc);
#endif /* __DPU_ENCODER_H__ */
void (*prepare_idle_pc)(struct dpu_encoder_phys *phys_enc);
void (*restore)(struct dpu_encoder_phys *phys);
int (*get_line_count)(struct dpu_encoder_phys *phys);
+ int (*get_frame_count)(struct dpu_encoder_phys *phys);
};
/**
return phys_enc->hw_intf->ops.get_line_count(phys_enc->hw_intf);
}
+static int dpu_encoder_phys_vid_get_frame_count(
+ struct dpu_encoder_phys *phys_enc)
+{
+ struct intf_status s = {0};
+ u32 fetch_start = 0;
+ struct drm_display_mode mode = phys_enc->cached_mode;
+
+ if (!dpu_encoder_phys_vid_is_master(phys_enc))
+ return -EINVAL;
+
+ if (!phys_enc->hw_intf || !phys_enc->hw_intf->ops.get_status)
+ return -EINVAL;
+
+ phys_enc->hw_intf->ops.get_status(phys_enc->hw_intf, &s);
+
+ if (s.is_prog_fetch_en && s.is_en) {
+ fetch_start = mode.vtotal - (mode.vsync_start - mode.vdisplay);
+ if ((s.line_count > fetch_start) &&
+ (s.line_count <= mode.vtotal))
+ return s.frame_count + 1;
+ }
+
+ return s.frame_count;
+}
+
static void dpu_encoder_phys_vid_init_ops(struct dpu_encoder_phys_ops *ops)
{
ops->is_master = dpu_encoder_phys_vid_is_master;
ops->handle_post_kickoff = dpu_encoder_phys_vid_handle_post_kickoff;
ops->needs_single_flush = dpu_encoder_phys_vid_needs_single_flush;
ops->get_line_count = dpu_encoder_phys_vid_get_line_count;
+ ops->get_frame_count = dpu_encoder_phys_vid_get_frame_count;
}
struct dpu_encoder_phys *dpu_encoder_phys_vid_init(
(VIG_MASK | BIT(DPU_SSPP_QOS_8LVL) | BIT(DPU_SSPP_SCALER_QSEED4))
#define VIG_SM8250_MASK \
- (VIG_MASK | BIT(DPU_SSPP_SCALER_QSEED3LITE))
+ (VIG_MASK | BIT(DPU_SSPP_QOS_8LVL) | BIT(DPU_SSPP_SCALER_QSEED3LITE))
#define DMA_SDM845_MASK \
(BIT(DPU_SSPP_SRC) | BIT(DPU_SSPP_QOS) | BIT(DPU_SSPP_QOS_8LVL) |\
#define PINGPONG_SDM845_SPLIT_MASK \
(PINGPONG_SDM845_MASK | BIT(DPU_PINGPONG_TE2))
+#define CTL_SC7280_MASK \
+ (BIT(DPU_CTL_ACTIVE_CFG) | BIT(DPU_CTL_FETCH_ACTIVE))
+
#define MERGE_3D_SM8150_MASK (0)
#define DSPP_SC7180_MASK BIT(DPU_DSPP_PCC)
#define INTF_SC7180_MASK BIT(DPU_INTF_INPUT_CTRL) | BIT(DPU_INTF_TE)
+#define INTF_SC7280_MASK INTF_SC7180_MASK | BIT(DPU_DATA_HCTL_EN)
+
+#define INTR_SC7180_MASK \
+ (BIT(DPU_IRQ_TYPE_PING_PONG_RD_PTR) |\
+ BIT(DPU_IRQ_TYPE_PING_PONG_WR_PTR) |\
+ BIT(DPU_IRQ_TYPE_PING_PONG_AUTO_REF) |\
+ BIT(DPU_IRQ_TYPE_PING_PONG_TEAR_CHECK) |\
+ BIT(DPU_IRQ_TYPE_PING_PONG_TE_CHECK))
+
#define DEFAULT_PIXEL_RAM_SIZE (50 * 1024)
#define DEFAULT_DPU_LINE_WIDTH 2048
#define DEFAULT_DPU_OUTPUT_LINE_WIDTH 2560
.pixel_ram_size = DEFAULT_PIXEL_RAM_SIZE,
};
+static const struct dpu_caps sc7280_dpu_caps = {
+ .max_mixer_width = DEFAULT_DPU_OUTPUT_LINE_WIDTH,
+ .max_mixer_blendstages = 0x7,
+ .qseed_type = DPU_SSPP_SCALER_QSEED4,
+ .smart_dma_rev = DPU_SSPP_SMART_DMA_V2,
+ .ubwc_version = DPU_HW_UBWC_VER_30,
+ .has_dim_layer = true,
+ .has_idle_pc = true,
+ .max_linewidth = 2400,
+ .pixel_ram_size = DEFAULT_PIXEL_RAM_SIZE,
+};
+
static const struct dpu_mdp_cfg sdm845_mdp[] = {
{
.name = "top_0", .id = MDP_TOP,
},
};
+static const struct dpu_mdp_cfg sc7280_mdp[] = {
+ {
+ .name = "top_0", .id = MDP_TOP,
+ .base = 0x0, .len = 0x2014,
+ .highest_bank_bit = 0x1,
+ .clk_ctrls[DPU_CLK_CTRL_VIG0] = {
+ .reg_off = 0x2AC, .bit_off = 0},
+ .clk_ctrls[DPU_CLK_CTRL_DMA0] = {
+ .reg_off = 0x2AC, .bit_off = 8},
+ .clk_ctrls[DPU_CLK_CTRL_CURSOR0] = {
+ .reg_off = 0x2B4, .bit_off = 8},
+ .clk_ctrls[DPU_CLK_CTRL_CURSOR1] = {
+ .reg_off = 0x2C4, .bit_off = 8},
+ },
+};
+
/*************************************************************
* CTL sub blocks config
*************************************************************/
},
};
+static const struct dpu_ctl_cfg sc7280_ctl[] = {
+ {
+ .name = "ctl_0", .id = CTL_0,
+ .base = 0x15000, .len = 0x1E8,
+ .features = CTL_SC7280_MASK
+ },
+ {
+ .name = "ctl_1", .id = CTL_1,
+ .base = 0x16000, .len = 0x1E8,
+ .features = CTL_SC7280_MASK
+ },
+ {
+ .name = "ctl_2", .id = CTL_2,
+ .base = 0x17000, .len = 0x1E8,
+ .features = CTL_SC7280_MASK
+ },
+ {
+ .name = "ctl_3", .id = CTL_3,
+ .base = 0x18000, .len = 0x1E8,
+ .features = CTL_SC7280_MASK
+ },
+};
+
/*************************************************************
* SSPP sub blocks config
*************************************************************/
sdm845_dma_sblk_3, 13, SSPP_TYPE_DMA, DPU_CLK_CTRL_CURSOR1),
};
+static const struct dpu_sspp_cfg sc7280_sspp[] = {
+ SSPP_BLK("sspp_0", SSPP_VIG0, 0x4000, VIG_SC7180_MASK,
+ sc7180_vig_sblk_0, 0, SSPP_TYPE_VIG, DPU_CLK_CTRL_VIG0),
+ SSPP_BLK("sspp_8", SSPP_DMA0, 0x24000, DMA_SDM845_MASK,
+ sdm845_dma_sblk_0, 1, SSPP_TYPE_DMA, DPU_CLK_CTRL_DMA0),
+ SSPP_BLK("sspp_9", SSPP_DMA1, 0x26000, DMA_CURSOR_SDM845_MASK,
+ sdm845_dma_sblk_1, 5, SSPP_TYPE_DMA, DPU_CLK_CTRL_CURSOR0),
+ SSPP_BLK("sspp_10", SSPP_DMA2, 0x28000, DMA_CURSOR_SDM845_MASK,
+ sdm845_dma_sblk_2, 9, SSPP_TYPE_DMA, DPU_CLK_CTRL_CURSOR1),
+};
+
/*************************************************************
* MIXER sub blocks config
*************************************************************/
&sdm845_lm_sblk, PINGPONG_5, LM_4, 0),
};
+static const struct dpu_lm_cfg sc7280_lm[] = {
+ LM_BLK("lm_0", LM_0, 0x44000, MIXER_SC7180_MASK,
+ &sc7180_lm_sblk, PINGPONG_0, 0, 0),
+ LM_BLK("lm_2", LM_2, 0x46000, MIXER_SC7180_MASK,
+ &sc7180_lm_sblk, PINGPONG_2, LM_3, 0),
+ LM_BLK("lm_3", LM_3, 0x47000, MIXER_SC7180_MASK,
+ &sc7180_lm_sblk, PINGPONG_3, LM_2, 0),
+};
+
/*************************************************************
* DSPP sub blocks config
*************************************************************/
.len = 0x20, .version = 0x10000},
};
-#define PP_BLK_TE(_name, _id, _base, _merge_3d) \
+static const struct dpu_pingpong_sub_blks sc7280_pp_sblk = {
+ .dither = {.id = DPU_PINGPONG_DITHER, .base = 0xe0,
+ .len = 0x20, .version = 0x20000},
+};
+
+#define PP_BLK_TE(_name, _id, _base, _merge_3d, _sblk) \
{\
.name = _name, .id = _id, \
.base = _base, .len = 0xd4, \
.features = PINGPONG_SDM845_SPLIT_MASK, \
.merge_3d = _merge_3d, \
- .sblk = &sdm845_pp_sblk_te \
+ .sblk = &_sblk \
}
-#define PP_BLK(_name, _id, _base, _merge_3d) \
+#define PP_BLK(_name, _id, _base, _merge_3d, _sblk) \
{\
.name = _name, .id = _id, \
.base = _base, .len = 0xd4, \
.features = PINGPONG_SDM845_MASK, \
.merge_3d = _merge_3d, \
- .sblk = &sdm845_pp_sblk \
+ .sblk = &_sblk \
}
static const struct dpu_pingpong_cfg sdm845_pp[] = {
- PP_BLK_TE("pingpong_0", PINGPONG_0, 0x70000, 0),
- PP_BLK_TE("pingpong_1", PINGPONG_1, 0x70800, 0),
- PP_BLK("pingpong_2", PINGPONG_2, 0x71000, 0),
- PP_BLK("pingpong_3", PINGPONG_3, 0x71800, 0),
+ PP_BLK_TE("pingpong_0", PINGPONG_0, 0x70000, 0, sdm845_pp_sblk_te),
+ PP_BLK_TE("pingpong_1", PINGPONG_1, 0x70800, 0, sdm845_pp_sblk_te),
+ PP_BLK("pingpong_2", PINGPONG_2, 0x71000, 0, sdm845_pp_sblk),
+ PP_BLK("pingpong_3", PINGPONG_3, 0x71800, 0, sdm845_pp_sblk),
};
static struct dpu_pingpong_cfg sc7180_pp[] = {
- PP_BLK_TE("pingpong_0", PINGPONG_0, 0x70000, 0),
- PP_BLK_TE("pingpong_1", PINGPONG_1, 0x70800, 0),
+ PP_BLK_TE("pingpong_0", PINGPONG_0, 0x70000, 0, sdm845_pp_sblk_te),
+ PP_BLK_TE("pingpong_1", PINGPONG_1, 0x70800, 0, sdm845_pp_sblk_te),
};
static const struct dpu_pingpong_cfg sm8150_pp[] = {
- PP_BLK_TE("pingpong_0", PINGPONG_0, 0x70000, MERGE_3D_0),
- PP_BLK_TE("pingpong_1", PINGPONG_1, 0x70800, MERGE_3D_0),
- PP_BLK("pingpong_2", PINGPONG_2, 0x71000, MERGE_3D_1),
- PP_BLK("pingpong_3", PINGPONG_3, 0x71800, MERGE_3D_1),
- PP_BLK("pingpong_4", PINGPONG_4, 0x72000, MERGE_3D_2),
- PP_BLK("pingpong_5", PINGPONG_5, 0x72800, MERGE_3D_2),
+ PP_BLK_TE("pingpong_0", PINGPONG_0, 0x70000, MERGE_3D_0, sdm845_pp_sblk_te),
+ PP_BLK_TE("pingpong_1", PINGPONG_1, 0x70800, MERGE_3D_0, sdm845_pp_sblk_te),
+ PP_BLK("pingpong_2", PINGPONG_2, 0x71000, MERGE_3D_1, sdm845_pp_sblk),
+ PP_BLK("pingpong_3", PINGPONG_3, 0x71800, MERGE_3D_1, sdm845_pp_sblk),
+ PP_BLK("pingpong_4", PINGPONG_4, 0x72000, MERGE_3D_2, sdm845_pp_sblk),
+ PP_BLK("pingpong_5", PINGPONG_5, 0x72800, MERGE_3D_2, sdm845_pp_sblk),
};
/*************************************************************
MERGE_3D_BLK("merge_3d_2", MERGE_3D_2, 0x83200),
};
+static const struct dpu_pingpong_cfg sc7280_pp[] = {
+ PP_BLK("pingpong_0", PINGPONG_0, 0x59000, 0, sc7280_pp_sblk),
+ PP_BLK("pingpong_1", PINGPONG_1, 0x6a000, 0, sc7280_pp_sblk),
+ PP_BLK("pingpong_2", PINGPONG_2, 0x6b000, 0, sc7280_pp_sblk),
+ PP_BLK("pingpong_3", PINGPONG_3, 0x6c000, 0, sc7280_pp_sblk),
+};
/*************************************************************
* INTF sub blocks config
*************************************************************/
INTF_BLK("intf_3", INTF_3, 0x6B800, INTF_DP, 1, 24, INTF_SC7180_MASK),
};
+static const struct dpu_intf_cfg sc7280_intf[] = {
+ INTF_BLK("intf_0", INTF_0, 0x34000, INTF_DP, 0, 24, INTF_SC7280_MASK),
+ INTF_BLK("intf_1", INTF_1, 0x35000, INTF_DSI, 0, 24, INTF_SC7280_MASK),
+ INTF_BLK("intf_5", INTF_5, 0x39000, INTF_EDP, 0, 24, INTF_SC7280_MASK),
+};
+
/*************************************************************
* VBIF sub blocks config
*************************************************************/
{.rd_enable = 1, .wr_enable = 1},
{.rd_enable = 1, .wr_enable = 0}
},
+ .clk_inefficiency_factor = 105,
+ .bw_inefficiency_factor = 120,
};
static const struct dpu_perf_cfg sc7180_perf_data = {
.min_core_ib = 2400000,
.min_llcc_ib = 800000,
.min_dram_ib = 800000,
+ .min_prefill_lines = 24,
.danger_lut_tbl = {0xf, 0xffff, 0x0},
.qos_lut_tbl = {
{.nentry = ARRAY_SIZE(sm8150_qos_linear),
{.rd_enable = 1, .wr_enable = 1},
{.rd_enable = 1, .wr_enable = 0}
},
+ .clk_inefficiency_factor = 105,
+ .bw_inefficiency_factor = 120,
};
static const struct dpu_perf_cfg sm8250_perf_data = {
.min_core_ib = 4800000,
.min_llcc_ib = 0,
.min_dram_ib = 800000,
+ .min_prefill_lines = 35,
.danger_lut_tbl = {0xf, 0xffff, 0x0},
.qos_lut_tbl = {
{.nentry = ARRAY_SIZE(sc7180_qos_linear),
{.rd_enable = 1, .wr_enable = 1},
{.rd_enable = 1, .wr_enable = 0}
},
+ .clk_inefficiency_factor = 105,
+ .bw_inefficiency_factor = 120,
+};
+
+static const struct dpu_perf_cfg sc7280_perf_data = {
+ .max_bw_low = 4700000,
+ .max_bw_high = 8800000,
+ .min_core_ib = 2500000,
+ .min_llcc_ib = 0,
+ .min_dram_ib = 1600000,
+ .min_prefill_lines = 24,
+ .danger_lut_tbl = {0xffff, 0xffff, 0x0},
+ .qos_lut_tbl = {
+ {.nentry = ARRAY_SIZE(sc7180_qos_macrotile),
+ .entries = sc7180_qos_macrotile
+ },
+ {.nentry = ARRAY_SIZE(sc7180_qos_macrotile),
+ .entries = sc7180_qos_macrotile
+ },
+ {.nentry = ARRAY_SIZE(sc7180_qos_nrt),
+ .entries = sc7180_qos_nrt
+ },
+ },
+ .cdp_cfg = {
+ {.rd_enable = 1, .wr_enable = 1},
+ {.rd_enable = 1, .wr_enable = 0}
+ },
+ .clk_inefficiency_factor = 105,
+ .bw_inefficiency_factor = 120,
};
/*************************************************************
.dma_cfg = sdm845_regdma,
.perf = sc7180_perf_data,
.mdss_irqs = 0x3f,
+ .obsolete_irq = INTR_SC7180_MASK,
};
}
};
}
+static void sc7280_cfg_init(struct dpu_mdss_cfg *dpu_cfg)
+{
+ *dpu_cfg = (struct dpu_mdss_cfg){
+ .caps = &sc7280_dpu_caps,
+ .mdp_count = ARRAY_SIZE(sc7280_mdp),
+ .mdp = sc7280_mdp,
+ .ctl_count = ARRAY_SIZE(sc7280_ctl),
+ .ctl = sc7280_ctl,
+ .sspp_count = ARRAY_SIZE(sc7280_sspp),
+ .sspp = sc7280_sspp,
+ .mixer_count = ARRAY_SIZE(sc7280_lm),
+ .mixer = sc7280_lm,
+ .pingpong_count = ARRAY_SIZE(sc7280_pp),
+ .pingpong = sc7280_pp,
+ .intf_count = ARRAY_SIZE(sc7280_intf),
+ .intf = sc7280_intf,
+ .vbif_count = ARRAY_SIZE(sdm845_vbif),
+ .vbif = sdm845_vbif,
+ .perf = sc7280_perf_data,
+ .mdss_irqs = 0x1c07,
+ .obsolete_irq = INTR_SC7180_MASK,
+ };
+}
+
static const struct dpu_mdss_hw_cfg_handler cfg_handler[] = {
{ .hw_rev = DPU_HW_VER_400, .cfg_init = sdm845_cfg_init},
{ .hw_rev = DPU_HW_VER_401, .cfg_init = sdm845_cfg_init},
{ .hw_rev = DPU_HW_VER_501, .cfg_init = sm8150_cfg_init},
{ .hw_rev = DPU_HW_VER_600, .cfg_init = sm8250_cfg_init},
{ .hw_rev = DPU_HW_VER_620, .cfg_init = sc7180_cfg_init},
+ { .hw_rev = DPU_HW_VER_720, .cfg_init = sc7280_cfg_init},
};
void dpu_hw_catalog_deinit(struct dpu_mdss_cfg *dpu_cfg)
#define DPU_HW_VER_501 DPU_HW_VER(5, 0, 1) /* sm8150 v2.0 */
#define DPU_HW_VER_600 DPU_HW_VER(6, 0, 0) /* sm8250 */
#define DPU_HW_VER_620 DPU_HW_VER(6, 2, 0) /* sc7180 v1.0 */
-
+#define DPU_HW_VER_720 DPU_HW_VER(7, 2, 0) /* sc7280 */
#define IS_MSM8996_TARGET(rev) IS_DPU_MAJOR_MINOR_SAME((rev), DPU_HW_VER_170)
#define IS_MSM8998_TARGET(rev) IS_DPU_MAJOR_MINOR_SAME((rev), DPU_HW_VER_300)
#define IS_SDM670_TARGET(rev) IS_DPU_MAJOR_MINOR_SAME((rev), DPU_HW_VER_410)
#define IS_SDM855_TARGET(rev) IS_DPU_MAJOR_MINOR_SAME((rev), DPU_HW_VER_500)
#define IS_SC7180_TARGET(rev) IS_DPU_MAJOR_MINOR_SAME((rev), DPU_HW_VER_620)
-
+#define IS_SC7280_TARGET(rev) IS_DPU_MAJOR_MINOR_SAME((rev), DPU_HW_VER_720)
#define DPU_HW_BLK_NAME_LEN 16
enum {
DPU_CTL_SPLIT_DISPLAY = 0x1,
DPU_CTL_ACTIVE_CFG,
+ DPU_CTL_FETCH_ACTIVE,
DPU_CTL_MAX
};
* @DPU_INTF_INPUT_CTRL Supports the setting of pp block from which
* pixel data arrives to this INTF
* @DPU_INTF_TE INTF block has TE configuration support
+ * @DPU_DATA_HCTL_EN Allows data to be transferred at different rate
+ than video timing
* @DPU_INTF_MAX
*/
enum {
DPU_INTF_INPUT_CTRL = 0x1,
DPU_INTF_TE,
+ DPU_DATA_HCTL_EN,
DPU_INTF_MAX
};
* @cursor_formats Supported formats for cursor pipe
* @vig_formats Supported formats for vig pipe
* @mdss_irqs: Bitmap with the irqs supported by the target
+ * @obsolete_irq: Irq types that are obsolete for a particular target
*/
struct dpu_mdss_cfg {
u32 hwversion;
const struct dpu_format_extended *vig_formats;
unsigned long mdss_irqs;
+ unsigned long obsolete_irq;
};
struct dpu_mdss_hw_cfg_handler {
#define CTL_MERGE_3D_FLUSH 0x100
#define CTL_INTF_FLUSH 0x110
#define CTL_INTF_MASTER 0x134
+#define CTL_FETCH_PIPE_ACTIVE 0x0FC
#define CTL_MIXER_BORDER_OUT BIT(24)
#define CTL_FLUSH_MASK_CTL BIT(17)
#define DPU_REG_RESET_TIMEOUT_US 2000
#define MERGE_3D_IDX 23
#define INTF_IDX 31
+#define CTL_INVALID_BIT 0xffff
+
+static const u32 fetch_tbl[SSPP_MAX] = {CTL_INVALID_BIT, 16, 17, 18, 19,
+ CTL_INVALID_BIT, CTL_INVALID_BIT, CTL_INVALID_BIT, CTL_INVALID_BIT, 0,
+ 1, 2, 3, CTL_INVALID_BIT, CTL_INVALID_BIT};
static const struct dpu_ctl_cfg *_ctl_offset(enum dpu_ctl ctl,
const struct dpu_mdss_cfg *m,
DPU_REG_WRITE(c, CTL_LAYER_EXT2(LM_0 + i), 0);
DPU_REG_WRITE(c, CTL_LAYER_EXT3(LM_0 + i), 0);
}
+
+ DPU_REG_WRITE(c, CTL_FETCH_PIPE_ACTIVE, 0);
}
static void dpu_hw_ctl_setup_blendstage(struct dpu_hw_ctl *ctx,
DPU_REG_WRITE(c, CTL_TOP, mode_sel);
DPU_REG_WRITE(c, CTL_INTF_ACTIVE, intf_active);
- DPU_REG_WRITE(c, CTL_MERGE_3D_ACTIVE, BIT(cfg->merge_3d - MERGE_3D_0));
+ if (cfg->merge_3d)
+ DPU_REG_WRITE(c, CTL_MERGE_3D_ACTIVE,
+ BIT(cfg->merge_3d - MERGE_3D_0));
}
static void dpu_hw_ctl_intf_cfg(struct dpu_hw_ctl *ctx,
DPU_REG_WRITE(c, CTL_TOP, intf_cfg);
}
+static void dpu_hw_ctl_set_fetch_pipe_active(struct dpu_hw_ctl *ctx,
+ unsigned long *fetch_active)
+{
+ int i;
+ u32 val = 0;
+
+ if (fetch_active) {
+ for (i = 0; i < SSPP_MAX; i++) {
+ if (test_bit(i, fetch_active) &&
+ fetch_tbl[i] != CTL_INVALID_BIT)
+ val |= BIT(fetch_tbl[i]);
+ }
+ }
+
+ DPU_REG_WRITE(&ctx->hw, CTL_FETCH_PIPE_ACTIVE, val);
+}
+
static void _setup_ctl_ops(struct dpu_hw_ctl_ops *ops,
unsigned long cap)
{
ops->get_bitmask_sspp = dpu_hw_ctl_get_bitmask_sspp;
ops->get_bitmask_mixer = dpu_hw_ctl_get_bitmask_mixer;
ops->get_bitmask_dspp = dpu_hw_ctl_get_bitmask_dspp;
+ if (cap & BIT(DPU_CTL_FETCH_ACTIVE))
+ ops->set_active_pipes = dpu_hw_ctl_set_fetch_pipe_active;
};
static struct dpu_hw_blk_ops dpu_hw_ops;
*/
void (*setup_blendstage)(struct dpu_hw_ctl *ctx,
enum dpu_lm lm, struct dpu_hw_stage_cfg *cfg);
+
+ void (*set_active_pipes)(struct dpu_hw_ctl *ctx,
+ unsigned long *fetch_active);
};
/**
#define MDP_AD4_INTR_EN_OFF 0x41c
#define MDP_AD4_INTR_CLEAR_OFF 0x424
#define MDP_AD4_INTR_STATUS_OFF 0x420
+#define MDP_INTF_0_OFF_REV_7xxx 0x34000
+#define MDP_INTF_1_OFF_REV_7xxx 0x35000
+#define MDP_INTF_5_OFF_REV_7xxx 0x39000
/**
* WB interrupt status bit definitions
#define DPU_INTR_INTF_1_UNDERRUN BIT(26)
#define DPU_INTR_INTF_2_UNDERRUN BIT(28)
#define DPU_INTR_INTF_3_UNDERRUN BIT(30)
+#define DPU_INTR_INTF_5_UNDERRUN BIT(22)
#define DPU_INTR_INTF_0_VSYNC BIT(25)
#define DPU_INTR_INTF_1_VSYNC BIT(27)
#define DPU_INTR_INTF_2_VSYNC BIT(29)
#define DPU_INTR_INTF_3_VSYNC BIT(31)
+#define DPU_INTR_INTF_5_VSYNC BIT(23)
/**
* Pingpong Secondary interrupt status bit definitions
MDP_AD4_1_OFF + MDP_AD4_INTR_CLEAR_OFF,
MDP_AD4_1_OFF + MDP_AD4_INTR_EN_OFF,
MDP_AD4_1_OFF + MDP_AD4_INTR_STATUS_OFF,
- }
+ },
+ {
+ MDP_INTF_0_OFF_REV_7xxx+INTF_INTR_CLEAR,
+ MDP_INTF_0_OFF_REV_7xxx+INTF_INTR_EN,
+ MDP_INTF_0_OFF_REV_7xxx+INTF_INTR_STATUS
+ },
+ {
+ MDP_INTF_1_OFF_REV_7xxx+INTF_INTR_CLEAR,
+ MDP_INTF_1_OFF_REV_7xxx+INTF_INTR_EN,
+ MDP_INTF_1_OFF_REV_7xxx+INTF_INTR_STATUS
+ },
+ {
+ MDP_INTF_5_OFF_REV_7xxx+INTF_INTR_CLEAR,
+ MDP_INTF_5_OFF_REV_7xxx+INTF_INTR_EN,
+ MDP_INTF_5_OFF_REV_7xxx+INTF_INTR_STATUS
+ },
};
/*
{ DPU_IRQ_TYPE_INTF_VSYNC, INTF_2, DPU_INTR_INTF_2_VSYNC, 0},
{ DPU_IRQ_TYPE_INTF_UNDER_RUN, INTF_3, DPU_INTR_INTF_3_UNDERRUN, 0},
{ DPU_IRQ_TYPE_INTF_VSYNC, INTF_3, DPU_INTR_INTF_3_VSYNC, 0},
-
- /* BEGIN MAP_RANGE: 32-64, INTR2 */
- /* irq_idx: 32-35 */
+ /* irq_idx:32-33 */
+ { DPU_IRQ_TYPE_INTF_UNDER_RUN, INTF_5, DPU_INTR_INTF_5_UNDERRUN, 0},
+ { DPU_IRQ_TYPE_INTF_VSYNC, INTF_5, DPU_INTR_INTF_5_VSYNC, 0},
+ /* irq_idx:34-63 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 0},
+ /* BEGIN MAP_RANGE: 64-95, INTR2 */
+ /* irq_idx: 64-67 */
{ DPU_IRQ_TYPE_PING_PONG_AUTO_REF, PINGPONG_S0,
DPU_INTR_PING_PONG_S0_AUTOREFRESH_DONE, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
- /* irq_idx: 36-39 */
+ /* irq_idx: 68-71 */
{ DPU_IRQ_TYPE_PING_PONG_WR_PTR, PINGPONG_S0,
DPU_INTR_PING_PONG_S0_WR_PTR, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
- /* irq_idx: 40 */
+ /* irq_idx: 72 */
{ DPU_IRQ_TYPE_PING_PONG_RD_PTR, PINGPONG_S0,
DPU_INTR_PING_PONG_S0_RD_PTR, 1},
- /* irq_idx: 41-45 */
+ /* irq_idx: 73-77 */
{ DPU_IRQ_TYPE_CTL_START, CTL_0,
DPU_INTR_CTL_0_START, 1},
{ DPU_IRQ_TYPE_CTL_START, CTL_1,
DPU_INTR_CTL_3_START, 1},
{ DPU_IRQ_TYPE_CTL_START, CTL_4,
DPU_INTR_CTL_4_START, 1},
- /* irq_idx: 46-47 */
+ /* irq_idx: 78-79 */
{ DPU_IRQ_TYPE_CWB_OVERFLOW, CWB_2, DPU_INTR_CWB_2_OVERFLOW, 1},
{ DPU_IRQ_TYPE_CWB_OVERFLOW, CWB_3, DPU_INTR_CWB_3_OVERFLOW, 1},
- /* irq_idx: 48-51 */
+ /* irq_idx: 80-83 */
{ DPU_IRQ_TYPE_PING_PONG_TEAR_CHECK, PINGPONG_0,
DPU_INTR_PING_PONG_0_TEAR_DETECTED, 1},
{ DPU_IRQ_TYPE_PING_PONG_TEAR_CHECK, PINGPONG_1,
DPU_INTR_PING_PONG_2_TEAR_DETECTED, 1},
{ DPU_IRQ_TYPE_PING_PONG_TEAR_CHECK, PINGPONG_3,
DPU_INTR_PING_PONG_3_TEAR_DETECTED, 1},
- /* irq_idx: 52-55 */
+ /* irq_idx: 84-87 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_PING_PONG_TEAR_CHECK, PINGPONG_S0,
DPU_INTR_PING_PONG_S0_TEAR_DETECTED, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
- /* irq_idx: 56-59 */
+ /* irq_idx: 88-91 */
{ DPU_IRQ_TYPE_PING_PONG_TE_CHECK, PINGPONG_0,
DPU_INTR_PING_PONG_0_TE_DETECTED, 1},
{ DPU_IRQ_TYPE_PING_PONG_TE_CHECK, PINGPONG_1,
DPU_INTR_PING_PONG_2_TE_DETECTED, 1},
{ DPU_IRQ_TYPE_PING_PONG_TE_CHECK, PINGPONG_3,
DPU_INTR_PING_PONG_3_TE_DETECTED, 1},
- /* irq_idx: 60-63 */
+ /* irq_idx: 92-95 */
{ DPU_IRQ_TYPE_PING_PONG_TE_CHECK, PINGPONG_S0,
DPU_INTR_PING_PONG_S0_TE_DETECTED, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
-
- /* BEGIN MAP_RANGE: 64-95 HIST */
- /* irq_idx: 64-67 */
+ /* irq_idx: 96-127 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 1},
+ /* BEGIN MAP_RANGE: 128-159 HIST */
+ /* irq_idx: 128-131 */
{ DPU_IRQ_TYPE_HIST_VIG_DONE, SSPP_VIG0, DPU_INTR_HIST_VIG_0_DONE, 2},
{ DPU_IRQ_TYPE_HIST_VIG_RSTSEQ, SSPP_VIG0,
DPU_INTR_HIST_VIG_0_RSTSEQ_DONE, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
- /* irq_idx: 68-71 */
+ /* irq_idx: 132-135 */
{ DPU_IRQ_TYPE_HIST_VIG_DONE, SSPP_VIG1, DPU_INTR_HIST_VIG_1_DONE, 2},
{ DPU_IRQ_TYPE_HIST_VIG_RSTSEQ, SSPP_VIG1,
DPU_INTR_HIST_VIG_1_RSTSEQ_DONE, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
- /* irq_idx: 72-75 */
+ /* irq_idx: 136-139 */
{ DPU_IRQ_TYPE_HIST_VIG_DONE, SSPP_VIG2, DPU_INTR_HIST_VIG_2_DONE, 2},
{ DPU_IRQ_TYPE_HIST_VIG_RSTSEQ, SSPP_VIG2,
DPU_INTR_HIST_VIG_2_RSTSEQ_DONE, 2},
{ DPU_IRQ_TYPE_HIST_VIG_DONE, SSPP_VIG3, DPU_INTR_HIST_VIG_3_DONE, 2},
{ DPU_IRQ_TYPE_HIST_VIG_RSTSEQ, SSPP_VIG3,
DPU_INTR_HIST_VIG_3_RSTSEQ_DONE, 2},
- /* irq_idx: 76-79 */
+ /* irq_idx: 140-143 */
{ DPU_IRQ_TYPE_HIST_DSPP_DONE, DSPP_0, DPU_INTR_HIST_DSPP_0_DONE, 2},
{ DPU_IRQ_TYPE_HIST_DSPP_RSTSEQ, DSPP_0,
DPU_INTR_HIST_DSPP_0_RSTSEQ_DONE, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
- /* irq_idx: 80-83 */
+ /* irq_idx: 144-147 */
{ DPU_IRQ_TYPE_HIST_DSPP_DONE, DSPP_1, DPU_INTR_HIST_DSPP_1_DONE, 2},
{ DPU_IRQ_TYPE_HIST_DSPP_RSTSEQ, DSPP_1,
DPU_INTR_HIST_DSPP_1_RSTSEQ_DONE, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
- /* irq_idx: 84-87 */
+ /* irq_idx: 148-151 */
{ DPU_IRQ_TYPE_HIST_DSPP_DONE, DSPP_2, DPU_INTR_HIST_DSPP_2_DONE, 2},
{ DPU_IRQ_TYPE_HIST_DSPP_RSTSEQ, DSPP_2,
DPU_INTR_HIST_DSPP_2_RSTSEQ_DONE, 2},
{ DPU_IRQ_TYPE_HIST_DSPP_DONE, DSPP_3, DPU_INTR_HIST_DSPP_3_DONE, 2},
{ DPU_IRQ_TYPE_HIST_DSPP_RSTSEQ, DSPP_3,
DPU_INTR_HIST_DSPP_3_RSTSEQ_DONE, 2},
- /* irq_idx: 88-91 */
+ /* irq_idx: 152-155 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
- /* irq_idx: 92-95 */
+ /* irq_idx: 156-159 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
-
- /* BEGIN MAP_RANGE: 96-127 INTF_0_INTR */
- /* irq_idx: 96-99 */
+ /* irq_idx: 160-191 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 2},
+ /* BEGIN MAP_RANGE: 192-255 INTF_0_INTR */
+ /* irq_idx: 192-195 */
{ DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_0,
DPU_INTR_VIDEO_INTO_STATIC, 3},
{ DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_0,
DPU_INTR_DSICMD_0_INTO_STATIC, 3},
{ DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_0,
DPU_INTR_DSICMD_0_OUTOF_STATIC, 3},
- /* irq_idx: 100-103 */
+ /* irq_idx: 196-199 */
{ DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_0,
DPU_INTR_DSICMD_1_INTO_STATIC, 3},
{ DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_0,
DPU_INTR_DSICMD_2_INTO_STATIC, 3},
{ DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_0,
DPU_INTR_DSICMD_2_OUTOF_STATIC, 3},
- /* irq_idx: 104-107 */
+ /* irq_idx: 200-203 */
{ DPU_IRQ_TYPE_PROG_LINE, INTF_0, DPU_INTR_PROG_LINE, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
- /* irq_idx: 108-111 */
+ /* irq_idx: 204-207 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ /* irq_idx: 208-211 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
- /* irq_idx: 112-115 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ /* irq_idx: 212-215 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
- /* irq_idx: 116-119 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ /* irq_idx: 216-219 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
- /* irq_idx: 120-123 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ /* irq_idx: 220-223 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
- /* irq_idx: 124-127 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ /* irq_idx: 224-255 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
-
- /* BEGIN MAP_RANGE: 128-159 INTF_1_INTR */
- /* irq_idx: 128-131 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 3},
+ /* BEGIN MAP_RANGE: 256-319 INTF_1_INTR */
+ /* irq_idx: 256-259 */
{ DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_1,
DPU_INTR_VIDEO_INTO_STATIC, 4},
{ DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_1,
DPU_INTR_DSICMD_0_INTO_STATIC, 4},
{ DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_1,
DPU_INTR_DSICMD_0_OUTOF_STATIC, 4},
- /* irq_idx: 132-135 */
+ /* irq_idx: 260-263 */
{ DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_1,
DPU_INTR_DSICMD_1_INTO_STATIC, 4},
{ DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_1,
DPU_INTR_DSICMD_2_INTO_STATIC, 4},
{ DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_1,
DPU_INTR_DSICMD_2_OUTOF_STATIC, 4},
- /* irq_idx: 136-139 */
+ /* irq_idx: 264-267 */
{ DPU_IRQ_TYPE_PROG_LINE, INTF_1, DPU_INTR_PROG_LINE, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
- /* irq_idx: 140-143 */
+ /* irq_idx: 268-271 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
- /* irq_idx: 144-147 */
+ /* irq_idx: 272-275 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
- /* irq_idx: 148-151 */
+ /* irq_idx: 276-279 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
- /* irq_idx: 152-155 */
+ /* irq_idx: 280-283 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
- /* irq_idx: 156-159 */
+ /* irq_idx: 284-287 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
-
- /* BEGIN MAP_RANGE: 160-191 INTF_2_INTR */
- /* irq_idx: 160-163 */
+ /* irq_idx: 288-319 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 4},
+ /* BEGIN MAP_RANGE: 320-383 INTF_2_INTR */
+ /* irq_idx: 320-323 */
{ DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_2,
DPU_INTR_VIDEO_INTO_STATIC, 5},
{ DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_2,
DPU_INTR_DSICMD_0_INTO_STATIC, 5},
{ DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_2,
DPU_INTR_DSICMD_0_OUTOF_STATIC, 5},
- /* irq_idx: 164-167 */
+ /* irq_idx: 324-327 */
{ DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_2,
DPU_INTR_DSICMD_1_INTO_STATIC, 5},
{ DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_2,
DPU_INTR_DSICMD_2_INTO_STATIC, 5},
{ DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_2,
DPU_INTR_DSICMD_2_OUTOF_STATIC, 5},
- /* irq_idx: 168-171 */
+ /* irq_idx: 328-331 */
{ DPU_IRQ_TYPE_PROG_LINE, INTF_2, DPU_INTR_PROG_LINE, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
- /* irq_idx: 172-175 */
+ /* irq_idx: 332-335 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
- /* irq_idx: 176-179 */
+ /* irq_idx: 336-339 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
- /* irq_idx: 180-183 */
+ /* irq_idx: 340-343 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
- /* irq_idx: 184-187 */
+ /* irq_idx: 344-347 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
- /* irq_idx: 188-191 */
+ /* irq_idx: 348-351 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
-
- /* BEGIN MAP_RANGE: 192-223 INTF_3_INTR */
- /* irq_idx: 192-195 */
+ /* irq_idx: 352-383 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 5},
+ /* BEGIN MAP_RANGE: 384-447 INTF_3_INTR */
+ /* irq_idx: 384-387 */
{ DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_3,
DPU_INTR_VIDEO_INTO_STATIC, 6},
{ DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_3,
DPU_INTR_DSICMD_0_INTO_STATIC, 6},
{ DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_3,
DPU_INTR_DSICMD_0_OUTOF_STATIC, 6},
- /* irq_idx: 196-199 */
+ /* irq_idx: 388-391 */
{ DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_3,
DPU_INTR_DSICMD_1_INTO_STATIC, 6},
{ DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_3,
DPU_INTR_DSICMD_2_INTO_STATIC, 6},
{ DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_3,
DPU_INTR_DSICMD_2_OUTOF_STATIC, 6},
- /* irq_idx: 200-203 */
+ /* irq_idx: 392-395 */
{ DPU_IRQ_TYPE_PROG_LINE, INTF_3, DPU_INTR_PROG_LINE, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
- /* irq_idx: 204-207 */
+ /* irq_idx: 396-399 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
- /* irq_idx: 208-211 */
+ /* irq_idx: 400-403 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
- /* irq_idx: 212-215 */
+ /* irq_idx: 404-407 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
- /* irq_idx: 216-219 */
+ /* irq_idx: 408-411 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
- /* irq_idx: 220-223 */
+ /* irq_idx: 412-415 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
-
- /* BEGIN MAP_RANGE: 224-255 INTF_4_INTR */
- /* irq_idx: 224-227 */
+ /* irq_idx: 416-447*/
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 6},
+ /* BEGIN MAP_RANGE: 448-511 INTF_4_INTR */
+ /* irq_idx: 448-451 */
{ DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_4,
DPU_INTR_VIDEO_INTO_STATIC, 7},
{ DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_4,
DPU_INTR_DSICMD_0_INTO_STATIC, 7},
{ DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_4,
DPU_INTR_DSICMD_0_OUTOF_STATIC, 7},
- /* irq_idx: 228-231 */
+ /* irq_idx: 452-455 */
{ DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_4,
DPU_INTR_DSICMD_1_INTO_STATIC, 7},
{ DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_4,
DPU_INTR_DSICMD_2_INTO_STATIC, 7},
{ DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_4,
DPU_INTR_DSICMD_2_OUTOF_STATIC, 7},
- /* irq_idx: 232-235 */
+ /* irq_idx: 456-459 */
{ DPU_IRQ_TYPE_PROG_LINE, INTF_4, DPU_INTR_PROG_LINE, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
- /* irq_idx: 236-239 */
+ /* irq_idx: 460-463 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
- /* irq_idx: 240-243 */
+ /* irq_idx: 464-467 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
- /* irq_idx: 244-247 */
+ /* irq_idx: 468-471 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
- /* irq_idx: 248-251 */
+ /* irq_idx: 472-475 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
- /* irq_idx: 252-255 */
+ /* irq_idx: 476-479 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
-
- /* BEGIN MAP_RANGE: 256-287 AD4_0_INTR */
- /* irq_idx: 256-259 */
+ /* irq_idx: 480-511 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 7},
+ /* BEGIN MAP_RANGE: 512-575 AD4_0_INTR */
+ /* irq_idx: 512-515 */
{ DPU_IRQ_TYPE_AD4_BL_DONE, DSPP_0, DPU_INTR_BACKLIGHT_UPDATED, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
- /* irq_idx: 260-263 */
+ /* irq_idx: 516-519 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
- /* irq_idx: 264-267 */
+ /* irq_idx: 520-523 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
- /* irq_idx: 268-271 */
+ /* irq_idx: 524-527 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
- /* irq_idx: 272-275 */
+ /* irq_idx: 528-531 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
- /* irq_idx: 276-279 */
+ /* irq_idx: 532-535 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
- /* irq_idx: 280-283 */
+ /* irq_idx: 536-539 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
- /* irq_idx: 284-287 */
+ /* irq_idx: 540-543 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
-
- /* BEGIN MAP_RANGE: 288-319 AD4_1_INTR */
- /* irq_idx: 288-291 */
+ /* irq_idx: 544-575*/
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 8},
+ /* BEGIN MAP_RANGE: 576-639 AD4_1_INTR */
+ /* irq_idx: 576-579 */
{ DPU_IRQ_TYPE_AD4_BL_DONE, DSPP_1, DPU_INTR_BACKLIGHT_UPDATED, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
- /* irq_idx: 292-295 */
+ /* irq_idx: 580-583 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* irq_idx: 584-587 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
- /* irq_idx: 296-299 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* irq_idx: 588-591 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
- /* irq_idx: 300-303 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* irq_idx: 592-595 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
- /* irq_idx: 304-307 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* irq_idx: 596-599 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
- /* irq_idx: 308-311 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* irq_idx: 600-603 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
- /* irq_idx: 312-315 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* irq_idx: 604-607 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
- /* irq_idx: 315-319 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* irq_idx: 608-639 */
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
{ DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 9},
+ /* BEGIN MAP_RANGE: 640-703 INTF_0_SC7280_INTR */
+ /* irq_idx: 640-643 */
+ { DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_0,
+ DPU_INTR_VIDEO_INTO_STATIC, 10},
+ { DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_0,
+ DPU_INTR_VIDEO_OUTOF_STATIC, 10},
+ { DPU_IRQ_TYPE_SFI_CMD_0_IN, INTF_0,
+ DPU_INTR_DSICMD_0_INTO_STATIC, 10},
+ { DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_0,
+ DPU_INTR_DSICMD_0_OUTOF_STATIC, 10},
+ /* irq_idx: 644-647 */
+ { DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_0,
+ DPU_INTR_DSICMD_1_INTO_STATIC, 10},
+ { DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_0,
+ DPU_INTR_DSICMD_1_OUTOF_STATIC, 10},
+ { DPU_IRQ_TYPE_SFI_CMD_2_IN, INTF_0,
+ DPU_INTR_DSICMD_2_INTO_STATIC, 10},
+ { DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_0,
+ DPU_INTR_DSICMD_2_OUTOF_STATIC, 10},
+ /* irq_idx: 648-651 */
+ { DPU_IRQ_TYPE_PROG_LINE, INTF_0, DPU_INTR_PROG_LINE, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ /* irq_idx: 652-655 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ /* irq_idx: 656-659 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ /* irq_idx: 660-663 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ /* irq_idx: 664-667 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ /* irq_idx: 668-671 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ /* irq_idx: 672-703 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 10},
+ /* BEGIN MAP_RANGE: 704-767 INTF_1_SC7280_INTR */
+ /* irq_idx: 704-707 */
+ { DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_1,
+ DPU_INTR_VIDEO_INTO_STATIC, 11},
+ { DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_1,
+ DPU_INTR_VIDEO_OUTOF_STATIC, 11},
+ { DPU_IRQ_TYPE_SFI_CMD_0_IN, INTF_1,
+ DPU_INTR_DSICMD_0_INTO_STATIC, 11},
+ { DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_1,
+ DPU_INTR_DSICMD_0_OUTOF_STATIC, 11},
+ /* irq_idx: 708-711 */
+ { DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_1,
+ DPU_INTR_DSICMD_1_INTO_STATIC, 11},
+ { DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_1,
+ DPU_INTR_DSICMD_1_OUTOF_STATIC, 11},
+ { DPU_IRQ_TYPE_SFI_CMD_2_IN, INTF_1,
+ DPU_INTR_DSICMD_2_INTO_STATIC, 11},
+ { DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_1,
+ DPU_INTR_DSICMD_2_OUTOF_STATIC, 11},
+ /* irq_idx: 712-715 */
+ { DPU_IRQ_TYPE_PROG_LINE, INTF_1, DPU_INTR_PROG_LINE, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ /* irq_idx: 716-719 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ /* irq_idx: 720-723 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ /* irq_idx: 724-727 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ /* irq_idx: 728-731 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ /* irq_idx: 732-735 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ /* irq_idx: 736-767 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 11},
+ /* BEGIN MAP_RANGE: 768-831 INTF_5_SC7280_INTR */
+ /* irq_idx: 768-771 */
+ { DPU_IRQ_TYPE_SFI_VIDEO_IN, INTF_5,
+ DPU_INTR_VIDEO_INTO_STATIC, 12},
+ { DPU_IRQ_TYPE_SFI_VIDEO_OUT, INTF_5,
+ DPU_INTR_VIDEO_OUTOF_STATIC, 12},
+ { DPU_IRQ_TYPE_SFI_CMD_0_IN, INTF_5,
+ DPU_INTR_DSICMD_0_INTO_STATIC, 12},
+ { DPU_IRQ_TYPE_SFI_CMD_0_OUT, INTF_5,
+ DPU_INTR_DSICMD_0_OUTOF_STATIC, 12},
+ /* irq_idx: 772-775 */
+ { DPU_IRQ_TYPE_SFI_CMD_1_IN, INTF_5,
+ DPU_INTR_DSICMD_1_INTO_STATIC, 12},
+ { DPU_IRQ_TYPE_SFI_CMD_1_OUT, INTF_5,
+ DPU_INTR_DSICMD_1_OUTOF_STATIC, 12},
+ { DPU_IRQ_TYPE_SFI_CMD_2_IN, INTF_5,
+ DPU_INTR_DSICMD_2_INTO_STATIC, 12},
+ { DPU_IRQ_TYPE_SFI_CMD_2_OUT, INTF_5,
+ DPU_INTR_DSICMD_2_OUTOF_STATIC, 12},
+ /* irq_idx: 776-779 */
+ { DPU_IRQ_TYPE_PROG_LINE, INTF_5, DPU_INTR_PROG_LINE, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ /* irq_idx: 780-783 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ /* irq_idx: 784-787 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ /* irq_idx: 788-791 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ /* irq_idx: 792-795 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ /* irq_idx: 796-799 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ /* irq_idx: 800-831 */
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
+ { DPU_IRQ_TYPE_RESERVED, 0, 0, 12},
};
-static int dpu_hw_intr_irqidx_lookup(enum dpu_intr_type intr_type,
- u32 instance_idx)
+static int dpu_hw_intr_irqidx_lookup(struct dpu_hw_intr *intr,
+ enum dpu_intr_type intr_type, u32 instance_idx)
{
int i;
for (i = 0; i < ARRAY_SIZE(dpu_irq_map); i++) {
if (intr_type == dpu_irq_map[i].intr_type &&
- instance_idx == dpu_irq_map[i].instance_idx)
+ instance_idx == dpu_irq_map[i].instance_idx &&
+ !(intr->obsolete_irq & BIT(dpu_irq_map[i].intr_type)))
return i;
}
irq_status = intr->save_irq_status[reg_idx];
/*
- * Each Interrupt register has a range of 32 indexes, and
+ * Each Interrupt register has a range of 64 indexes, and
* that is static for dpu_irq_map.
*/
- start_idx = reg_idx * 32;
- end_idx = start_idx + 32;
+ start_idx = reg_idx * 64;
+ end_idx = start_idx + 64;
if (!test_bit(reg_idx, &intr->irq_mask) ||
start_idx >= ARRAY_SIZE(dpu_irq_map))
(irq_idx < end_idx) && irq_status;
irq_idx++)
if ((irq_status & dpu_irq_map[irq_idx].irq_mask) &&
- (dpu_irq_map[irq_idx].reg_idx == reg_idx)) {
+ (dpu_irq_map[irq_idx].reg_idx == reg_idx) &&
+ !(intr->obsolete_irq &
+ BIT(dpu_irq_map[irq_idx].intr_type))) {
/*
* Once a match on irq mask, perform a callback
* to the given cbfunc. cbfunc will take care
}
intr->irq_mask = m->mdss_irqs;
+ intr->obsolete_irq = m->obsolete_irq;
+
spin_lock_init(&intr->irq_lock);
return intr;
/**
* irq_idx_lookup - Lookup IRQ index on the HW interrupt type
* Used for all irq related ops
+ * @intr: HW interrupt handle
* @intr_type: Interrupt type defined in dpu_intr_type
* @instance_idx: HW interrupt block instance
* @return: irq_idx or -EINVAL for lookup fail
*/
- int (*irq_idx_lookup)(
+ int (*irq_idx_lookup)(struct dpu_hw_intr *intr,
enum dpu_intr_type intr_type,
u32 instance_idx);
* @save_irq_status: array of IRQ status reg storage created during init
* @irq_idx_tbl_size: total number of irq_idx mapped in the hw_interrupts
* @irq_lock: spinlock for accessing IRQ resources
+ * @obsolete_irq: irq types that are obsolete for a particular target
*/
struct dpu_hw_intr {
struct dpu_hw_blk_reg_map hw;
u32 irq_idx_tbl_size;
spinlock_t irq_lock;
unsigned long irq_mask;
+ unsigned long obsolete_irq;
};
/**
#define INTF_TEST_CTL 0x054
#define INTF_TP_COLOR0 0x058
#define INTF_TP_COLOR1 0x05C
+#define INTF_CONFIG2 0x060
+#define INTF_DISPLAY_DATA_HCTL 0x064
#define INTF_FRAME_LINE_COUNT_EN 0x0A8
#define INTF_FRAME_COUNT 0x0AC
#define INTF_LINE_COUNT 0x0B0
u32 active_hctl, display_hctl, hsync_ctl;
u32 polarity_ctl, den_polarity, hsync_polarity, vsync_polarity;
u32 panel_format;
- u32 intf_cfg;
+ u32 intf_cfg, intf_cfg2 = 0, display_data_hctl = 0;
/* read interface_cfg */
intf_cfg = DPU_REG_READ(c, INTF_CONFIG);
(COLOR_8BIT << 4) |
(0x21 << 8));
+ if (ctx->cap->features & BIT(DPU_DATA_HCTL_EN)) {
+ intf_cfg2 |= BIT(4);
+ display_data_hctl = display_hctl;
+ DPU_REG_WRITE(c, INTF_CONFIG2, intf_cfg2);
+ DPU_REG_WRITE(c, INTF_DISPLAY_DATA_HCTL, display_data_hctl);
+ }
+
DPU_REG_WRITE(c, INTF_HSYNC_CTL, hsync_ctl);
DPU_REG_WRITE(c, INTF_VSYNC_PERIOD_F0, vsync_period * hsync_period);
DPU_REG_WRITE(c, INTF_VSYNC_PULSE_WIDTH_F0,
struct dpu_hw_blk_reg_map *c = &intf->hw;
s->is_en = DPU_REG_READ(c, INTF_TIMING_ENGINE_EN);
+ s->is_prog_fetch_en = !!(DPU_REG_READ(c, INTF_CONFIG) & BIT(31));
if (s->is_en) {
s->frame_count = DPU_REG_READ(c, INTF_FRAME_COUNT);
s->line_count = DPU_REG_READ(c, INTF_LINE_COUNT);
struct intf_status {
u8 is_en; /* interface timing engine is enabled or not */
+ u8 is_prog_fetch_en; /* interface prog fetch counter is enabled or not */
u32 frame_count; /* frame count since timing engine enabled */
u32 line_count; /* current line count including blanking */
};
/**
* struct split_pipe_cfg - pipe configuration for dual display panels
- * @en : Enable/disable dual pipe confguration
+ * @en : Enable/disable dual pipe configuration
* @mode : Panel interface mode
* @intf : Interface id for main control path
* @split_flush_en: Allows both the paths to be flushed when master path is
* @setup_traffic_shaper : programs traffic shaper control
*/
struct dpu_hw_mdp_ops {
- /** setup_split_pipe() : Regsiters are not double buffered, thisk
+ /** setup_split_pipe() : Registers are not double buffered, thisk
* function should be called before timing control enable
* @mdp : mdp top context driver
* @cfg : upper and lower part of pipe configuration
#include <drm/drm_crtc.h>
#include <drm/drm_file.h>
+#include <drm/drm_vblank.h>
#include "msm_drv.h"
#include "msm_mmu.h"
#define DPU_DEBUGFS_DIR "msm_dpu"
#define DPU_DEBUGFS_HWMASKNAME "hw_log_mask"
+#define MIN_IB_BW 400000000ULL /* Min ib vote 400MB */
+
static int dpu_kms_hw_init(struct msm_kms *kms);
static void _dpu_kms_mmu_destroy(struct dpu_kms *dpu_kms);
DPU_DEBUG("REG_DMA is not defined");
}
+ dpu_kms_parse_data_bus_icc_path(dpu_kms);
+
pm_runtime_get_sync(&dpu_kms->pdev->dev);
dpu_kms->core_rev = readl_relaxed(dpu_kms->mmio + 0x0);
*/
dev->mode_config.allow_fb_modifiers = true;
+ dev->max_vblank_count = 0xffffffff;
+ /* Disable vblank irqs aggressively for power-saving */
+ dev->vblank_disable_immediate = true;
+
/*
* _dpu_kms_drm_obj_init should create the DRM related objects
* i.e. CRTCs, planes, encoders, connectors and so forth
dpu_vbif_init_memtypes(dpu_kms);
- if (of_device_is_compatible(dev->dev->of_node, "qcom,sc7180-mdss"))
- dpu_kms_parse_data_bus_icc_path(dpu_kms);
-
pm_runtime_put_sync(&dpu_kms->pdev->dev);
return 0;
ddev = dpu_kms->dev;
+ WARN_ON(!(dpu_kms->num_paths));
/* Min vote of BW is required before turning on AXI clk */
for (i = 0; i < dpu_kms->num_paths; i++)
- icc_set_bw(dpu_kms->path[i], 0,
- dpu_kms->catalog->perf.min_dram_ib);
+ icc_set_bw(dpu_kms->path[i], 0, Bps_to_icc(MIN_IB_BW));
rc = msm_dss_enable_clk(mp->clk_config, mp->num_clk, true);
if (rc) {
static const struct of_device_id dpu_dt_match[] = {
{ .compatible = "qcom,sdm845-dpu", },
{ .compatible = "qcom,sc7180-dpu", },
+ { .compatible = "qcom,sc7280-dpu", },
+ { .compatible = "qcom,sm8150-dpu", },
+ { .compatible = "qcom,sm8250-dpu", },
{}
};
MODULE_DEVICE_TABLE(of, dpu_dt_match);
void __iomem *mmio;
struct dss_module_power mp;
struct dpu_irq_controller irq_controller;
- struct icc_path *path[2];
- u32 num_paths;
};
-static int dpu_mdss_parse_data_bus_icc_path(struct drm_device *dev,
- struct dpu_mdss *dpu_mdss)
-{
- struct icc_path *path0 = of_icc_get(dev->dev, "mdp0-mem");
- struct icc_path *path1 = of_icc_get(dev->dev, "mdp1-mem");
-
- if (IS_ERR_OR_NULL(path0))
- return PTR_ERR_OR_ZERO(path0);
-
- dpu_mdss->path[0] = path0;
- dpu_mdss->num_paths = 1;
-
- if (!IS_ERR_OR_NULL(path1)) {
- dpu_mdss->path[1] = path1;
- dpu_mdss->num_paths++;
- }
-
- return 0;
-}
-
-static void dpu_mdss_icc_request_bw(struct msm_mdss *mdss)
-{
- struct dpu_mdss *dpu_mdss = to_dpu_mdss(mdss);
- int i;
- u64 avg_bw = dpu_mdss->num_paths ? MAX_BW / dpu_mdss->num_paths : 0;
-
- for (i = 0; i < dpu_mdss->num_paths; i++)
- icc_set_bw(dpu_mdss->path[i], avg_bw, kBps_to_icc(MAX_BW));
-}
-
static void dpu_mdss_irq(struct irq_desc *desc)
{
struct dpu_mdss *dpu_mdss = irq_desc_get_handler_data(desc);
struct dss_module_power *mp = &dpu_mdss->mp;
int ret;
- dpu_mdss_icc_request_bw(mdss);
-
ret = msm_dss_enable_clk(mp->clk_config, mp->num_clk, true);
if (ret) {
DPU_ERROR("clock enable failed, ret:%d\n", ret);
case DPU_HW_VER_620:
writel_relaxed(0x1e, dpu_mdss->mmio + UBWC_STATIC);
break;
+ case DPU_HW_VER_720:
+ writel_relaxed(0x101e, dpu_mdss->mmio + UBWC_STATIC);
+ break;
}
return ret;
{
struct dpu_mdss *dpu_mdss = to_dpu_mdss(mdss);
struct dss_module_power *mp = &dpu_mdss->mp;
- int ret, i;
+ int ret;
ret = msm_dss_enable_clk(mp->clk_config, mp->num_clk, false);
if (ret)
DPU_ERROR("clock disable failed, ret:%d\n", ret);
- for (i = 0; i < dpu_mdss->num_paths; i++)
- icc_set_bw(dpu_mdss->path[i], 0, 0);
-
return ret;
}
struct dpu_mdss *dpu_mdss = to_dpu_mdss(priv->mdss);
struct dss_module_power *mp = &dpu_mdss->mp;
int irq;
- int i;
pm_runtime_suspend(dev->dev);
pm_runtime_disable(dev->dev);
msm_dss_put_clk(mp->clk_config, mp->num_clk);
devm_kfree(&pdev->dev, mp->clk_config);
- for (i = 0; i < dpu_mdss->num_paths; i++)
- icc_put(dpu_mdss->path[i]);
-
if (dpu_mdss->mmio)
devm_iounmap(&pdev->dev, dpu_mdss->mmio);
dpu_mdss->mmio = NULL;
DRM_DEBUG("mapped mdss address space @%pK\n", dpu_mdss->mmio);
- if (!of_device_is_compatible(dev->dev->of_node, "qcom,sc7180-mdss")) {
- ret = dpu_mdss_parse_data_bus_icc_path(dev, dpu_mdss);
- if (ret)
- return ret;
- }
-
mp = &dpu_mdss->mp;
ret = msm_dss_parse_clock(pdev, mp);
if (ret) {
pm_runtime_enable(dev->dev);
- dpu_mdss_icc_request_bw(priv->mdss);
-
return ret;
irq_error:
{
struct mdp5_kms *mdp5_kms = get_kms(encoder);
struct device *dev = encoder->dev->dev;
- u32 total_lines_x100, vclks_line, cfg;
+ u32 total_lines, vclks_line, cfg;
long vsync_clk_speed;
struct mdp5_hw_mixer *mixer = mdp5_crtc_get_mixer(encoder->crtc);
int pp_id = mixer->pp;
return -EINVAL;
}
- total_lines_x100 = mode->vtotal * drm_mode_vrefresh(mode);
- if (!total_lines_x100) {
+ total_lines = mode->vtotal * drm_mode_vrefresh(mode);
+ if (!total_lines) {
DRM_DEV_ERROR(dev, "%s: vtotal(%d) or vrefresh(%d) is 0\n",
__func__, mode->vtotal, drm_mode_vrefresh(mode));
return -EINVAL;
vsync_clk_speed);
return -EINVAL;
}
- vclks_line = vsync_clk_speed * 100 / total_lines_x100;
+ vclks_line = vsync_clk_speed / total_lines;
cfg = MDP5_PP_SYNC_CONFIG_VSYNC_COUNTER_EN
| MDP5_PP_SYNC_CONFIG_VSYNC_IN_EN;
cfg |= MDP5_PP_SYNC_CONFIG_VSYNC_COUNT(vclks_line);
+ /*
+ * Tearcheck emits a blanking signal every vclks_line * vtotal * 2 ticks on
+ * the vsync_clk equating to roughly half the desired panel refresh rate.
+ * This is only necessary as stability fallback if interrupts from the
+ * panel arrive too late or not at all, but is currently used by default
+ * because these panel interrupts are not wired up yet.
+ */
mdp5_write(mdp5_kms, REG_MDP5_PP_SYNC_CONFIG_VSYNC(pp_id), cfg);
mdp5_write(mdp5_kms,
- REG_MDP5_PP_SYNC_CONFIG_HEIGHT(pp_id), 0xfff0);
+ REG_MDP5_PP_SYNC_CONFIG_HEIGHT(pp_id), (2 * mode->vtotal));
+
mdp5_write(mdp5_kms,
REG_MDP5_PP_VSYNC_INIT_VAL(pp_id), mode->vdisplay);
mdp5_write(mdp5_kms, REG_MDP5_PP_RD_PTR_IRQ(pp_id), mode->vdisplay + 1);
mdp5_write(mdp5_kms, REG_MDP5_PP_SYNC_THRESH(pp_id),
MDP5_PP_SYNC_THRESH_START(4) |
MDP5_PP_SYNC_THRESH_CONTINUE(4));
+ mdp5_write(mdp5_kms, REG_MDP5_PP_AUTOREFRESH_CONFIG(pp_id), 0x0);
return 0;
}
struct drm_dp_aux dp_aux;
};
+#define MAX_AUX_RETRIES 5
+
static const char *dp_aux_get_error(u32 aux_error)
{
switch (aux_error) {
ret = dp_aux_cmd_fifo_tx(aux, msg);
if (ret < 0) {
+ if (aux->native) {
+ aux->retry_cnt++;
+ if (!(aux->retry_cnt % MAX_AUX_RETRIES))
+ dp_catalog_aux_update_cfg(aux->catalog);
+ }
usleep_range(400, 500); /* at least 400us to next try */
goto unlock_exit;
}
debug->link->test_video.test_h_width);
seq_printf(m, "vdisplay: %d\n",
debug->link->test_video.test_v_height);
- seq_printf(m, "bpc: %u\n",
+ seq_printf(m, "bpc: %u\n",
dp_link_bit_depth_to_bpc(bpc));
} else
seq_puts(m, "0");
int rc = 0;
struct dp_debug_private *debug = container_of(dp_debug,
struct dp_debug_private, dp_debug);
- struct dentry *file;
- struct dentry *test_active;
- struct dentry *test_data, *test_type;
- file = debugfs_create_file("dp_debug", 0444, minor->debugfs_root,
+ debugfs_create_file("dp_debug", 0444, minor->debugfs_root,
debug, &dp_debug_fops);
- if (IS_ERR_OR_NULL(file)) {
- rc = PTR_ERR(file);
- DRM_ERROR("[%s] debugfs create file failed, rc=%d\n",
- DEBUG_NAME, rc);
- }
- test_active = debugfs_create_file("msm_dp_test_active", 0444,
+ debugfs_create_file("msm_dp_test_active", 0444,
minor->debugfs_root,
debug, &test_active_fops);
- if (IS_ERR_OR_NULL(test_active)) {
- rc = PTR_ERR(test_active);
- DRM_ERROR("[%s] debugfs test_active failed, rc=%d\n",
- DEBUG_NAME, rc);
- }
- test_data = debugfs_create_file("msm_dp_test_data", 0444,
+ debugfs_create_file("msm_dp_test_data", 0444,
minor->debugfs_root,
debug, &dp_test_data_fops);
- if (IS_ERR_OR_NULL(test_data)) {
- rc = PTR_ERR(test_data);
- DRM_ERROR("[%s] debugfs test_data failed, rc=%d\n",
- DEBUG_NAME, rc);
- }
- test_type = debugfs_create_file("msm_dp_test_type", 0444,
+ debugfs_create_file("msm_dp_test_type", 0444,
minor->debugfs_root,
debug, &dp_test_type_fops);
- if (IS_ERR_OR_NULL(test_type)) {
- rc = PTR_ERR(test_type);
- DRM_ERROR("[%s] debugfs test_type failed, rc=%d\n",
- DEBUG_NAME, rc);
- }
debug->root = minor->debugfs_root;
dp_usbpd->hpd_high = hpd;
- if (!hpd_priv->dp_cb && !hpd_priv->dp_cb->configure
- && !hpd_priv->dp_cb->disconnect) {
+ if (!hpd_priv->dp_cb || !hpd_priv->dp_cb->configure
+ || !hpd_priv->dp_cb->disconnect) {
pr_err("hpd dp_cb not initialized\n");
return -EINVAL;
}
DRM_ERROR("failed to '%s' clks for: %s. err=%d\n",
enable ? "enable" : "disable",
dp_parser_pm_name(pm_type), rc);
- return rc;
+ return rc;
}
if (pm_type == DP_CORE_PM)
struct msm_dsi_phy_shared_timings;
struct msm_dsi_phy_clk_request;
-enum msm_dsi_phy_type {
- MSM_DSI_PHY_28NM_HPM,
- MSM_DSI_PHY_28NM_LP,
- MSM_DSI_PHY_20NM,
- MSM_DSI_PHY_28NM_8960,
- MSM_DSI_PHY_14NM,
- MSM_DSI_PHY_10NM,
- MSM_DSI_PHY_7NM,
- MSM_DSI_PHY_7NM_V4_1,
- MSM_DSI_PHY_MAX
-};
-
enum msm_dsi_phy_usecase {
MSM_DSI_PHY_STANDALONE,
MSM_DSI_PHY_MASTER,
struct drm_encoder *msm_dsi_get_encoder(struct msm_dsi *msm_dsi);
-/* dsi pll */
-struct msm_dsi_pll;
-#ifdef CONFIG_DRM_MSM_DSI_PLL
-struct msm_dsi_pll *msm_dsi_pll_init(struct platform_device *pdev,
- enum msm_dsi_phy_type type, int dsi_id);
-void msm_dsi_pll_destroy(struct msm_dsi_pll *pll);
-int msm_dsi_pll_get_clk_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider, struct clk **pixel_clk_provider);
-void msm_dsi_pll_save_state(struct msm_dsi_pll *pll);
-int msm_dsi_pll_restore_state(struct msm_dsi_pll *pll);
-int msm_dsi_pll_set_usecase(struct msm_dsi_pll *pll,
- enum msm_dsi_phy_usecase uc);
-#else
-static inline struct msm_dsi_pll *msm_dsi_pll_init(struct platform_device *pdev,
- enum msm_dsi_phy_type type, int id) {
- return ERR_PTR(-ENODEV);
-}
-static inline void msm_dsi_pll_destroy(struct msm_dsi_pll *pll)
-{
-}
-static inline int msm_dsi_pll_get_clk_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider, struct clk **pixel_clk_provider)
-{
- return -ENODEV;
-}
-static inline void msm_dsi_pll_save_state(struct msm_dsi_pll *pll)
-{
-}
-static inline int msm_dsi_pll_restore_state(struct msm_dsi_pll *pll)
-{
- return 0;
-}
-static inline int msm_dsi_pll_set_usecase(struct msm_dsi_pll *pll,
- enum msm_dsi_phy_usecase uc)
-{
- return -ENODEV;
-}
-#endif
-
/* dsi host */
struct msm_dsi_host;
int msm_dsi_host_xfer_prepare(struct mipi_dsi_host *host,
int msm_dsi_host_register(struct mipi_dsi_host *host, bool check_defer);
void msm_dsi_host_unregister(struct mipi_dsi_host *host);
int msm_dsi_host_set_src_pll(struct mipi_dsi_host *host,
- struct msm_dsi_pll *src_pll);
+ struct msm_dsi_phy *src_phy);
void msm_dsi_host_reset_phy(struct mipi_dsi_host *host);
void msm_dsi_host_get_phy_clk_req(struct mipi_dsi_host *host,
struct msm_dsi_phy_clk_request *clk_req,
void msm_dsi_phy_driver_register(void);
void msm_dsi_phy_driver_unregister(void);
-int msm_dsi_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+int msm_dsi_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req);
void msm_dsi_phy_disable(struct msm_dsi_phy *phy);
void msm_dsi_phy_get_shared_timings(struct msm_dsi_phy *phy,
struct msm_dsi_phy_shared_timings *shared_timing);
-struct msm_dsi_pll *msm_dsi_phy_get_pll(struct msm_dsi_phy *phy);
void msm_dsi_phy_set_usecase(struct msm_dsi_phy *phy,
enum msm_dsi_phy_usecase uc);
+int msm_dsi_phy_get_clk_provider(struct msm_dsi_phy *phy,
+ struct clk **byte_clk_provider, struct clk **pixel_clk_provider);
+void msm_dsi_phy_pll_save_state(struct msm_dsi_phy *phy);
+int msm_dsi_phy_pll_restore_state(struct msm_dsi_phy *phy);
#endif /* __DSI_CONNECTOR_H__ */
.num_dsi = 2,
};
-/*
- * TODO: core_mmss_clk fails to enable for some reason, but things work fine
- * without it too. Figure out why it doesn't enable and uncomment below
- */
static const char * const dsi_8996_bus_clk_names[] = {
- "mdp_core", "iface", "bus", /* "core_mmss", */
+ "mdp_core", "iface", "bus", "core_mmss",
};
static const struct msm_dsi_config msm8996_dsi_cfg = {
msm_host = devm_kzalloc(&pdev->dev, sizeof(*msm_host), GFP_KERNEL);
if (!msm_host) {
- pr_err("%s: FAILED: cannot alloc dsi host\n",
- __func__);
ret = -ENOMEM;
goto fail;
}
}
int msm_dsi_host_set_src_pll(struct mipi_dsi_host *host,
- struct msm_dsi_pll *src_pll)
+ struct msm_dsi_phy *src_phy)
{
struct msm_dsi_host *msm_host = to_msm_dsi_host(host);
struct clk *byte_clk_provider, *pixel_clk_provider;
int ret;
- ret = msm_dsi_pll_get_clk_provider(src_pll,
+ ret = msm_dsi_phy_get_clk_provider(src_phy,
&byte_clk_provider, &pixel_clk_provider);
if (ret) {
pr_info("%s: can't get provider from pll, don't set parent\n",
struct msm_dsi *other_dsi = dsi_mgr_get_other_dsi(id);
struct msm_dsi *clk_master_dsi = dsi_mgr_get_dsi(DSI_CLOCK_MASTER);
struct msm_dsi *clk_slave_dsi = dsi_mgr_get_dsi(DSI_CLOCK_SLAVE);
- struct msm_dsi_pll *src_pll;
int ret;
if (!IS_DUAL_DSI()) {
return ret;
msm_dsi_phy_set_usecase(msm_dsi->phy, MSM_DSI_PHY_STANDALONE);
- src_pll = msm_dsi_phy_get_pll(msm_dsi->phy);
- if (IS_ERR(src_pll))
- return PTR_ERR(src_pll);
- ret = msm_dsi_host_set_src_pll(msm_dsi->host, src_pll);
+ ret = msm_dsi_host_set_src_pll(msm_dsi->host, msm_dsi->phy);
} else if (!other_dsi) {
ret = 0;
} else {
MSM_DSI_PHY_MASTER);
msm_dsi_phy_set_usecase(clk_slave_dsi->phy,
MSM_DSI_PHY_SLAVE);
- src_pll = msm_dsi_phy_get_pll(clk_master_dsi->phy);
- if (IS_ERR(src_pll))
- return PTR_ERR(src_pll);
- ret = msm_dsi_host_set_src_pll(msm_dsi->host, src_pll);
+ ret = msm_dsi_host_set_src_pll(msm_dsi->host, clk_master_dsi->phy);
if (ret)
return ret;
- ret = msm_dsi_host_set_src_pll(other_dsi->host, src_pll);
+ ret = msm_dsi_host_set_src_pll(other_dsi->host, clk_master_dsi->phy);
}
return ret;
}
-static int enable_phy(struct msm_dsi *msm_dsi, int src_pll_id,
+static int enable_phy(struct msm_dsi *msm_dsi,
struct msm_dsi_phy_shared_timings *shared_timings)
{
struct msm_dsi_phy_clk_request clk_req;
msm_dsi_host_get_phy_clk_req(msm_dsi->host, &clk_req, is_dual_dsi);
- ret = msm_dsi_phy_enable(msm_dsi->phy, src_pll_id, &clk_req);
+ ret = msm_dsi_phy_enable(msm_dsi->phy, &clk_req);
msm_dsi_phy_get_shared_timings(msm_dsi->phy, shared_timings);
return ret;
struct msm_dsi *msm_dsi = dsi_mgr_get_dsi(id);
struct msm_dsi *mdsi = dsi_mgr_get_dsi(DSI_CLOCK_MASTER);
struct msm_dsi *sdsi = dsi_mgr_get_dsi(DSI_CLOCK_SLAVE);
- int src_pll_id = IS_DUAL_DSI() ? DSI_CLOCK_MASTER : id;
int ret;
/* In case of dual DSI, some registers in PHY1 have been programmed
msm_dsi_host_reset_phy(mdsi->host);
msm_dsi_host_reset_phy(sdsi->host);
- ret = enable_phy(mdsi, src_pll_id,
+ ret = enable_phy(mdsi,
&shared_timings[DSI_CLOCK_MASTER]);
if (ret)
return ret;
- ret = enable_phy(sdsi, src_pll_id,
+ ret = enable_phy(sdsi,
&shared_timings[DSI_CLOCK_SLAVE]);
if (ret) {
msm_dsi_phy_disable(mdsi->phy);
}
} else {
msm_dsi_host_reset_phy(msm_dsi->host);
- ret = enable_phy(msm_dsi, src_pll_id, &shared_timings[id]);
+ ret = enable_phy(msm_dsi, &shared_timings[id]);
if (ret)
return ret;
}
struct msm_dsi *msm_dsi1 = dsi_mgr_get_dsi(DSI_1);
struct mipi_dsi_host *host = msm_dsi->host;
struct drm_panel *panel = msm_dsi->panel;
- struct msm_dsi_pll *src_pll;
bool is_dual_dsi = IS_DUAL_DSI();
int ret;
id, ret);
}
- /* Save PLL status if it is a clock source */
- src_pll = msm_dsi_phy_get_pll(msm_dsi->phy);
- msm_dsi_pll_save_state(src_pll);
+ /* Save PHY status if it is a clock source */
+ msm_dsi_phy_pll_save_state(msm_dsi->phy);
ret = msm_dsi_host_power_off(host);
if (ret)
* Copyright (c) 2015, The Linux Foundation. All rights reserved.
*/
+#include <linux/clk-provider.h>
#include <linux/platform_device.h>
#include "dsi_phy.h"
return 0;
}
-void msm_dsi_phy_set_src_pll(struct msm_dsi_phy *phy, int pll_id, u32 reg,
- u32 bit_mask)
-{
- int phy_id = phy->id;
- u32 val;
-
- if ((phy_id >= DSI_MAX) || (pll_id >= DSI_MAX))
- return;
-
- val = dsi_phy_read(phy->base + reg);
-
- if (phy->cfg->src_pll_truthtable[phy_id][pll_id])
- dsi_phy_write(phy->base + reg, val | bit_mask);
- else
- dsi_phy_write(phy->base + reg, val & (~bit_mask));
-}
-
static int dsi_phy_regulator_init(struct msm_dsi_phy *phy)
{
struct regulator_bulk_data *s = phy->supplies;
return -EINVAL;
}
-int msm_dsi_phy_init_common(struct msm_dsi_phy *phy)
-{
- struct platform_device *pdev = phy->pdev;
- int ret = 0;
-
- phy->reg_base = msm_ioremap(pdev, "dsi_phy_regulator",
- "DSI_PHY_REG");
- if (IS_ERR(phy->reg_base)) {
- DRM_DEV_ERROR(&pdev->dev, "%s: failed to map phy regulator base\n",
- __func__);
- ret = -ENOMEM;
- goto fail;
- }
-
-fail:
- return ret;
-}
-
static int dsi_phy_driver_probe(struct platform_device *pdev)
{
struct msm_dsi_phy *phy;
if (!match)
return -ENODEV;
+ phy->provided_clocks = devm_kzalloc(dev,
+ struct_size(phy->provided_clocks, hws, NUM_PROVIDED_CLKS),
+ GFP_KERNEL);
+ if (!phy->provided_clocks)
+ return -ENOMEM;
+
+ phy->provided_clocks->num = NUM_PROVIDED_CLKS;
+
phy->cfg = match->data;
phy->pdev = pdev;
goto fail;
}
+ phy->pll_base = msm_ioremap(pdev, "dsi_pll", "DSI_PLL");
+ if (IS_ERR(phy->pll_base)) {
+ DRM_DEV_ERROR(&pdev->dev, "%s: failed to map pll base\n", __func__);
+ ret = -ENOMEM;
+ goto fail;
+ }
+
+ if (phy->cfg->has_phy_lane) {
+ phy->lane_base = msm_ioremap(pdev, "dsi_phy_lane", "DSI_PHY_LANE");
+ if (IS_ERR(phy->lane_base)) {
+ DRM_DEV_ERROR(&pdev->dev, "%s: failed to map phy lane base\n", __func__);
+ ret = -ENOMEM;
+ goto fail;
+ }
+ }
+
+ if (phy->cfg->has_phy_regulator) {
+ phy->reg_base = msm_ioremap(pdev, "dsi_phy_regulator", "DSI_PHY_REG");
+ if (IS_ERR(phy->reg_base)) {
+ DRM_DEV_ERROR(&pdev->dev, "%s: failed to map phy regulator base\n", __func__);
+ ret = -ENOMEM;
+ goto fail;
+ }
+ }
+
ret = dsi_phy_regulator_init(phy);
if (ret)
goto fail;
goto fail;
}
- if (phy->cfg->ops.init) {
- ret = phy->cfg->ops.init(phy);
- if (ret)
- goto fail;
- }
-
/* PLL init will call into clk_register which requires
* register access, so we need to enable power and ahb clock.
*/
if (ret)
goto fail;
- phy->pll = msm_dsi_pll_init(pdev, phy->cfg->type, phy->id);
- if (IS_ERR_OR_NULL(phy->pll)) {
- DRM_DEV_INFO(dev,
- "%s: pll init failed: %ld, need separate pll clk driver\n",
- __func__, PTR_ERR(phy->pll));
- phy->pll = NULL;
+ if (phy->cfg->ops.pll_init) {
+ ret = phy->cfg->ops.pll_init(phy);
+ if (ret) {
+ DRM_DEV_INFO(dev,
+ "%s: pll init failed: %d, need separate pll clk driver\n",
+ __func__, ret);
+ goto fail;
+ }
+ }
+
+ ret = devm_of_clk_add_hw_provider(dev, of_clk_hw_onecell_get,
+ phy->provided_clocks);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "%s: failed to register clk provider: %d\n", __func__, ret);
+ goto fail;
}
dsi_phy_disable_resource(phy);
return ret;
}
-static int dsi_phy_driver_remove(struct platform_device *pdev)
-{
- struct msm_dsi_phy *phy = platform_get_drvdata(pdev);
-
- if (phy && phy->pll) {
- msm_dsi_pll_destroy(phy->pll);
- phy->pll = NULL;
- }
-
- platform_set_drvdata(pdev, NULL);
-
- return 0;
-}
-
static struct platform_driver dsi_phy_platform_driver = {
.probe = dsi_phy_driver_probe,
- .remove = dsi_phy_driver_remove,
.driver = {
.name = "msm_dsi_phy",
.of_match_table = dsi_phy_dt_match,
platform_driver_unregister(&dsi_phy_platform_driver);
}
-int msm_dsi_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+int msm_dsi_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req)
{
struct device *dev = &phy->pdev->dev;
goto reg_en_fail;
}
- ret = phy->cfg->ops.enable(phy, src_pll_id, clk_req);
+ ret = phy->cfg->ops.enable(phy, clk_req);
if (ret) {
DRM_DEV_ERROR(dev, "%s: phy enable failed, %d\n", __func__, ret);
goto phy_en_fail;
* source.
*/
if (phy->usecase != MSM_DSI_PHY_SLAVE) {
- ret = msm_dsi_pll_restore_state(phy->pll);
+ ret = msm_dsi_phy_pll_restore_state(phy);
if (ret) {
- DRM_DEV_ERROR(dev, "%s: failed to restore pll state, %d\n",
+ DRM_DEV_ERROR(dev, "%s: failed to restore phy state, %d\n",
__func__, ret);
goto pll_restor_fail;
}
sizeof(*shared_timings));
}
-struct msm_dsi_pll *msm_dsi_phy_get_pll(struct msm_dsi_phy *phy)
-{
- if (!phy)
- return NULL;
-
- return phy->pll;
-}
-
void msm_dsi_phy_set_usecase(struct msm_dsi_phy *phy,
enum msm_dsi_phy_usecase uc)
{
if (phy)
phy->usecase = uc;
}
+
+int msm_dsi_phy_get_clk_provider(struct msm_dsi_phy *phy,
+ struct clk **byte_clk_provider, struct clk **pixel_clk_provider)
+{
+ if (byte_clk_provider)
+ *byte_clk_provider = phy->provided_clocks->hws[DSI_BYTE_PLL_CLK]->clk;
+ if (pixel_clk_provider)
+ *pixel_clk_provider = phy->provided_clocks->hws[DSI_PIXEL_PLL_CLK]->clk;
+
+ return -EINVAL;
+}
+
+void msm_dsi_phy_pll_save_state(struct msm_dsi_phy *phy)
+{
+ if (phy->cfg->ops.save_pll_state) {
+ phy->cfg->ops.save_pll_state(phy);
+ phy->state_saved = true;
+ }
+}
+
+int msm_dsi_phy_pll_restore_state(struct msm_dsi_phy *phy)
+{
+ int ret;
+
+ if (phy->cfg->ops.restore_pll_state && phy->state_saved) {
+ ret = phy->cfg->ops.restore_pll_state(phy);
+ if (ret)
+ return ret;
+
+ phy->state_saved = false;
+ }
+
+ return 0;
+}
#ifndef __DSI_PHY_H__
#define __DSI_PHY_H__
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
#include <linux/regulator/consumer.h>
#include "dsi.h"
#define dsi_phy_read(offset) msm_readl((offset))
#define dsi_phy_write(offset, data) msm_writel((data), (offset))
-
-/* v3.0.0 10nm implementation that requires the old timings settings */
-#define V3_0_0_10NM_OLD_TIMINGS_QUIRK BIT(0)
+#define dsi_phy_write_udelay(offset, data, delay_us) { msm_writel((data), (offset)); udelay(delay_us); }
+#define dsi_phy_write_ndelay(offset, data, delay_ns) { msm_writel((data), (offset)); ndelay(delay_ns); }
struct msm_dsi_phy_ops {
- int (*init) (struct msm_dsi_phy *phy);
- int (*enable)(struct msm_dsi_phy *phy, int src_pll_id,
+ int (*pll_init)(struct msm_dsi_phy *phy);
+ int (*enable)(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req);
void (*disable)(struct msm_dsi_phy *phy);
+ void (*save_pll_state)(struct msm_dsi_phy *phy);
+ int (*restore_pll_state)(struct msm_dsi_phy *phy);
};
struct msm_dsi_phy_cfg {
- enum msm_dsi_phy_type type;
struct dsi_reg_config reg_cfg;
struct msm_dsi_phy_ops ops;
- /*
- * Each cell {phy_id, pll_id} of the truth table indicates
- * if the source PLL selection bit should be set for each PHY.
- * Fill default H/W values in illegal cells, eg. cell {0, 1}.
- */
- bool src_pll_truthtable[DSI_MAX][DSI_MAX];
+ unsigned long min_pll_rate;
+ unsigned long max_pll_rate;
+
const resource_size_t io_start[DSI_MAX];
const int num_dsi_phy;
const int quirks;
+ bool has_phy_regulator;
+ bool has_phy_lane;
};
extern const struct msm_dsi_phy_cfg dsi_phy_28nm_hpm_cfgs;
u8 hs_halfbyte_en_ckln;
};
+#define DSI_BYTE_PLL_CLK 0
+#define DSI_PIXEL_PLL_CLK 1
+#define NUM_PROVIDED_CLKS 2
+
struct msm_dsi_phy {
struct platform_device *pdev;
void __iomem *base;
+ void __iomem *pll_base;
void __iomem *reg_base;
void __iomem *lane_base;
int id;
enum msm_dsi_phy_usecase usecase;
bool regulator_ldo_mode;
- struct msm_dsi_pll *pll;
+ struct clk_hw *vco_hw;
+ bool pll_on;
+
+ struct clk_hw_onecell_data *provided_clocks;
+
+ bool state_saved;
};
/*
struct msm_dsi_phy_clk_request *clk_req);
int msm_dsi_dphy_timing_calc_v4(struct msm_dsi_dphy_timing *timing,
struct msm_dsi_phy_clk_request *clk_req);
-void msm_dsi_phy_set_src_pll(struct msm_dsi_phy *phy, int pll_id, u32 reg,
- u32 bit_mask);
-int msm_dsi_phy_init_common(struct msm_dsi_phy *phy);
#endif /* __DSI_PHY_H__ */
-
* Copyright (c) 2018, The Linux Foundation
*/
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include <linux/iopoll.h>
#include "dsi_phy.h"
#include "dsi.xml.h"
+/*
+ * DSI PLL 10nm - clock diagram (eg: DSI0):
+ *
+ * dsi0_pll_out_div_clk dsi0_pll_bit_clk
+ * | |
+ * | |
+ * +---------+ | +----------+ | +----+
+ * dsi0vco_clk ---| out_div |--o--| divl_3_0 |--o--| /8 |-- dsi0_phy_pll_out_byteclk
+ * +---------+ | +----------+ | +----+
+ * | |
+ * | | dsi0_pll_by_2_bit_clk
+ * | | |
+ * | | +----+ | |\ dsi0_pclk_mux
+ * | |--| /2 |--o--| \ |
+ * | | +----+ | \ | +---------+
+ * | --------------| |--o--| div_7_4 |-- dsi0_phy_pll_out_dsiclk
+ * |------------------------------| / +---------+
+ * | +-----+ | /
+ * -----------| /4? |--o----------|/
+ * +-----+ | |
+ * | |dsiclk_sel
+ * |
+ * dsi0_pll_post_out_div_clk
+ */
+
+#define VCO_REF_CLK_RATE 19200000
+#define FRAC_BITS 18
+
+/* v3.0.0 10nm implementation that requires the old timings settings */
+#define DSI_PHY_10NM_QUIRK_OLD_TIMINGS BIT(0)
+
+struct dsi_pll_config {
+ bool enable_ssc;
+ bool ssc_center;
+ u32 ssc_freq;
+ u32 ssc_offset;
+ u32 ssc_adj_per;
+
+ /* out */
+ u32 pll_prop_gain_rate;
+ u32 decimal_div_start;
+ u32 frac_div_start;
+ u32 pll_clock_inverters;
+ u32 ssc_stepsize;
+ u32 ssc_div_per;
+};
+
+struct pll_10nm_cached_state {
+ unsigned long vco_rate;
+ u8 bit_clk_div;
+ u8 pix_clk_div;
+ u8 pll_out_div;
+ u8 pll_mux;
+};
+
+struct dsi_pll_10nm {
+ struct clk_hw clk_hw;
+
+ struct msm_dsi_phy *phy;
+
+ u64 vco_current_rate;
+
+ /* protects REG_DSI_10nm_PHY_CMN_CLK_CFG0 register */
+ spinlock_t postdiv_lock;
+
+ struct pll_10nm_cached_state cached_state;
+
+ struct dsi_pll_10nm *slave;
+};
+
+#define to_pll_10nm(x) container_of(x, struct dsi_pll_10nm, clk_hw)
+
+/*
+ * Global list of private DSI PLL struct pointers. We need this for Dual DSI
+ * mode, where the master PLL's clk_ops needs access the slave's private data
+ */
+static struct dsi_pll_10nm *pll_10nm_list[DSI_MAX];
+
+static void dsi_pll_setup_config(struct dsi_pll_config *config)
+{
+ config->ssc_freq = 31500;
+ config->ssc_offset = 5000;
+ config->ssc_adj_per = 2;
+
+ config->enable_ssc = false;
+ config->ssc_center = false;
+}
+
+static void dsi_pll_calc_dec_frac(struct dsi_pll_10nm *pll, struct dsi_pll_config *config)
+{
+ u64 fref = VCO_REF_CLK_RATE;
+ u64 pll_freq;
+ u64 divider;
+ u64 dec, dec_multiple;
+ u32 frac;
+ u64 multiplier;
+
+ pll_freq = pll->vco_current_rate;
+
+ divider = fref * 2;
+
+ multiplier = 1 << FRAC_BITS;
+ dec_multiple = div_u64(pll_freq * multiplier, divider);
+ dec = div_u64_rem(dec_multiple, multiplier, &frac);
+
+ if (pll_freq <= 1900000000UL)
+ config->pll_prop_gain_rate = 8;
+ else if (pll_freq <= 3000000000UL)
+ config->pll_prop_gain_rate = 10;
+ else
+ config->pll_prop_gain_rate = 12;
+ if (pll_freq < 1100000000UL)
+ config->pll_clock_inverters = 8;
+ else
+ config->pll_clock_inverters = 0;
+
+ config->decimal_div_start = dec;
+ config->frac_div_start = frac;
+}
+
+#define SSC_CENTER BIT(0)
+#define SSC_EN BIT(1)
+
+static void dsi_pll_calc_ssc(struct dsi_pll_10nm *pll, struct dsi_pll_config *config)
+{
+ u32 ssc_per;
+ u32 ssc_mod;
+ u64 ssc_step_size;
+ u64 frac;
+
+ if (!config->enable_ssc) {
+ DBG("SSC not enabled\n");
+ return;
+ }
+
+ ssc_per = DIV_ROUND_CLOSEST(VCO_REF_CLK_RATE, config->ssc_freq) / 2 - 1;
+ ssc_mod = (ssc_per + 1) % (config->ssc_adj_per + 1);
+ ssc_per -= ssc_mod;
+
+ frac = config->frac_div_start;
+ ssc_step_size = config->decimal_div_start;
+ ssc_step_size *= (1 << FRAC_BITS);
+ ssc_step_size += frac;
+ ssc_step_size *= config->ssc_offset;
+ ssc_step_size *= (config->ssc_adj_per + 1);
+ ssc_step_size = div_u64(ssc_step_size, (ssc_per + 1));
+ ssc_step_size = DIV_ROUND_CLOSEST_ULL(ssc_step_size, 1000000);
+
+ config->ssc_div_per = ssc_per;
+ config->ssc_stepsize = ssc_step_size;
+
+ pr_debug("SCC: Dec:%d, frac:%llu, frac_bits:%d\n",
+ config->decimal_div_start, frac, FRAC_BITS);
+ pr_debug("SSC: div_per:0x%X, stepsize:0x%X, adjper:0x%X\n",
+ ssc_per, (u32)ssc_step_size, config->ssc_adj_per);
+}
+
+static void dsi_pll_ssc_commit(struct dsi_pll_10nm *pll, struct dsi_pll_config *config)
+{
+ void __iomem *base = pll->phy->pll_base;
+
+ if (config->enable_ssc) {
+ pr_debug("SSC is enabled\n");
+
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_SSC_STEPSIZE_LOW_1,
+ config->ssc_stepsize & 0xff);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_SSC_STEPSIZE_HIGH_1,
+ config->ssc_stepsize >> 8);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_PER_LOW_1,
+ config->ssc_div_per & 0xff);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_PER_HIGH_1,
+ config->ssc_div_per >> 8);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_ADJPER_LOW_1,
+ config->ssc_adj_per & 0xff);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_ADJPER_HIGH_1,
+ config->ssc_adj_per >> 8);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_SSC_CONTROL,
+ SSC_EN | (config->ssc_center ? SSC_CENTER : 0));
+ }
+}
+
+static void dsi_pll_config_hzindep_reg(struct dsi_pll_10nm *pll)
+{
+ void __iomem *base = pll->phy->pll_base;
+
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_ANALOG_CONTROLS_ONE, 0x80);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_ANALOG_CONTROLS_TWO, 0x03);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_ANALOG_CONTROLS_THREE, 0x00);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_DSM_DIVIDER, 0x00);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_FEEDBACK_DIVIDER, 0x4e);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_CALIBRATION_SETTINGS, 0x40);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_BAND_SEL_CAL_SETTINGS_THREE,
+ 0xba);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_FREQ_DETECT_SETTINGS_ONE, 0x0c);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_OUTDIV, 0x00);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_CORE_OVERRIDE, 0x00);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_DIGITAL_TIMERS_TWO, 0x08);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_PROP_GAIN_RATE_1, 0x08);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_BAND_SET_RATE_1, 0xc0);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_INT_GAIN_IFILT_BAND_1, 0xfa);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_FL_INT_GAIN_PFILT_BAND_1,
+ 0x4c);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_LOCK_OVERRIDE, 0x80);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PFILT, 0x29);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_IFILT, 0x3f);
+}
+
+static void dsi_pll_commit(struct dsi_pll_10nm *pll, struct dsi_pll_config *config)
+{
+ void __iomem *base = pll->phy->pll_base;
+
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_CORE_INPUT_OVERRIDE, 0x12);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_DECIMAL_DIV_START_1,
+ config->decimal_div_start);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_LOW_1,
+ config->frac_div_start & 0xff);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_MID_1,
+ (config->frac_div_start & 0xff00) >> 8);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_HIGH_1,
+ (config->frac_div_start & 0x30000) >> 16);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_LOCKDET_RATE_1, 64);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_PLL_LOCK_DELAY, 0x06);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_CMODE, 0x10);
+ dsi_phy_write(base + REG_DSI_10nm_PHY_PLL_CLOCK_INVERTERS,
+ config->pll_clock_inverters);
+}
+
+static int dsi_pll_10nm_vco_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(hw);
+ struct dsi_pll_config config;
+
+ DBG("DSI PLL%d rate=%lu, parent's=%lu", pll_10nm->phy->id, rate,
+ parent_rate);
+
+ pll_10nm->vco_current_rate = rate;
+
+ dsi_pll_setup_config(&config);
+
+ dsi_pll_calc_dec_frac(pll_10nm, &config);
+
+ dsi_pll_calc_ssc(pll_10nm, &config);
+
+ dsi_pll_commit(pll_10nm, &config);
+
+ dsi_pll_config_hzindep_reg(pll_10nm);
+
+ dsi_pll_ssc_commit(pll_10nm, &config);
+
+ /* flush, ensure all register writes are done*/
+ wmb();
+
+ return 0;
+}
+
+static int dsi_pll_10nm_lock_status(struct dsi_pll_10nm *pll)
+{
+ struct device *dev = &pll->phy->pdev->dev;
+ int rc;
+ u32 status = 0;
+ u32 const delay_us = 100;
+ u32 const timeout_us = 5000;
+
+ rc = readl_poll_timeout_atomic(pll->phy->pll_base +
+ REG_DSI_10nm_PHY_PLL_COMMON_STATUS_ONE,
+ status,
+ ((status & BIT(0)) > 0),
+ delay_us,
+ timeout_us);
+ if (rc)
+ DRM_DEV_ERROR(dev, "DSI PLL(%d) lock failed, status=0x%08x\n",
+ pll->phy->id, status);
+
+ return rc;
+}
+
+static void dsi_pll_disable_pll_bias(struct dsi_pll_10nm *pll)
+{
+ u32 data = dsi_phy_read(pll->phy->base + REG_DSI_10nm_PHY_CMN_CTRL_0);
+
+ dsi_phy_write(pll->phy->pll_base + REG_DSI_10nm_PHY_PLL_SYSTEM_MUXES, 0);
+ dsi_phy_write(pll->phy->base + REG_DSI_10nm_PHY_CMN_CTRL_0,
+ data & ~BIT(5));
+ ndelay(250);
+}
+
+static void dsi_pll_enable_pll_bias(struct dsi_pll_10nm *pll)
+{
+ u32 data = dsi_phy_read(pll->phy->base + REG_DSI_10nm_PHY_CMN_CTRL_0);
+
+ dsi_phy_write(pll->phy->base + REG_DSI_10nm_PHY_CMN_CTRL_0,
+ data | BIT(5));
+ dsi_phy_write(pll->phy->pll_base + REG_DSI_10nm_PHY_PLL_SYSTEM_MUXES, 0xc0);
+ ndelay(250);
+}
+
+static void dsi_pll_disable_global_clk(struct dsi_pll_10nm *pll)
+{
+ u32 data;
+
+ data = dsi_phy_read(pll->phy->base + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
+ dsi_phy_write(pll->phy->base + REG_DSI_10nm_PHY_CMN_CLK_CFG1,
+ data & ~BIT(5));
+}
+
+static void dsi_pll_enable_global_clk(struct dsi_pll_10nm *pll)
+{
+ u32 data;
+
+ data = dsi_phy_read(pll->phy->base + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
+ dsi_phy_write(pll->phy->base + REG_DSI_10nm_PHY_CMN_CLK_CFG1,
+ data | BIT(5));
+}
+
+static int dsi_pll_10nm_vco_prepare(struct clk_hw *hw)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(hw);
+ struct device *dev = &pll_10nm->phy->pdev->dev;
+ int rc;
+
+ dsi_pll_enable_pll_bias(pll_10nm);
+ if (pll_10nm->slave)
+ dsi_pll_enable_pll_bias(pll_10nm->slave);
+
+ rc = dsi_pll_10nm_vco_set_rate(hw,pll_10nm->vco_current_rate, 0);
+ if (rc) {
+ DRM_DEV_ERROR(dev, "vco_set_rate failed, rc=%d\n", rc);
+ return rc;
+ }
+
+ /* Start PLL */
+ dsi_phy_write(pll_10nm->phy->base + REG_DSI_10nm_PHY_CMN_PLL_CNTRL,
+ 0x01);
+
+ /*
+ * ensure all PLL configurations are written prior to checking
+ * for PLL lock.
+ */
+ wmb();
+
+ /* Check for PLL lock */
+ rc = dsi_pll_10nm_lock_status(pll_10nm);
+ if (rc) {
+ DRM_DEV_ERROR(dev, "PLL(%d) lock failed\n", pll_10nm->phy->id);
+ goto error;
+ }
+
+ pll_10nm->phy->pll_on = true;
+
+ dsi_pll_enable_global_clk(pll_10nm);
+ if (pll_10nm->slave)
+ dsi_pll_enable_global_clk(pll_10nm->slave);
+
+ dsi_phy_write(pll_10nm->phy->base + REG_DSI_10nm_PHY_CMN_RBUF_CTRL,
+ 0x01);
+ if (pll_10nm->slave)
+ dsi_phy_write(pll_10nm->slave->phy->base +
+ REG_DSI_10nm_PHY_CMN_RBUF_CTRL, 0x01);
+
+error:
+ return rc;
+}
+
+static void dsi_pll_disable_sub(struct dsi_pll_10nm *pll)
+{
+ dsi_phy_write(pll->phy->base + REG_DSI_10nm_PHY_CMN_RBUF_CTRL, 0);
+ dsi_pll_disable_pll_bias(pll);
+}
+
+static void dsi_pll_10nm_vco_unprepare(struct clk_hw *hw)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(hw);
+
+ /*
+ * To avoid any stray glitches while abruptly powering down the PLL
+ * make sure to gate the clock using the clock enable bit before
+ * powering down the PLL
+ */
+ dsi_pll_disable_global_clk(pll_10nm);
+ dsi_phy_write(pll_10nm->phy->base + REG_DSI_10nm_PHY_CMN_PLL_CNTRL, 0);
+ dsi_pll_disable_sub(pll_10nm);
+ if (pll_10nm->slave) {
+ dsi_pll_disable_global_clk(pll_10nm->slave);
+ dsi_pll_disable_sub(pll_10nm->slave);
+ }
+ /* flush, ensure all register writes are done */
+ wmb();
+ pll_10nm->phy->pll_on = false;
+}
+
+static unsigned long dsi_pll_10nm_vco_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(hw);
+ void __iomem *base = pll_10nm->phy->pll_base;
+ u64 ref_clk = VCO_REF_CLK_RATE;
+ u64 vco_rate = 0x0;
+ u64 multiplier;
+ u32 frac;
+ u32 dec;
+ u64 pll_freq, tmp64;
+
+ dec = dsi_phy_read(base + REG_DSI_10nm_PHY_PLL_DECIMAL_DIV_START_1);
+ dec &= 0xff;
+
+ frac = dsi_phy_read(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_LOW_1);
+ frac |= ((dsi_phy_read(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_MID_1) &
+ 0xff) << 8);
+ frac |= ((dsi_phy_read(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_HIGH_1) &
+ 0x3) << 16);
+
+ /*
+ * TODO:
+ * 1. Assumes prescaler is disabled
+ */
+ multiplier = 1 << FRAC_BITS;
+ pll_freq = dec * (ref_clk * 2);
+ tmp64 = (ref_clk * 2 * frac);
+ pll_freq += div_u64(tmp64, multiplier);
+
+ vco_rate = pll_freq;
+
+ DBG("DSI PLL%d returning vco rate = %lu, dec = %x, frac = %x",
+ pll_10nm->phy->id, (unsigned long)vco_rate, dec, frac);
+
+ return (unsigned long)vco_rate;
+}
+
+static long dsi_pll_10nm_clk_round_rate(struct clk_hw *hw,
+ unsigned long rate, unsigned long *parent_rate)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(hw);
+
+ if (rate < pll_10nm->phy->cfg->min_pll_rate)
+ return pll_10nm->phy->cfg->min_pll_rate;
+ else if (rate > pll_10nm->phy->cfg->max_pll_rate)
+ return pll_10nm->phy->cfg->max_pll_rate;
+ else
+ return rate;
+}
+
+static const struct clk_ops clk_ops_dsi_pll_10nm_vco = {
+ .round_rate = dsi_pll_10nm_clk_round_rate,
+ .set_rate = dsi_pll_10nm_vco_set_rate,
+ .recalc_rate = dsi_pll_10nm_vco_recalc_rate,
+ .prepare = dsi_pll_10nm_vco_prepare,
+ .unprepare = dsi_pll_10nm_vco_unprepare,
+};
+
+/*
+ * PLL Callbacks
+ */
+
+static void dsi_10nm_pll_save_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(phy->vco_hw);
+ struct pll_10nm_cached_state *cached = &pll_10nm->cached_state;
+ void __iomem *phy_base = pll_10nm->phy->base;
+ u32 cmn_clk_cfg0, cmn_clk_cfg1;
+
+ cached->pll_out_div = dsi_phy_read(pll_10nm->phy->pll_base +
+ REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE);
+ cached->pll_out_div &= 0x3;
+
+ cmn_clk_cfg0 = dsi_phy_read(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG0);
+ cached->bit_clk_div = cmn_clk_cfg0 & 0xf;
+ cached->pix_clk_div = (cmn_clk_cfg0 & 0xf0) >> 4;
+
+ cmn_clk_cfg1 = dsi_phy_read(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
+ cached->pll_mux = cmn_clk_cfg1 & 0x3;
+
+ DBG("DSI PLL%d outdiv %x bit_clk_div %x pix_clk_div %x pll_mux %x",
+ pll_10nm->phy->id, cached->pll_out_div, cached->bit_clk_div,
+ cached->pix_clk_div, cached->pll_mux);
+}
+
+static int dsi_10nm_pll_restore_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(phy->vco_hw);
+ struct pll_10nm_cached_state *cached = &pll_10nm->cached_state;
+ void __iomem *phy_base = pll_10nm->phy->base;
+ u32 val;
+ int ret;
+
+ val = dsi_phy_read(pll_10nm->phy->pll_base + REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE);
+ val &= ~0x3;
+ val |= cached->pll_out_div;
+ dsi_phy_write(pll_10nm->phy->pll_base + REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE, val);
+
+ dsi_phy_write(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG0,
+ cached->bit_clk_div | (cached->pix_clk_div << 4));
+
+ val = dsi_phy_read(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
+ val &= ~0x3;
+ val |= cached->pll_mux;
+ dsi_phy_write(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG1, val);
+
+ ret = dsi_pll_10nm_vco_set_rate(phy->vco_hw,
+ pll_10nm->vco_current_rate,
+ VCO_REF_CLK_RATE);
+ if (ret) {
+ DRM_DEV_ERROR(&pll_10nm->phy->pdev->dev,
+ "restore vco rate failed. ret=%d\n", ret);
+ return ret;
+ }
+
+ DBG("DSI PLL%d", pll_10nm->phy->id);
+
+ return 0;
+}
+
+static int dsi_10nm_set_usecase(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_10nm *pll_10nm = to_pll_10nm(phy->vco_hw);
+ void __iomem *base = phy->base;
+ u32 data = 0x0; /* internal PLL */
+
+ DBG("DSI PLL%d", pll_10nm->phy->id);
+
+ switch (phy->usecase) {
+ case MSM_DSI_PHY_STANDALONE:
+ break;
+ case MSM_DSI_PHY_MASTER:
+ pll_10nm->slave = pll_10nm_list[(pll_10nm->phy->id + 1) % DSI_MAX];
+ break;
+ case MSM_DSI_PHY_SLAVE:
+ data = 0x1; /* external PLL */
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ /* set PLL src */
+ dsi_phy_write(base + REG_DSI_10nm_PHY_CMN_CLK_CFG1, (data << 2));
+
+ return 0;
+}
+
+/*
+ * The post dividers and mux clocks are created using the standard divider and
+ * mux API. Unlike the 14nm PHY, the slave PLL doesn't need its dividers/mux
+ * state to follow the master PLL's divider/mux state. Therefore, we don't
+ * require special clock ops that also configure the slave PLL registers
+ */
+static int pll_10nm_register(struct dsi_pll_10nm *pll_10nm, struct clk_hw **provided_clocks)
+{
+ char clk_name[32], parent[32], vco_name[32];
+ char parent2[32], parent3[32], parent4[32];
+ struct clk_init_data vco_init = {
+ .parent_names = (const char *[]){ "xo" },
+ .num_parents = 1,
+ .name = vco_name,
+ .flags = CLK_IGNORE_UNUSED,
+ .ops = &clk_ops_dsi_pll_10nm_vco,
+ };
+ struct device *dev = &pll_10nm->phy->pdev->dev;
+ struct clk_hw *hw;
+ int ret;
+
+ DBG("DSI%d", pll_10nm->phy->id);
+
+ snprintf(vco_name, 32, "dsi%dvco_clk", pll_10nm->phy->id);
+ pll_10nm->clk_hw.init = &vco_init;
+
+ ret = devm_clk_hw_register(dev, &pll_10nm->clk_hw);
+ if (ret)
+ return ret;
+
+ snprintf(clk_name, 32, "dsi%d_pll_out_div_clk", pll_10nm->phy->id);
+ snprintf(parent, 32, "dsi%dvco_clk", pll_10nm->phy->id);
+
+ hw = devm_clk_hw_register_divider(dev, clk_name,
+ parent, CLK_SET_RATE_PARENT,
+ pll_10nm->phy->pll_base +
+ REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE,
+ 0, 2, CLK_DIVIDER_POWER_OF_TWO, NULL);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_pll_bit_clk", pll_10nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_10nm->phy->id);
+
+ /* BIT CLK: DIV_CTRL_3_0 */
+ hw = devm_clk_hw_register_divider(dev, clk_name, parent,
+ CLK_SET_RATE_PARENT,
+ pll_10nm->phy->base +
+ REG_DSI_10nm_PHY_CMN_CLK_CFG0,
+ 0, 4, CLK_DIVIDER_ONE_BASED,
+ &pll_10nm->postdiv_lock);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_phy_pll_out_byteclk", pll_10nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_10nm->phy->id);
+
+ /* DSI Byte clock = VCO_CLK / OUT_DIV / BIT_DIV / 8 */
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent,
+ CLK_SET_RATE_PARENT, 1, 8);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ provided_clocks[DSI_BYTE_PLL_CLK] = hw;
+
+ snprintf(clk_name, 32, "dsi%d_pll_by_2_bit_clk", pll_10nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_10nm->phy->id);
+
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent,
+ 0, 1, 2);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_pll_post_out_div_clk", pll_10nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_10nm->phy->id);
+
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent,
+ 0, 1, 4);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_pclk_mux", pll_10nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_10nm->phy->id);
+ snprintf(parent2, 32, "dsi%d_pll_by_2_bit_clk", pll_10nm->phy->id);
+ snprintf(parent3, 32, "dsi%d_pll_out_div_clk", pll_10nm->phy->id);
+ snprintf(parent4, 32, "dsi%d_pll_post_out_div_clk", pll_10nm->phy->id);
+
+ hw = devm_clk_hw_register_mux(dev, clk_name,
+ ((const char *[]){
+ parent, parent2, parent3, parent4
+ }), 4, 0, pll_10nm->phy->base +
+ REG_DSI_10nm_PHY_CMN_CLK_CFG1,
+ 0, 2, 0, NULL);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_phy_pll_out_dsiclk", pll_10nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pclk_mux", pll_10nm->phy->id);
+
+ /* PIX CLK DIV : DIV_CTRL_7_4*/
+ hw = devm_clk_hw_register_divider(dev, clk_name, parent,
+ 0, pll_10nm->phy->base +
+ REG_DSI_10nm_PHY_CMN_CLK_CFG0,
+ 4, 4, CLK_DIVIDER_ONE_BASED,
+ &pll_10nm->postdiv_lock);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ provided_clocks[DSI_PIXEL_PLL_CLK] = hw;
+
+ return 0;
+
+fail:
+
+ return ret;
+}
+
+static int dsi_pll_10nm_init(struct msm_dsi_phy *phy)
+{
+ struct platform_device *pdev = phy->pdev;
+ struct dsi_pll_10nm *pll_10nm;
+ int ret;
+
+ pll_10nm = devm_kzalloc(&pdev->dev, sizeof(*pll_10nm), GFP_KERNEL);
+ if (!pll_10nm)
+ return -ENOMEM;
+
+ DBG("DSI PLL%d", phy->id);
+
+ pll_10nm_list[phy->id] = pll_10nm;
+
+ spin_lock_init(&pll_10nm->postdiv_lock);
+
+ pll_10nm->phy = phy;
+
+ ret = pll_10nm_register(pll_10nm, phy->provided_clocks->hws);
+ if (ret) {
+ DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
+ return ret;
+ }
+
+ phy->vco_hw = &pll_10nm->clk_hw;
+
+ /* TODO: Remove this when we have proper display handover support */
+ msm_dsi_phy_pll_save_state(phy);
+
+ return 0;
+}
+
static int dsi_phy_hw_v3_0_is_pll_on(struct msm_dsi_phy *phy)
{
void __iomem *base = phy->base;
u8 tx_dctrl[] = { 0x00, 0x00, 0x00, 0x04, 0x01 };
void __iomem *lane_base = phy->lane_base;
- if (phy->cfg->quirks & V3_0_0_10NM_OLD_TIMINGS_QUIRK)
+ if (phy->cfg->quirks & DSI_PHY_10NM_QUIRK_OLD_TIMINGS)
tx_dctrl[3] = 0x02;
/* Strength ctrl settings */
tx_dctrl[i]);
}
- if (!(phy->cfg->quirks & V3_0_0_10NM_OLD_TIMINGS_QUIRK)) {
+ if (!(phy->cfg->quirks & DSI_PHY_10NM_QUIRK_OLD_TIMINGS)) {
/* Toggle BIT 0 to release freeze I/0 */
dsi_phy_write(lane_base + REG_DSI_10nm_PHY_LN_TX_DCTRL(3), 0x05);
dsi_phy_write(lane_base + REG_DSI_10nm_PHY_LN_TX_DCTRL(3), 0x04);
}
}
-static int dsi_10nm_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+static int dsi_10nm_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req)
{
int ret;
/* Select full-rate mode */
dsi_phy_write(base + REG_DSI_10nm_PHY_CMN_CTRL_2, 0x40);
- ret = msm_dsi_pll_set_usecase(phy->pll, phy->usecase);
+ ret = dsi_10nm_set_usecase(phy);
if (ret) {
DRM_DEV_ERROR(&phy->pdev->dev, "%s: set pll usecase failed, %d\n",
__func__, ret);
DBG("DSI%d PHY disabled", phy->id);
}
-static int dsi_10nm_phy_init(struct msm_dsi_phy *phy)
-{
- struct platform_device *pdev = phy->pdev;
-
- phy->lane_base = msm_ioremap(pdev, "dsi_phy_lane",
- "DSI_PHY_LANE");
- if (IS_ERR(phy->lane_base)) {
- DRM_DEV_ERROR(&pdev->dev, "%s: failed to map phy lane base\n",
- __func__);
- return -ENOMEM;
- }
-
- return 0;
-}
-
const struct msm_dsi_phy_cfg dsi_phy_10nm_cfgs = {
- .type = MSM_DSI_PHY_10NM,
- .src_pll_truthtable = { {false, false}, {true, false} },
+ .has_phy_lane = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_10nm_phy_enable,
.disable = dsi_10nm_phy_disable,
- .init = dsi_10nm_phy_init,
+ .pll_init = dsi_pll_10nm_init,
+ .save_pll_state = dsi_10nm_pll_save_state,
+ .restore_pll_state = dsi_10nm_pll_restore_state,
},
+ .min_pll_rate = 1000000000UL,
+ .max_pll_rate = 3500000000UL,
.io_start = { 0xae94400, 0xae96400 },
.num_dsi_phy = 2,
};
const struct msm_dsi_phy_cfg dsi_phy_10nm_8998_cfgs = {
- .type = MSM_DSI_PHY_10NM,
- .src_pll_truthtable = { {false, false}, {true, false} },
+ .has_phy_lane = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_10nm_phy_enable,
.disable = dsi_10nm_phy_disable,
- .init = dsi_10nm_phy_init,
+ .pll_init = dsi_pll_10nm_init,
+ .save_pll_state = dsi_10nm_pll_save_state,
+ .restore_pll_state = dsi_10nm_pll_restore_state,
},
+ .min_pll_rate = 1000000000UL,
+ .max_pll_rate = 3500000000UL,
.io_start = { 0xc994400, 0xc996400 },
.num_dsi_phy = 2,
- .quirks = V3_0_0_10NM_OLD_TIMINGS_QUIRK,
+ .quirks = DSI_PHY_10NM_QUIRK_OLD_TIMINGS,
};
* Copyright (c) 2016, The Linux Foundation. All rights reserved.
*/
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include <linux/delay.h>
#include "dsi_phy.h"
#define PHY_14NM_CKLN_IDX 4
+/*
+ * DSI PLL 14nm - clock diagram (eg: DSI0):
+ *
+ * dsi0n1_postdiv_clk
+ * |
+ * |
+ * +----+ | +----+
+ * dsi0vco_clk ---| n1 |--o--| /8 |-- dsi0pllbyte
+ * +----+ | +----+
+ * | dsi0n1_postdivby2_clk
+ * | +----+ |
+ * o---| /2 |--o--|\
+ * | +----+ | \ +----+
+ * | | |--| n2 |-- dsi0pll
+ * o--------------| / +----+
+ * |/
+ */
+
+#define POLL_MAX_READS 15
+#define POLL_TIMEOUT_US 1000
+
+#define VCO_REF_CLK_RATE 19200000
+#define VCO_MIN_RATE 1300000000UL
+#define VCO_MAX_RATE 2600000000UL
+
+struct dsi_pll_config {
+ u64 vco_current_rate;
+
+ u32 ssc_en; /* SSC enable/disable */
+
+ /* fixed params */
+ u32 plllock_cnt;
+ u32 ssc_center;
+ u32 ssc_adj_period;
+ u32 ssc_spread;
+ u32 ssc_freq;
+
+ /* calculated */
+ u32 dec_start;
+ u32 div_frac_start;
+ u32 ssc_period;
+ u32 ssc_step_size;
+ u32 plllock_cmp;
+ u32 pll_vco_div_ref;
+ u32 pll_vco_count;
+ u32 pll_kvco_div_ref;
+ u32 pll_kvco_count;
+};
+
+struct pll_14nm_cached_state {
+ unsigned long vco_rate;
+ u8 n2postdiv;
+ u8 n1postdiv;
+};
+
+struct dsi_pll_14nm {
+ struct clk_hw clk_hw;
+
+ struct msm_dsi_phy *phy;
+
+ /* protects REG_DSI_14nm_PHY_CMN_CLK_CFG0 register */
+ spinlock_t postdiv_lock;
+
+ struct pll_14nm_cached_state cached_state;
+
+ struct dsi_pll_14nm *slave;
+};
+
+#define to_pll_14nm(x) container_of(x, struct dsi_pll_14nm, clk_hw)
+
+/*
+ * Private struct for N1/N2 post-divider clocks. These clocks are similar to
+ * the generic clk_divider class of clocks. The only difference is that it
+ * also sets the slave DSI PLL's post-dividers if in Dual DSI mode
+ */
+struct dsi_pll_14nm_postdiv {
+ struct clk_hw hw;
+
+ /* divider params */
+ u8 shift;
+ u8 width;
+ u8 flags; /* same flags as used by clk_divider struct */
+
+ struct dsi_pll_14nm *pll;
+};
+
+#define to_pll_14nm_postdiv(_hw) container_of(_hw, struct dsi_pll_14nm_postdiv, hw)
+
+/*
+ * Global list of private DSI PLL struct pointers. We need this for Dual DSI
+ * mode, where the master PLL's clk_ops needs access the slave's private data
+ */
+static struct dsi_pll_14nm *pll_14nm_list[DSI_MAX];
+
+static bool pll_14nm_poll_for_ready(struct dsi_pll_14nm *pll_14nm,
+ u32 nb_tries, u32 timeout_us)
+{
+ bool pll_locked = false;
+ void __iomem *base = pll_14nm->phy->pll_base;
+ u32 tries, val;
+
+ tries = nb_tries;
+ while (tries--) {
+ val = dsi_phy_read(base +
+ REG_DSI_14nm_PHY_PLL_RESET_SM_READY_STATUS);
+ pll_locked = !!(val & BIT(5));
+
+ if (pll_locked)
+ break;
+
+ udelay(timeout_us);
+ }
+
+ if (!pll_locked) {
+ tries = nb_tries;
+ while (tries--) {
+ val = dsi_phy_read(base +
+ REG_DSI_14nm_PHY_PLL_RESET_SM_READY_STATUS);
+ pll_locked = !!(val & BIT(0));
+
+ if (pll_locked)
+ break;
+
+ udelay(timeout_us);
+ }
+ }
+
+ DBG("DSI PLL is %slocked", pll_locked ? "" : "*not* ");
+
+ return pll_locked;
+}
+
+static void dsi_pll_14nm_config_init(struct dsi_pll_config *pconf)
+{
+ /* fixed input */
+ pconf->plllock_cnt = 1;
+
+ /*
+ * SSC is enabled by default. We might need DT props for configuring
+ * some SSC params like PPM and center/down spread etc.
+ */
+ pconf->ssc_en = 1;
+ pconf->ssc_center = 0; /* down spread by default */
+ pconf->ssc_spread = 5; /* PPM / 1000 */
+ pconf->ssc_freq = 31500; /* default recommended */
+ pconf->ssc_adj_period = 37;
+}
+
+#define CEIL(x, y) (((x) + ((y) - 1)) / (y))
+
+static void pll_14nm_ssc_calc(struct dsi_pll_14nm *pll, struct dsi_pll_config *pconf)
+{
+ u32 period, ssc_period;
+ u32 ref, rem;
+ u64 step_size;
+
+ DBG("vco=%lld ref=%d", pconf->vco_current_rate, VCO_REF_CLK_RATE);
+
+ ssc_period = pconf->ssc_freq / 500;
+ period = (u32)VCO_REF_CLK_RATE / 1000;
+ ssc_period = CEIL(period, ssc_period);
+ ssc_period -= 1;
+ pconf->ssc_period = ssc_period;
+
+ DBG("ssc freq=%d spread=%d period=%d", pconf->ssc_freq,
+ pconf->ssc_spread, pconf->ssc_period);
+
+ step_size = (u32)pconf->vco_current_rate;
+ ref = VCO_REF_CLK_RATE;
+ ref /= 1000;
+ step_size = div_u64(step_size, ref);
+ step_size <<= 20;
+ step_size = div_u64(step_size, 1000);
+ step_size *= pconf->ssc_spread;
+ step_size = div_u64(step_size, 1000);
+ step_size *= (pconf->ssc_adj_period + 1);
+
+ rem = 0;
+ step_size = div_u64_rem(step_size, ssc_period + 1, &rem);
+ if (rem)
+ step_size++;
+
+ DBG("step_size=%lld", step_size);
+
+ step_size &= 0x0ffff; /* take lower 16 bits */
+
+ pconf->ssc_step_size = step_size;
+}
+
+static void pll_14nm_dec_frac_calc(struct dsi_pll_14nm *pll, struct dsi_pll_config *pconf)
+{
+ u64 multiplier = BIT(20);
+ u64 dec_start_multiple, dec_start, pll_comp_val;
+ u32 duration, div_frac_start;
+ u64 vco_clk_rate = pconf->vco_current_rate;
+ u64 fref = VCO_REF_CLK_RATE;
+
+ DBG("vco_clk_rate=%lld ref_clk_rate=%lld", vco_clk_rate, fref);
+
+ dec_start_multiple = div_u64(vco_clk_rate * multiplier, fref);
+ div_u64_rem(dec_start_multiple, multiplier, &div_frac_start);
+
+ dec_start = div_u64(dec_start_multiple, multiplier);
+
+ pconf->dec_start = (u32)dec_start;
+ pconf->div_frac_start = div_frac_start;
+
+ if (pconf->plllock_cnt == 0)
+ duration = 1024;
+ else if (pconf->plllock_cnt == 1)
+ duration = 256;
+ else if (pconf->plllock_cnt == 2)
+ duration = 128;
+ else
+ duration = 32;
+
+ pll_comp_val = duration * dec_start_multiple;
+ pll_comp_val = div_u64(pll_comp_val, multiplier);
+ do_div(pll_comp_val, 10);
+
+ pconf->plllock_cmp = (u32)pll_comp_val;
+}
+
+static u32 pll_14nm_kvco_slop(u32 vrate)
+{
+ u32 slop = 0;
+
+ if (vrate > VCO_MIN_RATE && vrate <= 1800000000UL)
+ slop = 600;
+ else if (vrate > 1800000000UL && vrate < 2300000000UL)
+ slop = 400;
+ else if (vrate > 2300000000UL && vrate < VCO_MAX_RATE)
+ slop = 280;
+
+ return slop;
+}
+
+static void pll_14nm_calc_vco_count(struct dsi_pll_14nm *pll, struct dsi_pll_config *pconf)
+{
+ u64 vco_clk_rate = pconf->vco_current_rate;
+ u64 fref = VCO_REF_CLK_RATE;
+ u32 vco_measure_time = 5;
+ u32 kvco_measure_time = 5;
+ u64 data;
+ u32 cnt;
+
+ data = fref * vco_measure_time;
+ do_div(data, 1000000);
+ data &= 0x03ff; /* 10 bits */
+ data -= 2;
+ pconf->pll_vco_div_ref = data;
+
+ data = div_u64(vco_clk_rate, 1000000); /* unit is Mhz */
+ data *= vco_measure_time;
+ do_div(data, 10);
+ pconf->pll_vco_count = data;
+
+ data = fref * kvco_measure_time;
+ do_div(data, 1000000);
+ data &= 0x03ff; /* 10 bits */
+ data -= 1;
+ pconf->pll_kvco_div_ref = data;
+
+ cnt = pll_14nm_kvco_slop(vco_clk_rate);
+ cnt *= 2;
+ cnt /= 100;
+ cnt *= kvco_measure_time;
+ pconf->pll_kvco_count = cnt;
+}
+
+static void pll_db_commit_ssc(struct dsi_pll_14nm *pll, struct dsi_pll_config *pconf)
+{
+ void __iomem *base = pll->phy->pll_base;
+ u8 data;
+
+ data = pconf->ssc_adj_period;
+ data &= 0x0ff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SSC_ADJ_PER1, data);
+ data = (pconf->ssc_adj_period >> 8);
+ data &= 0x03;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SSC_ADJ_PER2, data);
+
+ data = pconf->ssc_period;
+ data &= 0x0ff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SSC_PER1, data);
+ data = (pconf->ssc_period >> 8);
+ data &= 0x0ff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SSC_PER2, data);
+
+ data = pconf->ssc_step_size;
+ data &= 0x0ff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SSC_STEP_SIZE1, data);
+ data = (pconf->ssc_step_size >> 8);
+ data &= 0x0ff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SSC_STEP_SIZE2, data);
+
+ data = (pconf->ssc_center & 0x01);
+ data <<= 1;
+ data |= 0x01; /* enable */
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SSC_EN_CENTER, data);
+
+ wmb(); /* make sure register committed */
+}
+
+static void pll_db_commit_common(struct dsi_pll_14nm *pll,
+ struct dsi_pll_config *pconf)
+{
+ void __iomem *base = pll->phy->pll_base;
+ u8 data;
+
+ /* confgiure the non frequency dependent pll registers */
+ data = 0;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_SYSCLK_EN_RESET, data);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_TXCLK_EN, 1);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_RESETSM_CNTRL, 48);
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_RESETSM_CNTRL2, 4 << 3); /* bandgap_timer */
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_RESETSM_CNTRL5, 5); /* pll_wakeup_timer */
+
+ data = pconf->pll_vco_div_ref & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_VCO_DIV_REF1, data);
+ data = (pconf->pll_vco_div_ref >> 8) & 0x3;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_VCO_DIV_REF2, data);
+
+ data = pconf->pll_kvco_div_ref & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_KVCO_DIV_REF1, data);
+ data = (pconf->pll_kvco_div_ref >> 8) & 0x3;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_KVCO_DIV_REF2, data);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_MISC1, 16);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_IE_TRIM, 4);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_IP_TRIM, 4);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_CP_SET_CUR, 1 << 3 | 1);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_ICPCSET, 0 << 3 | 0);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_ICPMSET, 0 << 3 | 0);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_ICP_SET, 4 << 3 | 4);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_LPF1, 1 << 4 | 11);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_IPTAT_TRIM, 7);
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_CRCTRL, 1 << 4 | 2);
+}
+
+static void pll_14nm_software_reset(struct dsi_pll_14nm *pll_14nm)
+{
+ void __iomem *cmn_base = pll_14nm->phy->base;
+
+ /* de assert pll start and apply pll sw reset */
+
+ /* stop pll */
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_PLL_CNTRL, 0);
+
+ /* pll sw reset */
+ dsi_phy_write_udelay(cmn_base + REG_DSI_14nm_PHY_CMN_CTRL_1, 0x20, 10);
+ wmb(); /* make sure register committed */
+
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_CTRL_1, 0);
+ wmb(); /* make sure register committed */
+}
+
+static void pll_db_commit_14nm(struct dsi_pll_14nm *pll,
+ struct dsi_pll_config *pconf)
+{
+ void __iomem *base = pll->phy->pll_base;
+ void __iomem *cmn_base = pll->phy->base;
+ u8 data;
+
+ DBG("DSI%d PLL", pll->phy->id);
+
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_LDO_CNTRL, 0x3c);
+
+ pll_db_commit_common(pll, pconf);
+
+ pll_14nm_software_reset(pll);
+
+ /* Use the /2 path in Mux */
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_CLK_CFG1, 1);
+
+ data = 0xff; /* data, clk, pll normal operation */
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_CTRL_0, data);
+
+ /* configure the frequency dependent pll registers */
+ data = pconf->dec_start;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_DEC_START, data);
+
+ data = pconf->div_frac_start & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START1, data);
+ data = (pconf->div_frac_start >> 8) & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START2, data);
+ data = (pconf->div_frac_start >> 16) & 0xf;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START3, data);
+
+ data = pconf->plllock_cmp & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP1, data);
+
+ data = (pconf->plllock_cmp >> 8) & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP2, data);
+
+ data = (pconf->plllock_cmp >> 16) & 0x3;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP3, data);
+
+ data = pconf->plllock_cnt << 1 | 0 << 3; /* plllock_rng */
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP_EN, data);
+
+ data = pconf->pll_vco_count & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_VCO_COUNT1, data);
+ data = (pconf->pll_vco_count >> 8) & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_VCO_COUNT2, data);
+
+ data = pconf->pll_kvco_count & 0xff;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_KVCO_COUNT1, data);
+ data = (pconf->pll_kvco_count >> 8) & 0x3;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_KVCO_COUNT2, data);
+
+ /*
+ * High nibble configures the post divider internal to the VCO. It's
+ * fixed to divide by 1 for now.
+ *
+ * 0: divided by 1
+ * 1: divided by 2
+ * 2: divided by 4
+ * 3: divided by 8
+ */
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_LPF2_POSTDIV, 0 << 4 | 3);
+
+ if (pconf->ssc_en)
+ pll_db_commit_ssc(pll, pconf);
+
+ wmb(); /* make sure register committed */
+}
+
+/*
+ * VCO clock Callbacks
+ */
+static int dsi_pll_14nm_vco_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(hw);
+ struct dsi_pll_config conf;
+
+ DBG("DSI PLL%d rate=%lu, parent's=%lu", pll_14nm->phy->id, rate,
+ parent_rate);
+
+ dsi_pll_14nm_config_init(&conf);
+ conf.vco_current_rate = rate;
+
+ pll_14nm_dec_frac_calc(pll_14nm, &conf);
+
+ if (conf.ssc_en)
+ pll_14nm_ssc_calc(pll_14nm, &conf);
+
+ pll_14nm_calc_vco_count(pll_14nm, &conf);
+
+ /* commit the slave DSI PLL registers if we're master. Note that we
+ * don't lock the slave PLL. We just ensure that the PLL/PHY registers
+ * of the master and slave are identical
+ */
+ if (pll_14nm->phy->usecase == MSM_DSI_PHY_MASTER) {
+ struct dsi_pll_14nm *pll_14nm_slave = pll_14nm->slave;
+
+ pll_db_commit_14nm(pll_14nm_slave, &conf);
+ }
+
+ pll_db_commit_14nm(pll_14nm, &conf);
+
+ return 0;
+}
+
+static unsigned long dsi_pll_14nm_vco_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(hw);
+ void __iomem *base = pll_14nm->phy->pll_base;
+ u64 vco_rate, multiplier = BIT(20);
+ u32 div_frac_start;
+ u32 dec_start;
+ u64 ref_clk = parent_rate;
+
+ dec_start = dsi_phy_read(base + REG_DSI_14nm_PHY_PLL_DEC_START);
+ dec_start &= 0x0ff;
+
+ DBG("dec_start = %x", dec_start);
+
+ div_frac_start = (dsi_phy_read(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START3)
+ & 0xf) << 16;
+ div_frac_start |= (dsi_phy_read(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START2)
+ & 0xff) << 8;
+ div_frac_start |= dsi_phy_read(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START1)
+ & 0xff;
+
+ DBG("div_frac_start = %x", div_frac_start);
+
+ vco_rate = ref_clk * dec_start;
+
+ vco_rate += ((ref_clk * div_frac_start) / multiplier);
+
+ /*
+ * Recalculating the rate from dec_start and frac_start doesn't end up
+ * the rate we originally set. Convert the freq to KHz, round it up and
+ * convert it back to MHz.
+ */
+ vco_rate = DIV_ROUND_UP_ULL(vco_rate, 1000) * 1000;
+
+ DBG("returning vco rate = %lu", (unsigned long)vco_rate);
+
+ return (unsigned long)vco_rate;
+}
+
+static int dsi_pll_14nm_vco_prepare(struct clk_hw *hw)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(hw);
+ void __iomem *base = pll_14nm->phy->pll_base;
+ void __iomem *cmn_base = pll_14nm->phy->base;
+ bool locked;
+
+ DBG("");
+
+ if (unlikely(pll_14nm->phy->pll_on))
+ return 0;
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_VREF_CFG1, 0x10);
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_PLL_CNTRL, 1);
+
+ locked = pll_14nm_poll_for_ready(pll_14nm, POLL_MAX_READS,
+ POLL_TIMEOUT_US);
+
+ if (unlikely(!locked)) {
+ DRM_DEV_ERROR(&pll_14nm->phy->pdev->dev, "DSI PLL lock failed\n");
+ return -EINVAL;
+ }
+
+ DBG("DSI PLL lock success");
+ pll_14nm->phy->pll_on = true;
+
+ return 0;
+}
+
+static void dsi_pll_14nm_vco_unprepare(struct clk_hw *hw)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(hw);
+ void __iomem *cmn_base = pll_14nm->phy->base;
+
+ DBG("");
+
+ if (unlikely(!pll_14nm->phy->pll_on))
+ return;
+
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_PLL_CNTRL, 0);
+
+ pll_14nm->phy->pll_on = false;
+}
+
+static long dsi_pll_14nm_clk_round_rate(struct clk_hw *hw,
+ unsigned long rate, unsigned long *parent_rate)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(hw);
+
+ if (rate < pll_14nm->phy->cfg->min_pll_rate)
+ return pll_14nm->phy->cfg->min_pll_rate;
+ else if (rate > pll_14nm->phy->cfg->max_pll_rate)
+ return pll_14nm->phy->cfg->max_pll_rate;
+ else
+ return rate;
+}
+
+static const struct clk_ops clk_ops_dsi_pll_14nm_vco = {
+ .round_rate = dsi_pll_14nm_clk_round_rate,
+ .set_rate = dsi_pll_14nm_vco_set_rate,
+ .recalc_rate = dsi_pll_14nm_vco_recalc_rate,
+ .prepare = dsi_pll_14nm_vco_prepare,
+ .unprepare = dsi_pll_14nm_vco_unprepare,
+};
+
+/*
+ * N1 and N2 post-divider clock callbacks
+ */
+#define div_mask(width) ((1 << (width)) - 1)
+static unsigned long dsi_pll_14nm_postdiv_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_14nm_postdiv *postdiv = to_pll_14nm_postdiv(hw);
+ struct dsi_pll_14nm *pll_14nm = postdiv->pll;
+ void __iomem *base = pll_14nm->phy->base;
+ u8 shift = postdiv->shift;
+ u8 width = postdiv->width;
+ u32 val;
+
+ DBG("DSI%d PLL parent rate=%lu", pll_14nm->phy->id, parent_rate);
+
+ val = dsi_phy_read(base + REG_DSI_14nm_PHY_CMN_CLK_CFG0) >> shift;
+ val &= div_mask(width);
+
+ return divider_recalc_rate(hw, parent_rate, val, NULL,
+ postdiv->flags, width);
+}
+
+static long dsi_pll_14nm_postdiv_round_rate(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long *prate)
+{
+ struct dsi_pll_14nm_postdiv *postdiv = to_pll_14nm_postdiv(hw);
+ struct dsi_pll_14nm *pll_14nm = postdiv->pll;
+
+ DBG("DSI%d PLL parent rate=%lu", pll_14nm->phy->id, rate);
+
+ return divider_round_rate(hw, rate, prate, NULL,
+ postdiv->width,
+ postdiv->flags);
+}
+
+static int dsi_pll_14nm_postdiv_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_14nm_postdiv *postdiv = to_pll_14nm_postdiv(hw);
+ struct dsi_pll_14nm *pll_14nm = postdiv->pll;
+ void __iomem *base = pll_14nm->phy->base;
+ spinlock_t *lock = &pll_14nm->postdiv_lock;
+ u8 shift = postdiv->shift;
+ u8 width = postdiv->width;
+ unsigned int value;
+ unsigned long flags = 0;
+ u32 val;
+
+ DBG("DSI%d PLL parent rate=%lu parent rate %lu", pll_14nm->phy->id, rate,
+ parent_rate);
+
+ value = divider_get_val(rate, parent_rate, NULL, postdiv->width,
+ postdiv->flags);
+
+ spin_lock_irqsave(lock, flags);
+
+ val = dsi_phy_read(base + REG_DSI_14nm_PHY_CMN_CLK_CFG0);
+ val &= ~(div_mask(width) << shift);
+
+ val |= value << shift;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, val);
+
+ /* If we're master in dual DSI mode, then the slave PLL's post-dividers
+ * follow the master's post dividers
+ */
+ if (pll_14nm->phy->usecase == MSM_DSI_PHY_MASTER) {
+ struct dsi_pll_14nm *pll_14nm_slave = pll_14nm->slave;
+ void __iomem *slave_base = pll_14nm_slave->phy->base;
+
+ dsi_phy_write(slave_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, val);
+ }
+
+ spin_unlock_irqrestore(lock, flags);
+
+ return 0;
+}
+
+static const struct clk_ops clk_ops_dsi_pll_14nm_postdiv = {
+ .recalc_rate = dsi_pll_14nm_postdiv_recalc_rate,
+ .round_rate = dsi_pll_14nm_postdiv_round_rate,
+ .set_rate = dsi_pll_14nm_postdiv_set_rate,
+};
+
+/*
+ * PLL Callbacks
+ */
+
+static void dsi_14nm_pll_save_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(phy->vco_hw);
+ struct pll_14nm_cached_state *cached_state = &pll_14nm->cached_state;
+ void __iomem *cmn_base = pll_14nm->phy->base;
+ u32 data;
+
+ data = dsi_phy_read(cmn_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0);
+
+ cached_state->n1postdiv = data & 0xf;
+ cached_state->n2postdiv = (data >> 4) & 0xf;
+
+ DBG("DSI%d PLL save state %x %x", pll_14nm->phy->id,
+ cached_state->n1postdiv, cached_state->n2postdiv);
+
+ cached_state->vco_rate = clk_hw_get_rate(phy->vco_hw);
+}
+
+static int dsi_14nm_pll_restore_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(phy->vco_hw);
+ struct pll_14nm_cached_state *cached_state = &pll_14nm->cached_state;
+ void __iomem *cmn_base = pll_14nm->phy->base;
+ u32 data;
+ int ret;
+
+ ret = dsi_pll_14nm_vco_set_rate(phy->vco_hw,
+ cached_state->vco_rate, 0);
+ if (ret) {
+ DRM_DEV_ERROR(&pll_14nm->phy->pdev->dev,
+ "restore vco rate failed. ret=%d\n", ret);
+ return ret;
+ }
+
+ data = cached_state->n1postdiv | (cached_state->n2postdiv << 4);
+
+ DBG("DSI%d PLL restore state %x %x", pll_14nm->phy->id,
+ cached_state->n1postdiv, cached_state->n2postdiv);
+
+ dsi_phy_write(cmn_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, data);
+
+ /* also restore post-dividers for slave DSI PLL */
+ if (phy->usecase == MSM_DSI_PHY_MASTER) {
+ struct dsi_pll_14nm *pll_14nm_slave = pll_14nm->slave;
+ void __iomem *slave_base = pll_14nm_slave->phy->base;
+
+ dsi_phy_write(slave_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, data);
+ }
+
+ return 0;
+}
+
+static int dsi_14nm_set_usecase(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_14nm *pll_14nm = to_pll_14nm(phy->vco_hw);
+ void __iomem *base = phy->pll_base;
+ u32 clkbuflr_en, bandgap = 0;
+
+ switch (phy->usecase) {
+ case MSM_DSI_PHY_STANDALONE:
+ clkbuflr_en = 0x1;
+ break;
+ case MSM_DSI_PHY_MASTER:
+ clkbuflr_en = 0x3;
+ pll_14nm->slave = pll_14nm_list[(pll_14nm->phy->id + 1) % DSI_MAX];
+ break;
+ case MSM_DSI_PHY_SLAVE:
+ clkbuflr_en = 0x0;
+ bandgap = 0x3;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_CLKBUFLR_EN, clkbuflr_en);
+ if (bandgap)
+ dsi_phy_write(base + REG_DSI_14nm_PHY_PLL_PLL_BANDGAP, bandgap);
+
+ return 0;
+}
+
+static struct clk_hw *pll_14nm_postdiv_register(struct dsi_pll_14nm *pll_14nm,
+ const char *name,
+ const char *parent_name,
+ unsigned long flags,
+ u8 shift)
+{
+ struct dsi_pll_14nm_postdiv *pll_postdiv;
+ struct device *dev = &pll_14nm->phy->pdev->dev;
+ struct clk_init_data postdiv_init = {
+ .parent_names = (const char *[]) { parent_name },
+ .num_parents = 1,
+ .name = name,
+ .flags = flags,
+ .ops = &clk_ops_dsi_pll_14nm_postdiv,
+ };
+ int ret;
+
+ pll_postdiv = devm_kzalloc(dev, sizeof(*pll_postdiv), GFP_KERNEL);
+ if (!pll_postdiv)
+ return ERR_PTR(-ENOMEM);
+
+ pll_postdiv->pll = pll_14nm;
+ pll_postdiv->shift = shift;
+ /* both N1 and N2 postdividers are 4 bits wide */
+ pll_postdiv->width = 4;
+ /* range of each divider is from 1 to 15 */
+ pll_postdiv->flags = CLK_DIVIDER_ONE_BASED;
+ pll_postdiv->hw.init = &postdiv_init;
+
+ ret = devm_clk_hw_register(dev, &pll_postdiv->hw);
+ if (ret)
+ return ERR_PTR(ret);
+
+ return &pll_postdiv->hw;
+}
+
+static int pll_14nm_register(struct dsi_pll_14nm *pll_14nm, struct clk_hw **provided_clocks)
+{
+ char clk_name[32], parent[32], vco_name[32];
+ struct clk_init_data vco_init = {
+ .parent_names = (const char *[]){ "xo" },
+ .num_parents = 1,
+ .name = vco_name,
+ .flags = CLK_IGNORE_UNUSED,
+ .ops = &clk_ops_dsi_pll_14nm_vco,
+ };
+ struct device *dev = &pll_14nm->phy->pdev->dev;
+ struct clk_hw *hw;
+ int ret;
+
+ DBG("DSI%d", pll_14nm->phy->id);
+
+ snprintf(vco_name, 32, "dsi%dvco_clk", pll_14nm->phy->id);
+ pll_14nm->clk_hw.init = &vco_init;
+
+ ret = devm_clk_hw_register(dev, &pll_14nm->clk_hw);
+ if (ret)
+ return ret;
+
+ snprintf(clk_name, 32, "dsi%dn1_postdiv_clk", pll_14nm->phy->id);
+ snprintf(parent, 32, "dsi%dvco_clk", pll_14nm->phy->id);
+
+ /* N1 postdiv, bits 0-3 in REG_DSI_14nm_PHY_CMN_CLK_CFG0 */
+ hw = pll_14nm_postdiv_register(pll_14nm, clk_name, parent,
+ CLK_SET_RATE_PARENT, 0);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+
+ snprintf(clk_name, 32, "dsi%dpllbyte", pll_14nm->phy->id);
+ snprintf(parent, 32, "dsi%dn1_postdiv_clk", pll_14nm->phy->id);
+
+ /* DSI Byte clock = VCO_CLK / N1 / 8 */
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent,
+ CLK_SET_RATE_PARENT, 1, 8);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+
+ provided_clocks[DSI_BYTE_PLL_CLK] = hw;
+
+ snprintf(clk_name, 32, "dsi%dn1_postdivby2_clk", pll_14nm->phy->id);
+ snprintf(parent, 32, "dsi%dn1_postdiv_clk", pll_14nm->phy->id);
+
+ /*
+ * Skip the mux for now, force DSICLK_SEL to 1, Add a /2 divider
+ * on the way. Don't let it set parent.
+ */
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent, 0, 1, 2);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+
+ snprintf(clk_name, 32, "dsi%dpll", pll_14nm->phy->id);
+ snprintf(parent, 32, "dsi%dn1_postdivby2_clk", pll_14nm->phy->id);
+
+ /* DSI pixel clock = VCO_CLK / N1 / 2 / N2
+ * This is the output of N2 post-divider, bits 4-7 in
+ * REG_DSI_14nm_PHY_CMN_CLK_CFG0. Don't let it set parent.
+ */
+ hw = pll_14nm_postdiv_register(pll_14nm, clk_name, parent, 0, 4);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+
+ provided_clocks[DSI_PIXEL_PLL_CLK] = hw;
+
+ return 0;
+}
+
+static int dsi_pll_14nm_init(struct msm_dsi_phy *phy)
+{
+ struct platform_device *pdev = phy->pdev;
+ struct dsi_pll_14nm *pll_14nm;
+ int ret;
+
+ if (!pdev)
+ return -ENODEV;
+
+ pll_14nm = devm_kzalloc(&pdev->dev, sizeof(*pll_14nm), GFP_KERNEL);
+ if (!pll_14nm)
+ return -ENOMEM;
+
+ DBG("PLL%d", phy->id);
+
+ pll_14nm_list[phy->id] = pll_14nm;
+
+ spin_lock_init(&pll_14nm->postdiv_lock);
+
+ pll_14nm->phy = phy;
+
+ ret = pll_14nm_register(pll_14nm, phy->provided_clocks->hws);
+ if (ret) {
+ DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
+ return ret;
+ }
+
+ phy->vco_hw = &pll_14nm->clk_hw;
+
+ return 0;
+}
+
static void dsi_14nm_dphy_set_timing(struct msm_dsi_phy *phy,
struct msm_dsi_dphy_timing *timing,
int lane_idx)
DSI_14nm_PHY_LN_TIMING_CTRL_11_TRIG3_CMD(0xa0));
}
-static int dsi_14nm_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+static int dsi_14nm_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req)
{
struct msm_dsi_dphy_timing *timing = &phy->timing;
int ret;
void __iomem *base = phy->base;
void __iomem *lane_base = phy->lane_base;
+ u32 glbl_test_ctrl;
if (msm_dsi_dphy_timing_calc_v2(timing, clk_req)) {
DRM_DEV_ERROR(&phy->pdev->dev,
udelay(100);
dsi_phy_write(base + REG_DSI_14nm_PHY_CMN_CTRL_1, 0x00);
- msm_dsi_phy_set_src_pll(phy, src_pll_id,
- REG_DSI_14nm_PHY_CMN_GLBL_TEST_CTRL,
- DSI_14nm_PHY_CMN_GLBL_TEST_CTRL_BITCLK_HS_SEL);
-
- ret = msm_dsi_pll_set_usecase(phy->pll, phy->usecase);
+ glbl_test_ctrl = dsi_phy_read(base + REG_DSI_14nm_PHY_CMN_GLBL_TEST_CTRL);
+ if (phy->id == DSI_1 && phy->usecase == MSM_DSI_PHY_SLAVE)
+ glbl_test_ctrl |= DSI_14nm_PHY_CMN_GLBL_TEST_CTRL_BITCLK_HS_SEL;
+ else
+ glbl_test_ctrl &= ~DSI_14nm_PHY_CMN_GLBL_TEST_CTRL_BITCLK_HS_SEL;
+ dsi_phy_write(base + REG_DSI_14nm_PHY_CMN_GLBL_TEST_CTRL, glbl_test_ctrl);
+ ret = dsi_14nm_set_usecase(phy);
if (ret) {
DRM_DEV_ERROR(&phy->pdev->dev, "%s: set pll usecase failed, %d\n",
__func__, ret);
wmb();
}
-static int dsi_14nm_phy_init(struct msm_dsi_phy *phy)
-{
- struct platform_device *pdev = phy->pdev;
-
- phy->lane_base = msm_ioremap(pdev, "dsi_phy_lane",
- "DSI_PHY_LANE");
- if (IS_ERR(phy->lane_base)) {
- DRM_DEV_ERROR(&pdev->dev, "%s: failed to map phy lane base\n",
- __func__);
- return -ENOMEM;
- }
-
- return 0;
-}
-
const struct msm_dsi_phy_cfg dsi_phy_14nm_cfgs = {
- .type = MSM_DSI_PHY_14NM,
- .src_pll_truthtable = { {false, false}, {true, false} },
+ .has_phy_lane = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_14nm_phy_enable,
.disable = dsi_14nm_phy_disable,
- .init = dsi_14nm_phy_init,
+ .pll_init = dsi_pll_14nm_init,
+ .save_pll_state = dsi_14nm_pll_save_state,
+ .restore_pll_state = dsi_14nm_pll_restore_state,
},
+ .min_pll_rate = VCO_MIN_RATE,
+ .max_pll_rate = VCO_MAX_RATE,
.io_start = { 0x994400, 0x996400 },
.num_dsi_phy = 2,
};
const struct msm_dsi_phy_cfg dsi_phy_14nm_660_cfgs = {
- .type = MSM_DSI_PHY_14NM,
- .src_pll_truthtable = { {false, false}, {true, false} },
+ .has_phy_lane = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_14nm_phy_enable,
.disable = dsi_14nm_phy_disable,
- .init = dsi_14nm_phy_init,
+ .pll_init = dsi_pll_14nm_init,
+ .save_pll_state = dsi_14nm_pll_save_state,
+ .restore_pll_state = dsi_14nm_pll_restore_state,
},
+ .min_pll_rate = VCO_MIN_RATE,
+ .max_pll_rate = VCO_MAX_RATE,
.io_start = { 0xc994400, 0xc996000 },
.num_dsi_phy = 2,
};
dsi_phy_write(base + REG_DSI_20nm_PHY_REGULATOR_CTRL_0, 0x03);
}
-static int dsi_20nm_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+static int dsi_20nm_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req)
{
struct msm_dsi_dphy_timing *timing = &phy->timing;
int i;
void __iomem *base = phy->base;
u32 cfg_4[4] = {0x20, 0x40, 0x20, 0x00};
+ u32 val;
DBG("");
dsi_phy_write(base + REG_DSI_20nm_PHY_STRENGTH_0, 0xff);
- msm_dsi_phy_set_src_pll(phy, src_pll_id,
- REG_DSI_20nm_PHY_GLBL_TEST_CTRL,
- DSI_20nm_PHY_GLBL_TEST_CTRL_BITCLK_HS_SEL);
+ val = dsi_phy_read(base + REG_DSI_20nm_PHY_GLBL_TEST_CTRL);
+ if (phy->id == DSI_1 && phy->usecase == MSM_DSI_PHY_STANDALONE)
+ val |= DSI_20nm_PHY_GLBL_TEST_CTRL_BITCLK_HS_SEL;
+ else
+ val &= ~DSI_20nm_PHY_GLBL_TEST_CTRL_BITCLK_HS_SEL;
+ dsi_phy_write(base + REG_DSI_20nm_PHY_GLBL_TEST_CTRL, val);
for (i = 0; i < 4; i++) {
dsi_phy_write(base + REG_DSI_20nm_PHY_LN_CFG_3(i),
}
const struct msm_dsi_phy_cfg dsi_phy_20nm_cfgs = {
- .type = MSM_DSI_PHY_20NM,
- .src_pll_truthtable = { {false, true}, {false, true} },
+ .has_phy_regulator = true,
.reg_cfg = {
.num = 2,
.regs = {
.ops = {
.enable = dsi_20nm_phy_enable,
.disable = dsi_20nm_phy_disable,
- .init = msm_dsi_phy_init_common,
},
.io_start = { 0xfd998500, 0xfd9a0500 },
.num_dsi_phy = 2,
* Copyright (c) 2015, The Linux Foundation. All rights reserved.
*/
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+
#include "dsi_phy.h"
#include "dsi.xml.h"
+/*
+ * DSI PLL 28nm - clock diagram (eg: DSI0):
+ *
+ * dsi0analog_postdiv_clk
+ * | dsi0indirect_path_div2_clk
+ * | |
+ * +------+ | +----+ | |\ dsi0byte_mux
+ * dsi0vco_clk --o--| DIV1 |--o--| /2 |--o--| \ |
+ * | +------+ +----+ | m| | +----+
+ * | | u|--o--| /4 |-- dsi0pllbyte
+ * | | x| +----+
+ * o--------------------------| /
+ * | |/
+ * | +------+
+ * o----------| DIV3 |------------------------- dsi0pll
+ * +------+
+ */
+
+#define POLL_MAX_READS 10
+#define POLL_TIMEOUT_US 50
+
+#define VCO_REF_CLK_RATE 19200000
+#define VCO_MIN_RATE 350000000
+#define VCO_MAX_RATE 750000000
+
+/* v2.0.0 28nm LP implementation */
+#define DSI_PHY_28NM_QUIRK_PHY_LP BIT(0)
+
+#define LPFR_LUT_SIZE 10
+struct lpfr_cfg {
+ unsigned long vco_rate;
+ u32 resistance;
+};
+
+/* Loop filter resistance: */
+static const struct lpfr_cfg lpfr_lut[LPFR_LUT_SIZE] = {
+ { 479500000, 8 },
+ { 480000000, 11 },
+ { 575500000, 8 },
+ { 576000000, 12 },
+ { 610500000, 8 },
+ { 659500000, 9 },
+ { 671500000, 10 },
+ { 672000000, 14 },
+ { 708500000, 10 },
+ { 750000000, 11 },
+};
+
+struct pll_28nm_cached_state {
+ unsigned long vco_rate;
+ u8 postdiv3;
+ u8 postdiv1;
+ u8 byte_mux;
+};
+
+struct dsi_pll_28nm {
+ struct clk_hw clk_hw;
+
+ struct msm_dsi_phy *phy;
+
+ struct pll_28nm_cached_state cached_state;
+};
+
+#define to_pll_28nm(x) container_of(x, struct dsi_pll_28nm, clk_hw)
+
+static bool pll_28nm_poll_for_ready(struct dsi_pll_28nm *pll_28nm,
+ u32 nb_tries, u32 timeout_us)
+{
+ bool pll_locked = false;
+ u32 val;
+
+ while (nb_tries--) {
+ val = dsi_phy_read(pll_28nm->phy->pll_base + REG_DSI_28nm_PHY_PLL_STATUS);
+ pll_locked = !!(val & DSI_28nm_PHY_PLL_STATUS_PLL_RDY);
+
+ if (pll_locked)
+ break;
+
+ udelay(timeout_us);
+ }
+ DBG("DSI PLL is %slocked", pll_locked ? "" : "*not* ");
+
+ return pll_locked;
+}
+
+static void pll_28nm_software_reset(struct dsi_pll_28nm *pll_28nm)
+{
+ void __iomem *base = pll_28nm->phy->pll_base;
+
+ /*
+ * Add HW recommended delays after toggling the software
+ * reset bit off and back on.
+ */
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_TEST_CFG,
+ DSI_28nm_PHY_PLL_TEST_CFG_PLL_SW_RESET, 1);
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_TEST_CFG, 0x00, 1);
+}
+
+/*
+ * Clock Callbacks
+ */
+static int dsi_pll_28nm_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+ struct device *dev = &pll_28nm->phy->pdev->dev;
+ void __iomem *base = pll_28nm->phy->pll_base;
+ unsigned long div_fbx1000, gen_vco_clk;
+ u32 refclk_cfg, frac_n_mode, frac_n_value;
+ u32 sdm_cfg0, sdm_cfg1, sdm_cfg2, sdm_cfg3;
+ u32 cal_cfg10, cal_cfg11;
+ u32 rem;
+ int i;
+
+ VERB("rate=%lu, parent's=%lu", rate, parent_rate);
+
+ /* Force postdiv2 to be div-4 */
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_POSTDIV2_CFG, 3);
+
+ /* Configure the Loop filter resistance */
+ for (i = 0; i < LPFR_LUT_SIZE; i++)
+ if (rate <= lpfr_lut[i].vco_rate)
+ break;
+ if (i == LPFR_LUT_SIZE) {
+ DRM_DEV_ERROR(dev, "unable to get loop filter resistance. vco=%lu\n",
+ rate);
+ return -EINVAL;
+ }
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_LPFR_CFG, lpfr_lut[i].resistance);
+
+ /* Loop filter capacitance values : c1 and c2 */
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_LPFC1_CFG, 0x70);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_LPFC2_CFG, 0x15);
+
+ rem = rate % VCO_REF_CLK_RATE;
+ if (rem) {
+ refclk_cfg = DSI_28nm_PHY_PLL_REFCLK_CFG_DBLR;
+ frac_n_mode = 1;
+ div_fbx1000 = rate / (VCO_REF_CLK_RATE / 500);
+ gen_vco_clk = div_fbx1000 * (VCO_REF_CLK_RATE / 500);
+ } else {
+ refclk_cfg = 0x0;
+ frac_n_mode = 0;
+ div_fbx1000 = rate / (VCO_REF_CLK_RATE / 1000);
+ gen_vco_clk = div_fbx1000 * (VCO_REF_CLK_RATE / 1000);
+ }
+
+ DBG("refclk_cfg = %d", refclk_cfg);
+
+ rem = div_fbx1000 % 1000;
+ frac_n_value = (rem << 16) / 1000;
+
+ DBG("div_fb = %lu", div_fbx1000);
+ DBG("frac_n_value = %d", frac_n_value);
+
+ DBG("Generated VCO Clock: %lu", gen_vco_clk);
+ rem = 0;
+ sdm_cfg1 = dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG1);
+ sdm_cfg1 &= ~DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET__MASK;
+ if (frac_n_mode) {
+ sdm_cfg0 = 0x0;
+ sdm_cfg0 |= DSI_28nm_PHY_PLL_SDM_CFG0_BYP_DIV(0);
+ sdm_cfg1 |= DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET(
+ (u32)(((div_fbx1000 / 1000) & 0x3f) - 1));
+ sdm_cfg3 = frac_n_value >> 8;
+ sdm_cfg2 = frac_n_value & 0xff;
+ } else {
+ sdm_cfg0 = DSI_28nm_PHY_PLL_SDM_CFG0_BYP;
+ sdm_cfg0 |= DSI_28nm_PHY_PLL_SDM_CFG0_BYP_DIV(
+ (u32)(((div_fbx1000 / 1000) & 0x3f) - 1));
+ sdm_cfg1 |= DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET(0);
+ sdm_cfg2 = 0;
+ sdm_cfg3 = 0;
+ }
+
+ DBG("sdm_cfg0=%d", sdm_cfg0);
+ DBG("sdm_cfg1=%d", sdm_cfg1);
+ DBG("sdm_cfg2=%d", sdm_cfg2);
+ DBG("sdm_cfg3=%d", sdm_cfg3);
+
+ cal_cfg11 = (u32)(gen_vco_clk / (256 * 1000000));
+ cal_cfg10 = (u32)((gen_vco_clk % (256 * 1000000)) / 1000000);
+ DBG("cal_cfg10=%d, cal_cfg11=%d", cal_cfg10, cal_cfg11);
+
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CHGPUMP_CFG, 0x02);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG3, 0x2b);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG4, 0x06);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x0d);
+
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG1, sdm_cfg1);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG2,
+ DSI_28nm_PHY_PLL_SDM_CFG2_FREQ_SEED_7_0(sdm_cfg2));
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG3,
+ DSI_28nm_PHY_PLL_SDM_CFG3_FREQ_SEED_15_8(sdm_cfg3));
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG4, 0x00);
+
+ /* Add hardware recommended delay for correct PLL configuration */
+ if (pll_28nm->phy->cfg->quirks & DSI_PHY_28NM_QUIRK_PHY_LP)
+ udelay(1000);
+ else
+ udelay(1);
+
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_REFCLK_CFG, refclk_cfg);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_PWRGEN_CFG, 0x00);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_VCOLPF_CFG, 0x31);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG0, sdm_cfg0);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG0, 0x12);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG6, 0x30);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG7, 0x00);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG8, 0x60);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG9, 0x00);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG10, cal_cfg10 & 0xff);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG11, cal_cfg11 & 0xff);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_EFUSE_CFG, 0x20);
+
+ return 0;
+}
+
+static int dsi_pll_28nm_clk_is_enabled(struct clk_hw *hw)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+
+ return pll_28nm_poll_for_ready(pll_28nm, POLL_MAX_READS,
+ POLL_TIMEOUT_US);
+}
+
+static unsigned long dsi_pll_28nm_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+ void __iomem *base = pll_28nm->phy->pll_base;
+ u32 sdm0, doubler, sdm_byp_div;
+ u32 sdm_dc_off, sdm_freq_seed, sdm2, sdm3;
+ u32 ref_clk = VCO_REF_CLK_RATE;
+ unsigned long vco_rate;
+
+ VERB("parent_rate=%lu", parent_rate);
+
+ /* Check to see if the ref clk doubler is enabled */
+ doubler = dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_REFCLK_CFG) &
+ DSI_28nm_PHY_PLL_REFCLK_CFG_DBLR;
+ ref_clk += (doubler * VCO_REF_CLK_RATE);
+
+ /* see if it is integer mode or sdm mode */
+ sdm0 = dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG0);
+ if (sdm0 & DSI_28nm_PHY_PLL_SDM_CFG0_BYP) {
+ /* integer mode */
+ sdm_byp_div = FIELD(
+ dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG0),
+ DSI_28nm_PHY_PLL_SDM_CFG0_BYP_DIV) + 1;
+ vco_rate = ref_clk * sdm_byp_div;
+ } else {
+ /* sdm mode */
+ sdm_dc_off = FIELD(
+ dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG1),
+ DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET);
+ DBG("sdm_dc_off = %d", sdm_dc_off);
+ sdm2 = FIELD(dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG2),
+ DSI_28nm_PHY_PLL_SDM_CFG2_FREQ_SEED_7_0);
+ sdm3 = FIELD(dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG3),
+ DSI_28nm_PHY_PLL_SDM_CFG3_FREQ_SEED_15_8);
+ sdm_freq_seed = (sdm3 << 8) | sdm2;
+ DBG("sdm_freq_seed = %d", sdm_freq_seed);
+
+ vco_rate = (ref_clk * (sdm_dc_off + 1)) +
+ mult_frac(ref_clk, sdm_freq_seed, BIT(16));
+ DBG("vco rate = %lu", vco_rate);
+ }
+
+ DBG("returning vco rate = %lu", vco_rate);
+
+ return vco_rate;
+}
+
+static int _dsi_pll_28nm_vco_prepare_hpm(struct dsi_pll_28nm *pll_28nm)
+{
+ struct device *dev = &pll_28nm->phy->pdev->dev;
+ void __iomem *base = pll_28nm->phy->pll_base;
+ u32 max_reads = 5, timeout_us = 100;
+ bool locked;
+ u32 val;
+ int i;
+
+ DBG("id=%d", pll_28nm->phy->id);
+
+ pll_28nm_software_reset(pll_28nm);
+
+ /*
+ * PLL power up sequence.
+ * Add necessary delays recommended by hardware.
+ */
+ val = DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 1);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRGEN_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 200);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_ENABLE;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 600);
+
+ for (i = 0; i < 2; i++) {
+ /* DSI Uniphy lock detect setting */
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2,
+ 0x0c, 100);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x0d);
+
+ /* poll for PLL ready status */
+ locked = pll_28nm_poll_for_ready(pll_28nm,
+ max_reads, timeout_us);
+ if (locked)
+ break;
+
+ pll_28nm_software_reset(pll_28nm);
+
+ /*
+ * PLL power up sequence.
+ * Add necessary delays recommended by hardware.
+ */
+ val = DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 1);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRGEN_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 200);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 250);
+
+ val &= ~DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 200);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_ENABLE;
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 600);
+ }
+
+ if (unlikely(!locked))
+ DRM_DEV_ERROR(dev, "DSI PLL lock failed\n");
+ else
+ DBG("DSI PLL Lock success");
+
+ return locked ? 0 : -EINVAL;
+}
+
+static int dsi_pll_28nm_vco_prepare_hpm(struct clk_hw *hw)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+ int i, ret;
+
+ if (unlikely(pll_28nm->phy->pll_on))
+ return 0;
+
+ for (i = 0; i < 3; i++) {
+ ret = _dsi_pll_28nm_vco_prepare_hpm(pll_28nm);
+ if (!ret) {
+ pll_28nm->phy->pll_on = true;
+ return 0;
+ }
+ }
+
+ return ret;
+}
+
+static int dsi_pll_28nm_vco_prepare_lp(struct clk_hw *hw)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+ struct device *dev = &pll_28nm->phy->pdev->dev;
+ void __iomem *base = pll_28nm->phy->pll_base;
+ bool locked;
+ u32 max_reads = 10, timeout_us = 50;
+ u32 val;
+
+ DBG("id=%d", pll_28nm->phy->id);
+
+ if (unlikely(pll_28nm->phy->pll_on))
+ return 0;
+
+ pll_28nm_software_reset(pll_28nm);
+
+ /*
+ * PLL power up sequence.
+ * Add necessary delays recommended by hardware.
+ */
+ dsi_phy_write_ndelay(base + REG_DSI_28nm_PHY_PLL_CAL_CFG1, 0x34, 500);
+
+ val = DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRDN_B;
+ dsi_phy_write_ndelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRGEN_PWRDN_B;
+ dsi_phy_write_ndelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
+
+ val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B |
+ DSI_28nm_PHY_PLL_GLB_CFG_PLL_ENABLE;
+ dsi_phy_write_ndelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
+
+ /* DSI PLL toggle lock detect setting */
+ dsi_phy_write_ndelay(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x04, 500);
+ dsi_phy_write_udelay(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x05, 512);
+
+ locked = pll_28nm_poll_for_ready(pll_28nm, max_reads, timeout_us);
+
+ if (unlikely(!locked)) {
+ DRM_DEV_ERROR(dev, "DSI PLL lock failed\n");
+ return -EINVAL;
+ }
+
+ DBG("DSI PLL lock success");
+ pll_28nm->phy->pll_on = true;
+
+ return 0;
+}
+
+static void dsi_pll_28nm_vco_unprepare(struct clk_hw *hw)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+
+ DBG("id=%d", pll_28nm->phy->id);
+
+ if (unlikely(!pll_28nm->phy->pll_on))
+ return;
+
+ dsi_phy_write(pll_28nm->phy->pll_base + REG_DSI_28nm_PHY_PLL_GLB_CFG, 0x00);
+
+ pll_28nm->phy->pll_on = false;
+}
+
+static long dsi_pll_28nm_clk_round_rate(struct clk_hw *hw,
+ unsigned long rate, unsigned long *parent_rate)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+
+ if (rate < pll_28nm->phy->cfg->min_pll_rate)
+ return pll_28nm->phy->cfg->min_pll_rate;
+ else if (rate > pll_28nm->phy->cfg->max_pll_rate)
+ return pll_28nm->phy->cfg->max_pll_rate;
+ else
+ return rate;
+}
+
+static const struct clk_ops clk_ops_dsi_pll_28nm_vco_hpm = {
+ .round_rate = dsi_pll_28nm_clk_round_rate,
+ .set_rate = dsi_pll_28nm_clk_set_rate,
+ .recalc_rate = dsi_pll_28nm_clk_recalc_rate,
+ .prepare = dsi_pll_28nm_vco_prepare_hpm,
+ .unprepare = dsi_pll_28nm_vco_unprepare,
+ .is_enabled = dsi_pll_28nm_clk_is_enabled,
+};
+
+static const struct clk_ops clk_ops_dsi_pll_28nm_vco_lp = {
+ .round_rate = dsi_pll_28nm_clk_round_rate,
+ .set_rate = dsi_pll_28nm_clk_set_rate,
+ .recalc_rate = dsi_pll_28nm_clk_recalc_rate,
+ .prepare = dsi_pll_28nm_vco_prepare_lp,
+ .unprepare = dsi_pll_28nm_vco_unprepare,
+ .is_enabled = dsi_pll_28nm_clk_is_enabled,
+};
+
+/*
+ * PLL Callbacks
+ */
+
+static void dsi_28nm_pll_save_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(phy->vco_hw);
+ struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
+ void __iomem *base = pll_28nm->phy->pll_base;
+
+ cached_state->postdiv3 =
+ dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_POSTDIV3_CFG);
+ cached_state->postdiv1 =
+ dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_POSTDIV1_CFG);
+ cached_state->byte_mux = dsi_phy_read(base + REG_DSI_28nm_PHY_PLL_VREG_CFG);
+ if (dsi_pll_28nm_clk_is_enabled(phy->vco_hw))
+ cached_state->vco_rate = clk_hw_get_rate(phy->vco_hw);
+ else
+ cached_state->vco_rate = 0;
+}
+
+static int dsi_28nm_pll_restore_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(phy->vco_hw);
+ struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
+ void __iomem *base = pll_28nm->phy->pll_base;
+ int ret;
+
+ ret = dsi_pll_28nm_clk_set_rate(phy->vco_hw,
+ cached_state->vco_rate, 0);
+ if (ret) {
+ DRM_DEV_ERROR(&pll_28nm->phy->pdev->dev,
+ "restore vco rate failed. ret=%d\n", ret);
+ return ret;
+ }
+
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_POSTDIV3_CFG,
+ cached_state->postdiv3);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_POSTDIV1_CFG,
+ cached_state->postdiv1);
+ dsi_phy_write(base + REG_DSI_28nm_PHY_PLL_VREG_CFG,
+ cached_state->byte_mux);
+
+ return 0;
+}
+
+static int pll_28nm_register(struct dsi_pll_28nm *pll_28nm, struct clk_hw **provided_clocks)
+{
+ char clk_name[32], parent1[32], parent2[32], vco_name[32];
+ struct clk_init_data vco_init = {
+ .parent_names = (const char *[]){ "xo" },
+ .num_parents = 1,
+ .name = vco_name,
+ .flags = CLK_IGNORE_UNUSED,
+ };
+ struct device *dev = &pll_28nm->phy->pdev->dev;
+ struct clk_hw *hw;
+ int ret;
+
+ DBG("%d", pll_28nm->phy->id);
+
+ if (pll_28nm->phy->cfg->quirks & DSI_PHY_28NM_QUIRK_PHY_LP)
+ vco_init.ops = &clk_ops_dsi_pll_28nm_vco_lp;
+ else
+ vco_init.ops = &clk_ops_dsi_pll_28nm_vco_hpm;
+
+ snprintf(vco_name, 32, "dsi%dvco_clk", pll_28nm->phy->id);
+ pll_28nm->clk_hw.init = &vco_init;
+ ret = devm_clk_hw_register(dev, &pll_28nm->clk_hw);
+ if (ret)
+ return ret;
+
+ snprintf(clk_name, 32, "dsi%danalog_postdiv_clk", pll_28nm->phy->id);
+ snprintf(parent1, 32, "dsi%dvco_clk", pll_28nm->phy->id);
+ hw = devm_clk_hw_register_divider(dev, clk_name,
+ parent1, CLK_SET_RATE_PARENT,
+ pll_28nm->phy->pll_base +
+ REG_DSI_28nm_PHY_PLL_POSTDIV1_CFG,
+ 0, 4, 0, NULL);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+
+ snprintf(clk_name, 32, "dsi%dindirect_path_div2_clk", pll_28nm->phy->id);
+ snprintf(parent1, 32, "dsi%danalog_postdiv_clk", pll_28nm->phy->id);
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name,
+ parent1, CLK_SET_RATE_PARENT,
+ 1, 2);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+
+ snprintf(clk_name, 32, "dsi%dpll", pll_28nm->phy->id);
+ snprintf(parent1, 32, "dsi%dvco_clk", pll_28nm->phy->id);
+ hw = devm_clk_hw_register_divider(dev, clk_name,
+ parent1, 0, pll_28nm->phy->pll_base +
+ REG_DSI_28nm_PHY_PLL_POSTDIV3_CFG,
+ 0, 8, 0, NULL);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+ provided_clocks[DSI_PIXEL_PLL_CLK] = hw;
+
+ snprintf(clk_name, 32, "dsi%dbyte_mux", pll_28nm->phy->id);
+ snprintf(parent1, 32, "dsi%dvco_clk", pll_28nm->phy->id);
+ snprintf(parent2, 32, "dsi%dindirect_path_div2_clk", pll_28nm->phy->id);
+ hw = devm_clk_hw_register_mux(dev, clk_name,
+ ((const char *[]){
+ parent1, parent2
+ }), 2, CLK_SET_RATE_PARENT, pll_28nm->phy->pll_base +
+ REG_DSI_28nm_PHY_PLL_VREG_CFG, 1, 1, 0, NULL);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+
+ snprintf(clk_name, 32, "dsi%dpllbyte", pll_28nm->phy->id);
+ snprintf(parent1, 32, "dsi%dbyte_mux", pll_28nm->phy->id);
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name,
+ parent1, CLK_SET_RATE_PARENT, 1, 4);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+ provided_clocks[DSI_BYTE_PLL_CLK] = hw;
+
+ return 0;
+}
+
+static int dsi_pll_28nm_init(struct msm_dsi_phy *phy)
+{
+ struct platform_device *pdev = phy->pdev;
+ struct dsi_pll_28nm *pll_28nm;
+ int ret;
+
+ if (!pdev)
+ return -ENODEV;
+
+ pll_28nm = devm_kzalloc(&pdev->dev, sizeof(*pll_28nm), GFP_KERNEL);
+ if (!pll_28nm)
+ return -ENOMEM;
+
+ pll_28nm->phy = phy;
+
+ ret = pll_28nm_register(pll_28nm, phy->provided_clocks->hws);
+ if (ret) {
+ DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
+ return ret;
+ }
+
+ phy->vco_hw = &pll_28nm->clk_hw;
+
+ return 0;
+}
+
static void dsi_28nm_dphy_set_timing(struct msm_dsi_phy *phy,
struct msm_dsi_dphy_timing *timing)
{
dsi_phy_write(base + REG_DSI_28nm_PHY_REGULATOR_CTRL_1, 0x1);
dsi_phy_write(base + REG_DSI_28nm_PHY_REGULATOR_CTRL_4, 0x20);
- if (phy->cfg->type == MSM_DSI_PHY_28NM_LP)
+ if (phy->cfg->quirks & DSI_PHY_28NM_QUIRK_PHY_LP)
dsi_phy_write(phy->base + REG_DSI_28nm_PHY_LDO_CNTRL, 0x05);
else
dsi_phy_write(phy->base + REG_DSI_28nm_PHY_LDO_CNTRL, 0x0d);
dsi_28nm_phy_regulator_enable_dcdc(phy);
}
-static int dsi_28nm_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+static int dsi_28nm_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req)
{
struct msm_dsi_dphy_timing *timing = &phy->timing;
int i;
void __iomem *base = phy->base;
+ u32 val;
DBG("");
dsi_phy_write(base + REG_DSI_28nm_PHY_CTRL_0, 0x5f);
- msm_dsi_phy_set_src_pll(phy, src_pll_id,
- REG_DSI_28nm_PHY_GLBL_TEST_CTRL,
- DSI_28nm_PHY_GLBL_TEST_CTRL_BITCLK_HS_SEL);
+ val = dsi_phy_read(base + REG_DSI_28nm_PHY_GLBL_TEST_CTRL);
+ if (phy->id == DSI_1 && phy->usecase == MSM_DSI_PHY_SLAVE)
+ val &= ~DSI_28nm_PHY_GLBL_TEST_CTRL_BITCLK_HS_SEL;
+ else
+ val |= DSI_28nm_PHY_GLBL_TEST_CTRL_BITCLK_HS_SEL;
+ dsi_phy_write(base + REG_DSI_28nm_PHY_GLBL_TEST_CTRL, val);
return 0;
}
}
const struct msm_dsi_phy_cfg dsi_phy_28nm_hpm_cfgs = {
- .type = MSM_DSI_PHY_28NM_HPM,
- .src_pll_truthtable = { {true, true}, {false, true} },
+ .has_phy_regulator = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_28nm_phy_enable,
.disable = dsi_28nm_phy_disable,
- .init = msm_dsi_phy_init_common,
+ .pll_init = dsi_pll_28nm_init,
+ .save_pll_state = dsi_28nm_pll_save_state,
+ .restore_pll_state = dsi_28nm_pll_restore_state,
},
+ .min_pll_rate = VCO_MIN_RATE,
+ .max_pll_rate = VCO_MAX_RATE,
.io_start = { 0xfd922b00, 0xfd923100 },
.num_dsi_phy = 2,
};
const struct msm_dsi_phy_cfg dsi_phy_28nm_hpm_famb_cfgs = {
- .type = MSM_DSI_PHY_28NM_HPM,
- .src_pll_truthtable = { {true, true}, {false, true} },
+ .has_phy_regulator = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_28nm_phy_enable,
.disable = dsi_28nm_phy_disable,
- .init = msm_dsi_phy_init_common,
+ .pll_init = dsi_pll_28nm_init,
+ .save_pll_state = dsi_28nm_pll_save_state,
+ .restore_pll_state = dsi_28nm_pll_restore_state,
},
+ .min_pll_rate = VCO_MIN_RATE,
+ .max_pll_rate = VCO_MAX_RATE,
.io_start = { 0x1a94400, 0x1a96400 },
.num_dsi_phy = 2,
};
const struct msm_dsi_phy_cfg dsi_phy_28nm_lp_cfgs = {
- .type = MSM_DSI_PHY_28NM_LP,
- .src_pll_truthtable = { {true, true}, {true, true} },
+ .has_phy_regulator = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_28nm_phy_enable,
.disable = dsi_28nm_phy_disable,
- .init = msm_dsi_phy_init_common,
+ .pll_init = dsi_pll_28nm_init,
+ .save_pll_state = dsi_28nm_pll_save_state,
+ .restore_pll_state = dsi_28nm_pll_restore_state,
},
+ .min_pll_rate = VCO_MIN_RATE,
+ .max_pll_rate = VCO_MAX_RATE,
.io_start = { 0x1a98500 },
.num_dsi_phy = 1,
+ .quirks = DSI_PHY_28NM_QUIRK_PHY_LP,
};
* Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
*/
+#include <linux/clk-provider.h>
#include <linux/delay.h>
#include "dsi_phy.h"
#include "dsi.xml.h"
+/*
+ * DSI PLL 28nm (8960/A family) - clock diagram (eg: DSI1):
+ *
+ *
+ * +------+
+ * dsi1vco_clk ----o-----| DIV1 |---dsi1pllbit (not exposed as clock)
+ * F * byte_clk | +------+
+ * | bit clock divider (F / 8)
+ * |
+ * | +------+
+ * o-----| DIV2 |---dsi0pllbyte---o---> To byte RCG
+ * | +------+ | (sets parent rate)
+ * | byte clock divider (F) |
+ * | |
+ * | o---> To esc RCG
+ * | (doesn't set parent rate)
+ * |
+ * | +------+
+ * o-----| DIV3 |----dsi0pll------o---> To dsi RCG
+ * +------+ | (sets parent rate)
+ * dsi clock divider (F * magic) |
+ * |
+ * o---> To pixel rcg
+ * (doesn't set parent rate)
+ */
+
+#define POLL_MAX_READS 8000
+#define POLL_TIMEOUT_US 1
+
+#define VCO_REF_CLK_RATE 27000000
+#define VCO_MIN_RATE 600000000
+#define VCO_MAX_RATE 1200000000
+
+#define VCO_PREF_DIV_RATIO 27
+
+struct pll_28nm_cached_state {
+ unsigned long vco_rate;
+ u8 postdiv3;
+ u8 postdiv2;
+ u8 postdiv1;
+};
+
+struct clk_bytediv {
+ struct clk_hw hw;
+ void __iomem *reg;
+};
+
+struct dsi_pll_28nm {
+ struct clk_hw clk_hw;
+
+ struct msm_dsi_phy *phy;
+
+ struct pll_28nm_cached_state cached_state;
+};
+
+#define to_pll_28nm(x) container_of(x, struct dsi_pll_28nm, clk_hw)
+
+static bool pll_28nm_poll_for_ready(struct dsi_pll_28nm *pll_28nm,
+ int nb_tries, int timeout_us)
+{
+ bool pll_locked = false;
+ u32 val;
+
+ while (nb_tries--) {
+ val = dsi_phy_read(pll_28nm->phy->pll_base + REG_DSI_28nm_8960_PHY_PLL_RDY);
+ pll_locked = !!(val & DSI_28nm_8960_PHY_PLL_RDY_PLL_RDY);
+
+ if (pll_locked)
+ break;
+
+ udelay(timeout_us);
+ }
+ DBG("DSI PLL is %slocked", pll_locked ? "" : "*not* ");
+
+ return pll_locked;
+}
+
+/*
+ * Clock Callbacks
+ */
+static int dsi_pll_28nm_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+ void __iomem *base = pll_28nm->phy->pll_base;
+ u32 val, temp, fb_divider;
+
+ DBG("rate=%lu, parent's=%lu", rate, parent_rate);
+
+ temp = rate / 10;
+ val = VCO_REF_CLK_RATE / 10;
+ fb_divider = (temp * VCO_PREF_DIV_RATIO) / val;
+ fb_divider = fb_divider / 2 - 1;
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_1,
+ fb_divider & 0xff);
+
+ val = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_2);
+
+ val |= (fb_divider >> 8) & 0x07;
+
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_2,
+ val);
+
+ val = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_3);
+
+ val |= (VCO_PREF_DIV_RATIO - 1) & 0x3f;
+
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_3,
+ val);
+
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_6,
+ 0xf);
+
+ val = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8);
+ val |= 0x7 << 4;
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8,
+ val);
+
+ return 0;
+}
+
+static int dsi_pll_28nm_clk_is_enabled(struct clk_hw *hw)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+
+ return pll_28nm_poll_for_ready(pll_28nm, POLL_MAX_READS,
+ POLL_TIMEOUT_US);
+}
+
+static unsigned long dsi_pll_28nm_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+ void __iomem *base = pll_28nm->phy->pll_base;
+ unsigned long vco_rate;
+ u32 status, fb_divider, temp, ref_divider;
+
+ VERB("parent_rate=%lu", parent_rate);
+
+ status = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_0);
+
+ if (status & DSI_28nm_8960_PHY_PLL_CTRL_0_ENABLE) {
+ fb_divider = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_1);
+ fb_divider &= 0xff;
+ temp = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_2) & 0x07;
+ fb_divider = (temp << 8) | fb_divider;
+ fb_divider += 1;
+
+ ref_divider = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_3);
+ ref_divider &= 0x3f;
+ ref_divider += 1;
+
+ /* multiply by 2 */
+ vco_rate = (parent_rate / ref_divider) * fb_divider * 2;
+ } else {
+ vco_rate = 0;
+ }
+
+ DBG("returning vco rate = %lu", vco_rate);
+
+ return vco_rate;
+}
+
+static int dsi_pll_28nm_vco_prepare(struct clk_hw *hw)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+ struct device *dev = &pll_28nm->phy->pdev->dev;
+ void __iomem *base = pll_28nm->phy->pll_base;
+ bool locked;
+ unsigned int bit_div, byte_div;
+ int max_reads = 1000, timeout_us = 100;
+ u32 val;
+
+ DBG("id=%d", pll_28nm->phy->id);
+
+ if (unlikely(pll_28nm->phy->pll_on))
+ return 0;
+
+ /*
+ * before enabling the PLL, configure the bit clock divider since we
+ * don't expose it as a clock to the outside world
+ * 1: read back the byte clock divider that should already be set
+ * 2: divide by 8 to get bit clock divider
+ * 3: write it to POSTDIV1
+ */
+ val = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_9);
+ byte_div = val + 1;
+ bit_div = byte_div / 8;
+
+ val = dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8);
+ val &= ~0xf;
+ val |= (bit_div - 1);
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8, val);
+
+ /* enable the PLL */
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_0,
+ DSI_28nm_8960_PHY_PLL_CTRL_0_ENABLE);
+
+ locked = pll_28nm_poll_for_ready(pll_28nm, max_reads, timeout_us);
+
+ if (unlikely(!locked)) {
+ DRM_DEV_ERROR(dev, "DSI PLL lock failed\n");
+ return -EINVAL;
+ }
+
+ DBG("DSI PLL lock success");
+ pll_28nm->phy->pll_on = true;
+
+ return 0;
+}
+
+static void dsi_pll_28nm_vco_unprepare(struct clk_hw *hw)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+
+ DBG("id=%d", pll_28nm->phy->id);
+
+ if (unlikely(!pll_28nm->phy->pll_on))
+ return;
+
+ dsi_phy_write(pll_28nm->phy->pll_base + REG_DSI_28nm_8960_PHY_PLL_CTRL_0, 0x00);
+
+ pll_28nm->phy->pll_on = false;
+}
+
+static long dsi_pll_28nm_clk_round_rate(struct clk_hw *hw,
+ unsigned long rate, unsigned long *parent_rate)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(hw);
+
+ if (rate < pll_28nm->phy->cfg->min_pll_rate)
+ return pll_28nm->phy->cfg->min_pll_rate;
+ else if (rate > pll_28nm->phy->cfg->max_pll_rate)
+ return pll_28nm->phy->cfg->max_pll_rate;
+ else
+ return rate;
+}
+
+static const struct clk_ops clk_ops_dsi_pll_28nm_vco = {
+ .round_rate = dsi_pll_28nm_clk_round_rate,
+ .set_rate = dsi_pll_28nm_clk_set_rate,
+ .recalc_rate = dsi_pll_28nm_clk_recalc_rate,
+ .prepare = dsi_pll_28nm_vco_prepare,
+ .unprepare = dsi_pll_28nm_vco_unprepare,
+ .is_enabled = dsi_pll_28nm_clk_is_enabled,
+};
+
+/*
+ * Custom byte clock divier clk_ops
+ *
+ * This clock is the entry point to configuring the PLL. The user (dsi host)
+ * will set this clock's rate to the desired byte clock rate. The VCO lock
+ * frequency is a multiple of the byte clock rate. The multiplication factor
+ * (shown as F in the diagram above) is a function of the byte clock rate.
+ *
+ * This custom divider clock ensures that its parent (VCO) is set to the
+ * desired rate, and that the byte clock postdivider (POSTDIV2) is configured
+ * accordingly
+ */
+#define to_clk_bytediv(_hw) container_of(_hw, struct clk_bytediv, hw)
+
+static unsigned long clk_bytediv_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct clk_bytediv *bytediv = to_clk_bytediv(hw);
+ unsigned int div;
+
+ div = dsi_phy_read(bytediv->reg) & 0xff;
+
+ return parent_rate / (div + 1);
+}
+
+/* find multiplication factor(wrt byte clock) at which the VCO should be set */
+static unsigned int get_vco_mul_factor(unsigned long byte_clk_rate)
+{
+ unsigned long bit_mhz;
+
+ /* convert to bit clock in Mhz */
+ bit_mhz = (byte_clk_rate * 8) / 1000000;
+
+ if (bit_mhz < 125)
+ return 64;
+ else if (bit_mhz < 250)
+ return 32;
+ else if (bit_mhz < 600)
+ return 16;
+ else
+ return 8;
+}
+
+static long clk_bytediv_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *prate)
+{
+ unsigned long best_parent;
+ unsigned int factor;
+
+ factor = get_vco_mul_factor(rate);
+
+ best_parent = rate * factor;
+ *prate = clk_hw_round_rate(clk_hw_get_parent(hw), best_parent);
+
+ return *prate / factor;
+}
+
+static int clk_bytediv_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_bytediv *bytediv = to_clk_bytediv(hw);
+ u32 val;
+ unsigned int factor;
+
+ factor = get_vco_mul_factor(rate);
+
+ val = dsi_phy_read(bytediv->reg);
+ val |= (factor - 1) & 0xff;
+ dsi_phy_write(bytediv->reg, val);
+
+ return 0;
+}
+
+/* Our special byte clock divider ops */
+static const struct clk_ops clk_bytediv_ops = {
+ .round_rate = clk_bytediv_round_rate,
+ .set_rate = clk_bytediv_set_rate,
+ .recalc_rate = clk_bytediv_recalc_rate,
+};
+
+/*
+ * PLL Callbacks
+ */
+static void dsi_28nm_pll_save_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(phy->vco_hw);
+ struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
+ void __iomem *base = pll_28nm->phy->pll_base;
+
+ cached_state->postdiv3 =
+ dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_10);
+ cached_state->postdiv2 =
+ dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_9);
+ cached_state->postdiv1 =
+ dsi_phy_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8);
+
+ cached_state->vco_rate = clk_hw_get_rate(phy->vco_hw);
+}
+
+static int dsi_28nm_pll_restore_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_28nm *pll_28nm = to_pll_28nm(phy->vco_hw);
+ struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
+ void __iomem *base = pll_28nm->phy->pll_base;
+ int ret;
+
+ ret = dsi_pll_28nm_clk_set_rate(phy->vco_hw,
+ cached_state->vco_rate, 0);
+ if (ret) {
+ DRM_DEV_ERROR(&pll_28nm->phy->pdev->dev,
+ "restore vco rate failed. ret=%d\n", ret);
+ return ret;
+ }
+
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_10,
+ cached_state->postdiv3);
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_9,
+ cached_state->postdiv2);
+ dsi_phy_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8,
+ cached_state->postdiv1);
+
+ return 0;
+}
+
+static int pll_28nm_register(struct dsi_pll_28nm *pll_28nm, struct clk_hw **provided_clocks)
+{
+ char *clk_name, *parent_name, *vco_name;
+ struct clk_init_data vco_init = {
+ .parent_names = (const char *[]){ "pxo" },
+ .num_parents = 1,
+ .flags = CLK_IGNORE_UNUSED,
+ .ops = &clk_ops_dsi_pll_28nm_vco,
+ };
+ struct device *dev = &pll_28nm->phy->pdev->dev;
+ struct clk_hw *hw;
+ struct clk_bytediv *bytediv;
+ struct clk_init_data bytediv_init = { };
+ int ret;
+
+ DBG("%d", pll_28nm->phy->id);
+
+ bytediv = devm_kzalloc(dev, sizeof(*bytediv), GFP_KERNEL);
+ if (!bytediv)
+ return -ENOMEM;
+
+ vco_name = devm_kzalloc(dev, 32, GFP_KERNEL);
+ if (!vco_name)
+ return -ENOMEM;
+
+ clk_name = devm_kzalloc(dev, 32, GFP_KERNEL);
+ if (!clk_name)
+ return -ENOMEM;
+
+ snprintf(vco_name, 32, "dsi%dvco_clk", pll_28nm->phy->id);
+ vco_init.name = vco_name;
+
+ pll_28nm->clk_hw.init = &vco_init;
+
+ ret = devm_clk_hw_register(dev, &pll_28nm->clk_hw);
+ if (ret)
+ return ret;
+
+ /* prepare and register bytediv */
+ bytediv->hw.init = &bytediv_init;
+ bytediv->reg = pll_28nm->phy->pll_base + REG_DSI_28nm_8960_PHY_PLL_CTRL_9;
+
+ snprintf(parent_name, 32, "dsi%dvco_clk", pll_28nm->phy->id);
+ snprintf(clk_name, 32, "dsi%dpllbyte", pll_28nm->phy->id);
+
+ bytediv_init.name = clk_name;
+ bytediv_init.ops = &clk_bytediv_ops;
+ bytediv_init.flags = CLK_SET_RATE_PARENT;
+ bytediv_init.parent_names = (const char * const *) &parent_name;
+ bytediv_init.num_parents = 1;
+
+ /* DIV2 */
+ ret = devm_clk_hw_register(dev, &bytediv->hw);
+ if (ret)
+ return ret;
+ provided_clocks[DSI_BYTE_PLL_CLK] = &bytediv->hw;
+
+ snprintf(clk_name, 32, "dsi%dpll", pll_28nm->phy->id);
+ /* DIV3 */
+ hw = devm_clk_hw_register_divider(dev, clk_name,
+ parent_name, 0, pll_28nm->phy->pll_base +
+ REG_DSI_28nm_8960_PHY_PLL_CTRL_10,
+ 0, 8, 0, NULL);
+ if (IS_ERR(hw))
+ return PTR_ERR(hw);
+ provided_clocks[DSI_PIXEL_PLL_CLK] = hw;
+
+ return 0;
+}
+
+static int dsi_pll_28nm_8960_init(struct msm_dsi_phy *phy)
+{
+ struct platform_device *pdev = phy->pdev;
+ struct dsi_pll_28nm *pll_28nm;
+ int ret;
+
+ if (!pdev)
+ return -ENODEV;
+
+ pll_28nm = devm_kzalloc(&pdev->dev, sizeof(*pll_28nm), GFP_KERNEL);
+ if (!pll_28nm)
+ return -ENOMEM;
+
+ pll_28nm->phy = phy;
+
+ ret = pll_28nm_register(pll_28nm, phy->provided_clocks->hws);
+ if (ret) {
+ DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
+ return ret;
+ }
+
+ phy->vco_hw = &pll_28nm->clk_hw;
+
+ return 0;
+}
+
static void dsi_28nm_dphy_set_timing(struct msm_dsi_phy *phy,
struct msm_dsi_dphy_timing *timing)
{
dsi_phy_write(base + REG_DSI_28nm_8960_PHY_LNCK_TEST_STR1, 0x88);
}
-static int dsi_28nm_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+static int dsi_28nm_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req)
{
struct msm_dsi_dphy_timing *timing = &phy->timing;
}
const struct msm_dsi_phy_cfg dsi_phy_28nm_8960_cfgs = {
- .type = MSM_DSI_PHY_28NM_8960,
- .src_pll_truthtable = { {true, true}, {false, true} },
+ .has_phy_regulator = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_28nm_phy_enable,
.disable = dsi_28nm_phy_disable,
- .init = msm_dsi_phy_init_common,
+ .pll_init = dsi_pll_28nm_8960_init,
+ .save_pll_state = dsi_28nm_pll_save_state,
+ .restore_pll_state = dsi_28nm_pll_restore_state,
},
+ .min_pll_rate = VCO_MIN_RATE,
+ .max_pll_rate = VCO_MAX_RATE,
.io_start = { 0x4700300, 0x5800300 },
.num_dsi_phy = 2,
};
* Copyright (c) 2018, The Linux Foundation
*/
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include <linux/iopoll.h>
#include "dsi_phy.h"
#include "dsi.xml.h"
+/*
+ * DSI PLL 7nm - clock diagram (eg: DSI0): TODO: updated CPHY diagram
+ *
+ * dsi0_pll_out_div_clk dsi0_pll_bit_clk
+ * | |
+ * | |
+ * +---------+ | +----------+ | +----+
+ * dsi0vco_clk ---| out_div |--o--| divl_3_0 |--o--| /8 |-- dsi0_phy_pll_out_byteclk
+ * +---------+ | +----------+ | +----+
+ * | |
+ * | | dsi0_pll_by_2_bit_clk
+ * | | |
+ * | | +----+ | |\ dsi0_pclk_mux
+ * | |--| /2 |--o--| \ |
+ * | | +----+ | \ | +---------+
+ * | --------------| |--o--| div_7_4 |-- dsi0_phy_pll_out_dsiclk
+ * |------------------------------| / +---------+
+ * | +-----+ | /
+ * -----------| /4? |--o----------|/
+ * +-----+ | |
+ * | |dsiclk_sel
+ * |
+ * dsi0_pll_post_out_div_clk
+ */
+
+#define VCO_REF_CLK_RATE 19200000
+#define FRAC_BITS 18
+
+/* Hardware is V4.1 */
+#define DSI_PHY_7NM_QUIRK_V4_1 BIT(0)
+
+struct dsi_pll_config {
+ bool enable_ssc;
+ bool ssc_center;
+ u32 ssc_freq;
+ u32 ssc_offset;
+ u32 ssc_adj_per;
+
+ /* out */
+ u32 decimal_div_start;
+ u32 frac_div_start;
+ u32 pll_clock_inverters;
+ u32 ssc_stepsize;
+ u32 ssc_div_per;
+};
+
+struct pll_7nm_cached_state {
+ unsigned long vco_rate;
+ u8 bit_clk_div;
+ u8 pix_clk_div;
+ u8 pll_out_div;
+ u8 pll_mux;
+};
+
+struct dsi_pll_7nm {
+ struct clk_hw clk_hw;
+
+ struct msm_dsi_phy *phy;
+
+ u64 vco_current_rate;
+
+ /* protects REG_DSI_7nm_PHY_CMN_CLK_CFG0 register */
+ spinlock_t postdiv_lock;
+
+ struct pll_7nm_cached_state cached_state;
+
+ struct dsi_pll_7nm *slave;
+};
+
+#define to_pll_7nm(x) container_of(x, struct dsi_pll_7nm, clk_hw)
+
+/*
+ * Global list of private DSI PLL struct pointers. We need this for Dual DSI
+ * mode, where the master PLL's clk_ops needs access the slave's private data
+ */
+static struct dsi_pll_7nm *pll_7nm_list[DSI_MAX];
+
+static void dsi_pll_setup_config(struct dsi_pll_config *config)
+{
+ config->ssc_freq = 31500;
+ config->ssc_offset = 4800;
+ config->ssc_adj_per = 2;
+
+ /* TODO: ssc enable */
+ config->enable_ssc = false;
+ config->ssc_center = 0;
+}
+
+static void dsi_pll_calc_dec_frac(struct dsi_pll_7nm *pll, struct dsi_pll_config *config)
+{
+ u64 fref = VCO_REF_CLK_RATE;
+ u64 pll_freq;
+ u64 divider;
+ u64 dec, dec_multiple;
+ u32 frac;
+ u64 multiplier;
+
+ pll_freq = pll->vco_current_rate;
+
+ divider = fref * 2;
+
+ multiplier = 1 << FRAC_BITS;
+ dec_multiple = div_u64(pll_freq * multiplier, divider);
+ div_u64_rem(dec_multiple, multiplier, &frac);
+
+ dec = div_u64(dec_multiple, multiplier);
+
+ if (!(pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_1))
+ config->pll_clock_inverters = 0x28;
+ else if (pll_freq <= 1000000000ULL)
+ config->pll_clock_inverters = 0xa0;
+ else if (pll_freq <= 2500000000ULL)
+ config->pll_clock_inverters = 0x20;
+ else if (pll_freq <= 3020000000ULL)
+ config->pll_clock_inverters = 0x00;
+ else
+ config->pll_clock_inverters = 0x40;
+
+ config->decimal_div_start = dec;
+ config->frac_div_start = frac;
+}
+
+#define SSC_CENTER BIT(0)
+#define SSC_EN BIT(1)
+
+static void dsi_pll_calc_ssc(struct dsi_pll_7nm *pll, struct dsi_pll_config *config)
+{
+ u32 ssc_per;
+ u32 ssc_mod;
+ u64 ssc_step_size;
+ u64 frac;
+
+ if (!config->enable_ssc) {
+ DBG("SSC not enabled\n");
+ return;
+ }
+
+ ssc_per = DIV_ROUND_CLOSEST(VCO_REF_CLK_RATE, config->ssc_freq) / 2 - 1;
+ ssc_mod = (ssc_per + 1) % (config->ssc_adj_per + 1);
+ ssc_per -= ssc_mod;
+
+ frac = config->frac_div_start;
+ ssc_step_size = config->decimal_div_start;
+ ssc_step_size *= (1 << FRAC_BITS);
+ ssc_step_size += frac;
+ ssc_step_size *= config->ssc_offset;
+ ssc_step_size *= (config->ssc_adj_per + 1);
+ ssc_step_size = div_u64(ssc_step_size, (ssc_per + 1));
+ ssc_step_size = DIV_ROUND_CLOSEST_ULL(ssc_step_size, 1000000);
+
+ config->ssc_div_per = ssc_per;
+ config->ssc_stepsize = ssc_step_size;
+
+ pr_debug("SCC: Dec:%d, frac:%llu, frac_bits:%d\n",
+ config->decimal_div_start, frac, FRAC_BITS);
+ pr_debug("SSC: div_per:0x%X, stepsize:0x%X, adjper:0x%X\n",
+ ssc_per, (u32)ssc_step_size, config->ssc_adj_per);
+}
+
+static void dsi_pll_ssc_commit(struct dsi_pll_7nm *pll, struct dsi_pll_config *config)
+{
+ void __iomem *base = pll->phy->pll_base;
+
+ if (config->enable_ssc) {
+ pr_debug("SSC is enabled\n");
+
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_SSC_STEPSIZE_LOW_1,
+ config->ssc_stepsize & 0xff);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_SSC_STEPSIZE_HIGH_1,
+ config->ssc_stepsize >> 8);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_SSC_DIV_PER_LOW_1,
+ config->ssc_div_per & 0xff);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_SSC_DIV_PER_HIGH_1,
+ config->ssc_div_per >> 8);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_SSC_ADJPER_LOW_1,
+ config->ssc_adj_per & 0xff);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_SSC_ADJPER_HIGH_1,
+ config->ssc_adj_per >> 8);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_SSC_CONTROL,
+ SSC_EN | (config->ssc_center ? SSC_CENTER : 0));
+ }
+}
+
+static void dsi_pll_config_hzindep_reg(struct dsi_pll_7nm *pll)
+{
+ void __iomem *base = pll->phy->pll_base;
+ u8 analog_controls_five_1 = 0x01, vco_config_1 = 0x00;
+
+ if (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_1) {
+ if (pll->vco_current_rate >= 3100000000ULL)
+ analog_controls_five_1 = 0x03;
+
+ if (pll->vco_current_rate < 1520000000ULL)
+ vco_config_1 = 0x08;
+ else if (pll->vco_current_rate < 2990000000ULL)
+ vco_config_1 = 0x01;
+ }
+
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_FIVE_1,
+ analog_controls_five_1);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_VCO_CONFIG_1, vco_config_1);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_FIVE, 0x01);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_TWO, 0x03);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_THREE, 0x00);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_DSM_DIVIDER, 0x00);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_FEEDBACK_DIVIDER, 0x4e);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_CALIBRATION_SETTINGS, 0x40);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_BAND_SEL_CAL_SETTINGS_THREE, 0xba);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_FREQ_DETECT_SETTINGS_ONE, 0x0c);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_OUTDIV, 0x00);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_CORE_OVERRIDE, 0x00);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_DIGITAL_TIMERS_TWO, 0x08);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_PROP_GAIN_RATE_1, 0x0a);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_BAND_SEL_RATE_1, 0xc0);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_INT_GAIN_IFILT_BAND_1, 0x84);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_INT_GAIN_IFILT_BAND_1, 0x82);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_FL_INT_GAIN_PFILT_BAND_1, 0x4c);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_LOCK_OVERRIDE, 0x80);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PFILT, 0x29);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PFILT, 0x2f);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_IFILT, 0x2a);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_IFILT,
+ pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_1 ? 0x3f : 0x22);
+
+ if (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_1) {
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PERF_OPTIMIZE, 0x22);
+ if (pll->slave)
+ dsi_phy_write(pll->slave->phy->pll_base + REG_DSI_7nm_PHY_PLL_PERF_OPTIMIZE, 0x22);
+ }
+}
+
+static void dsi_pll_commit(struct dsi_pll_7nm *pll, struct dsi_pll_config *config)
+{
+ void __iomem *base = pll->phy->pll_base;
+
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_CORE_INPUT_OVERRIDE, 0x12);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_DECIMAL_DIV_START_1, config->decimal_div_start);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_LOW_1,
+ config->frac_div_start & 0xff);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_MID_1,
+ (config->frac_div_start & 0xff00) >> 8);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_HIGH_1,
+ (config->frac_div_start & 0x30000) >> 16);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_LOCKDET_RATE_1, 0x40);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_PLL_LOCK_DELAY, 0x06);
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_CMODE_1, 0x10); /* TODO: 0x00 for CPHY */
+ dsi_phy_write(base + REG_DSI_7nm_PHY_PLL_CLOCK_INVERTERS, config->pll_clock_inverters);
+}
+
+static int dsi_pll_7nm_vco_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(hw);
+ struct dsi_pll_config config;
+
+ DBG("DSI PLL%d rate=%lu, parent's=%lu", pll_7nm->phy->id, rate,
+ parent_rate);
+
+ pll_7nm->vco_current_rate = rate;
+
+ dsi_pll_setup_config(&config);
+
+ dsi_pll_calc_dec_frac(pll_7nm, &config);
+
+ dsi_pll_calc_ssc(pll_7nm, &config);
+
+ dsi_pll_commit(pll_7nm, &config);
+
+ dsi_pll_config_hzindep_reg(pll_7nm);
+
+ dsi_pll_ssc_commit(pll_7nm, &config);
+
+ /* flush, ensure all register writes are done*/
+ wmb();
+
+ return 0;
+}
+
+static int dsi_pll_7nm_lock_status(struct dsi_pll_7nm *pll)
+{
+ int rc;
+ u32 status = 0;
+ u32 const delay_us = 100;
+ u32 const timeout_us = 5000;
+
+ rc = readl_poll_timeout_atomic(pll->phy->pll_base +
+ REG_DSI_7nm_PHY_PLL_COMMON_STATUS_ONE,
+ status,
+ ((status & BIT(0)) > 0),
+ delay_us,
+ timeout_us);
+ if (rc)
+ pr_err("DSI PLL(%d) lock failed, status=0x%08x\n",
+ pll->phy->id, status);
+
+ return rc;
+}
+
+static void dsi_pll_disable_pll_bias(struct dsi_pll_7nm *pll)
+{
+ u32 data = dsi_phy_read(pll->phy->base + REG_DSI_7nm_PHY_CMN_CTRL_0);
+
+ dsi_phy_write(pll->phy->pll_base + REG_DSI_7nm_PHY_PLL_SYSTEM_MUXES, 0);
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_CTRL_0, data & ~BIT(5));
+ ndelay(250);
+}
+
+static void dsi_pll_enable_pll_bias(struct dsi_pll_7nm *pll)
+{
+ u32 data = dsi_phy_read(pll->phy->base + REG_DSI_7nm_PHY_CMN_CTRL_0);
+
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_CTRL_0, data | BIT(5));
+ dsi_phy_write(pll->phy->pll_base + REG_DSI_7nm_PHY_PLL_SYSTEM_MUXES, 0xc0);
+ ndelay(250);
+}
+
+static void dsi_pll_disable_global_clk(struct dsi_pll_7nm *pll)
+{
+ u32 data;
+
+ data = dsi_phy_read(pll->phy->base + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_CLK_CFG1, data & ~BIT(5));
+}
+
+static void dsi_pll_enable_global_clk(struct dsi_pll_7nm *pll)
+{
+ u32 data;
+
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_CTRL_3, 0x04);
+
+ data = dsi_phy_read(pll->phy->base + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_CLK_CFG1,
+ data | BIT(5) | BIT(4));
+}
+
+static void dsi_pll_phy_dig_reset(struct dsi_pll_7nm *pll)
+{
+ /*
+ * Reset the PHY digital domain. This would be needed when
+ * coming out of a CX or analog rail power collapse while
+ * ensuring that the pads maintain LP00 or LP11 state
+ */
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE4, BIT(0));
+ wmb(); /* Ensure that the reset is deasserted */
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE4, 0x0);
+ wmb(); /* Ensure that the reset is deasserted */
+}
+
+static int dsi_pll_7nm_vco_prepare(struct clk_hw *hw)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(hw);
+ int rc;
+
+ dsi_pll_enable_pll_bias(pll_7nm);
+ if (pll_7nm->slave)
+ dsi_pll_enable_pll_bias(pll_7nm->slave);
+
+ /* Start PLL */
+ dsi_phy_write(pll_7nm->phy->base + REG_DSI_7nm_PHY_CMN_PLL_CNTRL, 0x01);
+
+ /*
+ * ensure all PLL configurations are written prior to checking
+ * for PLL lock.
+ */
+ wmb();
+
+ /* Check for PLL lock */
+ rc = dsi_pll_7nm_lock_status(pll_7nm);
+ if (rc) {
+ pr_err("PLL(%d) lock failed\n", pll_7nm->phy->id);
+ goto error;
+ }
+
+ pll_7nm->phy->pll_on = true;
+
+ /*
+ * assert power on reset for PHY digital in case the PLL is
+ * enabled after CX of analog domain power collapse. This needs
+ * to be done before enabling the global clk.
+ */
+ dsi_pll_phy_dig_reset(pll_7nm);
+ if (pll_7nm->slave)
+ dsi_pll_phy_dig_reset(pll_7nm->slave);
+
+ dsi_pll_enable_global_clk(pll_7nm);
+ if (pll_7nm->slave)
+ dsi_pll_enable_global_clk(pll_7nm->slave);
+
+error:
+ return rc;
+}
+
+static void dsi_pll_disable_sub(struct dsi_pll_7nm *pll)
+{
+ dsi_phy_write(pll->phy->base + REG_DSI_7nm_PHY_CMN_RBUF_CTRL, 0);
+ dsi_pll_disable_pll_bias(pll);
+}
+
+static void dsi_pll_7nm_vco_unprepare(struct clk_hw *hw)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(hw);
+
+ /*
+ * To avoid any stray glitches while abruptly powering down the PLL
+ * make sure to gate the clock using the clock enable bit before
+ * powering down the PLL
+ */
+ dsi_pll_disable_global_clk(pll_7nm);
+ dsi_phy_write(pll_7nm->phy->base + REG_DSI_7nm_PHY_CMN_PLL_CNTRL, 0);
+ dsi_pll_disable_sub(pll_7nm);
+ if (pll_7nm->slave) {
+ dsi_pll_disable_global_clk(pll_7nm->slave);
+ dsi_pll_disable_sub(pll_7nm->slave);
+ }
+ /* flush, ensure all register writes are done */
+ wmb();
+ pll_7nm->phy->pll_on = false;
+}
+
+static unsigned long dsi_pll_7nm_vco_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(hw);
+ void __iomem *base = pll_7nm->phy->pll_base;
+ u64 ref_clk = VCO_REF_CLK_RATE;
+ u64 vco_rate = 0x0;
+ u64 multiplier;
+ u32 frac;
+ u32 dec;
+ u64 pll_freq, tmp64;
+
+ dec = dsi_phy_read(base + REG_DSI_7nm_PHY_PLL_DECIMAL_DIV_START_1);
+ dec &= 0xff;
+
+ frac = dsi_phy_read(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_LOW_1);
+ frac |= ((dsi_phy_read(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_MID_1) &
+ 0xff) << 8);
+ frac |= ((dsi_phy_read(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_HIGH_1) &
+ 0x3) << 16);
+
+ /*
+ * TODO:
+ * 1. Assumes prescaler is disabled
+ */
+ multiplier = 1 << FRAC_BITS;
+ pll_freq = dec * (ref_clk * 2);
+ tmp64 = (ref_clk * 2 * frac);
+ pll_freq += div_u64(tmp64, multiplier);
+
+ vco_rate = pll_freq;
+
+ DBG("DSI PLL%d returning vco rate = %lu, dec = %x, frac = %x",
+ pll_7nm->phy->id, (unsigned long)vco_rate, dec, frac);
+
+ return (unsigned long)vco_rate;
+}
+
+static long dsi_pll_7nm_clk_round_rate(struct clk_hw *hw,
+ unsigned long rate, unsigned long *parent_rate)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(hw);
+
+ if (rate < pll_7nm->phy->cfg->min_pll_rate)
+ return pll_7nm->phy->cfg->min_pll_rate;
+ else if (rate > pll_7nm->phy->cfg->max_pll_rate)
+ return pll_7nm->phy->cfg->max_pll_rate;
+ else
+ return rate;
+}
+
+static const struct clk_ops clk_ops_dsi_pll_7nm_vco = {
+ .round_rate = dsi_pll_7nm_clk_round_rate,
+ .set_rate = dsi_pll_7nm_vco_set_rate,
+ .recalc_rate = dsi_pll_7nm_vco_recalc_rate,
+ .prepare = dsi_pll_7nm_vco_prepare,
+ .unprepare = dsi_pll_7nm_vco_unprepare,
+};
+
+/*
+ * PLL Callbacks
+ */
+
+static void dsi_7nm_pll_save_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(phy->vco_hw);
+ struct pll_7nm_cached_state *cached = &pll_7nm->cached_state;
+ void __iomem *phy_base = pll_7nm->phy->base;
+ u32 cmn_clk_cfg0, cmn_clk_cfg1;
+
+ cached->pll_out_div = dsi_phy_read(pll_7nm->phy->pll_base +
+ REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE);
+ cached->pll_out_div &= 0x3;
+
+ cmn_clk_cfg0 = dsi_phy_read(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG0);
+ cached->bit_clk_div = cmn_clk_cfg0 & 0xf;
+ cached->pix_clk_div = (cmn_clk_cfg0 & 0xf0) >> 4;
+
+ cmn_clk_cfg1 = dsi_phy_read(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
+ cached->pll_mux = cmn_clk_cfg1 & 0x3;
+
+ DBG("DSI PLL%d outdiv %x bit_clk_div %x pix_clk_div %x pll_mux %x",
+ pll_7nm->phy->id, cached->pll_out_div, cached->bit_clk_div,
+ cached->pix_clk_div, cached->pll_mux);
+}
+
+static int dsi_7nm_pll_restore_state(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(phy->vco_hw);
+ struct pll_7nm_cached_state *cached = &pll_7nm->cached_state;
+ void __iomem *phy_base = pll_7nm->phy->base;
+ u32 val;
+ int ret;
+
+ val = dsi_phy_read(pll_7nm->phy->pll_base + REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE);
+ val &= ~0x3;
+ val |= cached->pll_out_div;
+ dsi_phy_write(pll_7nm->phy->pll_base + REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE, val);
+
+ dsi_phy_write(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG0,
+ cached->bit_clk_div | (cached->pix_clk_div << 4));
+
+ val = dsi_phy_read(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
+ val &= ~0x3;
+ val |= cached->pll_mux;
+ dsi_phy_write(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG1, val);
+
+ ret = dsi_pll_7nm_vco_set_rate(phy->vco_hw,
+ pll_7nm->vco_current_rate,
+ VCO_REF_CLK_RATE);
+ if (ret) {
+ DRM_DEV_ERROR(&pll_7nm->phy->pdev->dev,
+ "restore vco rate failed. ret=%d\n", ret);
+ return ret;
+ }
+
+ DBG("DSI PLL%d", pll_7nm->phy->id);
+
+ return 0;
+}
+
+static int dsi_7nm_set_usecase(struct msm_dsi_phy *phy)
+{
+ struct dsi_pll_7nm *pll_7nm = to_pll_7nm(phy->vco_hw);
+ void __iomem *base = phy->base;
+ u32 data = 0x0; /* internal PLL */
+
+ DBG("DSI PLL%d", pll_7nm->phy->id);
+
+ switch (phy->usecase) {
+ case MSM_DSI_PHY_STANDALONE:
+ break;
+ case MSM_DSI_PHY_MASTER:
+ pll_7nm->slave = pll_7nm_list[(pll_7nm->phy->id + 1) % DSI_MAX];
+ break;
+ case MSM_DSI_PHY_SLAVE:
+ data = 0x1; /* external PLL */
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ /* set PLL src */
+ dsi_phy_write(base + REG_DSI_7nm_PHY_CMN_CLK_CFG1, (data << 2));
+
+ return 0;
+}
+
+/*
+ * The post dividers and mux clocks are created using the standard divider and
+ * mux API. Unlike the 14nm PHY, the slave PLL doesn't need its dividers/mux
+ * state to follow the master PLL's divider/mux state. Therefore, we don't
+ * require special clock ops that also configure the slave PLL registers
+ */
+static int pll_7nm_register(struct dsi_pll_7nm *pll_7nm, struct clk_hw **provided_clocks)
+{
+ char clk_name[32], parent[32], vco_name[32];
+ char parent2[32], parent3[32], parent4[32];
+ struct clk_init_data vco_init = {
+ .parent_names = (const char *[]){ "bi_tcxo" },
+ .num_parents = 1,
+ .name = vco_name,
+ .flags = CLK_IGNORE_UNUSED,
+ .ops = &clk_ops_dsi_pll_7nm_vco,
+ };
+ struct device *dev = &pll_7nm->phy->pdev->dev;
+ struct clk_hw *hw;
+ int ret;
+
+ DBG("DSI%d", pll_7nm->phy->id);
+
+ snprintf(vco_name, 32, "dsi%dvco_clk", pll_7nm->phy->id);
+ pll_7nm->clk_hw.init = &vco_init;
+
+ ret = devm_clk_hw_register(dev, &pll_7nm->clk_hw);
+ if (ret)
+ return ret;
+
+ snprintf(clk_name, 32, "dsi%d_pll_out_div_clk", pll_7nm->phy->id);
+ snprintf(parent, 32, "dsi%dvco_clk", pll_7nm->phy->id);
+
+ hw = devm_clk_hw_register_divider(dev, clk_name,
+ parent, CLK_SET_RATE_PARENT,
+ pll_7nm->phy->pll_base +
+ REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE,
+ 0, 2, CLK_DIVIDER_POWER_OF_TWO, NULL);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_pll_bit_clk", pll_7nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_7nm->phy->id);
+
+ /* BIT CLK: DIV_CTRL_3_0 */
+ hw = devm_clk_hw_register_divider(dev, clk_name, parent,
+ CLK_SET_RATE_PARENT,
+ pll_7nm->phy->base +
+ REG_DSI_7nm_PHY_CMN_CLK_CFG0,
+ 0, 4, CLK_DIVIDER_ONE_BASED,
+ &pll_7nm->postdiv_lock);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_phy_pll_out_byteclk", pll_7nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_7nm->phy->id);
+
+ /* DSI Byte clock = VCO_CLK / OUT_DIV / BIT_DIV / 8 */
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent,
+ CLK_SET_RATE_PARENT, 1, 8);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ provided_clocks[DSI_BYTE_PLL_CLK] = hw;
+
+ snprintf(clk_name, 32, "dsi%d_pll_by_2_bit_clk", pll_7nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_7nm->phy->id);
+
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent,
+ 0, 1, 2);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_pll_post_out_div_clk", pll_7nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_7nm->phy->id);
+
+ hw = devm_clk_hw_register_fixed_factor(dev, clk_name, parent,
+ 0, 1, 4);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_pclk_mux", pll_7nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_7nm->phy->id);
+ snprintf(parent2, 32, "dsi%d_pll_by_2_bit_clk", pll_7nm->phy->id);
+ snprintf(parent3, 32, "dsi%d_pll_out_div_clk", pll_7nm->phy->id);
+ snprintf(parent4, 32, "dsi%d_pll_post_out_div_clk", pll_7nm->phy->id);
+
+ hw = devm_clk_hw_register_mux(dev, clk_name,
+ ((const char *[]){
+ parent, parent2, parent3, parent4
+ }), 4, 0, pll_7nm->phy->base +
+ REG_DSI_7nm_PHY_CMN_CLK_CFG1,
+ 0, 2, 0, NULL);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ snprintf(clk_name, 32, "dsi%d_phy_pll_out_dsiclk", pll_7nm->phy->id);
+ snprintf(parent, 32, "dsi%d_pclk_mux", pll_7nm->phy->id);
+
+ /* PIX CLK DIV : DIV_CTRL_7_4*/
+ hw = devm_clk_hw_register_divider(dev, clk_name, parent,
+ 0, pll_7nm->phy->base +
+ REG_DSI_7nm_PHY_CMN_CLK_CFG0,
+ 4, 4, CLK_DIVIDER_ONE_BASED,
+ &pll_7nm->postdiv_lock);
+ if (IS_ERR(hw)) {
+ ret = PTR_ERR(hw);
+ goto fail;
+ }
+
+ provided_clocks[DSI_PIXEL_PLL_CLK] = hw;
+
+ return 0;
+
+fail:
+
+ return ret;
+}
+
+static int dsi_pll_7nm_init(struct msm_dsi_phy *phy)
+{
+ struct platform_device *pdev = phy->pdev;
+ struct dsi_pll_7nm *pll_7nm;
+ int ret;
+
+ pll_7nm = devm_kzalloc(&pdev->dev, sizeof(*pll_7nm), GFP_KERNEL);
+ if (!pll_7nm)
+ return -ENOMEM;
+
+ DBG("DSI PLL%d", phy->id);
+
+ pll_7nm_list[phy->id] = pll_7nm;
+
+ spin_lock_init(&pll_7nm->postdiv_lock);
+
+ pll_7nm->phy = phy;
+
+ ret = pll_7nm_register(pll_7nm, phy->provided_clocks->hws);
+ if (ret) {
+ DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
+ return ret;
+ }
+
+ phy->vco_hw = &pll_7nm->clk_hw;
+
+ /* TODO: Remove this when we have proper display handover support */
+ msm_dsi_phy_pll_save_state(phy);
+
+ return 0;
+}
+
static int dsi_phy_hw_v4_0_is_pll_on(struct msm_dsi_phy *phy)
{
void __iomem *base = phy->base;
const u8 *tx_dctrl = tx_dctrl_0;
void __iomem *lane_base = phy->lane_base;
- if (phy->cfg->type == MSM_DSI_PHY_7NM_V4_1)
+ if (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_1)
tx_dctrl = tx_dctrl_1;
/* Strength ctrl settings */
}
}
-static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy, int src_pll_id,
+static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy,
struct msm_dsi_phy_clk_request *clk_req)
{
int ret;
/* Alter PHY configurations if data rate less than 1.5GHZ*/
less_than_1500_mhz = (clk_req->bitclk_rate <= 1500000000);
- if (phy->cfg->type == MSM_DSI_PHY_7NM_V4_1) {
+ if (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_1) {
vreg_ctrl_0 = less_than_1500_mhz ? 0x53 : 0x52;
glbl_rescode_top_ctrl = less_than_1500_mhz ? 0x3d : 0x00;
glbl_rescode_bot_ctrl = less_than_1500_mhz ? 0x39 : 0x3c;
/* Select full-rate mode */
dsi_phy_write(base + REG_DSI_7nm_PHY_CMN_CTRL_2, 0x40);
- ret = msm_dsi_pll_set_usecase(phy->pll, phy->usecase);
+ ret = dsi_7nm_set_usecase(phy);
if (ret) {
DRM_DEV_ERROR(&phy->pdev->dev, "%s: set pll usecase failed, %d\n",
__func__, ret);
DBG("DSI%d PHY disabled", phy->id);
}
-static int dsi_7nm_phy_init(struct msm_dsi_phy *phy)
-{
- struct platform_device *pdev = phy->pdev;
-
- phy->lane_base = msm_ioremap(pdev, "dsi_phy_lane",
- "DSI_PHY_LANE");
- if (IS_ERR(phy->lane_base)) {
- DRM_DEV_ERROR(&pdev->dev, "%s: failed to map phy lane base\n",
- __func__);
- return -ENOMEM;
- }
-
- return 0;
-}
-
const struct msm_dsi_phy_cfg dsi_phy_7nm_cfgs = {
- .type = MSM_DSI_PHY_7NM_V4_1,
- .src_pll_truthtable = { {false, false}, {true, false} },
+ .has_phy_lane = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_7nm_phy_enable,
.disable = dsi_7nm_phy_disable,
- .init = dsi_7nm_phy_init,
+ .pll_init = dsi_pll_7nm_init,
+ .save_pll_state = dsi_7nm_pll_save_state,
+ .restore_pll_state = dsi_7nm_pll_restore_state,
},
+ .min_pll_rate = 600000000UL,
+ .max_pll_rate = (5000000000ULL < ULONG_MAX) ? 5000000000ULL : ULONG_MAX,
.io_start = { 0xae94400, 0xae96400 },
.num_dsi_phy = 2,
+ .quirks = DSI_PHY_7NM_QUIRK_V4_1,
};
const struct msm_dsi_phy_cfg dsi_phy_7nm_8150_cfgs = {
- .type = MSM_DSI_PHY_7NM,
- .src_pll_truthtable = { {false, false}, {true, false} },
+ .has_phy_lane = true,
.reg_cfg = {
.num = 1,
.regs = {
.ops = {
.enable = dsi_7nm_phy_enable,
.disable = dsi_7nm_phy_disable,
- .init = dsi_7nm_phy_init,
+ .pll_init = dsi_pll_7nm_init,
+ .save_pll_state = dsi_7nm_pll_save_state,
+ .restore_pll_state = dsi_7nm_pll_restore_state,
},
+ .min_pll_rate = 1000000000UL,
+ .max_pll_rate = 3500000000UL,
.io_start = { 0xae94400, 0xae96400 },
.num_dsi_phy = 2,
};
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
- */
-
-#include "dsi_pll.h"
-
-static int dsi_pll_enable(struct msm_dsi_pll *pll)
-{
- int i, ret = 0;
-
- /*
- * Certain PLLs do not allow VCO rate update when it is on.
- * Keep track of their status to turn on/off after set rate success.
- */
- if (unlikely(pll->pll_on))
- return 0;
-
- /* Try all enable sequences until one succeeds */
- for (i = 0; i < pll->en_seq_cnt; i++) {
- ret = pll->enable_seqs[i](pll);
- DBG("DSI PLL %s after sequence #%d",
- ret ? "unlocked" : "locked", i + 1);
- if (!ret)
- break;
- }
-
- if (ret) {
- DRM_ERROR("DSI PLL failed to lock\n");
- return ret;
- }
-
- pll->pll_on = true;
-
- return 0;
-}
-
-static void dsi_pll_disable(struct msm_dsi_pll *pll)
-{
- if (unlikely(!pll->pll_on))
- return;
-
- pll->disable_seq(pll);
-
- pll->pll_on = false;
-}
-
-/*
- * DSI PLL Helper functions
- */
-long msm_dsi_pll_helper_clk_round_rate(struct clk_hw *hw,
- unsigned long rate, unsigned long *parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
-
- if (rate < pll->min_rate)
- return pll->min_rate;
- else if (rate > pll->max_rate)
- return pll->max_rate;
- else
- return rate;
-}
-
-int msm_dsi_pll_helper_clk_prepare(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
-
- return dsi_pll_enable(pll);
-}
-
-void msm_dsi_pll_helper_clk_unprepare(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
-
- dsi_pll_disable(pll);
-}
-
-void msm_dsi_pll_helper_unregister_clks(struct platform_device *pdev,
- struct clk **clks, u32 num_clks)
-{
- of_clk_del_provider(pdev->dev.of_node);
-
- if (!num_clks || !clks)
- return;
-
- do {
- clk_unregister(clks[--num_clks]);
- clks[num_clks] = NULL;
- } while (num_clks);
-}
-
-/*
- * DSI PLL API
- */
-int msm_dsi_pll_get_clk_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider, struct clk **pixel_clk_provider)
-{
- if (pll->get_provider)
- return pll->get_provider(pll,
- byte_clk_provider,
- pixel_clk_provider);
-
- return -EINVAL;
-}
-
-void msm_dsi_pll_destroy(struct msm_dsi_pll *pll)
-{
- if (pll->destroy)
- pll->destroy(pll);
-}
-
-void msm_dsi_pll_save_state(struct msm_dsi_pll *pll)
-{
- if (pll->save_state) {
- pll->save_state(pll);
- pll->state_saved = true;
- }
-}
-
-int msm_dsi_pll_restore_state(struct msm_dsi_pll *pll)
-{
- int ret;
-
- if (pll->restore_state && pll->state_saved) {
- ret = pll->restore_state(pll);
- if (ret)
- return ret;
-
- pll->state_saved = false;
- }
-
- return 0;
-}
-
-int msm_dsi_pll_set_usecase(struct msm_dsi_pll *pll,
- enum msm_dsi_phy_usecase uc)
-{
- if (pll->set_usecase)
- return pll->set_usecase(pll, uc);
-
- return 0;
-}
-
-struct msm_dsi_pll *msm_dsi_pll_init(struct platform_device *pdev,
- enum msm_dsi_phy_type type, int id)
-{
- struct device *dev = &pdev->dev;
- struct msm_dsi_pll *pll;
-
- switch (type) {
- case MSM_DSI_PHY_28NM_HPM:
- case MSM_DSI_PHY_28NM_LP:
- pll = msm_dsi_pll_28nm_init(pdev, type, id);
- break;
- case MSM_DSI_PHY_28NM_8960:
- pll = msm_dsi_pll_28nm_8960_init(pdev, id);
- break;
- case MSM_DSI_PHY_14NM:
- pll = msm_dsi_pll_14nm_init(pdev, id);
- break;
- case MSM_DSI_PHY_10NM:
- pll = msm_dsi_pll_10nm_init(pdev, id);
- break;
- case MSM_DSI_PHY_7NM:
- case MSM_DSI_PHY_7NM_V4_1:
- pll = msm_dsi_pll_7nm_init(pdev, id);
- break;
- default:
- pll = ERR_PTR(-ENXIO);
- break;
- }
-
- if (IS_ERR(pll)) {
- DRM_DEV_ERROR(dev, "%s: failed to init DSI PLL\n", __func__);
- return pll;
- }
-
- pll->type = type;
-
- DBG("DSI:%d PLL registered", id);
-
- return pll;
-}
-
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
- */
-
-#ifndef __DSI_PLL_H__
-#define __DSI_PLL_H__
-
-#include <linux/clk-provider.h>
-#include <linux/delay.h>
-
-#include "dsi.h"
-
-#define NUM_DSI_CLOCKS_MAX 6
-#define MAX_DSI_PLL_EN_SEQS 10
-
-struct msm_dsi_pll {
- enum msm_dsi_phy_type type;
-
- struct clk_hw clk_hw;
- bool pll_on;
- bool state_saved;
-
- unsigned long min_rate;
- unsigned long max_rate;
- u32 en_seq_cnt;
-
- int (*enable_seqs[MAX_DSI_PLL_EN_SEQS])(struct msm_dsi_pll *pll);
- void (*disable_seq)(struct msm_dsi_pll *pll);
- int (*get_provider)(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider,
- struct clk **pixel_clk_provider);
- void (*destroy)(struct msm_dsi_pll *pll);
- void (*save_state)(struct msm_dsi_pll *pll);
- int (*restore_state)(struct msm_dsi_pll *pll);
- int (*set_usecase)(struct msm_dsi_pll *pll,
- enum msm_dsi_phy_usecase uc);
-};
-
-#define hw_clk_to_pll(x) container_of(x, struct msm_dsi_pll, clk_hw)
-
-static inline void pll_write(void __iomem *reg, u32 data)
-{
- msm_writel(data, reg);
-}
-
-static inline u32 pll_read(const void __iomem *reg)
-{
- return msm_readl(reg);
-}
-
-static inline void pll_write_udelay(void __iomem *reg, u32 data, u32 delay_us)
-{
- pll_write(reg, data);
- udelay(delay_us);
-}
-
-static inline void pll_write_ndelay(void __iomem *reg, u32 data, u32 delay_ns)
-{
- pll_write((reg), data);
- ndelay(delay_ns);
-}
-
-/*
- * DSI PLL Helper functions
- */
-
-/* clock callbacks */
-long msm_dsi_pll_helper_clk_round_rate(struct clk_hw *hw,
- unsigned long rate, unsigned long *parent_rate);
-int msm_dsi_pll_helper_clk_prepare(struct clk_hw *hw);
-void msm_dsi_pll_helper_clk_unprepare(struct clk_hw *hw);
-/* misc */
-void msm_dsi_pll_helper_unregister_clks(struct platform_device *pdev,
- struct clk **clks, u32 num_clks);
-
-/*
- * Initialization for Each PLL Type
- */
-#ifdef CONFIG_DRM_MSM_DSI_28NM_PHY
-struct msm_dsi_pll *msm_dsi_pll_28nm_init(struct platform_device *pdev,
- enum msm_dsi_phy_type type, int id);
-#else
-static inline struct msm_dsi_pll *msm_dsi_pll_28nm_init(
- struct platform_device *pdev, enum msm_dsi_phy_type type, int id)
-{
- return ERR_PTR(-ENODEV);
-}
-#endif
-#ifdef CONFIG_DRM_MSM_DSI_28NM_8960_PHY
-struct msm_dsi_pll *msm_dsi_pll_28nm_8960_init(struct platform_device *pdev,
- int id);
-#else
-static inline struct msm_dsi_pll *msm_dsi_pll_28nm_8960_init(
- struct platform_device *pdev, int id)
-{
- return ERR_PTR(-ENODEV);
-}
-#endif
-
-#ifdef CONFIG_DRM_MSM_DSI_14NM_PHY
-struct msm_dsi_pll *msm_dsi_pll_14nm_init(struct platform_device *pdev, int id);
-#else
-static inline struct msm_dsi_pll *
-msm_dsi_pll_14nm_init(struct platform_device *pdev, int id)
-{
- return ERR_PTR(-ENODEV);
-}
-#endif
-#ifdef CONFIG_DRM_MSM_DSI_10NM_PHY
-struct msm_dsi_pll *msm_dsi_pll_10nm_init(struct platform_device *pdev, int id);
-#else
-static inline struct msm_dsi_pll *
-msm_dsi_pll_10nm_init(struct platform_device *pdev, int id)
-{
- return ERR_PTR(-ENODEV);
-}
-#endif
-#ifdef CONFIG_DRM_MSM_DSI_7NM_PHY
-struct msm_dsi_pll *msm_dsi_pll_7nm_init(struct platform_device *pdev, int id);
-#else
-static inline struct msm_dsi_pll *
-msm_dsi_pll_7nm_init(struct platform_device *pdev, int id)
-{
- return ERR_PTR(-ENODEV);
-}
-#endif
-
-#endif /* __DSI_PLL_H__ */
-
+++ /dev/null
-/*
- * SPDX-License-Identifier: GPL-2.0
- * Copyright (c) 2018, The Linux Foundation
- */
-
-#include <linux/clk.h>
-#include <linux/clk-provider.h>
-#include <linux/iopoll.h>
-
-#include "dsi_pll.h"
-#include "dsi.xml.h"
-
-/*
- * DSI PLL 10nm - clock diagram (eg: DSI0):
- *
- * dsi0_pll_out_div_clk dsi0_pll_bit_clk
- * | |
- * | |
- * +---------+ | +----------+ | +----+
- * dsi0vco_clk ---| out_div |--o--| divl_3_0 |--o--| /8 |-- dsi0_phy_pll_out_byteclk
- * +---------+ | +----------+ | +----+
- * | |
- * | | dsi0_pll_by_2_bit_clk
- * | | |
- * | | +----+ | |\ dsi0_pclk_mux
- * | |--| /2 |--o--| \ |
- * | | +----+ | \ | +---------+
- * | --------------| |--o--| div_7_4 |-- dsi0_phy_pll_out_dsiclk
- * |------------------------------| / +---------+
- * | +-----+ | /
- * -----------| /4? |--o----------|/
- * +-----+ | |
- * | |dsiclk_sel
- * |
- * dsi0_pll_post_out_div_clk
- */
-
-#define DSI_BYTE_PLL_CLK 0
-#define DSI_PIXEL_PLL_CLK 1
-#define NUM_PROVIDED_CLKS 2
-
-#define VCO_REF_CLK_RATE 19200000
-
-struct dsi_pll_regs {
- u32 pll_prop_gain_rate;
- u32 pll_lockdet_rate;
- u32 decimal_div_start;
- u32 frac_div_start_low;
- u32 frac_div_start_mid;
- u32 frac_div_start_high;
- u32 pll_clock_inverters;
- u32 ssc_stepsize_low;
- u32 ssc_stepsize_high;
- u32 ssc_div_per_low;
- u32 ssc_div_per_high;
- u32 ssc_adjper_low;
- u32 ssc_adjper_high;
- u32 ssc_control;
-};
-
-struct dsi_pll_config {
- u32 ref_freq;
- bool div_override;
- u32 output_div;
- bool ignore_frac;
- bool disable_prescaler;
- bool enable_ssc;
- bool ssc_center;
- u32 dec_bits;
- u32 frac_bits;
- u32 lock_timer;
- u32 ssc_freq;
- u32 ssc_offset;
- u32 ssc_adj_per;
- u32 thresh_cycles;
- u32 refclk_cycles;
-};
-
-struct pll_10nm_cached_state {
- unsigned long vco_rate;
- u8 bit_clk_div;
- u8 pix_clk_div;
- u8 pll_out_div;
- u8 pll_mux;
-};
-
-struct dsi_pll_10nm {
- struct msm_dsi_pll base;
-
- int id;
- struct platform_device *pdev;
-
- void __iomem *phy_cmn_mmio;
- void __iomem *mmio;
-
- u64 vco_ref_clk_rate;
- u64 vco_current_rate;
-
- /* protects REG_DSI_10nm_PHY_CMN_CLK_CFG0 register */
- spinlock_t postdiv_lock;
-
- int vco_delay;
- struct dsi_pll_config pll_configuration;
- struct dsi_pll_regs reg_setup;
-
- /* private clocks: */
- struct clk_hw *out_div_clk_hw;
- struct clk_hw *bit_clk_hw;
- struct clk_hw *byte_clk_hw;
- struct clk_hw *by_2_bit_clk_hw;
- struct clk_hw *post_out_div_clk_hw;
- struct clk_hw *pclk_mux_hw;
- struct clk_hw *out_dsiclk_hw;
-
- /* clock-provider: */
- struct clk_hw_onecell_data *hw_data;
-
- struct pll_10nm_cached_state cached_state;
-
- enum msm_dsi_phy_usecase uc;
- struct dsi_pll_10nm *slave;
-};
-
-#define to_pll_10nm(x) container_of(x, struct dsi_pll_10nm, base)
-
-/*
- * Global list of private DSI PLL struct pointers. We need this for Dual DSI
- * mode, where the master PLL's clk_ops needs access the slave's private data
- */
-static struct dsi_pll_10nm *pll_10nm_list[DSI_MAX];
-
-static void dsi_pll_setup_config(struct dsi_pll_10nm *pll)
-{
- struct dsi_pll_config *config = &pll->pll_configuration;
-
- config->ref_freq = pll->vco_ref_clk_rate;
- config->output_div = 1;
- config->dec_bits = 8;
- config->frac_bits = 18;
- config->lock_timer = 64;
- config->ssc_freq = 31500;
- config->ssc_offset = 5000;
- config->ssc_adj_per = 2;
- config->thresh_cycles = 32;
- config->refclk_cycles = 256;
-
- config->div_override = false;
- config->ignore_frac = false;
- config->disable_prescaler = false;
-
- config->enable_ssc = false;
- config->ssc_center = 0;
-}
-
-static void dsi_pll_calc_dec_frac(struct dsi_pll_10nm *pll)
-{
- struct dsi_pll_config *config = &pll->pll_configuration;
- struct dsi_pll_regs *regs = &pll->reg_setup;
- u64 fref = pll->vco_ref_clk_rate;
- u64 pll_freq;
- u64 divider;
- u64 dec, dec_multiple;
- u32 frac;
- u64 multiplier;
-
- pll_freq = pll->vco_current_rate;
-
- if (config->disable_prescaler)
- divider = fref;
- else
- divider = fref * 2;
-
- multiplier = 1 << config->frac_bits;
- dec_multiple = div_u64(pll_freq * multiplier, divider);
- dec = div_u64_rem(dec_multiple, multiplier, &frac);
-
- if (pll_freq <= 1900000000UL)
- regs->pll_prop_gain_rate = 8;
- else if (pll_freq <= 3000000000UL)
- regs->pll_prop_gain_rate = 10;
- else
- regs->pll_prop_gain_rate = 12;
- if (pll_freq < 1100000000UL)
- regs->pll_clock_inverters = 8;
- else
- regs->pll_clock_inverters = 0;
-
- regs->pll_lockdet_rate = config->lock_timer;
- regs->decimal_div_start = dec;
- regs->frac_div_start_low = (frac & 0xff);
- regs->frac_div_start_mid = (frac & 0xff00) >> 8;
- regs->frac_div_start_high = (frac & 0x30000) >> 16;
-}
-
-#define SSC_CENTER BIT(0)
-#define SSC_EN BIT(1)
-
-static void dsi_pll_calc_ssc(struct dsi_pll_10nm *pll)
-{
- struct dsi_pll_config *config = &pll->pll_configuration;
- struct dsi_pll_regs *regs = &pll->reg_setup;
- u32 ssc_per;
- u32 ssc_mod;
- u64 ssc_step_size;
- u64 frac;
-
- if (!config->enable_ssc) {
- DBG("SSC not enabled\n");
- return;
- }
-
- ssc_per = DIV_ROUND_CLOSEST(config->ref_freq, config->ssc_freq) / 2 - 1;
- ssc_mod = (ssc_per + 1) % (config->ssc_adj_per + 1);
- ssc_per -= ssc_mod;
-
- frac = regs->frac_div_start_low |
- (regs->frac_div_start_mid << 8) |
- (regs->frac_div_start_high << 16);
- ssc_step_size = regs->decimal_div_start;
- ssc_step_size *= (1 << config->frac_bits);
- ssc_step_size += frac;
- ssc_step_size *= config->ssc_offset;
- ssc_step_size *= (config->ssc_adj_per + 1);
- ssc_step_size = div_u64(ssc_step_size, (ssc_per + 1));
- ssc_step_size = DIV_ROUND_CLOSEST_ULL(ssc_step_size, 1000000);
-
- regs->ssc_div_per_low = ssc_per & 0xFF;
- regs->ssc_div_per_high = (ssc_per & 0xFF00) >> 8;
- regs->ssc_stepsize_low = (u32)(ssc_step_size & 0xFF);
- regs->ssc_stepsize_high = (u32)((ssc_step_size & 0xFF00) >> 8);
- regs->ssc_adjper_low = config->ssc_adj_per & 0xFF;
- regs->ssc_adjper_high = (config->ssc_adj_per & 0xFF00) >> 8;
-
- regs->ssc_control = config->ssc_center ? SSC_CENTER : 0;
-
- pr_debug("SCC: Dec:%d, frac:%llu, frac_bits:%d\n",
- regs->decimal_div_start, frac, config->frac_bits);
- pr_debug("SSC: div_per:0x%X, stepsize:0x%X, adjper:0x%X\n",
- ssc_per, (u32)ssc_step_size, config->ssc_adj_per);
-}
-
-static void dsi_pll_ssc_commit(struct dsi_pll_10nm *pll)
-{
- void __iomem *base = pll->mmio;
- struct dsi_pll_regs *regs = &pll->reg_setup;
-
- if (pll->pll_configuration.enable_ssc) {
- pr_debug("SSC is enabled\n");
-
- pll_write(base + REG_DSI_10nm_PHY_PLL_SSC_STEPSIZE_LOW_1,
- regs->ssc_stepsize_low);
- pll_write(base + REG_DSI_10nm_PHY_PLL_SSC_STEPSIZE_HIGH_1,
- regs->ssc_stepsize_high);
- pll_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_PER_LOW_1,
- regs->ssc_div_per_low);
- pll_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_PER_HIGH_1,
- regs->ssc_div_per_high);
- pll_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_ADJPER_LOW_1,
- regs->ssc_adjper_low);
- pll_write(base + REG_DSI_10nm_PHY_PLL_SSC_DIV_ADJPER_HIGH_1,
- regs->ssc_adjper_high);
- pll_write(base + REG_DSI_10nm_PHY_PLL_SSC_CONTROL,
- SSC_EN | regs->ssc_control);
- }
-}
-
-static void dsi_pll_config_hzindep_reg(struct dsi_pll_10nm *pll)
-{
- void __iomem *base = pll->mmio;
-
- pll_write(base + REG_DSI_10nm_PHY_PLL_ANALOG_CONTROLS_ONE, 0x80);
- pll_write(base + REG_DSI_10nm_PHY_PLL_ANALOG_CONTROLS_TWO, 0x03);
- pll_write(base + REG_DSI_10nm_PHY_PLL_ANALOG_CONTROLS_THREE, 0x00);
- pll_write(base + REG_DSI_10nm_PHY_PLL_DSM_DIVIDER, 0x00);
- pll_write(base + REG_DSI_10nm_PHY_PLL_FEEDBACK_DIVIDER, 0x4e);
- pll_write(base + REG_DSI_10nm_PHY_PLL_CALIBRATION_SETTINGS, 0x40);
- pll_write(base + REG_DSI_10nm_PHY_PLL_BAND_SEL_CAL_SETTINGS_THREE,
- 0xba);
- pll_write(base + REG_DSI_10nm_PHY_PLL_FREQ_DETECT_SETTINGS_ONE, 0x0c);
- pll_write(base + REG_DSI_10nm_PHY_PLL_OUTDIV, 0x00);
- pll_write(base + REG_DSI_10nm_PHY_PLL_CORE_OVERRIDE, 0x00);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_DIGITAL_TIMERS_TWO, 0x08);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_PROP_GAIN_RATE_1, 0x08);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_BAND_SET_RATE_1, 0xc0);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_INT_GAIN_IFILT_BAND_1, 0xfa);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_FL_INT_GAIN_PFILT_BAND_1,
- 0x4c);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_LOCK_OVERRIDE, 0x80);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PFILT, 0x29);
- pll_write(base + REG_DSI_10nm_PHY_PLL_IFILT, 0x3f);
-}
-
-static void dsi_pll_commit(struct dsi_pll_10nm *pll)
-{
- void __iomem *base = pll->mmio;
- struct dsi_pll_regs *reg = &pll->reg_setup;
-
- pll_write(base + REG_DSI_10nm_PHY_PLL_CORE_INPUT_OVERRIDE, 0x12);
- pll_write(base + REG_DSI_10nm_PHY_PLL_DECIMAL_DIV_START_1,
- reg->decimal_div_start);
- pll_write(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_LOW_1,
- reg->frac_div_start_low);
- pll_write(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_MID_1,
- reg->frac_div_start_mid);
- pll_write(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_HIGH_1,
- reg->frac_div_start_high);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_LOCKDET_RATE_1,
- reg->pll_lockdet_rate);
- pll_write(base + REG_DSI_10nm_PHY_PLL_PLL_LOCK_DELAY, 0x06);
- pll_write(base + REG_DSI_10nm_PHY_PLL_CMODE, 0x10);
- pll_write(base + REG_DSI_10nm_PHY_PLL_CLOCK_INVERTERS,
- reg->pll_clock_inverters);
-}
-
-static int dsi_pll_10nm_vco_set_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
-
- DBG("DSI PLL%d rate=%lu, parent's=%lu", pll_10nm->id, rate,
- parent_rate);
-
- pll_10nm->vco_current_rate = rate;
- pll_10nm->vco_ref_clk_rate = VCO_REF_CLK_RATE;
-
- dsi_pll_setup_config(pll_10nm);
-
- dsi_pll_calc_dec_frac(pll_10nm);
-
- dsi_pll_calc_ssc(pll_10nm);
-
- dsi_pll_commit(pll_10nm);
-
- dsi_pll_config_hzindep_reg(pll_10nm);
-
- dsi_pll_ssc_commit(pll_10nm);
-
- /* flush, ensure all register writes are done*/
- wmb();
-
- return 0;
-}
-
-static int dsi_pll_10nm_lock_status(struct dsi_pll_10nm *pll)
-{
- struct device *dev = &pll->pdev->dev;
- int rc;
- u32 status = 0;
- u32 const delay_us = 100;
- u32 const timeout_us = 5000;
-
- rc = readl_poll_timeout_atomic(pll->mmio +
- REG_DSI_10nm_PHY_PLL_COMMON_STATUS_ONE,
- status,
- ((status & BIT(0)) > 0),
- delay_us,
- timeout_us);
- if (rc)
- DRM_DEV_ERROR(dev, "DSI PLL(%d) lock failed, status=0x%08x\n",
- pll->id, status);
-
- return rc;
-}
-
-static void dsi_pll_disable_pll_bias(struct dsi_pll_10nm *pll)
-{
- u32 data = pll_read(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CTRL_0);
-
- pll_write(pll->mmio + REG_DSI_10nm_PHY_PLL_SYSTEM_MUXES, 0);
- pll_write(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CTRL_0,
- data & ~BIT(5));
- ndelay(250);
-}
-
-static void dsi_pll_enable_pll_bias(struct dsi_pll_10nm *pll)
-{
- u32 data = pll_read(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CTRL_0);
-
- pll_write(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CTRL_0,
- data | BIT(5));
- pll_write(pll->mmio + REG_DSI_10nm_PHY_PLL_SYSTEM_MUXES, 0xc0);
- ndelay(250);
-}
-
-static void dsi_pll_disable_global_clk(struct dsi_pll_10nm *pll)
-{
- u32 data;
-
- data = pll_read(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
- pll_write(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CLK_CFG1,
- data & ~BIT(5));
-}
-
-static void dsi_pll_enable_global_clk(struct dsi_pll_10nm *pll)
-{
- u32 data;
-
- data = pll_read(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
- pll_write(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_CLK_CFG1,
- data | BIT(5));
-}
-
-static int dsi_pll_10nm_vco_prepare(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
- struct device *dev = &pll_10nm->pdev->dev;
- int rc;
-
- dsi_pll_enable_pll_bias(pll_10nm);
- if (pll_10nm->slave)
- dsi_pll_enable_pll_bias(pll_10nm->slave);
-
- rc = dsi_pll_10nm_vco_set_rate(hw,pll_10nm->vco_current_rate, 0);
- if (rc) {
- DRM_DEV_ERROR(dev, "vco_set_rate failed, rc=%d\n", rc);
- return rc;
- }
-
- /* Start PLL */
- pll_write(pll_10nm->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_PLL_CNTRL,
- 0x01);
-
- /*
- * ensure all PLL configurations are written prior to checking
- * for PLL lock.
- */
- wmb();
-
- /* Check for PLL lock */
- rc = dsi_pll_10nm_lock_status(pll_10nm);
- if (rc) {
- DRM_DEV_ERROR(dev, "PLL(%d) lock failed\n", pll_10nm->id);
- goto error;
- }
-
- pll->pll_on = true;
-
- dsi_pll_enable_global_clk(pll_10nm);
- if (pll_10nm->slave)
- dsi_pll_enable_global_clk(pll_10nm->slave);
-
- pll_write(pll_10nm->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_RBUF_CTRL,
- 0x01);
- if (pll_10nm->slave)
- pll_write(pll_10nm->slave->phy_cmn_mmio +
- REG_DSI_10nm_PHY_CMN_RBUF_CTRL, 0x01);
-
-error:
- return rc;
-}
-
-static void dsi_pll_disable_sub(struct dsi_pll_10nm *pll)
-{
- pll_write(pll->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_RBUF_CTRL, 0);
- dsi_pll_disable_pll_bias(pll);
-}
-
-static void dsi_pll_10nm_vco_unprepare(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
-
- /*
- * To avoid any stray glitches while abruptly powering down the PLL
- * make sure to gate the clock using the clock enable bit before
- * powering down the PLL
- */
- dsi_pll_disable_global_clk(pll_10nm);
- pll_write(pll_10nm->phy_cmn_mmio + REG_DSI_10nm_PHY_CMN_PLL_CNTRL, 0);
- dsi_pll_disable_sub(pll_10nm);
- if (pll_10nm->slave) {
- dsi_pll_disable_global_clk(pll_10nm->slave);
- dsi_pll_disable_sub(pll_10nm->slave);
- }
- /* flush, ensure all register writes are done */
- wmb();
- pll->pll_on = false;
-}
-
-static unsigned long dsi_pll_10nm_vco_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
- struct dsi_pll_config *config = &pll_10nm->pll_configuration;
- void __iomem *base = pll_10nm->mmio;
- u64 ref_clk = pll_10nm->vco_ref_clk_rate;
- u64 vco_rate = 0x0;
- u64 multiplier;
- u32 frac;
- u32 dec;
- u64 pll_freq, tmp64;
-
- dec = pll_read(base + REG_DSI_10nm_PHY_PLL_DECIMAL_DIV_START_1);
- dec &= 0xff;
-
- frac = pll_read(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_LOW_1);
- frac |= ((pll_read(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_MID_1) &
- 0xff) << 8);
- frac |= ((pll_read(base + REG_DSI_10nm_PHY_PLL_FRAC_DIV_START_HIGH_1) &
- 0x3) << 16);
-
- /*
- * TODO:
- * 1. Assumes prescaler is disabled
- */
- multiplier = 1 << config->frac_bits;
- pll_freq = dec * (ref_clk * 2);
- tmp64 = (ref_clk * 2 * frac);
- pll_freq += div_u64(tmp64, multiplier);
-
- vco_rate = pll_freq;
-
- DBG("DSI PLL%d returning vco rate = %lu, dec = %x, frac = %x",
- pll_10nm->id, (unsigned long)vco_rate, dec, frac);
-
- return (unsigned long)vco_rate;
-}
-
-static const struct clk_ops clk_ops_dsi_pll_10nm_vco = {
- .round_rate = msm_dsi_pll_helper_clk_round_rate,
- .set_rate = dsi_pll_10nm_vco_set_rate,
- .recalc_rate = dsi_pll_10nm_vco_recalc_rate,
- .prepare = dsi_pll_10nm_vco_prepare,
- .unprepare = dsi_pll_10nm_vco_unprepare,
-};
-
-/*
- * PLL Callbacks
- */
-
-static void dsi_pll_10nm_save_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
- struct pll_10nm_cached_state *cached = &pll_10nm->cached_state;
- void __iomem *phy_base = pll_10nm->phy_cmn_mmio;
- u32 cmn_clk_cfg0, cmn_clk_cfg1;
-
- cached->pll_out_div = pll_read(pll_10nm->mmio +
- REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE);
- cached->pll_out_div &= 0x3;
-
- cmn_clk_cfg0 = pll_read(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG0);
- cached->bit_clk_div = cmn_clk_cfg0 & 0xf;
- cached->pix_clk_div = (cmn_clk_cfg0 & 0xf0) >> 4;
-
- cmn_clk_cfg1 = pll_read(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
- cached->pll_mux = cmn_clk_cfg1 & 0x3;
-
- DBG("DSI PLL%d outdiv %x bit_clk_div %x pix_clk_div %x pll_mux %x",
- pll_10nm->id, cached->pll_out_div, cached->bit_clk_div,
- cached->pix_clk_div, cached->pll_mux);
-}
-
-static int dsi_pll_10nm_restore_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
- struct pll_10nm_cached_state *cached = &pll_10nm->cached_state;
- void __iomem *phy_base = pll_10nm->phy_cmn_mmio;
- u32 val;
- int ret;
-
- val = pll_read(pll_10nm->mmio + REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE);
- val &= ~0x3;
- val |= cached->pll_out_div;
- pll_write(pll_10nm->mmio + REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE, val);
-
- pll_write(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG0,
- cached->bit_clk_div | (cached->pix_clk_div << 4));
-
- val = pll_read(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG1);
- val &= ~0x3;
- val |= cached->pll_mux;
- pll_write(phy_base + REG_DSI_10nm_PHY_CMN_CLK_CFG1, val);
-
- ret = dsi_pll_10nm_vco_set_rate(&pll->clk_hw, pll_10nm->vco_current_rate, pll_10nm->vco_ref_clk_rate);
- if (ret) {
- DRM_DEV_ERROR(&pll_10nm->pdev->dev,
- "restore vco rate failed. ret=%d\n", ret);
- return ret;
- }
-
- DBG("DSI PLL%d", pll_10nm->id);
-
- return 0;
-}
-
-static int dsi_pll_10nm_set_usecase(struct msm_dsi_pll *pll,
- enum msm_dsi_phy_usecase uc)
-{
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
- void __iomem *base = pll_10nm->phy_cmn_mmio;
- u32 data = 0x0; /* internal PLL */
-
- DBG("DSI PLL%d", pll_10nm->id);
-
- switch (uc) {
- case MSM_DSI_PHY_STANDALONE:
- break;
- case MSM_DSI_PHY_MASTER:
- pll_10nm->slave = pll_10nm_list[(pll_10nm->id + 1) % DSI_MAX];
- break;
- case MSM_DSI_PHY_SLAVE:
- data = 0x1; /* external PLL */
- break;
- default:
- return -EINVAL;
- }
-
- /* set PLL src */
- pll_write(base + REG_DSI_10nm_PHY_CMN_CLK_CFG1, (data << 2));
-
- pll_10nm->uc = uc;
-
- return 0;
-}
-
-static int dsi_pll_10nm_get_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider,
- struct clk **pixel_clk_provider)
-{
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
- struct clk_hw_onecell_data *hw_data = pll_10nm->hw_data;
-
- DBG("DSI PLL%d", pll_10nm->id);
-
- if (byte_clk_provider)
- *byte_clk_provider = hw_data->hws[DSI_BYTE_PLL_CLK]->clk;
- if (pixel_clk_provider)
- *pixel_clk_provider = hw_data->hws[DSI_PIXEL_PLL_CLK]->clk;
-
- return 0;
-}
-
-static void dsi_pll_10nm_destroy(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_10nm *pll_10nm = to_pll_10nm(pll);
- struct device *dev = &pll_10nm->pdev->dev;
-
- DBG("DSI PLL%d", pll_10nm->id);
- of_clk_del_provider(dev->of_node);
-
- clk_hw_unregister_divider(pll_10nm->out_dsiclk_hw);
- clk_hw_unregister_mux(pll_10nm->pclk_mux_hw);
- clk_hw_unregister_fixed_factor(pll_10nm->post_out_div_clk_hw);
- clk_hw_unregister_fixed_factor(pll_10nm->by_2_bit_clk_hw);
- clk_hw_unregister_fixed_factor(pll_10nm->byte_clk_hw);
- clk_hw_unregister_divider(pll_10nm->bit_clk_hw);
- clk_hw_unregister_divider(pll_10nm->out_div_clk_hw);
- clk_hw_unregister(&pll_10nm->base.clk_hw);
-}
-
-/*
- * The post dividers and mux clocks are created using the standard divider and
- * mux API. Unlike the 14nm PHY, the slave PLL doesn't need its dividers/mux
- * state to follow the master PLL's divider/mux state. Therefore, we don't
- * require special clock ops that also configure the slave PLL registers
- */
-static int pll_10nm_register(struct dsi_pll_10nm *pll_10nm)
-{
- char clk_name[32], parent[32], vco_name[32];
- char parent2[32], parent3[32], parent4[32];
- struct clk_init_data vco_init = {
- .parent_names = (const char *[]){ "xo" },
- .num_parents = 1,
- .name = vco_name,
- .flags = CLK_IGNORE_UNUSED,
- .ops = &clk_ops_dsi_pll_10nm_vco,
- };
- struct device *dev = &pll_10nm->pdev->dev;
- struct clk_hw_onecell_data *hw_data;
- struct clk_hw *hw;
- int ret;
-
- DBG("DSI%d", pll_10nm->id);
-
- hw_data = devm_kzalloc(dev, sizeof(*hw_data) +
- NUM_PROVIDED_CLKS * sizeof(struct clk_hw *),
- GFP_KERNEL);
- if (!hw_data)
- return -ENOMEM;
-
- snprintf(vco_name, 32, "dsi%dvco_clk", pll_10nm->id);
- pll_10nm->base.clk_hw.init = &vco_init;
-
- ret = clk_hw_register(dev, &pll_10nm->base.clk_hw);
- if (ret)
- return ret;
-
- snprintf(clk_name, 32, "dsi%d_pll_out_div_clk", pll_10nm->id);
- snprintf(parent, 32, "dsi%dvco_clk", pll_10nm->id);
-
- hw = clk_hw_register_divider(dev, clk_name,
- parent, CLK_SET_RATE_PARENT,
- pll_10nm->mmio +
- REG_DSI_10nm_PHY_PLL_PLL_OUTDIV_RATE,
- 0, 2, CLK_DIVIDER_POWER_OF_TWO, NULL);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_base_clk_hw;
- }
-
- pll_10nm->out_div_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_pll_bit_clk", pll_10nm->id);
- snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_10nm->id);
-
- /* BIT CLK: DIV_CTRL_3_0 */
- hw = clk_hw_register_divider(dev, clk_name, parent,
- CLK_SET_RATE_PARENT,
- pll_10nm->phy_cmn_mmio +
- REG_DSI_10nm_PHY_CMN_CLK_CFG0,
- 0, 4, CLK_DIVIDER_ONE_BASED,
- &pll_10nm->postdiv_lock);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_out_div_clk_hw;
- }
-
- pll_10nm->bit_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_phy_pll_out_byteclk", pll_10nm->id);
- snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_10nm->id);
-
- /* DSI Byte clock = VCO_CLK / OUT_DIV / BIT_DIV / 8 */
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent,
- CLK_SET_RATE_PARENT, 1, 8);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_bit_clk_hw;
- }
-
- pll_10nm->byte_clk_hw = hw;
- hw_data->hws[DSI_BYTE_PLL_CLK] = hw;
-
- snprintf(clk_name, 32, "dsi%d_pll_by_2_bit_clk", pll_10nm->id);
- snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_10nm->id);
-
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent,
- 0, 1, 2);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_byte_clk_hw;
- }
-
- pll_10nm->by_2_bit_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_pll_post_out_div_clk", pll_10nm->id);
- snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_10nm->id);
-
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent,
- 0, 1, 4);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_by_2_bit_clk_hw;
- }
-
- pll_10nm->post_out_div_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_pclk_mux", pll_10nm->id);
- snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_10nm->id);
- snprintf(parent2, 32, "dsi%d_pll_by_2_bit_clk", pll_10nm->id);
- snprintf(parent3, 32, "dsi%d_pll_out_div_clk", pll_10nm->id);
- snprintf(parent4, 32, "dsi%d_pll_post_out_div_clk", pll_10nm->id);
-
- hw = clk_hw_register_mux(dev, clk_name,
- ((const char *[]){
- parent, parent2, parent3, parent4
- }), 4, 0, pll_10nm->phy_cmn_mmio +
- REG_DSI_10nm_PHY_CMN_CLK_CFG1,
- 0, 2, 0, NULL);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_post_out_div_clk_hw;
- }
-
- pll_10nm->pclk_mux_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_phy_pll_out_dsiclk", pll_10nm->id);
- snprintf(parent, 32, "dsi%d_pclk_mux", pll_10nm->id);
-
- /* PIX CLK DIV : DIV_CTRL_7_4*/
- hw = clk_hw_register_divider(dev, clk_name, parent,
- 0, pll_10nm->phy_cmn_mmio +
- REG_DSI_10nm_PHY_CMN_CLK_CFG0,
- 4, 4, CLK_DIVIDER_ONE_BASED,
- &pll_10nm->postdiv_lock);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_pclk_mux_hw;
- }
-
- pll_10nm->out_dsiclk_hw = hw;
- hw_data->hws[DSI_PIXEL_PLL_CLK] = hw;
-
- hw_data->num = NUM_PROVIDED_CLKS;
- pll_10nm->hw_data = hw_data;
-
- ret = of_clk_add_hw_provider(dev->of_node, of_clk_hw_onecell_get,
- pll_10nm->hw_data);
- if (ret) {
- DRM_DEV_ERROR(dev, "failed to register clk provider: %d\n", ret);
- goto err_dsiclk_hw;
- }
-
- return 0;
-
-err_dsiclk_hw:
- clk_hw_unregister_divider(pll_10nm->out_dsiclk_hw);
-err_pclk_mux_hw:
- clk_hw_unregister_mux(pll_10nm->pclk_mux_hw);
-err_post_out_div_clk_hw:
- clk_hw_unregister_fixed_factor(pll_10nm->post_out_div_clk_hw);
-err_by_2_bit_clk_hw:
- clk_hw_unregister_fixed_factor(pll_10nm->by_2_bit_clk_hw);
-err_byte_clk_hw:
- clk_hw_unregister_fixed_factor(pll_10nm->byte_clk_hw);
-err_bit_clk_hw:
- clk_hw_unregister_divider(pll_10nm->bit_clk_hw);
-err_out_div_clk_hw:
- clk_hw_unregister_divider(pll_10nm->out_div_clk_hw);
-err_base_clk_hw:
- clk_hw_unregister(&pll_10nm->base.clk_hw);
-
- return ret;
-}
-
-struct msm_dsi_pll *msm_dsi_pll_10nm_init(struct platform_device *pdev, int id)
-{
- struct dsi_pll_10nm *pll_10nm;
- struct msm_dsi_pll *pll;
- int ret;
-
- pll_10nm = devm_kzalloc(&pdev->dev, sizeof(*pll_10nm), GFP_KERNEL);
- if (!pll_10nm)
- return ERR_PTR(-ENOMEM);
-
- DBG("DSI PLL%d", id);
-
- pll_10nm->pdev = pdev;
- pll_10nm->id = id;
- pll_10nm_list[id] = pll_10nm;
-
- pll_10nm->phy_cmn_mmio = msm_ioremap(pdev, "dsi_phy", "DSI_PHY");
- if (IS_ERR_OR_NULL(pll_10nm->phy_cmn_mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "failed to map CMN PHY base\n");
- return ERR_PTR(-ENOMEM);
- }
-
- pll_10nm->mmio = msm_ioremap(pdev, "dsi_pll", "DSI_PLL");
- if (IS_ERR_OR_NULL(pll_10nm->mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "failed to map PLL base\n");
- return ERR_PTR(-ENOMEM);
- }
-
- spin_lock_init(&pll_10nm->postdiv_lock);
-
- pll = &pll_10nm->base;
- pll->min_rate = 1000000000UL;
- pll->max_rate = 3500000000UL;
- pll->get_provider = dsi_pll_10nm_get_provider;
- pll->destroy = dsi_pll_10nm_destroy;
- pll->save_state = dsi_pll_10nm_save_state;
- pll->restore_state = dsi_pll_10nm_restore_state;
- pll->set_usecase = dsi_pll_10nm_set_usecase;
-
- pll_10nm->vco_delay = 1;
-
- ret = pll_10nm_register(pll_10nm);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
- return ERR_PTR(ret);
- }
-
- /* TODO: Remove this when we have proper display handover support */
- msm_dsi_pll_save_state(pll);
-
- return pll;
-}
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2016, The Linux Foundation. All rights reserved.
- */
-
-#include <linux/clk.h>
-#include <linux/clk-provider.h>
-
-#include "dsi_pll.h"
-#include "dsi.xml.h"
-
-/*
- * DSI PLL 14nm - clock diagram (eg: DSI0):
- *
- * dsi0n1_postdiv_clk
- * |
- * |
- * +----+ | +----+
- * dsi0vco_clk ---| n1 |--o--| /8 |-- dsi0pllbyte
- * +----+ | +----+
- * | dsi0n1_postdivby2_clk
- * | +----+ |
- * o---| /2 |--o--|\
- * | +----+ | \ +----+
- * | | |--| n2 |-- dsi0pll
- * o--------------| / +----+
- * |/
- */
-
-#define POLL_MAX_READS 15
-#define POLL_TIMEOUT_US 1000
-
-#define NUM_PROVIDED_CLKS 2
-
-#define VCO_REF_CLK_RATE 19200000
-#define VCO_MIN_RATE 1300000000UL
-#define VCO_MAX_RATE 2600000000UL
-
-#define DSI_BYTE_PLL_CLK 0
-#define DSI_PIXEL_PLL_CLK 1
-
-#define DSI_PLL_DEFAULT_VCO_POSTDIV 1
-
-struct dsi_pll_input {
- u32 fref; /* reference clk */
- u32 fdata; /* bit clock rate */
- u32 dsiclk_sel; /* Mux configuration (see diagram) */
- u32 ssc_en; /* SSC enable/disable */
- u32 ldo_en;
-
- /* fixed params */
- u32 refclk_dbler_en;
- u32 vco_measure_time;
- u32 kvco_measure_time;
- u32 bandgap_timer;
- u32 pll_wakeup_timer;
- u32 plllock_cnt;
- u32 plllock_rng;
- u32 ssc_center;
- u32 ssc_adj_period;
- u32 ssc_spread;
- u32 ssc_freq;
- u32 pll_ie_trim;
- u32 pll_ip_trim;
- u32 pll_iptat_trim;
- u32 pll_cpcset_cur;
- u32 pll_cpmset_cur;
-
- u32 pll_icpmset;
- u32 pll_icpcset;
-
- u32 pll_icpmset_p;
- u32 pll_icpmset_m;
-
- u32 pll_icpcset_p;
- u32 pll_icpcset_m;
-
- u32 pll_lpf_res1;
- u32 pll_lpf_cap1;
- u32 pll_lpf_cap2;
- u32 pll_c3ctrl;
- u32 pll_r3ctrl;
-};
-
-struct dsi_pll_output {
- u32 pll_txclk_en;
- u32 dec_start;
- u32 div_frac_start;
- u32 ssc_period;
- u32 ssc_step_size;
- u32 plllock_cmp;
- u32 pll_vco_div_ref;
- u32 pll_vco_count;
- u32 pll_kvco_div_ref;
- u32 pll_kvco_count;
- u32 pll_misc1;
- u32 pll_lpf2_postdiv;
- u32 pll_resetsm_cntrl;
- u32 pll_resetsm_cntrl2;
- u32 pll_resetsm_cntrl5;
- u32 pll_kvco_code;
-
- u32 cmn_clk_cfg0;
- u32 cmn_clk_cfg1;
- u32 cmn_ldo_cntrl;
-
- u32 pll_postdiv;
- u32 fcvo;
-};
-
-struct pll_14nm_cached_state {
- unsigned long vco_rate;
- u8 n2postdiv;
- u8 n1postdiv;
-};
-
-struct dsi_pll_14nm {
- struct msm_dsi_pll base;
-
- int id;
- struct platform_device *pdev;
-
- void __iomem *phy_cmn_mmio;
- void __iomem *mmio;
-
- int vco_delay;
-
- struct dsi_pll_input in;
- struct dsi_pll_output out;
-
- /* protects REG_DSI_14nm_PHY_CMN_CLK_CFG0 register */
- spinlock_t postdiv_lock;
-
- u64 vco_current_rate;
- u64 vco_ref_clk_rate;
-
- /* private clocks: */
- struct clk_hw *hws[NUM_DSI_CLOCKS_MAX];
- u32 num_hws;
-
- /* clock-provider: */
- struct clk_hw_onecell_data *hw_data;
-
- struct pll_14nm_cached_state cached_state;
-
- enum msm_dsi_phy_usecase uc;
- struct dsi_pll_14nm *slave;
-};
-
-#define to_pll_14nm(x) container_of(x, struct dsi_pll_14nm, base)
-
-/*
- * Private struct for N1/N2 post-divider clocks. These clocks are similar to
- * the generic clk_divider class of clocks. The only difference is that it
- * also sets the slave DSI PLL's post-dividers if in Dual DSI mode
- */
-struct dsi_pll_14nm_postdiv {
- struct clk_hw hw;
-
- /* divider params */
- u8 shift;
- u8 width;
- u8 flags; /* same flags as used by clk_divider struct */
-
- struct dsi_pll_14nm *pll;
-};
-
-#define to_pll_14nm_postdiv(_hw) container_of(_hw, struct dsi_pll_14nm_postdiv, hw)
-
-/*
- * Global list of private DSI PLL struct pointers. We need this for Dual DSI
- * mode, where the master PLL's clk_ops needs access the slave's private data
- */
-static struct dsi_pll_14nm *pll_14nm_list[DSI_MAX];
-
-static bool pll_14nm_poll_for_ready(struct dsi_pll_14nm *pll_14nm,
- u32 nb_tries, u32 timeout_us)
-{
- bool pll_locked = false;
- void __iomem *base = pll_14nm->mmio;
- u32 tries, val;
-
- tries = nb_tries;
- while (tries--) {
- val = pll_read(base +
- REG_DSI_14nm_PHY_PLL_RESET_SM_READY_STATUS);
- pll_locked = !!(val & BIT(5));
-
- if (pll_locked)
- break;
-
- udelay(timeout_us);
- }
-
- if (!pll_locked) {
- tries = nb_tries;
- while (tries--) {
- val = pll_read(base +
- REG_DSI_14nm_PHY_PLL_RESET_SM_READY_STATUS);
- pll_locked = !!(val & BIT(0));
-
- if (pll_locked)
- break;
-
- udelay(timeout_us);
- }
- }
-
- DBG("DSI PLL is %slocked", pll_locked ? "" : "*not* ");
-
- return pll_locked;
-}
-
-static void dsi_pll_14nm_input_init(struct dsi_pll_14nm *pll)
-{
- pll->in.fref = pll->vco_ref_clk_rate;
- pll->in.fdata = 0;
- pll->in.dsiclk_sel = 1; /* Use the /2 path in Mux */
- pll->in.ldo_en = 0; /* disabled for now */
-
- /* fixed input */
- pll->in.refclk_dbler_en = 0;
- pll->in.vco_measure_time = 5;
- pll->in.kvco_measure_time = 5;
- pll->in.bandgap_timer = 4;
- pll->in.pll_wakeup_timer = 5;
- pll->in.plllock_cnt = 1;
- pll->in.plllock_rng = 0;
-
- /*
- * SSC is enabled by default. We might need DT props for configuring
- * some SSC params like PPM and center/down spread etc.
- */
- pll->in.ssc_en = 1;
- pll->in.ssc_center = 0; /* down spread by default */
- pll->in.ssc_spread = 5; /* PPM / 1000 */
- pll->in.ssc_freq = 31500; /* default recommended */
- pll->in.ssc_adj_period = 37;
-
- pll->in.pll_ie_trim = 4;
- pll->in.pll_ip_trim = 4;
- pll->in.pll_cpcset_cur = 1;
- pll->in.pll_cpmset_cur = 1;
- pll->in.pll_icpmset = 4;
- pll->in.pll_icpcset = 4;
- pll->in.pll_icpmset_p = 0;
- pll->in.pll_icpmset_m = 0;
- pll->in.pll_icpcset_p = 0;
- pll->in.pll_icpcset_m = 0;
- pll->in.pll_lpf_res1 = 3;
- pll->in.pll_lpf_cap1 = 11;
- pll->in.pll_lpf_cap2 = 1;
- pll->in.pll_iptat_trim = 7;
- pll->in.pll_c3ctrl = 2;
- pll->in.pll_r3ctrl = 1;
-}
-
-#define CEIL(x, y) (((x) + ((y) - 1)) / (y))
-
-static void pll_14nm_ssc_calc(struct dsi_pll_14nm *pll)
-{
- u32 period, ssc_period;
- u32 ref, rem;
- u64 step_size;
-
- DBG("vco=%lld ref=%lld", pll->vco_current_rate, pll->vco_ref_clk_rate);
-
- ssc_period = pll->in.ssc_freq / 500;
- period = (u32)pll->vco_ref_clk_rate / 1000;
- ssc_period = CEIL(period, ssc_period);
- ssc_period -= 1;
- pll->out.ssc_period = ssc_period;
-
- DBG("ssc freq=%d spread=%d period=%d", pll->in.ssc_freq,
- pll->in.ssc_spread, pll->out.ssc_period);
-
- step_size = (u32)pll->vco_current_rate;
- ref = pll->vco_ref_clk_rate;
- ref /= 1000;
- step_size = div_u64(step_size, ref);
- step_size <<= 20;
- step_size = div_u64(step_size, 1000);
- step_size *= pll->in.ssc_spread;
- step_size = div_u64(step_size, 1000);
- step_size *= (pll->in.ssc_adj_period + 1);
-
- rem = 0;
- step_size = div_u64_rem(step_size, ssc_period + 1, &rem);
- if (rem)
- step_size++;
-
- DBG("step_size=%lld", step_size);
-
- step_size &= 0x0ffff; /* take lower 16 bits */
-
- pll->out.ssc_step_size = step_size;
-}
-
-static void pll_14nm_dec_frac_calc(struct dsi_pll_14nm *pll)
-{
- struct dsi_pll_input *pin = &pll->in;
- struct dsi_pll_output *pout = &pll->out;
- u64 multiplier = BIT(20);
- u64 dec_start_multiple, dec_start, pll_comp_val;
- u32 duration, div_frac_start;
- u64 vco_clk_rate = pll->vco_current_rate;
- u64 fref = pll->vco_ref_clk_rate;
-
- DBG("vco_clk_rate=%lld ref_clk_rate=%lld", vco_clk_rate, fref);
-
- dec_start_multiple = div_u64(vco_clk_rate * multiplier, fref);
- div_u64_rem(dec_start_multiple, multiplier, &div_frac_start);
-
- dec_start = div_u64(dec_start_multiple, multiplier);
-
- pout->dec_start = (u32)dec_start;
- pout->div_frac_start = div_frac_start;
-
- if (pin->plllock_cnt == 0)
- duration = 1024;
- else if (pin->plllock_cnt == 1)
- duration = 256;
- else if (pin->plllock_cnt == 2)
- duration = 128;
- else
- duration = 32;
-
- pll_comp_val = duration * dec_start_multiple;
- pll_comp_val = div_u64(pll_comp_val, multiplier);
- do_div(pll_comp_val, 10);
-
- pout->plllock_cmp = (u32)pll_comp_val;
-
- pout->pll_txclk_en = 1;
- pout->cmn_ldo_cntrl = 0x3c;
-}
-
-static u32 pll_14nm_kvco_slop(u32 vrate)
-{
- u32 slop = 0;
-
- if (vrate > VCO_MIN_RATE && vrate <= 1800000000UL)
- slop = 600;
- else if (vrate > 1800000000UL && vrate < 2300000000UL)
- slop = 400;
- else if (vrate > 2300000000UL && vrate < VCO_MAX_RATE)
- slop = 280;
-
- return slop;
-}
-
-static void pll_14nm_calc_vco_count(struct dsi_pll_14nm *pll)
-{
- struct dsi_pll_input *pin = &pll->in;
- struct dsi_pll_output *pout = &pll->out;
- u64 vco_clk_rate = pll->vco_current_rate;
- u64 fref = pll->vco_ref_clk_rate;
- u64 data;
- u32 cnt;
-
- data = fref * pin->vco_measure_time;
- do_div(data, 1000000);
- data &= 0x03ff; /* 10 bits */
- data -= 2;
- pout->pll_vco_div_ref = data;
-
- data = div_u64(vco_clk_rate, 1000000); /* unit is Mhz */
- data *= pin->vco_measure_time;
- do_div(data, 10);
- pout->pll_vco_count = data;
-
- data = fref * pin->kvco_measure_time;
- do_div(data, 1000000);
- data &= 0x03ff; /* 10 bits */
- data -= 1;
- pout->pll_kvco_div_ref = data;
-
- cnt = pll_14nm_kvco_slop(vco_clk_rate);
- cnt *= 2;
- cnt /= 100;
- cnt *= pin->kvco_measure_time;
- pout->pll_kvco_count = cnt;
-
- pout->pll_misc1 = 16;
- pout->pll_resetsm_cntrl = 48;
- pout->pll_resetsm_cntrl2 = pin->bandgap_timer << 3;
- pout->pll_resetsm_cntrl5 = pin->pll_wakeup_timer;
- pout->pll_kvco_code = 0;
-}
-
-static void pll_db_commit_ssc(struct dsi_pll_14nm *pll)
-{
- void __iomem *base = pll->mmio;
- struct dsi_pll_input *pin = &pll->in;
- struct dsi_pll_output *pout = &pll->out;
- u8 data;
-
- data = pin->ssc_adj_period;
- data &= 0x0ff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_SSC_ADJ_PER1, data);
- data = (pin->ssc_adj_period >> 8);
- data &= 0x03;
- pll_write(base + REG_DSI_14nm_PHY_PLL_SSC_ADJ_PER2, data);
-
- data = pout->ssc_period;
- data &= 0x0ff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_SSC_PER1, data);
- data = (pout->ssc_period >> 8);
- data &= 0x0ff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_SSC_PER2, data);
-
- data = pout->ssc_step_size;
- data &= 0x0ff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_SSC_STEP_SIZE1, data);
- data = (pout->ssc_step_size >> 8);
- data &= 0x0ff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_SSC_STEP_SIZE2, data);
-
- data = (pin->ssc_center & 0x01);
- data <<= 1;
- data |= 0x01; /* enable */
- pll_write(base + REG_DSI_14nm_PHY_PLL_SSC_EN_CENTER, data);
-
- wmb(); /* make sure register committed */
-}
-
-static void pll_db_commit_common(struct dsi_pll_14nm *pll,
- struct dsi_pll_input *pin,
- struct dsi_pll_output *pout)
-{
- void __iomem *base = pll->mmio;
- u8 data;
-
- /* confgiure the non frequency dependent pll registers */
- data = 0;
- pll_write(base + REG_DSI_14nm_PHY_PLL_SYSCLK_EN_RESET, data);
-
- data = pout->pll_txclk_en;
- pll_write(base + REG_DSI_14nm_PHY_PLL_TXCLK_EN, data);
-
- data = pout->pll_resetsm_cntrl;
- pll_write(base + REG_DSI_14nm_PHY_PLL_RESETSM_CNTRL, data);
- data = pout->pll_resetsm_cntrl2;
- pll_write(base + REG_DSI_14nm_PHY_PLL_RESETSM_CNTRL2, data);
- data = pout->pll_resetsm_cntrl5;
- pll_write(base + REG_DSI_14nm_PHY_PLL_RESETSM_CNTRL5, data);
-
- data = pout->pll_vco_div_ref & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_VCO_DIV_REF1, data);
- data = (pout->pll_vco_div_ref >> 8) & 0x3;
- pll_write(base + REG_DSI_14nm_PHY_PLL_VCO_DIV_REF2, data);
-
- data = pout->pll_kvco_div_ref & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_KVCO_DIV_REF1, data);
- data = (pout->pll_kvco_div_ref >> 8) & 0x3;
- pll_write(base + REG_DSI_14nm_PHY_PLL_KVCO_DIV_REF2, data);
-
- data = pout->pll_misc1;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_MISC1, data);
-
- data = pin->pll_ie_trim;
- pll_write(base + REG_DSI_14nm_PHY_PLL_IE_TRIM, data);
-
- data = pin->pll_ip_trim;
- pll_write(base + REG_DSI_14nm_PHY_PLL_IP_TRIM, data);
-
- data = pin->pll_cpmset_cur << 3 | pin->pll_cpcset_cur;
- pll_write(base + REG_DSI_14nm_PHY_PLL_CP_SET_CUR, data);
-
- data = pin->pll_icpcset_p << 3 | pin->pll_icpcset_m;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_ICPCSET, data);
-
- data = pin->pll_icpmset_p << 3 | pin->pll_icpcset_m;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_ICPMSET, data);
-
- data = pin->pll_icpmset << 3 | pin->pll_icpcset;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_ICP_SET, data);
-
- data = pin->pll_lpf_cap2 << 4 | pin->pll_lpf_cap1;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_LPF1, data);
-
- data = pin->pll_iptat_trim;
- pll_write(base + REG_DSI_14nm_PHY_PLL_IPTAT_TRIM, data);
-
- data = pin->pll_c3ctrl | pin->pll_r3ctrl << 4;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_CRCTRL, data);
-}
-
-static void pll_14nm_software_reset(struct dsi_pll_14nm *pll_14nm)
-{
- void __iomem *cmn_base = pll_14nm->phy_cmn_mmio;
-
- /* de assert pll start and apply pll sw reset */
-
- /* stop pll */
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_PLL_CNTRL, 0);
-
- /* pll sw reset */
- pll_write_udelay(cmn_base + REG_DSI_14nm_PHY_CMN_CTRL_1, 0x20, 10);
- wmb(); /* make sure register committed */
-
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_CTRL_1, 0);
- wmb(); /* make sure register committed */
-}
-
-static void pll_db_commit_14nm(struct dsi_pll_14nm *pll,
- struct dsi_pll_input *pin,
- struct dsi_pll_output *pout)
-{
- void __iomem *base = pll->mmio;
- void __iomem *cmn_base = pll->phy_cmn_mmio;
- u8 data;
-
- DBG("DSI%d PLL", pll->id);
-
- data = pout->cmn_ldo_cntrl;
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_LDO_CNTRL, data);
-
- pll_db_commit_common(pll, pin, pout);
-
- pll_14nm_software_reset(pll);
-
- data = pin->dsiclk_sel; /* set dsiclk_sel = 1 */
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_CLK_CFG1, data);
-
- data = 0xff; /* data, clk, pll normal operation */
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_CTRL_0, data);
-
- /* configure the frequency dependent pll registers */
- data = pout->dec_start;
- pll_write(base + REG_DSI_14nm_PHY_PLL_DEC_START, data);
-
- data = pout->div_frac_start & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START1, data);
- data = (pout->div_frac_start >> 8) & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START2, data);
- data = (pout->div_frac_start >> 16) & 0xf;
- pll_write(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START3, data);
-
- data = pout->plllock_cmp & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP1, data);
-
- data = (pout->plllock_cmp >> 8) & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP2, data);
-
- data = (pout->plllock_cmp >> 16) & 0x3;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP3, data);
-
- data = pin->plllock_cnt << 1 | pin->plllock_rng << 3;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLLLOCK_CMP_EN, data);
-
- data = pout->pll_vco_count & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_VCO_COUNT1, data);
- data = (pout->pll_vco_count >> 8) & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_VCO_COUNT2, data);
-
- data = pout->pll_kvco_count & 0xff;
- pll_write(base + REG_DSI_14nm_PHY_PLL_KVCO_COUNT1, data);
- data = (pout->pll_kvco_count >> 8) & 0x3;
- pll_write(base + REG_DSI_14nm_PHY_PLL_KVCO_COUNT2, data);
-
- data = (pout->pll_postdiv - 1) << 4 | pin->pll_lpf_res1;
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_LPF2_POSTDIV, data);
-
- if (pin->ssc_en)
- pll_db_commit_ssc(pll);
-
- wmb(); /* make sure register committed */
-}
-
-/*
- * VCO clock Callbacks
- */
-static int dsi_pll_14nm_vco_set_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- struct dsi_pll_input *pin = &pll_14nm->in;
- struct dsi_pll_output *pout = &pll_14nm->out;
-
- DBG("DSI PLL%d rate=%lu, parent's=%lu", pll_14nm->id, rate,
- parent_rate);
-
- pll_14nm->vco_current_rate = rate;
- pll_14nm->vco_ref_clk_rate = VCO_REF_CLK_RATE;
-
- dsi_pll_14nm_input_init(pll_14nm);
-
- /*
- * This configures the post divider internal to the VCO. It's
- * fixed to divide by 1 for now.
- *
- * tx_band = pll_postdiv.
- * 0: divided by 1
- * 1: divided by 2
- * 2: divided by 4
- * 3: divided by 8
- */
- pout->pll_postdiv = DSI_PLL_DEFAULT_VCO_POSTDIV;
-
- pll_14nm_dec_frac_calc(pll_14nm);
-
- if (pin->ssc_en)
- pll_14nm_ssc_calc(pll_14nm);
-
- pll_14nm_calc_vco_count(pll_14nm);
-
- /* commit the slave DSI PLL registers if we're master. Note that we
- * don't lock the slave PLL. We just ensure that the PLL/PHY registers
- * of the master and slave are identical
- */
- if (pll_14nm->uc == MSM_DSI_PHY_MASTER) {
- struct dsi_pll_14nm *pll_14nm_slave = pll_14nm->slave;
-
- pll_db_commit_14nm(pll_14nm_slave, pin, pout);
- }
-
- pll_db_commit_14nm(pll_14nm, pin, pout);
-
- return 0;
-}
-
-static unsigned long dsi_pll_14nm_vco_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- void __iomem *base = pll_14nm->mmio;
- u64 vco_rate, multiplier = BIT(20);
- u32 div_frac_start;
- u32 dec_start;
- u64 ref_clk = parent_rate;
-
- dec_start = pll_read(base + REG_DSI_14nm_PHY_PLL_DEC_START);
- dec_start &= 0x0ff;
-
- DBG("dec_start = %x", dec_start);
-
- div_frac_start = (pll_read(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START3)
- & 0xf) << 16;
- div_frac_start |= (pll_read(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START2)
- & 0xff) << 8;
- div_frac_start |= pll_read(base + REG_DSI_14nm_PHY_PLL_DIV_FRAC_START1)
- & 0xff;
-
- DBG("div_frac_start = %x", div_frac_start);
-
- vco_rate = ref_clk * dec_start;
-
- vco_rate += ((ref_clk * div_frac_start) / multiplier);
-
- /*
- * Recalculating the rate from dec_start and frac_start doesn't end up
- * the rate we originally set. Convert the freq to KHz, round it up and
- * convert it back to MHz.
- */
- vco_rate = DIV_ROUND_UP_ULL(vco_rate, 1000) * 1000;
-
- DBG("returning vco rate = %lu", (unsigned long)vco_rate);
-
- return (unsigned long)vco_rate;
-}
-
-static const struct clk_ops clk_ops_dsi_pll_14nm_vco = {
- .round_rate = msm_dsi_pll_helper_clk_round_rate,
- .set_rate = dsi_pll_14nm_vco_set_rate,
- .recalc_rate = dsi_pll_14nm_vco_recalc_rate,
- .prepare = msm_dsi_pll_helper_clk_prepare,
- .unprepare = msm_dsi_pll_helper_clk_unprepare,
-};
-
-/*
- * N1 and N2 post-divider clock callbacks
- */
-#define div_mask(width) ((1 << (width)) - 1)
-static unsigned long dsi_pll_14nm_postdiv_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct dsi_pll_14nm_postdiv *postdiv = to_pll_14nm_postdiv(hw);
- struct dsi_pll_14nm *pll_14nm = postdiv->pll;
- void __iomem *base = pll_14nm->phy_cmn_mmio;
- u8 shift = postdiv->shift;
- u8 width = postdiv->width;
- u32 val;
-
- DBG("DSI%d PLL parent rate=%lu", pll_14nm->id, parent_rate);
-
- val = pll_read(base + REG_DSI_14nm_PHY_CMN_CLK_CFG0) >> shift;
- val &= div_mask(width);
-
- return divider_recalc_rate(hw, parent_rate, val, NULL,
- postdiv->flags, width);
-}
-
-static long dsi_pll_14nm_postdiv_round_rate(struct clk_hw *hw,
- unsigned long rate,
- unsigned long *prate)
-{
- struct dsi_pll_14nm_postdiv *postdiv = to_pll_14nm_postdiv(hw);
- struct dsi_pll_14nm *pll_14nm = postdiv->pll;
-
- DBG("DSI%d PLL parent rate=%lu", pll_14nm->id, rate);
-
- return divider_round_rate(hw, rate, prate, NULL,
- postdiv->width,
- postdiv->flags);
-}
-
-static int dsi_pll_14nm_postdiv_set_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long parent_rate)
-{
- struct dsi_pll_14nm_postdiv *postdiv = to_pll_14nm_postdiv(hw);
- struct dsi_pll_14nm *pll_14nm = postdiv->pll;
- void __iomem *base = pll_14nm->phy_cmn_mmio;
- spinlock_t *lock = &pll_14nm->postdiv_lock;
- u8 shift = postdiv->shift;
- u8 width = postdiv->width;
- unsigned int value;
- unsigned long flags = 0;
- u32 val;
-
- DBG("DSI%d PLL parent rate=%lu parent rate %lu", pll_14nm->id, rate,
- parent_rate);
-
- value = divider_get_val(rate, parent_rate, NULL, postdiv->width,
- postdiv->flags);
-
- spin_lock_irqsave(lock, flags);
-
- val = pll_read(base + REG_DSI_14nm_PHY_CMN_CLK_CFG0);
- val &= ~(div_mask(width) << shift);
-
- val |= value << shift;
- pll_write(base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, val);
-
- /* If we're master in dual DSI mode, then the slave PLL's post-dividers
- * follow the master's post dividers
- */
- if (pll_14nm->uc == MSM_DSI_PHY_MASTER) {
- struct dsi_pll_14nm *pll_14nm_slave = pll_14nm->slave;
- void __iomem *slave_base = pll_14nm_slave->phy_cmn_mmio;
-
- pll_write(slave_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, val);
- }
-
- spin_unlock_irqrestore(lock, flags);
-
- return 0;
-}
-
-static const struct clk_ops clk_ops_dsi_pll_14nm_postdiv = {
- .recalc_rate = dsi_pll_14nm_postdiv_recalc_rate,
- .round_rate = dsi_pll_14nm_postdiv_round_rate,
- .set_rate = dsi_pll_14nm_postdiv_set_rate,
-};
-
-/*
- * PLL Callbacks
- */
-
-static int dsi_pll_14nm_enable_seq(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- void __iomem *base = pll_14nm->mmio;
- void __iomem *cmn_base = pll_14nm->phy_cmn_mmio;
- bool locked;
-
- DBG("");
-
- pll_write(base + REG_DSI_14nm_PHY_PLL_VREF_CFG1, 0x10);
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_PLL_CNTRL, 1);
-
- locked = pll_14nm_poll_for_ready(pll_14nm, POLL_MAX_READS,
- POLL_TIMEOUT_US);
-
- if (unlikely(!locked))
- DRM_DEV_ERROR(&pll_14nm->pdev->dev, "DSI PLL lock failed\n");
- else
- DBG("DSI PLL lock success");
-
- return locked ? 0 : -EINVAL;
-}
-
-static void dsi_pll_14nm_disable_seq(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- void __iomem *cmn_base = pll_14nm->phy_cmn_mmio;
-
- DBG("");
-
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_PLL_CNTRL, 0);
-}
-
-static void dsi_pll_14nm_save_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- struct pll_14nm_cached_state *cached_state = &pll_14nm->cached_state;
- void __iomem *cmn_base = pll_14nm->phy_cmn_mmio;
- u32 data;
-
- data = pll_read(cmn_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0);
-
- cached_state->n1postdiv = data & 0xf;
- cached_state->n2postdiv = (data >> 4) & 0xf;
-
- DBG("DSI%d PLL save state %x %x", pll_14nm->id,
- cached_state->n1postdiv, cached_state->n2postdiv);
-
- cached_state->vco_rate = clk_hw_get_rate(&pll->clk_hw);
-}
-
-static int dsi_pll_14nm_restore_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- struct pll_14nm_cached_state *cached_state = &pll_14nm->cached_state;
- void __iomem *cmn_base = pll_14nm->phy_cmn_mmio;
- u32 data;
- int ret;
-
- ret = dsi_pll_14nm_vco_set_rate(&pll->clk_hw,
- cached_state->vco_rate, 0);
- if (ret) {
- DRM_DEV_ERROR(&pll_14nm->pdev->dev,
- "restore vco rate failed. ret=%d\n", ret);
- return ret;
- }
-
- data = cached_state->n1postdiv | (cached_state->n2postdiv << 4);
-
- DBG("DSI%d PLL restore state %x %x", pll_14nm->id,
- cached_state->n1postdiv, cached_state->n2postdiv);
-
- pll_write(cmn_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, data);
-
- /* also restore post-dividers for slave DSI PLL */
- if (pll_14nm->uc == MSM_DSI_PHY_MASTER) {
- struct dsi_pll_14nm *pll_14nm_slave = pll_14nm->slave;
- void __iomem *slave_base = pll_14nm_slave->phy_cmn_mmio;
-
- pll_write(slave_base + REG_DSI_14nm_PHY_CMN_CLK_CFG0, data);
- }
-
- return 0;
-}
-
-static int dsi_pll_14nm_set_usecase(struct msm_dsi_pll *pll,
- enum msm_dsi_phy_usecase uc)
-{
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- void __iomem *base = pll_14nm->mmio;
- u32 clkbuflr_en, bandgap = 0;
-
- switch (uc) {
- case MSM_DSI_PHY_STANDALONE:
- clkbuflr_en = 0x1;
- break;
- case MSM_DSI_PHY_MASTER:
- clkbuflr_en = 0x3;
- pll_14nm->slave = pll_14nm_list[(pll_14nm->id + 1) % DSI_MAX];
- break;
- case MSM_DSI_PHY_SLAVE:
- clkbuflr_en = 0x0;
- bandgap = 0x3;
- break;
- default:
- return -EINVAL;
- }
-
- pll_write(base + REG_DSI_14nm_PHY_PLL_CLKBUFLR_EN, clkbuflr_en);
- if (bandgap)
- pll_write(base + REG_DSI_14nm_PHY_PLL_PLL_BANDGAP, bandgap);
-
- pll_14nm->uc = uc;
-
- return 0;
-}
-
-static int dsi_pll_14nm_get_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider,
- struct clk **pixel_clk_provider)
-{
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- struct clk_hw_onecell_data *hw_data = pll_14nm->hw_data;
-
- if (byte_clk_provider)
- *byte_clk_provider = hw_data->hws[DSI_BYTE_PLL_CLK]->clk;
- if (pixel_clk_provider)
- *pixel_clk_provider = hw_data->hws[DSI_PIXEL_PLL_CLK]->clk;
-
- return 0;
-}
-
-static void dsi_pll_14nm_destroy(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_14nm *pll_14nm = to_pll_14nm(pll);
- struct platform_device *pdev = pll_14nm->pdev;
- int num_hws = pll_14nm->num_hws;
-
- of_clk_del_provider(pdev->dev.of_node);
-
- while (num_hws--)
- clk_hw_unregister(pll_14nm->hws[num_hws]);
-}
-
-static struct clk_hw *pll_14nm_postdiv_register(struct dsi_pll_14nm *pll_14nm,
- const char *name,
- const char *parent_name,
- unsigned long flags,
- u8 shift)
-{
- struct dsi_pll_14nm_postdiv *pll_postdiv;
- struct device *dev = &pll_14nm->pdev->dev;
- struct clk_init_data postdiv_init = {
- .parent_names = (const char *[]) { parent_name },
- .num_parents = 1,
- .name = name,
- .flags = flags,
- .ops = &clk_ops_dsi_pll_14nm_postdiv,
- };
- int ret;
-
- pll_postdiv = devm_kzalloc(dev, sizeof(*pll_postdiv), GFP_KERNEL);
- if (!pll_postdiv)
- return ERR_PTR(-ENOMEM);
-
- pll_postdiv->pll = pll_14nm;
- pll_postdiv->shift = shift;
- /* both N1 and N2 postdividers are 4 bits wide */
- pll_postdiv->width = 4;
- /* range of each divider is from 1 to 15 */
- pll_postdiv->flags = CLK_DIVIDER_ONE_BASED;
- pll_postdiv->hw.init = &postdiv_init;
-
- ret = clk_hw_register(dev, &pll_postdiv->hw);
- if (ret)
- return ERR_PTR(ret);
-
- return &pll_postdiv->hw;
-}
-
-static int pll_14nm_register(struct dsi_pll_14nm *pll_14nm)
-{
- char clk_name[32], parent[32], vco_name[32];
- struct clk_init_data vco_init = {
- .parent_names = (const char *[]){ "xo" },
- .num_parents = 1,
- .name = vco_name,
- .flags = CLK_IGNORE_UNUSED,
- .ops = &clk_ops_dsi_pll_14nm_vco,
- };
- struct device *dev = &pll_14nm->pdev->dev;
- struct clk_hw **hws = pll_14nm->hws;
- struct clk_hw_onecell_data *hw_data;
- struct clk_hw *hw;
- int num = 0;
- int ret;
-
- DBG("DSI%d", pll_14nm->id);
-
- hw_data = devm_kzalloc(dev, sizeof(*hw_data) +
- NUM_PROVIDED_CLKS * sizeof(struct clk_hw *),
- GFP_KERNEL);
- if (!hw_data)
- return -ENOMEM;
-
- snprintf(vco_name, 32, "dsi%dvco_clk", pll_14nm->id);
- pll_14nm->base.clk_hw.init = &vco_init;
-
- ret = clk_hw_register(dev, &pll_14nm->base.clk_hw);
- if (ret)
- return ret;
-
- hws[num++] = &pll_14nm->base.clk_hw;
-
- snprintf(clk_name, 32, "dsi%dn1_postdiv_clk", pll_14nm->id);
- snprintf(parent, 32, "dsi%dvco_clk", pll_14nm->id);
-
- /* N1 postdiv, bits 0-3 in REG_DSI_14nm_PHY_CMN_CLK_CFG0 */
- hw = pll_14nm_postdiv_register(pll_14nm, clk_name, parent,
- CLK_SET_RATE_PARENT, 0);
- if (IS_ERR(hw))
- return PTR_ERR(hw);
-
- hws[num++] = hw;
-
- snprintf(clk_name, 32, "dsi%dpllbyte", pll_14nm->id);
- snprintf(parent, 32, "dsi%dn1_postdiv_clk", pll_14nm->id);
-
- /* DSI Byte clock = VCO_CLK / N1 / 8 */
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent,
- CLK_SET_RATE_PARENT, 1, 8);
- if (IS_ERR(hw))
- return PTR_ERR(hw);
-
- hws[num++] = hw;
- hw_data->hws[DSI_BYTE_PLL_CLK] = hw;
-
- snprintf(clk_name, 32, "dsi%dn1_postdivby2_clk", pll_14nm->id);
- snprintf(parent, 32, "dsi%dn1_postdiv_clk", pll_14nm->id);
-
- /*
- * Skip the mux for now, force DSICLK_SEL to 1, Add a /2 divider
- * on the way. Don't let it set parent.
- */
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent, 0, 1, 2);
- if (IS_ERR(hw))
- return PTR_ERR(hw);
-
- hws[num++] = hw;
-
- snprintf(clk_name, 32, "dsi%dpll", pll_14nm->id);
- snprintf(parent, 32, "dsi%dn1_postdivby2_clk", pll_14nm->id);
-
- /* DSI pixel clock = VCO_CLK / N1 / 2 / N2
- * This is the output of N2 post-divider, bits 4-7 in
- * REG_DSI_14nm_PHY_CMN_CLK_CFG0. Don't let it set parent.
- */
- hw = pll_14nm_postdiv_register(pll_14nm, clk_name, parent, 0, 4);
- if (IS_ERR(hw))
- return PTR_ERR(hw);
-
- hws[num++] = hw;
- hw_data->hws[DSI_PIXEL_PLL_CLK] = hw;
-
- pll_14nm->num_hws = num;
-
- hw_data->num = NUM_PROVIDED_CLKS;
- pll_14nm->hw_data = hw_data;
-
- ret = of_clk_add_hw_provider(dev->of_node, of_clk_hw_onecell_get,
- pll_14nm->hw_data);
- if (ret) {
- DRM_DEV_ERROR(dev, "failed to register clk provider: %d\n", ret);
- return ret;
- }
-
- return 0;
-}
-
-struct msm_dsi_pll *msm_dsi_pll_14nm_init(struct platform_device *pdev, int id)
-{
- struct dsi_pll_14nm *pll_14nm;
- struct msm_dsi_pll *pll;
- int ret;
-
- if (!pdev)
- return ERR_PTR(-ENODEV);
-
- pll_14nm = devm_kzalloc(&pdev->dev, sizeof(*pll_14nm), GFP_KERNEL);
- if (!pll_14nm)
- return ERR_PTR(-ENOMEM);
-
- DBG("PLL%d", id);
-
- pll_14nm->pdev = pdev;
- pll_14nm->id = id;
- pll_14nm_list[id] = pll_14nm;
-
- pll_14nm->phy_cmn_mmio = msm_ioremap(pdev, "dsi_phy", "DSI_PHY");
- if (IS_ERR_OR_NULL(pll_14nm->phy_cmn_mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "failed to map CMN PHY base\n");
- return ERR_PTR(-ENOMEM);
- }
-
- pll_14nm->mmio = msm_ioremap(pdev, "dsi_pll", "DSI_PLL");
- if (IS_ERR_OR_NULL(pll_14nm->mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "failed to map PLL base\n");
- return ERR_PTR(-ENOMEM);
- }
-
- spin_lock_init(&pll_14nm->postdiv_lock);
-
- pll = &pll_14nm->base;
- pll->min_rate = VCO_MIN_RATE;
- pll->max_rate = VCO_MAX_RATE;
- pll->get_provider = dsi_pll_14nm_get_provider;
- pll->destroy = dsi_pll_14nm_destroy;
- pll->disable_seq = dsi_pll_14nm_disable_seq;
- pll->save_state = dsi_pll_14nm_save_state;
- pll->restore_state = dsi_pll_14nm_restore_state;
- pll->set_usecase = dsi_pll_14nm_set_usecase;
-
- pll_14nm->vco_delay = 1;
-
- pll->en_seq_cnt = 1;
- pll->enable_seqs[0] = dsi_pll_14nm_enable_seq;
-
- ret = pll_14nm_register(pll_14nm);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
- return ERR_PTR(ret);
- }
-
- return pll;
-}
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
- */
-
-#include <linux/clk.h>
-#include <linux/clk-provider.h>
-
-#include "dsi_pll.h"
-#include "dsi.xml.h"
-
-/*
- * DSI PLL 28nm - clock diagram (eg: DSI0):
- *
- * dsi0analog_postdiv_clk
- * | dsi0indirect_path_div2_clk
- * | |
- * +------+ | +----+ | |\ dsi0byte_mux
- * dsi0vco_clk --o--| DIV1 |--o--| /2 |--o--| \ |
- * | +------+ +----+ | m| | +----+
- * | | u|--o--| /4 |-- dsi0pllbyte
- * | | x| +----+
- * o--------------------------| /
- * | |/
- * | +------+
- * o----------| DIV3 |------------------------- dsi0pll
- * +------+
- */
-
-#define POLL_MAX_READS 10
-#define POLL_TIMEOUT_US 50
-
-#define NUM_PROVIDED_CLKS 2
-
-#define VCO_REF_CLK_RATE 19200000
-#define VCO_MIN_RATE 350000000
-#define VCO_MAX_RATE 750000000
-
-#define DSI_BYTE_PLL_CLK 0
-#define DSI_PIXEL_PLL_CLK 1
-
-#define LPFR_LUT_SIZE 10
-struct lpfr_cfg {
- unsigned long vco_rate;
- u32 resistance;
-};
-
-/* Loop filter resistance: */
-static const struct lpfr_cfg lpfr_lut[LPFR_LUT_SIZE] = {
- { 479500000, 8 },
- { 480000000, 11 },
- { 575500000, 8 },
- { 576000000, 12 },
- { 610500000, 8 },
- { 659500000, 9 },
- { 671500000, 10 },
- { 672000000, 14 },
- { 708500000, 10 },
- { 750000000, 11 },
-};
-
-struct pll_28nm_cached_state {
- unsigned long vco_rate;
- u8 postdiv3;
- u8 postdiv1;
- u8 byte_mux;
-};
-
-struct dsi_pll_28nm {
- struct msm_dsi_pll base;
-
- int id;
- struct platform_device *pdev;
- void __iomem *mmio;
-
- int vco_delay;
-
- /* private clocks: */
- struct clk *clks[NUM_DSI_CLOCKS_MAX];
- u32 num_clks;
-
- /* clock-provider: */
- struct clk *provided_clks[NUM_PROVIDED_CLKS];
- struct clk_onecell_data clk_data;
-
- struct pll_28nm_cached_state cached_state;
-};
-
-#define to_pll_28nm(x) container_of(x, struct dsi_pll_28nm, base)
-
-static bool pll_28nm_poll_for_ready(struct dsi_pll_28nm *pll_28nm,
- u32 nb_tries, u32 timeout_us)
-{
- bool pll_locked = false;
- u32 val;
-
- while (nb_tries--) {
- val = pll_read(pll_28nm->mmio + REG_DSI_28nm_PHY_PLL_STATUS);
- pll_locked = !!(val & DSI_28nm_PHY_PLL_STATUS_PLL_RDY);
-
- if (pll_locked)
- break;
-
- udelay(timeout_us);
- }
- DBG("DSI PLL is %slocked", pll_locked ? "" : "*not* ");
-
- return pll_locked;
-}
-
-static void pll_28nm_software_reset(struct dsi_pll_28nm *pll_28nm)
-{
- void __iomem *base = pll_28nm->mmio;
-
- /*
- * Add HW recommended delays after toggling the software
- * reset bit off and back on.
- */
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_TEST_CFG,
- DSI_28nm_PHY_PLL_TEST_CFG_PLL_SW_RESET, 1);
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_TEST_CFG, 0x00, 1);
-}
-
-/*
- * Clock Callbacks
- */
-static int dsi_pll_28nm_clk_set_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct device *dev = &pll_28nm->pdev->dev;
- void __iomem *base = pll_28nm->mmio;
- unsigned long div_fbx1000, gen_vco_clk;
- u32 refclk_cfg, frac_n_mode, frac_n_value;
- u32 sdm_cfg0, sdm_cfg1, sdm_cfg2, sdm_cfg3;
- u32 cal_cfg10, cal_cfg11;
- u32 rem;
- int i;
-
- VERB("rate=%lu, parent's=%lu", rate, parent_rate);
-
- /* Force postdiv2 to be div-4 */
- pll_write(base + REG_DSI_28nm_PHY_PLL_POSTDIV2_CFG, 3);
-
- /* Configure the Loop filter resistance */
- for (i = 0; i < LPFR_LUT_SIZE; i++)
- if (rate <= lpfr_lut[i].vco_rate)
- break;
- if (i == LPFR_LUT_SIZE) {
- DRM_DEV_ERROR(dev, "unable to get loop filter resistance. vco=%lu\n",
- rate);
- return -EINVAL;
- }
- pll_write(base + REG_DSI_28nm_PHY_PLL_LPFR_CFG, lpfr_lut[i].resistance);
-
- /* Loop filter capacitance values : c1 and c2 */
- pll_write(base + REG_DSI_28nm_PHY_PLL_LPFC1_CFG, 0x70);
- pll_write(base + REG_DSI_28nm_PHY_PLL_LPFC2_CFG, 0x15);
-
- rem = rate % VCO_REF_CLK_RATE;
- if (rem) {
- refclk_cfg = DSI_28nm_PHY_PLL_REFCLK_CFG_DBLR;
- frac_n_mode = 1;
- div_fbx1000 = rate / (VCO_REF_CLK_RATE / 500);
- gen_vco_clk = div_fbx1000 * (VCO_REF_CLK_RATE / 500);
- } else {
- refclk_cfg = 0x0;
- frac_n_mode = 0;
- div_fbx1000 = rate / (VCO_REF_CLK_RATE / 1000);
- gen_vco_clk = div_fbx1000 * (VCO_REF_CLK_RATE / 1000);
- }
-
- DBG("refclk_cfg = %d", refclk_cfg);
-
- rem = div_fbx1000 % 1000;
- frac_n_value = (rem << 16) / 1000;
-
- DBG("div_fb = %lu", div_fbx1000);
- DBG("frac_n_value = %d", frac_n_value);
-
- DBG("Generated VCO Clock: %lu", gen_vco_clk);
- rem = 0;
- sdm_cfg1 = pll_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG1);
- sdm_cfg1 &= ~DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET__MASK;
- if (frac_n_mode) {
- sdm_cfg0 = 0x0;
- sdm_cfg0 |= DSI_28nm_PHY_PLL_SDM_CFG0_BYP_DIV(0);
- sdm_cfg1 |= DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET(
- (u32)(((div_fbx1000 / 1000) & 0x3f) - 1));
- sdm_cfg3 = frac_n_value >> 8;
- sdm_cfg2 = frac_n_value & 0xff;
- } else {
- sdm_cfg0 = DSI_28nm_PHY_PLL_SDM_CFG0_BYP;
- sdm_cfg0 |= DSI_28nm_PHY_PLL_SDM_CFG0_BYP_DIV(
- (u32)(((div_fbx1000 / 1000) & 0x3f) - 1));
- sdm_cfg1 |= DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET(0);
- sdm_cfg2 = 0;
- sdm_cfg3 = 0;
- }
-
- DBG("sdm_cfg0=%d", sdm_cfg0);
- DBG("sdm_cfg1=%d", sdm_cfg1);
- DBG("sdm_cfg2=%d", sdm_cfg2);
- DBG("sdm_cfg3=%d", sdm_cfg3);
-
- cal_cfg11 = (u32)(gen_vco_clk / (256 * 1000000));
- cal_cfg10 = (u32)((gen_vco_clk % (256 * 1000000)) / 1000000);
- DBG("cal_cfg10=%d, cal_cfg11=%d", cal_cfg10, cal_cfg11);
-
- pll_write(base + REG_DSI_28nm_PHY_PLL_CHGPUMP_CFG, 0x02);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG3, 0x2b);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG4, 0x06);
- pll_write(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x0d);
-
- pll_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG1, sdm_cfg1);
- pll_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG2,
- DSI_28nm_PHY_PLL_SDM_CFG2_FREQ_SEED_7_0(sdm_cfg2));
- pll_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG3,
- DSI_28nm_PHY_PLL_SDM_CFG3_FREQ_SEED_15_8(sdm_cfg3));
- pll_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG4, 0x00);
-
- /* Add hardware recommended delay for correct PLL configuration */
- if (pll_28nm->vco_delay)
- udelay(pll_28nm->vco_delay);
-
- pll_write(base + REG_DSI_28nm_PHY_PLL_REFCLK_CFG, refclk_cfg);
- pll_write(base + REG_DSI_28nm_PHY_PLL_PWRGEN_CFG, 0x00);
- pll_write(base + REG_DSI_28nm_PHY_PLL_VCOLPF_CFG, 0x31);
- pll_write(base + REG_DSI_28nm_PHY_PLL_SDM_CFG0, sdm_cfg0);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG0, 0x12);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG6, 0x30);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG7, 0x00);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG8, 0x60);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG9, 0x00);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG10, cal_cfg10 & 0xff);
- pll_write(base + REG_DSI_28nm_PHY_PLL_CAL_CFG11, cal_cfg11 & 0xff);
- pll_write(base + REG_DSI_28nm_PHY_PLL_EFUSE_CFG, 0x20);
-
- return 0;
-}
-
-static int dsi_pll_28nm_clk_is_enabled(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
-
- return pll_28nm_poll_for_ready(pll_28nm, POLL_MAX_READS,
- POLL_TIMEOUT_US);
-}
-
-static unsigned long dsi_pll_28nm_clk_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- void __iomem *base = pll_28nm->mmio;
- u32 sdm0, doubler, sdm_byp_div;
- u32 sdm_dc_off, sdm_freq_seed, sdm2, sdm3;
- u32 ref_clk = VCO_REF_CLK_RATE;
- unsigned long vco_rate;
-
- VERB("parent_rate=%lu", parent_rate);
-
- /* Check to see if the ref clk doubler is enabled */
- doubler = pll_read(base + REG_DSI_28nm_PHY_PLL_REFCLK_CFG) &
- DSI_28nm_PHY_PLL_REFCLK_CFG_DBLR;
- ref_clk += (doubler * VCO_REF_CLK_RATE);
-
- /* see if it is integer mode or sdm mode */
- sdm0 = pll_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG0);
- if (sdm0 & DSI_28nm_PHY_PLL_SDM_CFG0_BYP) {
- /* integer mode */
- sdm_byp_div = FIELD(
- pll_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG0),
- DSI_28nm_PHY_PLL_SDM_CFG0_BYP_DIV) + 1;
- vco_rate = ref_clk * sdm_byp_div;
- } else {
- /* sdm mode */
- sdm_dc_off = FIELD(
- pll_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG1),
- DSI_28nm_PHY_PLL_SDM_CFG1_DC_OFFSET);
- DBG("sdm_dc_off = %d", sdm_dc_off);
- sdm2 = FIELD(pll_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG2),
- DSI_28nm_PHY_PLL_SDM_CFG2_FREQ_SEED_7_0);
- sdm3 = FIELD(pll_read(base + REG_DSI_28nm_PHY_PLL_SDM_CFG3),
- DSI_28nm_PHY_PLL_SDM_CFG3_FREQ_SEED_15_8);
- sdm_freq_seed = (sdm3 << 8) | sdm2;
- DBG("sdm_freq_seed = %d", sdm_freq_seed);
-
- vco_rate = (ref_clk * (sdm_dc_off + 1)) +
- mult_frac(ref_clk, sdm_freq_seed, BIT(16));
- DBG("vco rate = %lu", vco_rate);
- }
-
- DBG("returning vco rate = %lu", vco_rate);
-
- return vco_rate;
-}
-
-static const struct clk_ops clk_ops_dsi_pll_28nm_vco = {
- .round_rate = msm_dsi_pll_helper_clk_round_rate,
- .set_rate = dsi_pll_28nm_clk_set_rate,
- .recalc_rate = dsi_pll_28nm_clk_recalc_rate,
- .prepare = msm_dsi_pll_helper_clk_prepare,
- .unprepare = msm_dsi_pll_helper_clk_unprepare,
- .is_enabled = dsi_pll_28nm_clk_is_enabled,
-};
-
-/*
- * PLL Callbacks
- */
-static int dsi_pll_28nm_enable_seq_hpm(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct device *dev = &pll_28nm->pdev->dev;
- void __iomem *base = pll_28nm->mmio;
- u32 max_reads = 5, timeout_us = 100;
- bool locked;
- u32 val;
- int i;
-
- DBG("id=%d", pll_28nm->id);
-
- pll_28nm_software_reset(pll_28nm);
-
- /*
- * PLL power up sequence.
- * Add necessary delays recommended by hardware.
- */
- val = DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 1);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRGEN_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 200);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_ENABLE;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 600);
-
- for (i = 0; i < 2; i++) {
- /* DSI Uniphy lock detect setting */
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2,
- 0x0c, 100);
- pll_write(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x0d);
-
- /* poll for PLL ready status */
- locked = pll_28nm_poll_for_ready(pll_28nm,
- max_reads, timeout_us);
- if (locked)
- break;
-
- pll_28nm_software_reset(pll_28nm);
-
- /*
- * PLL power up sequence.
- * Add necessary delays recommended by hardware.
- */
- val = DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 1);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRGEN_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 200);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 250);
-
- val &= ~DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 200);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_ENABLE;
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 600);
- }
-
- if (unlikely(!locked))
- DRM_DEV_ERROR(dev, "DSI PLL lock failed\n");
- else
- DBG("DSI PLL Lock success");
-
- return locked ? 0 : -EINVAL;
-}
-
-static int dsi_pll_28nm_enable_seq_lp(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct device *dev = &pll_28nm->pdev->dev;
- void __iomem *base = pll_28nm->mmio;
- bool locked;
- u32 max_reads = 10, timeout_us = 50;
- u32 val;
-
- DBG("id=%d", pll_28nm->id);
-
- pll_28nm_software_reset(pll_28nm);
-
- /*
- * PLL power up sequence.
- * Add necessary delays recommended by hardware.
- */
- pll_write_ndelay(base + REG_DSI_28nm_PHY_PLL_CAL_CFG1, 0x34, 500);
-
- val = DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRDN_B;
- pll_write_ndelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_PWRGEN_PWRDN_B;
- pll_write_ndelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
-
- val |= DSI_28nm_PHY_PLL_GLB_CFG_PLL_LDO_PWRDN_B |
- DSI_28nm_PHY_PLL_GLB_CFG_PLL_ENABLE;
- pll_write_ndelay(base + REG_DSI_28nm_PHY_PLL_GLB_CFG, val, 500);
-
- /* DSI PLL toggle lock detect setting */
- pll_write_ndelay(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x04, 500);
- pll_write_udelay(base + REG_DSI_28nm_PHY_PLL_LKDET_CFG2, 0x05, 512);
-
- locked = pll_28nm_poll_for_ready(pll_28nm, max_reads, timeout_us);
-
- if (unlikely(!locked))
- DRM_DEV_ERROR(dev, "DSI PLL lock failed\n");
- else
- DBG("DSI PLL lock success");
-
- return locked ? 0 : -EINVAL;
-}
-
-static void dsi_pll_28nm_disable_seq(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
-
- DBG("id=%d", pll_28nm->id);
- pll_write(pll_28nm->mmio + REG_DSI_28nm_PHY_PLL_GLB_CFG, 0x00);
-}
-
-static void dsi_pll_28nm_save_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
- void __iomem *base = pll_28nm->mmio;
-
- cached_state->postdiv3 =
- pll_read(base + REG_DSI_28nm_PHY_PLL_POSTDIV3_CFG);
- cached_state->postdiv1 =
- pll_read(base + REG_DSI_28nm_PHY_PLL_POSTDIV1_CFG);
- cached_state->byte_mux = pll_read(base + REG_DSI_28nm_PHY_PLL_VREG_CFG);
- if (dsi_pll_28nm_clk_is_enabled(&pll->clk_hw))
- cached_state->vco_rate = clk_hw_get_rate(&pll->clk_hw);
- else
- cached_state->vco_rate = 0;
-}
-
-static int dsi_pll_28nm_restore_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
- void __iomem *base = pll_28nm->mmio;
- int ret;
-
- ret = dsi_pll_28nm_clk_set_rate(&pll->clk_hw,
- cached_state->vco_rate, 0);
- if (ret) {
- DRM_DEV_ERROR(&pll_28nm->pdev->dev,
- "restore vco rate failed. ret=%d\n", ret);
- return ret;
- }
-
- pll_write(base + REG_DSI_28nm_PHY_PLL_POSTDIV3_CFG,
- cached_state->postdiv3);
- pll_write(base + REG_DSI_28nm_PHY_PLL_POSTDIV1_CFG,
- cached_state->postdiv1);
- pll_write(base + REG_DSI_28nm_PHY_PLL_VREG_CFG,
- cached_state->byte_mux);
-
- return 0;
-}
-
-static int dsi_pll_28nm_get_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider,
- struct clk **pixel_clk_provider)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
-
- if (byte_clk_provider)
- *byte_clk_provider = pll_28nm->provided_clks[DSI_BYTE_PLL_CLK];
- if (pixel_clk_provider)
- *pixel_clk_provider =
- pll_28nm->provided_clks[DSI_PIXEL_PLL_CLK];
-
- return 0;
-}
-
-static void dsi_pll_28nm_destroy(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- int i;
-
- msm_dsi_pll_helper_unregister_clks(pll_28nm->pdev,
- pll_28nm->clks, pll_28nm->num_clks);
-
- for (i = 0; i < NUM_PROVIDED_CLKS; i++)
- pll_28nm->provided_clks[i] = NULL;
-
- pll_28nm->num_clks = 0;
- pll_28nm->clk_data.clks = NULL;
- pll_28nm->clk_data.clk_num = 0;
-}
-
-static int pll_28nm_register(struct dsi_pll_28nm *pll_28nm)
-{
- char clk_name[32], parent1[32], parent2[32], vco_name[32];
- struct clk_init_data vco_init = {
- .parent_names = (const char *[]){ "xo" },
- .num_parents = 1,
- .name = vco_name,
- .flags = CLK_IGNORE_UNUSED,
- .ops = &clk_ops_dsi_pll_28nm_vco,
- };
- struct device *dev = &pll_28nm->pdev->dev;
- struct clk **clks = pll_28nm->clks;
- struct clk **provided_clks = pll_28nm->provided_clks;
- int num = 0;
- int ret;
-
- DBG("%d", pll_28nm->id);
-
- snprintf(vco_name, 32, "dsi%dvco_clk", pll_28nm->id);
- pll_28nm->base.clk_hw.init = &vco_init;
- clks[num++] = clk_register(dev, &pll_28nm->base.clk_hw);
-
- snprintf(clk_name, 32, "dsi%danalog_postdiv_clk", pll_28nm->id);
- snprintf(parent1, 32, "dsi%dvco_clk", pll_28nm->id);
- clks[num++] = clk_register_divider(dev, clk_name,
- parent1, CLK_SET_RATE_PARENT,
- pll_28nm->mmio +
- REG_DSI_28nm_PHY_PLL_POSTDIV1_CFG,
- 0, 4, 0, NULL);
-
- snprintf(clk_name, 32, "dsi%dindirect_path_div2_clk", pll_28nm->id);
- snprintf(parent1, 32, "dsi%danalog_postdiv_clk", pll_28nm->id);
- clks[num++] = clk_register_fixed_factor(dev, clk_name,
- parent1, CLK_SET_RATE_PARENT,
- 1, 2);
-
- snprintf(clk_name, 32, "dsi%dpll", pll_28nm->id);
- snprintf(parent1, 32, "dsi%dvco_clk", pll_28nm->id);
- clks[num++] = provided_clks[DSI_PIXEL_PLL_CLK] =
- clk_register_divider(dev, clk_name,
- parent1, 0, pll_28nm->mmio +
- REG_DSI_28nm_PHY_PLL_POSTDIV3_CFG,
- 0, 8, 0, NULL);
-
- snprintf(clk_name, 32, "dsi%dbyte_mux", pll_28nm->id);
- snprintf(parent1, 32, "dsi%dvco_clk", pll_28nm->id);
- snprintf(parent2, 32, "dsi%dindirect_path_div2_clk", pll_28nm->id);
- clks[num++] = clk_register_mux(dev, clk_name,
- ((const char *[]){
- parent1, parent2
- }), 2, CLK_SET_RATE_PARENT, pll_28nm->mmio +
- REG_DSI_28nm_PHY_PLL_VREG_CFG, 1, 1, 0, NULL);
-
- snprintf(clk_name, 32, "dsi%dpllbyte", pll_28nm->id);
- snprintf(parent1, 32, "dsi%dbyte_mux", pll_28nm->id);
- clks[num++] = provided_clks[DSI_BYTE_PLL_CLK] =
- clk_register_fixed_factor(dev, clk_name,
- parent1, CLK_SET_RATE_PARENT, 1, 4);
-
- pll_28nm->num_clks = num;
-
- pll_28nm->clk_data.clk_num = NUM_PROVIDED_CLKS;
- pll_28nm->clk_data.clks = provided_clks;
-
- ret = of_clk_add_provider(dev->of_node,
- of_clk_src_onecell_get, &pll_28nm->clk_data);
- if (ret) {
- DRM_DEV_ERROR(dev, "failed to register clk provider: %d\n", ret);
- return ret;
- }
-
- return 0;
-}
-
-struct msm_dsi_pll *msm_dsi_pll_28nm_init(struct platform_device *pdev,
- enum msm_dsi_phy_type type, int id)
-{
- struct dsi_pll_28nm *pll_28nm;
- struct msm_dsi_pll *pll;
- int ret;
-
- if (!pdev)
- return ERR_PTR(-ENODEV);
-
- pll_28nm = devm_kzalloc(&pdev->dev, sizeof(*pll_28nm), GFP_KERNEL);
- if (!pll_28nm)
- return ERR_PTR(-ENOMEM);
-
- pll_28nm->pdev = pdev;
- pll_28nm->id = id;
-
- pll_28nm->mmio = msm_ioremap(pdev, "dsi_pll", "DSI_PLL");
- if (IS_ERR_OR_NULL(pll_28nm->mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "%s: failed to map pll base\n", __func__);
- return ERR_PTR(-ENOMEM);
- }
-
- pll = &pll_28nm->base;
- pll->min_rate = VCO_MIN_RATE;
- pll->max_rate = VCO_MAX_RATE;
- pll->get_provider = dsi_pll_28nm_get_provider;
- pll->destroy = dsi_pll_28nm_destroy;
- pll->disable_seq = dsi_pll_28nm_disable_seq;
- pll->save_state = dsi_pll_28nm_save_state;
- pll->restore_state = dsi_pll_28nm_restore_state;
-
- if (type == MSM_DSI_PHY_28NM_HPM) {
- pll_28nm->vco_delay = 1;
-
- pll->en_seq_cnt = 3;
- pll->enable_seqs[0] = dsi_pll_28nm_enable_seq_hpm;
- pll->enable_seqs[1] = dsi_pll_28nm_enable_seq_hpm;
- pll->enable_seqs[2] = dsi_pll_28nm_enable_seq_hpm;
- } else if (type == MSM_DSI_PHY_28NM_LP) {
- pll_28nm->vco_delay = 1000;
-
- pll->en_seq_cnt = 1;
- pll->enable_seqs[0] = dsi_pll_28nm_enable_seq_lp;
- } else {
- DRM_DEV_ERROR(&pdev->dev, "phy type (%d) is not 28nm\n", type);
- return ERR_PTR(-EINVAL);
- }
-
- ret = pll_28nm_register(pll_28nm);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
- return ERR_PTR(ret);
- }
-
- return pll;
-}
-
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2012-2015, The Linux Foundation. All rights reserved.
- */
-
-#include <linux/clk-provider.h>
-
-#include "dsi_pll.h"
-#include "dsi.xml.h"
-
-/*
- * DSI PLL 28nm (8960/A family) - clock diagram (eg: DSI1):
- *
- *
- * +------+
- * dsi1vco_clk ----o-----| DIV1 |---dsi1pllbit (not exposed as clock)
- * F * byte_clk | +------+
- * | bit clock divider (F / 8)
- * |
- * | +------+
- * o-----| DIV2 |---dsi0pllbyte---o---> To byte RCG
- * | +------+ | (sets parent rate)
- * | byte clock divider (F) |
- * | |
- * | o---> To esc RCG
- * | (doesn't set parent rate)
- * |
- * | +------+
- * o-----| DIV3 |----dsi0pll------o---> To dsi RCG
- * +------+ | (sets parent rate)
- * dsi clock divider (F * magic) |
- * |
- * o---> To pixel rcg
- * (doesn't set parent rate)
- */
-
-#define POLL_MAX_READS 8000
-#define POLL_TIMEOUT_US 1
-
-#define NUM_PROVIDED_CLKS 2
-
-#define VCO_REF_CLK_RATE 27000000
-#define VCO_MIN_RATE 600000000
-#define VCO_MAX_RATE 1200000000
-
-#define DSI_BYTE_PLL_CLK 0
-#define DSI_PIXEL_PLL_CLK 1
-
-#define VCO_PREF_DIV_RATIO 27
-
-struct pll_28nm_cached_state {
- unsigned long vco_rate;
- u8 postdiv3;
- u8 postdiv2;
- u8 postdiv1;
-};
-
-struct clk_bytediv {
- struct clk_hw hw;
- void __iomem *reg;
-};
-
-struct dsi_pll_28nm {
- struct msm_dsi_pll base;
-
- int id;
- struct platform_device *pdev;
- void __iomem *mmio;
-
- /* custom byte clock divider */
- struct clk_bytediv *bytediv;
-
- /* private clocks: */
- struct clk *clks[NUM_DSI_CLOCKS_MAX];
- u32 num_clks;
-
- /* clock-provider: */
- struct clk *provided_clks[NUM_PROVIDED_CLKS];
- struct clk_onecell_data clk_data;
-
- struct pll_28nm_cached_state cached_state;
-};
-
-#define to_pll_28nm(x) container_of(x, struct dsi_pll_28nm, base)
-
-static bool pll_28nm_poll_for_ready(struct dsi_pll_28nm *pll_28nm,
- int nb_tries, int timeout_us)
-{
- bool pll_locked = false;
- u32 val;
-
- while (nb_tries--) {
- val = pll_read(pll_28nm->mmio + REG_DSI_28nm_8960_PHY_PLL_RDY);
- pll_locked = !!(val & DSI_28nm_8960_PHY_PLL_RDY_PLL_RDY);
-
- if (pll_locked)
- break;
-
- udelay(timeout_us);
- }
- DBG("DSI PLL is %slocked", pll_locked ? "" : "*not* ");
-
- return pll_locked;
-}
-
-/*
- * Clock Callbacks
- */
-static int dsi_pll_28nm_clk_set_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- void __iomem *base = pll_28nm->mmio;
- u32 val, temp, fb_divider;
-
- DBG("rate=%lu, parent's=%lu", rate, parent_rate);
-
- temp = rate / 10;
- val = VCO_REF_CLK_RATE / 10;
- fb_divider = (temp * VCO_PREF_DIV_RATIO) / val;
- fb_divider = fb_divider / 2 - 1;
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_1,
- fb_divider & 0xff);
-
- val = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_2);
-
- val |= (fb_divider >> 8) & 0x07;
-
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_2,
- val);
-
- val = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_3);
-
- val |= (VCO_PREF_DIV_RATIO - 1) & 0x3f;
-
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_3,
- val);
-
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_6,
- 0xf);
-
- val = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8);
- val |= 0x7 << 4;
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8,
- val);
-
- return 0;
-}
-
-static int dsi_pll_28nm_clk_is_enabled(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
-
- return pll_28nm_poll_for_ready(pll_28nm, POLL_MAX_READS,
- POLL_TIMEOUT_US);
-}
-
-static unsigned long dsi_pll_28nm_clk_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- void __iomem *base = pll_28nm->mmio;
- unsigned long vco_rate;
- u32 status, fb_divider, temp, ref_divider;
-
- VERB("parent_rate=%lu", parent_rate);
-
- status = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_0);
-
- if (status & DSI_28nm_8960_PHY_PLL_CTRL_0_ENABLE) {
- fb_divider = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_1);
- fb_divider &= 0xff;
- temp = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_2) & 0x07;
- fb_divider = (temp << 8) | fb_divider;
- fb_divider += 1;
-
- ref_divider = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_3);
- ref_divider &= 0x3f;
- ref_divider += 1;
-
- /* multiply by 2 */
- vco_rate = (parent_rate / ref_divider) * fb_divider * 2;
- } else {
- vco_rate = 0;
- }
-
- DBG("returning vco rate = %lu", vco_rate);
-
- return vco_rate;
-}
-
-static const struct clk_ops clk_ops_dsi_pll_28nm_vco = {
- .round_rate = msm_dsi_pll_helper_clk_round_rate,
- .set_rate = dsi_pll_28nm_clk_set_rate,
- .recalc_rate = dsi_pll_28nm_clk_recalc_rate,
- .prepare = msm_dsi_pll_helper_clk_prepare,
- .unprepare = msm_dsi_pll_helper_clk_unprepare,
- .is_enabled = dsi_pll_28nm_clk_is_enabled,
-};
-
-/*
- * Custom byte clock divier clk_ops
- *
- * This clock is the entry point to configuring the PLL. The user (dsi host)
- * will set this clock's rate to the desired byte clock rate. The VCO lock
- * frequency is a multiple of the byte clock rate. The multiplication factor
- * (shown as F in the diagram above) is a function of the byte clock rate.
- *
- * This custom divider clock ensures that its parent (VCO) is set to the
- * desired rate, and that the byte clock postdivider (POSTDIV2) is configured
- * accordingly
- */
-#define to_clk_bytediv(_hw) container_of(_hw, struct clk_bytediv, hw)
-
-static unsigned long clk_bytediv_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct clk_bytediv *bytediv = to_clk_bytediv(hw);
- unsigned int div;
-
- div = pll_read(bytediv->reg) & 0xff;
-
- return parent_rate / (div + 1);
-}
-
-/* find multiplication factor(wrt byte clock) at which the VCO should be set */
-static unsigned int get_vco_mul_factor(unsigned long byte_clk_rate)
-{
- unsigned long bit_mhz;
-
- /* convert to bit clock in Mhz */
- bit_mhz = (byte_clk_rate * 8) / 1000000;
-
- if (bit_mhz < 125)
- return 64;
- else if (bit_mhz < 250)
- return 32;
- else if (bit_mhz < 600)
- return 16;
- else
- return 8;
-}
-
-static long clk_bytediv_round_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long *prate)
-{
- unsigned long best_parent;
- unsigned int factor;
-
- factor = get_vco_mul_factor(rate);
-
- best_parent = rate * factor;
- *prate = clk_hw_round_rate(clk_hw_get_parent(hw), best_parent);
-
- return *prate / factor;
-}
-
-static int clk_bytediv_set_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long parent_rate)
-{
- struct clk_bytediv *bytediv = to_clk_bytediv(hw);
- u32 val;
- unsigned int factor;
-
- factor = get_vco_mul_factor(rate);
-
- val = pll_read(bytediv->reg);
- val |= (factor - 1) & 0xff;
- pll_write(bytediv->reg, val);
-
- return 0;
-}
-
-/* Our special byte clock divider ops */
-static const struct clk_ops clk_bytediv_ops = {
- .round_rate = clk_bytediv_round_rate,
- .set_rate = clk_bytediv_set_rate,
- .recalc_rate = clk_bytediv_recalc_rate,
-};
-
-/*
- * PLL Callbacks
- */
-static int dsi_pll_28nm_enable_seq(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct device *dev = &pll_28nm->pdev->dev;
- void __iomem *base = pll_28nm->mmio;
- bool locked;
- unsigned int bit_div, byte_div;
- int max_reads = 1000, timeout_us = 100;
- u32 val;
-
- DBG("id=%d", pll_28nm->id);
-
- /*
- * before enabling the PLL, configure the bit clock divider since we
- * don't expose it as a clock to the outside world
- * 1: read back the byte clock divider that should already be set
- * 2: divide by 8 to get bit clock divider
- * 3: write it to POSTDIV1
- */
- val = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_9);
- byte_div = val + 1;
- bit_div = byte_div / 8;
-
- val = pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8);
- val &= ~0xf;
- val |= (bit_div - 1);
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8, val);
-
- /* enable the PLL */
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_0,
- DSI_28nm_8960_PHY_PLL_CTRL_0_ENABLE);
-
- locked = pll_28nm_poll_for_ready(pll_28nm, max_reads, timeout_us);
-
- if (unlikely(!locked))
- DRM_DEV_ERROR(dev, "DSI PLL lock failed\n");
- else
- DBG("DSI PLL lock success");
-
- return locked ? 0 : -EINVAL;
-}
-
-static void dsi_pll_28nm_disable_seq(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
-
- DBG("id=%d", pll_28nm->id);
- pll_write(pll_28nm->mmio + REG_DSI_28nm_8960_PHY_PLL_CTRL_0, 0x00);
-}
-
-static void dsi_pll_28nm_save_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
- void __iomem *base = pll_28nm->mmio;
-
- cached_state->postdiv3 =
- pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_10);
- cached_state->postdiv2 =
- pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_9);
- cached_state->postdiv1 =
- pll_read(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8);
-
- cached_state->vco_rate = clk_hw_get_rate(&pll->clk_hw);
-}
-
-static int dsi_pll_28nm_restore_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
- struct pll_28nm_cached_state *cached_state = &pll_28nm->cached_state;
- void __iomem *base = pll_28nm->mmio;
- int ret;
-
- ret = dsi_pll_28nm_clk_set_rate(&pll->clk_hw,
- cached_state->vco_rate, 0);
- if (ret) {
- DRM_DEV_ERROR(&pll_28nm->pdev->dev,
- "restore vco rate failed. ret=%d\n", ret);
- return ret;
- }
-
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_10,
- cached_state->postdiv3);
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_9,
- cached_state->postdiv2);
- pll_write(base + REG_DSI_28nm_8960_PHY_PLL_CTRL_8,
- cached_state->postdiv1);
-
- return 0;
-}
-
-static int dsi_pll_28nm_get_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider,
- struct clk **pixel_clk_provider)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
-
- if (byte_clk_provider)
- *byte_clk_provider = pll_28nm->provided_clks[DSI_BYTE_PLL_CLK];
- if (pixel_clk_provider)
- *pixel_clk_provider =
- pll_28nm->provided_clks[DSI_PIXEL_PLL_CLK];
-
- return 0;
-}
-
-static void dsi_pll_28nm_destroy(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_28nm *pll_28nm = to_pll_28nm(pll);
-
- msm_dsi_pll_helper_unregister_clks(pll_28nm->pdev,
- pll_28nm->clks, pll_28nm->num_clks);
-}
-
-static int pll_28nm_register(struct dsi_pll_28nm *pll_28nm)
-{
- char *clk_name, *parent_name, *vco_name;
- struct clk_init_data vco_init = {
- .parent_names = (const char *[]){ "pxo" },
- .num_parents = 1,
- .flags = CLK_IGNORE_UNUSED,
- .ops = &clk_ops_dsi_pll_28nm_vco,
- };
- struct device *dev = &pll_28nm->pdev->dev;
- struct clk **clks = pll_28nm->clks;
- struct clk **provided_clks = pll_28nm->provided_clks;
- struct clk_bytediv *bytediv;
- struct clk_init_data bytediv_init = { };
- int ret, num = 0;
-
- DBG("%d", pll_28nm->id);
-
- bytediv = devm_kzalloc(dev, sizeof(*bytediv), GFP_KERNEL);
- if (!bytediv)
- return -ENOMEM;
-
- vco_name = devm_kzalloc(dev, 32, GFP_KERNEL);
- if (!vco_name)
- return -ENOMEM;
-
- parent_name = devm_kzalloc(dev, 32, GFP_KERNEL);
- if (!parent_name)
- return -ENOMEM;
-
- clk_name = devm_kzalloc(dev, 32, GFP_KERNEL);
- if (!clk_name)
- return -ENOMEM;
-
- pll_28nm->bytediv = bytediv;
-
- snprintf(vco_name, 32, "dsi%dvco_clk", pll_28nm->id);
- vco_init.name = vco_name;
-
- pll_28nm->base.clk_hw.init = &vco_init;
-
- clks[num++] = clk_register(dev, &pll_28nm->base.clk_hw);
-
- /* prepare and register bytediv */
- bytediv->hw.init = &bytediv_init;
- bytediv->reg = pll_28nm->mmio + REG_DSI_28nm_8960_PHY_PLL_CTRL_9;
-
- snprintf(parent_name, 32, "dsi%dvco_clk", pll_28nm->id);
- snprintf(clk_name, 32, "dsi%dpllbyte", pll_28nm->id);
-
- bytediv_init.name = clk_name;
- bytediv_init.ops = &clk_bytediv_ops;
- bytediv_init.flags = CLK_SET_RATE_PARENT;
- bytediv_init.parent_names = (const char * const *) &parent_name;
- bytediv_init.num_parents = 1;
-
- /* DIV2 */
- clks[num++] = provided_clks[DSI_BYTE_PLL_CLK] =
- clk_register(dev, &bytediv->hw);
-
- snprintf(clk_name, 32, "dsi%dpll", pll_28nm->id);
- /* DIV3 */
- clks[num++] = provided_clks[DSI_PIXEL_PLL_CLK] =
- clk_register_divider(dev, clk_name,
- parent_name, 0, pll_28nm->mmio +
- REG_DSI_28nm_8960_PHY_PLL_CTRL_10,
- 0, 8, 0, NULL);
-
- pll_28nm->num_clks = num;
-
- pll_28nm->clk_data.clk_num = NUM_PROVIDED_CLKS;
- pll_28nm->clk_data.clks = provided_clks;
-
- ret = of_clk_add_provider(dev->of_node,
- of_clk_src_onecell_get, &pll_28nm->clk_data);
- if (ret) {
- DRM_DEV_ERROR(dev, "failed to register clk provider: %d\n", ret);
- return ret;
- }
-
- return 0;
-}
-
-struct msm_dsi_pll *msm_dsi_pll_28nm_8960_init(struct platform_device *pdev,
- int id)
-{
- struct dsi_pll_28nm *pll_28nm;
- struct msm_dsi_pll *pll;
- int ret;
-
- if (!pdev)
- return ERR_PTR(-ENODEV);
-
- pll_28nm = devm_kzalloc(&pdev->dev, sizeof(*pll_28nm), GFP_KERNEL);
- if (!pll_28nm)
- return ERR_PTR(-ENOMEM);
-
- pll_28nm->pdev = pdev;
- pll_28nm->id = id + 1;
-
- pll_28nm->mmio = msm_ioremap(pdev, "dsi_pll", "DSI_PLL");
- if (IS_ERR_OR_NULL(pll_28nm->mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "%s: failed to map pll base\n", __func__);
- return ERR_PTR(-ENOMEM);
- }
-
- pll = &pll_28nm->base;
- pll->min_rate = VCO_MIN_RATE;
- pll->max_rate = VCO_MAX_RATE;
- pll->get_provider = dsi_pll_28nm_get_provider;
- pll->destroy = dsi_pll_28nm_destroy;
- pll->disable_seq = dsi_pll_28nm_disable_seq;
- pll->save_state = dsi_pll_28nm_save_state;
- pll->restore_state = dsi_pll_28nm_restore_state;
-
- pll->en_seq_cnt = 1;
- pll->enable_seqs[0] = dsi_pll_28nm_enable_seq;
-
- ret = pll_28nm_register(pll_28nm);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
- return ERR_PTR(ret);
- }
-
- return pll;
-}
+++ /dev/null
-/*
- * SPDX-License-Identifier: GPL-2.0
- * Copyright (c) 2018, The Linux Foundation
- */
-
-#include <linux/clk.h>
-#include <linux/clk-provider.h>
-#include <linux/iopoll.h>
-
-#include "dsi_pll.h"
-#include "dsi.xml.h"
-
-/*
- * DSI PLL 7nm - clock diagram (eg: DSI0): TODO: updated CPHY diagram
- *
- * dsi0_pll_out_div_clk dsi0_pll_bit_clk
- * | |
- * | |
- * +---------+ | +----------+ | +----+
- * dsi0vco_clk ---| out_div |--o--| divl_3_0 |--o--| /8 |-- dsi0_phy_pll_out_byteclk
- * +---------+ | +----------+ | +----+
- * | |
- * | | dsi0_pll_by_2_bit_clk
- * | | |
- * | | +----+ | |\ dsi0_pclk_mux
- * | |--| /2 |--o--| \ |
- * | | +----+ | \ | +---------+
- * | --------------| |--o--| div_7_4 |-- dsi0_phy_pll_out_dsiclk
- * |------------------------------| / +---------+
- * | +-----+ | /
- * -----------| /4? |--o----------|/
- * +-----+ | |
- * | |dsiclk_sel
- * |
- * dsi0_pll_post_out_div_clk
- */
-
-#define DSI_BYTE_PLL_CLK 0
-#define DSI_PIXEL_PLL_CLK 1
-#define NUM_PROVIDED_CLKS 2
-
-#define VCO_REF_CLK_RATE 19200000
-
-struct dsi_pll_regs {
- u32 pll_prop_gain_rate;
- u32 pll_lockdet_rate;
- u32 decimal_div_start;
- u32 frac_div_start_low;
- u32 frac_div_start_mid;
- u32 frac_div_start_high;
- u32 pll_clock_inverters;
- u32 ssc_stepsize_low;
- u32 ssc_stepsize_high;
- u32 ssc_div_per_low;
- u32 ssc_div_per_high;
- u32 ssc_adjper_low;
- u32 ssc_adjper_high;
- u32 ssc_control;
-};
-
-struct dsi_pll_config {
- u32 ref_freq;
- bool div_override;
- u32 output_div;
- bool ignore_frac;
- bool disable_prescaler;
- bool enable_ssc;
- bool ssc_center;
- u32 dec_bits;
- u32 frac_bits;
- u32 lock_timer;
- u32 ssc_freq;
- u32 ssc_offset;
- u32 ssc_adj_per;
- u32 thresh_cycles;
- u32 refclk_cycles;
-};
-
-struct pll_7nm_cached_state {
- unsigned long vco_rate;
- u8 bit_clk_div;
- u8 pix_clk_div;
- u8 pll_out_div;
- u8 pll_mux;
-};
-
-struct dsi_pll_7nm {
- struct msm_dsi_pll base;
-
- int id;
- struct platform_device *pdev;
-
- void __iomem *phy_cmn_mmio;
- void __iomem *mmio;
-
- u64 vco_ref_clk_rate;
- u64 vco_current_rate;
-
- /* protects REG_DSI_7nm_PHY_CMN_CLK_CFG0 register */
- spinlock_t postdiv_lock;
-
- int vco_delay;
- struct dsi_pll_config pll_configuration;
- struct dsi_pll_regs reg_setup;
-
- /* private clocks: */
- struct clk_hw *out_div_clk_hw;
- struct clk_hw *bit_clk_hw;
- struct clk_hw *byte_clk_hw;
- struct clk_hw *by_2_bit_clk_hw;
- struct clk_hw *post_out_div_clk_hw;
- struct clk_hw *pclk_mux_hw;
- struct clk_hw *out_dsiclk_hw;
-
- /* clock-provider: */
- struct clk_hw_onecell_data *hw_data;
-
- struct pll_7nm_cached_state cached_state;
-
- enum msm_dsi_phy_usecase uc;
- struct dsi_pll_7nm *slave;
-};
-
-#define to_pll_7nm(x) container_of(x, struct dsi_pll_7nm, base)
-
-/*
- * Global list of private DSI PLL struct pointers. We need this for Dual DSI
- * mode, where the master PLL's clk_ops needs access the slave's private data
- */
-static struct dsi_pll_7nm *pll_7nm_list[DSI_MAX];
-
-static void dsi_pll_setup_config(struct dsi_pll_7nm *pll)
-{
- struct dsi_pll_config *config = &pll->pll_configuration;
-
- config->ref_freq = pll->vco_ref_clk_rate;
- config->output_div = 1;
- config->dec_bits = 8;
- config->frac_bits = 18;
- config->lock_timer = 64;
- config->ssc_freq = 31500;
- config->ssc_offset = 4800;
- config->ssc_adj_per = 2;
- config->thresh_cycles = 32;
- config->refclk_cycles = 256;
-
- config->div_override = false;
- config->ignore_frac = false;
- config->disable_prescaler = false;
-
- /* TODO: ssc enable */
- config->enable_ssc = false;
- config->ssc_center = 0;
-}
-
-static void dsi_pll_calc_dec_frac(struct dsi_pll_7nm *pll)
-{
- struct dsi_pll_config *config = &pll->pll_configuration;
- struct dsi_pll_regs *regs = &pll->reg_setup;
- u64 fref = pll->vco_ref_clk_rate;
- u64 pll_freq;
- u64 divider;
- u64 dec, dec_multiple;
- u32 frac;
- u64 multiplier;
-
- pll_freq = pll->vco_current_rate;
-
- if (config->disable_prescaler)
- divider = fref;
- else
- divider = fref * 2;
-
- multiplier = 1 << config->frac_bits;
- dec_multiple = div_u64(pll_freq * multiplier, divider);
- div_u64_rem(dec_multiple, multiplier, &frac);
-
- dec = div_u64(dec_multiple, multiplier);
-
- if (pll->base.type != MSM_DSI_PHY_7NM_V4_1)
- regs->pll_clock_inverters = 0x28;
- else if (pll_freq <= 1000000000ULL)
- regs->pll_clock_inverters = 0xa0;
- else if (pll_freq <= 2500000000ULL)
- regs->pll_clock_inverters = 0x20;
- else if (pll_freq <= 3020000000ULL)
- regs->pll_clock_inverters = 0x00;
- else
- regs->pll_clock_inverters = 0x40;
-
- regs->pll_lockdet_rate = config->lock_timer;
- regs->decimal_div_start = dec;
- regs->frac_div_start_low = (frac & 0xff);
- regs->frac_div_start_mid = (frac & 0xff00) >> 8;
- regs->frac_div_start_high = (frac & 0x30000) >> 16;
-}
-
-#define SSC_CENTER BIT(0)
-#define SSC_EN BIT(1)
-
-static void dsi_pll_calc_ssc(struct dsi_pll_7nm *pll)
-{
- struct dsi_pll_config *config = &pll->pll_configuration;
- struct dsi_pll_regs *regs = &pll->reg_setup;
- u32 ssc_per;
- u32 ssc_mod;
- u64 ssc_step_size;
- u64 frac;
-
- if (!config->enable_ssc) {
- DBG("SSC not enabled\n");
- return;
- }
-
- ssc_per = DIV_ROUND_CLOSEST(config->ref_freq, config->ssc_freq) / 2 - 1;
- ssc_mod = (ssc_per + 1) % (config->ssc_adj_per + 1);
- ssc_per -= ssc_mod;
-
- frac = regs->frac_div_start_low |
- (regs->frac_div_start_mid << 8) |
- (regs->frac_div_start_high << 16);
- ssc_step_size = regs->decimal_div_start;
- ssc_step_size *= (1 << config->frac_bits);
- ssc_step_size += frac;
- ssc_step_size *= config->ssc_offset;
- ssc_step_size *= (config->ssc_adj_per + 1);
- ssc_step_size = div_u64(ssc_step_size, (ssc_per + 1));
- ssc_step_size = DIV_ROUND_CLOSEST_ULL(ssc_step_size, 1000000);
-
- regs->ssc_div_per_low = ssc_per & 0xFF;
- regs->ssc_div_per_high = (ssc_per & 0xFF00) >> 8;
- regs->ssc_stepsize_low = (u32)(ssc_step_size & 0xFF);
- regs->ssc_stepsize_high = (u32)((ssc_step_size & 0xFF00) >> 8);
- regs->ssc_adjper_low = config->ssc_adj_per & 0xFF;
- regs->ssc_adjper_high = (config->ssc_adj_per & 0xFF00) >> 8;
-
- regs->ssc_control = config->ssc_center ? SSC_CENTER : 0;
-
- pr_debug("SCC: Dec:%d, frac:%llu, frac_bits:%d\n",
- regs->decimal_div_start, frac, config->frac_bits);
- pr_debug("SSC: div_per:0x%X, stepsize:0x%X, adjper:0x%X\n",
- ssc_per, (u32)ssc_step_size, config->ssc_adj_per);
-}
-
-static void dsi_pll_ssc_commit(struct dsi_pll_7nm *pll)
-{
- void __iomem *base = pll->mmio;
- struct dsi_pll_regs *regs = &pll->reg_setup;
-
- if (pll->pll_configuration.enable_ssc) {
- pr_debug("SSC is enabled\n");
-
- pll_write(base + REG_DSI_7nm_PHY_PLL_SSC_STEPSIZE_LOW_1,
- regs->ssc_stepsize_low);
- pll_write(base + REG_DSI_7nm_PHY_PLL_SSC_STEPSIZE_HIGH_1,
- regs->ssc_stepsize_high);
- pll_write(base + REG_DSI_7nm_PHY_PLL_SSC_DIV_PER_LOW_1,
- regs->ssc_div_per_low);
- pll_write(base + REG_DSI_7nm_PHY_PLL_SSC_DIV_PER_HIGH_1,
- regs->ssc_div_per_high);
- pll_write(base + REG_DSI_7nm_PHY_PLL_SSC_ADJPER_LOW_1,
- regs->ssc_adjper_low);
- pll_write(base + REG_DSI_7nm_PHY_PLL_SSC_ADJPER_HIGH_1,
- regs->ssc_adjper_high);
- pll_write(base + REG_DSI_7nm_PHY_PLL_SSC_CONTROL,
- SSC_EN | regs->ssc_control);
- }
-}
-
-static void dsi_pll_config_hzindep_reg(struct dsi_pll_7nm *pll)
-{
- void __iomem *base = pll->mmio;
- u8 analog_controls_five_1 = 0x01, vco_config_1 = 0x00;
-
- if (pll->base.type == MSM_DSI_PHY_7NM_V4_1) {
- if (pll->vco_current_rate >= 3100000000ULL)
- analog_controls_five_1 = 0x03;
-
- if (pll->vco_current_rate < 1520000000ULL)
- vco_config_1 = 0x08;
- else if (pll->vco_current_rate < 2990000000ULL)
- vco_config_1 = 0x01;
- }
-
- pll_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_FIVE_1,
- analog_controls_five_1);
- pll_write(base + REG_DSI_7nm_PHY_PLL_VCO_CONFIG_1, vco_config_1);
- pll_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_FIVE, 0x01);
- pll_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_TWO, 0x03);
- pll_write(base + REG_DSI_7nm_PHY_PLL_ANALOG_CONTROLS_THREE, 0x00);
- pll_write(base + REG_DSI_7nm_PHY_PLL_DSM_DIVIDER, 0x00);
- pll_write(base + REG_DSI_7nm_PHY_PLL_FEEDBACK_DIVIDER, 0x4e);
- pll_write(base + REG_DSI_7nm_PHY_PLL_CALIBRATION_SETTINGS, 0x40);
- pll_write(base + REG_DSI_7nm_PHY_PLL_BAND_SEL_CAL_SETTINGS_THREE, 0xba);
- pll_write(base + REG_DSI_7nm_PHY_PLL_FREQ_DETECT_SETTINGS_ONE, 0x0c);
- pll_write(base + REG_DSI_7nm_PHY_PLL_OUTDIV, 0x00);
- pll_write(base + REG_DSI_7nm_PHY_PLL_CORE_OVERRIDE, 0x00);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_DIGITAL_TIMERS_TWO, 0x08);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_PROP_GAIN_RATE_1, 0x0a);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_BAND_SEL_RATE_1, 0xc0);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_INT_GAIN_IFILT_BAND_1, 0x84);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_INT_GAIN_IFILT_BAND_1, 0x82);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_FL_INT_GAIN_PFILT_BAND_1, 0x4c);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_LOCK_OVERRIDE, 0x80);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PFILT, 0x29);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PFILT, 0x2f);
- pll_write(base + REG_DSI_7nm_PHY_PLL_IFILT, 0x2a);
- pll_write(base + REG_DSI_7nm_PHY_PLL_IFILT,
- pll->base.type == MSM_DSI_PHY_7NM_V4_1 ? 0x3f : 0x22);
-
- if (pll->base.type == MSM_DSI_PHY_7NM_V4_1) {
- pll_write(base + REG_DSI_7nm_PHY_PLL_PERF_OPTIMIZE, 0x22);
- if (pll->slave)
- pll_write(pll->slave->mmio + REG_DSI_7nm_PHY_PLL_PERF_OPTIMIZE, 0x22);
- }
-}
-
-static void dsi_pll_commit(struct dsi_pll_7nm *pll)
-{
- void __iomem *base = pll->mmio;
- struct dsi_pll_regs *reg = &pll->reg_setup;
-
- pll_write(base + REG_DSI_7nm_PHY_PLL_CORE_INPUT_OVERRIDE, 0x12);
- pll_write(base + REG_DSI_7nm_PHY_PLL_DECIMAL_DIV_START_1, reg->decimal_div_start);
- pll_write(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_LOW_1, reg->frac_div_start_low);
- pll_write(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_MID_1, reg->frac_div_start_mid);
- pll_write(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_HIGH_1, reg->frac_div_start_high);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_LOCKDET_RATE_1, 0x40);
- pll_write(base + REG_DSI_7nm_PHY_PLL_PLL_LOCK_DELAY, 0x06);
- pll_write(base + REG_DSI_7nm_PHY_PLL_CMODE_1, 0x10); /* TODO: 0x00 for CPHY */
- pll_write(base + REG_DSI_7nm_PHY_PLL_CLOCK_INVERTERS, reg->pll_clock_inverters);
-}
-
-static int dsi_pll_7nm_vco_set_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
-
- DBG("DSI PLL%d rate=%lu, parent's=%lu", pll_7nm->id, rate,
- parent_rate);
-
- pll_7nm->vco_current_rate = rate;
- pll_7nm->vco_ref_clk_rate = VCO_REF_CLK_RATE;
-
- dsi_pll_setup_config(pll_7nm);
-
- dsi_pll_calc_dec_frac(pll_7nm);
-
- dsi_pll_calc_ssc(pll_7nm);
-
- dsi_pll_commit(pll_7nm);
-
- dsi_pll_config_hzindep_reg(pll_7nm);
-
- dsi_pll_ssc_commit(pll_7nm);
-
- /* flush, ensure all register writes are done*/
- wmb();
-
- return 0;
-}
-
-static int dsi_pll_7nm_lock_status(struct dsi_pll_7nm *pll)
-{
- int rc;
- u32 status = 0;
- u32 const delay_us = 100;
- u32 const timeout_us = 5000;
-
- rc = readl_poll_timeout_atomic(pll->mmio +
- REG_DSI_7nm_PHY_PLL_COMMON_STATUS_ONE,
- status,
- ((status & BIT(0)) > 0),
- delay_us,
- timeout_us);
- if (rc)
- pr_err("DSI PLL(%d) lock failed, status=0x%08x\n",
- pll->id, status);
-
- return rc;
-}
-
-static void dsi_pll_disable_pll_bias(struct dsi_pll_7nm *pll)
-{
- u32 data = pll_read(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CTRL_0);
-
- pll_write(pll->mmio + REG_DSI_7nm_PHY_PLL_SYSTEM_MUXES, 0);
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CTRL_0, data & ~BIT(5));
- ndelay(250);
-}
-
-static void dsi_pll_enable_pll_bias(struct dsi_pll_7nm *pll)
-{
- u32 data = pll_read(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CTRL_0);
-
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CTRL_0, data | BIT(5));
- pll_write(pll->mmio + REG_DSI_7nm_PHY_PLL_SYSTEM_MUXES, 0xc0);
- ndelay(250);
-}
-
-static void dsi_pll_disable_global_clk(struct dsi_pll_7nm *pll)
-{
- u32 data;
-
- data = pll_read(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CLK_CFG1, data & ~BIT(5));
-}
-
-static void dsi_pll_enable_global_clk(struct dsi_pll_7nm *pll)
-{
- u32 data;
-
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CTRL_3, 0x04);
-
- data = pll_read(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_CLK_CFG1,
- data | BIT(5) | BIT(4));
-}
-
-static void dsi_pll_phy_dig_reset(struct dsi_pll_7nm *pll)
-{
- /*
- * Reset the PHY digital domain. This would be needed when
- * coming out of a CX or analog rail power collapse while
- * ensuring that the pads maintain LP00 or LP11 state
- */
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE4, BIT(0));
- wmb(); /* Ensure that the reset is deasserted */
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE4, 0x0);
- wmb(); /* Ensure that the reset is deasserted */
-}
-
-static int dsi_pll_7nm_vco_prepare(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
- int rc;
-
- dsi_pll_enable_pll_bias(pll_7nm);
- if (pll_7nm->slave)
- dsi_pll_enable_pll_bias(pll_7nm->slave);
-
- /* Start PLL */
- pll_write(pll_7nm->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_PLL_CNTRL, 0x01);
-
- /*
- * ensure all PLL configurations are written prior to checking
- * for PLL lock.
- */
- wmb();
-
- /* Check for PLL lock */
- rc = dsi_pll_7nm_lock_status(pll_7nm);
- if (rc) {
- pr_err("PLL(%d) lock failed\n", pll_7nm->id);
- goto error;
- }
-
- pll->pll_on = true;
-
- /*
- * assert power on reset for PHY digital in case the PLL is
- * enabled after CX of analog domain power collapse. This needs
- * to be done before enabling the global clk.
- */
- dsi_pll_phy_dig_reset(pll_7nm);
- if (pll_7nm->slave)
- dsi_pll_phy_dig_reset(pll_7nm->slave);
-
- dsi_pll_enable_global_clk(pll_7nm);
- if (pll_7nm->slave)
- dsi_pll_enable_global_clk(pll_7nm->slave);
-
-error:
- return rc;
-}
-
-static void dsi_pll_disable_sub(struct dsi_pll_7nm *pll)
-{
- pll_write(pll->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_RBUF_CTRL, 0);
- dsi_pll_disable_pll_bias(pll);
-}
-
-static void dsi_pll_7nm_vco_unprepare(struct clk_hw *hw)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
-
- /*
- * To avoid any stray glitches while abruptly powering down the PLL
- * make sure to gate the clock using the clock enable bit before
- * powering down the PLL
- */
- dsi_pll_disable_global_clk(pll_7nm);
- pll_write(pll_7nm->phy_cmn_mmio + REG_DSI_7nm_PHY_CMN_PLL_CNTRL, 0);
- dsi_pll_disable_sub(pll_7nm);
- if (pll_7nm->slave) {
- dsi_pll_disable_global_clk(pll_7nm->slave);
- dsi_pll_disable_sub(pll_7nm->slave);
- }
- /* flush, ensure all register writes are done */
- wmb();
- pll->pll_on = false;
-}
-
-static unsigned long dsi_pll_7nm_vco_recalc_rate(struct clk_hw *hw,
- unsigned long parent_rate)
-{
- struct msm_dsi_pll *pll = hw_clk_to_pll(hw);
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
- void __iomem *base = pll_7nm->mmio;
- u64 ref_clk = pll_7nm->vco_ref_clk_rate;
- u64 vco_rate = 0x0;
- u64 multiplier;
- u32 frac;
- u32 dec;
- u64 pll_freq, tmp64;
-
- dec = pll_read(base + REG_DSI_7nm_PHY_PLL_DECIMAL_DIV_START_1);
- dec &= 0xff;
-
- frac = pll_read(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_LOW_1);
- frac |= ((pll_read(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_MID_1) &
- 0xff) << 8);
- frac |= ((pll_read(base + REG_DSI_7nm_PHY_PLL_FRAC_DIV_START_HIGH_1) &
- 0x3) << 16);
-
- /*
- * TODO:
- * 1. Assumes prescaler is disabled
- * 2. Multiplier is 2^18. it should be 2^(num_of_frac_bits)
- */
- multiplier = 1 << 18;
- pll_freq = dec * (ref_clk * 2);
- tmp64 = (ref_clk * 2 * frac);
- pll_freq += div_u64(tmp64, multiplier);
-
- vco_rate = pll_freq;
-
- DBG("DSI PLL%d returning vco rate = %lu, dec = %x, frac = %x",
- pll_7nm->id, (unsigned long)vco_rate, dec, frac);
-
- return (unsigned long)vco_rate;
-}
-
-static const struct clk_ops clk_ops_dsi_pll_7nm_vco = {
- .round_rate = msm_dsi_pll_helper_clk_round_rate,
- .set_rate = dsi_pll_7nm_vco_set_rate,
- .recalc_rate = dsi_pll_7nm_vco_recalc_rate,
- .prepare = dsi_pll_7nm_vco_prepare,
- .unprepare = dsi_pll_7nm_vco_unprepare,
-};
-
-/*
- * PLL Callbacks
- */
-
-static void dsi_pll_7nm_save_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
- struct pll_7nm_cached_state *cached = &pll_7nm->cached_state;
- void __iomem *phy_base = pll_7nm->phy_cmn_mmio;
- u32 cmn_clk_cfg0, cmn_clk_cfg1;
-
- cached->pll_out_div = pll_read(pll_7nm->mmio +
- REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE);
- cached->pll_out_div &= 0x3;
-
- cmn_clk_cfg0 = pll_read(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG0);
- cached->bit_clk_div = cmn_clk_cfg0 & 0xf;
- cached->pix_clk_div = (cmn_clk_cfg0 & 0xf0) >> 4;
-
- cmn_clk_cfg1 = pll_read(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
- cached->pll_mux = cmn_clk_cfg1 & 0x3;
-
- DBG("DSI PLL%d outdiv %x bit_clk_div %x pix_clk_div %x pll_mux %x",
- pll_7nm->id, cached->pll_out_div, cached->bit_clk_div,
- cached->pix_clk_div, cached->pll_mux);
-}
-
-static int dsi_pll_7nm_restore_state(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
- struct pll_7nm_cached_state *cached = &pll_7nm->cached_state;
- void __iomem *phy_base = pll_7nm->phy_cmn_mmio;
- u32 val;
- int ret;
-
- val = pll_read(pll_7nm->mmio + REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE);
- val &= ~0x3;
- val |= cached->pll_out_div;
- pll_write(pll_7nm->mmio + REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE, val);
-
- pll_write(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG0,
- cached->bit_clk_div | (cached->pix_clk_div << 4));
-
- val = pll_read(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG1);
- val &= ~0x3;
- val |= cached->pll_mux;
- pll_write(phy_base + REG_DSI_7nm_PHY_CMN_CLK_CFG1, val);
-
- ret = dsi_pll_7nm_vco_set_rate(&pll->clk_hw, pll_7nm->vco_current_rate, pll_7nm->vco_ref_clk_rate);
- if (ret) {
- DRM_DEV_ERROR(&pll_7nm->pdev->dev,
- "restore vco rate failed. ret=%d\n", ret);
- return ret;
- }
-
- DBG("DSI PLL%d", pll_7nm->id);
-
- return 0;
-}
-
-static int dsi_pll_7nm_set_usecase(struct msm_dsi_pll *pll,
- enum msm_dsi_phy_usecase uc)
-{
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
- void __iomem *base = pll_7nm->phy_cmn_mmio;
- u32 data = 0x0; /* internal PLL */
-
- DBG("DSI PLL%d", pll_7nm->id);
-
- switch (uc) {
- case MSM_DSI_PHY_STANDALONE:
- break;
- case MSM_DSI_PHY_MASTER:
- pll_7nm->slave = pll_7nm_list[(pll_7nm->id + 1) % DSI_MAX];
- break;
- case MSM_DSI_PHY_SLAVE:
- data = 0x1; /* external PLL */
- break;
- default:
- return -EINVAL;
- }
-
- /* set PLL src */
- pll_write(base + REG_DSI_7nm_PHY_CMN_CLK_CFG1, (data << 2));
-
- pll_7nm->uc = uc;
-
- return 0;
-}
-
-static int dsi_pll_7nm_get_provider(struct msm_dsi_pll *pll,
- struct clk **byte_clk_provider,
- struct clk **pixel_clk_provider)
-{
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
- struct clk_hw_onecell_data *hw_data = pll_7nm->hw_data;
-
- DBG("DSI PLL%d", pll_7nm->id);
-
- if (byte_clk_provider)
- *byte_clk_provider = hw_data->hws[DSI_BYTE_PLL_CLK]->clk;
- if (pixel_clk_provider)
- *pixel_clk_provider = hw_data->hws[DSI_PIXEL_PLL_CLK]->clk;
-
- return 0;
-}
-
-static void dsi_pll_7nm_destroy(struct msm_dsi_pll *pll)
-{
- struct dsi_pll_7nm *pll_7nm = to_pll_7nm(pll);
- struct device *dev = &pll_7nm->pdev->dev;
-
- DBG("DSI PLL%d", pll_7nm->id);
- of_clk_del_provider(dev->of_node);
-
- clk_hw_unregister_divider(pll_7nm->out_dsiclk_hw);
- clk_hw_unregister_mux(pll_7nm->pclk_mux_hw);
- clk_hw_unregister_fixed_factor(pll_7nm->post_out_div_clk_hw);
- clk_hw_unregister_fixed_factor(pll_7nm->by_2_bit_clk_hw);
- clk_hw_unregister_fixed_factor(pll_7nm->byte_clk_hw);
- clk_hw_unregister_divider(pll_7nm->bit_clk_hw);
- clk_hw_unregister_divider(pll_7nm->out_div_clk_hw);
- clk_hw_unregister(&pll_7nm->base.clk_hw);
-}
-
-/*
- * The post dividers and mux clocks are created using the standard divider and
- * mux API. Unlike the 14nm PHY, the slave PLL doesn't need its dividers/mux
- * state to follow the master PLL's divider/mux state. Therefore, we don't
- * require special clock ops that also configure the slave PLL registers
- */
-static int pll_7nm_register(struct dsi_pll_7nm *pll_7nm)
-{
- char clk_name[32], parent[32], vco_name[32];
- char parent2[32], parent3[32], parent4[32];
- struct clk_init_data vco_init = {
- .parent_names = (const char *[]){ "bi_tcxo" },
- .num_parents = 1,
- .name = vco_name,
- .flags = CLK_IGNORE_UNUSED,
- .ops = &clk_ops_dsi_pll_7nm_vco,
- };
- struct device *dev = &pll_7nm->pdev->dev;
- struct clk_hw_onecell_data *hw_data;
- struct clk_hw *hw;
- int ret;
-
- DBG("DSI%d", pll_7nm->id);
-
- hw_data = devm_kzalloc(dev, sizeof(*hw_data) +
- NUM_PROVIDED_CLKS * sizeof(struct clk_hw *),
- GFP_KERNEL);
- if (!hw_data)
- return -ENOMEM;
-
- snprintf(vco_name, 32, "dsi%dvco_clk", pll_7nm->id);
- pll_7nm->base.clk_hw.init = &vco_init;
-
- ret = clk_hw_register(dev, &pll_7nm->base.clk_hw);
- if (ret)
- return ret;
-
- snprintf(clk_name, 32, "dsi%d_pll_out_div_clk", pll_7nm->id);
- snprintf(parent, 32, "dsi%dvco_clk", pll_7nm->id);
-
- hw = clk_hw_register_divider(dev, clk_name,
- parent, CLK_SET_RATE_PARENT,
- pll_7nm->mmio +
- REG_DSI_7nm_PHY_PLL_PLL_OUTDIV_RATE,
- 0, 2, CLK_DIVIDER_POWER_OF_TWO, NULL);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_base_clk_hw;
- }
-
- pll_7nm->out_div_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_pll_bit_clk", pll_7nm->id);
- snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_7nm->id);
-
- /* BIT CLK: DIV_CTRL_3_0 */
- hw = clk_hw_register_divider(dev, clk_name, parent,
- CLK_SET_RATE_PARENT,
- pll_7nm->phy_cmn_mmio +
- REG_DSI_7nm_PHY_CMN_CLK_CFG0,
- 0, 4, CLK_DIVIDER_ONE_BASED,
- &pll_7nm->postdiv_lock);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_out_div_clk_hw;
- }
-
- pll_7nm->bit_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_phy_pll_out_byteclk", pll_7nm->id);
- snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_7nm->id);
-
- /* DSI Byte clock = VCO_CLK / OUT_DIV / BIT_DIV / 8 */
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent,
- CLK_SET_RATE_PARENT, 1, 8);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_bit_clk_hw;
- }
-
- pll_7nm->byte_clk_hw = hw;
- hw_data->hws[DSI_BYTE_PLL_CLK] = hw;
-
- snprintf(clk_name, 32, "dsi%d_pll_by_2_bit_clk", pll_7nm->id);
- snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_7nm->id);
-
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent,
- 0, 1, 2);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_byte_clk_hw;
- }
-
- pll_7nm->by_2_bit_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_pll_post_out_div_clk", pll_7nm->id);
- snprintf(parent, 32, "dsi%d_pll_out_div_clk", pll_7nm->id);
-
- hw = clk_hw_register_fixed_factor(dev, clk_name, parent,
- 0, 1, 4);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_by_2_bit_clk_hw;
- }
-
- pll_7nm->post_out_div_clk_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_pclk_mux", pll_7nm->id);
- snprintf(parent, 32, "dsi%d_pll_bit_clk", pll_7nm->id);
- snprintf(parent2, 32, "dsi%d_pll_by_2_bit_clk", pll_7nm->id);
- snprintf(parent3, 32, "dsi%d_pll_out_div_clk", pll_7nm->id);
- snprintf(parent4, 32, "dsi%d_pll_post_out_div_clk", pll_7nm->id);
-
- hw = clk_hw_register_mux(dev, clk_name,
- ((const char *[]){
- parent, parent2, parent3, parent4
- }), 4, 0, pll_7nm->phy_cmn_mmio +
- REG_DSI_7nm_PHY_CMN_CLK_CFG1,
- 0, 2, 0, NULL);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_post_out_div_clk_hw;
- }
-
- pll_7nm->pclk_mux_hw = hw;
-
- snprintf(clk_name, 32, "dsi%d_phy_pll_out_dsiclk", pll_7nm->id);
- snprintf(parent, 32, "dsi%d_pclk_mux", pll_7nm->id);
-
- /* PIX CLK DIV : DIV_CTRL_7_4*/
- hw = clk_hw_register_divider(dev, clk_name, parent,
- 0, pll_7nm->phy_cmn_mmio +
- REG_DSI_7nm_PHY_CMN_CLK_CFG0,
- 4, 4, CLK_DIVIDER_ONE_BASED,
- &pll_7nm->postdiv_lock);
- if (IS_ERR(hw)) {
- ret = PTR_ERR(hw);
- goto err_pclk_mux_hw;
- }
-
- pll_7nm->out_dsiclk_hw = hw;
- hw_data->hws[DSI_PIXEL_PLL_CLK] = hw;
-
- hw_data->num = NUM_PROVIDED_CLKS;
- pll_7nm->hw_data = hw_data;
-
- ret = of_clk_add_hw_provider(dev->of_node, of_clk_hw_onecell_get,
- pll_7nm->hw_data);
- if (ret) {
- DRM_DEV_ERROR(dev, "failed to register clk provider: %d\n", ret);
- goto err_dsiclk_hw;
- }
-
- return 0;
-
-err_dsiclk_hw:
- clk_hw_unregister_divider(pll_7nm->out_dsiclk_hw);
-err_pclk_mux_hw:
- clk_hw_unregister_mux(pll_7nm->pclk_mux_hw);
-err_post_out_div_clk_hw:
- clk_hw_unregister_fixed_factor(pll_7nm->post_out_div_clk_hw);
-err_by_2_bit_clk_hw:
- clk_hw_unregister_fixed_factor(pll_7nm->by_2_bit_clk_hw);
-err_byte_clk_hw:
- clk_hw_unregister_fixed_factor(pll_7nm->byte_clk_hw);
-err_bit_clk_hw:
- clk_hw_unregister_divider(pll_7nm->bit_clk_hw);
-err_out_div_clk_hw:
- clk_hw_unregister_divider(pll_7nm->out_div_clk_hw);
-err_base_clk_hw:
- clk_hw_unregister(&pll_7nm->base.clk_hw);
-
- return ret;
-}
-
-struct msm_dsi_pll *msm_dsi_pll_7nm_init(struct platform_device *pdev, int id)
-{
- struct dsi_pll_7nm *pll_7nm;
- struct msm_dsi_pll *pll;
- int ret;
-
- pll_7nm = devm_kzalloc(&pdev->dev, sizeof(*pll_7nm), GFP_KERNEL);
- if (!pll_7nm)
- return ERR_PTR(-ENOMEM);
-
- DBG("DSI PLL%d", id);
-
- pll_7nm->pdev = pdev;
- pll_7nm->id = id;
- pll_7nm_list[id] = pll_7nm;
-
- pll_7nm->phy_cmn_mmio = msm_ioremap(pdev, "dsi_phy", "DSI_PHY");
- if (IS_ERR_OR_NULL(pll_7nm->phy_cmn_mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "failed to map CMN PHY base\n");
- return ERR_PTR(-ENOMEM);
- }
-
- pll_7nm->mmio = msm_ioremap(pdev, "dsi_pll", "DSI_PLL");
- if (IS_ERR_OR_NULL(pll_7nm->mmio)) {
- DRM_DEV_ERROR(&pdev->dev, "failed to map PLL base\n");
- return ERR_PTR(-ENOMEM);
- }
-
- spin_lock_init(&pll_7nm->postdiv_lock);
-
- pll = &pll_7nm->base;
- pll->min_rate = 1000000000UL;
- pll->max_rate = 3500000000UL;
- if (pll->type == MSM_DSI_PHY_7NM_V4_1) {
- pll->min_rate = 600000000UL;
- pll->max_rate = (unsigned long)5000000000ULL;
- /* workaround for max rate overflowing on 32-bit builds: */
- pll->max_rate = max(pll->max_rate, 0xffffffffUL);
- }
- pll->get_provider = dsi_pll_7nm_get_provider;
- pll->destroy = dsi_pll_7nm_destroy;
- pll->save_state = dsi_pll_7nm_save_state;
- pll->restore_state = dsi_pll_7nm_restore_state;
- pll->set_usecase = dsi_pll_7nm_set_usecase;
-
- pll_7nm->vco_delay = 1;
-
- ret = pll_7nm_register(pll_7nm);
- if (ret) {
- DRM_DEV_ERROR(&pdev->dev, "failed to register PLL: %d\n", ret);
- return ERR_PTR(ret);
- }
-
- /* TODO: Remove this when we have proper display handover support */
- msm_dsi_pll_save_state(pll);
-
- return pll;
-}
static void lock_crtcs(struct msm_kms *kms, unsigned int crtc_mask)
{
+ int crtc_index;
struct drm_crtc *crtc;
- for_each_crtc_mask(kms->dev, crtc, crtc_mask)
- mutex_lock(&kms->commit_lock[drm_crtc_index(crtc)]);
+ for_each_crtc_mask(kms->dev, crtc, crtc_mask) {
+ crtc_index = drm_crtc_index(crtc);
+ mutex_lock_nested(&kms->commit_lock[crtc_index], crtc_index);
+ }
}
static void unlock_crtcs(struct msm_kms *kms, unsigned int crtc_mask)
static int msm_gem_show(struct drm_device *dev, struct seq_file *m)
{
struct msm_drm_private *priv = dev->dev_private;
- struct msm_gpu *gpu = priv->gpu;
int ret;
- ret = mutex_lock_interruptible(&priv->mm_lock);
+ ret = mutex_lock_interruptible(&priv->obj_lock);
if (ret)
return ret;
- if (gpu) {
- seq_printf(m, "Active Objects (%s):\n", gpu->name);
- msm_gem_describe_objects(&gpu->active_list, m);
- }
-
- seq_printf(m, "Inactive Objects:\n");
- msm_gem_describe_objects(&priv->inactive_dontneed, m);
- msm_gem_describe_objects(&priv->inactive_willneed, m);
+ msm_gem_describe_objects(&priv->objects, m);
- mutex_unlock(&priv->mm_lock);
+ mutex_unlock(&priv->obj_lock);
return 0;
}
* GEM object's debug name
* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl
* - 1.6.0 - Syncobj support
+ * - 1.7.0 - Add MSM_PARAM_SUSPENDS to access suspend count
*/
#define MSM_VERSION_MAJOR 1
#define MSM_VERSION_MINOR 6
priv->wq = alloc_ordered_workqueue("msm", 0);
+ INIT_LIST_HEAD(&priv->objects);
+ mutex_init(&priv->obj_lock);
+
INIT_LIST_HEAD(&priv->inactive_willneed);
INIT_LIST_HEAD(&priv->inactive_dontneed);
+ INIT_LIST_HEAD(&priv->inactive_unpinned);
mutex_init(&priv->mm_lock);
/* Teach lockdep about lock ordering wrt. shrinker: */
kfree(priv);
err_put_drm_dev:
drm_dev_put(ddev);
+ platform_set_drvdata(pdev, NULL);
return ret;
}
static int __maybe_unused msm_pm_prepare(struct device *dev)
{
struct drm_device *ddev = dev_get_drvdata(dev);
+ struct msm_drm_private *priv = ddev ? ddev->dev_private : NULL;
+
+ if (!priv || !priv->kms)
+ return 0;
return drm_mode_config_helper_suspend(ddev);
}
static void __maybe_unused msm_pm_complete(struct device *dev)
{
struct drm_device *ddev = dev_get_drvdata(dev);
+ struct msm_drm_private *priv = ddev ? ddev->dev_private : NULL;
+
+ if (!priv || !priv->kms)
+ return;
drm_mode_config_helper_resume(ddev);
}
return (strstr(dev_name(dev), "mdp") != NULL);
}
-static int add_display_components(struct device *dev,
+static int add_display_components(struct platform_device *pdev,
struct component_match **matchptr)
{
struct device *mdp_dev;
+ struct device *dev = &pdev->dev;
int ret;
/*
* Populate the children devices, find the MDP5/DPU node, and then add
* the interfaces to our components list.
*/
- if (of_device_is_compatible(dev->of_node, "qcom,mdss") ||
- of_device_is_compatible(dev->of_node, "qcom,sdm845-mdss") ||
- of_device_is_compatible(dev->of_node, "qcom,sc7180-mdss")) {
+ switch (get_mdp_ver(pdev)) {
+ case KMS_MDP5:
+ case KMS_DPU:
ret = of_platform_populate(dev->of_node, NULL, NULL, dev);
if (ret) {
DRM_DEV_ERROR(dev, "failed to populate children devices\n");
/* add the MDP component itself */
drm_of_component_match_add(dev, matchptr, compare_of,
mdp_dev->of_node);
- } else {
+ break;
+ case KMS_MDP4:
/* MDP4 */
mdp_dev = dev;
+ break;
}
ret = add_components_mdp(mdp_dev, matchptr);
int ret;
if (get_mdp_ver(pdev)) {
- ret = add_display_components(&pdev->dev, &match);
+ ret = add_display_components(pdev, &match);
if (ret)
return ret;
}
static void msm_pdev_shutdown(struct platform_device *pdev)
{
struct drm_device *drm = platform_get_drvdata(pdev);
+ struct msm_drm_private *priv = drm ? drm->dev_private : NULL;
+
+ if (!priv || !priv->kms)
+ return;
drm_atomic_helper_shutdown(drm);
}
{ .compatible = "qcom,mdss", .data = (void *)KMS_MDP5 },
{ .compatible = "qcom,sdm845-mdss", .data = (void *)KMS_DPU },
{ .compatible = "qcom,sc7180-mdss", .data = (void *)KMS_DPU },
+ { .compatible = "qcom,sc7280-mdss", .data = (void *)KMS_DPU },
+ { .compatible = "qcom,sm8150-mdss", .data = (void *)KMS_DPU },
+ { .compatible = "qcom,sm8250-mdss", .data = (void *)KMS_DPU },
{}
};
MODULE_DEVICE_TABLE(of, dt_match);
struct msm_rd_state *hangrd; /* debugfs to dump hanging submits */
struct msm_perf_state *perf;
- /*
- * Lists of inactive GEM objects. Every bo is either in one of the
+ /**
+ * List of all GEM objects (mainly for debugfs, protected by obj_lock
+ * (acquire before per GEM object lock)
+ */
+ struct list_head objects;
+ struct mutex obj_lock;
+
+ /**
+ * LRUs of inactive GEM objects. Every bo is either in one of the
* inactive lists (depending on whether or not it is shrinkable) or
- * gpu->active_list (for the gpu it is active on[1])
+ * gpu->active_list (for the gpu it is active on[1]), or transiently
+ * on a temporary list as the shrinker is running.
+ *
+ * Note that inactive_willneed also contains pinned and vmap'd bos,
+ * but the number of pinned-but-not-active objects is small (scanout
+ * buffers, ringbuffer, etc).
*
- * These lists are protected by mm_lock. If struct_mutex is involved, it
- * should be aquired prior to mm_lock. One should *not* hold mm_lock in
+ * These lists are protected by mm_lock (which should be acquired
+ * before per GEM object lock). One should *not* hold mm_lock in
* get_pages()/vmap()/etc paths, as they can trigger the shrinker.
*
* [1] if someone ever added support for the old 2d cores, there could be
* more than one gpu object
*/
- struct list_head inactive_willneed; /* inactive + !shrinkable */
- struct list_head inactive_dontneed; /* inactive + shrinkable */
+ struct list_head inactive_willneed; /* inactive + potentially unpin/evictable */
+ struct list_head inactive_dontneed; /* inactive + shrinkable */
+ struct list_head inactive_unpinned; /* inactive + purged or unpinned */
+ long shrinkable_count; /* write access under mm_lock */
+ long evictable_count; /* write access under mm_lock */
struct mutex mm_lock;
struct workqueue_struct *wq;
#ifdef CONFIG_DEBUG_FS
void msm_framebuffer_describe(struct drm_framebuffer *fb, struct seq_file *m)
{
+ struct msm_gem_stats stats = {};
int i, n = fb->format->num_planes;
seq_printf(m, "fb: %dx%d@%4.4s (%2d, ID:%d)\n",
for (i = 0; i < n; i++) {
seq_printf(m, " %d: offset=%d pitch=%d, obj: ",
i, fb->offsets[i], fb->pitches[i]);
- msm_gem_describe(fb->obj[i], m);
+ msm_gem_describe(fb->obj[i], m, &stats);
}
}
#endif
int ret;
if (fence > fctx->last_fence) {
- DRM_ERROR("%s: waiting on invalid fence: %u (of %u)\n",
+ DRM_ERROR_RATELIMITED("%s: waiting on invalid fence: %u (of %u)\n",
fctx->name, fence, fctx->last_fence);
return -EINVAL;
}
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
if (!msm_obj->pages) {
struct drm_device *dev = obj->dev;
*/
if (msm_obj->flags & (MSM_BO_WC|MSM_BO_UNCACHED))
sync_for_device(msm_obj);
+
+ GEM_WARN_ON(msm_obj->active_count);
+ update_inactive(msm_obj);
}
return msm_obj->pages;
sg_free_table(msm_obj->sgt);
kfree(msm_obj->sgt);
+ msm_obj->sgt = NULL;
}
if (use_pages(obj))
msm_gem_lock(obj);
- if (WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) {
+ if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) {
msm_gem_unlock(obj);
return ERR_PTR(-EBUSY);
}
goto out;
}
- if (WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) {
+ if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) {
msm_gem_unlock(obj);
return VM_FAULT_SIGBUS;
}
struct drm_device *dev = obj->dev;
int ret;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
/* Make it mmapable */
ret = drm_gem_create_mmap_offset(obj);
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct msm_gem_vma *vma;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
vma = kzalloc(sizeof(*vma), GFP_KERNEL);
if (!vma)
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct msm_gem_vma *vma;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
list_for_each_entry(vma, &msm_obj->vmas, list) {
if (vma->aspace == aspace)
kfree(vma);
}
-/* Called with msm_obj locked */
+/**
+ * If close is true, this also closes the VMA (releasing the allocated
+ * iova range) in addition to removing the iommu mapping. In the eviction
+ * case (!close), we keep the iova allocated, but only remove the iommu
+ * mapping.
+ */
static void
-put_iova_spaces(struct drm_gem_object *obj)
+put_iova_spaces(struct drm_gem_object *obj, bool close)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct msm_gem_vma *vma;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
list_for_each_entry(vma, &msm_obj->vmas, list) {
if (vma->aspace) {
msm_gem_purge_vma(vma->aspace, vma);
- msm_gem_close_vma(vma->aspace, vma);
+ if (close)
+ msm_gem_close_vma(vma->aspace, vma);
}
}
}
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct msm_gem_vma *vma, *tmp;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
list_for_each_entry_safe(vma, tmp, &msm_obj->vmas, list) {
del_vma(vma);
struct msm_gem_vma *vma;
int ret = 0;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
vma = lookup_vma(obj, aspace);
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct msm_gem_vma *vma;
struct page **pages;
- int prot = IOMMU_READ;
+ int ret, prot = IOMMU_READ;
if (!(msm_obj->flags & MSM_BO_GPU_READONLY))
prot |= IOMMU_WRITE;
if (msm_obj->flags & MSM_BO_MAP_PRIV)
prot |= IOMMU_PRIV;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
- if (WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED))
+ if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED))
return -EBUSY;
vma = lookup_vma(obj, aspace);
- if (WARN_ON(!vma))
+ if (GEM_WARN_ON(!vma))
return -EINVAL;
pages = get_pages(obj);
if (IS_ERR(pages))
return PTR_ERR(pages);
- return msm_gem_map_vma(aspace, vma, prot,
+ ret = msm_gem_map_vma(aspace, vma, prot,
msm_obj->sgt, obj->size >> PAGE_SHIFT);
+
+ if (!ret)
+ msm_obj->pin_count++;
+
+ return ret;
}
static int get_and_pin_iova_range_locked(struct drm_gem_object *obj,
u64 local;
int ret;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
ret = get_iova_locked(obj, aspace, &local,
range_start, range_end);
msm_gem_lock(obj);
vma = lookup_vma(obj, aspace);
msm_gem_unlock(obj);
- WARN_ON(!vma);
+ GEM_WARN_ON(!vma);
return vma ? vma->iova : 0;
}
void msm_gem_unpin_iova_locked(struct drm_gem_object *obj,
struct msm_gem_address_space *aspace)
{
+ struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct msm_gem_vma *vma;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
vma = lookup_vma(obj, aspace);
- if (!WARN_ON(!vma))
+ if (!GEM_WARN_ON(!vma)) {
msm_gem_unmap_vma(aspace, vma);
+
+ msm_obj->pin_count--;
+ GEM_WARN_ON(msm_obj->pin_count < 0);
+
+ update_inactive(msm_obj);
+ }
}
/*
struct msm_gem_object *msm_obj = to_msm_bo(obj);
int ret = 0;
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
if (obj->import_attach)
return ERR_PTR(-ENODEV);
- if (WARN_ON(msm_obj->madv > madv)) {
+ if (GEM_WARN_ON(msm_obj->madv > madv)) {
DRM_DEV_ERROR(obj->dev->dev, "Invalid madv state: %u vs %u\n",
msm_obj->madv, madv);
return ERR_PTR(-EBUSY);
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
- WARN_ON(!msm_gem_is_locked(obj));
- WARN_ON(msm_obj->vmap_count < 1);
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(msm_obj->vmap_count < 1);
msm_obj->vmap_count--;
}
struct drm_device *dev = obj->dev;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
- WARN_ON(!is_purgeable(msm_obj));
- WARN_ON(obj->import_attach);
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!is_purgeable(msm_obj));
- put_iova_spaces(obj);
+ /* Get rid of any iommu mapping(s): */
+ put_iova_spaces(obj, true);
msm_gem_vunmap(obj);
+ drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
+
put_pages(obj);
put_iova_vmas(obj);
msm_obj->madv = __MSM_MADV_PURGED;
+ update_inactive(msm_obj);
- drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
drm_gem_free_mmap_offset(obj);
/* Our goal here is to return as much of the memory as
0, (loff_t)-1);
}
+/**
+ * Unpin the backing pages and make them available to be swapped out.
+ */
+void msm_gem_evict(struct drm_gem_object *obj)
+{
+ struct drm_device *dev = obj->dev;
+ struct msm_gem_object *msm_obj = to_msm_bo(obj);
+
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(is_unevictable(msm_obj));
+ GEM_WARN_ON(!msm_obj->evictable);
+ GEM_WARN_ON(msm_obj->active_count);
+
+ /* Get rid of any iommu mapping(s): */
+ put_iova_spaces(obj, false);
+
+ drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
+
+ put_pages(obj);
+
+ update_inactive(msm_obj);
+}
+
void msm_gem_vunmap(struct drm_gem_object *obj)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
- if (!msm_obj->vaddr || WARN_ON(!is_vunmapable(msm_obj)))
+ if (!msm_obj->vaddr || GEM_WARN_ON(!is_vunmapable(msm_obj)))
return;
vunmap(msm_obj->vaddr);
struct msm_drm_private *priv = obj->dev->dev_private;
might_sleep();
- WARN_ON(!msm_gem_is_locked(obj));
- WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED);
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED);
+ GEM_WARN_ON(msm_obj->dontneed);
+ GEM_WARN_ON(!msm_obj->sgt);
if (msm_obj->active_count++ == 0) {
mutex_lock(&priv->mm_lock);
- list_del_init(&msm_obj->mm_list);
+ if (msm_obj->evictable)
+ mark_unevictable(msm_obj);
+ list_del(&msm_obj->mm_list);
list_add_tail(&msm_obj->mm_list, &gpu->active_list);
mutex_unlock(&priv->mm_lock);
}
struct msm_gem_object *msm_obj = to_msm_bo(obj);
might_sleep();
- WARN_ON(!msm_gem_is_locked(obj));
+ GEM_WARN_ON(!msm_gem_is_locked(obj));
if (--msm_obj->active_count == 0) {
update_inactive(msm_obj);
{
struct msm_drm_private *priv = msm_obj->base.dev->dev_private;
+ GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base));
+
+ if (msm_obj->active_count != 0)
+ return;
+
mutex_lock(&priv->mm_lock);
- WARN_ON(msm_obj->active_count != 0);
- list_del_init(&msm_obj->mm_list);
- if (msm_obj->madv == MSM_MADV_WILLNEED)
+ if (msm_obj->dontneed)
+ mark_unpurgeable(msm_obj);
+ if (msm_obj->evictable)
+ mark_unevictable(msm_obj);
+
+ list_del(&msm_obj->mm_list);
+ if ((msm_obj->madv == MSM_MADV_WILLNEED) && msm_obj->sgt) {
list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed);
- else
+ mark_evictable(msm_obj);
+ } else if (msm_obj->madv == MSM_MADV_DONTNEED) {
list_add_tail(&msm_obj->mm_list, &priv->inactive_dontneed);
+ mark_purgeable(msm_obj);
+ } else {
+ GEM_WARN_ON((msm_obj->madv != __MSM_MADV_PURGED) && msm_obj->sgt);
+ list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned);
+ }
mutex_unlock(&priv->mm_lock);
}
fence->seqno);
}
-void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m)
+void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m,
+ struct msm_gem_stats *stats)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct dma_resv *robj = obj->resv;
msm_gem_lock(obj);
+ stats->all.count++;
+ stats->all.size += obj->size;
+
+ if (is_active(msm_obj)) {
+ stats->active.count++;
+ stats->active.size += obj->size;
+ }
+
+ if (msm_obj->pages) {
+ stats->resident.count++;
+ stats->resident.size += obj->size;
+ }
+
switch (msm_obj->madv) {
case __MSM_MADV_PURGED:
+ stats->purged.count++;
+ stats->purged.size += obj->size;
madv = " purged";
break;
case MSM_MADV_DONTNEED:
+ stats->purgeable.count++;
+ stats->purgeable.size += obj->size;
madv = " purgeable";
break;
case MSM_MADV_WILLNEED:
void msm_gem_describe_objects(struct list_head *list, struct seq_file *m)
{
+ struct msm_gem_stats stats = {};
struct msm_gem_object *msm_obj;
- int count = 0;
- size_t size = 0;
seq_puts(m, " flags id ref offset kaddr size madv name\n");
- list_for_each_entry(msm_obj, list, mm_list) {
+ list_for_each_entry(msm_obj, list, node) {
struct drm_gem_object *obj = &msm_obj->base;
seq_puts(m, " ");
- msm_gem_describe(obj, m);
- count++;
- size += obj->size;
+ msm_gem_describe(obj, m, &stats);
}
- seq_printf(m, "Total %d objects, %zu bytes\n", count, size);
+ seq_printf(m, "Total: %4d objects, %9zu bytes\n",
+ stats.all.count, stats.all.size);
+ seq_printf(m, "Active: %4d objects, %9zu bytes\n",
+ stats.active.count, stats.active.size);
+ seq_printf(m, "Resident: %4d objects, %9zu bytes\n",
+ stats.resident.count, stats.resident.size);
+ seq_printf(m, "Purgeable: %4d objects, %9zu bytes\n",
+ stats.purgeable.count, stats.purgeable.size);
+ seq_printf(m, "Purged: %4d objects, %9zu bytes\n",
+ stats.purged.count, stats.purged.size);
}
#endif
struct drm_device *dev = obj->dev;
struct msm_drm_private *priv = dev->dev_private;
+ mutex_lock(&priv->obj_lock);
+ list_del(&msm_obj->node);
+ mutex_unlock(&priv->obj_lock);
+
mutex_lock(&priv->mm_lock);
+ if (msm_obj->dontneed)
+ mark_unpurgeable(msm_obj);
list_del(&msm_obj->mm_list);
mutex_unlock(&priv->mm_lock);
msm_gem_lock(obj);
/* object should not be on active list: */
- WARN_ON(is_active(msm_obj));
+ GEM_WARN_ON(is_active(msm_obj));
- put_iova_spaces(obj);
+ put_iova_spaces(obj, true);
if (obj->import_attach) {
- WARN_ON(msm_obj->vaddr);
+ GEM_WARN_ON(msm_obj->vaddr);
/* Don't drop the pages for imported dmabuf, as they are not
* ours, just free the array we allocated:
else if ((flags & (MSM_BO_STOLEN | MSM_BO_SCANOUT)) && priv->vram.size)
use_vram = true;
- if (WARN_ON(use_vram && !priv->vram.size))
+ if (GEM_WARN_ON(use_vram && !priv->vram.size))
return ERR_PTR(-EINVAL);
/* Disallow zero sized objects as they make the underlying
}
mutex_lock(&priv->mm_lock);
- /* Initially obj is idle, obj->madv == WILLNEED: */
- list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed);
+ list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned);
mutex_unlock(&priv->mm_lock);
+ mutex_lock(&priv->obj_lock);
+ list_add_tail(&msm_obj->node, &priv->objects);
+ mutex_unlock(&priv->obj_lock);
+
return obj;
fail:
msm_gem_unlock(obj);
mutex_lock(&priv->mm_lock);
- list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed);
+ list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned);
mutex_unlock(&priv->mm_lock);
+ mutex_lock(&priv->obj_lock);
+ list_add_tail(&msm_obj->node, &priv->objects);
+ mutex_unlock(&priv->obj_lock);
+
return obj;
fail:
#include <linux/dma-resv.h>
#include "msm_drv.h"
+/* Make all GEM related WARN_ON()s ratelimited.. when things go wrong they
+ * tend to go wrong 1000s of times in a short timespan.
+ */
+#define GEM_WARN_ON(x) WARN_RATELIMIT(x, "%s", __stringify(x))
+
/* Additional internal-use only BO flags: */
#define MSM_BO_STOLEN 0x10000000 /* try to use stolen/splash memory */
#define MSM_BO_MAP_PRIV 0x20000000 /* use IOMMU_PRIV when mapping */
*/
uint8_t madv;
+ /**
+ * Is object on inactive_dontneed list (ie. counted in priv->shrinkable_count)?
+ */
+ bool dontneed : 1;
+
+ /**
+ * Is object evictable (ie. counted in priv->evictable_count)?
+ */
+ bool evictable : 1;
+
/**
* count of active vmap'ing
*/
uint8_t vmap_count;
- /* And object is either:
- * inactive - on priv->inactive_list
+ /**
+ * Node in list of all objects (mainly for debugfs, protected by
+ * priv->obj_lock
+ */
+ struct list_head node;
+
+ /**
+ * An object is either:
+ * inactive - on priv->inactive_dontneed or priv->inactive_willneed
+ * (depending on purgeability status)
* active - on one one of the gpu's active_list.. well, at
* least for now we don't have (I don't think) hw sync between
* 2d and 3d one devices which have both, meaning we need to
* block on submit if a bo is already on other ring
- *
*/
struct list_head mm_list;
struct list_head vmas; /* list of msm_gem_vma */
- struct llist_node freed;
-
/* For physically contiguous buffers. Used when we don't have
* an IOMMU. Also used for stolen/splashscreen buffer.
*/
char name[32]; /* Identifier to print for the debugfs files */
int active_count;
+ int pin_count;
};
#define to_msm_bo(x) container_of(x, struct msm_gem_object, base)
struct dma_buf *dmabuf, struct sg_table *sgt);
__printf(2, 3)
void msm_gem_object_set_name(struct drm_gem_object *bo, const char *fmt, ...);
+
#ifdef CONFIG_DEBUG_FS
-void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m);
+struct msm_gem_stats {
+ struct {
+ unsigned count;
+ size_t size;
+ } all, active, resident, purgeable, purged;
+};
+
+void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m,
+ struct msm_gem_stats *stats);
void msm_gem_describe_objects(struct list_head *list, struct seq_file *m);
#endif
static inline bool is_active(struct msm_gem_object *msm_obj)
{
- WARN_ON(!msm_gem_is_locked(&msm_obj->base));
+ GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base));
return msm_obj->active_count;
}
+/* imported/exported objects are not purgeable: */
+static inline bool is_unpurgeable(struct msm_gem_object *msm_obj)
+{
+ return msm_obj->base.dma_buf && msm_obj->base.import_attach;
+}
+
static inline bool is_purgeable(struct msm_gem_object *msm_obj)
{
return (msm_obj->madv == MSM_MADV_DONTNEED) && msm_obj->sgt &&
- !msm_obj->base.dma_buf && !msm_obj->base.import_attach;
+ !is_unpurgeable(msm_obj);
}
static inline bool is_vunmapable(struct msm_gem_object *msm_obj)
{
- WARN_ON(!msm_gem_is_locked(&msm_obj->base));
+ GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base));
return (msm_obj->vmap_count == 0) && msm_obj->vaddr;
}
+static inline void mark_purgeable(struct msm_gem_object *msm_obj)
+{
+ struct msm_drm_private *priv = msm_obj->base.dev->dev_private;
+
+ GEM_WARN_ON(!mutex_is_locked(&priv->mm_lock));
+
+ if (is_unpurgeable(msm_obj))
+ return;
+
+ if (GEM_WARN_ON(msm_obj->dontneed))
+ return;
+
+ priv->shrinkable_count += msm_obj->base.size >> PAGE_SHIFT;
+ msm_obj->dontneed = true;
+}
+
+static inline void mark_unpurgeable(struct msm_gem_object *msm_obj)
+{
+ struct msm_drm_private *priv = msm_obj->base.dev->dev_private;
+
+ GEM_WARN_ON(!mutex_is_locked(&priv->mm_lock));
+
+ if (is_unpurgeable(msm_obj))
+ return;
+
+ if (GEM_WARN_ON(!msm_obj->dontneed))
+ return;
+
+ priv->shrinkable_count -= msm_obj->base.size >> PAGE_SHIFT;
+ GEM_WARN_ON(priv->shrinkable_count < 0);
+ msm_obj->dontneed = false;
+}
+
+static inline bool is_unevictable(struct msm_gem_object *msm_obj)
+{
+ return is_unpurgeable(msm_obj) || msm_obj->pin_count || msm_obj->vaddr;
+}
+
+static inline void mark_evictable(struct msm_gem_object *msm_obj)
+{
+ struct msm_drm_private *priv = msm_obj->base.dev->dev_private;
+
+ WARN_ON(!mutex_is_locked(&priv->mm_lock));
+
+ if (is_unevictable(msm_obj))
+ return;
+
+ if (WARN_ON(msm_obj->evictable))
+ return;
+
+ priv->evictable_count += msm_obj->base.size >> PAGE_SHIFT;
+ msm_obj->evictable = true;
+}
+
+static inline void mark_unevictable(struct msm_gem_object *msm_obj)
+{
+ struct msm_drm_private *priv = msm_obj->base.dev->dev_private;
+
+ WARN_ON(!mutex_is_locked(&priv->mm_lock));
+
+ if (is_unevictable(msm_obj))
+ return;
+
+ if (WARN_ON(!msm_obj->evictable))
+ return;
+
+ priv->evictable_count -= msm_obj->base.size >> PAGE_SHIFT;
+ WARN_ON(priv->evictable_count < 0);
+ msm_obj->evictable = false;
+}
+
void msm_gem_purge(struct drm_gem_object *obj);
+void msm_gem_evict(struct drm_gem_object *obj);
void msm_gem_vunmap(struct drm_gem_object *obj);
/* Created per submit-ioctl, to track bo's and cmdstream bufs, etc,
#include "msm_gpu.h"
#include "msm_gpu_trace.h"
+/* Default disabled for now until it has some more testing on the different
+ * iommu combinations that can be paired with the driver:
+ */
+bool enable_eviction = false;
+MODULE_PARM_DESC(enable_eviction, "Enable swappable GEM buffers");
+module_param(enable_eviction, bool, 0600);
+
+static bool can_swap(void)
+{
+ return enable_eviction && get_nr_swap_pages() > 0;
+}
+
static unsigned long
msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
{
struct msm_drm_private *priv =
container_of(shrinker, struct msm_drm_private, shrinker);
- struct msm_gem_object *msm_obj;
- unsigned long count = 0;
+ unsigned count = priv->shrinkable_count;
- mutex_lock(&priv->mm_lock);
+ if (can_swap())
+ count += priv->evictable_count;
- list_for_each_entry(msm_obj, &priv->inactive_dontneed, mm_list) {
- if (!msm_gem_trylock(&msm_obj->base))
- continue;
- if (is_purgeable(msm_obj))
- count += msm_obj->base.size >> PAGE_SHIFT;
- msm_gem_unlock(&msm_obj->base);
- }
+ return count;
+}
- mutex_unlock(&priv->mm_lock);
+static bool
+purge(struct msm_gem_object *msm_obj)
+{
+ if (!is_purgeable(msm_obj))
+ return false;
- return count;
+ /*
+ * This will move the obj out of still_in_list to
+ * the purged list
+ */
+ msm_gem_purge(&msm_obj->base);
+
+ return true;
+}
+
+static bool
+evict(struct msm_gem_object *msm_obj)
+{
+ if (is_unevictable(msm_obj))
+ return false;
+
+ msm_gem_evict(&msm_obj->base);
+
+ return true;
}
static unsigned long
-msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
+scan(struct msm_drm_private *priv, unsigned nr_to_scan, struct list_head *list,
+ bool (*shrink)(struct msm_gem_object *msm_obj))
{
- struct msm_drm_private *priv =
- container_of(shrinker, struct msm_drm_private, shrinker);
- struct msm_gem_object *msm_obj;
- unsigned long freed = 0;
+ unsigned freed = 0;
+ struct list_head still_in_list;
+
+ INIT_LIST_HEAD(&still_in_list);
mutex_lock(&priv->mm_lock);
- list_for_each_entry(msm_obj, &priv->inactive_dontneed, mm_list) {
- if (freed >= sc->nr_to_scan)
+ while (freed < nr_to_scan) {
+ struct msm_gem_object *msm_obj = list_first_entry_or_null(
+ list, typeof(*msm_obj), mm_list);
+
+ if (!msm_obj)
break;
- if (!msm_gem_trylock(&msm_obj->base))
+
+ list_move_tail(&msm_obj->mm_list, &still_in_list);
+
+ /*
+ * If it is in the process of being freed, msm_gem_free_object
+ * can be blocked on mm_lock waiting to remove it. So just
+ * skip it.
+ */
+ if (!kref_get_unless_zero(&msm_obj->base.refcount))
continue;
- if (is_purgeable(msm_obj)) {
- msm_gem_purge(&msm_obj->base);
+
+ /*
+ * Now that we own a reference, we can drop mm_lock for the
+ * rest of the loop body, to reduce contention with the
+ * retire_submit path (which could make more objects purgeable)
+ */
+
+ mutex_unlock(&priv->mm_lock);
+
+ /*
+ * Note that this still needs to be trylock, since we can
+ * hit shrinker in response to trying to get backing pages
+ * for this obj (ie. while it's lock is already held)
+ */
+ if (!msm_gem_trylock(&msm_obj->base))
+ goto tail;
+
+ if (shrink(msm_obj))
freed += msm_obj->base.size >> PAGE_SHIFT;
- }
+
msm_gem_unlock(&msm_obj->base);
+
+tail:
+ drm_gem_object_put(&msm_obj->base);
+ mutex_lock(&priv->mm_lock);
}
+ list_splice_tail(&still_in_list, list);
mutex_unlock(&priv->mm_lock);
+ return freed;
+}
+
+static unsigned long
+msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
+{
+ struct msm_drm_private *priv =
+ container_of(shrinker, struct msm_drm_private, shrinker);
+ unsigned long freed;
+
+ freed = scan(priv, sc->nr_to_scan, &priv->inactive_dontneed, purge);
+
if (freed > 0)
trace_msm_gem_purge(freed << PAGE_SHIFT);
- return freed;
+ if (can_swap() && freed < sc->nr_to_scan) {
+ int evicted = scan(priv, sc->nr_to_scan - freed,
+ &priv->inactive_willneed, evict);
+
+ if (evicted > 0)
+ trace_msm_gem_evict(evicted << PAGE_SHIFT);
+
+ freed += evicted;
+ }
+
+ return (freed > 0) ? freed : SHRINK_STOP;
}
/* since we don't know any better, lets bail after a few
*/
static const int vmap_shrink_limit = 15;
-static unsigned
-vmap_shrink(struct list_head *mm_list)
+static bool
+vmap_shrink(struct msm_gem_object *msm_obj)
{
- struct msm_gem_object *msm_obj;
- unsigned unmapped = 0;
+ if (!is_vunmapable(msm_obj))
+ return false;
- list_for_each_entry(msm_obj, mm_list, mm_list) {
- if (!msm_gem_trylock(&msm_obj->base))
- continue;
- if (is_vunmapable(msm_obj)) {
- msm_gem_vunmap(&msm_obj->base);
- unmapped++;
- }
- msm_gem_unlock(&msm_obj->base);
+ msm_gem_vunmap(&msm_obj->base);
- if (++unmapped >= vmap_shrink_limit)
- break;
- }
-
- return unmapped;
+ return true;
}
static int
};
unsigned idx, unmapped = 0;
- mutex_lock(&priv->mm_lock);
-
- for (idx = 0; mm_lists[idx]; idx++) {
- unmapped += vmap_shrink(mm_lists[idx]);
-
- if (unmapped >= vmap_shrink_limit)
- break;
+ for (idx = 0; mm_lists[idx] && unmapped < vmap_shrink_limit; idx++) {
+ unmapped += scan(priv, vmap_shrink_limit - unmapped,
+ mm_lists[idx], vmap_shrink);
}
- mutex_unlock(&priv->mm_lock);
-
*(unsigned long *)ptr += unmapped;
if (unmapped > 0)
if (ret)
return ret;
+ gpu->suspend_count++;
+
return 0;
}
ktime_t time;
} devfreq;
+ uint32_t suspend_count;
+
struct msm_gpu_state *crashstate;
/* True if the hardware supports expanded apriv (a650 and newer) */
bool hw_apriv;
);
+TRACE_EVENT(msm_gem_evict,
+ TP_PROTO(u32 bytes),
+ TP_ARGS(bytes),
+ TP_STRUCT__entry(
+ __field(u32, bytes)
+ ),
+ TP_fast_assign(
+ __entry->bytes = bytes;
+ ),
+ TP_printk("Evicting %u bytes", __entry->bytes)
+);
+
+
TRACE_EVENT(msm_gem_purge_vmaps,
TP_PROTO(u32 unmapped),
TP_ARGS(unmapped),
* from the crtc's pending_timer close to end of the frame:
*/
struct mutex commit_lock[MAX_CRTCS];
- struct lock_class_key commit_lock_keys[MAX_CRTCS];
unsigned pending_crtc_mask;
struct msm_pending_timer pending_timers[MAX_CRTCS];
};
{
unsigned i, ret;
- for (i = 0; i < ARRAY_SIZE(kms->commit_lock); i++) {
- lockdep_register_key(&kms->commit_lock_keys[i]);
- __mutex_init(&kms->commit_lock[i], "&kms->commit_lock[i]",
- &kms->commit_lock_keys[i]);
- }
+ for (i = 0; i < ARRAY_SIZE(kms->commit_lock); i++)
+ mutex_init(&kms->commit_lock[i]);
kms->funcs = funcs;
else
nouveau_display(dev)->format_modifiers = disp50xx_modifiers;
- if (disp->disp->object.oclass >= GK104_DISP) {
+ /* FIXME: 256x256 cursors are supported on Kepler, however unlike Maxwell and later
+ * generations Kepler requires that we use small pages (4K) for cursor scanout surfaces. The
+ * proper fix for this is to teach nouveau to migrate fbs being used for the cursor plane to
+ * small page allocations in prepare_fb(). When this is implemented, we should also force
+ * large pages (128K) for ovly fbs in order to fix Kepler ovlys.
+ * But until then, just limit cursors to 128x128 - which is small enough to avoid ever using
+ * large pages.
+ */
+ if (disp->disp->object.oclass >= GM107_DISP) {
dev->mode_config.cursor_width = 256;
dev->mode_config.cursor_height = 256;
+ } else if (disp->disp->object.oclass >= GK104_DISP) {
+ dev->mode_config.cursor_width = 128;
+ dev->mode_config.cursor_height = 128;
} else {
dev->mode_config.cursor_width = 64;
dev->mode_config.cursor_height = 64;
if (!ttm_dma)
return;
+ if (!ttm_dma->pages) {
+ NV_DEBUG(drm, "ttm_dma 0x%p: pages NULL\n", ttm_dma);
+ return;
+ }
/* Don't waste time looping if the object is coherent */
if (nvbo->force_coherent)
if (!ttm_dma)
return;
+ if (!ttm_dma->pages) {
+ NV_DEBUG(drm, "ttm_dma 0x%p: pages NULL\n", ttm_dma);
+ return;
+ }
/* Don't waste time looping if the object is coherent */
if (nvbo->force_coherent)
const struct mipi_dsi_msg *msg)
{
struct mipi_dsi_packet pkt;
- int err;
+ int ret;
u32 r;
- err = mipi_dsi_create_packet(&pkt, msg);
- if (err)
- return err;
+ ret = mipi_dsi_create_packet(&pkt, msg);
+ if (ret < 0)
+ return ret;
WARN_ON(!dsi_bus_is_locked(dsi));
u32 height_mm;
u32 max_hs_rate;
u32 max_lp_rate;
+ bool te_support;
};
struct panel_drv_data {
if (r)
goto err;
- r = mipi_dsi_dcs_set_tear_on(ddata->dsi, MIPI_DSI_DCS_TEAR_MODE_VBLANK);
- if (r)
- goto err;
+ if (ddata->panel_data->te_support) {
+ r = mipi_dsi_dcs_set_tear_on(ddata->dsi, MIPI_DSI_DCS_TEAR_MODE_VBLANK);
+ if (r)
+ goto err;
+ }
/* possible panel bug */
msleep(100);
.height_mm = 0,
.max_hs_rate = 300000000,
.max_lp_rate = 10000000,
+ .te_support = true,
};
static const struct dsic_panel_data himalaya_data = {
.height_mm = 88,
.max_hs_rate = 300000000,
.max_lp_rate = 10000000,
+ .te_support = false,
};
static const struct dsic_panel_data droid4_data = {
.height_mm = 89,
.max_hs_rate = 300000000,
.max_lp_rate = 10000000,
+ .te_support = false,
};
static const struct of_device_id dsicm_of_match[] = {
{
struct td043mtea1_panel *lcd = dev_get_drvdata(dev);
- return snprintf(buf, PAGE_SIZE, "%d\n", lcd->vmirror);
+ return sysfs_emit(buf, "%d\n", lcd->vmirror);
}
static ssize_t vmirror_store(struct device *dev, struct device_attribute *attr,
{
struct td043mtea1_panel *lcd = dev_get_drvdata(dev);
- return snprintf(buf, PAGE_SIZE, "%d\n", lcd->mode);
+ return sysfs_emit(buf, "%d\n", lcd->mode);
}
static ssize_t mode_store(struct device *dev, struct device_attribute *attr,
pr_err("r600_cp: Bogus length %zu in firmware \"%s\"\n",
rdev->me_fw->size, fw_name);
err = -EINVAL;
+ goto out;
}
snprintf(fw_name, sizeof(fw_name), "radeon/%s_rlc.bin", rlc_chip_name);
pr_err("r600_rlc: Bogus length %zu in firmware \"%s\"\n",
rdev->rlc_fw->size, fw_name);
err = -EINVAL;
+ goto out;
}
if ((rdev->family >= CHIP_RV770) && (rdev->family <= CHIP_HEMLOCK)) {
struct r600_cs_track {
- /* configuration we miror so that we use same code btw kms/ums */
+ /* configuration we mirror so that we use same code btw kms/ums */
u32 group_size;
u32 nbanks;
u32 npipes;
*
* This function will test against r600_reg_safe_bm and return 0
* if register is safe. If register is not flag as safe this function
- * will test it against a list of register needind special handling.
+ * will test it against a list of register needing special handling.
*/
static int r600_cs_check_reg(struct radeon_cs_parser *p, u32 reg, u32 idx)
{
/**
* r600_dma_cs_next_reloc() - parse next reloc
* @p: parser structure holding parsing context.
- * @cs_reloc: reloc informations
+ * @cs_reloc: reloc information
*
* Return the next reloc, do bo validation and compute
* GPU offset using the provided start.
to_radeon_connector(connector);
struct radeon_connector *master = radeon_connector->mst_port;
+ if (drm_connector_is_unregistered(connector))
+ return connector_status_disconnected;
+
return drm_dp_mst_detect_port(connector, ctx, &master->mst_mgr,
radeon_connector->port);
}
}
#endif
man = ttm_manager_type(bdev, TTM_PL_VRAM);
+ if (!man)
+ return 0;
return ttm_resource_manager_evict_all(bdev, man);
}
struct radeon_device *rdev = ddev->dev_private;
int cp = rdev->pm.profile;
- return snprintf(buf, PAGE_SIZE, "%s\n",
- (cp == PM_PROFILE_AUTO) ? "auto" :
- (cp == PM_PROFILE_LOW) ? "low" :
- (cp == PM_PROFILE_MID) ? "mid" :
- (cp == PM_PROFILE_HIGH) ? "high" : "default");
+ return sysfs_emit(buf, "%s\n", (cp == PM_PROFILE_AUTO) ? "auto" :
+ (cp == PM_PROFILE_LOW) ? "low" :
+ (cp == PM_PROFILE_MID) ? "mid" :
+ (cp == PM_PROFILE_HIGH) ? "high" : "default");
}
static ssize_t radeon_set_pm_profile(struct device *dev,
struct radeon_device *rdev = ddev->dev_private;
int pm = rdev->pm.pm_method;
- return snprintf(buf, PAGE_SIZE, "%s\n",
- (pm == PM_METHOD_DYNPM) ? "dynpm" :
- (pm == PM_METHOD_PROFILE) ? "profile" : "dpm");
+ return sysfs_emit(buf, "%s\n", (pm == PM_METHOD_DYNPM) ? "dynpm" :
+ (pm == PM_METHOD_PROFILE) ? "profile" : "dpm");
}
static ssize_t radeon_set_pm_method(struct device *dev,
struct radeon_device *rdev = ddev->dev_private;
enum radeon_pm_state_type pm = rdev->pm.dpm.user_state;
- return snprintf(buf, PAGE_SIZE, "%s\n",
- (pm == POWER_STATE_TYPE_BATTERY) ? "battery" :
- (pm == POWER_STATE_TYPE_BALANCED) ? "balanced" : "performance");
+ return sysfs_emit(buf, "%s\n",
+ (pm == POWER_STATE_TYPE_BATTERY) ? "battery" :
+ (pm == POWER_STATE_TYPE_BALANCED) ? "balanced" : "performance");
}
static ssize_t radeon_set_dpm_state(struct device *dev,
if ((rdev->flags & RADEON_IS_PX) &&
(ddev->switch_power_state != DRM_SWITCH_POWER_ON))
- return snprintf(buf, PAGE_SIZE, "off\n");
+ return sysfs_emit(buf, "off\n");
- return snprintf(buf, PAGE_SIZE, "%s\n",
- (level == RADEON_DPM_FORCED_LEVEL_AUTO) ? "auto" :
- (level == RADEON_DPM_FORCED_LEVEL_LOW) ? "low" : "high");
+ return sysfs_emit(buf, "%s\n",
+ (level == RADEON_DPM_FORCED_LEVEL_AUTO) ? "auto" :
+ (level == RADEON_DPM_FORCED_LEVEL_LOW) ? "low" : "high");
}
static ssize_t radeon_set_dpm_forced_performance_level(struct device *dev,
else
temp = 0;
- return snprintf(buf, PAGE_SIZE, "%d\n", temp);
+ return sysfs_emit(buf, "%d\n", temp);
}
static ssize_t radeon_hwmon_show_temp_thresh(struct device *dev,
else
temp = rdev->pm.dpm.thermal.max_temp;
- return snprintf(buf, PAGE_SIZE, "%d\n", temp);
+ return sysfs_emit(buf, "%d\n", temp);
}
static SENSOR_DEVICE_ATTR(temp1_input, S_IRUGO, radeon_hwmon_show_temp, NULL, 0);
for hwmon */
sclk *= 10000;
- return snprintf(buf, PAGE_SIZE, "%u\n", sclk);
+ return sysfs_emit(buf, "%u\n", sclk);
}
static SENSOR_DEVICE_ATTR(freq1_input, S_IRUGO, radeon_hwmon_show_sclk, NULL,
if (rdev->asic->dpm.get_current_vddc)
vddc = rdev->asic->dpm.get_current_vddc(rdev);
- return snprintf(buf, PAGE_SIZE, "%u\n", vddc);
+ return sysfs_emit(buf, "%u\n", vddc);
}
static SENSOR_DEVICE_ATTR(in0_input, S_IRUGO, radeon_hwmon_show_vddc, NULL,
if (gtt->userflags & RADEON_GEM_USERPTR_ANONONLY) {
/* check that we only pin down anonymous memory
to prevent problems with writeback */
- unsigned long end = gtt->userptr + ttm->num_pages * PAGE_SIZE;
+ unsigned long end = gtt->userptr + (u64)ttm->num_pages * PAGE_SIZE;
struct vm_area_struct *vma;
vma = find_vma(gtt->usermm, gtt->userptr);
if (!vma || vma->vm_file || vma->vm_end < end)
} while (pinned < ttm->num_pages);
r = sg_alloc_table_from_pages(ttm->sg, ttm->pages, ttm->num_pages, 0,
- ttm->num_pages << PAGE_SHIFT,
+ (u64)ttm->num_pages << PAGE_SHIFT,
GFP_KERNEL);
if (r)
goto release_sg;
DMA_BIDIRECTIONAL : DMA_TO_DEVICE;
/* double check that we don't free the table twice */
- if (!ttm->sg->sgl)
+ if (!ttm->sg || !ttm->sg->sgl)
return;
/* free the sg table and pages again */
struct radeon_ttm_tt *gtt = (void *)ttm;
struct radeon_device *rdev = radeon_get_rdev(bdev);
+ if (gtt->userptr)
+ radeon_ttm_tt_unpin_userptr(bdev, ttm);
+
if (!gtt->bound)
return;
radeon_gart_unbind(rdev, gtt->offset, ttm->num_pages);
- if (gtt->userptr)
- radeon_ttm_tt_unpin_userptr(bdev, ttm);
gtt->bound = false;
}
static const struct drm_encoder_funcs rcar_du_encoder_funcs = {
};
-static void rcar_du_encoder_release(struct drm_device *dev, void *res)
-{
- struct rcar_du_encoder *renc = res;
-
- drm_encoder_cleanup(&renc->base);
- kfree(renc);
-}
-
int rcar_du_encoder_init(struct rcar_du_device *rcdu,
enum rcar_du_output output,
struct device_node *enc_node)
{
struct rcar_du_encoder *renc;
struct drm_bridge *bridge;
- int ret;
/*
* Locate the DRM bridge from the DT node. For the DPAD outputs, if the
return -ENOLINK;
}
- renc = kzalloc(sizeof(*renc), GFP_KERNEL);
- if (renc == NULL)
- return -ENOMEM;
-
- renc->output = output;
-
dev_dbg(rcdu->dev, "initializing encoder %pOF for output %u\n",
enc_node, output);
- ret = drm_encoder_init(&rcdu->ddev, &renc->base, &rcar_du_encoder_funcs,
- DRM_MODE_ENCODER_NONE, NULL);
- if (ret < 0) {
- kfree(renc);
- return ret;
- }
+ renc = drmm_encoder_alloc(&rcdu->ddev, struct rcar_du_encoder, base,
+ &rcar_du_encoder_funcs, DRM_MODE_ENCODER_NONE,
+ NULL);
+ if (!renc)
+ return -ENOMEM;
- ret = drmm_add_action_or_reset(&rcdu->ddev, rcar_du_encoder_release,
- renc);
- if (ret)
- return ret;
+ renc->output = output;
/*
* Attach the bridge to the encoder. The bridge will create the
*/
void drm_sched_increase_karma(struct drm_sched_job *bad)
{
- int i;
- struct drm_sched_entity *tmp;
- struct drm_sched_entity *entity;
- struct drm_gpu_scheduler *sched = bad->sched;
-
- /* don't increase @bad's karma if it's from KERNEL RQ,
- * because sometimes GPU hang would cause kernel jobs (like VM updating jobs)
- * corrupt but keep in mind that kernel jobs always considered good.
- */
- if (bad->s_priority != DRM_SCHED_PRIORITY_KERNEL) {
- atomic_inc(&bad->karma);
- for (i = DRM_SCHED_PRIORITY_MIN; i < DRM_SCHED_PRIORITY_KERNEL;
- i++) {
- struct drm_sched_rq *rq = &sched->sched_rq[i];
-
- spin_lock(&rq->lock);
- list_for_each_entry_safe(entity, tmp, &rq->entities, list) {
- if (bad->s_fence->scheduled.context ==
- entity->fence_context) {
- if (atomic_read(&bad->karma) >
- bad->sched->hang_limit)
- if (entity->guilty)
- atomic_set(entity->guilty, 1);
- break;
- }
- }
- spin_unlock(&rq->lock);
- if (&entity->list != &rq->entities)
- break;
- }
- }
+ drm_sched_increase_karma_ext(bad, 1);
}
EXPORT_SYMBOL(drm_sched_increase_karma);
+void drm_sched_reset_karma(struct drm_sched_job *bad)
+{
+ drm_sched_increase_karma_ext(bad, 0);
+}
+EXPORT_SYMBOL(drm_sched_reset_karma);
+
/**
* drm_sched_stop - stop the scheduler
*
*
*/
void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched)
+{
+ drm_sched_resubmit_jobs_ext(sched, INT_MAX);
+}
+EXPORT_SYMBOL(drm_sched_resubmit_jobs);
+
+/**
+ * drm_sched_resubmit_jobs_ext - helper to relunch certain number of jobs from mirror ring list
+ *
+ * @sched: scheduler instance
+ * @max: job numbers to relaunch
+ *
+ */
+void drm_sched_resubmit_jobs_ext(struct drm_gpu_scheduler *sched, int max)
{
struct drm_sched_job *s_job, *tmp;
uint64_t guilty_context;
bool found_guilty = false;
struct dma_fence *fence;
+ int i = 0;
list_for_each_entry_safe(s_job, tmp, &sched->pending_list, list) {
struct drm_sched_fence *s_fence = s_job->s_fence;
+ if (i >= max)
+ break;
+
if (!found_guilty && atomic_read(&s_job->karma) > sched->hang_limit) {
found_guilty = true;
guilty_context = s_job->s_fence->scheduled.context;
dma_fence_put(s_job->s_fence->parent);
fence = sched->ops->run_job(s_job);
+ i++;
if (IS_ERR_OR_NULL(fence)) {
if (IS_ERR(fence))
}
}
}
-EXPORT_SYMBOL(drm_sched_resubmit_jobs);
+EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
/**
* drm_sched_job_init - init a scheduler job
sched->ready = false;
}
EXPORT_SYMBOL(drm_sched_fini);
+
+/**
+ * drm_sched_increase_karma_ext - Update sched_entity guilty flag
+ *
+ * @bad: The job guilty of time out
+ * @type: type for increase/reset karma
+ *
+ */
+void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type)
+{
+ int i;
+ struct drm_sched_entity *tmp;
+ struct drm_sched_entity *entity;
+ struct drm_gpu_scheduler *sched = bad->sched;
+
+ /* don't change @bad's karma if it's from KERNEL RQ,
+ * because sometimes GPU hang would cause kernel jobs (like VM updating jobs)
+ * corrupt but keep in mind that kernel jobs always considered good.
+ */
+ if (bad->s_priority != DRM_SCHED_PRIORITY_KERNEL) {
+ if (type == 0)
+ atomic_set(&bad->karma, 0);
+ else if (type == 1)
+ atomic_inc(&bad->karma);
+
+ for (i = DRM_SCHED_PRIORITY_MIN; i < DRM_SCHED_PRIORITY_KERNEL;
+ i++) {
+ struct drm_sched_rq *rq = &sched->sched_rq[i];
+
+ spin_lock(&rq->lock);
+ list_for_each_entry_safe(entity, tmp, &rq->entities, list) {
+ if (bad->s_fence->scheduled.context ==
+ entity->fence_context) {
+ if (entity->guilty)
+ atomic_set(entity->guilty, type);
+ break;
+ }
+ }
+ spin_unlock(&rq->lock);
+ if (&entity->list != &rq->entities)
+ break;
+ }
+ }
+}
+EXPORT_SYMBOL(drm_sched_increase_karma_ext);
return &plane->base;
}
-static const u32 tegra_cursor_plane_formats[] = {
+static const u32 tegra_legacy_cursor_plane_formats[] = {
DRM_FORMAT_RGBA8888,
};
+static const u32 tegra_cursor_plane_formats[] = {
+ DRM_FORMAT_ARGB8888,
+};
+
static int tegra_cursor_atomic_check(struct drm_plane *plane,
struct drm_atomic_state *state)
{
plane);
struct tegra_plane_state *tegra_plane_state = to_tegra_plane_state(new_state);
struct tegra_dc *dc = to_tegra_dc(new_state->crtc);
- u32 value = CURSOR_CLIP_DISPLAY;
+ struct tegra_drm *tegra = plane->dev->dev_private;
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ u64 dma_mask = *dc->dev->dma_mask;
+#endif
+ unsigned int x, y;
+ u32 value = 0;
/* rien ne va plus */
if (!new_state->crtc || !new_state->fb)
return;
+ /*
+ * Legacy display supports hardware clipping of the cursor, but
+ * nvdisplay relies on software to clip the cursor to the screen.
+ */
+ if (!dc->soc->has_nvdisplay)
+ value |= CURSOR_CLIP_DISPLAY;
+
switch (new_state->crtc_w) {
case 32:
value |= CURSOR_SIZE_32x32;
tegra_dc_writel(dc, value, DC_DISP_CURSOR_START_ADDR);
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- value = (tegra_plane_state->iova[0] >> 32) & 0x3;
+ value = (tegra_plane_state->iova[0] >> 32) & (dma_mask >> 32);
tegra_dc_writel(dc, value, DC_DISP_CURSOR_START_ADDR_HI);
#endif
value = tegra_dc_readl(dc, DC_DISP_BLEND_CURSOR_CONTROL);
value &= ~CURSOR_DST_BLEND_MASK;
value &= ~CURSOR_SRC_BLEND_MASK;
- value |= CURSOR_MODE_NORMAL;
+
+ if (dc->soc->has_nvdisplay)
+ value &= ~CURSOR_COMPOSITION_MODE_XOR;
+ else
+ value |= CURSOR_MODE_NORMAL;
+
value |= CURSOR_DST_BLEND_NEG_K1_TIMES_SRC;
value |= CURSOR_SRC_BLEND_K1_TIMES_SRC;
value |= CURSOR_ALPHA;
tegra_dc_writel(dc, value, DC_DISP_BLEND_CURSOR_CONTROL);
+ /* nvdisplay relies on software for clipping */
+ if (dc->soc->has_nvdisplay) {
+ struct drm_rect src;
+
+ x = new_state->dst.x1;
+ y = new_state->dst.y1;
+
+ drm_rect_fp_to_int(&src, &new_state->src);
+
+ value = (src.y1 & tegra->vmask) << 16 | (src.x1 & tegra->hmask);
+ tegra_dc_writel(dc, value, DC_DISP_PCALC_HEAD_SET_CROPPED_POINT_IN_CURSOR);
+
+ value = (drm_rect_height(&src) & tegra->vmask) << 16 |
+ (drm_rect_width(&src) & tegra->hmask);
+ tegra_dc_writel(dc, value, DC_DISP_PCALC_HEAD_SET_CROPPED_SIZE_IN_CURSOR);
+ } else {
+ x = new_state->crtc_x;
+ y = new_state->crtc_y;
+ }
+
/* position the cursor */
- value = (new_state->crtc_y & 0x3fff) << 16 |
- (new_state->crtc_x & 0x3fff);
+ value = ((y & tegra->vmask) << 16) | (x & tegra->hmask);
tegra_dc_writel(dc, value, DC_DISP_CURSOR_POSITION);
}
plane->index = 6;
plane->dc = dc;
- num_formats = ARRAY_SIZE(tegra_cursor_plane_formats);
- formats = tegra_cursor_plane_formats;
+ if (!dc->soc->has_nvdisplay) {
+ num_formats = ARRAY_SIZE(tegra_legacy_cursor_plane_formats);
+ formats = tegra_legacy_cursor_plane_formats;
+ } else {
+ num_formats = ARRAY_SIZE(tegra_cursor_plane_formats);
+ formats = tegra_cursor_plane_formats;
+ }
err = drm_universal_plane_init(drm, &plane->base, possible_crtcs,
&tegra_plane_funcs, formats,
dev_err(dc->dev,
"failed to set clock rate to %lu Hz\n",
state->pclk);
+
+ err = clk_set_rate(dc->clk, state->pclk);
+ if (err < 0)
+ dev_err(dc->dev, "failed to set clock %pC to %lu Hz: %d\n",
+ dc->clk, state->pclk, err);
}
DRM_DEBUG_KMS("rate: %lu, div: %u\n", clk_get_rate(dc->clk),
value = SHIFT_CLK_DIVIDER(state->div) | PIXEL_CLK_DIVIDER_PCD1;
tegra_dc_writel(dc, value, DC_DISP_DISP_CLOCK_CONTROL);
}
-
- err = clk_set_rate(dc->clk, state->pclk);
- if (err < 0)
- dev_err(dc->dev, "failed to set clock %pC to %lu Hz: %d\n",
- dc->clk, state->pclk, err);
}
static void tegra_dc_stop(struct tegra_dc *dc)
return false;
}
+static int tegra_dc_early_init(struct host1x_client *client)
+{
+ struct drm_device *drm = dev_get_drvdata(client->host);
+ struct tegra_drm *tegra = drm->dev_private;
+
+ tegra->num_crtcs++;
+
+ return 0;
+}
+
static int tegra_dc_init(struct host1x_client *client)
{
struct drm_device *drm = dev_get_drvdata(client->host);
struct drm_plane *cursor = NULL;
int err;
+ /*
+ * DC has been reset by now, so VBLANK syncpoint can be released
+ * for general use.
+ */
+ host1x_syncpt_release_vblank_reservation(client, 26 + dc->pipe);
+
/*
* XXX do not register DCs with no window groups because we cannot
* assign a primary plane to them, which in turn will cause KMS to
if (dc->soc->pitch_align > tegra->pitch_align)
tegra->pitch_align = dc->soc->pitch_align;
+ /* track maximum resolution */
+ if (dc->soc->has_nvdisplay)
+ drm->mode_config.max_width = drm->mode_config.max_height = 16384;
+ else
+ drm->mode_config.max_width = drm->mode_config.max_height = 4096;
+
err = tegra_dc_rgb_init(drm, dc);
if (err < 0 && err != -ENODEV) {
dev_err(dc->dev, "failed to initialize RGB output: %d\n", err);
drm_plane_cleanup(primary);
host1x_client_iommu_detach(client);
- host1x_syncpt_free(dc->syncpt);
+ host1x_syncpt_put(dc->syncpt);
return err;
}
}
host1x_client_iommu_detach(client);
- host1x_syncpt_free(dc->syncpt);
+ host1x_syncpt_put(dc->syncpt);
+
+ return 0;
+}
+
+static int tegra_dc_late_exit(struct host1x_client *client)
+{
+ struct drm_device *drm = dev_get_drvdata(client->host);
+ struct tegra_drm *tegra = drm->dev_private;
+
+ tegra->num_crtcs--;
return 0;
}
}
static const struct host1x_client_ops dc_client_ops = {
+ .early_init = tegra_dc_early_init,
.init = tegra_dc_init,
.exit = tegra_dc_exit,
+ .late_exit = tegra_dc_late_exit,
.suspend = tegra_dc_runtime_suspend,
.resume = tegra_dc_runtime_resume,
};
.supports_interlacing = false,
.supports_cursor = false,
.supports_block_linear = false,
+ .supports_sector_layout = false,
.has_legacy_blending = true,
.pitch_align = 8,
.has_powergate = false,
.supports_interlacing = false,
.supports_cursor = false,
.supports_block_linear = false,
+ .supports_sector_layout = false,
.has_legacy_blending = true,
.pitch_align = 8,
.has_powergate = false,
.supports_interlacing = false,
.supports_cursor = false,
.supports_block_linear = false,
+ .supports_sector_layout = false,
.has_legacy_blending = true,
.pitch_align = 64,
.has_powergate = true,
.supports_interlacing = true,
.supports_cursor = true,
.supports_block_linear = true,
+ .supports_sector_layout = false,
.has_legacy_blending = false,
.pitch_align = 64,
.has_powergate = true,
.supports_interlacing = true,
.supports_cursor = true,
.supports_block_linear = true,
+ .supports_sector_layout = false,
.has_legacy_blending = false,
.pitch_align = 64,
.has_powergate = true,
.supports_interlacing = true,
.supports_cursor = true,
.supports_block_linear = true,
+ .supports_sector_layout = false,
.has_legacy_blending = false,
.pitch_align = 64,
.has_powergate = false,
.supports_interlacing = true,
.supports_cursor = true,
.supports_block_linear = true,
+ .supports_sector_layout = true,
.has_legacy_blending = false,
.pitch_align = 64,
.has_powergate = false,
* POWER_CONTROL registers during CRTC enabling.
*/
if (dc->soc->coupled_pm && dc->pipe == 1) {
- u32 flags = DL_FLAG_PM_RUNTIME | DL_FLAG_AUTOREMOVE_CONSUMER;
- struct device_link *link;
- struct device *partner;
+ struct device *companion;
+ struct tegra_dc *parent;
- partner = driver_find_device(dc->dev->driver, NULL, NULL,
- tegra_dc_match_by_pipe);
- if (!partner)
+ companion = driver_find_device(dc->dev->driver, NULL, (const void *)0,
+ tegra_dc_match_by_pipe);
+ if (!companion)
return -EPROBE_DEFER;
- link = device_link_add(dc->dev, partner, flags);
- if (!link) {
- dev_err(dc->dev, "failed to link controllers\n");
- return -EINVAL;
- }
+ parent = dev_get_drvdata(companion);
+ dc->client.parent = &parent->client;
- dev_dbg(dc->dev, "coupled to %s\n", dev_name(partner));
+ dev_dbg(dc->dev, "coupled to %s\n", dev_name(companion));
}
return 0;
static int tegra_dc_probe(struct platform_device *pdev)
{
+ u64 dma_mask = dma_get_mask(pdev->dev.parent);
struct tegra_dc *dc;
int err;
+ err = dma_coerce_mask_and_coherent(&pdev->dev, dma_mask);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to set DMA mask: %d\n", err);
+ return err;
+ }
+
dc = devm_kzalloc(&pdev->dev, sizeof(*dc), GFP_KERNEL);
if (!dc)
return -ENOMEM;
bool supports_interlacing;
bool supports_cursor;
bool supports_block_linear;
+ bool supports_sector_layout;
bool has_legacy_blending;
unsigned int pitch_align;
bool has_powergate;
#define DC_DISP_CURSOR_START_ADDR_HI 0x4ec
#define DC_DISP_BLEND_CURSOR_CONTROL 0x4f1
+#define CURSOR_COMPOSITION_MODE_BLEND (0 << 25)
+#define CURSOR_COMPOSITION_MODE_XOR (1 << 25)
#define CURSOR_MODE_LEGACY (0 << 24)
#define CURSOR_MODE_NORMAL (1 << 24)
#define CURSOR_DST_BLEND_ZERO (0 << 16)
#define PROTOCOL_MASK (0xf << 8)
#define PROTOCOL_SINGLE_TMDS_A (0x1 << 8)
+#define DC_DISP_PCALC_HEAD_SET_CROPPED_POINT_IN_CURSOR 0x442
+#define DC_DISP_PCALC_HEAD_SET_CROPPED_SIZE_IN_CURSOR 0x446
+
#define DC_WIN_CORE_WINDOWGROUP_SET_CONTROL 0x702
#define OWNER_MASK (0xf << 0)
#define OWNER(x) (((x) & 0xf) << 0)
dpaux->aux.transfer = tegra_dpaux_transfer;
dpaux->aux.dev = &pdev->dev;
- err = drm_dp_aux_register(&dpaux->aux);
- if (err < 0)
- return err;
+ drm_dp_aux_init(&dpaux->aux);
/*
* Assume that by default the DPAUX/I2C pads will be used for HDMI,
pm_runtime_put_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
- drm_dp_aux_unregister(&dpaux->aux);
-
mutex_lock(&dpaux_lock);
list_del(&dpaux->list);
mutex_unlock(&dpaux_lock);
unsigned long timeout;
int err;
+ err = drm_dp_aux_register(aux);
+ if (err < 0)
+ return err;
+
output->connector.polled = DRM_CONNECTOR_POLL_HPD;
dpaux->output = output;
unsigned long timeout;
int err;
+ drm_dp_aux_unregister(aux);
disable_irq(dpaux->irq);
if (dpaux->output->panel) {
struct drm_tegra_syncpt syncpt;
struct host1x *host1x = dev_get_drvdata(drm->dev->parent);
struct drm_gem_object **refs;
- struct host1x_syncpt *sp;
+ struct host1x_syncpt *sp = NULL;
struct host1x_job *job;
unsigned int num_refs;
int err;
goto fail;
}
- /* check whether syncpoint ID is valid */
- sp = host1x_syncpt_get(host1x, syncpt.id);
+ /* Syncpoint ref will be dropped on job release. */
+ sp = host1x_syncpt_get_by_id(host1x, syncpt.id);
if (!sp) {
err = -ENOENT;
goto fail;
job->is_addr_reg = context->client->ops->is_addr_reg;
job->is_valid_class = context->client->ops->is_valid_class;
job->syncpt_incrs = syncpt.incrs;
- job->syncpt_id = syncpt.id;
+ job->syncpt = sp;
job->timeout = 10000;
if (args->timeout && args->timeout < 10000)
struct drm_tegra_syncpt_read *args = data;
struct host1x_syncpt *sp;
- sp = host1x_syncpt_get(host, args->id);
+ sp = host1x_syncpt_get_by_id_noref(host, args->id);
if (!sp)
return -EINVAL;
struct drm_tegra_syncpt_incr *args = data;
struct host1x_syncpt *sp;
- sp = host1x_syncpt_get(host1x, args->id);
+ sp = host1x_syncpt_get_by_id_noref(host1x, args->id);
if (!sp)
return -EINVAL;
struct drm_tegra_syncpt_wait *args = data;
struct host1x_syncpt *sp;
- sp = host1x_syncpt_get(host1x, args->id);
+ sp = host1x_syncpt_get_by_id_noref(host1x, args->id);
if (!sp)
return -EINVAL;
drm->mode_config.min_width = 0;
drm->mode_config.min_height = 0;
-
- drm->mode_config.max_width = 4096;
- drm->mode_config.max_height = 4096;
+ drm->mode_config.max_width = 0;
+ drm->mode_config.max_height = 0;
drm->mode_config.allow_fb_modifiers = true;
if (err < 0)
goto fbdev;
+ /*
+ * Now that all display controller have been initialized, the maximum
+ * supported resolution is known and the bitmask for horizontal and
+ * vertical bitfields can be computed.
+ */
+ tegra->hmask = drm->mode_config.max_width - 1;
+ tegra->vmask = drm->mode_config.max_height - 1;
+
if (tegra->use_explicit_iommu) {
u64 carveout_start, carveout_end, gem_start, gem_end;
u64 dma_mask = dma_get_mask(&dev->dev);
#include "hub.h"
#include "trace.h"
+/* XXX move to include/uapi/drm/drm_fourcc.h? */
+#define DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT BIT(22)
+
struct reset_control;
#ifdef CONFIG_DRM_FBDEV_EMULATION
struct tegra_fbdev *fbdev;
#endif
+ unsigned int hmask, vmask;
unsigned int pitch_align;
+ unsigned int num_crtcs;
struct tegra_display_hub *hub;
};
{
uint64_t modifier = framebuffer->modifier;
+ if ((modifier >> 56) == DRM_FORMAT_MOD_VENDOR_NVIDIA) {
+ if ((modifier & DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT) == 0)
+ tiling->sector_layout = TEGRA_BO_SECTOR_LAYOUT_TEGRA;
+ else
+ tiling->sector_layout = TEGRA_BO_SECTOR_LAYOUT_GPU;
+
+ modifier &= ~DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT;
+ }
+
switch (modifier) {
case DRM_FORMAT_MOD_LINEAR:
tiling->mode = TEGRA_BO_TILING_MODE_PITCH;
break;
default:
+ DRM_DEBUG_KMS("unknown format modifier: %llx\n", modifier);
return -EINVAL;
}
TEGRA_BO_TILING_MODE_BLOCK,
};
+enum tegra_bo_sector_layout {
+ TEGRA_BO_SECTOR_LAYOUT_TEGRA,
+ TEGRA_BO_SECTOR_LAYOUT_GPU,
+};
+
struct tegra_bo_tiling {
enum tegra_bo_tiling_mode mode;
unsigned long value;
+ enum tegra_bo_sector_layout sector_layout;
};
struct tegra_bo {
detach:
host1x_client_iommu_detach(client);
free:
- host1x_syncpt_free(client->syncpts[0]);
+ host1x_syncpt_put(client->syncpts[0]);
put:
host1x_channel_put(gr2d->channel);
return err;
return err;
host1x_client_iommu_detach(client);
- host1x_syncpt_free(client->syncpts[0]);
+ host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(gr2d->channel);
return 0;
detach:
host1x_client_iommu_detach(client);
free:
- host1x_syncpt_free(client->syncpts[0]);
+ host1x_syncpt_put(client->syncpts[0]);
put:
host1x_channel_put(gr3d->channel);
return err;
return err;
host1x_client_iommu_detach(client);
- host1x_syncpt_free(client->syncpts[0]);
+ host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(gr3d->channel);
return 0;
DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(3),
DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(4),
DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(5),
+ /*
+ * The GPU sector layout is only supported on Tegra194, but these will
+ * be filtered out later on by ->format_mod_supported() on SoCs where
+ * it isn't supported.
+ */
+ DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(0) | DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT,
+ DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(1) | DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT,
+ DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(2) | DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT,
+ DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(3) | DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT,
+ DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(4) | DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT,
+ DRM_FORMAT_MOD_NVIDIA_16BX2_BLOCK(5) | DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT,
+ /* sentinel */
DRM_FORMAT_MOD_INVALID
};
return -EINVAL;
}
+ if (tiling->sector_layout == TEGRA_BO_SECTOR_LAYOUT_GPU &&
+ !dc->soc->supports_sector_layout) {
+ DRM_ERROR("hardware doesn't support GPU sector layout\n");
+ return -EINVAL;
+ }
+
/*
* Tegra doesn't support different strides for U and V planes so we
* error out if the user tries to display a framebuffer with such a
base = tegra_plane_state->iova[0] + fb->offsets[0];
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ /*
+ * Physical address bit 39 in Tegra194 is used as a switch for special
+ * logic that swizzles the memory using either the legacy Tegra or the
+ * dGPU sector layout.
+ */
+ if (tegra_plane_state->tiling.sector_layout == TEGRA_BO_SECTOR_LAYOUT_GPU)
+ base |= BIT(39);
+#endif
+
tegra_plane_writel(p, tegra_plane_state->format, DC_WIN_COLOR_DEPTH);
tegra_plane_writel(p, 0, DC_WIN_PRECOMP_WGRP_PARAMS);
enum drm_plane_type type = DRM_PLANE_TYPE_OVERLAY;
struct tegra_drm *tegra = drm->dev_private;
struct tegra_display_hub *hub = tegra->hub;
- /* planes can be assigned to arbitrary CRTCs */
- unsigned int possible_crtcs = 0x7;
struct tegra_shared_plane *plane;
+ unsigned int possible_crtcs;
unsigned int num_formats;
const u64 *modifiers;
struct drm_plane *p;
p = &plane->base.base;
+ /* planes can be assigned to arbitrary CRTCs */
+ possible_crtcs = BIT(tegra->num_crtcs) - 1;
+
num_formats = ARRAY_SIZE(tegra_shared_plane_formats);
formats = tegra_shared_plane_formats;
modifiers = tegra_shared_plane_modifiers;
static int tegra_display_hub_probe(struct platform_device *pdev)
{
+ u64 dma_mask = dma_get_mask(pdev->dev.parent);
struct device_node *child = NULL;
struct tegra_display_hub *hub;
struct clk *clk;
unsigned int i;
int err;
+ err = dma_coerce_mask_and_coherent(&pdev->dev, dma_mask);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to set DMA mask: %d\n", err);
+ return err;
+ }
+
hub = devm_kzalloc(&pdev->dev, sizeof(*hub), GFP_KERNEL);
if (!hub)
return -ENOMEM;
kfree(state);
}
+static bool tegra_plane_supports_sector_layout(struct drm_plane *plane)
+{
+ struct drm_crtc *crtc;
+
+ drm_for_each_crtc(crtc, plane->dev) {
+ if (plane->possible_crtcs & drm_crtc_mask(crtc)) {
+ struct tegra_dc *dc = to_tegra_dc(crtc);
+
+ if (!dc->soc->supports_sector_layout)
+ return false;
+ }
+ }
+
+ return true;
+}
+
static bool tegra_plane_format_mod_supported(struct drm_plane *plane,
uint32_t format,
uint64_t modifier)
if (modifier == DRM_FORMAT_MOD_LINEAR)
return true;
+ /* check for the sector layout bit */
+ if ((modifier >> 56) == DRM_FORMAT_MOD_VENDOR_NVIDIA) {
+ if (modifier & DRM_FORMAT_MOD_NVIDIA_SECTOR_LAYOUT) {
+ if (!tegra_plane_supports_sector_layout(plane))
+ return false;
+ }
+ }
+
if (info->num_planes == 1)
return true;
dma_addr_t phys_addr, *phys;
struct sg_table *sgt;
+ /*
+ * If we're not attached to a domain, we already stored the
+ * physical address when the buffer was allocated. If we're
+ * part of a group that's shared between all display
+ * controllers, we've also already mapped the framebuffer
+ * through the SMMU. In both cases we can short-circuit the
+ * code below and retrieve the stored IOV address.
+ */
if (!domain || dc->client.group)
phys = &phys_addr;
else
* kernel is possible.
*/
if (sor->rst) {
+ err = pm_runtime_resume_and_get(sor->dev);
+ if (err < 0) {
+ dev_err(sor->dev, "failed to get runtime PM: %d\n", err);
+ return err;
+ }
+
err = reset_control_acquire(sor->rst);
if (err < 0) {
dev_err(sor->dev, "failed to acquire SOR reset: %d\n",
}
reset_control_release(sor->rst);
+ pm_runtime_put(sor->dev);
}
err = clk_prepare_enable(sor->clk_safe);
return 0;
free_syncpt:
- host1x_syncpt_free(client->syncpts[0]);
+ host1x_syncpt_put(client->syncpts[0]);
free_channel:
host1x_channel_put(vic->channel);
detach:
if (err < 0)
return err;
- host1x_syncpt_free(client->syncpts[0]);
+ host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(vic->channel);
host1x_client_iommu_detach(client);
return 0;
if (no_wait_gpu) {
+ ret = dma_fence_is_signaled(fence) ? 0 : -EBUSY;
dma_fence_put(fence);
- return -EBUSY;
+ return ret;
}
dma_resv_add_shared_fence(bo->base.resv, fence);
if (mem->bus.caching == ttm_write_combined)
addr = ioremap_wc(mem->bus.offset, bus_size);
+#ifdef CONFIG_X86
+ else if (mem->bus.caching == ttm_cached)
+ addr = ioremap_cache(mem->bus.offset, bus_size);
+#endif
else
addr = ioremap(mem->bus.offset, bus_size);
if (!addr) {
if (mem->bus.caching == ttm_write_combined)
map->virtual = ioremap_wc(bo->mem.bus.offset + offset,
size);
+#ifdef CONFIG_X86
+ else if (mem->bus.caching == ttm_cached)
+ map->virtual = ioremap_cache(bo->mem.bus.offset + offset,
+ size);
+#endif
else
map->virtual = ioremap(bo->mem.bus.offset + offset,
size);
else if (mem->bus.caching == ttm_write_combined)
vaddr_iomem = ioremap_wc(mem->bus.offset,
bo->base.size);
+#ifdef CONFIG_X86
+ else if (mem->bus.caching == ttm_cached)
+ vaddr_iomem = ioremap_cache(mem->bus.offset,
+ bo->base.size);
+#endif
else
vaddr_iomem = ioremap(mem->bus.offset, bo->base.size);
{
const struct vc4_crtc_data *crtc_data = vc4_crtc_to_vc4_crtc_data(vc4_crtc);
const struct vc4_pv_data *pv_data = vc4_crtc_to_vc4_pv_data(vc4_crtc);
+ struct vc4_dev *vc4 = to_vc4_dev(vc4_crtc->base.dev);
u32 fifo_len_bytes = pv_data->fifo_depth;
/*
if (crtc_data->hvs_output == 5)
return 32;
+ /*
+ * It looks like in some situations, we will overflow
+ * the PixelValve FIFO (with the bit 10 of PV stat being
+ * set) and stall the HVS / PV, eventually resulting in
+ * a page flip timeout.
+ *
+ * Displaying the video overlay during a playback with
+ * Kodi on an RPi3 seems to be a great solution with a
+ * failure rate around 50%.
+ *
+ * Removing 1 from the FIFO full level however
+ * seems to completely remove that issue.
+ */
+ if (!vc4->hvs->hvs5)
+ return fifo_len_bytes - 3 * HVS_FIFO_LATENCY_PIX - 1;
+
return fifo_len_bytes - 3 * HVS_FIFO_LATENCY_PIX;
}
}
plane->state->src_y = new_plane_state->src_y;
plane->state->src_w = new_plane_state->src_w;
plane->state->src_h = new_plane_state->src_h;
- plane->state->src_h = new_plane_state->src_h;
plane->state->alpha = new_plane_state->alpha;
plane->state->pixel_blend_mode = new_plane_state->pixel_blend_mode;
plane->state->rotation = new_plane_state->rotation;
drm_dev = drm_dev_alloc(&xen_drm_driver, dev);
if (IS_ERR(drm_dev)) {
ret = PTR_ERR(drm_dev);
- goto fail;
+ goto fail_dev;
}
drm_info->drm_dev = drm_dev;
drm_kms_helper_poll_fini(drm_dev);
drm_mode_config_cleanup(drm_dev);
drm_dev_put(drm_dev);
-fail:
+fail_dev:
kfree(drm_info);
+ front_info->drm_info = NULL;
+fail:
return ret;
}
struct drm_connector;
struct xen_drm_front_drm_info;
-struct xen_drm_front_drm_info;
int xen_drm_front_conn_init(struct xen_drm_front_drm_info *drm_info,
struct drm_connector *connector);
return ret;
zynqmp_dp_write(dp, ZYNQMP_DP_SCRAMBLING_DISABLE, 1);
- memset(dp->train_set, 0, 4);
+ memset(dp->train_set, 0, sizeof(dp->train_set));
ret = zynqmp_dp_link_train_cr(dp);
if (ret)
return ret;
mutex_lock(&device->clients_lock);
+ list_for_each_entry(client, &device->clients, list) {
+ if (client->ops && client->ops->early_init) {
+ err = client->ops->early_init(client);
+ if (err < 0) {
+ dev_err(&device->dev, "failed to early initialize %s: %d\n",
+ dev_name(client->dev), err);
+ goto teardown_late;
+ }
+ }
+ }
+
list_for_each_entry(client, &device->clients, list) {
if (client->ops && client->ops->init) {
err = client->ops->init(client);
if (client->ops->exit)
client->ops->exit(client);
+ /* reset client to end of list for late teardown */
+ client = list_entry(&device->clients, struct host1x_client, list);
+
+teardown_late:
+ list_for_each_entry_continue_reverse(client, &device->clients, list)
+ if (client->ops->late_exit)
+ client->ops->late_exit(client);
+
mutex_unlock(&device->clients_lock);
return err;
}
}
}
+ list_for_each_entry_reverse(client, &device->clients, list) {
+ if (client->ops && client->ops->late_exit) {
+ err = client->ops->late_exit(client);
+ if (err < 0) {
+ dev_err(&device->dev, "failed to late cleanup %s: %d\n",
+ dev_name(client->dev), err);
+ mutex_unlock(&device->clients_lock);
+ return err;
+ }
+ }
+ }
+
mutex_unlock(&device->clients_lock);
return 0;
EXPORT_SYMBOL(host1x_driver_unregister);
/**
- * host1x_client_register() - register a host1x client
+ * __host1x_client_register() - register a host1x client
* @client: host1x client
+ * @key: lock class key for the client-specific mutex
*
* Registers a host1x client with each host1x controller instance. Note that
* each client will only match their parent host1x controller and will only be
* device and call host1x_device_init(), which will in turn call each client's
* &host1x_client_ops.init implementation.
*/
-int host1x_client_register(struct host1x_client *client)
+int __host1x_client_register(struct host1x_client *client,
+ struct lock_class_key *key)
{
struct host1x *host1x;
int err;
INIT_LIST_HEAD(&client->list);
- mutex_init(&client->lock);
+ __mutex_init(&client->lock, "host1x client lock", key);
client->usecount = 0;
mutex_lock(&devices_lock);
return 0;
}
-EXPORT_SYMBOL(host1x_client_register);
+EXPORT_SYMBOL(__host1x_client_register);
/**
* host1x_client_unregister() - unregister a host1x client
static void cdma_start_timer_locked(struct host1x_cdma *cdma,
struct host1x_job *job)
{
- struct host1x *host = cdma_to_host1x(cdma);
-
if (cdma->timeout.client) {
/* timer already started */
return;
}
cdma->timeout.client = job->client;
- cdma->timeout.syncpt = host1x_syncpt_get(host, job->syncpt_id);
+ cdma->timeout.syncpt = job->syncpt;
cdma->timeout.syncpt_val = job->syncpt_end;
cdma->timeout.start_ktime = ktime_get();
static void update_cdma_locked(struct host1x_cdma *cdma)
{
bool signal = false;
- struct host1x *host1x = cdma_to_host1x(cdma);
struct host1x_job *job, *n;
/* If CDMA is stopped, queue is cleared and we can return */
* to consume as many sync queue entries as possible without blocking
*/
list_for_each_entry_safe(job, n, &cdma->sync_queue, list) {
- struct host1x_syncpt *sp =
- host1x_syncpt_get(host1x, job->syncpt_id);
+ struct host1x_syncpt *sp = job->syncpt;
/* Check whether this syncpt has completed, and bail if not */
if (!host1x_syncpt_is_expired(sp, job->syncpt_end)) {
if (!cdma->timeout.initialized) {
int err;
- err = host1x_hw_cdma_timeout_init(host1x, cdma,
- job->syncpt_id);
+ err = host1x_hw_cdma_timeout_init(host1x, cdma);
if (err) {
mutex_unlock(&cdma->lock);
return err;
static void show_syncpts(struct host1x *m, struct output *o)
{
+ struct list_head *pos;
unsigned int i;
host1x_debug_output(o, "---- syncpts ----\n");
for (i = 0; i < host1x_syncpt_nb_pts(m); i++) {
u32 max = host1x_syncpt_read_max(m->syncpt + i);
u32 min = host1x_syncpt_load(m->syncpt + i);
+ unsigned int waiters = 0;
- if (!min && !max)
+ spin_lock(&m->syncpt[i].intr.lock);
+ list_for_each(pos, &m->syncpt[i].intr.wait_head)
+ waiters++;
+ spin_unlock(&m->syncpt[i].intr.lock);
+
+ if (!min && !max && !waiters)
continue;
- host1x_debug_output(o, "id %u (%s) min %d max %d\n",
- i, m->syncpt[i].name, min, max);
+ host1x_debug_output(o,
+ "id %u (%s) min %d max %d (%d waiters)\n",
+ i, m->syncpt[i].name, min, max, waiters);
}
for (i = 0; i < host1x_syncpt_nb_bases(m); i++) {
.has_hypervisor = false,
.num_sid_entries = 0,
.sid_table = NULL,
+ .reserve_vblank_syncpts = true,
};
static const struct host1x_info host1x02_info = {
.has_hypervisor = false,
.num_sid_entries = 0,
.sid_table = NULL,
+ .reserve_vblank_syncpts = true,
};
static const struct host1x_info host1x04_info = {
.has_hypervisor = false,
.num_sid_entries = 0,
.sid_table = NULL,
+ .reserve_vblank_syncpts = false,
};
static const struct host1x_info host1x05_info = {
.has_hypervisor = false,
.num_sid_entries = 0,
.sid_table = NULL,
+ .reserve_vblank_syncpts = false,
};
static const struct host1x_sid_entry tegra186_sid_table[] = {
.has_hypervisor = true,
.num_sid_entries = ARRAY_SIZE(tegra186_sid_table),
.sid_table = tegra186_sid_table,
+ .reserve_vblank_syncpts = false,
};
static const struct host1x_sid_entry tegra194_sid_table[] = {
.has_hypervisor = true,
.num_sid_entries = ARRAY_SIZE(tegra194_sid_table),
.sid_table = tegra194_sid_table,
+ .reserve_vblank_syncpts = false,
};
static const struct of_device_id host1x_of_match[] = {
void (*start)(struct host1x_cdma *cdma);
void (*stop)(struct host1x_cdma *cdma);
void (*flush)(struct host1x_cdma *cdma);
- int (*timeout_init)(struct host1x_cdma *cdma, unsigned int syncpt);
+ int (*timeout_init)(struct host1x_cdma *cdma);
void (*timeout_destroy)(struct host1x_cdma *cdma);
void (*freeze)(struct host1x_cdma *cdma);
void (*resume)(struct host1x_cdma *cdma, u32 getptr);
bool has_hypervisor; /* has hypervisor registers */
unsigned int num_sid_entries;
const struct host1x_sid_entry *sid_table;
+ /*
+ * On T20-T148, the boot chain may setup DC to increment syncpoints
+ * 26/27 on VBLANK. As such we cannot use these syncpoints until
+ * the display driver disables VBLANK increments.
+ */
+ bool reserve_vblank_syncpts;
};
struct host1x {
}
static inline int host1x_hw_cdma_timeout_init(struct host1x *host,
- struct host1x_cdma *cdma,
- unsigned int syncpt)
+ struct host1x_cdma *cdma)
{
- return host->cdma_op->timeout_init(cdma, syncpt);
+ return host->cdma_op->timeout_init(cdma);
}
static inline void host1x_hw_cdma_timeout_destroy(struct host1x *host,
/*
* Init timeout resources
*/
-static int cdma_timeout_init(struct host1x_cdma *cdma, unsigned int syncpt)
+static int cdma_timeout_init(struct host1x_cdma *cdma)
{
INIT_DELAYED_WORK(&cdma->timeout.wq, cdma_timeout_handler);
cdma->timeout.initialized = true;
static inline void synchronize_syncpt_base(struct host1x_job *job)
{
- struct host1x *host = dev_get_drvdata(job->channel->dev->parent);
- struct host1x_syncpt *sp = host->syncpt + job->syncpt_id;
+ struct host1x_syncpt *sp = job->syncpt;
unsigned int id;
u32 value;
static int channel_submit(struct host1x_job *job)
{
struct host1x_channel *ch = job->channel;
- struct host1x_syncpt *sp;
+ struct host1x_syncpt *sp = job->syncpt;
u32 user_syncpt_incrs = job->syncpt_incrs;
u32 prev_max = 0;
u32 syncval;
struct host1x_waitlist *completed_waiter = NULL;
struct host1x *host = dev_get_drvdata(ch->dev->parent);
- sp = host->syncpt + job->syncpt_id;
trace_host1x_channel_submit(dev_name(ch->dev),
job->num_gathers, job->num_relocs,
- job->syncpt_id, job->syncpt_incrs);
+ job->syncpt->id, job->syncpt_incrs);
/* before error checks, return current max */
prev_max = job->syncpt_end = host1x_syncpt_read_max(sp);
host1x_cdma_push(&ch->cdma,
host1x_opcode_setclass(HOST1X_CLASS_HOST1X,
host1x_uclass_wait_syncpt_r(), 1),
- host1x_class_host_wait_syncpt(job->syncpt_id,
+ host1x_class_host_wait_syncpt(job->syncpt->id,
host1x_syncpt_read_max(sp)));
}
unsigned int i;
host1x_debug_output(o, "\n%p: JOB, syncpt_id=%d, syncpt_val=%d, first_get=%08x, timeout=%d num_slots=%d, num_handles=%d\n",
- job, job->syncpt_id, job->syncpt_end,
+ job, job->syncpt->id, job->syncpt_end,
job->first_get, job->timeout,
job->num_slots, job->num_unpins);
#define HOST1X_SYNC_SYNCPT_THRESH_INT_ENABLE_CPU0(x) (0x652c + 4 * (x))
#define HOST1X_SYNC_SYNCPT_THRESH_INT_DISABLE(x) (0x6590 + 4 * (x))
#define HOST1X_SYNC_SYNCPT(x) (0x8080 + 4 * (x))
-#define HOST1X_SYNC_SYNCPT_INT_THRESH(x) (0x8d00 + 4 * (x))
+#define HOST1X_SYNC_SYNCPT_INT_THRESH(x) (0x9980 + 4 * (x))
#define HOST1X_SYNC_SYNCPT_CH_APP(x) (0xa604 + 4 * (x))
#define HOST1X_SYNC_SYNCPT_CH_APP_CH(v) (((v) & 0x3f) << 8)
host1x_hw_intr_enable_syncpt_intr(host, syncpt->id);
}
- spin_unlock(&syncpt->intr.lock);
-
if (ref)
*ref = waiter;
+
+ spin_unlock(&syncpt->intr.lock);
+
return 0;
}
-void host1x_intr_put_ref(struct host1x *host, unsigned int id, void *ref)
+void host1x_intr_put_ref(struct host1x *host, unsigned int id, void *ref,
+ bool flush)
{
struct host1x_waitlist *waiter = ref;
struct host1x_syncpt *syncpt;
- while (atomic_cmpxchg(&waiter->state, WLS_PENDING, WLS_CANCELLED) ==
- WLS_REMOVED)
- schedule();
+ atomic_cmpxchg(&waiter->state, WLS_PENDING, WLS_CANCELLED);
syncpt = host->syncpt + id;
- (void)process_wait_list(host, syncpt,
- host1x_syncpt_load(host->syncpt + id));
+
+ spin_lock(&syncpt->intr.lock);
+ if (atomic_cmpxchg(&waiter->state, WLS_CANCELLED, WLS_HANDLED) ==
+ WLS_CANCELLED) {
+ list_del(&waiter->list);
+ kref_put(&waiter->refcount, waiter_release);
+ }
+ spin_unlock(&syncpt->intr.lock);
+
+ if (flush) {
+ /* Wait until any concurrently executing handler has finished. */
+ while (atomic_read(&waiter->state) != WLS_HANDLED)
+ schedule();
+ }
kref_put(&waiter->refcount, waiter_release);
}
* Unreference an action submitted to host1x_intr_add_action().
* You must call this if you passed non-NULL as ref.
* @ref the ref returned from host1x_intr_add_action()
+ * @flush wait until any pending handlers have completed before returning.
*/
-void host1x_intr_put_ref(struct host1x *host, unsigned int id, void *ref);
+void host1x_intr_put_ref(struct host1x *host, unsigned int id, void *ref,
+ bool flush);
/* Initialize host1x sync point interrupt */
int host1x_intr_init(struct host1x *host, unsigned int irq_sync);
{
struct host1x_job *job = container_of(ref, struct host1x_job, ref);
+ if (job->syncpt)
+ host1x_syncpt_put(job->syncpt);
+
kfree(job);
}
*/
void host1x_job_dump(struct device *dev, struct host1x_job *job)
{
- dev_dbg(dev, " SYNCPT_ID %d\n", job->syncpt_id);
+ dev_dbg(dev, " SYNCPT_ID %d\n", job->syncpt->id);
dev_dbg(dev, " SYNCPT_VAL %d\n", job->syncpt_end);
dev_dbg(dev, " FIRST_GET 0x%x\n", job->first_get);
dev_dbg(dev, " TIMEOUT %d\n", job->timeout);
base->requested = false;
}
-static struct host1x_syncpt *host1x_syncpt_alloc(struct host1x *host,
- struct host1x_client *client,
- unsigned long flags)
+/**
+ * host1x_syncpt_alloc() - allocate a syncpoint
+ * @host: host1x device data
+ * @flags: bitfield of HOST1X_SYNCPT_* flags
+ * @name: name for the syncpoint for use in debug prints
+ *
+ * Allocates a hardware syncpoint for the caller's use. The caller then has
+ * the sole authority to mutate the syncpoint's value until it is freed again.
+ *
+ * If no free syncpoints are available, or a NULL name was specified, returns
+ * NULL.
+ */
+struct host1x_syncpt *host1x_syncpt_alloc(struct host1x *host,
+ unsigned long flags,
+ const char *name)
{
struct host1x_syncpt *sp = host->syncpt;
+ char *full_name;
unsigned int i;
- char *name;
+
+ if (!name)
+ return NULL;
mutex_lock(&host->syncpt_mutex);
- for (i = 0; i < host->info->nb_pts && sp->name; i++, sp++)
+ for (i = 0; i < host->info->nb_pts && kref_read(&sp->ref); i++, sp++)
;
if (i >= host->info->nb_pts)
goto unlock;
}
- name = kasprintf(GFP_KERNEL, "%02u-%s", sp->id,
- client ? dev_name(client->dev) : NULL);
- if (!name)
+ full_name = kasprintf(GFP_KERNEL, "%u-%s", sp->id, name);
+ if (!full_name)
goto free_base;
- sp->client = client;
- sp->name = name;
+ sp->name = full_name;
if (flags & HOST1X_SYNCPT_CLIENT_MANAGED)
sp->client_managed = true;
else
sp->client_managed = false;
+ kref_init(&sp->ref);
+
mutex_unlock(&host->syncpt_mutex);
return sp;
mutex_unlock(&host->syncpt_mutex);
return NULL;
}
+EXPORT_SYMBOL(host1x_syncpt_alloc);
/**
* host1x_syncpt_id() - retrieve syncpoint ID
}
}
- host1x_intr_put_ref(sp->host, sp->id, ref);
+ host1x_intr_put_ref(sp->host, sp->id, ref, true);
done:
return err;
bool host1x_syncpt_is_expired(struct host1x_syncpt *sp, u32 thresh)
{
u32 current_val;
- u32 future_val;
smp_rmb();
current_val = (u32)atomic_read(&sp->min_val);
- future_val = (u32)atomic_read(&sp->max_val);
-
- /* Note the use of unsigned arithmetic here (mod 1<<32).
- *
- * c = current_val = min_val = the current value of the syncpoint.
- * t = thresh = the value we are checking
- * f = future_val = max_val = the value c will reach when all
- * outstanding increments have completed.
- *
- * Note that c always chases f until it reaches f.
- *
- * Dtf = (f - t)
- * Dtc = (c - t)
- *
- * Consider all cases:
- *
- * A) .....c..t..f..... Dtf < Dtc need to wait
- * B) .....c.....f..t.. Dtf > Dtc expired
- * C) ..t..c.....f..... Dtf > Dtc expired (Dct very large)
- *
- * Any case where f==c: always expired (for any t). Dtf == Dcf
- * Any case where t==c: always expired (for any f). Dtf >= Dtc (because Dtc==0)
- * Any case where t==f!=c: always wait. Dtf < Dtc (because Dtf==0,
- * Dtc!=0)
- *
- * Other cases:
- *
- * A) .....t..f..c..... Dtf < Dtc need to wait
- * A) .....f..c..t..... Dtf < Dtc need to wait
- * A) .....f..t..c..... Dtf > Dtc expired
- *
- * So:
- * Dtf >= Dtc implies EXPIRED (return true)
- * Dtf < Dtc implies WAIT (return false)
- *
- * Note: If t is expired then we *cannot* wait on it. We would wait
- * forever (hang the system).
- *
- * Note: do NOT get clever and remove the -thresh from both sides. It
- * is NOT the same.
- *
- * If future valueis zero, we have a client managed sync point. In that
- * case we do a direct comparison.
- */
- if (!host1x_syncpt_client_managed(sp))
- return future_val - thresh >= current_val - thresh;
- else
- return (s32)(current_val - thresh) >= 0;
+
+ return ((current_val - thresh) & 0x80000000U) == 0U;
}
int host1x_syncpt_init(struct host1x *host)
host1x_hw_syncpt_enable_protection(host);
/* Allocate sync point to use for clearing waits for expired fences */
- host->nop_sp = host1x_syncpt_alloc(host, NULL, 0);
+ host->nop_sp = host1x_syncpt_alloc(host, 0, "reserved-nop");
if (!host->nop_sp)
return -ENOMEM;
+ if (host->info->reserve_vblank_syncpts) {
+ kref_init(&host->syncpt[26].ref);
+ kref_init(&host->syncpt[27].ref);
+ }
+
return 0;
}
* host1x client drivers can use this function to allocate a syncpoint for
* subsequent use. A syncpoint returned by this function will be reserved for
* use by the client exclusively. When no longer using a syncpoint, a host1x
- * client driver needs to release it using host1x_syncpt_free().
+ * client driver needs to release it using host1x_syncpt_put().
*/
struct host1x_syncpt *host1x_syncpt_request(struct host1x_client *client,
unsigned long flags)
{
struct host1x *host = dev_get_drvdata(client->host->parent);
- return host1x_syncpt_alloc(host, client, flags);
+ return host1x_syncpt_alloc(host, flags, dev_name(client->dev));
}
EXPORT_SYMBOL(host1x_syncpt_request);
-/**
- * host1x_syncpt_free() - free a requested syncpoint
- * @sp: host1x syncpoint
- *
- * Release a syncpoint previously allocated using host1x_syncpt_request(). A
- * host1x client driver should call this when the syncpoint is no longer in
- * use. Note that client drivers must ensure that the syncpoint doesn't remain
- * under the control of hardware after calling this function, otherwise two
- * clients may end up trying to access the same syncpoint concurrently.
- */
-void host1x_syncpt_free(struct host1x_syncpt *sp)
+static void syncpt_release(struct kref *ref)
{
- if (!sp)
- return;
+ struct host1x_syncpt *sp = container_of(ref, struct host1x_syncpt, ref);
+
+ atomic_set(&sp->max_val, host1x_syncpt_read(sp));
mutex_lock(&sp->host->syncpt_mutex);
host1x_syncpt_base_free(sp->base);
kfree(sp->name);
sp->base = NULL;
- sp->client = NULL;
sp->name = NULL;
sp->client_managed = false;
mutex_unlock(&sp->host->syncpt_mutex);
}
-EXPORT_SYMBOL(host1x_syncpt_free);
+
+/**
+ * host1x_syncpt_put() - free a requested syncpoint
+ * @sp: host1x syncpoint
+ *
+ * Release a syncpoint previously allocated using host1x_syncpt_request(). A
+ * host1x client driver should call this when the syncpoint is no longer in
+ * use.
+ */
+void host1x_syncpt_put(struct host1x_syncpt *sp)
+{
+ if (!sp)
+ return;
+
+ kref_put(&sp->ref, syncpt_release);
+}
+EXPORT_SYMBOL(host1x_syncpt_put);
void host1x_syncpt_deinit(struct host1x *host)
{
}
/**
- * host1x_syncpt_get() - obtain a syncpoint by ID
+ * host1x_syncpt_get_by_id() - obtain a syncpoint by ID
* @host: host1x controller
* @id: syncpoint ID
*/
-struct host1x_syncpt *host1x_syncpt_get(struct host1x *host, unsigned int id)
+struct host1x_syncpt *host1x_syncpt_get_by_id(struct host1x *host,
+ unsigned int id)
{
if (id >= host->info->nb_pts)
return NULL;
- return host->syncpt + id;
+ if (kref_get_unless_zero(&host->syncpt[id].ref))
+ return &host->syncpt[id];
+ else
+ return NULL;
+}
+EXPORT_SYMBOL(host1x_syncpt_get_by_id);
+
+/**
+ * host1x_syncpt_get_by_id_noref() - obtain a syncpoint by ID but don't
+ * increase the refcount.
+ * @host: host1x controller
+ * @id: syncpoint ID
+ */
+struct host1x_syncpt *host1x_syncpt_get_by_id_noref(struct host1x *host,
+ unsigned int id)
+{
+ if (id >= host->info->nb_pts)
+ return NULL;
+
+ return &host->syncpt[id];
+}
+EXPORT_SYMBOL(host1x_syncpt_get_by_id_noref);
+
+/**
+ * host1x_syncpt_get() - increment syncpoint refcount
+ * @sp: syncpoint
+ */
+struct host1x_syncpt *host1x_syncpt_get(struct host1x_syncpt *sp)
+{
+ kref_get(&sp->ref);
+
+ return sp;
}
EXPORT_SYMBOL(host1x_syncpt_get);
return base->id;
}
EXPORT_SYMBOL(host1x_syncpt_base_id);
+
+static void do_nothing(struct kref *ref)
+{
+}
+
+/**
+ * host1x_syncpt_release_vblank_reservation() - Make VBLANK syncpoint
+ * available for allocation
+ *
+ * @client: host1x bus client
+ * @syncpt_id: syncpoint ID to make available
+ *
+ * Makes VBLANK<i> syncpoint available for allocatation if it was
+ * reserved at initialization time. This should be called by the display
+ * driver after it has ensured that any VBLANK increment programming configured
+ * by the boot chain has been disabled.
+ */
+void host1x_syncpt_release_vblank_reservation(struct host1x_client *client,
+ u32 syncpt_id)
+{
+ struct host1x *host = dev_get_drvdata(client->host->parent);
+
+ if (!host->info->reserve_vblank_syncpts)
+ return;
+
+ kref_put(&host->syncpt[syncpt_id].ref, do_nothing);
+}
+EXPORT_SYMBOL(host1x_syncpt_release_vblank_reservation);
#include <linux/atomic.h>
#include <linux/host1x.h>
#include <linux/kernel.h>
+#include <linux/kref.h>
#include <linux/sched.h>
#include "intr.h"
};
struct host1x_syncpt {
+ struct kref ref;
+
unsigned int id;
atomic_t min_val;
atomic_t max_val;
const char *name;
bool client_managed;
struct host1x *host;
- struct host1x_client *client;
struct host1x_syncpt_base *base;
/* interrupt data */
if ((comp_param1 & DW_IC_COMP_PARAM_1_SPEED_MODE_MASK)
!= DW_IC_COMP_PARAM_1_SPEED_MODE_HIGH) {
dev_err(dev->dev, "High Speed not supported!\n");
+ t->bus_freq_hz = I2C_MAX_FAST_MODE_FREQ;
dev->master_cfg &= ~DW_IC_CON_SPEED_MASK;
dev->master_cfg |= DW_IC_CON_SPEED_FAST;
dev->hs_hcnt = 0;
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* i2c-exynos5.c - Samsung Exynos5 I2C Controller Driver
*
* Copyright (C) 2013 Samsung Electronics Co., Ltd.
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2014 Linaro Ltd.
- * Copyright (c) 2014 Hisilicon Limited.
+ * Copyright (c) 2014 HiSilicon Limited.
*
* Now only support 7 bit address.
*/
i2c_sta = jz4780_i2c_readw(i2c, JZ4780_I2C_STA);
data = *i2c->wbuf;
data &= ~JZ4780_I2C_DC_READ;
- if ((!i2c->stop_hold) && (i2c->cdata->version >=
- ID_X1000))
+ if ((i2c->wt_len == 1) && (!i2c->stop_hold) &&
+ (i2c->cdata->version >= ID_X1000))
data |= X1000_I2C_DC_STOP;
jz4780_i2c_writew(i2c, JZ4780_I2C_DC, data);
i2c->wbuf++;
default:
/*
* N-byte reception:
- * Enable ACK, reset POS (ACK postion) and clear ADDR flag.
+ * Enable ACK, reset POS (ACK position) and clear ADDR flag.
* In that way, ACK will be sent as soon as the current byte
* will be received in the shift register
*/
static int i2c_init_recovery(struct i2c_adapter *adap)
{
struct i2c_bus_recovery_info *bri = adap->bus_recovery_info;
- char *err_str;
+ char *err_str, *err_level = KERN_ERR;
if (!bri)
return 0;
return -EPROBE_DEFER;
if (!bri->recover_bus) {
- err_str = "no recover_bus() found";
+ err_str = "no suitable method provided";
+ err_level = KERN_DEBUG;
goto err;
}
return 0;
err:
- dev_err(&adap->dev, "Not using recovery: %s\n", err_str);
+ dev_printk(err_level, &adap->dev, "Not using recovery: %s\n", err_str);
adap->bus_recovery_info = NULL;
return -EINVAL;
select IIO_BUFFER
select IIO_BUFFER_HW_CONSUMER
select IIO_BUFFER_DMAENGINE
+ depends on HAS_IOMEM
+ depends on OF
help
Say yes here to build support for Analog Devices Generic
AXI ADC IP core. The IP core is used for interfacing with
depends on ARCH_STM32 || COMPILE_TEST
depends on OF
depends on REGULATOR
+ depends on HAS_IOMEM
select IIO_BUFFER
select MFD_STM32_TIMERS
select IIO_STM32_TIMER_TRIGGER
return processed;
/* Return millivolt or milliamps or millicentigrades */
- *val = processed * 1000;
+ *val = processed;
return IIO_VAL_INT;
}
int ret;
int i;
int bits_per_word = ad7949_adc->resolution;
- int mask = GENMASK(ad7949_adc->resolution, 0);
+ int mask = GENMASK(ad7949_adc->resolution - 1, 0);
struct spi_message msg;
struct spi_transfer tx[] = {
{
VADC_CHAN_NO_SCALE(P_MUX16_1_3, 1)
VADC_CHAN_NO_SCALE(LR_MUX1_BAT_THERM, 0)
- VADC_CHAN_NO_SCALE(LR_MUX2_BAT_ID, 0)
+ VADC_CHAN_VOLT(LR_MUX2_BAT_ID, 0, SCALE_DEFAULT)
VADC_CHAN_NO_SCALE(LR_MUX3_XO_THERM, 0)
VADC_CHAN_NO_SCALE(LR_MUX4_AMUX_THM1, 0)
VADC_CHAN_NO_SCALE(LR_MUX5_AMUX_THM2, 0)
MPU3050_FIFO_R,
&fifo_values[offset],
toread);
+ if (ret)
+ goto out_trigger_unlock;
dev_dbg(mpu3050->dev,
"%04x %04x %04x %04x %04x\n",
struct hid_humidity_state {
struct hid_sensor_common common_attributes;
struct hid_sensor_hub_attribute_info humidity_attr;
- s32 humidity_data;
+ struct {
+ s32 humidity_data;
+ u64 timestamp __aligned(8);
+ } scan;
int scale_pre_decml;
int scale_post_decml;
int scale_precision;
struct hid_humidity_state *humid_st = iio_priv(indio_dev);
if (atomic_read(&humid_st->common_attributes.data_ready))
- iio_push_to_buffers_with_timestamp(indio_dev,
- &humid_st->humidity_data,
- iio_get_time_ns(indio_dev));
+ iio_push_to_buffers_with_timestamp(indio_dev, &humid_st->scan,
+ iio_get_time_ns(indio_dev));
return 0;
}
switch (usage_id) {
case HID_USAGE_SENSOR_ATMOSPHERIC_HUMIDITY:
- humid_st->humidity_data = *(s32 *)raw_data;
+ humid_st->scan.humidity_data = *(s32 *)raw_data;
return 0;
default:
if (ret)
goto err_ret;
- ret = sscanf(indio_dev->name, "adis%u\n", &device_id);
- if (ret != 1) {
+ if (sscanf(indio_dev->name, "adis%u\n", &device_id) != 1) {
ret = -EINVAL;
goto err_ret;
}
struct hid_sensor_common common_attributes;
struct hid_sensor_hub_attribute_info prox_attr;
u32 human_presence;
+ int scale_pre_decml;
+ int scale_post_decml;
+ int scale_precision;
};
/* Channel definitions */
ret_type = IIO_VAL_INT;
break;
case IIO_CHAN_INFO_SCALE:
- *val = prox_state->prox_attr.units;
- ret_type = IIO_VAL_INT;
+ *val = prox_state->scale_pre_decml;
+ *val2 = prox_state->scale_post_decml;
+ ret_type = prox_state->scale_precision;
break;
case IIO_CHAN_INFO_OFFSET:
*val = hid_sensor_convert_exponent(
HID_USAGE_SENSOR_HUMAN_PRESENCE,
&st->common_attributes.sensitivity);
+ st->scale_precision = hid_sensor_format_scale(
+ hsdev->usage,
+ &st->prox_attr,
+ &st->scale_pre_decml, &st->scale_post_decml);
+
return ret;
}
struct temperature_state {
struct hid_sensor_common common_attributes;
struct hid_sensor_hub_attribute_info temperature_attr;
- s32 temperature_data;
+ struct {
+ s32 temperature_data;
+ u64 timestamp __aligned(8);
+ } scan;
int scale_pre_decml;
int scale_post_decml;
int scale_precision;
BIT(IIO_CHAN_INFO_SAMP_FREQ) |
BIT(IIO_CHAN_INFO_HYSTERESIS),
},
- IIO_CHAN_SOFT_TIMESTAMP(3),
+ IIO_CHAN_SOFT_TIMESTAMP(1),
};
/* Adjust channel real bits based on report descriptor */
struct temperature_state *temp_st = iio_priv(indio_dev);
if (atomic_read(&temp_st->common_attributes.data_ready))
- iio_push_to_buffers_with_timestamp(indio_dev,
- &temp_st->temperature_data,
- iio_get_time_ns(indio_dev));
+ iio_push_to_buffers_with_timestamp(indio_dev, &temp_st->scan,
+ iio_get_time_ns(indio_dev));
return 0;
}
switch (usage_id) {
case HID_USAGE_SENSOR_DATA_ENVIRONMENTAL_TEMPERATURE:
- temp_st->temperature_data = *(s32 *)raw_data;
+ temp_st->scan.temperature_data = *(s32 *)raw_data;
return 0;
default:
return -EINVAL;
static const struct nla_policy ib_nl_addr_policy[LS_NLA_TYPE_MAX] = {
[LS_NLA_TYPE_DGID] = {.type = NLA_BINARY,
- .len = sizeof(struct rdma_nla_ls_gid)},
+ .len = sizeof(struct rdma_nla_ls_gid),
+ .validation_type = NLA_VALIDATE_MIN,
+ .min = sizeof(struct rdma_nla_ls_gid)},
};
static inline bool ib_nl_is_good_ip_resp(const struct nlmsghdr *nlh)
ep->com.local_addr.ss_family == AF_INET) {
err = cxgb4_remove_server_filter(
ep->com.dev->rdev.lldi.ports[0], ep->stid,
- ep->com.dev->rdev.lldi.rxq_ids[0], 0);
+ ep->com.dev->rdev.lldi.rxq_ids[0], false);
} else {
struct sockaddr_in6 *sin6;
c4iw_init_wr_wait(ep->com.wr_waitp);
err = cxgb4_remove_server(
ep->com.dev->rdev.lldi.ports[0], ep->stid,
- ep->com.dev->rdev.lldi.rxq_ids[0], 0);
+ ep->com.dev->rdev.lldi.rxq_ids[0],
+ ep->com.local_addr.ss_family == AF_INET6);
if (err)
goto done;
err = c4iw_wait_for_reply(&ep->com.dev->rdev, ep->com.wr_waitp,
*/
int hfi1_dev_affinity_init(struct hfi1_devdata *dd)
{
- int node = pcibus_to_node(dd->pcidev->bus);
struct hfi1_affinity_node *entry;
const struct cpumask *local_mask;
int curr_cpu, possible, i, ret;
bool new_entry = false;
- /*
- * If the BIOS does not have the NUMA node information set, select
- * NUMA 0 so we get consistent performance.
- */
- if (node < 0) {
- dd_dev_err(dd, "Invalid PCI NUMA node. Performance may be affected\n");
- node = 0;
- }
- dd->node = node;
-
local_mask = cpumask_of_node(dd->node);
if (cpumask_first(local_mask) >= nr_cpu_ids)
local_mask = topology_core_cpumask(0);
* create an entry in the global affinity structure and initialize it.
*/
if (!entry) {
- entry = node_affinity_allocate(node);
+ entry = node_affinity_allocate(dd->node);
if (!entry) {
dd_dev_err(dd,
"Unable to allocate global affinity node\n");
if (new_entry)
node_affinity_add_tail(entry);
+ dd->affinity_entry = entry;
mutex_unlock(&node_affinity.lock);
return 0;
{
struct hfi1_affinity_node *entry;
- if (dd->node < 0)
- return;
-
mutex_lock(&node_affinity.lock);
+ if (!dd->affinity_entry)
+ goto unlock;
entry = node_affinity_lookup(dd->node);
if (!entry)
goto unlock;
*/
_dev_comp_vect_cpu_mask_clean_up(dd, entry);
unlock:
+ dd->affinity_entry = NULL;
mutex_unlock(&node_affinity.lock);
- dd->node = NUMA_NO_NODE;
}
/*
spinlock_t irq_src_lock;
int vnic_num_vports;
struct net_device *dummy_netdev;
+ struct hfi1_affinity_node *affinity_entry;
/* Keeps track of IPoIB RSM rule users */
atomic_t ipoib_rsm_usr_num;
dd->pport = (struct hfi1_pportdata *)(dd + 1);
dd->pcidev = pdev;
pci_set_drvdata(pdev, dd);
- dd->node = NUMA_NO_NODE;
ret = xa_alloc_irq(&hfi1_dev_table, &dd->unit, dd, xa_limit_32b,
GFP_KERNEL);
goto bail;
}
rvt_set_ibdev_name(&dd->verbs_dev.rdi, "%s_%d", class_name(), dd->unit);
+ /*
+ * If the BIOS does not have the NUMA node information set, select
+ * NUMA 0 so we get consistent performance.
+ */
+ dd->node = pcibus_to_node(pdev->bus);
+ if (dd->node == NUMA_NO_NODE) {
+ dd_dev_err(dd, "Invalid PCI NUMA node. Performance may be affected\n");
+ dd->node = 0;
+ }
/*
* Initialize all locks for the device. This needs to be as early as
return 0;
}
- cpumask_and(node_cpu_mask, cpu_mask,
- cpumask_of_node(pcibus_to_node(dd->pcidev->bus)));
+ cpumask_and(node_cpu_mask, cpu_mask, cpumask_of_node(dd->node));
available_cpus = cpumask_weight(node_cpu_mask);
upper_32_bits(dma));
roce_write(hr_dev, ROCEE_TX_CMQ_DEPTH_REG,
(u32)ring->desc_num >> HNS_ROCE_CMQ_DESC_NUM_S);
- roce_write(hr_dev, ROCEE_TX_CMQ_HEAD_REG, 0);
+
+ /* Make sure to write tail first and then head */
roce_write(hr_dev, ROCEE_TX_CMQ_TAIL_REG, 0);
+ roce_write(hr_dev, ROCEE_TX_CMQ_HEAD_REG, 0);
} else {
roce_write(hr_dev, ROCEE_RX_CMQ_BASEADDR_L_REG, (u32)dma);
roce_write(hr_dev, ROCEE_RX_CMQ_BASEADDR_H_REG,
case MLX5_CMD_OP_CREATE_MKEY:
MLX5_SET(destroy_mkey_in, din, opcode,
MLX5_CMD_OP_DESTROY_MKEY);
- MLX5_SET(destroy_mkey_in, in, mkey_index, *obj_id);
+ MLX5_SET(destroy_mkey_in, din, mkey_index, *obj_id);
break;
case MLX5_CMD_OP_CREATE_CQ:
MLX5_SET(destroy_cq_in, din, opcode, MLX5_CMD_OP_DESTROY_CQ);
qpc = MLX5_ADDR_OF(create_qp_in, *in, qpc);
MLX5_SET(qpc, qpc, uar_page, uar_index);
- MLX5_SET(qpc, qpc, ts_format, MLX5_QPC_TIMESTAMP_FORMAT_DEFAULT);
+ MLX5_SET(qpc, qpc, ts_format, mlx5_get_qp_default_ts(dev->mdev));
MLX5_SET(qpc, qpc, log_page_size, qp->buf.page_shift - MLX5_ADAPTER_PAGE_SHIFT);
/* Set "fast registration enabled" for all kernel QPs */
}
return MLX5_RQC_TIMESTAMP_FORMAT_FREE_RUNNING;
}
- return MLX5_RQC_TIMESTAMP_FORMAT_DEFAULT;
+ return fr_supported ? MLX5_RQC_TIMESTAMP_FORMAT_FREE_RUNNING :
+ MLX5_RQC_TIMESTAMP_FORMAT_DEFAULT;
}
static int get_sq_ts_format(struct mlx5_ib_dev *dev, struct mlx5_ib_cq *send_cq)
}
return MLX5_SQC_TIMESTAMP_FORMAT_FREE_RUNNING;
}
- return MLX5_SQC_TIMESTAMP_FORMAT_DEFAULT;
+ return fr_supported ? MLX5_SQC_TIMESTAMP_FORMAT_FREE_RUNNING :
+ MLX5_SQC_TIMESTAMP_FORMAT_DEFAULT;
}
static int get_qp_ts_format(struct mlx5_ib_dev *dev, struct mlx5_ib_cq *send_cq,
MLX5_QP_TIMESTAMP_FORMAT_CAP_FREE_RUNNING ||
MLX5_CAP_ROCE(dev->mdev, qp_ts_format) ==
MLX5_QP_TIMESTAMP_FORMAT_CAP_FREE_RUNNING_AND_REAL_TIME;
- int ts_format = MLX5_QPC_TIMESTAMP_FORMAT_DEFAULT;
+ int ts_format = fr_supported ? MLX5_QPC_TIMESTAMP_FORMAT_FREE_RUNNING :
+ MLX5_QPC_TIMESTAMP_FORMAT_DEFAULT;
if (recv_cq &&
recv_cq->create_flags & IB_UVERBS_CQ_FLAGS_TIMESTAMP_COMPLETION)
if (qp->flags & IB_QP_CREATE_MANAGED_RECV)
MLX5_SET(qpc, qpc, cd_slave_receive, 1);
+ MLX5_SET(qpc, qpc, ts_format, mlx5_get_qp_default_ts(dev->mdev));
MLX5_SET(qpc, qpc, rq_type, MLX5_SRQ_RQ);
MLX5_SET(qpc, qpc, no_sq, 1);
MLX5_SET(qpc, qpc, cqn_rcv, to_mcq(devr->c0)->mcq.cqn);
struct mlx5_ib_dev *dev;
int has_net_offloads;
__be64 *rq_pas0;
+ int ts_format;
void *in;
void *rqc;
void *wq;
dev = to_mdev(pd->device);
+ ts_format = get_rq_ts_format(dev, to_mcq(init_attr->cq));
+ if (ts_format < 0)
+ return ts_format;
+
inlen = MLX5_ST_SZ_BYTES(create_rq_in) + sizeof(u64) * rwq->rq_num_pas;
in = kvzalloc(inlen, GFP_KERNEL);
if (!in)
rqc = MLX5_ADDR_OF(create_rq_in, in, ctx);
MLX5_SET(rqc, rqc, mem_rq_type,
MLX5_RQC_MEM_RQ_TYPE_MEMORY_RQ_INLINE);
+ MLX5_SET(rqc, rqc, ts_format, ts_format);
MLX5_SET(rqc, rqc, user_index, rwq->user_index);
MLX5_SET(rqc, rqc, cqn, to_mcq(init_attr->cq)->mcq.cqn);
MLX5_SET(rqc, rqc, state, MLX5_RQC_STATE_RST);
* TGT QP isn't associated with RQ/SQ
*/
if ((attrs->qp_type != IB_QPT_GSI) && (dev->gsi_qp_created) &&
- (attrs->qp_type != IB_QPT_XRC_TGT)) {
+ (attrs->qp_type != IB_QPT_XRC_TGT) &&
+ (attrs->qp_type != IB_QPT_XRC_INI)) {
struct qedr_cq *send_cq = get_qedr_cq(attrs->send_cq);
struct qedr_cq *recv_cq = get_qedr_cq(attrs->recv_cq);
/* Now it is safe to iterate over all paths without locks */
list_for_each_entry_safe(sess, tmp, &clt->paths_list, s.entry) {
- rtrs_clt_destroy_sess_files(sess, NULL);
rtrs_clt_close_conns(sess, true);
+ rtrs_clt_destroy_sess_files(sess, NULL);
kobject_put(&sess->kobj);
}
free_clt(clt);
MODULE_AUTHOR("Vojtech Pavlik <vojtech@ucw.cz>");
MODULE_DESCRIPTION("Joystick device interfaces");
-MODULE_SUPPORTED_DEVICE("input/js");
MODULE_LICENSE("GPL");
#define JOYDEV_MINOR_BASE 0
EXPORT_SYMBOL_GPL(icc_bulk_put);
/**
- * icc_bulk_set() - set bandwidth to a set of paths
+ * icc_bulk_set_bw() - set bandwidth to a set of paths
* @num_paths: the number of icc_bulk_data
* @paths: the icc_bulk_data table containing the paths and bandwidth
*
GFP_KERNEL);
if (new)
src->links = new;
+ else
+ ret = -ENOMEM;
out:
mutex_unlock(&icc_lock);
DEFINE_QNODE(mas_pcnoc_sdcc_2, MSM8939_MASTER_SDCC_2, 8, -1, -1, MSM8939_PNOC_INT_1);
DEFINE_QNODE(mas_qdss_bam, MSM8939_MASTER_QDSS_BAM, 8, -1, -1, MSM8939_SNOC_QDSS_INT);
DEFINE_QNODE(mas_qdss_etr, MSM8939_MASTER_QDSS_ETR, 8, -1, -1, MSM8939_SNOC_QDSS_INT);
-DEFINE_QNODE(mas_snoc_cfg, MSM8939_MASTER_SNOC_CFG, 4, 20, -1, MSM8939_SLAVE_SRVC_SNOC);
+DEFINE_QNODE(mas_snoc_cfg, MSM8939_MASTER_SNOC_CFG, 4, -1, -1, MSM8939_SLAVE_SRVC_SNOC);
DEFINE_QNODE(mas_spdm, MSM8939_MASTER_SPDM, 4, -1, -1, MSM8939_PNOC_MAS_0);
DEFINE_QNODE(mas_tcu0, MSM8939_MASTER_TCU0, 16, -1, -1, MSM8939_SLAVE_EBI_CH0, MSM8939_BIMC_SNOC_MAS, MSM8939_SLAVE_AMPSS_L2);
DEFINE_QNODE(mas_usb_hs1, MSM8939_MASTER_USB_HS1, 4, -1, -1, MSM8939_PNOC_MAS_1);
DEFINE_QNODE(pcnoc_snoc_slv, MSM8939_PNOC_SNOC_SLV, 8, -1, 45, MSM8939_SNOC_INT_0, MSM8939_SNOC_INT_BIMC, MSM8939_SNOC_INT_1);
DEFINE_QNODE(qdss_int, MSM8939_SNOC_QDSS_INT, 8, -1, -1, MSM8939_SNOC_INT_0, MSM8939_SNOC_INT_BIMC);
DEFINE_QNODE(slv_apps_l2, MSM8939_SLAVE_AMPSS_L2, 16, -1, -1, 0);
-DEFINE_QNODE(slv_apss, MSM8939_SLAVE_APSS, 4, -1, 20, 0);
+DEFINE_QNODE(slv_apss, MSM8939_SLAVE_APSS, 4, -1, -1, 0);
DEFINE_QNODE(slv_audio, MSM8939_SLAVE_LPASS, 4, -1, -1, 0);
DEFINE_QNODE(slv_bimc_cfg, MSM8939_SLAVE_BIMC_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_blsp_1, MSM8939_SLAVE_BLSP_1, 4, -1, -1, 0);
DEFINE_QNODE(slv_boot_rom, MSM8939_SLAVE_BOOT_ROM, 4, -1, -1, 0);
DEFINE_QNODE(slv_camera_cfg, MSM8939_SLAVE_CAMERA_CFG, 4, -1, -1, 0);
-DEFINE_QNODE(slv_cats_0, MSM8939_SLAVE_CATS_128, 16, -1, 106, 0);
-DEFINE_QNODE(slv_cats_1, MSM8939_SLAVE_OCMEM_64, 8, -1, 107, 0);
+DEFINE_QNODE(slv_cats_0, MSM8939_SLAVE_CATS_128, 16, -1, -1, 0);
+DEFINE_QNODE(slv_cats_1, MSM8939_SLAVE_OCMEM_64, 8, -1, -1, 0);
DEFINE_QNODE(slv_clk_ctl, MSM8939_SLAVE_CLK_CTL, 4, -1, -1, 0);
DEFINE_QNODE(slv_crypto_0_cfg, MSM8939_SLAVE_CRYPTO_0_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_dehr_cfg, MSM8939_SLAVE_DEHR_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_security, MSM8939_SLAVE_SECURITY, 4, -1, -1, 0);
DEFINE_QNODE(slv_snoc_cfg, MSM8939_SLAVE_SNOC_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_spdm, MSM8939_SLAVE_SPDM, 4, -1, -1, 0);
-DEFINE_QNODE(slv_srvc_snoc, MSM8939_SLAVE_SRVC_SNOC, 8, -1, 29, 0);
+DEFINE_QNODE(slv_srvc_snoc, MSM8939_SLAVE_SRVC_SNOC, 8, -1, -1, 0);
DEFINE_QNODE(slv_tcsr, MSM8939_SLAVE_TCSR, 4, -1, -1, 0);
DEFINE_QNODE(slv_tlmm, MSM8939_SLAVE_TLMM, 4, -1, -1, 0);
DEFINE_QNODE(slv_usb_hs1, MSM8939_SLAVE_USB_HS1, 4, -1, -1, 0);
DEFINE_QNODE(slv_usb_hs2, MSM8939_SLAVE_USB_HS2, 4, -1, -1, 0);
DEFINE_QNODE(slv_venus_cfg, MSM8939_SLAVE_VENUS_CFG, 4, -1, -1, 0);
-DEFINE_QNODE(snoc_bimc_0_mas, MSM8939_SNOC_BIMC_0_MAS, 16, 3, -1, MSM8939_SNOC_BIMC_0_SLV);
-DEFINE_QNODE(snoc_bimc_0_slv, MSM8939_SNOC_BIMC_0_SLV, 16, -1, 24, MSM8939_SLAVE_EBI_CH0);
+DEFINE_QNODE(snoc_bimc_0_mas, MSM8939_SNOC_BIMC_0_MAS, 16, -1, -1, MSM8939_SNOC_BIMC_0_SLV);
+DEFINE_QNODE(snoc_bimc_0_slv, MSM8939_SNOC_BIMC_0_SLV, 16, -1, -1, MSM8939_SLAVE_EBI_CH0);
DEFINE_QNODE(snoc_bimc_1_mas, MSM8939_SNOC_BIMC_1_MAS, 16, 76, -1, MSM8939_SNOC_BIMC_1_SLV);
DEFINE_QNODE(snoc_bimc_1_slv, MSM8939_SNOC_BIMC_1_SLV, 16, -1, 104, MSM8939_SLAVE_EBI_CH0);
DEFINE_QNODE(snoc_bimc_2_mas, MSM8939_SNOC_BIMC_2_MAS, 16, -1, -1, MSM8939_SNOC_BIMC_2_SLV);
DEFINE_QNODE(snoc_bimc_2_slv, MSM8939_SNOC_BIMC_2_SLV, 16, -1, -1, MSM8939_SLAVE_EBI_CH0);
DEFINE_QNODE(snoc_int_0, MSM8939_SNOC_INT_0, 8, 99, 130, MSM8939_SLAVE_QDSS_STM, MSM8939_SLAVE_IMEM, MSM8939_SNOC_PNOC_MAS);
-DEFINE_QNODE(snoc_int_1, MSM8939_SNOC_INT_1, 8, 100, 131, MSM8939_SLAVE_APSS, MSM8939_SLAVE_CATS_128, MSM8939_SLAVE_OCMEM_64);
+DEFINE_QNODE(snoc_int_1, MSM8939_SNOC_INT_1, 8, -1, -1, MSM8939_SLAVE_APSS, MSM8939_SLAVE_CATS_128, MSM8939_SLAVE_OCMEM_64);
DEFINE_QNODE(snoc_int_bimc, MSM8939_SNOC_INT_BIMC, 8, 101, 132, MSM8939_SNOC_BIMC_1_MAS);
DEFINE_QNODE(snoc_pcnoc_mas, MSM8939_SNOC_PNOC_MAS, 8, -1, -1, MSM8939_SNOC_PNOC_SLV);
DEFINE_QNODE(snoc_pcnoc_slv, MSM8939_SNOC_PNOC_SLV, 8, -1, -1, MSM8939_PNOC_INT_0);
struct acpi_table_header *ivrs_base;
int i, remap_cache_sz, ret;
acpi_status status;
- u32 pci_id;
if (!amd_iommu_detected)
return -ENODEV;
if (ret)
goto out;
- /* Disable IOMMU if there's Stoney Ridge graphics */
- for (i = 0; i < 32; i++) {
- pci_id = read_pci_config(0, i, 0, 0);
- if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) {
- pr_info("Disable IOMMU on Stoney Ridge\n");
- amd_iommu_disabled = true;
- break;
- }
- }
-
/* Disable any previously enabled IOMMUs */
if (!is_kdump_kernel() || amd_iommu_disabled)
disable_iommus();
{
struct acpi_table_header *ivrs_base;
acpi_status status;
+ int i;
status = acpi_get_table("IVRS", 0, &ivrs_base);
if (status == AE_NOT_FOUND)
acpi_put_table(ivrs_base);
+ /* Don't use IOMMU if there is Stoney Ridge graphics */
+ for (i = 0; i < 32; i++) {
+ u32 pci_id;
+
+ pci_id = read_pci_config(0, i, 0, 0);
+ if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) {
+ pr_info("Disable IOMMU on Stoney Ridge\n");
+ return false;
+ }
+ }
+
/* Make sure ACS will be enabled during PCI probe */
pci_request_acs();
}
break;
case IOMMU_IVRS_DETECTED:
- ret = early_amd_iommu_init();
- init_state = ret ? IOMMU_INIT_ERROR : IOMMU_ACPI_FINISHED;
- if (init_state == IOMMU_ACPI_FINISHED && amd_iommu_disabled) {
- pr_info("AMD IOMMU disabled\n");
+ if (amd_iommu_disabled) {
init_state = IOMMU_CMDLINE_DISABLED;
ret = -EINVAL;
+ } else {
+ ret = early_amd_iommu_init();
+ init_state = ret ? IOMMU_INIT_ERROR : IOMMU_ACPI_FINISHED;
}
break;
case IOMMU_ACPI_FINISHED:
amd_iommu_irq_remap = true;
ret = iommu_go_to_state(IOMMU_ACPI_FINISHED);
- if (ret)
+ if (ret) {
+ amd_iommu_irq_remap = false;
return ret;
+ }
+
return amd_iommu_irq_remap ? 0 : -ENODEV;
}
smmu = tegra_smmu_find(args.np);
if (smmu) {
err = tegra_smmu_configure(smmu, dev, &args);
- of_node_put(args.np);
- if (err < 0)
+ if (err < 0) {
+ of_node_put(args.np);
return ERR_PTR(err);
-
- break;
+ }
}
of_node_put(args.np);
* Return value: CAPI result code
*/
-u16 capi20_get_manufacturer(u32 contr, u8 *buf)
+u16 capi20_get_manufacturer(u32 contr, u8 buf[CAPI_MANUFACTURER_LEN])
{
struct capi_ctr *ctr;
u16 ret;
* Return value: CAPI result code
*/
-u16 capi20_get_serial(u32 contr, u8 *serial)
+u16 capi20_get_serial(u32 contr, u8 serial[CAPI_SERIAL_LEN])
{
struct capi_ctr *ctr;
u16 ret;
{
if (isac->type & IPAC_TYPE_ISACX)
WriteISAC(isac, ISACX_MASK, 0xff);
- else
+ else if (isac->type != 0)
WriteISAC(isac, ISAC_MASK, 0xff);
if (isac->dch.timer.function != NULL) {
del_timer(&isac->dch.timer);
/*
* CZ.NIC's Turris Omnia LEDs driver
*
- * 2020 by Marek Behun <marek.behun@nic.cz>
+ * 2020 by Marek Behún <kabel@kernel.org>
*/
#include <linux/i2c.h>
module_i2c_driver(omnia_leds_driver);
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
MODULE_DESCRIPTION("CZ.NIC's Turris Omnia LEDs");
MODULE_LICENSE("GPL v2");
/*
* rWTM BIU Mailbox driver for Armada 37xx
*
- * Author: Marek Behun <marek.behun@nic.cz>
+ * Author: Marek Behún <kabel@kernel.org>
*/
#include <linux/device.h>
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("rWTM BIU Mailbox driver for Armada 37xx");
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
* Grab our output buffer.
*/
nl = orig_nl = get_result_buffer(param, param_size, &len);
- if (len < needed) {
+ if (len < needed || len < sizeof(nl->dev)) {
param->flags |= DM_BUFFER_FULL_FLAG;
goto out;
}
return blk_queue_zoned_model(q) != *zoned_model;
}
+/*
+ * Check the device zoned model based on the target feature flag. If the target
+ * has the DM_TARGET_ZONED_HM feature flag set, host-managed zoned devices are
+ * also accepted but all devices must have the same zoned model. If the target
+ * has the DM_TARGET_MIXED_ZONED_MODEL feature set, the devices can have any
+ * zoned model with all zoned devices having the same zone size.
+ */
static bool dm_table_supports_zoned_model(struct dm_table *t,
enum blk_zoned_model zoned_model)
{
for (i = 0; i < dm_table_get_num_targets(t); i++) {
ti = dm_table_get_target(t, i);
- if (zoned_model == BLK_ZONED_HM &&
- !dm_target_supports_zoned_hm(ti->type))
- return false;
-
- if (!ti->type->iterate_devices ||
- ti->type->iterate_devices(ti, device_not_zoned_model, &zoned_model))
- return false;
+ if (dm_target_supports_zoned_hm(ti->type)) {
+ if (!ti->type->iterate_devices ||
+ ti->type->iterate_devices(ti, device_not_zoned_model,
+ &zoned_model))
+ return false;
+ } else if (!dm_target_supports_mixed_zoned_model(ti->type)) {
+ if (zoned_model == BLK_ZONED_HM)
+ return false;
+ }
}
return true;
struct request_queue *q = bdev_get_queue(dev->bdev);
unsigned int *zone_sectors = data;
+ if (!blk_queue_is_zoned(q))
+ return 0;
+
return blk_queue_zone_sectors(q) != *zone_sectors;
}
+/*
+ * Check consistency of zoned model and zone sectors across all targets. For
+ * zone sectors, if the destination device is a zoned block device, it shall
+ * have the specified zone_sectors.
+ */
static int validate_hardware_zoned_model(struct dm_table *table,
enum blk_zoned_model zoned_model,
unsigned int zone_sectors)
return -EINVAL;
if (dm_table_any_dev_attr(table, device_not_matches_zone_sectors, &zone_sectors)) {
- DMERR("%s: zone sectors is not consistent across all devices",
+ DMERR("%s: zone sectors is not consistent across all zoned devices",
dm_device_name(table->md));
return -EINVAL;
}
#define DM_VERITY_OPT_IGN_ZEROES "ignore_zero_blocks"
#define DM_VERITY_OPT_AT_MOST_ONCE "check_at_most_once"
-#define DM_VERITY_OPTS_MAX (2 + DM_VERITY_OPTS_FEC + \
+#define DM_VERITY_OPTS_MAX (3 + DM_VERITY_OPTS_FEC + \
DM_VERITY_ROOT_HASH_VERIFICATION_OPTS)
static unsigned dm_verity_prefetch_cluster = DM_VERITY_DEFAULT_PREFETCH_SIZE;
static struct target_type dmz_type = {
.name = "zoned",
.version = {2, 0, 0},
- .features = DM_TARGET_SINGLETON | DM_TARGET_ZONED_HM,
+ .features = DM_TARGET_SINGLETON | DM_TARGET_MIXED_ZONED_MODEL,
.module = THIS_MODULE,
.ctr = dmz_ctr,
.dtr = dmz_dtr,
if (size != dm_get_size(md))
memset(&md->geometry, 0, sizeof(md->geometry));
- set_capacity_and_notify(md->disk, size);
+ if (!get_capacity(md->disk))
+ set_capacity(md->disk, size);
+ else
+ set_capacity_and_notify(md->disk, size);
dm_table_event_callback(t, event_callback, md);
MODULE_AUTHOR("Ben Backx <ben@bbackx.com>");
MODULE_DESCRIPTION("FireDTV DVB Driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("FireDTV DVB");
MODULE_AUTHOR("Andy Walls");
MODULE_DESCRIPTION("CX23418 ALSA Interface");
-MODULE_SUPPORTED_DEVICE("CX23418 MPEG2 encoder");
MODULE_LICENSE("GPL");
MODULE_VERSION(CX18_VERSION);
MODULE_AUTHOR("Hans Verkuil");
MODULE_DESCRIPTION("CX23418 driver");
-MODULE_SUPPORTED_DEVICE("CX23418 MPEG2 encoder");
MODULE_LICENSE("GPL");
MODULE_VERSION(CX18_VERSION);
MODULE_DESCRIPTION("ALSA driver module for cx25821 based capture cards");
MODULE_AUTHOR("Hiep Huynh");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Conexant,25821}"); /* "{{Conexant,23881}," */
static unsigned int debug;
module_param(debug, int, 0644);
MODULE_LICENSE("GPL v2");
MODULE_VERSION(CX88_VERSION);
-MODULE_SUPPORTED_DEVICE("{{Conexant,23881},{{Conexant,23882},{{Conexant,23883}");
static unsigned int debug;
module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, "enable debug messages");
MODULE_AUTHOR("Andy Walls");
MODULE_DESCRIPTION("CX23415/CX23416 ALSA Interface");
-MODULE_SUPPORTED_DEVICE("CX23415/CX23416 MPEG2 encoder");
MODULE_LICENSE("GPL");
MODULE_VERSION(IVTV_VERSION);
MODULE_AUTHOR("Kevin Thayer, Chris Kennedy, Hans Verkuil");
MODULE_DESCRIPTION("CX23415/CX23416 driver");
-MODULE_SUPPORTED_DEVICE
- ("CX23415/CX23416 MPEG2 encoder (WinTV PVR-150/250/350/500,\n"
- "\t\t\tYuan MPG series and similar)");
MODULE_LICENSE("GPL");
MODULE_VERSION(IVTV_VERSION);
MODULE_DESCRIPTION("STA2X11 Video Input Port driver");
MODULE_AUTHOR("Wind River");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("sta2x11 video input");
MODULE_VERSION(DRV_VERSION);
MODULE_DEVICE_TABLE(pci, sta2x11_vip_pci_tbl);
MODULE_AUTHOR("Josh Wu <josh.wu@atmel.com>");
MODULE_DESCRIPTION("The V4L2 driver for Atmel Linux");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("video");
MODULE_AUTHOR("Songjun Wu");
MODULE_DESCRIPTION("The V4L2 driver for Atmel-ISC");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("video");
MODULE_AUTHOR("Jonathan Corbet <corbet@lwn.net>");
MODULE_DESCRIPTION("Marvell 88ALP01 CMOS Camera Controller driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("Video");
-
-
-
struct cafe_camera {
int registered; /* Fully initialized? */
MODULE_AUTHOR("Hugues Fruchet <hugues.fruchet@st.com>");
MODULE_DESCRIPTION("STMicroelectronics STM32 Digital Camera Memory Interface driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("video");
MODULE_AUTHOR("Steve Miller (STMicroelectronics) <steve.miller@st.com>");
MODULE_DESCRIPTION("V4L-driver for STMicroelectronics CPiA2 based cameras");
-MODULE_SUPPORTED_DEVICE("video");
MODULE_LICENSE("GPL");
MODULE_VERSION(CPIA_VERSION);
MODULE_DESCRIPTION("ALSA driver module for tm5600/tm6000/tm6010 based TV cards");
MODULE_AUTHOR("Mauro Carvalho Chehab");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{Trident,tm5600},{{Trident,tm6000},{{Trident,tm6010}");
static unsigned int debug;
module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, "enable debug messages");
MODULE_AUTHOR("Mauro Carvalho Chehab");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Trident, tm5600},{{Trident, tm6000},{{Trident, tm6010}");
-
static int debug;
module_param(debug, int, 0644);
{}
};
-static const struct resource intel_quark_i2c_res[] = {
+/* This is used as a place holder and will be modified at run-time */
+static struct resource intel_quark_i2c_res[] = {
[INTEL_QUARK_IORES_MEM] = {
.flags = IORESOURCE_MEM,
},
.adr = MFD_ACPI_MATCH_I2C,
};
-static const struct resource intel_quark_gpio_res[] = {
+/* This is used as a place holder and will be modified at run-time */
+static struct resource intel_quark_gpio_res[] = {
[INTEL_QUARK_IORES_MEM] = {
.flags = IORESOURCE_MEM,
},
if (buffer_id == 0)
return -EINVAL;
- if (!mei_cl_is_connected(cl))
- return -ENODEV;
+ if (mei_cl_is_connected(cl))
+ return -EPROTO;
if (cl->dma_mapped)
return -EPROTO;
mutex_unlock(&dev->device_lock);
wait_event_timeout(cl->wait,
- cl->dma_mapped ||
- cl->status ||
- !mei_cl_is_connected(cl),
+ cl->dma_mapped || cl->status,
mei_secs_to_jiffies(MEI_CL_CONNECT_TIMEOUT));
mutex_lock(&dev->device_lock);
return -EOPNOTSUPP;
}
- if (!mei_cl_is_connected(cl))
- return -ENODEV;
+ /* do not allow unmap for connected client */
+ if (mei_cl_is_connected(cl))
+ return -EPROTO;
if (!cl->dma_mapped)
return -EPROTO;
mutex_unlock(&dev->device_lock);
wait_event_timeout(cl->wait,
- !cl->dma_mapped ||
- cl->status ||
- !mei_cl_is_connected(cl),
+ !cl->dma_mapped || cl->status,
mei_secs_to_jiffies(MEI_CL_CONNECT_TIMEOUT));
mutex_lock(&dev->device_lock);
MODULE_AUTHOR("Eric Brower <ebrower@usa.net>");
MODULE_DESCRIPTION("User-programmable flash device on Sun Microsystems boardsets");
-MODULE_SUPPORTED_DEVICE(DRIVER_NAME);
MODULE_LICENSE("GPL");
MODULE_VERSION("2.1");
int i, ioaddr, ret;
struct resource *r;
+ ret = 0;
+
if (pci_enable_device(pdev))
return -EIO;
priv->ci = ci;
mm = &ci->misc_map;
+ pci_set_drvdata(pdev, priv);
+
INIT_LIST_HEAD(&priv->list_dev);
if (mm->size) {
dev = alloc_arcdev(device);
if (!dev) {
ret = -ENOMEM;
- goto out_port;
+ break;
}
dev->dev_port = i;
pr_err("IO region %xh-%xh already allocated\n",
ioaddr, ioaddr + cm->size - 1);
ret = -EBUSY;
- goto out_port;
+ goto err_free_arcdev;
}
/* Dummy access after Reset
if (arcnet_inb(ioaddr, COM20020_REG_R_STATUS) == 0xFF) {
pr_err("IO address %Xh is empty!\n", ioaddr);
ret = -EIO;
- goto out_port;
+ goto err_free_arcdev;
}
if (com20020_check(dev)) {
ret = -EIO;
- goto out_port;
+ goto err_free_arcdev;
}
card = devm_kzalloc(&pdev->dev, sizeof(struct com20020_dev),
GFP_KERNEL);
if (!card) {
ret = -ENOMEM;
- goto out_port;
+ goto err_free_arcdev;
}
card->index = i;
ret = devm_led_classdev_register(&pdev->dev, &card->tx_led);
if (ret)
- goto out_port;
+ goto err_free_arcdev;
ret = devm_led_classdev_register(&pdev->dev, &card->recon_led);
if (ret)
- goto out_port;
+ goto err_free_arcdev;
dev_set_drvdata(&dev->dev, card);
ret = com20020_found(dev, IRQF_SHARED);
if (ret)
- goto out_port;
+ goto err_free_arcdev;
devm_arcnet_led_init(dev, dev->dev_id, i);
list_add(&card->list, &priv->list_dev);
- }
+ continue;
- pci_set_drvdata(pdev, priv);
-
- return 0;
-
-out_port:
- com20020pci_remove(pdev);
+err_free_arcdev:
+ free_arcdev(dev);
+ break;
+ }
+ if (ret)
+ com20020pci_remove(pdev);
return ret;
}
rcu_read_lock();
slave = bond_first_slave_rcu(bond);
- if (!slave) {
- ret = -EINVAL;
+ if (!slave)
goto out;
- }
slave_ops = slave->dev->netdev_ops;
- if (!slave_ops->ndo_neigh_setup) {
- ret = -EINVAL;
+ if (!slave_ops->ndo_neigh_setup)
goto out;
- }
/* TODO: find another way [1] to implement this.
* Passing a zeroed structure is fragile,
.brp_inc = 1,
};
-static inline void c_can_pm_runtime_enable(const struct c_can_priv *priv)
-{
- if (priv->device)
- pm_runtime_enable(priv->device);
-}
-
-static inline void c_can_pm_runtime_disable(const struct c_can_priv *priv)
-{
- if (priv->device)
- pm_runtime_disable(priv->device);
-}
-
static inline void c_can_pm_runtime_get_sync(const struct c_can_priv *priv)
{
if (priv->device)
int register_c_can_dev(struct net_device *dev)
{
- struct c_can_priv *priv = netdev_priv(dev);
int err;
/* Deactivate pins to prevent DRA7 DCAN IP from being
*/
pinctrl_pm_select_sleep_state(dev->dev.parent);
- c_can_pm_runtime_enable(priv);
-
dev->flags |= IFF_ECHO; /* we support local echo */
dev->netdev_ops = &c_can_netdev_ops;
err = register_candev(dev);
- if (err)
- c_can_pm_runtime_disable(priv);
- else
+ if (!err)
devm_can_led_init(dev);
-
return err;
}
EXPORT_SYMBOL_GPL(register_c_can_dev);
void unregister_c_can_dev(struct net_device *dev)
{
- struct c_can_priv *priv = netdev_priv(dev);
-
unregister_candev(dev);
-
- c_can_pm_runtime_disable(priv);
}
EXPORT_SYMBOL_GPL(unregister_c_can_dev);
{
struct net_device *dev = pci_get_drvdata(pdev);
struct c_can_priv *priv = netdev_priv(dev);
+ void __iomem *addr = priv->base;
unregister_c_can_dev(dev);
free_c_can_dev(dev);
- pci_iounmap(pdev, priv->base);
+ pci_iounmap(pdev, addr);
pci_disable_msi(pdev);
pci_clear_master(pdev);
pci_release_regions(pdev);
#include <linux/list.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
#include <linux/clk.h>
#include <linux/of.h>
#include <linux/of_device.h>
platform_set_drvdata(pdev, dev);
SET_NETDEV_DEV(dev, &pdev->dev);
+ pm_runtime_enable(priv->device);
ret = register_c_can_dev(dev);
if (ret) {
dev_err(&pdev->dev, "registering %s failed (err=%d)\n",
return 0;
exit_free_device:
+ pm_runtime_disable(priv->device);
free_c_can_dev(dev);
exit:
dev_err(&pdev->dev, "probe failed\n");
static int c_can_plat_remove(struct platform_device *pdev)
{
struct net_device *dev = platform_get_drvdata(pdev);
+ struct c_can_priv *priv = netdev_priv(dev);
unregister_c_can_dev(dev);
-
+ pm_runtime_disable(priv->device);
free_c_can_dev(dev);
return 0;
struct rtnl_link_ops can_link_ops __read_mostly = {
.kind = "can",
+ .netns_refund = true,
.maxtype = IFLA_CAN_MAX,
.policy = can_policy,
.setup = can_setup,
static int flexcan_chip_freeze(struct flexcan_priv *priv)
{
struct flexcan_regs __iomem *regs = priv->regs;
- unsigned int timeout = 1000 * 1000 * 10 / priv->can.bittiming.bitrate;
+ unsigned int timeout;
+ u32 bitrate = priv->can.bittiming.bitrate;
u32 reg;
+ if (bitrate)
+ timeout = 1000 * 1000 * 10 / bitrate;
+ else
+ timeout = FLEXCAN_TIMEOUT_US / 10;
+
reg = priv->read(®s->mcr);
reg |= FLEXCAN_MCR_FRZ | FLEXCAN_MCR_HALT;
priv->write(reg, ®s->mcr);
#define KVASER_PCIEFD_KCAN_STAT_REG 0x418
#define KVASER_PCIEFD_KCAN_MODE_REG 0x41c
#define KVASER_PCIEFD_KCAN_BTRN_REG 0x420
+#define KVASER_PCIEFD_KCAN_BUS_LOAD_REG 0x424
#define KVASER_PCIEFD_KCAN_BTRD_REG 0x428
#define KVASER_PCIEFD_KCAN_PWM_REG 0x430
/* Loopback control register */
timer_setup(&can->bec_poll_timer, kvaser_pciefd_bec_poll_timer,
0);
+ /* Disable Bus load reporting */
+ iowrite32(0, can->reg_base + KVASER_PCIEFD_KCAN_BUS_LOAD_REG);
+
tx_npackets = ioread32(can->reg_base +
KVASER_PCIEFD_KCAN_TX_NPACKETS_REG);
if (((tx_npackets >> KVASER_PCIEFD_KCAN_TX_NPACKETS_MAX_SHIFT) &
}
while ((rxfs & RXFS_FFL_MASK) && (quota > 0)) {
- if (rxfs & RXFS_RFL)
- netdev_warn(dev, "Rx FIFO 0 Message Lost\n");
-
m_can_read_fifo(dev, rxfs);
quota--;
{
struct m_can_classdev *cdev = netdev_priv(dev);
- m_can_rx_handler(dev, 1);
+ m_can_rx_handler(dev, M_CAN_NAPI_WEIGHT);
m_can_enable_all_interrupts(cdev);
MODULE_AUTHOR("Stephane Grosjean <s.grosjean@peak-system.com>");
MODULE_DESCRIPTION("Socket-CAN driver for PEAK PCAN PCIe/M.2 FD family cards");
-MODULE_SUPPORTED_DEVICE("PEAK PCAN PCIe/M.2 FD CAN cards");
MODULE_LICENSE("GPL v2");
#define PCIEFD_DRV_NAME "peak_pciefd"
MODULE_AUTHOR("Sebastian Haas <haas@ems-wuenche.com>");
MODULE_DESCRIPTION("Socket-CAN driver for EMS CPC-PCI/PCIe/104P CAN cards");
-MODULE_SUPPORTED_DEVICE("EMS CPC-PCI/PCIe/104P CAN card");
MODULE_LICENSE("GPL v2");
#define EMS_PCI_V1_MAX_CHAN 2
MODULE_AUTHOR("Markus Plessing <plessing@ems-wuensche.com>");
MODULE_DESCRIPTION("Socket-CAN driver for EMS CPC-CARD cards");
-MODULE_SUPPORTED_DEVICE("EMS CPC-CARD CAN card");
MODULE_LICENSE("GPL v2");
#define EMS_PCMCIA_MAX_CHAN 2
MODULE_AUTHOR("Per Dalen <per.dalen@cnw.se>");
MODULE_DESCRIPTION("Socket-CAN driver for KVASER PCAN PCI cards");
-MODULE_SUPPORTED_DEVICE("KVASER PCAN PCI CAN card");
MODULE_LICENSE("GPL v2");
#define MAX_NO_OF_CHANNELS 4 /* max no of channels on a single card */
MODULE_AUTHOR("Stephane Grosjean <s.grosjean@peak-system.com>");
MODULE_DESCRIPTION("Socket-CAN driver for PEAK PCAN PCI family cards");
-MODULE_SUPPORTED_DEVICE("PEAK PCAN PCI/PCIe/PCIeC miniPCI CAN cards");
-MODULE_SUPPORTED_DEVICE("PEAK PCAN miniPCIe/cPCI PC/104+ PCI/104e CAN Cards");
MODULE_LICENSE("GPL v2");
#define DRV_NAME "peak_pci"
MODULE_AUTHOR("Stephane Grosjean <s.grosjean@peak-system.com>");
MODULE_DESCRIPTION("CAN driver for PEAK-System PCAN-PC Cards");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("PEAK PCAN-PC Card");
/* PEAK-System PCMCIA driver name */
#define PCC_NAME "peak_pcmcia"
MODULE_AUTHOR("Pavel Cheblakov <P.B.Cheblakov@inp.nsk.su>");
MODULE_DESCRIPTION("Socket-CAN driver for PLX90xx PCI-bridge cards with "
"the SJA1000 chips");
-MODULE_SUPPORTED_DEVICE("Adlink PCI-7841/cPCI-7841, "
- "Adlink PCI-7841/cPCI-7841 SE, "
- "Marathon CAN-bus-PCI, "
- "Marathon CAN-bus-PCIe, "
- "TEWS TECHNOLOGIES TPMC810, "
- "esd CAN-PCI/CPCI/PCI104/200, "
- "esd CAN-PCI/PMC/266, "
- "esd CAN-PCIe/2000, "
- "Connect Tech Inc. CANpro/104-Plus Opto (CRG001), "
- "IXXAT PC-I 04/PCI, "
- "ELCUS CAN-200-PCI, "
- "ASEM DUAL CAN-RAW")
MODULE_LICENSE("GPL v2");
#define PLX_PCI_MAX_CHAN 2
return ret;
}
+static int mcp251x_spi_write(struct spi_device *spi, int len)
+{
+ struct mcp251x_priv *priv = spi_get_drvdata(spi);
+ int ret;
+
+ ret = spi_write(spi, priv->spi_tx_buf, len);
+ if (ret)
+ dev_err(&spi->dev, "spi write failed: ret = %d\n", ret);
+
+ return ret;
+}
+
static u8 mcp251x_read_reg(struct spi_device *spi, u8 reg)
{
struct mcp251x_priv *priv = spi_get_drvdata(spi);
priv->spi_tx_buf[1] = reg;
priv->spi_tx_buf[2] = val;
- mcp251x_spi_trans(spi, 3);
+ mcp251x_spi_write(spi, 3);
}
static void mcp251x_write_2regs(struct spi_device *spi, u8 reg, u8 v1, u8 v2)
priv->spi_tx_buf[2] = v1;
priv->spi_tx_buf[3] = v2;
- mcp251x_spi_trans(spi, 4);
+ mcp251x_spi_write(spi, 4);
}
static void mcp251x_write_bits(struct spi_device *spi, u8 reg,
priv->spi_tx_buf[2] = mask;
priv->spi_tx_buf[3] = val;
- mcp251x_spi_trans(spi, 4);
+ mcp251x_spi_write(spi, 4);
}
static u8 mcp251x_read_stat(struct spi_device *spi)
buf[i]);
} else {
memcpy(priv->spi_tx_buf, buf, TXBDAT_OFF + len);
- mcp251x_spi_trans(spi, TXBDAT_OFF + len);
+ mcp251x_spi_write(spi, TXBDAT_OFF + len);
}
}
/* use INSTRUCTION_RTS, to avoid "repeated frame problem" */
priv->spi_tx_buf[0] = INSTRUCTION_RTS(1 << tx_buf_idx);
- mcp251x_spi_trans(priv->spi, 1);
+ mcp251x_spi_write(priv->spi, 1);
}
static void mcp251x_hw_rx_frame(struct spi_device *spi, u8 *buf,
mdelay(MCP251X_OST_DELAY_MS);
priv->spi_tx_buf[0] = INSTRUCTION_RESET;
- ret = mcp251x_spi_trans(spi, 1);
+ ret = mcp251x_spi_write(spi, 1);
if (ret)
return ret;
- Kvaser Memorator Pro 5xHS
- Kvaser USBcan Light 4xHS
- Kvaser USBcan Pro 2xHS v2
+ - Kvaser USBcan Pro 4xHS
- Kvaser USBcan Pro 5xHS
- Kvaser U100
- Kvaser U100P
#define USB_U100_PRODUCT_ID 273
#define USB_U100P_PRODUCT_ID 274
#define USB_U100S_PRODUCT_ID 275
+#define USB_USBCAN_PRO_4HS_PRODUCT_ID 276
#define USB_HYDRA_PRODUCT_ID_END \
- USB_U100S_PRODUCT_ID
+ USB_USBCAN_PRO_4HS_PRODUCT_ID
static inline bool kvaser_is_leaf(const struct usb_device_id *id)
{
{ USB_DEVICE(KVASER_VENDOR_ID, USB_U100_PRODUCT_ID) },
{ USB_DEVICE(KVASER_VENDOR_ID, USB_U100P_PRODUCT_ID) },
{ USB_DEVICE(KVASER_VENDOR_ID, USB_U100S_PRODUCT_ID) },
+ { USB_DEVICE(KVASER_VENDOR_ID, USB_USBCAN_PRO_4HS_PRODUCT_ID) },
{ }
};
MODULE_DEVICE_TABLE(usb, kvaser_usb_table);
#include "pcan_usb_core.h"
-MODULE_SUPPORTED_DEVICE("PEAK-System PCAN-USB adapter");
-
/* PCAN-USB Endpoints */
#define PCAN_USB_EP_CMDOUT 1
#define PCAN_USB_EP_CMDIN (PCAN_USB_EP_CMDOUT | USB_DIR_IN)
if (dev->adapter->dev_set_bus) {
err = dev->adapter->dev_set_bus(dev, 0);
if (err)
- goto lbl_unregister_candev;
+ goto adap_dev_free;
}
/* get device number early */
return 0;
+adap_dev_free:
+ if (dev->adapter->dev_free)
+ dev->adapter->dev_free(dev);
+
lbl_unregister_candev:
unregister_candev(netdev);
#include "pcan_usb_core.h"
#include "pcan_usb_pro.h"
-MODULE_SUPPORTED_DEVICE("PEAK-System PCAN-USB FD adapter");
-MODULE_SUPPORTED_DEVICE("PEAK-System PCAN-USB Pro FD adapter");
-
#define PCAN_USBPROFD_CHANNEL_COUNT 2
#define PCAN_USBFD_CHANNEL_COUNT 1
#include "pcan_usb_core.h"
#include "pcan_usb_pro.h"
-MODULE_SUPPORTED_DEVICE("PEAK-System PCAN-USB Pro adapter");
-
#define PCAN_USBPRO_CHANNEL_COUNT 2
/* PCAN-USB Pro adapter internal clock (MHz) */
b53_disable_port(ds, port);
}
- /* Let DSA handle the case were multiple bridges span the same switch
- * device and different VLAN awareness settings are requested, which
- * would be breaking filtering semantics for any of the other bridge
- * devices. (not hardware supported)
- */
- ds->vlan_filtering_is_global = true;
-
return b53_setup_devlink_resources(ds);
}
ds->ops = &b53_switch_ops;
ds->untag_bridge_pvid = true;
dev->vlan_enabled = true;
+ /* Let DSA handle the case were multiple bridges span the same switch
+ * device and different VLAN awareness settings are requested, which
+ * would be breaking filtering semantics for any of the other bridge
+ * devices. (not hardware supported)
+ */
+ ds->vlan_filtering_is_global = true;
+
mutex_init(&dev->reg_mutex);
mutex_init(&dev->stats_mutex);
/* Force link status for IMP port */
reg = core_readl(priv, offset);
reg |= (MII_SW_OR | LINK_STS);
- reg &= ~GMII_SPEED_UP_2G;
+ if (priv->type == BCM4908_DEVICE_ID)
+ reg |= GMII_SPEED_UP_2G;
+ else
+ reg &= ~GMII_SPEED_UP_2G;
core_writel(priv, reg, offset);
/* Enable Broadcast, Multicast, Unicast forwarding to IMP port */
* in bits 15:8 and the patch level in bits 7:0 which is exactly what
* the REG_PHY_REVISION register layout is.
*/
-
- return priv->hw_params.gphy_rev;
+ if (priv->int_phy_mask & BIT(port))
+ return priv->hw_params.gphy_rev;
+ else
+ return 0;
}
static void bcm_sf2_sw_validate(struct dsa_switch *ds, int port,
/* GSWIP MII Registers */
#define GSWIP_MII_CFGp(p) (0x2 * (p))
+#define GSWIP_MII_CFG_RESET BIT(15)
#define GSWIP_MII_CFG_EN BIT(14)
+#define GSWIP_MII_CFG_ISOLATE BIT(13)
#define GSWIP_MII_CFG_LDCLKDIS BIT(12)
+#define GSWIP_MII_CFG_RGMII_IBS BIT(8)
+#define GSWIP_MII_CFG_RMII_CLK BIT(7)
#define GSWIP_MII_CFG_MODE_MIIP 0x0
#define GSWIP_MII_CFG_MODE_MIIM 0x1
#define GSWIP_MII_CFG_MODE_RMIIP 0x2
#define GSWIP_PCE_DEFPVID(p) (0x486 + ((p) * 0xA))
#define GSWIP_MAC_FLEN 0x8C5
+#define GSWIP_MAC_CTRL_0p(p) (0x903 + ((p) * 0xC))
+#define GSWIP_MAC_CTRL_0_PADEN BIT(8)
+#define GSWIP_MAC_CTRL_0_FCS_EN BIT(7)
+#define GSWIP_MAC_CTRL_0_FCON_MASK 0x0070
+#define GSWIP_MAC_CTRL_0_FCON_AUTO 0x0000
+#define GSWIP_MAC_CTRL_0_FCON_RX 0x0010
+#define GSWIP_MAC_CTRL_0_FCON_TX 0x0020
+#define GSWIP_MAC_CTRL_0_FCON_RXTX 0x0030
+#define GSWIP_MAC_CTRL_0_FCON_NONE 0x0040
+#define GSWIP_MAC_CTRL_0_FDUP_MASK 0x000C
+#define GSWIP_MAC_CTRL_0_FDUP_AUTO 0x0000
+#define GSWIP_MAC_CTRL_0_FDUP_EN 0x0004
+#define GSWIP_MAC_CTRL_0_FDUP_DIS 0x000C
+#define GSWIP_MAC_CTRL_0_GMII_MASK 0x0003
+#define GSWIP_MAC_CTRL_0_GMII_AUTO 0x0000
+#define GSWIP_MAC_CTRL_0_GMII_MII 0x0001
+#define GSWIP_MAC_CTRL_0_GMII_RGMII 0x0002
#define GSWIP_MAC_CTRL_2p(p) (0x905 + ((p) * 0xC))
#define GSWIP_MAC_CTRL_2_MLEN BIT(3) /* Maximum Untagged Frame Lnegth */
GSWIP_SDMA_PCTRLp(port));
if (!dsa_is_cpu_port(ds, port)) {
- u32 macconf = GSWIP_MDIO_PHY_LINK_AUTO |
- GSWIP_MDIO_PHY_SPEED_AUTO |
- GSWIP_MDIO_PHY_FDUP_AUTO |
- GSWIP_MDIO_PHY_FCONTX_AUTO |
- GSWIP_MDIO_PHY_FCONRX_AUTO |
- (phydev->mdio.addr & GSWIP_MDIO_PHY_ADDR_MASK);
-
- gswip_mdio_w(priv, macconf, GSWIP_MDIO_PHYp(port));
- /* Activate MDIO auto polling */
- gswip_mdio_mask(priv, 0, BIT(port), GSWIP_MDIO_MDC_CFG0);
+ u32 mdio_phy = 0;
+
+ if (phydev)
+ mdio_phy = phydev->mdio.addr & GSWIP_MDIO_PHY_ADDR_MASK;
+
+ gswip_mdio_mask(priv, GSWIP_MDIO_PHY_ADDR_MASK, mdio_phy,
+ GSWIP_MDIO_PHYp(port));
}
return 0;
if (!dsa_is_user_port(ds, port))
return;
- if (!dsa_is_cpu_port(ds, port)) {
- gswip_mdio_mask(priv, GSWIP_MDIO_PHY_LINK_DOWN,
- GSWIP_MDIO_PHY_LINK_MASK,
- GSWIP_MDIO_PHYp(port));
- /* Deactivate MDIO auto polling */
- gswip_mdio_mask(priv, BIT(port), 0, GSWIP_MDIO_MDC_CFG0);
- }
-
gswip_switch_mask(priv, GSWIP_FDMA_PCTRL_EN, 0,
GSWIP_FDMA_PCTRLp(port));
gswip_switch_mask(priv, GSWIP_SDMA_PCTRL_EN, 0,
gswip_switch_w(priv, BIT(cpu_port), GSWIP_PCE_PMAP2);
gswip_switch_w(priv, BIT(cpu_port), GSWIP_PCE_PMAP3);
- /* disable PHY auto polling */
+ /* Deactivate MDIO PHY auto polling. Some PHYs as the AR8030 have an
+ * interoperability problem with this auto polling mechanism because
+ * their status registers think that the link is in a different state
+ * than it actually is. For the AR8030 it has the BMSR_ESTATEN bit set
+ * as well as ESTATUS_1000_TFULL and ESTATUS_1000_XFULL. This makes the
+ * auto polling state machine consider the link being negotiated with
+ * 1Gbit/s. Since the PHY itself is a Fast Ethernet RMII PHY this leads
+ * to the switch port being completely dead (RX and TX are both not
+ * working).
+ * Also with various other PHY / port combinations (PHY11G GPHY, PHY22F
+ * GPHY, external RGMII PEF7071/7072) any traffic would stop. Sometimes
+ * it would work fine for a few minutes to hours and then stop, on
+ * other device it would no traffic could be sent or received at all.
+ * Testing shows that when PHY auto polling is disabled these problems
+ * go away.
+ */
gswip_mdio_w(priv, 0x0, GSWIP_MDIO_MDC_CFG0);
+
/* Configure the MDIO Clock 2.5 MHz */
gswip_mdio_mask(priv, 0xff, 0x09, GSWIP_MDIO_MDC_CFG1);
- /* Disable the xMII link */
+ /* Disable the xMII interface and clear it's isolation bit */
for (i = 0; i < priv->hw_info->max_ports; i++)
- gswip_mii_mask_cfg(priv, GSWIP_MII_CFG_EN, 0, i);
+ gswip_mii_mask_cfg(priv,
+ GSWIP_MII_CFG_EN | GSWIP_MII_CFG_ISOLATE,
+ 0, i);
/* enable special tag insertion on cpu port */
gswip_switch_mask(priv, 0, GSWIP_FDMA_PCTRL_STEN,
return;
}
+static void gswip_port_set_link(struct gswip_priv *priv, int port, bool link)
+{
+ u32 mdio_phy;
+
+ if (link)
+ mdio_phy = GSWIP_MDIO_PHY_LINK_UP;
+ else
+ mdio_phy = GSWIP_MDIO_PHY_LINK_DOWN;
+
+ gswip_mdio_mask(priv, GSWIP_MDIO_PHY_LINK_MASK, mdio_phy,
+ GSWIP_MDIO_PHYp(port));
+}
+
+static void gswip_port_set_speed(struct gswip_priv *priv, int port, int speed,
+ phy_interface_t interface)
+{
+ u32 mdio_phy = 0, mii_cfg = 0, mac_ctrl_0 = 0;
+
+ switch (speed) {
+ case SPEED_10:
+ mdio_phy = GSWIP_MDIO_PHY_SPEED_M10;
+
+ if (interface == PHY_INTERFACE_MODE_RMII)
+ mii_cfg = GSWIP_MII_CFG_RATE_M50;
+ else
+ mii_cfg = GSWIP_MII_CFG_RATE_M2P5;
+
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_GMII_MII;
+ break;
+
+ case SPEED_100:
+ mdio_phy = GSWIP_MDIO_PHY_SPEED_M100;
+
+ if (interface == PHY_INTERFACE_MODE_RMII)
+ mii_cfg = GSWIP_MII_CFG_RATE_M50;
+ else
+ mii_cfg = GSWIP_MII_CFG_RATE_M25;
+
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_GMII_MII;
+ break;
+
+ case SPEED_1000:
+ mdio_phy = GSWIP_MDIO_PHY_SPEED_G1;
+
+ mii_cfg = GSWIP_MII_CFG_RATE_M125;
+
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_GMII_RGMII;
+ break;
+ }
+
+ gswip_mdio_mask(priv, GSWIP_MDIO_PHY_SPEED_MASK, mdio_phy,
+ GSWIP_MDIO_PHYp(port));
+ gswip_mii_mask_cfg(priv, GSWIP_MII_CFG_RATE_MASK, mii_cfg, port);
+ gswip_switch_mask(priv, GSWIP_MAC_CTRL_0_GMII_MASK, mac_ctrl_0,
+ GSWIP_MAC_CTRL_0p(port));
+}
+
+static void gswip_port_set_duplex(struct gswip_priv *priv, int port, int duplex)
+{
+ u32 mac_ctrl_0, mdio_phy;
+
+ if (duplex == DUPLEX_FULL) {
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_FDUP_EN;
+ mdio_phy = GSWIP_MDIO_PHY_FDUP_EN;
+ } else {
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_FDUP_DIS;
+ mdio_phy = GSWIP_MDIO_PHY_FDUP_DIS;
+ }
+
+ gswip_switch_mask(priv, GSWIP_MAC_CTRL_0_FDUP_MASK, mac_ctrl_0,
+ GSWIP_MAC_CTRL_0p(port));
+ gswip_mdio_mask(priv, GSWIP_MDIO_PHY_FDUP_MASK, mdio_phy,
+ GSWIP_MDIO_PHYp(port));
+}
+
+static void gswip_port_set_pause(struct gswip_priv *priv, int port,
+ bool tx_pause, bool rx_pause)
+{
+ u32 mac_ctrl_0, mdio_phy;
+
+ if (tx_pause && rx_pause) {
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_FCON_RXTX;
+ mdio_phy = GSWIP_MDIO_PHY_FCONTX_EN |
+ GSWIP_MDIO_PHY_FCONRX_EN;
+ } else if (tx_pause) {
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_FCON_TX;
+ mdio_phy = GSWIP_MDIO_PHY_FCONTX_EN |
+ GSWIP_MDIO_PHY_FCONRX_DIS;
+ } else if (rx_pause) {
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_FCON_RX;
+ mdio_phy = GSWIP_MDIO_PHY_FCONTX_DIS |
+ GSWIP_MDIO_PHY_FCONRX_EN;
+ } else {
+ mac_ctrl_0 = GSWIP_MAC_CTRL_0_FCON_NONE;
+ mdio_phy = GSWIP_MDIO_PHY_FCONTX_DIS |
+ GSWIP_MDIO_PHY_FCONRX_DIS;
+ }
+
+ gswip_switch_mask(priv, GSWIP_MAC_CTRL_0_FCON_MASK,
+ mac_ctrl_0, GSWIP_MAC_CTRL_0p(port));
+ gswip_mdio_mask(priv,
+ GSWIP_MDIO_PHY_FCONTX_MASK |
+ GSWIP_MDIO_PHY_FCONRX_MASK,
+ mdio_phy, GSWIP_MDIO_PHYp(port));
+}
+
static void gswip_phylink_mac_config(struct dsa_switch *ds, int port,
unsigned int mode,
const struct phylink_link_state *state)
break;
case PHY_INTERFACE_MODE_RMII:
miicfg |= GSWIP_MII_CFG_MODE_RMIIM;
+
+ /* Configure the RMII clock as output: */
+ miicfg |= GSWIP_MII_CFG_RMII_CLK;
break;
case PHY_INTERFACE_MODE_RGMII:
case PHY_INTERFACE_MODE_RGMII_ID:
"Unsupported interface: %d\n", state->interface);
return;
}
- gswip_mii_mask_cfg(priv, GSWIP_MII_CFG_MODE_MASK, miicfg, port);
+
+ gswip_mii_mask_cfg(priv,
+ GSWIP_MII_CFG_MODE_MASK | GSWIP_MII_CFG_RMII_CLK |
+ GSWIP_MII_CFG_RGMII_IBS | GSWIP_MII_CFG_LDCLKDIS,
+ miicfg, port);
switch (state->interface) {
case PHY_INTERFACE_MODE_RGMII_ID:
struct gswip_priv *priv = ds->priv;
gswip_mii_mask_cfg(priv, GSWIP_MII_CFG_EN, 0, port);
+
+ if (!dsa_is_cpu_port(ds, port))
+ gswip_port_set_link(priv, port, false);
}
static void gswip_phylink_mac_link_up(struct dsa_switch *ds, int port,
{
struct gswip_priv *priv = ds->priv;
+ if (!dsa_is_cpu_port(ds, port)) {
+ gswip_port_set_link(priv, port, true);
+ gswip_port_set_speed(priv, port, speed, interface);
+ gswip_port_set_duplex(priv, port, duplex);
+ gswip_port_set_pause(priv, port, tx_pause, rx_pause);
+ }
+
gswip_mii_mask_cfg(priv, 0, GSWIP_MII_CFG_EN, port);
}
TD_DM_DRVP(8) | TD_DM_DRVN(8));
/* Setup core clock for MT7530 */
- if (!trgint) {
- /* Disable MT7530 core clock */
- core_clear(priv, CORE_TRGMII_GSW_CLK_CG, REG_GSWCK_EN);
-
- /* Disable PLL, since phy_device has not yet been created
- * provided for phy_[read,write]_mmd_indirect is called, we
- * provide our own core_write_mmd_indirect to complete this
- * function.
- */
- core_write_mmd_indirect(priv,
- CORE_GSWPLL_GRP1,
- MDIO_MMD_VEND2,
- 0);
-
- /* Set core clock into 500Mhz */
- core_write(priv, CORE_GSWPLL_GRP2,
- RG_GSWPLL_POSDIV_500M(1) |
- RG_GSWPLL_FBKDIV_500M(25));
+ /* Disable MT7530 core clock */
+ core_clear(priv, CORE_TRGMII_GSW_CLK_CG, REG_GSWCK_EN);
- /* Enable PLL */
- core_write(priv, CORE_GSWPLL_GRP1,
- RG_GSWPLL_EN_PRE |
- RG_GSWPLL_POSDIV_200M(2) |
- RG_GSWPLL_FBKDIV_200M(32));
-
- /* Enable MT7530 core clock */
- core_set(priv, CORE_TRGMII_GSW_CLK_CG, REG_GSWCK_EN);
- }
+ /* Disable PLL, since phy_device has not yet been created
+ * provided for phy_[read,write]_mmd_indirect is called, we
+ * provide our own core_write_mmd_indirect to complete this
+ * function.
+ */
+ core_write_mmd_indirect(priv,
+ CORE_GSWPLL_GRP1,
+ MDIO_MMD_VEND2,
+ 0);
+
+ /* Set core clock into 500Mhz */
+ core_write(priv, CORE_GSWPLL_GRP2,
+ RG_GSWPLL_POSDIV_500M(1) |
+ RG_GSWPLL_FBKDIV_500M(25));
+
+ /* Enable PLL */
+ core_write(priv, CORE_GSWPLL_GRP1,
+ RG_GSWPLL_EN_PRE |
+ RG_GSWPLL_POSDIV_200M(2) |
+ RG_GSWPLL_FBKDIV_200M(32));
+
+ /* Enable MT7530 core clock */
+ core_set(priv, CORE_TRGMII_GSW_CLK_CG, REG_GSWCK_EN);
/* Setup the MT7530 TRGMII Tx Clock */
core_set(priv, CORE_TRGMII_GSW_CLK_CG, REG_GSWCK_EN);
}
pci_set_master(pdev);
- ioaddr = pci_resource_start(pdev, 0);
- if (!ioaddr) {
+ if (!pci_resource_len(pdev, 0)) {
if (pcnet32_debug & NETIF_MSG_PROBE)
pr_err("card has no PCI IO resources, aborting\n");
err = -ENODEV;
pr_err("architecture does not support 32bit PCI busmaster DMA\n");
goto err_disable_dev;
}
+
+ ioaddr = pci_resource_start(pdev, 0);
if (!request_region(ioaddr, PCNET32_TOTAL_SIZE, "pcnet32_probe_pci")) {
if (pcnet32_debug & NETIF_MSG_PROBE)
pr_err("io address range already allocated\n");
#define XGBE_DMA_SYS_AWCR 0x30303030
/* DMA cache settings - PCI device */
-#define XGBE_DMA_PCI_ARCR 0x00000003
-#define XGBE_DMA_PCI_AWCR 0x13131313
-#define XGBE_DMA_PCI_AWARCR 0x00000313
+#define XGBE_DMA_PCI_ARCR 0x000f0f0f
+#define XGBE_DMA_PCI_AWCR 0x0f0f0f0f
+#define XGBE_DMA_PCI_AWARCR 0x00000f0f
/* DMA channel interrupt modes */
#define XGBE_IRQ_MODE_EDGE 0
config BCM4908_ENET
tristate "Broadcom BCM4908 internal mac support"
depends on ARCH_BCM4908 || COMPILE_TEST
- default y
+ default y if ARCH_BCM4908
help
This driver supports Ethernet controller integrated into Broadcom
BCM4908 family SoCs.
err_free_buf_descs:
dma_free_coherent(dev, size, ring->cpu_addr, ring->dma_addr);
+ ring->cpu_addr = NULL;
return -ENOMEM;
}
bool cmp_b = false;
bool cmp_c = false;
+ if (!macb_is_gem(bp))
+ return;
+
tp4sp_v = &(fs->h_u.tcp_ip4_spec);
tp4sp_m = &(fs->m_u.tcp_ip4_spec);
{
struct net_device *netdev = bp->dev;
netdev_features_t features = netdev->features;
+ struct ethtool_rx_fs_item *item;
/* TX checksum offload */
macb_set_txcsum_feature(bp, features);
macb_set_rxcsum_feature(bp, features);
/* RX Flow Filters */
+ list_for_each_entry(item, &bp->rx_fs_list.list, list)
+ gem_prog_cmp_regs(bp, &item->fs);
+
macb_set_rxflow_feature(bp, features);
}
struct cudbg_buffer temp_buff = { 0 };
struct sge_qbase_reg_field *sge_qbase;
struct ireg_buf *ch_sge_dbg;
+ u8 padap_running = 0;
int i, rc;
+ u32 size;
- rc = cudbg_get_buff(pdbg_init, dbg_buff,
- sizeof(*ch_sge_dbg) * 2 + sizeof(*sge_qbase),
- &temp_buff);
+ /* Accessing SGE_QBASE_MAP[0-3] and SGE_QBASE_INDEX regs can
+ * lead to SGE missing doorbells under heavy traffic. So, only
+ * collect them when adapter is idle.
+ */
+ for_each_port(padap, i) {
+ padap_running = netif_running(padap->port[i]);
+ if (padap_running)
+ break;
+ }
+
+ size = sizeof(*ch_sge_dbg) * 2;
+ if (!padap_running)
+ size += sizeof(*sge_qbase);
+
+ rc = cudbg_get_buff(pdbg_init, dbg_buff, size, &temp_buff);
if (rc)
return rc;
ch_sge_dbg++;
}
- if (CHELSIO_CHIP_VERSION(padap->params.chip) > CHELSIO_T5) {
+ if (CHELSIO_CHIP_VERSION(padap->params.chip) > CHELSIO_T5 &&
+ !padap_running) {
sge_qbase = (struct sge_qbase_reg_field *)ch_sge_dbg;
/* 1 addr reg SGE_QBASE_INDEX and 4 data reg
* SGE_QBASE_MAP[0-3]
0x1190, 0x1194,
0x11a0, 0x11a4,
0x11b0, 0x11b4,
- 0x11fc, 0x1274,
+ 0x11fc, 0x123c,
+ 0x1254, 0x1274,
0x1280, 0x133c,
0x1800, 0x18fc,
0x3000, 0x302c,
kvfree(tx_info);
return 0;
}
- tx_info->open_state = false;
+ tx_info->open_state = CH_KTLS_OPEN_SUCCESS;
spin_unlock(&tx_info->lock);
complete(&tx_info->completion);
*/
if (unlikely(priv->need_mac_restart)) {
ftgmac100_start_hw(priv);
+ priv->need_mac_restart = false;
/* Re-enable "bad" interrupts */
iowrite32(FTGMAC100_INT_BAD,
static int gfar_set_mac_addr(struct net_device *dev, void *p)
{
- eth_mac_addr(dev, p);
+ int ret;
+
+ ret = eth_mac_addr(dev, p);
+ if (ret)
+ return ret;
gfar_set_mac_for_addr(dev, 0, dev->dev_addr);
* normalcy is to reset.
* 2. A new reset request from the stack due to timeout
*
- * For the first case,error event might not have ae handle available.
* check if this is a new reset request and we are not here just because
* last reset attempt did not succeed and watchdog hit us again. We will
* know this if last reset request did not occur very recently (watchdog
* want to make sure we throttle the reset request. Therefore, we will
* not allow it again before 3*HZ times.
*/
- if (!handle)
- handle = &hdev->vport[0].nic;
if (time_before(jiffies, (hdev->last_reset_time +
HCLGE_RESET_INTERVAL))) {
mod_timer(&hdev->reset_timer, jiffies + HCLGE_RESET_INTERVAL);
return;
- } else if (hdev->default_reset_request) {
+ }
+
+ if (hdev->default_reset_request) {
hdev->reset_level =
hclge_get_reset_level(ae_dev,
&hdev->default_reset_request);
if (ret)
return ret;
- /* RSS indirection table has been configuared by user */
+ /* RSS indirection table has been configured by user */
if (rxfh_configured)
goto out;
if (test_and_clear_bit(HCLGEVF_RESET_PENDING,
&hdev->reset_state)) {
- /* PF has initmated that it is about to reset the hardware.
+ /* PF has intimated that it is about to reset the hardware.
* We now have to poll & check if hardware has actually
* completed the reset sequence. On hardware reset completion,
* VF needs to reset the client and ae device.
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
+ clear_bit(HCLGEVF_STATE_DOWN, &hdev->state);
+
hclgevf_reset_tqp_stats(handle);
hclgevf_request_link_info(hdev);
hclgevf_update_link_mode(hdev);
- clear_bit(HCLGEVF_STATE_DOWN, &hdev->state);
-
return 0;
}
if (ret)
return ret;
- /* RSS indirection table has been configuared by user */
+ /* RSS indirection table has been configured by user */
if (rxfh_configured)
goto out;
} else {
data &= ~IGP02E1000_PM_D0_LPLU;
ret_val = e1e_wphy(hw, IGP02E1000_PHY_POWER_MGMT, data);
+ if (ret_val)
+ return ret_val;
/* LPLU and SmartSpeed are mutually exclusive. LPLU is used
* during Dx states where the power conservation is most
* important. During driver activity we should enable
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright(c) 1999 - 2018 Intel Corporation. */
-#ifndef _E1000_HW_H_
-#define _E1000_HW_H_
+#ifndef _E1000E_HW_H_
+#define _E1000E_HW_H_
#include "regs.h"
#include "defines.h"
#include "80003es2lan.h"
#include "ich8lan.h"
-#endif
+#endif /* _E1000E_HW_H_ */
struct e1000_adapter *adapter;
adapter = container_of(work, struct e1000_adapter, reset_task);
+ rtnl_lock();
/* don't run the task if already down */
- if (test_bit(__E1000_DOWN, &adapter->state))
+ if (test_bit(__E1000_DOWN, &adapter->state)) {
+ rtnl_unlock();
return;
+ }
if (!(adapter->flags & FLAG_RESTART_NOW)) {
e1000e_dump(adapter);
e_err("Reset adapter unexpectedly\n");
}
e1000e_reinit_locked(adapter);
+ rtnl_unlock();
}
/**
__I40E_VIRTCHNL_OP_PENDING,
__I40E_RECOVERY_MODE,
__I40E_VF_RESETS_DISABLED, /* disable resets during i40e_remove */
+ __I40E_VFS_RELEASING,
/* This must be last as it determines the size of the BITMAP */
__I40E_STATE_SIZE__,
};
case RING_TYPE_XDP:
ring = kmemdup(vsi->xdp_rings[ring_id], sizeof(*ring), GFP_KERNEL);
break;
+ default:
+ ring = NULL;
+ break;
}
if (!ring)
return;
I40E_STAT(struct i40e_vsi, _name, _stat)
#define I40E_VEB_STAT(_name, _stat) \
I40E_STAT(struct i40e_veb, _name, _stat)
+#define I40E_VEB_TC_STAT(_name, _stat) \
+ I40E_STAT(struct i40e_cp_veb_tc_stats, _name, _stat)
#define I40E_PFC_STAT(_name, _stat) \
I40E_STAT(struct i40e_pfc_stats, _name, _stat)
#define I40E_QUEUE_STAT(_name, _stat) \
I40E_VEB_STAT("veb.rx_unknown_protocol", stats.rx_unknown_protocol),
};
+struct i40e_cp_veb_tc_stats {
+ u64 tc_rx_packets;
+ u64 tc_rx_bytes;
+ u64 tc_tx_packets;
+ u64 tc_tx_bytes;
+};
+
static const struct i40e_stats i40e_gstrings_veb_tc_stats[] = {
- I40E_VEB_STAT("veb.tc_%u_tx_packets", tc_stats.tc_tx_packets),
- I40E_VEB_STAT("veb.tc_%u_tx_bytes", tc_stats.tc_tx_bytes),
- I40E_VEB_STAT("veb.tc_%u_rx_packets", tc_stats.tc_rx_packets),
- I40E_VEB_STAT("veb.tc_%u_rx_bytes", tc_stats.tc_rx_bytes),
+ I40E_VEB_TC_STAT("veb.tc_%u_tx_packets", tc_tx_packets),
+ I40E_VEB_TC_STAT("veb.tc_%u_tx_bytes", tc_tx_bytes),
+ I40E_VEB_TC_STAT("veb.tc_%u_rx_packets", tc_rx_packets),
+ I40E_VEB_TC_STAT("veb.tc_%u_rx_bytes", tc_rx_bytes),
};
static const struct i40e_stats i40e_gstrings_misc_stats[] = {
/* Set flow control settings */
ethtool_link_ksettings_add_link_mode(ks, supported, Pause);
+ ethtool_link_ksettings_add_link_mode(ks, supported, Asym_Pause);
switch (hw->fc.requested_mode) {
case I40E_FC_FULL:
}
}
+/**
+ * i40e_get_veb_tc_stats - copy VEB TC statistics to formatted structure
+ * @tc: the TC statistics in VEB structure (veb->tc_stats)
+ * @i: the index of traffic class in (veb->tc_stats) structure to copy
+ *
+ * Copy VEB TC statistics from structure of arrays (veb->tc_stats) to
+ * one dimensional structure i40e_cp_veb_tc_stats.
+ * Produce formatted i40e_cp_veb_tc_stats structure of the VEB TC
+ * statistics for the given TC.
+ **/
+static struct i40e_cp_veb_tc_stats
+i40e_get_veb_tc_stats(struct i40e_veb_tc_stats *tc, unsigned int i)
+{
+ struct i40e_cp_veb_tc_stats veb_tc = {
+ .tc_rx_packets = tc->tc_rx_packets[i],
+ .tc_rx_bytes = tc->tc_rx_bytes[i],
+ .tc_tx_packets = tc->tc_tx_packets[i],
+ .tc_tx_bytes = tc->tc_tx_bytes[i],
+ };
+
+ return veb_tc;
+}
+
/**
* i40e_get_pfc_stats - copy HW PFC statistics to formatted structure
* @pf: the PF device structure
i40e_gstrings_veb_stats);
for (i = 0; i < I40E_MAX_TRAFFIC_CLASS; i++)
- i40e_add_ethtool_stats(&data, veb_stats ? veb : NULL,
- i40e_gstrings_veb_tc_stats);
+ if (veb_stats) {
+ struct i40e_cp_veb_tc_stats veb_tc =
+ i40e_get_veb_tc_stats(&veb->tc_stats, i);
+
+ i40e_add_ethtool_stats(&data, &veb_tc,
+ i40e_gstrings_veb_tc_stats);
+ } else {
+ i40e_add_ethtool_stats(&data, NULL,
+ i40e_gstrings_veb_tc_stats);
+ }
i40e_add_ethtool_stats(&data, pf, i40e_gstrings_stats);
status = i40e_aq_get_phy_register(hw,
I40E_AQ_PHY_REG_ACCESS_EXTERNAL_MODULE,
- true, addr, offset, &value, NULL);
+ addr, true, offset, &value, NULL);
if (status)
return -EIO;
data[i] = value;
i40e_stat_str(hw, aq_ret),
i40e_aq_str(hw, hw->aq.asq_last_status));
} else {
- dev_info(&pf->pdev->dev, "%s is %s allmulti mode.\n",
- vsi->netdev->name,
+ dev_info(&pf->pdev->dev, "%s allmulti mode.\n",
cur_multipromisc ? "entering" : "leaving");
}
}
return 0;
}
+/**
+ * i40e_rx_offset - Return expected offset into page to access data
+ * @rx_ring: Ring we are requesting offset of
+ *
+ * Returns the offset value for ring into the data buffer.
+ */
+static unsigned int i40e_rx_offset(struct i40e_ring *rx_ring)
+{
+ return ring_uses_build_skb(rx_ring) ? I40E_SKB_PAD : 0;
+}
+
/**
* i40e_configure_rx_ring - Configure a receive ring context
* @ring: The Rx ring to configure
else
set_ring_build_skb_enabled(ring);
+ ring->rx_offset = i40e_rx_offset(ring);
+
/* cache tail for quicker writes, and clear the reg before use */
ring->tail = hw->hw_addr + I40E_QRX_TAIL(pf_q);
writel(0, ring->tail);
set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
set_bit(__I40E_CLIENT_L2_CHANGE, pf->state);
}
- /* registers are set, lets apply */
- if (pf->hw_features & I40E_HW_USE_SET_LLDP_MIB)
- ret = i40e_hw_set_dcb_config(pf, new_cfg);
+ /* registers are set, lets apply */
+ if (pf->hw_features & I40E_HW_USE_SET_LLDP_MIB)
+ ret = i40e_hw_set_dcb_config(pf, new_cfg);
}
err:
goto end_core_reset;
}
- if (!lock_acquired)
- rtnl_lock();
- ret = i40e_setup_pf_switch(pf, reinit);
- if (ret)
- goto end_unlock;
-
#ifdef CONFIG_I40E_DCB
/* Enable FW to write a default DCB config on link-up
* unless I40E_FLAG_TC_MQPRIO was enabled or DCB
i40e_aq_set_dcb_parameters(hw, false, NULL);
dev_warn(&pf->pdev->dev,
"DCB is not supported for X710-T*L 2.5/5G speeds\n");
- pf->flags &= ~I40E_FLAG_DCB_CAPABLE;
+ pf->flags &= ~I40E_FLAG_DCB_CAPABLE;
} else {
i40e_aq_set_dcb_parameters(hw, true, NULL);
ret = i40e_init_pf_dcb(pf);
}
#endif /* CONFIG_I40E_DCB */
+ if (!lock_acquired)
+ rtnl_lock();
+ ret = i40e_setup_pf_switch(pf, reinit);
+ if (ret)
+ goto end_unlock;
/* The driver only wants link up/down and module qualification
* reports from firmware. Note the negative logic.
* in order to register the netdev
*/
v_idx = i40e_vsi_mem_alloc(pf, I40E_VSI_MAIN);
- if (v_idx < 0)
+ if (v_idx < 0) {
+ err = v_idx;
goto err_switch_setup;
+ }
pf->lan_vsi = v_idx;
vsi = pf->vsi[v_idx];
- if (!vsi)
+ if (!vsi) {
+ err = -EFAULT;
goto err_switch_setup;
+ }
vsi->alloc_queue_pairs = 1;
err = i40e_config_netdev(vsi);
if (err)
}
}
-/**
- * i40e_rx_offset - Return expected offset into page to access data
- * @rx_ring: Ring we are requesting offset of
- *
- * Returns the offset value for ring into the data buffer.
- */
-static unsigned int i40e_rx_offset(struct i40e_ring *rx_ring)
-{
- return ring_uses_build_skb(rx_ring) ? I40E_SKB_PAD : 0;
-}
-
/**
* i40e_setup_rx_descriptors - Allocate Rx descriptors
* @rx_ring: Rx descriptor ring (for a specific queue) to setup
rx_ring->next_to_alloc = 0;
rx_ring->next_to_clean = 0;
rx_ring->next_to_use = 0;
- rx_ring->rx_offset = i40e_rx_offset(rx_ring);
/* XDP RX-queue info only needed for RX rings exposed to XDP */
if (rx_ring->vsi->type == I40E_VSI_MAIN) {
* @rx_ring: Rx ring being processed
* @xdp: XDP buffer containing the frame
**/
-static struct sk_buff *i40e_run_xdp(struct i40e_ring *rx_ring,
- struct xdp_buff *xdp)
+static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
{
int err, result = I40E_XDP_PASS;
struct i40e_ring *xdp_ring;
}
xdp_out:
rcu_read_unlock();
- return ERR_PTR(-result);
+ return result;
}
/**
unsigned int xdp_xmit = 0;
bool failure = false;
struct xdp_buff xdp;
+ int xdp_res = 0;
#if (PAGE_SIZE < 8192)
frame_sz = i40e_rx_frame_truesize(rx_ring, 0);
/* At larger PAGE_SIZE, frame_sz depend on len size */
xdp.frame_sz = i40e_rx_frame_truesize(rx_ring, size);
#endif
- skb = i40e_run_xdp(rx_ring, &xdp);
+ xdp_res = i40e_run_xdp(rx_ring, &xdp);
}
- if (IS_ERR(skb)) {
- unsigned int xdp_res = -PTR_ERR(skb);
-
+ if (xdp_res) {
if (xdp_res & (I40E_XDP_TX | I40E_XDP_REDIR)) {
xdp_xmit |= xdp_res;
i40e_rx_buffer_flip(rx_ring, rx_buffer, size);
**/
static inline void i40e_vc_disable_vf(struct i40e_vf *vf)
{
+ struct i40e_pf *pf = vf->pf;
int i;
i40e_vc_notify_vf_reset(vf);
* ensure a reset.
*/
for (i = 0; i < 20; i++) {
+ /* If PF is in VFs releasing state reset VF is impossible,
+ * so leave it.
+ */
+ if (test_bit(__I40E_VFS_RELEASING, pf->state))
+ return;
if (i40e_reset_vf(vf, false))
return;
usleep_range(10000, 20000);
if (!pf->vf)
return;
+
+ set_bit(__I40E_VFS_RELEASING, pf->state);
while (test_and_set_bit(__I40E_VF_DISABLE, pf->state))
usleep_range(1000, 2000);
}
}
clear_bit(__I40E_VF_DISABLE, pf->state);
+ clear_bit(__I40E_VFS_RELEASING, pf->state);
}
#ifdef CONFIG_PCI_IOV
nb_pkts = xsk_tx_peek_release_desc_batch(xdp_ring->xsk_pool, descs, budget);
if (!nb_pkts)
- return false;
+ return true;
if (xdp_ring->next_to_use + nb_pkts >= xdp_ring->count) {
nb_processed = xdp_ring->count - xdp_ring->next_to_use;
i40e_update_tx_stats(xdp_ring, nb_pkts, total_bytes);
- return true;
+ return nb_pkts < budget;
}
/**
__ICE_NEEDS_RESTART,
__ICE_PREPARED_FOR_RESET, /* set by driver when prepared */
__ICE_RESET_OICR_RECV, /* set by driver after rcv reset OICR */
- __ICE_DCBNL_DEVRESET, /* set by dcbnl devreset */
__ICE_PFR_REQ, /* set by driver and peers */
__ICE_CORER_REQ, /* set by driver and peers */
__ICE_GLOBR_REQ, /* set by driver and peers */
void ice_print_link_msg(struct ice_vsi *vsi, bool isup);
const char *ice_stat_str(enum ice_status stat_err);
const char *ice_aq_str(enum ice_aq_err aq_err);
-bool ice_is_wol_supported(struct ice_pf *pf);
+bool ice_is_wol_supported(struct ice_hw *hw);
int
ice_fdir_write_fltr(struct ice_pf *pf, struct ice_fdir_fltr *input, bool add,
bool is_tun);
int ice_aq_wait_for_event(struct ice_pf *pf, u16 opcode, unsigned long timeout,
struct ice_rq_event_info *event);
int ice_open(struct net_device *netdev);
+int ice_open_internal(struct net_device *netdev);
int ice_stop(struct net_device *netdev);
void ice_service_task_schedule(struct ice_pf *pf);
tlan_ctx->legacy_int = ICE_TX_LEGACY;
}
+/**
+ * ice_rx_offset - Return expected offset into page to access data
+ * @rx_ring: Ring we are requesting offset of
+ *
+ * Returns the offset value for ring into the data buffer.
+ */
+static unsigned int ice_rx_offset(struct ice_ring *rx_ring)
+{
+ if (ice_ring_uses_build_skb(rx_ring))
+ return ICE_SKB_PAD;
+ else if (ice_is_xdp_ena_vsi(rx_ring->vsi))
+ return XDP_PACKET_HEADROOM;
+
+ return 0;
+}
+
/**
* ice_setup_rx_ctx - Configure a receive ring context
* @ring: The Rx ring to configure
else
ice_set_ring_build_skb_ena(ring);
+ ring->rx_offset = ice_rx_offset(ring);
+
/* init queue specific tail register */
ring->tail = hw->hw_addr + QRX_TAIL(pf_q);
writel(0, ring->tail);
if (ring->xsk_pool) {
+ bool ok;
+
if (!xsk_buff_can_alloc(ring->xsk_pool, num_bufs)) {
dev_warn(dev, "XSK buffer pool does not provide enough addresses to fill %d buffers on Rx ring %d\n",
num_bufs, ring->q_index);
return 0;
}
- err = ice_alloc_rx_bufs_zc(ring, num_bufs);
- if (err)
+ ok = ice_alloc_rx_bufs_zc(ring, num_bufs);
+ if (!ok)
dev_info(dev, "Failed to allocate some buffers on XSK buffer pool enabled Rx ring %d (pf_q %d)\n",
ring->q_index, pf_q);
return 0;
if (!data) {
data = devm_kcalloc(ice_hw_to_dev(hw),
- sizeof(*data),
ICE_AQC_FW_LOG_ID_MAX,
+ sizeof(*data),
GFP_KERNEL);
if (!data)
return ICE_ERR_NO_MEMORY;
ICE_CTL_Q_MAILBOX,
};
-/* Control Queue timeout settings - max delay 250ms */
-#define ICE_CTL_Q_SQ_CMD_TIMEOUT 2500 /* Count 2500 times */
+/* Control Queue timeout settings - max delay 1s */
+#define ICE_CTL_Q_SQ_CMD_TIMEOUT 10000 /* Count 10000 times */
#define ICE_CTL_Q_SQ_CMD_USEC 100 /* Check every 100usec */
#define ICE_CTL_Q_ADMIN_INIT_TIMEOUT 10 /* Count 10 times */
#define ICE_CTL_Q_ADMIN_INIT_MSEC 100 /* Check every 100msec */
/**
* ice_cee_to_dcb_cfg
* @cee_cfg: pointer to CEE configuration struct
- * @dcbcfg: DCB configuration struct
+ * @pi: port information structure
*
* Convert CEE configuration from firmware to DCB configuration
*/
static void
ice_cee_to_dcb_cfg(struct ice_aqc_get_cee_dcb_cfg_resp *cee_cfg,
- struct ice_dcbx_cfg *dcbcfg)
+ struct ice_port_info *pi)
{
u32 status, tlv_status = le32_to_cpu(cee_cfg->tlv_status);
u32 ice_aqc_cee_status_mask, ice_aqc_cee_status_shift;
+ u8 i, j, err, sync, oper, app_index, ice_app_sel_type;
u16 app_prio = le16_to_cpu(cee_cfg->oper_app_prio);
- u8 i, err, sync, oper, app_index, ice_app_sel_type;
u16 ice_aqc_cee_app_mask, ice_aqc_cee_app_shift;
+ struct ice_dcbx_cfg *cmp_dcbcfg, *dcbcfg;
u16 ice_app_prot_id_type;
- /* CEE PG data to ETS config */
+ dcbcfg = &pi->qos_cfg.local_dcbx_cfg;
+ dcbcfg->dcbx_mode = ICE_DCBX_MODE_CEE;
+ dcbcfg->tlv_status = tlv_status;
+
+ /* CEE PG data */
dcbcfg->etscfg.maxtcs = cee_cfg->oper_num_tc;
/* Note that the FW creates the oper_prio_tc nibbles reversed
}
}
- /* CEE PFC data to ETS config */
+ /* CEE PFC data */
dcbcfg->pfc.pfcena = cee_cfg->oper_pfc_en;
dcbcfg->pfc.pfccap = ICE_MAX_TRAFFIC_CLASS;
+ /* CEE APP TLV data */
+ if (dcbcfg->app_mode == ICE_DCBX_APPS_NON_WILLING)
+ cmp_dcbcfg = &pi->qos_cfg.desired_dcbx_cfg;
+ else
+ cmp_dcbcfg = &pi->qos_cfg.remote_dcbx_cfg;
+
app_index = 0;
for (i = 0; i < 3; i++) {
if (i == 0) {
ice_aqc_cee_app_shift = ICE_AQC_CEE_APP_ISCSI_S;
ice_app_sel_type = ICE_APP_SEL_TCPIP;
ice_app_prot_id_type = ICE_APP_PROT_ID_ISCSI;
+
+ for (j = 0; j < cmp_dcbcfg->numapps; j++) {
+ u16 prot_id = cmp_dcbcfg->app[j].prot_id;
+ u8 sel = cmp_dcbcfg->app[j].selector;
+
+ if (sel == ICE_APP_SEL_TCPIP &&
+ (prot_id == ICE_APP_PROT_ID_ISCSI ||
+ prot_id == ICE_APP_PROT_ID_ISCSI_860)) {
+ ice_app_prot_id_type = prot_id;
+ break;
+ }
+ }
} else {
/* FIP APP */
ice_aqc_cee_status_mask = ICE_AQC_CEE_FIP_STATUS_M;
ret = ice_aq_get_cee_dcb_cfg(pi->hw, &cee_cfg, NULL);
if (!ret) {
/* CEE mode */
- dcbx_cfg = &pi->qos_cfg.local_dcbx_cfg;
- dcbx_cfg->dcbx_mode = ICE_DCBX_MODE_CEE;
- dcbx_cfg->tlv_status = le32_to_cpu(cee_cfg.tlv_status);
- ice_cee_to_dcb_cfg(&cee_cfg, dcbx_cfg);
ret = ice_get_ieee_or_cee_dcb_cfg(pi, ICE_DCBX_MODE_CEE);
+ ice_cee_to_dcb_cfg(&cee_cfg, pi);
} else if (pi->hw->adminq.sq_last_status == ICE_AQ_RC_ENOENT) {
/* CEE mode not enabled try querying IEEE data */
dcbx_cfg = &pi->qos_cfg.local_dcbx_cfg;
while (ice_is_reset_in_progress(pf->state))
usleep_range(1000, 2000);
- set_bit(__ICE_DCBNL_DEVRESET, pf->state);
dev_close(netdev);
netdev_state_change(netdev);
dev_open(netdev, NULL);
netdev_state_change(netdev);
- clear_bit(__ICE_DCBNL_DEVRESET, pf->state);
}
/**
netdev_warn(netdev, "Wake on LAN is not supported on this interface!\n");
/* Get WoL settings based on the HW capability */
- if (ice_is_wol_supported(pf)) {
+ if (ice_is_wol_supported(&pf->hw)) {
wol->supported = WAKE_MAGIC;
wol->wolopts = pf->wol_ena ? WAKE_MAGIC : 0;
} else {
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
- if (vsi->type != ICE_VSI_PF || !ice_is_wol_supported(pf))
+ if (vsi->type != ICE_VSI_PF || !ice_is_wol_supported(&pf->hw))
return -EOPNOTSUPP;
/* only magic packet is supported */
if (!locked)
rtnl_lock();
- err = ice_open(vsi->netdev);
+ err = ice_open_internal(vsi->netdev);
if (!locked)
rtnl_unlock();
if (!locked)
rtnl_lock();
- ice_stop(vsi->netdev);
+ ice_vsi_close(vsi);
if (!locked)
rtnl_unlock();
bool ice_is_reset_in_progress(unsigned long *state)
{
return test_bit(__ICE_RESET_OICR_RECV, state) ||
- test_bit(__ICE_DCBNL_DEVRESET, state) ||
test_bit(__ICE_PFR_REQ, state) ||
test_bit(__ICE_CORER_REQ, state) ||
test_bit(__ICE_GLOBR_REQ, state);
}
/**
- * ice_is_wol_supported - get NVM state of WoL
- * @pf: board private structure
+ * ice_is_wol_supported - check if WoL is supported
+ * @hw: pointer to hardware info
*
* Check if WoL is supported based on the HW configuration.
* Returns true if NVM supports and enables WoL for this port, false otherwise
*/
-bool ice_is_wol_supported(struct ice_pf *pf)
+bool ice_is_wol_supported(struct ice_hw *hw)
{
- struct ice_hw *hw = &pf->hw;
u16 wol_ctrl;
/* A bit set to 1 in the NVM Software Reserved Word 2 (WoL control
if (ice_read_sr_word(hw, ICE_SR_NVM_WOL_CFG, &wol_ctrl))
return false;
- return !(BIT(hw->pf_id) & wol_ctrl);
+ return !(BIT(hw->port_info->lport) & wol_ctrl);
}
/**
goto err_send_version_unroll;
}
+ /* not a fatal error if this fails */
err = ice_init_nvm_phy_type(pf->hw.port_info);
- if (err) {
+ if (err)
dev_err(dev, "ice_init_nvm_phy_type failed: %d\n", err);
- goto err_send_version_unroll;
- }
+ /* not a fatal error if this fails */
err = ice_update_link_info(pf->hw.port_info);
- if (err) {
+ if (err)
dev_err(dev, "ice_update_link_info failed: %d\n", err);
- goto err_send_version_unroll;
- }
ice_init_link_dflt_override(pf->hw.port_info);
/* if media available, initialize PHY settings */
if (pf->hw.port_info->phy.link_info.link_info &
ICE_AQ_MEDIA_AVAILABLE) {
+ /* not a fatal error if this fails */
err = ice_init_phy_user_cfg(pf->hw.port_info);
- if (err) {
+ if (err)
dev_err(dev, "ice_init_phy_user_cfg failed: %d\n", err);
- goto err_send_version_unroll;
- }
if (!test_bit(ICE_FLAG_LINK_DOWN_ON_CLOSE_ENA, pf->flags)) {
struct ice_vsi *vsi = ice_get_main_vsi(pf);
continue;
ice_vsi_free_q_vectors(pf->vsi[v]);
}
+ ice_free_cpu_rx_rmap(ice_get_main_vsi(pf));
ice_clear_interrupt_scheme(pf);
pci_save_state(pdev);
* Returns 0 on success, negative value on failure
*/
int ice_open(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_pf *pf = np->vsi->back;
+
+ if (ice_is_reset_in_progress(pf->state)) {
+ netdev_err(netdev, "can't open net device while reset is in progress");
+ return -EBUSY;
+ }
+
+ return ice_open_internal(netdev);
+}
+
+/**
+ * ice_open_internal - Called when a network interface becomes active
+ * @netdev: network interface device structure
+ *
+ * Internal ice_open implementation. Should not be used directly except for ice_open and reset
+ * handling routine
+ *
+ * Returns 0 on success, negative value on failure
+ */
+int ice_open_internal(struct net_device *netdev)
{
struct ice_netdev_priv *np = netdev_priv(netdev);
struct ice_vsi *vsi = np->vsi;
{
struct ice_netdev_priv *np = netdev_priv(netdev);
struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+
+ if (ice_is_reset_in_progress(pf->state)) {
+ netdev_err(netdev, "can't stop net device while reset is in progress");
+ return -EBUSY;
+ }
ice_vsi_close(vsi);
ice_create_vsi_list_map(hw, &vsi_handle_arr[0], 2,
vsi_list_id);
+ if (!m_entry->vsi_list_info)
+ return ICE_ERR_NO_MEMORY;
+
/* If this entry was large action then the large action needs
* to be updated to point to FWD to VSI list
*/
return ((fm_entry->fltr_info.fltr_act == ICE_FWD_TO_VSI &&
fm_entry->fltr_info.vsi_handle == vsi_handle) ||
(fm_entry->fltr_info.fltr_act == ICE_FWD_TO_VSI_LIST &&
+ fm_entry->vsi_list_info &&
(test_bit(vsi_handle, fm_entry->vsi_list_info->vsi_map))));
}
return ICE_ERR_PARAM;
list_for_each_entry(fm_entry, lkup_list_head, list_entry) {
- struct ice_fltr_info *fi;
-
- fi = &fm_entry->fltr_info;
- if (!fi || !ice_vsi_uses_fltr(fm_entry, vsi_handle))
+ if (!ice_vsi_uses_fltr(fm_entry, vsi_handle))
continue;
status = ice_add_entry_to_vsi_fltr_list(hw, vsi_handle,
- vsi_list_head, fi);
+ vsi_list_head,
+ &fm_entry->fltr_info);
if (status)
return status;
}
&remove_list_head);
mutex_unlock(rule_lock);
if (status)
- return;
+ goto free_fltr_list;
switch (lkup) {
case ICE_SW_LKUP_MAC:
break;
}
+free_fltr_list:
list_for_each_entry_safe(fm_entry, tmp, &remove_list_head, list_entry) {
list_del(&fm_entry->list_entry);
devm_kfree(ice_hw_to_dev(hw), fm_entry);
}
}
-/**
- * ice_rx_offset - Return expected offset into page to access data
- * @rx_ring: Ring we are requesting offset of
- *
- * Returns the offset value for ring into the data buffer.
- */
-static unsigned int ice_rx_offset(struct ice_ring *rx_ring)
-{
- if (ice_ring_uses_build_skb(rx_ring))
- return ICE_SKB_PAD;
- else if (ice_is_xdp_ena_vsi(rx_ring->vsi))
- return XDP_PACKET_HEADROOM;
-
- return 0;
-}
-
/**
* ice_setup_rx_ring - Allocate the Rx descriptors
* @rx_ring: the Rx ring to set up
rx_ring->next_to_use = 0;
rx_ring->next_to_clean = 0;
- rx_ring->rx_offset = ice_rx_offset(rx_ring);
if (ice_is_xdp_ena_vsi(rx_ring->vsi))
WRITE_ONCE(rx_ring->xdp_prog, rx_ring->vsi->xdp_prog);
#define ICE_TLV_STATUS_ERR 0x4
#define ICE_APP_PROT_ID_FCOE 0x8906
#define ICE_APP_PROT_ID_ISCSI 0x0cbc
+#define ICE_APP_PROT_ID_ISCSI_860 0x035c
#define ICE_APP_PROT_ID_FIP 0x8914
#define ICE_APP_SEL_ETHTYPE 0x1
#define ICE_APP_SEL_TCPIP 0x2
* This function allocates a number of Rx buffers from the fill ring
* or the internal recycle mechanism and places them on the Rx ring.
*
- * Returns false if all allocations were successful, true if any fail.
+ * Returns true if all allocations were successful, false if any fail.
*/
bool ice_alloc_rx_bufs_zc(struct ice_ring *rx_ring, u16 count)
{
union ice_32b_rx_flex_desc *rx_desc;
u16 ntu = rx_ring->next_to_use;
struct ice_rx_buf *rx_buf;
- bool ret = false;
+ bool ok = true;
dma_addr_t dma;
if (!count)
- return false;
+ return true;
rx_desc = ICE_RX_DESC(rx_ring, ntu);
rx_buf = &rx_ring->rx_buf[ntu];
do {
rx_buf->xdp = xsk_buff_alloc(rx_ring->xsk_pool);
if (!rx_buf->xdp) {
- ret = true;
+ ok = false;
break;
}
ice_release_rx_desc(rx_ring, ntu);
}
- return ret;
+ return ok;
}
/**
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright(c) 2007 - 2018 Intel Corporation. */
-#ifndef _E1000_HW_H_
-#define _E1000_HW_H_
+#ifndef _E1000_IGB_HW_H_
+#define _E1000_IGB_HW_H_
#include <linux/types.h>
#include <linux/delay.h>
void igb_read_pci_cfg(struct e1000_hw *hw, u32 reg, u16 *value);
void igb_write_pci_cfg(struct e1000_hw *hw, u32 reg, u16 *value);
-#endif /* _E1000_HW_H_ */
+#endif /* _E1000_IGB_HW_H_ */
void igb_ptp_rx_hang(struct igb_adapter *adapter);
void igb_ptp_tx_hang(struct igb_adapter *adapter);
void igb_ptp_rx_rgtstamp(struct igb_q_vector *q_vector, struct sk_buff *skb);
-void igb_ptp_rx_pktstamp(struct igb_q_vector *q_vector, void *va,
- struct sk_buff *skb);
+int igb_ptp_rx_pktstamp(struct igb_q_vector *q_vector, void *va,
+ struct sk_buff *skb);
int igb_ptp_set_ts_config(struct net_device *netdev, struct ifreq *ifr);
int igb_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr);
void igb_set_flag_queue_pairs(struct igb_adapter *, const u32);
new_buff->pagecnt_bias = old_buff->pagecnt_bias;
}
-static bool igb_can_reuse_rx_page(struct igb_rx_buffer *rx_buffer)
+static bool igb_can_reuse_rx_page(struct igb_rx_buffer *rx_buffer,
+ int rx_buf_pgcnt)
{
unsigned int pagecnt_bias = rx_buffer->pagecnt_bias;
struct page *page = rx_buffer->page;
#if (PAGE_SIZE < 8192)
/* if we are only owner of page we can reuse it */
- if (unlikely((page_ref_count(page) - pagecnt_bias) > 1))
+ if (unlikely((rx_buf_pgcnt - pagecnt_bias) > 1))
return false;
#else
#define IGB_LAST_OFFSET \
return NULL;
if (unlikely(igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP))) {
- igb_ptp_rx_pktstamp(rx_ring->q_vector, xdp->data, skb);
- xdp->data += IGB_TS_HDR_LEN;
- size -= IGB_TS_HDR_LEN;
+ if (!igb_ptp_rx_pktstamp(rx_ring->q_vector, xdp->data, skb)) {
+ xdp->data += IGB_TS_HDR_LEN;
+ size -= IGB_TS_HDR_LEN;
+ }
}
/* Determine available headroom for copy */
/* pull timestamp out of packet data */
if (igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
- igb_ptp_rx_pktstamp(rx_ring->q_vector, skb->data, skb);
- __skb_pull(skb, IGB_TS_HDR_LEN);
+ if (!igb_ptp_rx_pktstamp(rx_ring->q_vector, skb->data, skb))
+ __skb_pull(skb, IGB_TS_HDR_LEN);
}
/* update buffer offset */
}
static struct igb_rx_buffer *igb_get_rx_buffer(struct igb_ring *rx_ring,
- const unsigned int size)
+ const unsigned int size, int *rx_buf_pgcnt)
{
struct igb_rx_buffer *rx_buffer;
rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean];
+ *rx_buf_pgcnt =
+#if (PAGE_SIZE < 8192)
+ page_count(rx_buffer->page);
+#else
+ 0;
+#endif
prefetchw(rx_buffer->page);
/* we are reusing so sync this buffer for CPU use */
}
static void igb_put_rx_buffer(struct igb_ring *rx_ring,
- struct igb_rx_buffer *rx_buffer)
+ struct igb_rx_buffer *rx_buffer, int rx_buf_pgcnt)
{
- if (igb_can_reuse_rx_page(rx_buffer)) {
+ if (igb_can_reuse_rx_page(rx_buffer, rx_buf_pgcnt)) {
/* hand second half of page back to the ring */
igb_reuse_rx_page(rx_ring, rx_buffer);
} else {
unsigned int xdp_xmit = 0;
struct xdp_buff xdp;
u32 frame_sz = 0;
+ int rx_buf_pgcnt;
/* Frame size depend on rx_ring setup when PAGE_SIZE=4K */
#if (PAGE_SIZE < 8192)
*/
dma_rmb();
- rx_buffer = igb_get_rx_buffer(rx_ring, size);
+ rx_buffer = igb_get_rx_buffer(rx_ring, size, &rx_buf_pgcnt);
/* retrieve a buffer from the ring */
if (!skb) {
break;
}
- igb_put_rx_buffer(rx_ring, rx_buffer);
+ igb_put_rx_buffer(rx_ring, rx_buffer, rx_buf_pgcnt);
cleaned_count++;
/* fetch next buffer in frame if non-eop */
dev_kfree_skb_any(skb);
}
+#define IGB_RET_PTP_DISABLED 1
+#define IGB_RET_PTP_INVALID 2
+
/**
* igb_ptp_rx_pktstamp - retrieve Rx per packet timestamp
* @q_vector: Pointer to interrupt specific structure
*
* This function is meant to retrieve a timestamp from the first buffer of an
* incoming frame. The value is stored in little endian format starting on
- * byte 8.
+ * byte 8
+ *
+ * Returns: 0 if success, nonzero if failure
**/
-void igb_ptp_rx_pktstamp(struct igb_q_vector *q_vector, void *va,
- struct sk_buff *skb)
+int igb_ptp_rx_pktstamp(struct igb_q_vector *q_vector, void *va,
+ struct sk_buff *skb)
{
- __le64 *regval = (__le64 *)va;
struct igb_adapter *adapter = q_vector->adapter;
+ __le64 *regval = (__le64 *)va;
int adjust = 0;
+ if (!(adapter->ptp_flags & IGB_PTP_ENABLED))
+ return IGB_RET_PTP_DISABLED;
+
/* The timestamp is recorded in little endian format.
* DWORD: 0 1 2 3
* Field: Reserved Reserved SYSTIML SYSTIMH
*/
+
+ /* check reserved dwords are zero, be/le doesn't matter for zero */
+ if (regval[0])
+ return IGB_RET_PTP_INVALID;
+
igb_ptp_systim_to_hwtstamp(adapter, skb_hwtstamps(skb),
le64_to_cpu(regval[1]));
}
skb_hwtstamps(skb)->hwtstamp =
ktime_sub_ns(skb_hwtstamps(skb)->hwtstamp, adjust);
+
+ return 0;
}
/**
* This function is meant to retrieve a timestamp from the internal registers
* of the adapter and store it in the skb.
**/
-void igb_ptp_rx_rgtstamp(struct igb_q_vector *q_vector,
- struct sk_buff *skb)
+void igb_ptp_rx_rgtstamp(struct igb_q_vector *q_vector, struct sk_buff *skb)
{
struct igb_adapter *adapter = q_vector->adapter;
struct e1000_hw *hw = &adapter->hw;
- u64 regval;
int adjust = 0;
+ u64 regval;
+
+ if (!(adapter->ptp_flags & IGB_PTP_ENABLED))
+ return;
/* If this bit is set, then the RX registers contain the time stamp. No
* other packet will be time stamped until we read these registers, so
void igc_ptp_reset(struct igc_adapter *adapter);
void igc_ptp_suspend(struct igc_adapter *adapter);
void igc_ptp_stop(struct igc_adapter *adapter);
-void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, void *va,
+void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, __le32 *va,
struct sk_buff *skb);
int igc_ptp_set_ts_config(struct net_device *netdev, struct ifreq *ifr);
int igc_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr);
Autoneg);
}
+ /* Set pause flow control settings */
+ ethtool_link_ksettings_add_link_mode(cmd, supported, Pause);
+
switch (hw->fc.requested_mode) {
case igc_fc_full:
ethtool_link_ksettings_add_link_mode(cmd, advertising, Pause);
Asym_Pause);
break;
default:
- ethtool_link_ksettings_add_link_mode(cmd, advertising, Pause);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- Asym_Pause);
+ break;
}
status = pm_runtime_suspended(&adapter->pdev->dev) ?
adapter = container_of(work, struct igc_adapter, reset_task);
+ rtnl_lock();
+ /* If we're already down or resetting, just bail */
+ if (test_bit(__IGC_DOWN, &adapter->state) ||
+ test_bit(__IGC_RESETTING, &adapter->state)) {
+ rtnl_unlock();
+ return;
+ }
+
igc_rings_dump(adapter);
igc_regs_dump(adapter);
netdev_err(adapter->netdev, "Reset adapter\n");
igc_reinit_locked(adapter);
+ rtnl_unlock();
}
/**
}
/**
- * igc_ptp_rx_pktstamp - retrieve Rx per packet timestamp
+ * igc_ptp_rx_pktstamp - Retrieve timestamp from Rx packet buffer
* @q_vector: Pointer to interrupt specific structure
* @va: Pointer to address containing Rx buffer
* @skb: Buffer containing timestamp and packet
*
- * This function is meant to retrieve the first timestamp from the
- * first buffer of an incoming frame. The value is stored in little
- * endian format starting on byte 0. There's a second timestamp
- * starting on byte 8.
- **/
-void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, void *va,
+ * This function retrieves the timestamp saved in the beginning of packet
+ * buffer. While two timestamps are available, one in timer0 reference and the
+ * other in timer1 reference, this function considers only the timestamp in
+ * timer0 reference.
+ */
+void igc_ptp_rx_pktstamp(struct igc_q_vector *q_vector, __le32 *va,
struct sk_buff *skb)
{
struct igc_adapter *adapter = q_vector->adapter;
- __le64 *regval = (__le64 *)va;
- int adjust = 0;
-
- /* The timestamp is recorded in little endian format.
- * DWORD: | 0 | 1 | 2 | 3
- * Field: | Timer0 Low | Timer0 High | Timer1 Low | Timer1 High
+ u64 regval;
+ int adjust;
+
+ /* Timestamps are saved in little endian at the beginning of the packet
+ * buffer following the layout:
+ *
+ * DWORD: | 0 | 1 | 2 | 3 |
+ * Field: | Timer1 SYSTIML | Timer1 SYSTIMH | Timer0 SYSTIML | Timer0 SYSTIMH |
+ *
+ * SYSTIML holds the nanoseconds part while SYSTIMH holds the seconds
+ * part of the timestamp.
*/
- igc_ptp_systim_to_hwtstamp(adapter, skb_hwtstamps(skb),
- le64_to_cpu(regval[0]));
-
- /* adjust timestamp for the RX latency based on link speed */
- if (adapter->hw.mac.type == igc_i225) {
- switch (adapter->link_speed) {
- case SPEED_10:
- adjust = IGC_I225_RX_LATENCY_10;
- break;
- case SPEED_100:
- adjust = IGC_I225_RX_LATENCY_100;
- break;
- case SPEED_1000:
- adjust = IGC_I225_RX_LATENCY_1000;
- break;
- case SPEED_2500:
- adjust = IGC_I225_RX_LATENCY_2500;
- break;
- }
+ regval = le32_to_cpu(va[2]);
+ regval |= (u64)le32_to_cpu(va[3]) << 32;
+ igc_ptp_systim_to_hwtstamp(adapter, skb_hwtstamps(skb), regval);
+
+ /* Adjust timestamp for the RX latency based on link speed */
+ switch (adapter->link_speed) {
+ case SPEED_10:
+ adjust = IGC_I225_RX_LATENCY_10;
+ break;
+ case SPEED_100:
+ adjust = IGC_I225_RX_LATENCY_100;
+ break;
+ case SPEED_1000:
+ adjust = IGC_I225_RX_LATENCY_1000;
+ break;
+ case SPEED_2500:
+ adjust = IGC_I225_RX_LATENCY_2500;
+ break;
+ default:
+ adjust = 0;
+ netdev_warn_once(adapter->netdev, "Imprecise timestamp\n");
+ break;
}
skb_hwtstamps(skb)->hwtstamp =
ktime_sub_ns(skb_hwtstamps(skb)->hwtstamp, adjust);
#endif
}
+ ring->rx_offset = ixgbe_rx_offset(ring);
+
if (ring->xsk_pool && hw->mac.type != ixgbe_mac_82599EB) {
u32 xsk_buf_len = xsk_pool_get_rx_frame_size(ring->xsk_pool);
rx_ring->next_to_clean = 0;
rx_ring->next_to_use = 0;
- rx_ring->rx_offset = ixgbe_rx_offset(rx_ring);
/* XDP RX-queue info */
if (xdp_rxq_info_reg(&rx_ring->xdp_rxq, adapter->netdev,
config NET_VENDOR_MARVELL
bool "Marvell devices"
default y
- depends on PCI || CPU_PXA168 || MV64X60 || PPC32 || PLAT_ORION || INET || COMPILE_TEST
+ depends on PCI || CPU_PXA168 || PPC32 || PLAT_ORION || INET || COMPILE_TEST
help
If you have a network (Ethernet) card belonging to this class, say Y.
config MV643XX_ETH
tristate "Marvell Discovery (643XX) and Orion ethernet support"
- depends on MV64X60 || PPC32 || PLAT_ORION || COMPILE_TEST
+ depends on PPC32 || PLAT_ORION || COMPILE_TEST
depends on INET
select PHYLIB
select MVMDIO
MODULE_DEVICE_TABLE(of, mv643xx_eth_shared_ids);
#endif
-#if defined(CONFIG_OF_IRQ) && !defined(CONFIG_MV64X60)
+#ifdef CONFIG_OF_IRQ
#define mv643xx_eth_property(_np, _name, _v) \
do { \
u32 tmp; \
[NPC_LT_LC_IP] = {
/* SIP+DIP: 8 bytes, KW2[63:0] */
KEX_LD_CFG(0x07, 0xc, 0x1, 0x0, 0x10),
- /* TOS: 1 byte, KW1[63:56] */
- KEX_LD_CFG(0x0, 0x1, 0x1, 0x0, 0xf),
},
/* Layer C: IPv6 */
[NPC_LT_LC_IP6] = {
INTR_MASK(rvu->hw->total_pfs) & ~1ULL);
for (irq = 0; irq < rvu->num_vec; irq++) {
- if (rvu->irq_allocated[irq])
+ if (rvu->irq_allocated[irq]) {
free_irq(pci_irq_vector(rvu->pdev, irq), rvu);
+ rvu->irq_allocated[irq] = false;
+ }
}
pci_free_irq_vectors(rvu->pdev);
struct rvu *rvu = pci_get_drvdata(pdev);
rvu_dbg_exit(rvu);
- rvu_unregister_interrupts(rvu);
rvu_unregister_dl(rvu);
+ rvu_unregister_interrupts(rvu);
rvu_flr_wq_destroy(rvu);
rvu_cgx_exit(rvu);
rvu_fwdata_exit(rvu);
u8 *intf, u8 *ena);
bool is_mac_feature_supported(struct rvu *rvu, int pf, int feature);
u32 rvu_cgx_get_fifolen(struct rvu *rvu);
+void *rvu_first_cgx_pdata(struct rvu *rvu);
/* CPT APIs */
int rvu_cpt_lf_teardown(struct rvu *rvu, u16 pcifunc, int lf, int slot);
return rvu->cgx_idmap[cgx_id];
}
+/* Return first enabled CGX instance if none are enabled then return NULL */
+void *rvu_first_cgx_pdata(struct rvu *rvu)
+{
+ int first_enabled_cgx = 0;
+ void *cgxd = NULL;
+
+ for (; first_enabled_cgx < rvu->cgx_cnt_max; first_enabled_cgx++) {
+ cgxd = rvu_cgx_pdata(first_enabled_cgx, rvu);
+ if (cgxd)
+ break;
+ }
+
+ return cgxd;
+}
+
/* Based on P2X connectivity find mapped NIX block for a PF */
static void rvu_map_cgx_nix_block(struct rvu *rvu, int pf,
int cgx_id, int lmac_id)
u32 rvu_cgx_get_fifolen(struct rvu *rvu)
{
struct mac_ops *mac_ops;
- int rvu_def_cgx_id = 0;
u32 fifo_len;
- mac_ops = get_mac_ops(rvu_cgx_pdata(rvu_def_cgx_id, rvu));
+ mac_ops = get_mac_ops(rvu_first_cgx_pdata(rvu));
fifo_len = mac_ops ? mac_ops->fifo_len : 0;
return fifo_len;
char __user *buffer,
size_t count, loff_t *ppos)
{
- int index, off = 0, flag = 0, go_back = 0, off_prev;
+ int index, off = 0, flag = 0, go_back = 0, len = 0;
struct rvu *rvu = filp->private_data;
int lf, pf, vf, pcifunc;
struct rvu_block block;
int bytes_not_copied;
+ int lf_str_size = 12;
int buf_size = 2048;
+ char *lfs;
char *buf;
/* don't allow partial reads */
buf = kzalloc(buf_size, GFP_KERNEL);
if (!buf)
return -ENOSPC;
- off += scnprintf(&buf[off], buf_size - 1 - off, "\npcifunc\t\t");
+
+ lfs = kzalloc(lf_str_size, GFP_KERNEL);
+ if (!lfs) {
+ kfree(buf);
+ return -ENOMEM;
+ }
+ off += scnprintf(&buf[off], buf_size - 1 - off, "%-*s", lf_str_size,
+ "pcifunc");
for (index = 0; index < BLK_COUNT; index++)
- if (strlen(rvu->hw->block[index].name))
- off += scnprintf(&buf[off], buf_size - 1 - off,
- "%*s\t", (index - 1) * 2,
- rvu->hw->block[index].name);
+ if (strlen(rvu->hw->block[index].name)) {
+ off += scnprintf(&buf[off], buf_size - 1 - off,
+ "%-*s", lf_str_size,
+ rvu->hw->block[index].name);
+ }
off += scnprintf(&buf[off], buf_size - 1 - off, "\n");
for (pf = 0; pf < rvu->hw->total_pfs; pf++) {
for (vf = 0; vf <= rvu->hw->total_vfs; vf++) {
continue;
if (vf) {
+ sprintf(lfs, "PF%d:VF%d", pf, vf - 1);
go_back = scnprintf(&buf[off],
buf_size - 1 - off,
- "PF%d:VF%d\t\t", pf,
- vf - 1);
+ "%-*s", lf_str_size, lfs);
} else {
+ sprintf(lfs, "PF%d", pf);
go_back = scnprintf(&buf[off],
buf_size - 1 - off,
- "PF%d\t\t", pf);
+ "%-*s", lf_str_size, lfs);
}
off += go_back;
block = rvu->hw->block[index];
if (!strlen(block.name))
continue;
- off_prev = off;
+ len = 0;
+ lfs[len] = '\0';
for (lf = 0; lf < block.lf.max; lf++) {
if (block.fn_map[lf] != pcifunc)
continue;
flag = 1;
- off += scnprintf(&buf[off], buf_size - 1
- - off, "%3d,", lf);
+ len += sprintf(&lfs[len], "%d,", lf);
}
- if (flag && off_prev != off)
- off--;
- else
- go_back++;
+
+ if (flag)
+ len--;
+ lfs[len] = '\0';
off += scnprintf(&buf[off], buf_size - 1 - off,
- "\t");
+ "%-*s", lf_str_size, lfs);
+ if (!strlen(lfs))
+ go_back += lf_str_size;
}
if (!flag)
off -= go_back;
}
bytes_not_copied = copy_to_user(buffer, buf, off);
+ kfree(lfs);
kfree(buf);
if (bytes_not_copied)
struct rvu *rvu = filp->private;
struct pci_dev *pdev = NULL;
struct mac_ops *mac_ops;
- int rvu_def_cgx_id = 0;
char cgx[10], lmac[10];
struct rvu_pfvf *pfvf;
int pf, domain, blkid;
u16 pcifunc;
domain = 2;
- mac_ops = get_mac_ops(rvu_cgx_pdata(rvu_def_cgx_id, rvu));
+ mac_ops = get_mac_ops(rvu_first_cgx_pdata(rvu));
+ /* There can be no CGX devices at all */
+ if (!mac_ops)
+ return 0;
seq_printf(filp, "PCI dev\t\tRVU PF Func\tNIX block\t%s\tLMAC\n",
mac_ops->name);
for (pf = 0; pf < rvu->hw->total_pfs; pf++) {
{
struct mac_ops *mac_ops;
unsigned long lmac_bmap;
- int rvu_def_cgx_id = 0;
int i, lmac_id;
char dname[20];
void *cgx;
if (!cgx_get_cgxcnt_max())
return;
- mac_ops = get_mac_ops(rvu_cgx_pdata(rvu_def_cgx_id, rvu));
+ mac_ops = get_mac_ops(rvu_first_cgx_pdata(rvu));
if (!mac_ops)
return;
struct nix_rx_flowkey_alg *field;
struct nix_rx_flowkey_alg tmp;
u32 key_type, valid_key;
- int l4_key_offset;
+ int l4_key_offset = 0;
if (!alg)
return -EINVAL;
index = find_next_bit(mcam->bmap, mcam->bmap_entries, entry);
if (index >= mcam->bmap_entries)
break;
+ entry = index + 1;
if (mcam->entry2cntr_map[index] != req->cntr)
continue;
- entry = index + 1;
npc_unmap_mcam_entry_and_cntr(rvu, mcam, blkaddr,
index, req->cntr);
}
int otx2_get_all_flows(struct otx2_nic *pfvf, struct ethtool_rxnfc *nfc,
u32 *rule_locs)
{
+ u32 rule_cnt = nfc->rule_cnt;
u32 location = 0;
int idx = 0;
int err = 0;
nfc->data = pfvf->flow_cfg->ntuple_max_flows;
- while ((!err || err == -ENOENT) && idx < nfc->rule_cnt) {
+ while ((!err || err == -ENOENT) && idx < rule_cnt) {
err = otx2_get_flow(pfvf, nfc, location);
if (!err)
rule_locs[idx++] = location;
location++;
}
+ nfc->rule_cnt = rule_cnt;
return err;
}
struct otx2_nic *pf = netdev_priv(netdev);
struct otx2_cq_poll *cq_poll = NULL;
struct otx2_qset *qset = &pf->qset;
+ struct otx2_rss_info *rss;
int qidx, vec, wrk;
netif_carrier_off(netdev);
/* First stop packet Rx/Tx */
otx2_rxtx_enable(pf, false);
+ /* Clear RSS enable flag */
+ rss = &pf->hw.rss_info;
+ rss->enable = false;
+
/* Cleanup Queue IRQ */
vec = pci_irq_vector(pf->pdev,
pf->hw.nix_msixoff + NIX_LF_QINT_VEC_START);
clk_disable_unprepare(pep->clk);
mdiobus_unregister(pep->smi_bus);
mdiobus_free(pep->smi_bus);
- unregister_netdev(dev);
cancel_work_sync(&pep->tx_timeout_task);
+ unregister_netdev(dev);
free_netdev(dev);
return 0;
}
}
enum {
- MLX5_INTERFACE_PROTOCOL_ETH_REP,
MLX5_INTERFACE_PROTOCOL_ETH,
+ MLX5_INTERFACE_PROTOCOL_ETH_REP,
+ MLX5_INTERFACE_PROTOCOL_IB,
MLX5_INTERFACE_PROTOCOL_IB_REP,
MLX5_INTERFACE_PROTOCOL_MPIB,
- MLX5_INTERFACE_PROTOCOL_IB,
MLX5_INTERFACE_PROTOCOL_VNET,
};
MLX5_MPWRQ_LOG_WQE_SZ - PAGE_SHIFT : 0)
#define MLX5_MPWRQ_PAGES_PER_WQE BIT(MLX5_MPWRQ_WQE_PAGE_ORDER)
-#define MLX5_MTT_OCTW(npages) (ALIGN(npages, 8) / 2)
+#define MLX5_ALIGN_MTTS(mtts) (ALIGN(mtts, 8))
+#define MLX5_ALIGNED_MTTS_OCTW(mtts) ((mtts) / 2)
+#define MLX5_MTT_OCTW(mtts) (MLX5_ALIGNED_MTTS_OCTW(MLX5_ALIGN_MTTS(mtts)))
/* Add another page to MLX5E_REQUIRED_WQE_MTTS as a buffer between
* WQEs, This page will absorb write overflow by the hardware, when
* receiving packets larger than MTU. These oversize packets are
* dropped by the driver at a later stage.
*/
-#define MLX5E_REQUIRED_WQE_MTTS (ALIGN(MLX5_MPWRQ_PAGES_PER_WQE + 1, 8))
-#define MLX5E_LOG_ALIGNED_MPWQE_PPW (ilog2(MLX5E_REQUIRED_WQE_MTTS))
+#define MLX5E_REQUIRED_WQE_MTTS (MLX5_ALIGN_MTTS(MLX5_MPWRQ_PAGES_PER_WQE + 1))
#define MLX5E_REQUIRED_MTTS(wqes) (wqes * MLX5E_REQUIRED_WQE_MTTS)
#define MLX5E_MAX_RQ_NUM_MTTS \
((1 << 16) * 2) /* So that MLX5_MTT_OCTW(num_mtts) fits into u16 */
struct mlx5_wq_cyc wq;
void __iomem *uar_map;
u32 sqn;
+ u16 reserved_room;
unsigned long state;
/* control path */
return !!(entry->tuple_nat_node.next);
}
+static int
+mlx5_get_label_mapping(struct mlx5_tc_ct_priv *ct_priv,
+ u32 *labels, u32 *id)
+{
+ if (!memchr_inv(labels, 0, sizeof(u32) * 4)) {
+ *id = 0;
+ return 0;
+ }
+
+ if (mapping_add(ct_priv->labels_mapping, labels, id))
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+static void
+mlx5_put_label_mapping(struct mlx5_tc_ct_priv *ct_priv, u32 id)
+{
+ if (id)
+ mapping_remove(ct_priv->labels_mapping, id);
+}
+
static int
mlx5_tc_ct_rule_to_tuple(struct mlx5_ct_tuple *tuple, struct flow_rule *rule)
{
mlx5_tc_rule_delete(netdev_priv(ct_priv->netdev), zone_rule->rule, attr);
mlx5e_mod_hdr_detach(ct_priv->dev,
ct_priv->mod_hdr_tbl, zone_rule->mh);
- mapping_remove(ct_priv->labels_mapping, attr->ct_attr.ct_labels_id);
+ mlx5_put_label_mapping(ct_priv, attr->ct_attr.ct_labels_id);
kfree(attr);
}
if (!meta)
return -EOPNOTSUPP;
- err = mapping_add(ct_priv->labels_mapping, meta->ct_metadata.labels,
- &attr->ct_attr.ct_labels_id);
+ err = mlx5_get_label_mapping(ct_priv, meta->ct_metadata.labels,
+ &attr->ct_attr.ct_labels_id);
if (err)
return -EOPNOTSUPP;
if (nat) {
err_mapping:
dealloc_mod_hdr_actions(&mod_acts);
- mapping_remove(ct_priv->labels_mapping, attr->ct_attr.ct_labels_id);
+ mlx5_put_label_mapping(ct_priv, attr->ct_attr.ct_labels_id);
return err;
}
err_rule:
mlx5e_mod_hdr_detach(ct_priv->dev,
ct_priv->mod_hdr_tbl, zone_rule->mh);
- mapping_remove(ct_priv->labels_mapping, attr->ct_attr.ct_labels_id);
+ mlx5_put_label_mapping(ct_priv, attr->ct_attr.ct_labels_id);
err_mod_hdr:
kfree(attr);
err_attr:
mlx5e_tc_match_to_reg_get_match(spec, CTSTATE_TO_REG,
&ctstate, &ctstate_mask);
- if (ctstate_mask)
+
+ if ((ctstate & ctstate_mask) == MLX5_CT_STATE_TRK_BIT)
return -EOPNOTSUPP;
ctstate_mask |= MLX5_CT_STATE_TRK_BIT;
if (!priv || !ct_attr->ct_labels_id)
return;
- mapping_remove(priv->labels_mapping, ct_attr->ct_labels_id);
+ mlx5_put_label_mapping(priv, ct_attr->ct_labels_id);
}
int
ct_labels[1] = key->ct_labels[1] & mask->ct_labels[1];
ct_labels[2] = key->ct_labels[2] & mask->ct_labels[2];
ct_labels[3] = key->ct_labels[3] & mask->ct_labels[3];
- if (mapping_add(priv->labels_mapping, ct_labels, &ct_attr->ct_labels_id))
+ if (mlx5_get_label_mapping(priv, ct_labels, &ct_attr->ct_labels_id))
return -EOPNOTSUPP;
mlx5e_tc_match_to_reg_match(spec, LABELS_TO_REG, ct_attr->ct_labels_id,
MLX5_CT_LABELS_MASK);
u16 vport_num;
int err = 0;
- if (flow_attr->ip_version == 4) {
+ if (flow_attr->tun_ip_version == 4) {
/* Addresses are swapped for decap */
attr.fl.fl4.saddr = esw_attr->rx_tun_attr->dst_ip.v4;
attr.fl.fl4.daddr = esw_attr->rx_tun_attr->src_ip.v4;
err = mlx5e_route_lookup_ipv4_get(priv, priv->netdev, &attr);
}
#if IS_ENABLED(CONFIG_INET) && IS_ENABLED(CONFIG_IPV6)
- else if (flow_attr->ip_version == 6) {
+ else if (flow_attr->tun_ip_version == 6) {
/* Addresses are swapped for decap */
attr.fl.fl6.saddr = esw_attr->rx_tun_attr->dst_ip.v6;
attr.fl.fl6.daddr = esw_attr->rx_tun_attr->src_ip.v6;
esw_attr->rx_tun_attr->decap_vport = vport_num;
out:
- if (flow_attr->ip_version == 4)
+ if (flow_attr->tun_ip_version == 4)
mlx5e_route_lookup_ipv4_put(&attr);
#if IS_ENABLED(CONFIG_INET) && IS_ENABLED(CONFIG_IPV6)
- else if (flow_attr->ip_version == 6)
+ else if (flow_attr->tun_ip_version == 6)
mlx5e_route_lookup_ipv6_put(&attr);
#endif
return err;
MLX5E_TC_TUNNEL_TYPE_MPLSOUDP,
};
+struct mlx5e_encap_key {
+ const struct ip_tunnel_key *ip_tun_key;
+ struct mlx5e_tc_tunnel *tc_tunnel;
+};
+
struct mlx5e_tc_tunnel {
int tunnel_type;
enum mlx5_flow_match_level match_level;
struct flow_cls_offload *f,
void *headers_c,
void *headers_v);
+ bool (*encap_info_equal)(struct mlx5e_encap_key *a,
+ struct mlx5e_encap_key *b);
};
extern struct mlx5e_tc_tunnel vxlan_tunnel;
void *headers_c,
void *headers_v);
+bool mlx5e_tc_tun_encap_info_equal_generic(struct mlx5e_encap_key *a,
+ struct mlx5e_encap_key *b);
+
#endif /* CONFIG_MLX5_ESWITCH */
#endif //__MLX5_EN_TC_TUNNEL_H__
* required to establish routing.
*/
flow_flag_set(flow, TUN_RX);
+ flow->attr->tun_ip_version = ip_version;
return 0;
}
mlx5e_decap_dealloc(priv, d);
}
-struct encap_key {
- const struct ip_tunnel_key *ip_tun_key;
- struct mlx5e_tc_tunnel *tc_tunnel;
-};
-
-static int cmp_encap_info(struct encap_key *a,
- struct encap_key *b)
+bool mlx5e_tc_tun_encap_info_equal_generic(struct mlx5e_encap_key *a,
+ struct mlx5e_encap_key *b)
{
- return memcmp(a->ip_tun_key, b->ip_tun_key, sizeof(*a->ip_tun_key)) ||
- a->tc_tunnel->tunnel_type != b->tc_tunnel->tunnel_type;
+ return memcmp(a->ip_tun_key, b->ip_tun_key, sizeof(*a->ip_tun_key)) == 0 &&
+ a->tc_tunnel->tunnel_type == b->tc_tunnel->tunnel_type;
}
static int cmp_decap_info(struct mlx5e_decap_key *a,
return memcmp(&a->key, &b->key, sizeof(b->key));
}
-static int hash_encap_info(struct encap_key *key)
+static int hash_encap_info(struct mlx5e_encap_key *key)
{
return jhash(key->ip_tun_key, sizeof(*key->ip_tun_key),
key->tc_tunnel->tunnel_type);
}
static struct mlx5e_encap_entry *
-mlx5e_encap_get(struct mlx5e_priv *priv, struct encap_key *key,
+mlx5e_encap_get(struct mlx5e_priv *priv, struct mlx5e_encap_key *key,
uintptr_t hash_key)
{
struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
+ struct mlx5e_encap_key e_key;
struct mlx5e_encap_entry *e;
- struct encap_key e_key;
hash_for_each_possible_rcu(esw->offloads.encap_tbl, e,
encap_hlist, hash_key) {
e_key.ip_tun_key = &e->tun_info->key;
e_key.tc_tunnel = e->tunnel;
- if (!cmp_encap_info(&e_key, key) &&
+ if (e->tunnel->encap_info_equal(&e_key, key) &&
mlx5e_encap_take(e))
return e;
}
struct mlx5_flow_attr *attr = flow->attr;
const struct ip_tunnel_info *tun_info;
unsigned long tbl_time_before = 0;
- struct encap_key key;
struct mlx5e_encap_entry *e;
+ struct mlx5e_encap_key key;
bool entry_created = false;
unsigned short family;
uintptr_t hash_key;
if (err || !esw_attr->rx_tun_attr->decap_vport)
goto out;
- key.ip_version = attr->ip_version;
+ key.ip_version = attr->tun_ip_version;
if (key.ip_version == 4)
key.endpoint_ip.v4 = esw_attr->rx_tun_attr->dst_ip.v4;
else
option_key = (struct geneve_opt *)&enc_opts.key->data[0];
option_mask = (struct geneve_opt *)&enc_opts.mask->data[0];
+ if (option_mask->opt_class == 0 && option_mask->type == 0 &&
+ !memchr_inv(option_mask->opt_data, 0, option_mask->length * 4))
+ return 0;
+
if (option_key->length > max_tlv_option_data_len) {
NL_SET_ERR_MSG_MOD(extack,
"Matching on GENEVE options: unsupported option len");
return mlx5e_tc_tun_parse_geneve_options(priv, spec, f);
}
+static bool mlx5e_tc_tun_encap_info_equal_geneve(struct mlx5e_encap_key *a,
+ struct mlx5e_encap_key *b)
+{
+ struct ip_tunnel_info *a_info;
+ struct ip_tunnel_info *b_info;
+ bool a_has_opts, b_has_opts;
+
+ if (!mlx5e_tc_tun_encap_info_equal_generic(a, b))
+ return false;
+
+ a_has_opts = !!(a->ip_tun_key->tun_flags & TUNNEL_GENEVE_OPT);
+ b_has_opts = !!(b->ip_tun_key->tun_flags & TUNNEL_GENEVE_OPT);
+
+ /* keys are equal when both don't have any options attached */
+ if (!a_has_opts && !b_has_opts)
+ return true;
+
+ if (a_has_opts != b_has_opts)
+ return false;
+
+ /* geneve options stored in memory next to ip_tunnel_info struct */
+ a_info = container_of(a->ip_tun_key, struct ip_tunnel_info, key);
+ b_info = container_of(b->ip_tun_key, struct ip_tunnel_info, key);
+
+ return a_info->options_len == b_info->options_len &&
+ memcmp(a_info + 1, b_info + 1, a_info->options_len) == 0;
+}
+
struct mlx5e_tc_tunnel geneve_tunnel = {
.tunnel_type = MLX5E_TC_TUNNEL_TYPE_GENEVE,
.match_level = MLX5_MATCH_L4,
.generate_ip_tun_hdr = mlx5e_gen_ip_tunnel_header_geneve,
.parse_udp_ports = mlx5e_tc_tun_parse_udp_ports_geneve,
.parse_tunnel = mlx5e_tc_tun_parse_geneve,
+ .encap_info_equal = mlx5e_tc_tun_encap_info_equal_geneve,
};
.generate_ip_tun_hdr = mlx5e_gen_ip_tunnel_header_gretap,
.parse_udp_ports = NULL,
.parse_tunnel = mlx5e_tc_tun_parse_gretap,
+ .encap_info_equal = mlx5e_tc_tun_encap_info_equal_generic,
};
.generate_ip_tun_hdr = generate_ip_tun_hdr,
.parse_udp_ports = parse_udp_ports,
.parse_tunnel = parse_tunnel,
+ .encap_info_equal = mlx5e_tc_tun_encap_info_equal_generic,
};
.generate_ip_tun_hdr = mlx5e_gen_ip_tunnel_header_vxlan,
.parse_udp_ports = mlx5e_tc_tun_parse_udp_ports_vxlan,
.parse_tunnel = mlx5e_tc_tun_parse_vxlan,
+ .encap_info_equal = mlx5e_tc_tun_encap_info_equal_generic,
};
return wqe_size * 2 - 1;
}
+static inline bool mlx5e_icosq_can_post_wqe(struct mlx5e_icosq *sq, u16 wqe_size)
+{
+ u16 room = sq->reserved_room + mlx5e_stop_room_for_wqe(wqe_size);
+
+ return mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room);
+}
#endif
struct tls12_crypto_info_aes_gcm_128 crypto_info;
struct accel_rule rule;
struct sock *sk;
- struct mlx5e_rq_stats *stats;
+ struct mlx5e_rq_stats *rq_stats;
+ struct mlx5e_tls_sw_stats *sw_stats;
struct completion add_ctx;
u32 tirn;
u32 key_id;
{
struct mlx5e_set_tls_static_params_wqe *wqe;
struct mlx5e_icosq_wqe_info wi;
- u16 pi, num_wqebbs, room;
+ u16 pi, num_wqebbs;
num_wqebbs = MLX5E_TLS_SET_STATIC_PARAMS_WQEBBS;
- room = mlx5e_stop_room_for_wqe(num_wqebbs);
- if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room)))
+ if (unlikely(!mlx5e_icosq_can_post_wqe(sq, num_wqebbs)))
return ERR_PTR(-ENOSPC);
pi = mlx5e_icosq_get_next_pi(sq, num_wqebbs);
{
struct mlx5e_set_tls_progress_params_wqe *wqe;
struct mlx5e_icosq_wqe_info wi;
- u16 pi, num_wqebbs, room;
+ u16 pi, num_wqebbs;
num_wqebbs = MLX5E_TLS_SET_PROGRESS_PARAMS_WQEBBS;
- room = mlx5e_stop_room_for_wqe(num_wqebbs);
- if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room)))
+ if (unlikely(!mlx5e_icosq_can_post_wqe(sq, num_wqebbs)))
return ERR_PTR(-ENOSPC);
pi = mlx5e_icosq_get_next_pi(sq, num_wqebbs);
return err;
err_out:
- priv_rx->stats->tls_resync_req_skip++;
+ priv_rx->rq_stats->tls_resync_req_skip++;
err = PTR_ERR(cseg);
complete(&priv_rx->add_ctx);
goto unlock;
buf->priv_rx = priv_rx;
- BUILD_BUG_ON(MLX5E_KTLS_GET_PROGRESS_WQEBBS != 1);
-
spin_lock_bh(&sq->channel->async_icosq_lock);
- if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, 1))) {
+ if (unlikely(!mlx5e_icosq_can_post_wqe(sq, MLX5E_KTLS_GET_PROGRESS_WQEBBS))) {
spin_unlock_bh(&sq->channel->async_icosq_lock);
err = -ENOSPC;
goto err_dma_unmap;
}
- pi = mlx5e_icosq_get_next_pi(sq, 1);
+ pi = mlx5e_icosq_get_next_pi(sq, MLX5E_KTLS_GET_PROGRESS_WQEBBS);
wqe = MLX5E_TLS_FETCH_GET_PROGRESS_PARAMS_WQE(sq, pi);
#define GET_PSV_DS_CNT (DIV_ROUND_UP(sizeof(*wqe), MLX5_SEND_WQE_DS))
wi = (struct mlx5e_icosq_wqe_info) {
.wqe_type = MLX5E_ICOSQ_WQE_GET_PSV_TLS,
- .num_wqebbs = 1,
+ .num_wqebbs = MLX5E_KTLS_GET_PROGRESS_WQEBBS,
.tls_get_params.buf = buf,
};
icosq_fill_wi(sq, pi, &wi);
err_free:
kfree(buf);
err_out:
- priv_rx->stats->tls_resync_req_skip++;
+ priv_rx->rq_stats->tls_resync_req_skip++;
return err;
}
cseg = post_static_params(sq, priv_rx);
if (IS_ERR(cseg)) {
- priv_rx->stats->tls_resync_res_skip++;
+ priv_rx->rq_stats->tls_resync_res_skip++;
err = PTR_ERR(cseg);
goto unlock;
}
/* Do not increment priv_rx refcnt, CQE handling is empty */
mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, cseg);
- priv_rx->stats->tls_resync_res_ok++;
+ priv_rx->rq_stats->tls_resync_res_ok++;
unlock:
spin_unlock_bh(&c->async_icosq_lock);
auth_state = MLX5_GET(tls_progress_params, ctx, auth_state);
if (tracker_state != MLX5E_TLS_PROGRESS_PARAMS_RECORD_TRACKER_STATE_TRACKING ||
auth_state != MLX5E_TLS_PROGRESS_PARAMS_AUTH_STATE_NO_OFFLOAD) {
- priv_rx->stats->tls_resync_req_skip++;
+ priv_rx->rq_stats->tls_resync_req_skip++;
goto out;
}
hw_seq = MLX5_GET(tls_progress_params, ctx, hw_resync_tcp_sn);
tls_offload_rx_resync_async_request_end(priv_rx->sk, cpu_to_be32(hw_seq));
- priv_rx->stats->tls_resync_req_end++;
+ priv_rx->rq_stats->tls_resync_req_end++;
out:
mlx5e_ktls_priv_rx_put(priv_rx);
dma_unmap_single(dev, buf->dma_addr, PROGRESS_PARAMS_PADDED_SIZE, DMA_FROM_DEVICE);
priv_rx->rxq = rxq;
priv_rx->sk = sk;
- priv_rx->stats = &priv->channel_stats[rxq].rq;
+ priv_rx->rq_stats = &priv->channel_stats[rxq].rq;
+ priv_rx->sw_stats = &priv->tls->sw_stats;
mlx5e_set_ktls_rx_priv_ctx(tls_ctx, priv_rx);
rqtn = priv->direct_tir[rxq].rqt.rqtn;
if (err)
goto err_post_wqes;
- priv_rx->stats->tls_ctx++;
+ atomic64_inc(&priv_rx->sw_stats->rx_tls_ctx);
return 0;
if (cancel_work_sync(&resync->work))
mlx5e_ktls_priv_rx_put(priv_rx);
- priv_rx->stats->tls_del++;
+ atomic64_inc(&priv_rx->sw_stats->rx_tls_del);
if (priv_rx->rule.rule)
mlx5e_accel_fs_del_sk(priv_rx->rule.rule);
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
// Copyright (c) 2019 Mellanox Technologies.
+#include "en_accel/tls.h"
#include "en_accel/ktls_txrx.h"
#include "en_accel/ktls_utils.h"
struct mlx5e_ktls_offload_context_tx {
struct tls_offload_context_tx *tx_ctx;
struct tls12_crypto_info_aes_gcm_128 crypto_info;
+ struct mlx5e_tls_sw_stats *sw_stats;
u32 expected_seq;
u32 tisn;
u32 key_id;
if (err)
goto err_create_key;
+ priv_tx->sw_stats = &priv->tls->sw_stats;
priv_tx->expected_seq = start_offload_tcp_sn;
priv_tx->crypto_info =
*(struct tls12_crypto_info_aes_gcm_128 *)crypto_info;
goto err_create_tis;
priv_tx->ctx_post_pending = true;
+ atomic64_inc(&priv_tx->sw_stats->tx_tls_ctx);
return 0;
if (unlikely(mlx5e_ktls_tx_offload_test_and_clear_pending(priv_tx))) {
mlx5e_ktls_tx_post_param_wqes(sq, priv_tx, false, false);
- stats->tls_ctx++;
}
seq = ntohl(tcp_hdr(skb)->seq);
#include "en.h"
struct mlx5e_tls_sw_stats {
+ atomic64_t tx_tls_ctx;
atomic64_t tx_tls_drop_metadata;
atomic64_t tx_tls_drop_resync_alloc;
atomic64_t tx_tls_drop_no_sync_data;
atomic64_t tx_tls_drop_bypass_required;
+ atomic64_t rx_tls_ctx;
+ atomic64_t rx_tls_del;
atomic64_t rx_tls_drop_resync_request;
atomic64_t rx_tls_resync_request;
atomic64_t rx_tls_resync_reply;
{ MLX5E_DECLARE_STAT(struct mlx5e_tls_sw_stats, tx_tls_drop_bypass_required) },
};
+static const struct counter_desc mlx5e_ktls_sw_stats_desc[] = {
+ { MLX5E_DECLARE_STAT(struct mlx5e_tls_sw_stats, tx_tls_ctx) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_tls_sw_stats, rx_tls_ctx) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_tls_sw_stats, rx_tls_del) },
+};
+
#define MLX5E_READ_CTR_ATOMIC64(ptr, dsc, i) \
atomic64_read((atomic64_t *)((char *)(ptr) + (dsc)[i].offset))
-#define NUM_TLS_SW_COUNTERS ARRAY_SIZE(mlx5e_tls_sw_stats_desc)
-
-static bool is_tls_atomic_stats(struct mlx5e_priv *priv)
+static const struct counter_desc *get_tls_atomic_stats(struct mlx5e_priv *priv)
{
- return priv->tls && !mlx5_accel_is_ktls_device(priv->mdev);
+ if (!priv->tls)
+ return NULL;
+ if (mlx5_accel_is_ktls_device(priv->mdev))
+ return mlx5e_ktls_sw_stats_desc;
+ return mlx5e_tls_sw_stats_desc;
}
int mlx5e_tls_get_count(struct mlx5e_priv *priv)
{
- if (!is_tls_atomic_stats(priv))
+ if (!priv->tls)
return 0;
-
- return NUM_TLS_SW_COUNTERS;
+ if (mlx5_accel_is_ktls_device(priv->mdev))
+ return ARRAY_SIZE(mlx5e_ktls_sw_stats_desc);
+ return ARRAY_SIZE(mlx5e_tls_sw_stats_desc);
}
int mlx5e_tls_get_strings(struct mlx5e_priv *priv, uint8_t *data)
{
- unsigned int i, idx = 0;
+ const struct counter_desc *stats_desc;
+ unsigned int i, n, idx = 0;
- if (!is_tls_atomic_stats(priv))
- return 0;
+ stats_desc = get_tls_atomic_stats(priv);
+ n = mlx5e_tls_get_count(priv);
- for (i = 0; i < NUM_TLS_SW_COUNTERS; i++)
+ for (i = 0; i < n; i++)
strcpy(data + (idx++) * ETH_GSTRING_LEN,
- mlx5e_tls_sw_stats_desc[i].format);
+ stats_desc[i].format);
- return NUM_TLS_SW_COUNTERS;
+ return n;
}
int mlx5e_tls_get_stats(struct mlx5e_priv *priv, u64 *data)
{
- int i, idx = 0;
+ const struct counter_desc *stats_desc;
+ unsigned int i, n, idx = 0;
- if (!is_tls_atomic_stats(priv))
- return 0;
+ stats_desc = get_tls_atomic_stats(priv);
+ n = mlx5e_tls_get_count(priv);
- for (i = 0; i < NUM_TLS_SW_COUNTERS; i++)
+ for (i = 0; i < n; i++)
data[idx++] =
MLX5E_READ_CTR_ATOMIC64(&priv->tls->sw_stats,
- mlx5e_tls_sw_stats_desc, i);
+ stats_desc, i);
- return NUM_TLS_SW_COUNTERS;
+ return n;
}
return 0;
}
-static void ptys2ethtool_supported_advertised_port(struct ethtool_link_ksettings *link_ksettings,
- u32 eth_proto_cap,
- u8 connector_type, bool ext)
+static void ptys2ethtool_supported_advertised_port(struct mlx5_core_dev *mdev,
+ struct ethtool_link_ksettings *link_ksettings,
+ u32 eth_proto_cap, u8 connector_type)
{
- if ((!connector_type && !ext) || connector_type >= MLX5E_CONNECTOR_TYPE_NUMBER) {
+ if (!MLX5_CAP_PCAM_FEATURE(mdev, ptys_connector_type)) {
if (eth_proto_cap & (MLX5E_PROT_MASK(MLX5E_10GBASE_CR)
| MLX5E_PROT_MASK(MLX5E_10GBASE_SR)
| MLX5E_PROT_MASK(MLX5E_40GBASE_CR4)
[MLX5E_PORT_OTHER] = PORT_OTHER,
};
-static u8 get_connector_port(u32 eth_proto, u8 connector_type, bool ext)
+static u8 get_connector_port(struct mlx5_core_dev *mdev, u32 eth_proto, u8 connector_type)
{
- if ((connector_type || ext) && connector_type < MLX5E_CONNECTOR_TYPE_NUMBER)
+ if (MLX5_CAP_PCAM_FEATURE(mdev, ptys_connector_type))
return ptys2connector_type[connector_type];
if (eth_proto &
data_rate_oper, link_ksettings);
eth_proto_oper = eth_proto_oper ? eth_proto_oper : eth_proto_cap;
-
- link_ksettings->base.port = get_connector_port(eth_proto_oper,
- connector_type, ext);
- ptys2ethtool_supported_advertised_port(link_ksettings, eth_proto_admin,
- connector_type, ext);
+ connector_type = connector_type < MLX5E_CONNECTOR_TYPE_NUMBER ?
+ connector_type : MLX5E_PORT_UNKNOWN;
+ link_ksettings->base.port = get_connector_port(mdev, eth_proto_oper, connector_type);
+ ptys2ethtool_supported_advertised_port(mdev, link_ksettings, eth_proto_admin,
+ connector_type);
get_lp_advertising(mdev, eth_proto_lp, link_ksettings);
if (an_status == MLX5_AN_COMPLETE)
{
struct mlx5e_priv *priv = netdev_priv(netdev);
struct mlx5_core_dev *mdev = priv->mdev;
+ int err;
if (!MLX5_CAP_GEN(mdev, cqe_compression))
return -EOPNOTSUPP;
return -EINVAL;
}
- mlx5e_modify_rx_cqe_compression_locked(priv, enable);
+ err = mlx5e_modify_rx_cqe_compression_locked(priv, enable);
+ if (err)
+ return err;
+
priv->channels.params.rx_cqe_compress_def = enable;
return 0;
*/
if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) {
+ struct mlx5e_params old_params;
+
+ old_params = priv->channels.params;
priv->channels.params = new_channels.params;
err = mlx5e_num_channels_changed(priv);
+ if (err)
+ priv->channels.params = old_params;
goto out;
}
rq->wqe_overflow.addr);
}
-static inline u64 mlx5e_get_mpwqe_offset(struct mlx5e_rq *rq, u16 wqe_ix)
+static u64 mlx5e_get_mpwqe_offset(u16 wqe_ix)
{
- return (wqe_ix << MLX5E_LOG_ALIGNED_MPWQE_PPW) << PAGE_SHIFT;
+ return MLX5E_REQUIRED_MTTS(wqe_ix) << PAGE_SHIFT;
}
static void mlx5e_init_frags_partition(struct mlx5e_rq *rq)
mlx5_wq_ll_get_wqe(&rq->mpwqe.wq, i);
u32 byte_count =
rq->mpwqe.num_strides << rq->mpwqe.log_stride_sz;
- u64 dma_offset = mlx5e_get_mpwqe_offset(rq, i);
+ u64 dma_offset = mlx5e_get_mpwqe_offset(i);
wqe->data[0].addr = cpu_to_be64(dma_offset + rq->buff.headroom);
wqe->data[0].byte_count = cpu_to_be32(byte_count);
sq->channel = c;
sq->uar_map = mdev->mlx5e_res.bfreg.map;
+ sq->reserved_room = param->stop_room;
param->wq.db_numa_node = cpu_to_node(c->cpu);
err = mlx5_wq_cyc_create(mdev, ¶m->wq, sqc_wq, wq, &sq->wq_ctrl);
mlx5e_build_ico_cq_param(priv, log_wq_size, ¶m->cqp);
}
+static void mlx5e_build_async_icosq_param(struct mlx5e_priv *priv,
+ struct mlx5e_params *params,
+ u8 log_wq_size,
+ struct mlx5e_sq_param *param)
+{
+ void *sqc = param->sqc;
+ void *wq = MLX5_ADDR_OF(sqc, sqc, wq);
+
+ mlx5e_build_sq_param_common(priv, param);
+
+ /* async_icosq is used by XSK only if xdp_prog is active */
+ if (params->xdp_prog)
+ param->stop_room = mlx5e_stop_room_for_wqe(1); /* for XSK NOP */
+ MLX5_SET(sqc, sqc, reg_umr, MLX5_CAP_ETH(priv->mdev, reg_umr_sq));
+ MLX5_SET(wq, wq, log_wq_sz, log_wq_size);
+ mlx5e_build_ico_cq_param(priv, log_wq_size, ¶m->cqp);
+}
+
void mlx5e_build_xdpsq_param(struct mlx5e_priv *priv,
struct mlx5e_params *params,
struct mlx5e_sq_param *param)
{
switch (params->rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
- return order_base_2(MLX5E_UMR_WQEBBS) +
- mlx5e_get_rq_log_wq_sz(rqp->rqc);
+ return max_t(u8, MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE,
+ order_base_2(MLX5E_UMR_WQEBBS) +
+ mlx5e_get_rq_log_wq_sz(rqp->rqc));
default: /* MLX5_WQ_TYPE_CYCLIC */
return MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE;
}
mlx5e_build_sq_param(priv, params, &cparam->txq_sq);
mlx5e_build_xdpsq_param(priv, params, &cparam->xdp_sq);
mlx5e_build_icosq_param(priv, icosq_log_wq_sz, &cparam->icosq);
- mlx5e_build_icosq_param(priv, async_icosq_log_wq_sz, &cparam->async_icosq);
+ mlx5e_build_async_icosq_param(priv, params, async_icosq_log_wq_sz, &cparam->async_icosq);
}
int mlx5e_open_channels(struct mlx5e_priv *priv,
{
int i;
- if (chs->port_ptp)
+ if (chs->port_ptp) {
mlx5e_port_ptp_close(chs->port_ptp);
+ chs->port_ptp = NULL;
+ }
for (i = 0; i < chs->num; i++)
mlx5e_close_channel(chs->c[i]);
for (j = 0; j < priv->max_opened_tc; j++) {
struct mlx5e_sq_stats *sq_stats = &channel_stats->sq[j];
+ s->tx_packets += sq_stats->packets;
+ s->tx_bytes += sq_stats->bytes;
+ s->tx_dropped += sq_stats->dropped;
+ }
+ }
+ if (priv->port_ptp_opened) {
+ for (i = 0; i < priv->max_opened_tc; i++) {
+ struct mlx5e_sq_stats *sq_stats = &priv->port_ptp_stats.sq[i];
+
s->tx_packets += sq_stats->packets;
s->tx_bytes += sq_stats->bytes;
s->tx_dropped += sq_stats->dropped;
}
if (mlx5e_is_uplink_rep(priv)) {
+ struct mlx5e_vport_stats *vstats = &priv->stats.vport;
+
stats->rx_packets = PPORT_802_3_GET(pstats, a_frames_received_ok);
stats->rx_bytes = PPORT_802_3_GET(pstats, a_octets_received_ok);
stats->tx_packets = PPORT_802_3_GET(pstats, a_frames_transmitted_ok);
stats->tx_bytes = PPORT_802_3_GET(pstats, a_octets_transmitted_ok);
+
+ /* vport multicast also counts packets that are dropped due to steering
+ * or rx out of buffer
+ */
+ stats->multicast = VPORT_COUNTER_GET(vstats, received_eth_multicast.packets);
} else {
mlx5e_fold_sw_stats64(priv, stats);
}
struct mlx5e_channel *c = priv->channels.c[i];
mlx5e_rq_replace_xdp_prog(&c->rq, prog);
- if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state))
+ if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)) {
+ bpf_prog_inc(prog);
mlx5e_rq_replace_xdp_prog(&c->xskrq, prog);
+ }
}
unlock:
priv->max_nch);
params->num_tc = 1;
+ /* Set an initial non-zero value, so that mlx5e_select_queue won't
+ * divide by zero if called before first activating channels.
+ */
+ priv->num_tc_x_num_ch = params->num_channels * params->num_tc;
+
/* SQ */
params->log_sq_size = is_kdump_kernel() ?
MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE :
struct net_device *netdev,
struct mlx5_core_dev *mdev)
{
- memset(priv, 0, sizeof(*priv));
-
/* priv init */
priv->mdev = mdev;
priv->netdev = netdev;
{
int i;
+ /* bail if change profile failed and also rollback failed */
+ if (!priv->mdev)
+ return;
+
destroy_workqueue(priv->wq);
free_cpumask_var(priv->scratchpad.cpumask);
for (i = 0; i < priv->htb.max_qos_sqs; i++)
kfree(priv->htb.qos_sq_stats[i]);
kvfree(priv->htb.qos_sq_stats);
+
+ memset(priv, 0, sizeof(*priv));
}
struct net_device *
}
static int
-mlx5e_netdev_attach_profile(struct mlx5e_priv *priv,
+mlx5e_netdev_attach_profile(struct net_device *netdev, struct mlx5_core_dev *mdev,
const struct mlx5e_profile *new_profile, void *new_ppriv)
{
- struct net_device *netdev = priv->netdev;
- struct mlx5_core_dev *mdev = priv->mdev;
+ struct mlx5e_priv *priv = netdev_priv(netdev);
int err;
err = mlx5e_priv_init(priv, netdev, mdev);
priv->ppriv = new_ppriv;
err = new_profile->init(priv->mdev, priv->netdev);
if (err)
- return err;
+ goto priv_cleanup;
err = mlx5e_attach_netdev(priv);
if (err)
- new_profile->cleanup(priv);
+ goto profile_cleanup;
+ return err;
+
+profile_cleanup:
+ new_profile->cleanup(priv);
+priv_cleanup:
+ mlx5e_priv_cleanup(priv);
return err;
}
{
unsigned int new_max_nch = mlx5e_calc_max_nch(priv, new_profile);
const struct mlx5e_profile *orig_profile = priv->profile;
+ struct net_device *netdev = priv->netdev;
+ struct mlx5_core_dev *mdev = priv->mdev;
void *orig_ppriv = priv->ppriv;
int err, rollback_err;
/* sanity */
if (new_max_nch != priv->max_nch) {
- netdev_warn(priv->netdev,
- "%s: Replacing profile with different max channels\n",
+ netdev_warn(netdev, "%s: Replacing profile with different max channels\n",
__func__);
return -EINVAL;
}
priv->profile->cleanup(priv);
mlx5e_priv_cleanup(priv);
- err = mlx5e_netdev_attach_profile(priv, new_profile, new_ppriv);
+ err = mlx5e_netdev_attach_profile(netdev, mdev, new_profile, new_ppriv);
if (err) { /* roll back to original profile */
- netdev_warn(priv->netdev, "%s: new profile init failed, %d\n",
- __func__, err);
+ netdev_warn(netdev, "%s: new profile init failed, %d\n", __func__, err);
goto rollback;
}
return 0;
rollback:
- rollback_err = mlx5e_netdev_attach_profile(priv, orig_profile, orig_ppriv);
- if (rollback_err) {
- netdev_err(priv->netdev,
- "%s: failed to rollback to orig profile, %d\n",
+ rollback_err = mlx5e_netdev_attach_profile(netdev, mdev, orig_profile, orig_ppriv);
+ if (rollback_err)
+ netdev_err(netdev, "%s: failed to rollback to orig profile, %d\n",
__func__, rollback_err);
- }
return err;
}
mlx5e_rep_tc_enable(priv);
- mlx5_modify_vport_admin_state(mdev, MLX5_VPORT_STATE_OP_MOD_UPLINK,
- 0, 0, MLX5_VPORT_ADMIN_STATE_AUTO);
+ if (MLX5_CAP_GEN(mdev, uplink_follow))
+ mlx5_modify_vport_admin_state(mdev, MLX5_VPORT_STATE_OP_MOD_UPLINK,
+ 0, 0, MLX5_VPORT_ADMIN_STATE_AUTO);
mlx5_lag_add(mdev, netdev);
priv->events_nb.notifier_call = uplink_rep_async_event;
mlx5_notifier_register(mdev, &priv->events_nb);
struct mlx5e_icosq *sq = rq->icosq;
struct mlx5_wq_cyc *wq = &sq->wq;
struct mlx5e_umr_wqe *umr_wqe;
- u16 xlt_offset = ix << (MLX5E_LOG_ALIGNED_MPWQE_PPW - 1);
u16 pi;
int err;
int i;
umr_wqe->ctrl.opmod_idx_opcode =
cpu_to_be32((sq->pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
MLX5_OPCODE_UMR);
- umr_wqe->uctrl.xlt_offset = cpu_to_be16(xlt_offset);
+ umr_wqe->uctrl.xlt_offset =
+ cpu_to_be16(MLX5_ALIGNED_MTTS_OCTW(MLX5E_REQUIRED_MTTS(ix)));
sq->db.wqe_info[pi] = (struct mlx5e_icosq_wqe_info) {
.wqe_type = MLX5E_ICOSQ_WQE_UMR_RX,
#ifdef CONFIG_MLX5_EN_TLS
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_tls_encrypted_packets) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_tls_encrypted_bytes) },
- { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_tls_ctx) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_tls_ooo) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_tls_dump_packets) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_tls_dump_bytes) },
#ifdef CONFIG_MLX5_EN_TLS
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_decrypted_packets) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_decrypted_bytes) },
- { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_ctx) },
- { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_del) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_resync_req_pkt) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_resync_req_start) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_resync_req_end) },
#ifdef CONFIG_MLX5_EN_TLS
s->rx_tls_decrypted_packets += rq_stats->tls_decrypted_packets;
s->rx_tls_decrypted_bytes += rq_stats->tls_decrypted_bytes;
- s->rx_tls_ctx += rq_stats->tls_ctx;
- s->rx_tls_del += rq_stats->tls_del;
s->rx_tls_resync_req_pkt += rq_stats->tls_resync_req_pkt;
s->rx_tls_resync_req_start += rq_stats->tls_resync_req_start;
s->rx_tls_resync_req_end += rq_stats->tls_resync_req_end;
#ifdef CONFIG_MLX5_EN_TLS
s->tx_tls_encrypted_packets += sq_stats->tls_encrypted_packets;
s->tx_tls_encrypted_bytes += sq_stats->tls_encrypted_bytes;
- s->tx_tls_ctx += sq_stats->tls_ctx;
s->tx_tls_ooo += sq_stats->tls_ooo;
s->tx_tls_dump_bytes += sq_stats->tls_dump_bytes;
s->tx_tls_dump_packets += sq_stats->tls_dump_packets;
#ifdef CONFIG_MLX5_EN_TLS
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_decrypted_packets) },
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_decrypted_bytes) },
- { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_ctx) },
- { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_del) },
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_resync_req_pkt) },
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_resync_req_start) },
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_resync_req_end) },
#ifdef CONFIG_MLX5_EN_TLS
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, tls_encrypted_packets) },
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, tls_encrypted_bytes) },
- { MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, tls_ctx) },
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, tls_ooo) },
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, tls_dump_packets) },
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, tls_dump_bytes) },
#ifdef CONFIG_MLX5_EN_TLS
{ MLX5E_DECLARE_QOS_TX_STAT(struct mlx5e_sq_stats, tls_encrypted_packets) },
{ MLX5E_DECLARE_QOS_TX_STAT(struct mlx5e_sq_stats, tls_encrypted_bytes) },
- { MLX5E_DECLARE_QOS_TX_STAT(struct mlx5e_sq_stats, tls_ctx) },
{ MLX5E_DECLARE_QOS_TX_STAT(struct mlx5e_sq_stats, tls_ooo) },
{ MLX5E_DECLARE_QOS_TX_STAT(struct mlx5e_sq_stats, tls_dump_packets) },
{ MLX5E_DECLARE_QOS_TX_STAT(struct mlx5e_sq_stats, tls_dump_bytes) },
#ifdef CONFIG_MLX5_EN_TLS
u64 tx_tls_encrypted_packets;
u64 tx_tls_encrypted_bytes;
- u64 tx_tls_ctx;
u64 tx_tls_ooo;
u64 tx_tls_dump_packets;
u64 tx_tls_dump_bytes;
u64 rx_tls_decrypted_packets;
u64 rx_tls_decrypted_bytes;
- u64 rx_tls_ctx;
- u64 rx_tls_del;
u64 rx_tls_resync_req_pkt;
u64 rx_tls_resync_req_start;
u64 rx_tls_resync_req_end;
#ifdef CONFIG_MLX5_EN_TLS
u64 tls_decrypted_packets;
u64 tls_decrypted_bytes;
- u64 tls_ctx;
- u64 tls_del;
u64 tls_resync_req_pkt;
u64 tls_resync_req_start;
u64 tls_resync_req_end;
#ifdef CONFIG_MLX5_EN_TLS
u64 tls_encrypted_packets;
u64 tls_encrypted_bytes;
- u64 tls_ctx;
u64 tls_ooo;
u64 tls_dump_packets;
u64 tls_dump_bytes;
*match_level = MLX5_MATCH_L4;
}
+ /* Currenlty supported only for MPLS over UDP */
+ if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_MPLS) &&
+ !netif_is_bareudp(filter_dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Matching on MPLS is supported only for MPLS over UDP");
+ netdev_err(priv->netdev,
+ "Matching on MPLS is supported only for MPLS over UDP\n");
+ return -EOPNOTSUPP;
+ }
+
return 0;
}
return 0;
}
+static bool modify_tuple_supported(bool modify_tuple, bool ct_clear,
+ bool ct_flow, struct netlink_ext_ack *extack,
+ struct mlx5e_priv *priv,
+ struct mlx5_flow_spec *spec)
+{
+ if (!modify_tuple || ct_clear)
+ return true;
+
+ if (ct_flow) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "can't offload tuple modification with non-clear ct()");
+ netdev_info(priv->netdev,
+ "can't offload tuple modification with non-clear ct()");
+ return false;
+ }
+
+ /* Add ct_state=-trk match so it will be offloaded for non ct flows
+ * (or after clear action), as otherwise, since the tuple is changed,
+ * we can't restore ct state
+ */
+ if (mlx5_tc_ct_add_no_trk_match(spec)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "can't offload tuple modification with ct matches and no ct(clear) action");
+ netdev_info(priv->netdev,
+ "can't offload tuple modification with ct matches and no ct(clear) action");
+ return false;
+ }
+
+ return true;
+}
+
static bool modify_header_match_supported(struct mlx5e_priv *priv,
struct mlx5_flow_spec *spec,
struct flow_action *flow_action,
return err;
}
- /* Add ct_state=-trk match so it will be offloaded for non ct flows
- * (or after clear action), as otherwise, since the tuple is changed,
- * we can't restore ct state
- */
- if (!ct_clear && modify_tuple &&
- mlx5_tc_ct_add_no_trk_match(spec)) {
- NL_SET_ERR_MSG_MOD(extack,
- "can't offload tuple modify header with ct matches");
- netdev_info(priv->netdev,
- "can't offload tuple modify header with ct matches");
+ if (!modify_tuple_supported(modify_tuple, ct_clear, ct_flow, extack,
+ priv, spec))
return false;
- }
ip_proto = MLX5_GET(fte_match_set_lyr_2_4, headers_v, ip_protocol);
if (modify_ip_header && ip_proto != IPPROTO_TCP &&
*/
if (rate) {
rate = (rate * BITS_PER_BYTE) + 500000;
- rate_mbps = max_t(u64, do_div(rate, 1000000), 1);
+ do_div(rate, 1000000);
+ rate_mbps = max_t(u32, rate, 1);
}
err = mlx5_esw_modify_vport_rate(esw, vport_num, rate_mbps);
u8 inner_match_level;
u8 outer_match_level;
u8 ip_version;
+ u8 tun_ip_version;
u32 flags;
union {
struct mlx5_esw_flow_attr esw_attr[0];
mutex_unlock(&table->lock);
}
+#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
+#define MLX5_MAX_ASYNC_EQS 4
+#else
+#define MLX5_MAX_ASYNC_EQS 3
+#endif
+
int mlx5_eq_table_create(struct mlx5_core_dev *dev)
{
struct mlx5_eq_table *eq_table = dev->priv.eq_table;
+ int num_eqs = MLX5_CAP_GEN(dev, max_num_eqs) ?
+ MLX5_CAP_GEN(dev, max_num_eqs) :
+ 1 << MLX5_CAP_GEN(dev, log_max_eq);
int err;
eq_table->num_comp_eqs =
- mlx5_irq_get_num_comp(eq_table->irq_table);
+ min_t(int,
+ mlx5_irq_get_num_comp(eq_table->irq_table),
+ num_eqs - MLX5_MAX_ASYNC_EQS);
err = create_async_eqs(dev);
if (err) {
err_ethertype:
kfree(rule);
out:
- kfree(rule_spec);
+ kvfree(rule_spec);
return err;
}
e->recirc_cnt = 0;
out:
- kfree(in);
+ kvfree(in);
return err;
}
spec = kvzalloc(sizeof(*spec), GFP_KERNEL);
if (!spec) {
- kfree(in);
+ kvfree(in);
return -ENOMEM;
}
}
err_out:
- kfree(spec);
- kfree(in);
+ kvfree(spec);
+ kvfree(in);
return err;
}
return i;
}
+static bool
+esw_src_port_rewrite_supported(struct mlx5_eswitch *esw)
+{
+ return MLX5_CAP_GEN(esw->dev, reg_c_preserve) &&
+ mlx5_eswitch_vport_match_metadata_enabled(esw) &&
+ MLX5_CAP_ESW_FLOWTABLE_FDB(esw->dev, ignore_flow_level);
+}
+
static int
esw_setup_dests(struct mlx5_flow_destination *dest,
struct mlx5_flow_act *flow_act,
int err = 0;
if (!mlx5_eswitch_termtbl_required(esw, attr, flow_act, spec) &&
- MLX5_CAP_GEN(esw_attr->in_mdev, reg_c_preserve) &&
- mlx5_eswitch_vport_match_metadata_enabled(esw))
+ esw_src_port_rewrite_supported(esw))
attr->flags |= MLX5_ESW_ATTR_FLAG_SRC_REWRITE;
if (attr->dest_ft) {
}
esw->fdb_table.offloads.send_to_vport_grp = g;
- /* meta send to vport */
- memset(flow_group_in, 0, inlen);
- MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable,
- MLX5_MATCH_MISC_PARAMETERS_2);
-
- match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, match_criteria);
+ if (esw_src_port_rewrite_supported(esw)) {
+ /* meta send to vport */
+ memset(flow_group_in, 0, inlen);
+ MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable,
+ MLX5_MATCH_MISC_PARAMETERS_2);
- MLX5_SET(fte_match_param, match_criteria,
- misc_parameters_2.metadata_reg_c_0, mlx5_eswitch_get_vport_metadata_mask());
- MLX5_SET(fte_match_param, match_criteria,
- misc_parameters_2.metadata_reg_c_1, ESW_TUN_MASK);
+ match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, match_criteria);
- num_vfs = esw->esw_funcs.num_vfs;
- if (num_vfs) {
- MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, ix);
- MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, ix + num_vfs - 1);
- ix += num_vfs;
+ MLX5_SET(fte_match_param, match_criteria,
+ misc_parameters_2.metadata_reg_c_0,
+ mlx5_eswitch_get_vport_metadata_mask());
+ MLX5_SET(fte_match_param, match_criteria,
+ misc_parameters_2.metadata_reg_c_1, ESW_TUN_MASK);
- g = mlx5_create_flow_group(fdb, flow_group_in);
- if (IS_ERR(g)) {
- err = PTR_ERR(g);
- esw_warn(dev, "Failed to create send-to-vport meta flow group err(%d)\n",
- err);
- goto send_vport_meta_err;
+ num_vfs = esw->esw_funcs.num_vfs;
+ if (num_vfs) {
+ MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, ix);
+ MLX5_SET(create_flow_group_in, flow_group_in,
+ end_flow_index, ix + num_vfs - 1);
+ ix += num_vfs;
+
+ g = mlx5_create_flow_group(fdb, flow_group_in);
+ if (IS_ERR(g)) {
+ err = PTR_ERR(g);
+ esw_warn(dev, "Failed to create send-to-vport meta flow group err(%d)\n",
+ err);
+ goto send_vport_meta_err;
+ }
+ esw->fdb_table.offloads.send_to_vport_meta_grp = g;
+
+ err = mlx5_eswitch_add_send_to_vport_meta_rules(esw);
+ if (err)
+ goto meta_rule_err;
}
- esw->fdb_table.offloads.send_to_vport_meta_grp = g;
-
- err = mlx5_eswitch_add_send_to_vport_meta_rules(esw);
- if (err)
- goto meta_rule_err;
}
if (MLX5_CAP_ESW(esw->dev, merged_eswitch)) {
MLX5_SET(qpc, qpc, log_sq_size, ilog2(conn->qp.sq.size));
MLX5_SET(qpc, qpc, cqn_snd, conn->cq.mcq.cqn);
MLX5_SET(qpc, qpc, cqn_rcv, conn->cq.mcq.cqn);
+ MLX5_SET(qpc, qpc, ts_format, mlx5_get_qp_default_ts(mdev));
MLX5_SET64(qpc, qpc, dbr_addr, conn->qp.wq_ctrl.db.dma);
if (MLX5_CAP_GEN(mdev, cqe_version) == 1)
MLX5_SET(qpc, qpc, user_index, 0xFFFFFF);
}
qpc = MLX5_ADDR_OF(create_qp_in, in, qpc);
+ MLX5_SET(qpc, qpc, ts_format, mlx5_get_qp_default_ts(priv->mdev));
MLX5_SET(qpc, qpc, st, MLX5_QP_ST_UD);
MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED);
MLX5_SET(qpc, qpc, ulp_stateless_offload_mode,
static void mlx5_rdma_netdev_free(struct net_device *netdev)
{
struct mlx5e_priv *priv = mlx5i_epriv(netdev);
+ struct mlx5_core_dev *mdev = priv->mdev;
struct mlx5i_priv *ipriv = priv->ppriv;
const struct mlx5e_profile *profile = priv->profile;
if (!ipriv->sub_interface) {
mlx5i_pkey_qpn_ht_cleanup(netdev);
- mlx5e_destroy_mdev_resources(priv->mdev);
+ mlx5e_destroy_mdev_resources(mdev);
}
}
return -EINVAL;
field_select = MLX5_MTPPS_FS_ENABLE;
+ pin = ptp_find_pin(clock->ptp, PTP_PF_PEROUT, rq->perout.index);
+ if (pin < 0)
+ return -EBUSY;
+
if (on) {
bool rt_mode = mlx5_real_time_mode(mdev);
u32 nsec;
s64 sec;
- pin = ptp_find_pin(clock->ptp, PTP_PF_PEROUT, rq->perout.index);
- if (pin < 0)
- return -EBUSY;
-
pin_mode = MLX5_PIN_MODE_OUT;
pattern = MLX5_OUT_PATTERN_PERIODIC;
ts.tv_sec = rq->perout.period.sec;
u16 max_functions;
u16 function_id;
int err = 0;
- bool ecpu;
int i;
max_functions = mlx5_sf_max_functions(dev);
function_id = MLX5_CAP_GEN(dev, sf_base_id);
- ecpu = mlx5_read_embedded_cpu(dev);
/* Arm the vhca context as the vhca event notifier */
for (i = 0; i < max_functions; i++) {
- err = mlx5_vhca_event_arm(dev, function_id, ecpu);
+ err = mlx5_vhca_event_arm(dev, function_id);
if (err)
return err;
#include "sf.h"
#include "mlx5_ifc_vhca_event.h"
#include "vhca_event.h"
-#include "ecpf.h"
+#include "mlx5_core.h"
struct mlx5_sf_hw {
u32 usr_sfnum;
struct mlx5_core_dev *dev;
struct mlx5_sf_hw *sfs;
int max_local_functions;
- u8 ecpu: 1;
struct mutex table_lock; /* Serializes sf deletion and vhca state change handler. */
struct notifier_block vhca_nb;
};
}
if (sw_id == -ENOSPC) {
err = -ENOSPC;
- goto err;
+ goto exist_err;
}
hw_fn_id = mlx5_sf_sw_to_hw_id(table->dev, sw_id);
if (err)
goto err;
- err = mlx5_modify_vhca_sw_id(dev, hw_fn_id, table->ecpu, usr_sfnum);
+ err = mlx5_modify_vhca_sw_id(dev, hw_fn_id, usr_sfnum);
if (err)
goto vhca_err;
hw_fn_id = mlx5_sf_sw_to_hw_id(dev, id);
mutex_lock(&table->table_lock);
- err = mlx5_cmd_query_vhca_state(dev, hw_fn_id, table->ecpu, out, sizeof(out));
+ err = mlx5_cmd_query_vhca_state(dev, hw_fn_id, out, sizeof(out));
if (err)
goto err;
state = MLX5_GET(query_vhca_state_out, out, vhca_state_context.vhca_state);
table->dev = dev;
table->sfs = sfs;
table->max_local_functions = max_functions;
- table->ecpu = mlx5_read_embedded_cpu(dev);
dev->priv.sf_hw_table = table;
mlx5_core_dbg(dev, "SF HW table: max sfs = %d\n", max_functions);
return 0;
u8 sw_function_id[0x20];
- u8 reserved_at_40[0x80];
+ u8 reserved_at_40[0x40];
};
struct mlx5_ifc_query_vhca_state_out_bits {
struct mlx5_vhca_state_event event;
};
-int mlx5_cmd_query_vhca_state(struct mlx5_core_dev *dev, u16 function_id,
- bool ecpu, u32 *out, u32 outlen)
+int mlx5_cmd_query_vhca_state(struct mlx5_core_dev *dev, u16 function_id, u32 *out, u32 outlen)
{
u32 in[MLX5_ST_SZ_DW(query_vhca_state_in)] = {};
MLX5_SET(query_vhca_state_in, in, opcode, MLX5_CMD_OP_QUERY_VHCA_STATE);
MLX5_SET(query_vhca_state_in, in, function_id, function_id);
- MLX5_SET(query_vhca_state_in, in, embedded_cpu_function, ecpu);
+ MLX5_SET(query_vhca_state_in, in, embedded_cpu_function, 0);
return mlx5_cmd_exec(dev, in, sizeof(in), out, outlen);
}
static int mlx5_cmd_modify_vhca_state(struct mlx5_core_dev *dev, u16 function_id,
- bool ecpu, u32 *in, u32 inlen)
+ u32 *in, u32 inlen)
{
u32 out[MLX5_ST_SZ_DW(modify_vhca_state_out)] = {};
MLX5_SET(modify_vhca_state_in, in, opcode, MLX5_CMD_OP_MODIFY_VHCA_STATE);
MLX5_SET(modify_vhca_state_in, in, function_id, function_id);
- MLX5_SET(modify_vhca_state_in, in, embedded_cpu_function, ecpu);
+ MLX5_SET(modify_vhca_state_in, in, embedded_cpu_function, 0);
return mlx5_cmd_exec(dev, in, inlen, out, sizeof(out));
}
-int mlx5_modify_vhca_sw_id(struct mlx5_core_dev *dev, u16 function_id, bool ecpu, u32 sw_fn_id)
+int mlx5_modify_vhca_sw_id(struct mlx5_core_dev *dev, u16 function_id, u32 sw_fn_id)
{
u32 out[MLX5_ST_SZ_DW(modify_vhca_state_out)] = {};
u32 in[MLX5_ST_SZ_DW(modify_vhca_state_in)] = {};
MLX5_SET(modify_vhca_state_in, in, opcode, MLX5_CMD_OP_MODIFY_VHCA_STATE);
MLX5_SET(modify_vhca_state_in, in, function_id, function_id);
- MLX5_SET(modify_vhca_state_in, in, embedded_cpu_function, ecpu);
+ MLX5_SET(modify_vhca_state_in, in, embedded_cpu_function, 0);
MLX5_SET(modify_vhca_state_in, in, vhca_state_field_select.sw_function_id, 1);
MLX5_SET(modify_vhca_state_in, in, vhca_state_context.sw_function_id, sw_fn_id);
return mlx5_cmd_exec_inout(dev, modify_vhca_state, in, out);
}
-int mlx5_vhca_event_arm(struct mlx5_core_dev *dev, u16 function_id, bool ecpu)
+int mlx5_vhca_event_arm(struct mlx5_core_dev *dev, u16 function_id)
{
u32 in[MLX5_ST_SZ_DW(modify_vhca_state_in)] = {};
MLX5_SET(modify_vhca_state_in, in, vhca_state_context.arm_change_event, 1);
MLX5_SET(modify_vhca_state_in, in, vhca_state_field_select.arm_change_event, 1);
- return mlx5_cmd_modify_vhca_state(dev, function_id, ecpu, in, sizeof(in));
+ return mlx5_cmd_modify_vhca_state(dev, function_id, in, sizeof(in));
}
static void
u32 out[MLX5_ST_SZ_DW(query_vhca_state_out)] = {};
int err;
- err = mlx5_cmd_query_vhca_state(dev, event->function_id, event->ecpu, out, sizeof(out));
+ err = mlx5_cmd_query_vhca_state(dev, event->function_id, out, sizeof(out));
if (err)
return;
event->new_vhca_state = MLX5_GET(query_vhca_state_out, out,
vhca_state_context.vhca_state);
- mlx5_vhca_event_arm(dev, event->function_id, event->ecpu);
+ mlx5_vhca_event_arm(dev, event->function_id);
blocking_notifier_call_chain(&dev->priv.vhca_state_notifier->n_head, 0, event);
}
struct mlx5_core_dev *dev = notifier->dev;
mlx5_vhca_event_notify(dev, &work->event);
+ kfree(work);
}
static int
INIT_WORK(&work->work, &mlx5_vhca_state_work_handler);
work->notifier = notifier;
work->event.function_id = be16_to_cpu(eqe->data.vhca_state.function_id);
- work->event.ecpu = be16_to_cpu(eqe->data.vhca_state.ec_function);
mlx5_events_work_enqueue(notifier->dev, &work->work);
return NOTIFY_OK;
}
u16 function_id;
u16 sw_function_id;
u8 new_vhca_state;
- bool ecpu;
};
static inline bool mlx5_vhca_event_supported(const struct mlx5_core_dev *dev)
void mlx5_vhca_event_stop(struct mlx5_core_dev *dev);
int mlx5_vhca_event_notifier_register(struct mlx5_core_dev *dev, struct notifier_block *nb);
void mlx5_vhca_event_notifier_unregister(struct mlx5_core_dev *dev, struct notifier_block *nb);
-int mlx5_modify_vhca_sw_id(struct mlx5_core_dev *dev, u16 function_id, bool ecpu, u32 sw_fn_id);
-int mlx5_vhca_event_arm(struct mlx5_core_dev *dev, u16 function_id, bool ecpu);
+int mlx5_modify_vhca_sw_id(struct mlx5_core_dev *dev, u16 function_id, u32 sw_fn_id);
+int mlx5_vhca_event_arm(struct mlx5_core_dev *dev, u16 function_id);
int mlx5_cmd_query_vhca_state(struct mlx5_core_dev *dev, u16 function_id,
- bool ecpu, u32 *out, u32 outlen);
+ u32 *out, u32 outlen);
#else
static inline void mlx5_vhca_state_cap_handle(struct mlx5_core_dev *dev, void *set_hca_cap)
MLX5_SET(qpc, qpc, log_rq_size, ilog2(dr_qp->rq.wqe_cnt));
MLX5_SET(qpc, qpc, rq_type, MLX5_NON_ZERO_RQ);
MLX5_SET(qpc, qpc, log_sq_size, ilog2(dr_qp->sq.wqe_cnt));
+ MLX5_SET(qpc, qpc, ts_format, mlx5_get_qp_default_ts(mdev));
MLX5_SET64(qpc, qpc, dbr_addr, dr_qp->wq_ctrl.db.dma);
if (MLX5_CAP_GEN(mdev, cqe_version) == 1)
MLX5_SET(qpc, qpc, user_index, 0xFFFFFF);
static u64 dr_ste_v1_get_miss_addr(u8 *hw_ste_p)
{
u64 index =
- (MLX5_GET(ste_match_bwc_v1, hw_ste_p, miss_address_31_6) |
- MLX5_GET(ste_match_bwc_v1, hw_ste_p, miss_address_39_32) << 26);
+ ((u64)MLX5_GET(ste_match_bwc_v1, hw_ste_p, miss_address_31_6) |
+ ((u64)MLX5_GET(ste_match_bwc_v1, hw_ste_p, miss_address_39_32)) << 26);
return index << 6;
}
#include <net/red.h>
#include <net/vxlan.h>
#include <net/flow_offload.h>
+#include <net/inet_ecn.h>
#include "port.h"
#include "core.h"
u32 (*ptys_proto_cap_masked_get)(u32 eth_proto_cap);
};
+static inline u8 mlxsw_sp_tunnel_ecn_decap(u8 outer_ecn, u8 inner_ecn,
+ bool *trap_en)
+{
+ bool set_ce = false;
+
+ *trap_en = !!__INET_ECN_decapsulate(outer_ecn, inner_ecn, &set_ce);
+ if (set_ce)
+ return INET_ECN_CE;
+ else if (outer_ecn == INET_ECN_ECT_1 && inner_ecn == INET_ECN_ECT_0)
+ return INET_ECN_ECT_1;
+ else
+ return inner_ecn;
+}
+
static inline struct net_device *
mlxsw_sp_bridge_vxlan_dev_find(struct net_device *br_dev)
{
u32 ptys_eth_proto,
struct ethtool_link_ksettings *cmd)
{
+ struct mlxsw_sp1_port_link_mode link;
int i;
- cmd->link_mode = -1;
+ cmd->base.speed = SPEED_UNKNOWN;
+ cmd->base.duplex = DUPLEX_UNKNOWN;
+ cmd->lanes = 0;
if (!carrier_ok)
return;
for (i = 0; i < MLXSW_SP1_PORT_LINK_MODE_LEN; i++) {
- if (ptys_eth_proto & mlxsw_sp1_port_link_mode[i].mask)
- cmd->link_mode = mlxsw_sp1_port_link_mode[i].mask_ethtool;
+ if (ptys_eth_proto & mlxsw_sp1_port_link_mode[i].mask) {
+ link = mlxsw_sp1_port_link_mode[i];
+ ethtool_params_from_link_mode(cmd,
+ link.mask_ethtool);
+ }
}
}
struct mlxsw_sp2_port_link_mode link;
int i;
- cmd->link_mode = -1;
+ cmd->base.speed = SPEED_UNKNOWN;
+ cmd->base.duplex = DUPLEX_UNKNOWN;
+ cmd->lanes = 0;
if (!carrier_ok)
return;
for (i = 0; i < MLXSW_SP2_PORT_LINK_MODE_LEN; i++) {
if (ptys_eth_proto & mlxsw_sp2_port_link_mode[i].mask) {
link = mlxsw_sp2_port_link_mode[i];
- cmd->link_mode = link.mask_ethtool[1];
+ ethtool_params_from_link_mode(cmd,
+ link.mask_ethtool[1]);
}
}
}
u8 inner_ecn, u8 outer_ecn)
{
char tidem_pl[MLXSW_REG_TIDEM_LEN];
- bool trap_en, set_ce = false;
u8 new_inner_ecn;
+ bool trap_en;
- trap_en = __INET_ECN_decapsulate(outer_ecn, inner_ecn, &set_ce);
- new_inner_ecn = set_ce ? INET_ECN_CE : inner_ecn;
-
+ new_inner_ecn = mlxsw_sp_tunnel_ecn_decap(outer_ecn, inner_ecn,
+ &trap_en);
mlxsw_reg_tidem_pack(tidem_pl, outer_ecn, inner_ecn, new_inner_ecn,
trap_en, trap_en ? MLXSW_TRAP_ID_DECAP_ECN0 : 0);
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(tidem), tidem_pl);
u8 inner_ecn, u8 outer_ecn)
{
char tndem_pl[MLXSW_REG_TNDEM_LEN];
- bool trap_en, set_ce = false;
u8 new_inner_ecn;
+ bool trap_en;
- trap_en = !!__INET_ECN_decapsulate(outer_ecn, inner_ecn, &set_ce);
- new_inner_ecn = set_ce ? INET_ECN_CE : inner_ecn;
-
+ new_inner_ecn = mlxsw_sp_tunnel_ecn_decap(outer_ecn, inner_ecn,
+ &trap_en);
mlxsw_reg_tndem_pack(tndem_pl, outer_ecn, inner_ecn, new_inner_ecn,
trap_en, trap_en ? MLXSW_TRAP_ID_DECAP_ECN0 : 0);
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(tndem), tndem_pl);
}
mac_rx &= ~(MAC_RX_MAX_SIZE_MASK_);
- mac_rx |= (((new_mtu + ETH_HLEN + 4) << MAC_RX_MAX_SIZE_SHIFT_) &
- MAC_RX_MAX_SIZE_MASK_);
+ mac_rx |= (((new_mtu + ETH_HLEN + ETH_FCS_LEN)
+ << MAC_RX_MAX_SIZE_SHIFT_) & MAC_RX_MAX_SIZE_MASK_);
lan743x_csr_write(adapter, MAC_RX, mac_rx);
if (enabled) {
struct sk_buff *skb;
dma_addr_t dma_ptr;
- buffer_length = netdev->mtu + ETH_HLEN + 4 + RX_HEAD_PADDING;
+ buffer_length = netdev->mtu + ETH_HLEN + ETH_FCS_LEN + RX_HEAD_PADDING;
descriptor = &rx->ring_cpu_ptr[index];
buffer_info = &rx->buffer_info[index];
dev_kfree_skb_irq(skb);
return NULL;
}
- frame_length = max_t(int, 0, frame_length - RX_HEAD_PADDING - 4);
+ frame_length = max_t(int, 0, frame_length - ETH_FCS_LEN);
if (skb->len > frame_length) {
skb->tail -= skb->len - frame_length;
skb->len = frame_length;
dev_kfree_skb_any(curr);
if (segs != NULL) {
curr = segs;
- segs = segs->next;
+ segs = next;
curr->next = NULL;
dev_kfree_skb_any(segs);
}
dev_consume_skb_any(skb);
else
dev_kfree_skb_any(skb);
+ return;
}
nfp_ccm_rx(&bpf->ccm, skb);
* @qos_rate_limiters: Current active qos rate limiters
* @qos_stats_lock: Lock on qos stats updates
* @pre_tun_rule_cnt: Number of pre-tunnel rules offloaded
+ * @merge_table: Hash table to store merged flows
*/
struct nfp_flower_priv {
struct nfp_app *app;
unsigned int qos_rate_limiters;
spinlock_t qos_stats_lock; /* Protect the qos stats */
int pre_tun_rule_cnt;
+ struct rhashtable merge_table;
};
/**
};
extern const struct rhashtable_params nfp_flower_table_params;
+extern const struct rhashtable_params merge_table_params;
+
+struct nfp_merge_info {
+ u64 parent_ctx;
+ struct rhash_head ht_node;
+};
struct nfp_fl_stats_frame {
__be32 stats_con_id;
goto err_free_ctx_entry;
}
+ /* Do net allocate a mask-id for pre_tun_rules. These flows are used to
+ * configure the pre_tun table and are never actually send to the
+ * firmware as an add-flow message. This causes the mask-id allocation
+ * on the firmware to get out of sync if allocated here.
+ */
new_mask_id = 0;
- if (!nfp_check_mask_add(app, nfp_flow->mask_data,
+ if (!nfp_flow->pre_tun_rule.dev &&
+ !nfp_check_mask_add(app, nfp_flow->mask_data,
nfp_flow->meta.mask_len,
&nfp_flow->meta.flags, &new_mask_id)) {
NL_SET_ERR_MSG_MOD(extack, "invalid entry: cannot allocate a new mask id");
goto err_remove_mask;
}
- if (!nfp_check_mask_remove(app, nfp_flow->mask_data,
+ if (!nfp_flow->pre_tun_rule.dev &&
+ !nfp_check_mask_remove(app, nfp_flow->mask_data,
nfp_flow->meta.mask_len,
NULL, &new_mask_id)) {
NL_SET_ERR_MSG_MOD(extack, "invalid entry: cannot release mask id");
return 0;
err_remove_mask:
- nfp_check_mask_remove(app, nfp_flow->mask_data, nfp_flow->meta.mask_len,
- NULL, &new_mask_id);
+ if (!nfp_flow->pre_tun_rule.dev)
+ nfp_check_mask_remove(app, nfp_flow->mask_data,
+ nfp_flow->meta.mask_len,
+ NULL, &new_mask_id);
err_remove_rhash:
WARN_ON_ONCE(rhashtable_remove_fast(&priv->stats_ctx_table,
&ctx_entry->ht_node,
__nfp_modify_flow_metadata(priv, nfp_flow);
- nfp_check_mask_remove(app, nfp_flow->mask_data,
- nfp_flow->meta.mask_len, &nfp_flow->meta.flags,
- &new_mask_id);
+ if (!nfp_flow->pre_tun_rule.dev)
+ nfp_check_mask_remove(app, nfp_flow->mask_data,
+ nfp_flow->meta.mask_len, &nfp_flow->meta.flags,
+ &new_mask_id);
/* Update flow payload with mask ids. */
nfp_flow->unmasked_data[NFP_FL_MASK_ID_LOCATION] = new_mask_id;
.automatic_shrinking = true,
};
+const struct rhashtable_params merge_table_params = {
+ .key_offset = offsetof(struct nfp_merge_info, parent_ctx),
+ .head_offset = offsetof(struct nfp_merge_info, ht_node),
+ .key_len = sizeof(u64),
+};
+
int nfp_flower_metadata_init(struct nfp_app *app, u64 host_ctx_count,
unsigned int host_num_mems)
{
if (err)
goto err_free_flow_table;
+ err = rhashtable_init(&priv->merge_table, &merge_table_params);
+ if (err)
+ goto err_free_stats_ctx_table;
+
get_random_bytes(&priv->mask_id_seed, sizeof(priv->mask_id_seed));
/* Init ring buffer and unallocated mask_ids. */
kmalloc_array(NFP_FLOWER_MASK_ENTRY_RS,
NFP_FLOWER_MASK_ELEMENT_RS, GFP_KERNEL);
if (!priv->mask_ids.mask_id_free_list.buf)
- goto err_free_stats_ctx_table;
+ goto err_free_merge_table;
priv->mask_ids.init_unallocated = NFP_FLOWER_MASK_ENTRY_RS - 1;
kfree(priv->mask_ids.last_used);
err_free_mask_id:
kfree(priv->mask_ids.mask_id_free_list.buf);
+err_free_merge_table:
+ rhashtable_destroy(&priv->merge_table);
err_free_stats_ctx_table:
rhashtable_destroy(&priv->stats_ctx_table);
err_free_flow_table:
nfp_check_rhashtable_empty, NULL);
rhashtable_free_and_destroy(&priv->stats_ctx_table,
nfp_check_rhashtable_empty, NULL);
+ rhashtable_free_and_destroy(&priv->merge_table,
+ nfp_check_rhashtable_empty, NULL);
kvfree(priv->stats);
kfree(priv->mask_ids.mask_id_free_list.buf);
kfree(priv->mask_ids.last_used);
struct netlink_ext_ack *extack = NULL;
struct nfp_fl_payload *merge_flow;
struct nfp_fl_key_ls merge_key_ls;
+ struct nfp_merge_info *merge_info;
+ u64 parent_ctx = 0;
int err;
ASSERT_RTNL();
nfp_flower_is_merge_flow(sub_flow2))
return -EINVAL;
+ /* check if the two flows are already merged */
+ parent_ctx = (u64)(be32_to_cpu(sub_flow1->meta.host_ctx_id)) << 32;
+ parent_ctx |= (u64)(be32_to_cpu(sub_flow2->meta.host_ctx_id));
+ if (rhashtable_lookup_fast(&priv->merge_table,
+ &parent_ctx, merge_table_params)) {
+ nfp_flower_cmsg_warn(app, "The two flows are already merged.\n");
+ return 0;
+ }
+
err = nfp_flower_can_merge(sub_flow1, sub_flow2);
if (err)
return err;
if (err)
goto err_release_metadata;
+ merge_info = kmalloc(sizeof(*merge_info), GFP_KERNEL);
+ if (!merge_info) {
+ err = -ENOMEM;
+ goto err_remove_rhash;
+ }
+ merge_info->parent_ctx = parent_ctx;
+ err = rhashtable_insert_fast(&priv->merge_table, &merge_info->ht_node,
+ merge_table_params);
+ if (err)
+ goto err_destroy_merge_info;
+
err = nfp_flower_xmit_flow(app, merge_flow,
NFP_FLOWER_CMSG_TYPE_FLOW_MOD);
if (err)
- goto err_remove_rhash;
+ goto err_remove_merge_info;
merge_flow->in_hw = true;
sub_flow1->in_hw = false;
return 0;
+err_remove_merge_info:
+ WARN_ON_ONCE(rhashtable_remove_fast(&priv->merge_table,
+ &merge_info->ht_node,
+ merge_table_params));
+err_destroy_merge_info:
+ kfree(merge_info);
err_remove_rhash:
WARN_ON_ONCE(rhashtable_remove_fast(&priv->flow_table,
&merge_flow->fl_node,
return -EOPNOTSUPP;
}
+ if (!(key_layer & NFP_FLOWER_LAYER_IPV4) &&
+ !(key_layer & NFP_FLOWER_LAYER_IPV6)) {
+ NL_SET_ERR_MSG_MOD(extack, "unsupported pre-tunnel rule: match on ipv4/ipv6 eth_type must be present");
+ return -EOPNOTSUPP;
+ }
+
/* Skip fields known to exist. */
mask += sizeof(struct nfp_flower_meta_tci);
ext += sizeof(struct nfp_flower_meta_tci);
mask += sizeof(struct nfp_flower_in_port);
ext += sizeof(struct nfp_flower_in_port);
+ /* Ensure destination MAC address matches pre_tun_dev. */
+ mac = (struct nfp_flower_mac_mpls *)ext;
+ if (memcmp(&mac->mac_dst[0], flow->pre_tun_rule.dev->dev_addr, 6)) {
+ NL_SET_ERR_MSG_MOD(extack, "unsupported pre-tunnel rule: dest MAC must match output dev MAC");
+ return -EOPNOTSUPP;
+ }
+
/* Ensure destination MAC address is fully matched. */
mac = (struct nfp_flower_mac_mpls *)mask;
if (!is_broadcast_ether_addr(&mac->mac_dst[0])) {
return -EOPNOTSUPP;
}
+ if (mac->mpls_lse) {
+ NL_SET_ERR_MSG_MOD(extack, "unsupported pre-tunnel rule: MPLS not supported");
+ return -EOPNOTSUPP;
+ }
+
mask += sizeof(struct nfp_flower_mac_mpls);
ext += sizeof(struct nfp_flower_mac_mpls);
if (key_layer & NFP_FLOWER_LAYER_IPV4 ||
{
struct nfp_flower_priv *priv = app->priv;
struct nfp_fl_payload_link *link, *temp;
+ struct nfp_merge_info *merge_info;
struct nfp_fl_payload *origin;
+ u64 parent_ctx = 0;
bool mod = false;
int err;
err_free_links:
/* Clean any links connected with the merged flow. */
list_for_each_entry_safe(link, temp, &merge_flow->linked_flows,
- merge_flow.list)
+ merge_flow.list) {
+ u32 ctx_id = be32_to_cpu(link->sub_flow.flow->meta.host_ctx_id);
+
+ parent_ctx = (parent_ctx << 32) | (u64)(ctx_id);
nfp_flower_unlink_flow(link);
+ }
+
+ merge_info = rhashtable_lookup_fast(&priv->merge_table,
+ &parent_ctx,
+ merge_table_params);
+ if (merge_info) {
+ WARN_ON_ONCE(rhashtable_remove_fast(&priv->merge_table,
+ &merge_info->ht_node,
+ merge_table_params));
+ kfree(merge_info);
+ }
kfree(merge_flow->action_data);
kfree(merge_flow->mask_data);
#define NFP_FL_MAX_ROUTES 32
#define NFP_TUN_PRE_TUN_RULE_LIMIT 32
-#define NFP_TUN_PRE_TUN_RULE_DEL 0x1
-#define NFP_TUN_PRE_TUN_IDX_BIT 0x8
+#define NFP_TUN_PRE_TUN_RULE_DEL BIT(0)
+#define NFP_TUN_PRE_TUN_IDX_BIT BIT(3)
+#define NFP_TUN_PRE_TUN_IPV6_BIT BIT(7)
/**
* struct nfp_tun_pre_run_rule - rule matched before decap
{
struct nfp_flower_priv *app_priv = app->priv;
struct nfp_tun_offloaded_mac *mac_entry;
+ struct nfp_flower_meta_tci *key_meta;
struct nfp_tun_pre_tun_rule payload;
struct net_device *internal_dev;
int err;
if (!mac_entry)
return -ENOENT;
+ /* Set/clear IPV6 bit. cpu_to_be16() swap will lead to MSB being
+ * set/clear for port_idx.
+ */
+ key_meta = (struct nfp_flower_meta_tci *)flow->unmasked_data;
+ if (key_meta->nfp_flow_key_layer & NFP_FLOWER_LAYER_IPV6)
+ mac_entry->index |= NFP_TUN_PRE_TUN_IPV6_BIT;
+ else
+ mac_entry->index &= ~NFP_TUN_PRE_TUN_IPV6_BIT;
+
payload.port_idx = cpu_to_be16(mac_entry->index);
/* Copy mac id and vlan to flow - dev may not exist at delete time. */
{
int sg_elems = q->lif->qtype_info[IONIC_QTYPE_TXQ].max_sg_elems;
struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ int ndescs;
int err;
- /* If TSO, need roundup(skb->len/mss) descs */
+ /* Each desc is mss long max, so a descriptor for each gso_seg */
if (skb_is_gso(skb))
- return (skb->len / skb_shinfo(skb)->gso_size) + 1;
+ ndescs = skb_shinfo(skb)->gso_segs;
+ else
+ ndescs = 1;
- /* If non-TSO, just need 1 desc and nr_frags sg elems */
if (skb_shinfo(skb)->nr_frags <= sg_elems)
- return 1;
+ return ndescs;
/* Too many frags, so linearize */
err = skb_linearize(skb);
stats->linearize++;
- /* Need 1 desc and zero sg elems */
- return 1;
+ return ndescs;
}
static int ionic_maybe_stop_tx(struct ionic_queue *q, int ndescs)
if (fw_dump->tmpl_hdr == NULL || current_version > prev_version) {
vfree(fw_dump->tmpl_hdr);
+ fw_dump->tmpl_hdr = NULL;
if (qlcnic_83xx_md_check_extended_dump_capability(adapter))
extended = !qlcnic_83xx_extend_md_capab(adapter);
struct qlcnic_83xx_dump_template_hdr *hdr;
hdr = fw_dump->tmpl_hdr;
+ if (!hdr)
+ return;
hdr->drv_cap_mask = 0x1f;
fw_dump->cap_mask = 0x1f;
dev_info(&pdev->dev,
rtl8169_update_counters(tp);
+ pci_clear_master(tp->pci_dev);
+ rtl_pci_commit(tp);
+
rtl8169_cleanup(tp, true);
rtl_prepare_power_down(tp);
static void rtl8169_up(struct rtl8169_private *tp)
{
+ pci_set_master(tp->pci_dev);
phy_resume(tp->phydev);
rtl8169_init_phy(tp);
napi_enable(&tp->napi);
rtl_hw_reset(tp);
- pci_set_master(pdev);
-
rc = rtl_alloc_irq(tp);
if (rc < 0) {
dev_err(&pdev->dev, "Can't allocate interrupt\n");
goto err1;
/* set phy power down */
- data = netsec_phy_read(priv->mii_bus, priv->phy_addr, MII_BMCR) |
- BMCR_PDOWN;
- netsec_phy_write(priv->mii_bus, priv->phy_addr, MII_BMCR, data);
+ data = netsec_phy_read(priv->mii_bus, priv->phy_addr, MII_BMCR);
+ netsec_phy_write(priv->mii_bus, priv->phy_addr, MII_BMCR,
+ data | BMCR_PDOWN);
ret = netsec_reset_hardware(priv, true);
if (ret)
goto err2;
+ /* Restore phy power state */
+ netsec_phy_write(priv->mii_bus, priv->phy_addr, MII_BMCR, data);
+
spin_lock_init(&priv->desc_ring[NETSEC_RING_TX].lock);
spin_lock_init(&priv->desc_ring[NETSEC_RING_RX].lock);
plat_dat->init = sun8i_dwmac_init;
plat_dat->exit = sun8i_dwmac_exit;
plat_dat->setup = sun8i_dwmac_setup;
+ plat_dat->tx_fifo_size = 4096;
+ plat_dat->rx_fifo_size = 16384;
ret = sun8i_dwmac_set_syscon(&pdev->dev, plat_dat);
if (ret)
return axienet_ior(lp, XAE_MDIO_MCR_OFFSET);
}
+static inline void axienet_lock_mii(struct axienet_local *lp)
+{
+ if (lp->mii_bus)
+ mutex_lock(&lp->mii_bus->mdio_lock);
+}
+
+static inline void axienet_unlock_mii(struct axienet_local *lp)
+{
+ if (lp->mii_bus)
+ mutex_unlock(&lp->mii_bus->mdio_lock);
+}
+
/**
* axienet_iow - Memory mapped Axi Ethernet register write
* @lp: Pointer to axienet local structure
* including the MDIO. MDIO must be disabled before resetting.
* Hold MDIO bus lock to avoid MDIO accesses during the reset.
*/
- mutex_lock(&lp->mii_bus->mdio_lock);
+ axienet_lock_mii(lp);
ret = axienet_device_reset(ndev);
- mutex_unlock(&lp->mii_bus->mdio_lock);
+ axienet_unlock_mii(lp);
ret = phylink_of_phy_connect(lp->phylink, lp->dev->of_node, 0);
if (ret) {
}
/* Do a reset to ensure DMA is really stopped */
- mutex_lock(&lp->mii_bus->mdio_lock);
+ axienet_lock_mii(lp);
__axienet_device_reset(lp);
- mutex_unlock(&lp->mii_bus->mdio_lock);
+ axienet_unlock_mii(lp);
cancel_work_sync(&lp->dma_err_task);
* including the MDIO. MDIO must be disabled before resetting.
* Hold MDIO bus lock to avoid MDIO accesses during the reset.
*/
- mutex_lock(&lp->mii_bus->mdio_lock);
+ axienet_lock_mii(lp);
__axienet_device_reset(lp);
- mutex_unlock(&lp->mii_bus->mdio_lock);
+ axienet_unlock_mii(lp);
for (i = 0; i < lp->tx_bd_num; i++) {
cur_p = &lp->tx_bd_v[i];
if (IS_ERR(lp->regs)) {
dev_err(&pdev->dev, "could not map Axi Ethernet regs.\n");
ret = PTR_ERR(lp->regs);
- goto free_netdev;
+ goto cleanup_clk;
}
lp->regs_start = ethres->start;
break;
default:
ret = -EINVAL;
- goto free_netdev;
+ goto cleanup_clk;
}
} else {
ret = of_get_phy_mode(pdev->dev.of_node, &lp->phy_mode);
if (ret)
- goto free_netdev;
+ goto cleanup_clk;
}
if (lp->switch_x_sgmii && lp->phy_mode != PHY_INTERFACE_MODE_SGMII &&
lp->phy_mode != PHY_INTERFACE_MODE_1000BASEX) {
dev_err(&pdev->dev, "xlnx,switch-x-sgmii only supported with SGMII or 1000BaseX\n");
ret = -EINVAL;
- goto free_netdev;
+ goto cleanup_clk;
}
/* Find the DMA node, map the DMA registers, and decode the DMA IRQs */
dev_err(&pdev->dev,
"unable to get DMA resource\n");
of_node_put(np);
- goto free_netdev;
+ goto cleanup_clk;
}
lp->dma_regs = devm_ioremap_resource(&pdev->dev,
&dmares);
if (IS_ERR(lp->dma_regs)) {
dev_err(&pdev->dev, "could not map DMA regs\n");
ret = PTR_ERR(lp->dma_regs);
- goto free_netdev;
+ goto cleanup_clk;
}
if ((lp->rx_irq <= 0) || (lp->tx_irq <= 0)) {
dev_err(&pdev->dev, "could not determine irqs\n");
ret = -ENOMEM;
- goto free_netdev;
+ goto cleanup_clk;
}
/* Autodetect the need for 64-bit DMA pointers.
ret = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(addr_width));
if (ret) {
dev_err(&pdev->dev, "No suitable DMA available\n");
- goto free_netdev;
+ goto cleanup_clk;
}
/* Check for Ethernet core IRQ (optional) */
if (!lp->phy_node) {
dev_err(&pdev->dev, "phy-handle required for 1000BaseX/SGMII\n");
ret = -EINVAL;
- goto free_netdev;
+ goto cleanup_mdio;
}
lp->pcs_phy = of_mdio_find_device(lp->phy_node);
if (!lp->pcs_phy) {
ret = -EPROBE_DEFER;
- goto free_netdev;
+ goto cleanup_mdio;
}
lp->phylink_config.pcs_poll = true;
}
if (IS_ERR(lp->phylink)) {
ret = PTR_ERR(lp->phylink);
dev_err(&pdev->dev, "phylink_create error (%i)\n", ret);
- goto free_netdev;
+ goto cleanup_mdio;
}
ret = register_netdev(lp->ndev);
if (ret) {
dev_err(lp->dev, "register_netdev() error (%i)\n", ret);
- goto free_netdev;
+ goto cleanup_phylink;
}
return 0;
+cleanup_phylink:
+ phylink_destroy(lp->phylink);
+
+cleanup_mdio:
+ if (lp->pcs_phy)
+ put_device(&lp->pcs_phy->dev);
+ if (lp->mii_bus)
+ axienet_mdio_teardown(lp);
+ of_node_put(lp->phy_node);
+
+cleanup_clk:
+ clk_disable_unprepare(lp->clk);
+
free_netdev:
free_netdev(ndev);
info = skb_tunnel_info(skb);
if (info) {
- info->key.u.ipv4.dst = fl4.saddr;
- info->key.u.ipv4.src = fl4.daddr;
+ struct ip_tunnel_info *unclone;
+
+ unclone = skb_tunnel_info_unclone(skb);
+ if (unlikely(!unclone)) {
+ dst_release(&rt->dst);
+ return -ENOMEM;
+ }
+
+ unclone->key.u.ipv4.dst = fl4.saddr;
+ unclone->key.u.ipv4.src = fl4.daddr;
}
if (!pskb_may_pull(skb, ETH_HLEN)) {
struct ip_tunnel_info *info = skb_tunnel_info(skb);
if (info) {
- info->key.u.ipv6.dst = fl6.saddr;
- info->key.u.ipv6.src = fl6.daddr;
+ struct ip_tunnel_info *unclone;
+
+ unclone = skb_tunnel_info_unclone(skb);
+ if (unlikely(!unclone)) {
+ dst_release(dst);
+ return -ENOMEM;
+ }
+
+ unclone->key.u.ipv6.dst = fl6.saddr;
+ unclone->key.u.ipv6.src = fl6.daddr;
}
if (!pskb_may_pull(skb, ETH_HLEN)) {
MODULE_AUTHOR("Joerg Reuter <jreuter@yaina.de>");
MODULE_DESCRIPTION("AX.25 Device Driver for Z8530 based HDLC cards");
-MODULE_SUPPORTED_DEVICE("Z8530 based SCC cards for Amateur Radio");
MODULE_LICENSE("GPL");
module_init(scc_init_driver);
module_exit(scc_cleanup_driver);
return -ENOMEM;
}
usb_anchor_urb(urb, &atusb->idle_urbs);
+ usb_free_urb(urb);
n--;
}
return 0;
: field_max(IP_FLTRT_FLAGS_NHASH_ADDR_FMASK);
if (mem->offset > offset_max ||
ipa->mem_offset > offset_max - mem->offset) {
- dev_err(dev, "IPv%c %s%s table region offset too large "
- "(0x%04x + 0x%04x > 0x%04x)\n",
- ipv6 ? '6' : '4', hashed ? "hashed " : "",
- route ? "route" : "filter",
- ipa->mem_offset, mem->offset, offset_max);
+ dev_err(dev, "IPv%c %s%s table region offset too large\n",
+ ipv6 ? '6' : '4', hashed ? "hashed " : "",
+ route ? "route" : "filter");
+ dev_err(dev, " (0x%04x + 0x%04x > 0x%04x)\n",
+ ipa->mem_offset, mem->offset, offset_max);
+
return false;
}
if (mem->offset > ipa->mem_size ||
mem->size > ipa->mem_size - mem->offset) {
- dev_err(dev, "IPv%c %s%s table region out of range "
- "(0x%04x + 0x%04x > 0x%04x)\n",
- ipv6 ? '6' : '4', hashed ? "hashed " : "",
- route ? "route" : "filter",
- mem->offset, mem->size, ipa->mem_size);
+ dev_err(dev, "IPv%c %s%s table region out of range\n",
+ ipv6 ? '6' : '4', hashed ? "hashed " : "",
+ route ? "route" : "filter");
+ dev_err(dev, " (0x%04x + 0x%04x > 0x%04x)\n",
+ mem->offset, mem->size, ipa->mem_size);
+
return false;
}
u32 size_max;
u32 size;
+ /* In ipa_cmd_hdr_init_local_add() we record the offset and size
+ * of the header table memory area. Make sure the offset and size
+ * fit in the fields that need to hold them, and that the entire
+ * range is within the overall IPA memory range.
+ */
offset_max = field_max(HDR_INIT_LOCAL_FLAGS_HDR_ADDR_FMASK);
if (mem->offset > offset_max ||
ipa->mem_offset > offset_max - mem->offset) {
- dev_err(dev, "header table region offset too large "
- "(0x%04x + 0x%04x > 0x%04x)\n",
- ipa->mem_offset + mem->offset, offset_max);
+ dev_err(dev, "header table region offset too large\n");
+ dev_err(dev, " (0x%04x + 0x%04x > 0x%04x)\n",
+ ipa->mem_offset, mem->offset, offset_max);
+
return false;
}
size_max = field_max(HDR_INIT_LOCAL_FLAGS_TABLE_SIZE_FMASK);
size = ipa->mem[IPA_MEM_MODEM_HEADER].size;
size += ipa->mem[IPA_MEM_AP_HEADER].size;
- if (mem->offset > ipa->mem_size || size > ipa->mem_size - mem->offset) {
- dev_err(dev, "header table region out of range "
- "(0x%04x + 0x%04x > 0x%04x)\n",
- mem->offset, size, ipa->mem_size);
+
+ if (size > size_max) {
+ dev_err(dev, "header table region size too large\n");
+ dev_err(dev, " (0x%04x > 0x%08x)\n", size, size_max);
+
+ return false;
+ }
+ if (size > ipa->mem_size || mem->offset > ipa->mem_size - size) {
+ dev_err(dev, "header table region out of range\n");
+ dev_err(dev, " (0x%04x + 0x%04x > 0x%04x)\n",
+ mem->offset, size, ipa->mem_size);
+
return false;
}
.decoded_size = IPA_QMI_DRIVER_INIT_COMPLETE_REQ_SZ,
.fn = ipa_server_driver_init_complete,
},
+ { },
};
/* Handle an INIT_DRIVER response message from the modem. */
.decoded_size = IPA_QMI_INIT_DRIVER_RSP_SZ,
.fn = ipa_client_init_driver,
},
+ { },
};
/* Return a pointer to an init modem driver request structure, which contains
int bcm_phy_set_eee(struct phy_device *phydev, bool enable)
{
- int val;
+ int val, mask = 0;
/* Enable EEE at PHY level */
val = phy_read_mmd(phydev, MDIO_MMD_AN, BRCM_CL45VEN_EEE_CONTROL);
if (val < 0)
return val;
+ if (linkmode_test_bit(ETHTOOL_LINK_MODE_1000baseT_Full_BIT,
+ phydev->supported))
+ mask |= MDIO_EEE_1000T;
+ if (linkmode_test_bit(ETHTOOL_LINK_MODE_100baseT_Full_BIT,
+ phydev->supported))
+ mask |= MDIO_EEE_100TX;
+
if (enable)
- val |= (MDIO_EEE_100TX | MDIO_EEE_1000T);
+ val |= mask;
else
- val &= ~(MDIO_EEE_100TX | MDIO_EEE_1000T);
+ val &= ~mask;
phy_write_mmd(phydev, MDIO_MMD_AN, BCM_CL45VEN_EEE_ADV, (u32)val);
bcm54xx_adjust_rxrefclk(phydev);
switch (BRCM_PHY_MODEL(phydev)) {
+ case PHY_ID_BCM50610:
+ case PHY_ID_BCM50610M:
+ err = bcm54xx_config_clock_delay(phydev);
+ break;
case PHY_ID_BCM54210E:
err = bcm54210e_config_init(phydev);
break;
if (ret < 0)
return ret;
+ /* Upon exiting power down, the PHY remains in an internal reset state
+ * for 40us
+ */
+ fsleep(40);
+
return bcm54xx_config_init(phydev);
}
err = pl->mac_ops->mac_finish(pl->config, pl->cur_link_an_mode,
state->interface);
if (err < 0)
- phylink_err(pl, "mac_prepare failed: %pe\n",
+ phylink_err(pl, "mac_finish failed: %pe\n",
ERR_PTR(err));
}
}
#include <linux/bpf.h>
#include <linux/bpf_trace.h>
#include <linux/mutex.h>
+#include <linux/ieee802154.h>
+#include <linux/if_ltalk.h>
+#include <uapi/linux/if_fddi.h>
+#include <uapi/linux/if_hippi.h>
+#include <uapi/linux/if_fc.h>
+#include <net/ax25.h>
+#include <net/rose.h>
+#include <net/6lowpan.h>
#include <linux/uaccess.h>
#include <linux/proc_fs.h>
return __tun_set_ebpf(tun, prog_p, prog);
}
+/* Return correct value for tun->dev->addr_len based on tun->dev->type. */
+static unsigned char tun_get_addr_len(unsigned short type)
+{
+ switch (type) {
+ case ARPHRD_IP6GRE:
+ case ARPHRD_TUNNEL6:
+ return sizeof(struct in6_addr);
+ case ARPHRD_IPGRE:
+ case ARPHRD_TUNNEL:
+ case ARPHRD_SIT:
+ return 4;
+ case ARPHRD_ETHER:
+ return ETH_ALEN;
+ case ARPHRD_IEEE802154:
+ case ARPHRD_IEEE802154_MONITOR:
+ return IEEE802154_EXTENDED_ADDR_LEN;
+ case ARPHRD_PHONET_PIPE:
+ case ARPHRD_PPP:
+ case ARPHRD_NONE:
+ return 0;
+ case ARPHRD_6LOWPAN:
+ return EUI64_ADDR_LEN;
+ case ARPHRD_FDDI:
+ return FDDI_K_ALEN;
+ case ARPHRD_HIPPI:
+ return HIPPI_ALEN;
+ case ARPHRD_IEEE802:
+ return FC_ALEN;
+ case ARPHRD_ROSE:
+ return ROSE_ADDR_LEN;
+ case ARPHRD_NETROM:
+ return AX25_ADDR_LEN;
+ case ARPHRD_LOCALTLK:
+ return LTALK_ALEN;
+ default:
+ return 0;
+ }
+}
+
static long __tun_chr_ioctl(struct file *file, unsigned int cmd,
unsigned long arg, int ifreq_len)
{
break;
}
tun->dev->type = (int) arg;
+ tun->dev->addr_len = tun_get_addr_len(tun->dev->type);
netif_info(tun, drv, tun->dev, "linktype set to %d\n",
tun->dev->type);
call_netdevice_notifiers(NETDEV_POST_TYPE_CHANGE,
err = register_netdev(dev);
if (err) {
+ /* Set disconnected flag so that disconnect() returns early. */
+ pnd->disconnected = 1;
usb_driver_release_interface(&usbpn_driver, data_intf);
goto out;
}
return serial;
}
-static int get_free_serial_index(void)
+static int obtain_minor(struct hso_serial *serial)
{
int index;
unsigned long flags;
spin_lock_irqsave(&serial_table_lock, flags);
for (index = 0; index < HSO_SERIAL_TTY_MINORS; index++) {
if (serial_table[index] == NULL) {
+ serial_table[index] = serial->parent;
+ serial->minor = index;
spin_unlock_irqrestore(&serial_table_lock, flags);
- return index;
+ return 0;
}
}
spin_unlock_irqrestore(&serial_table_lock, flags);
return -1;
}
-static void set_serial_by_index(unsigned index, struct hso_serial *serial)
+static void release_minor(struct hso_serial *serial)
{
unsigned long flags;
spin_lock_irqsave(&serial_table_lock, flags);
- if (serial)
- serial_table[index] = serial->parent;
- else
- serial_table[index] = NULL;
+ serial_table[serial->minor] = NULL;
spin_unlock_irqrestore(&serial_table_lock, flags);
}
static void hso_serial_tty_unregister(struct hso_serial *serial)
{
tty_unregister_device(tty_drv, serial->minor);
+ release_minor(serial);
}
static void hso_serial_common_free(struct hso_serial *serial)
static int hso_serial_common_create(struct hso_serial *serial, int num_urbs,
int rx_size, int tx_size)
{
- int minor;
int i;
tty_port_init(&serial->port);
- minor = get_free_serial_index();
- if (minor < 0)
+ if (obtain_minor(serial))
goto exit2;
/* register our minor number */
serial->parent->dev = tty_port_register_device_attr(&serial->port,
- tty_drv, minor, &serial->parent->interface->dev,
+ tty_drv, serial->minor, &serial->parent->interface->dev,
serial->parent, hso_serial_dev_groups);
- if (IS_ERR(serial->parent->dev))
+ if (IS_ERR(serial->parent->dev)) {
+ release_minor(serial);
goto exit2;
+ }
- /* fill in specific data for later use */
- serial->minor = minor;
serial->magic = HSO_SERIAL_MAGIC;
spin_lock_init(&serial->serial_lock);
serial->num_rx_urbs = num_urbs;
serial->write_data = hso_std_serial_write_data;
- /* and record this serial */
- set_serial_by_index(serial->minor, serial);
-
/* setup the proc dirs and files if needed */
hso_log_port(hso_dev);
serial->shared_int->ref_count++;
mutex_unlock(&serial->shared_int->shared_int_lock);
- /* and record this serial */
- set_serial_by_index(serial->minor, serial);
-
/* setup the proc dirs and files if needed */
hso_log_port(hso_dev);
cancel_work_sync(&serial_table[i]->async_get_intf);
hso_serial_tty_unregister(serial);
kref_put(&serial_table[i]->ref, hso_serial_ref_free);
- set_serial_by_index(i, NULL);
}
}
ops->in_nway = rtl8153_in_nway;
ops->hw_phy_cfg = r8153_hw_phy_cfg;
ops->autosuspend_en = rtl8153_runtime_enable;
- tp->rx_buf_sz = 32 * 1024;
+ if (tp->udev->speed < USB_SPEED_SUPER)
+ tp->rx_buf_sz = 16 * 1024;
+ else
+ tp->rx_buf_sz = 32 * 1024;
tp->eee_en = true;
tp->eee_adv = MDIO_EEE_1000T | MDIO_EEE_100TX;
break;
if (rxq < rcv->real_num_rx_queues) {
rq = &rcv_priv->rq[rxq];
rcv_xdp = rcu_access_pointer(rq->xdp_prog);
- if (rcv_xdp)
- skb_record_rx_queue(skb, rxq);
+ skb_record_rx_queue(skb, rxq);
}
skb_tx_timestamp(skb);
offset += hdr_padded_len;
p += hdr_padded_len;
- copy = len;
- if (copy > skb_tailroom(skb))
- copy = skb_tailroom(skb);
+ /* Copy all frame if it fits skb->head, otherwise
+ * we let virtio_net_hdr_to_skb() and GRO pull headers as needed.
+ */
+ if (len <= skb_tailroom(skb))
+ copy = len;
+ else
+ copy = ETH_HLEN + metasize;
skb_put_data(skb, p, copy);
if (metasize) {
goto tx_error;
} else if (err) {
if (info) {
+ struct ip_tunnel_info *unclone;
struct in_addr src, dst;
+ unclone = skb_tunnel_info_unclone(skb);
+ if (unlikely(!unclone))
+ goto tx_error;
+
src = remote_ip.sin.sin_addr;
dst = local_ip.sin.sin_addr;
- info->key.u.ipv4.src = src.s_addr;
- info->key.u.ipv4.dst = dst.s_addr;
+ unclone->key.u.ipv4.src = src.s_addr;
+ unclone->key.u.ipv4.dst = dst.s_addr;
}
vxlan_encap_bypass(skb, vxlan, vxlan, vni, false);
dst_release(ndst);
goto tx_error;
} else if (err) {
if (info) {
+ struct ip_tunnel_info *unclone;
struct in6_addr src, dst;
+ unclone = skb_tunnel_info_unclone(skb);
+ if (unlikely(!unclone))
+ goto tx_error;
+
src = remote_ip.sin6.sin6_addr;
dst = local_ip.sin6.sin6_addr;
- info->key.u.ipv6.src = src;
- info->key.u.ipv6.dst = dst;
+ unclone->key.u.ipv6.src = src;
+ unclone->key.u.ipv6.dst = dst;
}
vxlan_encap_bypass(skb, vxlan, vxlan, vni, false);
if (pad > 0) { /* Pad the frame with zeros */
if (__skb_pad(skb, pad, false))
- goto drop;
+ goto out;
skb_put(skb, pad);
}
}
return NETDEV_TX_OK;
drop:
- dev->stats.tx_dropped++;
kfree_skb(skb);
+out:
+ dev->stats.tx_dropped++;
return NETDEV_TX_OK;
}
struct x25_state {
x25_hdlc_proto settings;
+ bool up;
+ spinlock_t up_lock; /* Protects "up" */
};
static int x25_ioctl(struct net_device *dev, struct ifreq *ifr);
static netdev_tx_t x25_xmit(struct sk_buff *skb, struct net_device *dev)
{
+ hdlc_device *hdlc = dev_to_hdlc(dev);
+ struct x25_state *x25st = state(hdlc);
int result;
/* There should be a pseudo header of 1 byte added by upper layers.
return NETDEV_TX_OK;
}
+ spin_lock_bh(&x25st->up_lock);
+ if (!x25st->up) {
+ spin_unlock_bh(&x25st->up_lock);
+ kfree_skb(skb);
+ return NETDEV_TX_OK;
+ }
+
switch (skb->data[0]) {
case X25_IFACE_DATA: /* Data to be transmitted */
skb_pull(skb, 1);
if ((result = lapb_data_request(dev, skb)) != LAPB_OK)
dev_kfree_skb(skb);
+ spin_unlock_bh(&x25st->up_lock);
return NETDEV_TX_OK;
case X25_IFACE_CONNECT:
break;
}
+ spin_unlock_bh(&x25st->up_lock);
dev_kfree_skb(skb);
return NETDEV_TX_OK;
}
.data_transmit = x25_data_transmit,
};
hdlc_device *hdlc = dev_to_hdlc(dev);
+ struct x25_state *x25st = state(hdlc);
struct lapb_parms_struct params;
int result;
if (result != LAPB_OK)
return -EINVAL;
+ spin_lock_bh(&x25st->up_lock);
+ x25st->up = true;
+ spin_unlock_bh(&x25st->up_lock);
+
return 0;
}
static void x25_close(struct net_device *dev)
{
+ hdlc_device *hdlc = dev_to_hdlc(dev);
+ struct x25_state *x25st = state(hdlc);
+
+ spin_lock_bh(&x25st->up_lock);
+ x25st->up = false;
+ spin_unlock_bh(&x25st->up_lock);
+
lapb_unregister(dev);
}
static int x25_rx(struct sk_buff *skb)
{
struct net_device *dev = skb->dev;
+ hdlc_device *hdlc = dev_to_hdlc(dev);
+ struct x25_state *x25st = state(hdlc);
if ((skb = skb_share_check(skb, GFP_ATOMIC)) == NULL) {
dev->stats.rx_dropped++;
return NET_RX_DROP;
}
- if (lapb_data_received(dev, skb) == LAPB_OK)
+ spin_lock_bh(&x25st->up_lock);
+ if (!x25st->up) {
+ spin_unlock_bh(&x25st->up_lock);
+ kfree_skb(skb);
+ dev->stats.rx_dropped++;
+ return NET_RX_DROP;
+ }
+
+ if (lapb_data_received(dev, skb) == LAPB_OK) {
+ spin_unlock_bh(&x25st->up_lock);
return NET_RX_SUCCESS;
+ }
+ spin_unlock_bh(&x25st->up_lock);
dev->stats.rx_errors++;
dev_kfree_skb_any(skb);
return NET_RX_DROP;
return result;
memcpy(&state(hdlc)->settings, &new_settings, size);
+ state(hdlc)->up = false;
+ spin_lock_init(&state(hdlc)->up_lock);
/* There's no header_ops so hard_header_len should be 0. */
dev->hard_header_len = 0;
MODULE_AUTHOR("Michael Wu <flamingice@sourmilk.net>");
MODULE_AUTHOR("Jouni Malinen <j@w1.fi>");
MODULE_DESCRIPTION("Driver for IEEE 802.11b wireless cards based on ADMtek ADM8211");
-MODULE_SUPPORTED_DEVICE("ADM8211");
MODULE_LICENSE("GPL");
static unsigned int tx_ring_size __read_mostly = 16;
MODULE_AUTHOR("Jiri Slaby");
MODULE_AUTHOR("Nick Kossifidis");
MODULE_DESCRIPTION("Support for 5xxx series of Atheros 802.11 wireless LAN cards.");
-MODULE_SUPPORTED_DEVICE("Atheros 5xxx WLAN cards");
MODULE_LICENSE("Dual BSD/GPL");
static int ath5k_init(struct ieee80211_hw *hw);
MODULE_AUTHOR("Atheros Communications");
MODULE_DESCRIPTION("Support for Atheros 802.11n wireless LAN cards.");
-MODULE_SUPPORTED_DEVICE("Atheros 802.11n WLAN cards");
MODULE_LICENSE("Dual BSD/GPL");
static void ath9k_hw_set_clockrate(struct ath_hw *ah)
MODULE_AUTHOR("Atheros Communications");
MODULE_DESCRIPTION("Support for Atheros 802.11n wireless LAN cards.");
-MODULE_SUPPORTED_DEVICE("Atheros 802.11n WLAN cards");
MODULE_LICENSE("Dual BSD/GPL");
static unsigned int ath9k_debug = ATH_DBG_DEFAULT;
MODULE_AUTHOR("Simon Kelley");
MODULE_DESCRIPTION("Support for Atmel at76c50x 802.11 wireless ethernet cards.");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("Atmel at76c50x wireless cards");
/* The name of the firmware file to be loaded
over-rides any automatic selection */
MODULE_AUTHOR("Simon Kelley");
MODULE_DESCRIPTION("Support for Atmel at76c50x 802.11 wireless ethernet cards.");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("Atmel at76c50x PCMCIA cards");
/*====================================================================*/
MODULE_AUTHOR("Simon Kelley");
MODULE_DESCRIPTION("Support for Atmel at76c50x 802.11 wireless ethernet cards.");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("Atmel at76c506 PCI wireless cards");
static const struct pci_device_id card_ids[] = {
{ 0x1114, 0x0506, PCI_ANY_ID, PCI_ANY_ID },
vif = ifp->vif;
cfg = wdev_to_cfg(&vif->wdev);
cfg->p2p.bss_idx[P2PAPI_BSSCFG_DEVICE].vif = NULL;
- if (locked) {
+ if (!locked) {
rtnl_lock();
wiphy_lock(cfg->wiphy);
cfg80211_unregister_wdev(&vif->wdev);
MODULE_AUTHOR("Broadcom Corporation");
MODULE_DESCRIPTION("Broadcom 802.11n wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Broadcom 802.11n WLAN cards");
MODULE_LICENSE("Dual BSD/GPL");
/* This needs to be adjusted when brcms_firmwares changes */
MODULE_FIRMWARE("brcm/bcm43xx-0.fw");
MODULE_AUTHOR("Broadcom Corporation");
MODULE_DESCRIPTION("Broadcom 802.11n wireless LAN driver utilities.");
-MODULE_SUPPORTED_DEVICE("Broadcom 802.11n WLAN cards");
MODULE_LICENSE("Dual BSD/GPL");
struct sk_buff *brcmu_pkt_buf_get_skb(uint len)
MODULE_DESCRIPTION("Support for Cisco/Aironet 802.11 wireless ethernet cards. "
"Direct support for ISA/PCI/MPI cards and support for PCMCIA when used with airo_cs.");
MODULE_LICENSE("Dual BSD/GPL");
-MODULE_SUPPORTED_DEVICE("Aironet 4500, 4800 and Cisco 340/350");
module_param_hw_array(io, int, ioport, NULL, 0);
module_param_hw_array(irq, int, irq, NULL, 0);
module_param_array(rates, int, NULL, 0);
"cards. This is the module that links the PCMCIA card "
"with the airo module.");
MODULE_LICENSE("Dual BSD/GPL");
-MODULE_SUPPORTED_DEVICE("Aironet 4500, 4800 and Cisco 340 PCMCIA cards");
/*====================================================================*/
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2005-2014 Intel Corporation
+ * Copyright (C) 2005-2014, 2021 Intel Corporation
* Copyright (C) 2015-2017 Intel Deutschland GmbH
*/
#include <linux/sched.h>
if (!list_empty(¬if_wait->notif_waits)) {
struct iwl_notification_wait *w;
- spin_lock(¬if_wait->notif_wait_lock);
+ spin_lock_bh(¬if_wait->notif_wait_lock);
list_for_each_entry(w, ¬if_wait->notif_waits, list) {
int i;
bool found = false;
triggered = true;
}
}
- spin_unlock(¬if_wait->notif_wait_lock);
+ spin_unlock_bh(¬if_wait->notif_wait_lock);
}
return triggered;
{
struct iwl_notification_wait *wait_entry;
- spin_lock(¬if_wait->notif_wait_lock);
+ spin_lock_bh(¬if_wait->notif_wait_lock);
list_for_each_entry(wait_entry, ¬if_wait->notif_waits, list)
wait_entry->aborted = true;
- spin_unlock(¬if_wait->notif_wait_lock);
+ spin_unlock_bh(¬if_wait->notif_wait_lock);
wake_up_all(¬if_wait->notif_waitq);
}
#define IWL_CFG_MAC_TYPE_QNJ 0x36
#define IWL_CFG_MAC_TYPE_SO 0x37
#define IWL_CFG_MAC_TYPE_SNJ 0x42
+#define IWL_CFG_MAC_TYPE_SOF 0x43
#define IWL_CFG_MAC_TYPE_MA 0x44
#define IWL_CFG_RF_TYPE_TH 0x105
REG_CAPA_V2_MCS_9_ALLOWED = BIT(6),
REG_CAPA_V2_WEATHER_DISABLED = BIT(7),
REG_CAPA_V2_40MHZ_ALLOWED = BIT(8),
- REG_CAPA_V2_11AX_DISABLED = BIT(13),
+ REG_CAPA_V2_11AX_DISABLED = BIT(10),
};
/*
return -EINVAL;
/* value zero triggers re-sending the default table to the device */
- if (!op_id)
+ if (!op_id) {
+ mutex_lock(&mvm->mutex);
ret = iwl_rfi_send_config_cmd(mvm, NULL);
- else
+ mutex_unlock(&mvm->mutex);
+ } else {
ret = -EOPNOTSUPP; /* in the future a new table will be added */
+ }
return ret ?: count;
}
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2020 Intel Corporation
+ * Copyright (C) 2020 - 2021 Intel Corporation
*/
#include "mvm.h"
if (!fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_RFIM_SUPPORT))
return -EOPNOTSUPP;
+ lockdep_assert_held(&mvm->mutex);
+
/* in case no table is passed, use the default one */
if (!rfi_table) {
memcpy(cmd.table, iwl_rfi_table, sizeof(cmd.table));
cmd.oem = 1;
}
- mutex_lock(&mvm->mutex);
ret = iwl_mvm_send_cmd(mvm, &hcmd);
- mutex_unlock(&mvm->mutex);
if (ret)
IWL_ERR(mvm, "Failed to send RFI config cmd %d\n", ret);
rx_status->chain_signal[2] = S8_MIN;
}
-static int iwl_mvm_rx_mgmt_crypto(struct ieee80211_sta *sta,
- struct ieee80211_hdr *hdr,
- struct iwl_rx_mpdu_desc *desc,
- u32 status)
+static int iwl_mvm_rx_mgmt_prot(struct ieee80211_sta *sta,
+ struct ieee80211_hdr *hdr,
+ struct iwl_rx_mpdu_desc *desc,
+ u32 status)
{
struct iwl_mvm_sta *mvmsta;
struct iwl_mvm_vif *mvmvif;
u32 len = le16_to_cpu(desc->mpdu_len);
const u8 *frame = (void *)hdr;
+ if ((status & IWL_RX_MPDU_STATUS_SEC_MASK) == IWL_RX_MPDU_STATUS_SEC_NONE)
+ return 0;
+
/*
* For non-beacon, we don't really care. But beacons may
* be filtered out, and we thus need the firmware's replay
IWL_RX_MPDU_STATUS_SEC_UNKNOWN && !mvm->monitor_on)
return -1;
+ if (unlikely(ieee80211_is_mgmt(hdr->frame_control) &&
+ !ieee80211_has_protected(hdr->frame_control)))
+ return iwl_mvm_rx_mgmt_prot(sta, hdr, desc, status);
+
if (!ieee80211_has_protected(hdr->frame_control) ||
(status & IWL_RX_MPDU_STATUS_SEC_MASK) ==
IWL_RX_MPDU_STATUS_SEC_NONE)
stats->flag |= RX_FLAG_DECRYPTED;
return 0;
case RX_MPDU_RES_STATUS_SEC_CMAC_GMAC_ENC:
- return iwl_mvm_rx_mgmt_crypto(sta, hdr, desc, status);
+ break;
default:
/*
* Sometimes we can get frames that were not decrypted
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2018-2020 Intel Corporation
+ * Copyright (C) 2018-2021 Intel Corporation
*/
#include "iwl-trans.h"
#include "iwl-fh.h"
const struct fw_img *fw)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- u32 ltr_val = CSR_LTR_LONG_VAL_AD_NO_SNOOP_REQ |
- u32_encode_bits(CSR_LTR_LONG_VAL_AD_SCALE_USEC,
- CSR_LTR_LONG_VAL_AD_NO_SNOOP_SCALE) |
- u32_encode_bits(250,
- CSR_LTR_LONG_VAL_AD_NO_SNOOP_VAL) |
- CSR_LTR_LONG_VAL_AD_SNOOP_REQ |
- u32_encode_bits(CSR_LTR_LONG_VAL_AD_SCALE_USEC,
- CSR_LTR_LONG_VAL_AD_SNOOP_SCALE) |
- u32_encode_bits(250, CSR_LTR_LONG_VAL_AD_SNOOP_VAL);
struct iwl_context_info_gen3 *ctxt_info_gen3;
struct iwl_prph_scratch *prph_scratch;
struct iwl_prph_scratch_ctrl_cfg *prph_sc_ctrl;
iwl_set_bit(trans, CSR_CTXT_INFO_BOOT_CTRL,
CSR_AUTO_FUNC_BOOT_ENA);
- /*
- * To workaround hardware latency issues during the boot process,
- * initialize the LTR to ~250 usec (see ltr_val above).
- * The firmware initializes this again later (to a smaller value).
- */
- if ((trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210 ||
- trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000) &&
- !trans->trans_cfg->integrated) {
- iwl_write32(trans, CSR_LTR_LONG_VAL_AD, ltr_val);
- } else if (trans->trans_cfg->integrated &&
- trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000) {
- iwl_write_prph(trans, HPM_MAC_LTR_CSR, HPM_MAC_LRT_ENABLE_ALL);
- iwl_write_prph(trans, HPM_UMAC_LTR, ltr_val);
- }
-
- if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210)
- iwl_write_umac_prph(trans, UREG_CPU_INIT_RUN, 1);
- else
- iwl_set_bit(trans, CSR_GP_CNTRL, CSR_AUTO_FUNC_INIT);
-
return 0;
err_free_ctxt_info:
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
* Copyright (C) 2017 Intel Deutschland GmbH
- * Copyright (C) 2018-2020 Intel Corporation
+ * Copyright (C) 2018-2021 Intel Corporation
*/
#include "iwl-trans.h"
#include "iwl-fh.h"
/* kick FW self load */
iwl_write64(trans, CSR_CTXT_INFO_BA, trans_pcie->ctxt_info_dma_addr);
- iwl_write_prph(trans, UREG_CPU_INIT_RUN, 1);
/* Context info will be released upon alive or failure to get one */
IWL_DEV_INFO(0x4DF0, 0x1652, killer1650i_2ax_cfg_qu_b0_hr_b0, NULL),
IWL_DEV_INFO(0x4DF0, 0x2074, iwl_ax201_cfg_qu_hr, NULL),
IWL_DEV_INFO(0x4DF0, 0x4070, iwl_ax201_cfg_qu_hr, NULL),
+ IWL_DEV_INFO(0x4DF0, 0x6074, iwl_ax201_cfg_qu_hr, NULL),
/* So with HR */
IWL_DEV_INFO(0x2725, 0x0090, iwlax211_2ax_cfg_so_gf_a0, NULL),
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
- iwl_cfg_so_a0_hr_a0, iwl_ax201_name)
+ iwl_cfg_so_a0_hr_a0, iwl_ax201_name),
+
+/* So-F with Hr */
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ iwl_cfg_so_a0_hr_a0, iwl_ax203_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ iwl_cfg_so_a0_hr_a0, iwl_ax101_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ iwl_cfg_so_a0_hr_a0, iwl_ax201_name),
+
+/* So-F with Gf */
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB,
+ iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_name),
#endif /* CONFIG_IWLMVM */
};
mutex_unlock(&trans_pcie->mutex);
}
+static void iwl_pcie_set_ltr(struct iwl_trans *trans)
+{
+ u32 ltr_val = CSR_LTR_LONG_VAL_AD_NO_SNOOP_REQ |
+ u32_encode_bits(CSR_LTR_LONG_VAL_AD_SCALE_USEC,
+ CSR_LTR_LONG_VAL_AD_NO_SNOOP_SCALE) |
+ u32_encode_bits(250,
+ CSR_LTR_LONG_VAL_AD_NO_SNOOP_VAL) |
+ CSR_LTR_LONG_VAL_AD_SNOOP_REQ |
+ u32_encode_bits(CSR_LTR_LONG_VAL_AD_SCALE_USEC,
+ CSR_LTR_LONG_VAL_AD_SNOOP_SCALE) |
+ u32_encode_bits(250, CSR_LTR_LONG_VAL_AD_SNOOP_VAL);
+
+ /*
+ * To workaround hardware latency issues during the boot process,
+ * initialize the LTR to ~250 usec (see ltr_val above).
+ * The firmware initializes this again later (to a smaller value).
+ */
+ if ((trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210 ||
+ trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000) &&
+ !trans->trans_cfg->integrated) {
+ iwl_write32(trans, CSR_LTR_LONG_VAL_AD, ltr_val);
+ } else if (trans->trans_cfg->integrated &&
+ trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000) {
+ iwl_write_prph(trans, HPM_MAC_LTR_CSR, HPM_MAC_LRT_ENABLE_ALL);
+ iwl_write_prph(trans, HPM_UMAC_LTR, ltr_val);
+ }
+}
+
int iwl_trans_pcie_gen2_start_fw(struct iwl_trans *trans,
const struct fw_img *fw, bool run_in_rfkill)
{
if (ret)
goto out;
+ iwl_pcie_set_ltr(trans);
+
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210)
+ iwl_write_umac_prph(trans, UREG_CPU_INIT_RUN, 1);
+ else
+ iwl_write_prph(trans, UREG_CPU_INIT_RUN, 1);
+
/* re-check RF-Kill state since we may have missed the interrupt */
hw_rfkill = iwl_pcie_check_hw_rf_kill(trans);
if (hw_rfkill && !run_in_rfkill)
u32 cmd_pos;
const u8 *cmddata[IWL_MAX_CMD_TBS_PER_TFD];
u16 cmdlen[IWL_MAX_CMD_TBS_PER_TFD];
+ unsigned long flags;
if (WARN(!trans->wide_cmd_header &&
group_id > IWL_ALWAYS_LONG_GROUP,
goto free_dup_buf;
}
- spin_lock_bh(&txq->lock);
+ spin_lock_irqsave(&txq->lock, flags);
if (iwl_txq_space(trans, txq) < ((cmd->flags & CMD_ASYNC) ? 2 : 1)) {
- spin_unlock_bh(&txq->lock);
+ spin_unlock_irqrestore(&txq->lock, flags);
IWL_ERR(trans, "No space in command queue\n");
iwl_op_mode_cmd_queue_full(trans->op_mode);
unlock_reg:
spin_unlock(&trans_pcie->reg_lock);
out:
- spin_unlock_bh(&txq->lock);
+ spin_unlock_irqrestore(&txq->lock, flags);
free_dup_buf:
if (idx < 0)
kfree(dup_buf);
MODULE_AUTHOR("Jouni Malinen");
MODULE_DESCRIPTION("Support for Intersil Prism2-based 802.11 wireless LAN "
"cards (PC Card).");
-MODULE_SUPPORTED_DEVICE("Intersil Prism2-based WLAN cards (PC Card)");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Jouni Malinen");
MODULE_DESCRIPTION("Support for Intersil Prism2.5-based 802.11 wireless LAN "
"PCI cards.");
-MODULE_SUPPORTED_DEVICE("Intersil Prism2.5-based WLAN PCI cards");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Jouni Malinen");
MODULE_DESCRIPTION("Support for Intersil Prism2-based 802.11 wireless LAN "
"cards (PLX).");
-MODULE_SUPPORTED_DEVICE("Intersil Prism2-based WLAN cards (PLX)");
MODULE_LICENSE("GPL");
#define MT_WTBLON_TOP_BASE 0x34000
#define MT_WTBLON_TOP(ofs) (MT_WTBLON_TOP_BASE + (ofs))
-#define MT_WTBLON_TOP_WDUCR MT_WTBLON_TOP(0x0)
+#define MT_WTBLON_TOP_WDUCR MT_WTBLON_TOP(0x200)
#define MT_WTBLON_TOP_WDUCR_GROUP GENMASK(2, 0)
-#define MT_WTBL_UPDATE MT_WTBLON_TOP(0x030)
+#define MT_WTBL_UPDATE MT_WTBLON_TOP(0x230)
#define MT_WTBL_UPDATE_WLAN_IDX GENMASK(9, 0)
#define MT_WTBL_UPDATE_ADM_COUNT_CLEAR BIT(12)
#define MT_WTBL_UPDATE_BUSY BIT(31)
MODULE_AUTHOR(DRV_PROJECT);
MODULE_VERSION(DRV_VERSION);
MODULE_DESCRIPTION("Ralink RT2400 PCI & PCMCIA Wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Ralink RT2460 PCI & PCMCIA chipset based cards");
MODULE_DEVICE_TABLE(pci, rt2400pci_device_table);
MODULE_LICENSE("GPL");
MODULE_AUTHOR(DRV_PROJECT);
MODULE_VERSION(DRV_VERSION);
MODULE_DESCRIPTION("Ralink RT2500 PCI & PCMCIA Wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Ralink RT2560 PCI & PCMCIA chipset based cards");
MODULE_DEVICE_TABLE(pci, rt2500pci_device_table);
MODULE_LICENSE("GPL");
MODULE_AUTHOR(DRV_PROJECT);
MODULE_VERSION(DRV_VERSION);
MODULE_DESCRIPTION("Ralink RT2500 USB Wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Ralink RT2570 USB chipset based cards");
MODULE_DEVICE_TABLE(usb, rt2500usb_device_table);
MODULE_LICENSE("GPL");
MODULE_AUTHOR(DRV_PROJECT);
MODULE_VERSION(DRV_VERSION);
MODULE_DESCRIPTION("Ralink RT2800 PCI & PCMCIA Wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Ralink RT2860 PCI & PCMCIA chipset based cards");
MODULE_FIRMWARE(FIRMWARE_RT2860);
MODULE_DEVICE_TABLE(pci, rt2800pci_device_table);
MODULE_LICENSE("GPL");
MODULE_AUTHOR(DRV_PROJECT);
MODULE_VERSION(DRV_VERSION);
MODULE_DESCRIPTION("Ralink RT2800 USB Wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Ralink RT2870 USB chipset based cards");
MODULE_DEVICE_TABLE(usb, rt2800usb_device_table);
MODULE_FIRMWARE(FIRMWARE_RT2870);
MODULE_LICENSE("GPL");
MODULE_AUTHOR(DRV_PROJECT);
MODULE_VERSION(DRV_VERSION);
MODULE_DESCRIPTION("Ralink RT61 PCI & PCMCIA Wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Ralink RT2561, RT2561s & RT2661 "
- "PCI & PCMCIA chipset based cards");
MODULE_DEVICE_TABLE(pci, rt61pci_device_table);
MODULE_FIRMWARE(FIRMWARE_RT2561);
MODULE_FIRMWARE(FIRMWARE_RT2561s);
MODULE_AUTHOR(DRV_PROJECT);
MODULE_VERSION(DRV_VERSION);
MODULE_DESCRIPTION("Ralink RT73 USB Wireless LAN driver.");
-MODULE_SUPPORTED_DEVICE("Ralink RT2571W & RT2671 USB chipset based cards");
MODULE_DEVICE_TABLE(usb, rt73usb_device_table);
MODULE_FIRMWARE(FIRMWARE_RT2571);
MODULE_LICENSE("GPL");
module_exit(rsi_91x_hal_module_exit);
MODULE_AUTHOR("Redpine Signals Inc");
MODULE_DESCRIPTION("Station driver for RSI 91x devices");
-MODULE_SUPPORTED_DEVICE("RSI-91x");
MODULE_VERSION("0.1");
MODULE_LICENSE("Dual BSD/GPL");
MODULE_AUTHOR("Redpine Signals Inc");
MODULE_DESCRIPTION("Common SDIO layer for RSI drivers");
-MODULE_SUPPORTED_DEVICE("RSI-91x");
MODULE_DEVICE_TABLE(sdio, rsi_dev_table);
MODULE_FIRMWARE(FIRMWARE_RSI9113);
MODULE_VERSION("0.1");
MODULE_AUTHOR("Redpine Signals Inc");
MODULE_DESCRIPTION("Common USB layer for RSI drivers");
-MODULE_SUPPORTED_DEVICE("RSI-91x");
MODULE_DEVICE_TABLE(usb, rsi_dev_table);
MODULE_FIRMWARE(FIRMWARE_RSI9113);
MODULE_VERSION("0.1");
#include <net/cfg80211.h>
#include <net/rtnetlink.h>
#include <linux/etherdevice.h>
+#include <linux/math64.h>
#include <linux/module.h>
static struct wiphy *common_wiphy;
scan_result.work);
struct wiphy *wiphy = priv_to_wiphy(priv);
struct cfg80211_scan_info scan_info = { .aborted = false };
+ u64 tsf = div_u64(ktime_get_boottime_ns(), 1000);
informed_bss = cfg80211_inform_bss(wiphy, &channel_5ghz,
CFG80211_BSS_FTYPE_PRESP,
- fake_router_bssid,
- ktime_get_boottime_ns(),
+ fake_router_bssid, tsf,
WLAN_CAPABILITY_ESS, 0,
(void *)&ssid, sizeof(ssid),
DBM_TO_MBM(-50), GFP_KERNEL);
queue_delayed_work(nvme_wq, &ctrl->ka_work, ctrl->kato * HZ);
}
-static int nvme_keep_alive(struct nvme_ctrl *ctrl)
-{
- struct request *rq;
-
- rq = nvme_alloc_request(ctrl->admin_q, &ctrl->ka_cmd,
- BLK_MQ_REQ_RESERVED);
- if (IS_ERR(rq))
- return PTR_ERR(rq);
-
- rq->timeout = ctrl->kato * HZ;
- rq->end_io_data = ctrl;
-
- blk_execute_rq_nowait(NULL, rq, 0, nvme_keep_alive_end_io);
-
- return 0;
-}
-
static void nvme_keep_alive_work(struct work_struct *work)
{
struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
struct nvme_ctrl, ka_work);
bool comp_seen = ctrl->comp_seen;
+ struct request *rq;
if ((ctrl->ctratt & NVME_CTRL_ATTR_TBKAS) && comp_seen) {
dev_dbg(ctrl->device,
return;
}
- if (nvme_keep_alive(ctrl)) {
+ rq = nvme_alloc_request(ctrl->admin_q, &ctrl->ka_cmd,
+ BLK_MQ_REQ_RESERVED | BLK_MQ_REQ_NOWAIT);
+ if (IS_ERR(rq)) {
/* allocation failure, reset the controller */
- dev_err(ctrl->device, "keep-alive failed\n");
+ dev_err(ctrl->device, "keep-alive failed: %ld\n", PTR_ERR(rq));
nvme_reset_ctrl(ctrl);
return;
}
+
+ rq->timeout = ctrl->kato * HZ;
+ rq->end_io_data = ctrl;
+ blk_execute_rq_nowait(NULL, rq, 0, nvme_keep_alive_end_io);
}
static void nvme_start_keep_alive(struct nvme_ctrl *ctrl)
blk_queue_max_write_zeroes_sectors(queue, UINT_MAX);
}
-static void nvme_config_write_zeroes(struct gendisk *disk, struct nvme_ns *ns)
+/*
+ * Even though NVMe spec explicitly states that MDTS is not applicable to the
+ * write-zeroes, we are cautious and limit the size to the controllers
+ * max_hw_sectors value, which is based on the MDTS field and possibly other
+ * limiting factors.
+ */
+static void nvme_config_write_zeroes(struct request_queue *q,
+ struct nvme_ctrl *ctrl)
{
- u64 max_blocks;
-
- if (!(ns->ctrl->oncs & NVME_CTRL_ONCS_WRITE_ZEROES) ||
- (ns->ctrl->quirks & NVME_QUIRK_DISABLE_WRITE_ZEROES))
- return;
- /*
- * Even though NVMe spec explicitly states that MDTS is not
- * applicable to the write-zeroes:- "The restriction does not apply to
- * commands that do not transfer data between the host and the
- * controller (e.g., Write Uncorrectable ro Write Zeroes command).".
- * In order to be more cautious use controller's max_hw_sectors value
- * to configure the maximum sectors for the write-zeroes which is
- * configured based on the controller's MDTS field in the
- * nvme_init_identify() if available.
- */
- if (ns->ctrl->max_hw_sectors == UINT_MAX)
- max_blocks = (u64)USHRT_MAX + 1;
- else
- max_blocks = ns->ctrl->max_hw_sectors + 1;
-
- blk_queue_max_write_zeroes_sectors(disk->queue,
- nvme_lba_to_sect(ns, max_blocks));
+ if ((ctrl->oncs & NVME_CTRL_ONCS_WRITE_ZEROES) &&
+ !(ctrl->quirks & NVME_QUIRK_DISABLE_WRITE_ZEROES))
+ blk_queue_max_write_zeroes_sectors(q, ctrl->max_hw_sectors);
}
static bool nvme_ns_ids_valid(struct nvme_ns_ids *ids)
set_capacity_and_notify(disk, capacity);
nvme_config_discard(disk, ns);
- nvme_config_write_zeroes(disk, ns);
+ nvme_config_write_zeroes(disk->queue, ns->ctrl);
set_disk_ro(disk, (id->nsattr & NVME_NS_ATTR_RO) ||
test_bit(NVME_NS_FORCE_RO, &ns->flags));
/* default is -1: the fail fast mechanism is disabled */
#define NVMF_DEF_FAIL_FAST_TMO -1
+/*
+ * Reserved one command for internal usage. This command is used for sending
+ * the connect command, as well as for the keep alive command on the admin
+ * queue once live.
+ */
+#define NVMF_RESERVED_TAGS 1
+
/*
* Define a host as seen by the target. We allocate one at boot, but also
* allow the override it when creating controllers. This is both to provide
memset(&ctrl->tag_set, 0, sizeof(ctrl->tag_set));
ctrl->tag_set.ops = &nvme_fc_mq_ops;
ctrl->tag_set.queue_depth = ctrl->ctrl.opts->queue_size;
- ctrl->tag_set.reserved_tags = 1; /* fabric connect */
+ ctrl->tag_set.reserved_tags = NVMF_RESERVED_TAGS;
ctrl->tag_set.numa_node = ctrl->ctrl.numa_node;
ctrl->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
ctrl->tag_set.cmd_size =
memset(&ctrl->admin_tag_set, 0, sizeof(ctrl->admin_tag_set));
ctrl->admin_tag_set.ops = &nvme_fc_admin_mq_ops;
ctrl->admin_tag_set.queue_depth = NVME_AQ_MQ_TAG_DEPTH;
- ctrl->admin_tag_set.reserved_tags = 2; /* fabric connect + Keep-Alive */
+ ctrl->admin_tag_set.reserved_tags = NVMF_RESERVED_TAGS;
ctrl->admin_tag_set.numa_node = ctrl->ctrl.numa_node;
ctrl->admin_tag_set.cmd_size =
struct_size((struct nvme_fcp_op_w_sgl *)NULL, priv,
return ret;
ctrl->ctrl.queue_count = nr_io_queues + 1;
- if (ctrl->ctrl.queue_count < 2)
- return 0;
+ if (ctrl->ctrl.queue_count < 2) {
+ dev_err(ctrl->ctrl.device,
+ "unable to set any I/O queues\n");
+ return -ENOMEM;
+ }
dev_info(ctrl->ctrl.device,
"creating %d I/O queues.\n", nr_io_queues);
memset(set, 0, sizeof(*set));
set->ops = &nvme_rdma_admin_mq_ops;
set->queue_depth = NVME_AQ_MQ_TAG_DEPTH;
- set->reserved_tags = 2; /* connect + keep-alive */
+ set->reserved_tags = NVMF_RESERVED_TAGS;
set->numa_node = nctrl->numa_node;
set->cmd_size = sizeof(struct nvme_rdma_request) +
NVME_RDMA_DATA_SGL_SIZE;
memset(set, 0, sizeof(*set));
set->ops = &nvme_rdma_mq_ops;
set->queue_depth = nctrl->sqsize + 1;
- set->reserved_tags = 1; /* fabric connect */
+ set->reserved_tags = NVMF_RESERVED_TAGS;
set->numa_node = nctrl->numa_node;
set->flags = BLK_MQ_F_SHOULD_MERGE;
set->cmd_size = sizeof(struct nvme_rdma_request) +
* directly, otherwise queue io_work. Also, only do that if we
* are on the same cpu, so we don't introduce contention.
*/
- if (queue->io_cpu == __smp_processor_id() &&
+ if (queue->io_cpu == raw_smp_processor_id() &&
sync && empty && mutex_trylock(&queue->send_mutex)) {
queue->more_requests = !last;
nvme_tcp_send_all(queue);
req->pdu_len = le32_to_cpu(pdu->r2t_length);
req->pdu_sent = 0;
+ if (unlikely(!req->pdu_len)) {
+ dev_err(queue->ctrl->ctrl.device,
+ "req %d r2t len is %u, probably a bug...\n",
+ rq->tag, req->pdu_len);
+ return -EPROTO;
+ }
+
if (unlikely(req->data_sent + req->pdu_len > req->data_len)) {
dev_err(queue->ctrl->ctrl.device,
"req %d r2t len %u exceeded data len %u (%zu sent)\n",
memset(set, 0, sizeof(*set));
set->ops = &nvme_tcp_admin_mq_ops;
set->queue_depth = NVME_AQ_MQ_TAG_DEPTH;
- set->reserved_tags = 2; /* connect + keep-alive */
+ set->reserved_tags = NVMF_RESERVED_TAGS;
set->numa_node = nctrl->numa_node;
set->flags = BLK_MQ_F_BLOCKING;
set->cmd_size = sizeof(struct nvme_tcp_request);
memset(set, 0, sizeof(*set));
set->ops = &nvme_tcp_mq_ops;
set->queue_depth = nctrl->sqsize + 1;
- set->reserved_tags = 1; /* fabric connect */
+ set->reserved_tags = NVMF_RESERVED_TAGS;
set->numa_node = nctrl->numa_node;
set->flags = BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_BLOCKING;
set->cmd_size = sizeof(struct nvme_tcp_request);
return ret;
ctrl->queue_count = nr_io_queues + 1;
- if (ctrl->queue_count < 2)
- return 0;
+ if (ctrl->queue_count < 2) {
+ dev_err(ctrl->device,
+ "unable to set any I/O queues\n");
+ return -ENOMEM;
+ }
dev_info(ctrl->device,
"creating %d I/O queues.\n", nr_io_queues);
{
lockdep_assert_held(&ctrl->lock);
- if (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
- nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES ||
- nvmet_cc_mps(ctrl->cc) != 0 ||
+ /*
+ * Only I/O controllers should verify iosqes,iocqes.
+ * Strictly speaking, the spec says a discovery controller
+ * should verify iosqes,iocqes are zeroed, however that
+ * would break backwards compatibility, so don't enforce it.
+ */
+ if (ctrl->subsys->type != NVME_NQN_DISC &&
+ (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
+ nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES)) {
+ ctrl->csts = NVME_CSTS_CFS;
+ return;
+ }
+
+ if (nvmet_cc_mps(ctrl->cc) != 0 ||
nvmet_cc_ams(ctrl->cc) != 0 ||
nvmet_cc_css(ctrl->cc) != 0) {
ctrl->csts = NVME_CSTS_CFS;
memset(&ctrl->admin_tag_set, 0, sizeof(ctrl->admin_tag_set));
ctrl->admin_tag_set.ops = &nvme_loop_admin_mq_ops;
ctrl->admin_tag_set.queue_depth = NVME_AQ_MQ_TAG_DEPTH;
- ctrl->admin_tag_set.reserved_tags = 2; /* connect + keep-alive */
+ ctrl->admin_tag_set.reserved_tags = NVMF_RESERVED_TAGS;
ctrl->admin_tag_set.numa_node = ctrl->ctrl.numa_node;
ctrl->admin_tag_set.cmd_size = sizeof(struct nvme_loop_iod) +
NVME_INLINE_SG_CNT * sizeof(struct scatterlist);
memset(&ctrl->tag_set, 0, sizeof(ctrl->tag_set));
ctrl->tag_set.ops = &nvme_loop_mq_ops;
ctrl->tag_set.queue_depth = ctrl->ctrl.opts->queue_size;
- ctrl->tag_set.reserved_tags = 1; /* fabric connect */
+ ctrl->tag_set.reserved_tags = NVMF_RESERVED_TAGS;
ctrl->tag_set.numa_node = ctrl->ctrl.numa_node;
ctrl->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
ctrl->tag_set.cmd_size = sizeof(struct nvme_loop_iod) +
cmd->rbytes_done += ret;
}
+ nvmet_tcp_unmap_pdu_iovec(cmd);
if (queue->data_digest) {
nvmet_tcp_prep_recv_ddgst(cmd);
return 0;
}
- nvmet_tcp_unmap_pdu_iovec(cmd);
if (!(cmd->flags & NVMET_TCP_F_INIT_FAILED) &&
cmd->rbytes_done == cmd->req.transfer_len) {
*pprev = NULL;
}
-static bool populate_node(const void *blob,
+static int populate_node(const void *blob,
int offset,
void **mem,
struct device_node *dad,
{
struct device_node *np;
const char *pathp;
- unsigned int l, allocl;
+ int len;
- pathp = fdt_get_name(blob, offset, &l);
+ pathp = fdt_get_name(blob, offset, &len);
if (!pathp) {
*pnp = NULL;
- return false;
+ return len;
}
- allocl = ++l;
+ len++;
- np = unflatten_dt_alloc(mem, sizeof(struct device_node) + allocl,
+ np = unflatten_dt_alloc(mem, sizeof(struct device_node) + len,
__alignof__(struct device_node));
if (!dryrun) {
char *fn;
of_node_init(np);
np->full_name = fn = ((char *)np) + sizeof(*np);
- memcpy(fn, pathp, l);
+ memcpy(fn, pathp, len);
if (dad != NULL) {
np->parent = dad;
struct device_node *nps[FDT_MAX_DEPTH];
void *base = mem;
bool dryrun = !base;
+ int ret;
if (nodepp)
*nodepp = NULL;
!of_fdt_device_is_available(blob, offset))
continue;
- if (!populate_node(blob, offset, &mem, nps[depth],
- &nps[depth+1], dryrun))
- return mem - base;
+ ret = populate_node(blob, offset, &mem, nps[depth],
+ &nps[depth+1], dryrun);
+ if (ret < 0)
+ return ret;
if (!dryrun && nodepp && !*nodepp)
*nodepp = nps[depth+1];
{
int size;
void *mem;
+ int ret;
+
+ if (mynodes)
+ *mynodes = NULL;
pr_debug(" -> unflatten_device_tree()\n");
/* First pass, scan for size */
size = unflatten_dt_nodes(blob, NULL, dad, NULL);
- if (size < 0)
+ if (size <= 0)
return NULL;
size = ALIGN(size, 4);
pr_debug(" unflattening %p...\n", mem);
/* Second pass, do actual unflattening */
- unflatten_dt_nodes(blob, mem, dad, mynodes);
+ ret = unflatten_dt_nodes(blob, mem, dad, mynodes);
+
if (be32_to_cpup(mem + size) != 0xdeadbeef)
pr_warn("End of tree marker overwritten: %08x\n",
be32_to_cpup(mem + size));
- if (detached && mynodes) {
+ if (ret <= 0)
+ return NULL;
+
+ if (detached && mynodes && *mynodes) {
of_node_set_flag(*mynodes, OF_DETACHED);
pr_debug("unflattened tree is detached\n");
}
* Copyright (C) 1996-2005 Paul Mackerras.
*/
+#define FDT_ALIGN_SIZE 8
+
/**
* struct alias_prop - Alias property in 'aliases' node
* @link: List node to link the structure in aliases_lookup list
* struct overlay_changeset
* @id: changeset identifier
* @ovcs_list: list on which we are located
- * @fdt: FDT that was unflattened to create @overlay_tree
+ * @fdt: base of memory allocated to hold aligned FDT that was unflattened to create @overlay_tree
* @overlay_tree: expanded device tree that contains the fragment nodes
* @count: count of fragment structures
* @fragments: fragment nodes in the overlay expanded device tree
/**
* init_overlay_changeset() - initialize overlay changeset from overlay tree
* @ovcs: Overlay changeset to build
- * @fdt: the FDT that was unflattened to create @tree
- * @tree: Contains all the overlay fragments and overlay fixup nodes
+ * @fdt: base of memory allocated to hold aligned FDT that was unflattened to create @tree
+ * @tree: Contains the overlay fragments and overlay fixup nodes
*
* Initialize @ovcs. Populate @ovcs->fragments with node information from
* the top level of @tree. The relevant top level nodes are the fragment
* internal documentation
*
* of_overlay_apply() - Create and apply an overlay changeset
- * @fdt: the FDT that was unflattened to create @tree
+ * @fdt: base of memory allocated to hold the aligned FDT
* @tree: Expanded overlay device tree
* @ovcs_id: Pointer to overlay changeset id
*
/*
* after overlay_notify(), ovcs->overlay_tree related pointers may have
* leaked to drivers, so can not kfree() tree, aka ovcs->overlay_tree;
- * and can not free fdt, aka ovcs->fdt
+ * and can not free memory containing aligned fdt. The aligned fdt
+ * is contained within the memory at ovcs->fdt, possibly at an offset
+ * from ovcs->fdt.
*/
ret = overlay_notify(ovcs, OF_OVERLAY_PRE_APPLY);
if (ret) {
int of_overlay_fdt_apply(const void *overlay_fdt, u32 overlay_fdt_size,
int *ovcs_id)
{
- const void *new_fdt;
+ void *new_fdt;
+ void *new_fdt_align;
int ret;
u32 size;
- struct device_node *overlay_root;
+ struct device_node *overlay_root = NULL;
*ovcs_id = 0;
ret = 0;
* Must create permanent copy of FDT because of_fdt_unflatten_tree()
* will create pointers to the passed in FDT in the unflattened tree.
*/
- new_fdt = kmemdup(overlay_fdt, size, GFP_KERNEL);
+ new_fdt = kmalloc(size + FDT_ALIGN_SIZE, GFP_KERNEL);
if (!new_fdt)
return -ENOMEM;
- of_fdt_unflatten_tree(new_fdt, NULL, &overlay_root);
+ new_fdt_align = PTR_ALIGN(new_fdt, FDT_ALIGN_SIZE);
+ memcpy(new_fdt_align, overlay_fdt, size);
+
+ of_fdt_unflatten_tree(new_fdt_align, NULL, &overlay_root);
if (!overlay_root) {
pr_err("unable to unflatten overlay_fdt\n");
ret = -EINVAL;
DEFINE_SIMPLE_PROP(pinctrl8, "pinctrl-8", NULL)
DEFINE_SUFFIX_PROP(regulators, "-supply", NULL)
DEFINE_SUFFIX_PROP(gpio, "-gpio", "#gpio-cells")
-DEFINE_SUFFIX_PROP(gpios, "-gpios", "#gpio-cells")
+
+static struct device_node *parse_gpios(struct device_node *np,
+ const char *prop_name, int index)
+{
+ if (!strcmp_suffix(prop_name, ",nr-gpios"))
+ return NULL;
+
+ return parse_suffix_prop_cells(np, prop_name, index, "-gpios",
+ "#gpio-cells");
+}
static struct device_node *parse_iommu_maps(struct device_node *np,
const char *prop_name, int index)
#include <linux/slab.h>
#include <linux/device.h>
#include <linux/platform_device.h>
+#include <linux/kernel.h>
#include <linux/i2c.h>
#include <linux/i2c-mux.h>
static int __init unittest_data_add(void)
{
void *unittest_data;
- struct device_node *unittest_data_node, *np;
+ void *unittest_data_align;
+ struct device_node *unittest_data_node = NULL, *np;
/*
* __dtb_testcases_begin[] and __dtb_testcases_end[] are magically
* created by cmd_dt_S_dtb in scripts/Makefile.lib
extern uint8_t __dtb_testcases_end[];
const int size = __dtb_testcases_end - __dtb_testcases_begin;
int rc;
+ void *ret;
if (!size) {
- pr_warn("%s: No testcase data to attach; not running tests\n",
- __func__);
+ pr_warn("%s: testcases is empty\n", __func__);
return -ENODATA;
}
/* creating copy */
- unittest_data = kmemdup(__dtb_testcases_begin, size, GFP_KERNEL);
+ unittest_data = kmalloc(size + FDT_ALIGN_SIZE, GFP_KERNEL);
if (!unittest_data)
return -ENOMEM;
- of_fdt_unflatten_tree(unittest_data, NULL, &unittest_data_node);
+ unittest_data_align = PTR_ALIGN(unittest_data, FDT_ALIGN_SIZE);
+ memcpy(unittest_data_align, __dtb_testcases_begin, size);
+
+ ret = of_fdt_unflatten_tree(unittest_data_align, NULL, &unittest_data_node);
+ if (!ret) {
+ pr_warn("%s: unflatten testcases tree failed\n", __func__);
+ kfree(unittest_data);
+ return -ENODATA;
+ }
if (!unittest_data_node) {
- pr_warn("%s: No tree to attach; not running tests\n", __func__);
+ pr_warn("%s: testcases tree is empty\n", __func__);
kfree(unittest_data);
return -ENODATA;
}
MODULE_AUTHOR("Joerg Dorchain <joerg@dorchain.net>");
MODULE_DESCRIPTION("Parport Driver for Amiga builtin Port");
-MODULE_SUPPORTED_DEVICE("Amiga builtin Parallel Port");
MODULE_LICENSE("GPL");
MODULE_ALIAS("platform:amiga-parallel");
MODULE_AUTHOR("Andreas Schwab");
MODULE_DESCRIPTION("Parport Driver for Atari builtin Port");
-MODULE_SUPPORTED_DEVICE("Atari builtin Parallel Port");
MODULE_LICENSE("GPL");
module_init(parport_atari_init)
MODULE_AUTHOR("Helge Deller <deller@gmx.de>");
MODULE_DESCRIPTION("HP-PARISC PC-style parallel port driver");
-MODULE_SUPPORTED_DEVICE("integrated PC-style parallel port");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Joerg Dorchain <joerg@dorchain.net>");
MODULE_DESCRIPTION("Parport Driver for Multiface 3 expansion cards Parallel Port");
-MODULE_SUPPORTED_DEVICE("Multiface 3 Parallel Port");
MODULE_LICENSE("GPL");
module_init(parport_mfc3_init)
MODULE_AUTHOR("Derrick J Brashear");
MODULE_DESCRIPTION("Parport Driver for Sparc bidirectional Port");
-MODULE_SUPPORTED_DEVICE("Sparc Bidirectional Parallel Port");
MODULE_VERSION("2.0");
MODULE_LICENSE("GPL");
if (nbytes >= MAX_DRC_NAME_LEN)
return 0;
- memcpy(drc_name, buf, nbytes);
+ strscpy(drc_name, buf, nbytes + 1);
end = strchr(drc_name, '\n');
- if (!end)
- end = &drc_name[nbytes];
- *end = '\0';
+ if (end)
+ *end = '\0';
rc = dlpar_add_slot(drc_name);
if (rc)
if (nbytes >= MAX_DRC_NAME_LEN)
return 0;
- memcpy(drc_name, buf, nbytes);
+ strscpy(drc_name, buf, nbytes + 1);
end = strchr(drc_name, '\n');
- if (!end)
- end = &drc_name[nbytes];
- *end = '\0';
+ if (end)
+ *end = '\0';
rc = dlpar_remove_slot(drc_name);
if (rc)
pci_dev_put(pdev);
return -EBUSY;
}
+ pci_dev_put(pdev);
- zpci_remove_device(zdev);
+ zpci_remove_device(zdev, false);
rc = zpci_disable_device(zdev);
if (rc)
gpps[i].gpio_base = 0;
break;
case INTEL_GPIO_BASE_NOMAP:
+ break;
default:
break;
}
gpps[i].size = min(gpp_size, npins);
npins -= gpps[i].size;
+ gpps[i].gpio_base = gpps[i].base;
gpps[i].padown_num = padown_num;
/*
if (IS_ERR(regs))
return PTR_ERR(regs);
- /* Determine community features based on the revision */
+ /*
+ * Determine community features based on the revision.
+ * A value of all ones means the device is not present.
+ */
value = readl(regs + REVID);
+ if (value == ~0u)
+ return -ENODEV;
if (((value & REVID_MASK) >> REVID_SHIFT) >= 0x94) {
community->features |= PINCTRL_FEATURE_DEBOUNCE;
community->features |= PINCTRL_FEATURE_1K_PD;
/* Type value spread over 2 registers sets: low, high bit */
sgpio_clrsetbits(bank->priv, REG_INT_TRIGGER, addr.bit,
BIT(addr.port), (!!(type & 0x1)) << addr.port);
- sgpio_clrsetbits(bank->priv, REG_INT_TRIGGER + SGPIO_MAX_BITS, addr.bit,
+ sgpio_clrsetbits(bank->priv, REG_INT_TRIGGER, SGPIO_MAX_BITS + addr.bit,
BIT(addr.port), (!!(type & 0x2)) << addr.port);
if (type == SGPIO_INT_TRG_LEVEL)
static int __maybe_unused rockchip_pinctrl_resume(struct device *dev)
{
struct rockchip_pinctrl *info = dev_get_drvdata(dev);
- int ret = regmap_write(info->regmap_base, RK3288_GRF_GPIO6C_IOMUX,
- rk3288_grf_gpio6c_iomux |
- GPIO6C6_SEL_WRITE_ENABLE);
+ int ret;
- if (ret)
- return ret;
+ if (info->ctrl->type == RK3288) {
+ ret = regmap_write(info->regmap_base, RK3288_GRF_GPIO6C_IOMUX,
+ rk3288_grf_gpio6c_iomux |
+ GPIO6C6_SEL_WRITE_ENABLE);
+ if (ret)
+ return ret;
+ }
return pinctrl_force_default(info->pctl_dev);
}
unsigned long *configs, unsigned int nconfs)
{
struct lpi_pinctrl *pctrl = dev_get_drvdata(pctldev->dev);
- unsigned int param, arg, pullup, strength;
+ unsigned int param, arg, pullup = LPI_GPIO_BIAS_DISABLE, strength = 2;
bool value, output_enabled = false;
const struct lpi_pingroup *g;
unsigned long sval;
[172] = PINGROUP(172, qdss, _, _, _, _, _, _, _, _),
[173] = PINGROUP(173, qdss, _, _, _, _, _, _, _, _),
[174] = PINGROUP(174, qdss, _, _, _, _, _, _, _, _),
- [175] = UFS_RESET(ufs_reset, 0x1be000),
- [176] = SDC_QDSD_PINGROUP(sdc1_rclk, 0x1b3000, 15, 0),
- [177] = SDC_QDSD_PINGROUP(sdc1_clk, 0x1b3000, 13, 6),
- [178] = SDC_QDSD_PINGROUP(sdc1_cmd, 0x1b3000, 11, 3),
- [179] = SDC_QDSD_PINGROUP(sdc1_data, 0x1b3000, 9, 0),
- [180] = SDC_QDSD_PINGROUP(sdc2_clk, 0x1b4000, 14, 6),
- [181] = SDC_QDSD_PINGROUP(sdc2_cmd, 0x1b4000, 11, 3),
- [182] = SDC_QDSD_PINGROUP(sdc2_data, 0x1b4000, 9, 0),
+ [175] = UFS_RESET(ufs_reset, 0xbe000),
+ [176] = SDC_QDSD_PINGROUP(sdc1_rclk, 0xb3004, 0, 6),
+ [177] = SDC_QDSD_PINGROUP(sdc1_clk, 0xb3000, 13, 6),
+ [178] = SDC_QDSD_PINGROUP(sdc1_cmd, 0xb3000, 11, 3),
+ [179] = SDC_QDSD_PINGROUP(sdc1_data, 0xb3000, 9, 0),
+ [180] = SDC_QDSD_PINGROUP(sdc2_clk, 0xb4000, 14, 6),
+ [181] = SDC_QDSD_PINGROUP(sdc2_cmd, 0xb4000, 11, 3),
+ [182] = SDC_QDSD_PINGROUP(sdc2_data, 0xb4000, 9, 0),
};
static const struct msm_pinctrl_soc_data sc7280_pinctrl = {
static const char * const qdss_stm_groups[] = {
"gpio0", "gpio1", "gpio2", "gpio3", "gpio4", "gpio5", "gpio6", "gpio7", "gpio12", "gpio13",
- "gpio14", "gpio15", "gpio16", "gpio17", "gpio18", "gpio19" "gpio20", "gpio21", "gpio22",
+ "gpio14", "gpio15", "gpio16", "gpio17", "gpio18", "gpio19", "gpio20", "gpio21", "gpio22",
"gpio23", "gpio44", "gpio45", "gpio52", "gpio53", "gpio56", "gpio57", "gpio61", "gpio62",
"gpio63", "gpio64", "gpio65", "gpio66",
};
depends on PCI
help
The Intel Platform Controller Hub for Intel Core SoCs provides access
- to Power Management Controller registers via a PCI interface. This
+ to Power Management Controller registers via various interfaces. This
driver can utilize debugging capabilities and supported features as
- exposed by the Power Management Controller.
+ exposed by the Power Management Controller. It also may perform some
+ tasks in the PMC in order to enable transition into the SLPS0 state.
+ It should be selected on all Intel platforms supported by the driver.
Supported features:
- SLP_S0_RESIDENCY counter
- PCH IP Power Gating status
- - LTR Ignore
+ - LTR Ignore / LTR Show
- MPHY/PLL gating status (Sunrisepoint PCH only)
+ - SLPS0 Debug registers (Cannonlake/Icelake PCH)
+ - Low Power Mode registers (Tigerlake and beyond)
+ - PMC quirks as needed to enable SLPS0/S0ix
config INTEL_PMT_CLASS
tristate
sysfs_remove_group(wmi_priv.enumeration_data[instance_id].attr_name_kobj,
&enumeration_attr_group);
}
+ wmi_priv.enumeration_instances_count = 0;
+
kfree(wmi_priv.enumeration_data);
+ wmi_priv.enumeration_data = NULL;
}
sysfs_remove_group(wmi_priv.integer_data[instance_id].attr_name_kobj,
&integer_attr_group);
}
+ wmi_priv.integer_instances_count = 0;
+
kfree(wmi_priv.integer_data);
+ wmi_priv.integer_data = NULL;
}
sysfs_remove_group(wmi_priv.po_data[instance_id].attr_name_kobj,
&po_attr_group);
}
+ wmi_priv.po_instances_count = 0;
+
kfree(wmi_priv.po_data);
+ wmi_priv.po_data = NULL;
}
sysfs_remove_group(wmi_priv.str_data[instance_id].attr_name_kobj,
&str_attr_group);
}
+ wmi_priv.str_instances_count = 0;
+
kfree(wmi_priv.str_data);
+ wmi_priv.str_data = NULL;
}
*/
static int create_attributes_level_sysfs_files(void)
{
- int ret = sysfs_create_file(&wmi_priv.main_dir_kset->kobj, &reset_bios.attr);
+ int ret;
- if (ret) {
- pr_debug("could not create reset_bios file\n");
+ ret = sysfs_create_file(&wmi_priv.main_dir_kset->kobj, &reset_bios.attr);
+ if (ret)
return ret;
- }
ret = sysfs_create_file(&wmi_priv.main_dir_kset->kobj, &pending_reboot.attr);
- if (ret) {
- pr_debug("could not create changing_pending_reboot file\n");
- sysfs_remove_file(&wmi_priv.main_dir_kset->kobj, &reset_bios.attr);
- }
- return ret;
-}
+ if (ret)
+ return ret;
-static void release_reset_bios_data(void)
-{
- sysfs_remove_file(&wmi_priv.main_dir_kset->kobj, &reset_bios.attr);
- sysfs_remove_file(&wmi_priv.main_dir_kset->kobj, &pending_reboot.attr);
+ return 0;
}
static ssize_t wmi_sysman_attr_show(struct kobject *kobj, struct attribute *attr,
*/
static void release_attributes_data(void)
{
- release_reset_bios_data();
-
mutex_lock(&wmi_priv.mutex);
exit_enum_attributes();
exit_int_attributes();
wmi_priv.authentication_dir_kset = NULL;
}
if (wmi_priv.main_dir_kset) {
+ sysfs_remove_file(&wmi_priv.main_dir_kset->kobj, &reset_bios.attr);
+ sysfs_remove_file(&wmi_priv.main_dir_kset->kobj, &pending_reboot.attr);
destroy_attribute_objs(wmi_priv.main_dir_kset);
kset_unregister(wmi_priv.main_dir_kset);
+ wmi_priv.main_dir_kset = NULL;
}
mutex_unlock(&wmi_priv.mutex);
-
}
/**
err_attr_init:
mutex_unlock(&wmi_priv.mutex);
- release_attributes_data();
kfree(obj);
return retval;
}
}
ret = init_bios_attr_set_interface();
- if (ret || !wmi_priv.bios_attr_wdev) {
- pr_debug("failed to initialize set interface\n");
- goto fail_set_interface;
- }
+ if (ret)
+ return ret;
ret = init_bios_attr_pass_interface();
- if (ret || !wmi_priv.password_attr_wdev) {
- pr_debug("failed to initialize pass interface\n");
- goto fail_pass_interface;
+ if (ret)
+ goto err_exit_bios_attr_set_interface;
+
+ if (!wmi_priv.bios_attr_wdev || !wmi_priv.password_attr_wdev) {
+ pr_debug("failed to find set or pass interface\n");
+ ret = -ENODEV;
+ goto err_exit_bios_attr_pass_interface;
}
ret = class_register(&firmware_attributes_class);
if (ret)
- goto fail_class;
+ goto err_exit_bios_attr_pass_interface;
wmi_priv.class_dev = device_create(&firmware_attributes_class, NULL, MKDEV(0, 0),
NULL, "%s", DRIVER_NAME);
if (IS_ERR(wmi_priv.class_dev)) {
ret = PTR_ERR(wmi_priv.class_dev);
- goto fail_classdev;
+ goto err_unregister_class;
}
wmi_priv.main_dir_kset = kset_create_and_add("attributes", NULL,
&wmi_priv.class_dev->kobj);
if (!wmi_priv.main_dir_kset) {
ret = -ENOMEM;
- goto fail_main_kset;
+ goto err_destroy_classdev;
}
wmi_priv.authentication_dir_kset = kset_create_and_add("authentication", NULL,
&wmi_priv.class_dev->kobj);
if (!wmi_priv.authentication_dir_kset) {
ret = -ENOMEM;
- goto fail_authentication_kset;
+ goto err_release_attributes_data;
}
ret = create_attributes_level_sysfs_files();
if (ret) {
pr_debug("could not create reset BIOS attribute\n");
- goto fail_reset_bios;
+ goto err_release_attributes_data;
}
ret = init_bios_attributes(ENUM, DELL_WMI_BIOS_ENUMERATION_ATTRIBUTE_GUID);
if (ret) {
pr_debug("failed to populate enumeration type attributes\n");
- goto fail_create_group;
+ goto err_release_attributes_data;
}
ret = init_bios_attributes(INT, DELL_WMI_BIOS_INTEGER_ATTRIBUTE_GUID);
if (ret) {
pr_debug("failed to populate integer type attributes\n");
- goto fail_create_group;
+ goto err_release_attributes_data;
}
ret = init_bios_attributes(STR, DELL_WMI_BIOS_STRING_ATTRIBUTE_GUID);
if (ret) {
pr_debug("failed to populate string type attributes\n");
- goto fail_create_group;
+ goto err_release_attributes_data;
}
ret = init_bios_attributes(PO, DELL_WMI_BIOS_PASSOBJ_ATTRIBUTE_GUID);
if (ret) {
pr_debug("failed to populate pass object type attributes\n");
- goto fail_create_group;
+ goto err_release_attributes_data;
}
return 0;
-fail_create_group:
+err_release_attributes_data:
release_attributes_data();
-fail_reset_bios:
- if (wmi_priv.authentication_dir_kset) {
- kset_unregister(wmi_priv.authentication_dir_kset);
- wmi_priv.authentication_dir_kset = NULL;
- }
-
-fail_authentication_kset:
- if (wmi_priv.main_dir_kset) {
- kset_unregister(wmi_priv.main_dir_kset);
- wmi_priv.main_dir_kset = NULL;
- }
-
-fail_main_kset:
+err_destroy_classdev:
device_destroy(&firmware_attributes_class, MKDEV(0, 0));
-fail_classdev:
+err_unregister_class:
class_unregister(&firmware_attributes_class);
-fail_class:
+err_exit_bios_attr_pass_interface:
exit_bios_attr_pass_interface();
-fail_pass_interface:
+err_exit_bios_attr_set_interface:
exit_bios_attr_set_interface();
-fail_set_interface:
return ret;
}
DMI_MATCH(DMI_PRODUCT_NAME, "HP Spectre x2 Detachable"),
},
},
+ {
+ .ident = "Lenovo ThinkPad X1 Tablet Gen 2",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "LENOVO"),
+ DMI_MATCH(DMI_PRODUCT_FAMILY, "ThinkPad X1 Tablet Gen 2"),
+ },
+ },
{ }
};
goto wakeup;
/*
- * Switch events will wake the device and report the new switch
- * position to the input subsystem.
+ * Some devices send (duplicate) tablet-mode events when moved
+ * around even though the mode has not changed; and they do this
+ * even when suspended.
+ * Update the switch state in case it changed and then return
+ * without waking up to avoid spurious wakeups.
*/
- if (priv->switches && (event == 0xcc || event == 0xcd))
- goto wakeup;
+ if (event == 0xcc || event == 0xcd) {
+ report_tablet_mode_event(priv->switches, event);
+ return;
+ }
/* Wake up on 5-button array events only. */
if (event == 0xc0 || !priv->array)
wakeup:
pm_wakeup_hard_event(&device->dev);
- if (report_tablet_mode_event(priv->switches, event))
- return;
-
return;
}
};
static const struct key_entry intel_vbtn_switchmap[] = {
- { KE_SW, 0xCA, { .sw = { SW_DOCK, 1 } } }, /* Docked */
- { KE_SW, 0xCB, { .sw = { SW_DOCK, 0 } } }, /* Undocked */
+ /*
+ * SW_DOCK should only be reported for docking stations, but DSDTs using the
+ * intel-vbtn code, always seem to use this for 2-in-1s / convertibles and set
+ * SW_DOCK=1 when in laptop-mode (in tandem with setting SW_TABLET_MODE=0).
+ * This causes userspace to think the laptop is docked to a port-replicator
+ * and to disable suspend-on-lid-close, which is undesirable.
+ * Map the dock events to KEY_IGNORE to avoid this broken SW_DOCK reporting.
+ */
+ { KE_IGNORE, 0xCA, { .sw = { SW_DOCK, 1 } } }, /* Docked */
+ { KE_IGNORE, 0xCB, { .sw = { SW_DOCK, 0 } } }, /* Undocked */
{ KE_SW, 0xCC, { .sw = { SW_TABLET_MODE, 1 } } }, /* Tablet */
{ KE_SW, 0xCD, { .sw = { SW_TABLET_MODE, 0 } } }, /* Laptop */
{ KE_END }
}
DEFINE_SHOW_ATTRIBUTE(pmc_core_pll);
-static ssize_t pmc_core_ltr_ignore_write(struct file *file,
- const char __user *userbuf,
- size_t count, loff_t *ppos)
+static int pmc_core_send_ltr_ignore(u32 value)
{
struct pmc_dev *pmcdev = &pmc;
const struct pmc_reg_map *map = pmcdev->map;
- u32 val, buf_size, fd;
- int err;
-
- buf_size = count < 64 ? count : 64;
-
- err = kstrtou32_from_user(userbuf, buf_size, 10, &val);
- if (err)
- return err;
+ u32 reg;
+ int err = 0;
mutex_lock(&pmcdev->lock);
- if (val > map->ltr_ignore_max) {
+ if (value > map->ltr_ignore_max) {
err = -EINVAL;
goto out_unlock;
}
- fd = pmc_core_reg_read(pmcdev, map->ltr_ignore_offset);
- fd |= (1U << val);
- pmc_core_reg_write(pmcdev, map->ltr_ignore_offset, fd);
+ reg = pmc_core_reg_read(pmcdev, map->ltr_ignore_offset);
+ reg |= BIT(value);
+ pmc_core_reg_write(pmcdev, map->ltr_ignore_offset, reg);
out_unlock:
mutex_unlock(&pmcdev->lock);
+
+ return err;
+}
+
+static ssize_t pmc_core_ltr_ignore_write(struct file *file,
+ const char __user *userbuf,
+ size_t count, loff_t *ppos)
+{
+ u32 buf_size, value;
+ int err;
+
+ buf_size = min_t(u32, count, 64);
+
+ err = kstrtou32_from_user(userbuf, buf_size, 10, &value);
+ if (err)
+ return err;
+
+ err = pmc_core_send_ltr_ignore(value);
+
return err == 0 ? count : err;
}
pmcdev->pmc_xram_read_bit = pmc_core_check_read_lock_bit();
dmi_check_system(pmc_core_dmi_table);
+ /*
+ * On TGL, due to a hardware limitation, the GBE LTR blocks PC10 when
+ * a cable is attached. Tell the PMC to ignore it.
+ */
+ if (pmcdev->map == &tgl_reg_map) {
+ dev_dbg(&pdev->dev, "ignoring GBE LTR\n");
+ pmc_core_send_ltr_ignore(3);
+ }
+
pmc_core_dbgfs_register(pmcdev);
device_initialized = true;
struct intel_pmt_namespace *ns,
struct device *parent)
{
- struct resource res;
+ struct resource res = {0};
struct device *dev;
int ret;
#define CRASH_TYPE_OOBMSM 1
/* Control Flags */
-#define CRASHLOG_FLAG_DISABLE BIT(27)
+#define CRASHLOG_FLAG_DISABLE BIT(28)
/*
- * Bits 28 and 29 control the state of bit 31.
+ * Bits 29 and 30 control the state of bit 31.
*
- * Bit 28 will clear bit 31, if set, allowing a new crashlog to be captured.
- * Bit 29 will immediately trigger a crashlog to be generated, setting bit 31.
- * Bit 30 is read-only and reserved as 0.
+ * Bit 29 will clear bit 31, if set, allowing a new crashlog to be captured.
+ * Bit 30 will immediately trigger a crashlog to be generated, setting bit 31.
* Bit 31 is the read-only status with a 1 indicating log is complete.
*/
-#define CRASHLOG_FLAG_TRIGGER_CLEAR BIT(28)
-#define CRASHLOG_FLAG_TRIGGER_EXECUTE BIT(29)
+#define CRASHLOG_FLAG_TRIGGER_CLEAR BIT(29)
+#define CRASHLOG_FLAG_TRIGGER_EXECUTE BIT(30)
#define CRASHLOG_FLAG_TRIGGER_COMPLETE BIT(31)
#define CRASHLOG_FLAG_TRIGGER_MASK GENMASK(31, 28)
case TP_HKEY_EV_KEY_NUMLOCK:
case TP_HKEY_EV_KEY_FN:
- case TP_HKEY_EV_KEY_FN_ESC:
/* key press events, we just ignore them as long as the EC
* is still reporting them in the normal keyboard stream */
*send_acpi_ev = false;
*ignore_acpi_ev = true;
return true;
+ case TP_HKEY_EV_KEY_FN_ESC:
+ /* Get the media key status to foce the status LED to update */
+ acpi_evalf(hkey_handle, NULL, "GMKS", "v");
+ *send_acpi_ev = false;
+ *ignore_acpi_ev = true;
+ return true;
+
case TP_HKEY_EV_TABLET_CHANGED:
tpacpi_input_send_tabletsw();
hotkey_tablet_mode_notify_change();
* Thinkpad sensor interfaces
*/
+#define DYTC_CMD_QUERY 0 /* To get DYTC status - enable/revision */
+#define DYTC_QUERY_ENABLE_BIT 8 /* Bit 8 - 0 = disabled, 1 = enabled */
+#define DYTC_QUERY_SUBREV_BIT 16 /* Bits 16 - 27 - sub revision */
+#define DYTC_QUERY_REV_BIT 28 /* Bits 28 - 31 - revision */
+
#define DYTC_CMD_GET 2 /* To get current IC function and mode */
#define DYTC_GET_LAPMODE_BIT 17 /* Set when in lapmode */
static bool has_lapsensor;
static bool palm_state;
static bool lap_state;
+static int dytc_version;
static int dytc_command(int command, int *output)
{
return 0;
}
+static int dytc_get_version(void)
+{
+ int err, output;
+
+ /* Check if we've been called before - and just return cached value */
+ if (dytc_version)
+ return dytc_version;
+
+ /* Otherwise query DYTC and extract version information */
+ err = dytc_command(DYTC_CMD_QUERY, &output);
+ /*
+ * If support isn't available (ENODEV) then don't return an error
+ * and don't create the sysfs group
+ */
+ if (err == -ENODEV)
+ return 0;
+ /* For all other errors we can flag the failure */
+ if (err)
+ return err;
+
+ /* Check DYTC is enabled and supports mode setting */
+ if (output & BIT(DYTC_QUERY_ENABLE_BIT))
+ dytc_version = (output >> DYTC_QUERY_REV_BIT) & 0xF;
+
+ return 0;
+}
+
static int lapsensor_get(bool *present, bool *state)
{
int output, err;
if (err)
return err;
}
- if (has_lapsensor) {
+
+ /* Check if we know the DYTC version, if we don't then get it */
+ if (!dytc_version) {
+ err = dytc_get_version();
+ if (err)
+ return err;
+ }
+ /*
+ * Platforms before DYTC version 5 claim to have a lap sensor, but it doesn't work, so we
+ * ignore them
+ */
+ if (has_lapsensor && (dytc_version >= 5)) {
err = sysfs_create_file(&tpacpi_pdev->dev.kobj, &dev_attr_dytc_lapmode.attr);
if (err)
return err;
* DYTC Platform Profile interface
*/
-#define DYTC_CMD_QUERY 0 /* To get DYTC status - enable/revision */
#define DYTC_CMD_SET 1 /* To enable/disable IC function mode */
#define DYTC_CMD_RESET 0x1ff /* To reset back to default */
-#define DYTC_QUERY_ENABLE_BIT 8 /* Bit 8 - 0 = disabled, 1 = enabled */
-#define DYTC_QUERY_SUBREV_BIT 16 /* Bits 16 - 27 - sub revision */
-#define DYTC_QUERY_REV_BIT 28 /* Bits 28 - 31 - revision */
-
#define DYTC_GET_FUNCTION_BIT 8 /* Bits 8-11 - function setting */
#define DYTC_GET_MODE_BIT 12 /* Bits 12-15 - mode setting */
return err;
if (profile == PLATFORM_PROFILE_BALANCED) {
- /* To get back to balanced mode we just issue a reset command */
- err = dytc_command(DYTC_CMD_RESET, &output);
+ /*
+ * To get back to balanced mode we need to issue a reset command.
+ * Note we still need to disable CQL mode before hand and re-enable
+ * it afterwards, otherwise dytc_lapmode gets reset to 0 and stays
+ * stuck at 0 for aprox. 30 minutes.
+ */
+ err = dytc_cql_command(DYTC_CMD_RESET, &output);
if (err)
goto unlock;
} else {
if (err)
return err;
+ /* Check if we know the DYTC version, if we don't then get it */
+ if (!dytc_version) {
+ err = dytc_get_version();
+ if (err)
+ return err;
+ }
/* Check DYTC is enabled and supports mode setting */
- if (output & BIT(DYTC_QUERY_ENABLE_BIT)) {
- /* Only DYTC v5.0 and later has this feature. */
- int dytc_version;
-
- dytc_version = (output >> DYTC_QUERY_REV_BIT) & 0xF;
- if (dytc_version >= 5) {
- dbg_printk(TPACPI_DBG_INIT,
- "DYTC version %d: thermal mode available\n", dytc_version);
- /* Create platform_profile structure and register */
- err = platform_profile_register(&dytc_profile);
- /*
- * If for some reason platform_profiles aren't enabled
- * don't quit terminally.
- */
- if (err)
- return 0;
+ if (dytc_version >= 5) {
+ dbg_printk(TPACPI_DBG_INIT,
+ "DYTC version %d: thermal mode available\n", dytc_version);
+ /* Create platform_profile structure and register */
+ err = platform_profile_register(&dytc_profile);
+ /*
+ * If for some reason platform_profiles aren't enabled
+ * don't quit terminally.
+ */
+ if (err)
+ return 0;
- dytc_profile_available = true;
- /* Ensure initial values are correct */
- dytc_profile_refresh();
- }
+ dytc_profile_available = true;
+ /* Ensure initial values are correct */
+ dytc_profile_refresh();
}
return 0;
}
tmr_add = ptp_qoriq->tmr_add;
adj = tmr_add;
- /* calculate diff as adj*(scaled_ppm/65536)/1000000
- * and round() to the nearest integer
+ /*
+ * Calculate diff and round() to the nearest integer
+ *
+ * diff = adj * (ppb / 1000000000)
+ * = adj * scaled_ppm / 65536000000
*/
- adj *= scaled_ppm;
- diff = div_u64(adj, 8000000);
- diff = (diff >> 13) + ((diff >> 12) & 1);
+ diff = mul_u64_u64_div_u64(adj, scaled_ppm, 32768000000);
+ diff = DIV64_U64_ROUND_UP(diff, 2);
tmr_add = neg_adj ? tmr_add - diff : tmr_add + diff;
-
ptp_qoriq->write(®s->ctrl_regs->tmr_add, tmr_add);
return 0;
return ret;
}
+/**
+ * cec_add_elem - Add an element to the CEC array.
+ * @pfn: page frame number to insert
+ *
+ * Return values:
+ * - <0: on error
+ * - 0: on success
+ * - >0: when the inserted pfn was offlined
+ */
static int cec_add_elem(u64 pfn)
{
struct ce_array *ca = &ce_arr;
+ int count, err, ret = 0;
unsigned int to = 0;
- int count, ret = 0;
/*
* We can be called very early on the identify_cpu() path where we are
if (ca->n == MAX_ELEMS)
WARN_ON(!del_lru_elem_unlocked(ca));
- ret = find_elem(ca, pfn, &to);
- if (ret < 0) {
+ err = find_elem(ca, pfn, &to);
+ if (err < 0) {
/*
* Shift range [to-end] to make room for one more element.
*/
static const struct regulator_desc regulators[] = {
BD9571MWV_REG("VD09", "vd09", VD09, avs_ops, 0, 0x7f,
- 0x80, 600000, 10000, 0x3c),
+ 0x6f, 600000, 10000, 0x3c),
BD9571MWV_REG("VD18", "vd18", VD18, vid_ops, BD9571MWV_VD18_VID, 0xf,
16, 1625000, 25000, 0),
BD9571MWV_REG("VD25", "vd25", VD25, vid_ops, BD9571MWV_VD25_VID, 0xf,
11, 2800000, 100000, 0),
BD9571MWV_REG("DVFS", "dvfs", DVFS, reg_ops,
BD9571MWV_DVFS_MONIVDAC, 0x7f,
- 0x80, 600000, 10000, 0x3c),
+ 0x6f, 600000, 10000, 0x3c),
};
#ifdef CONFIG_PM_SLEEP
{
struct bd9571mwv_reg *bdreg = dev_get_drvdata(dev);
- return sprintf(buf, "%s\n", bdreg->bkup_mode_enabled ? "on" : "off");
+ return sysfs_emit(buf, "%s\n", bdreg->bkup_mode_enabled ? "on" : "off");
}
static ssize_t backup_mode_store(struct device *dev,
&config);
if (IS_ERR(rdev)) {
dev_err(&pdev->dev, "failed to register %s regulator\n",
- pdev->name);
+ regulators[i].name);
return PTR_ERR(rdev);
}
}
if (len == 0)
return NULL;
+ /*
+ * GNU binutils do not support multiple address spaces. The GNU
+ * linker's default linker script places IRAM at an arbitrary high
+ * offset, in order to differentiate it from DRAM. Hence we need to
+ * strip the artificial offset in the IRAM addresses coming from the
+ * ELF file.
+ *
+ * The TI proprietary linker would never set those higher IRAM address
+ * bits anyway. PRU architecture limits the program counter to 16-bit
+ * word-address range. This in turn corresponds to 18-bit IRAM
+ * byte-address range for ELF.
+ *
+ * Two more bits are added just in case to make the final 20-bit mask.
+ * Idea is to have a safeguard in case TI decides to add banking
+ * in future SoCs.
+ */
+ da &= 0xfffff;
+
if (da >= PRU_IRAM_DA &&
da + len <= PRU_IRAM_DA + pru->mem_regions[PRU_IOMEM_IRAM].size) {
offset = da - PRU_IRAM_DA;
break;
}
- if (pru->data->is_k3 && is_iram) {
+ if (pru->data->is_k3) {
ret = pru_rproc_memcpy(ptr, elf_data + phdr->p_offset,
filesz);
if (ret) {
memset_io(base, 0, resource_size(&imem));
_reloc.base = base;
- _reloc.num_entries = resource_size(&imem) / PIL_RELOC_ENTRY_SIZE;
+ _reloc.num_entries = (u32)resource_size(&imem) / PIL_RELOC_ENTRY_SIZE;
return 0;
}
MODULE_AUTHOR("Holger Smolinski <Holger.Smolinski@de.ibm.com>");
MODULE_DESCRIPTION("Linux on S/390 DASD device driver,"
" Copyright IBM Corp. 2000");
-MODULE_SUPPORTED_DEVICE("dasd");
MODULE_LICENSE("GPL");
/*
MODULE_AUTHOR("Eric Brower <ebrower@usa.net>");
MODULE_DESCRIPTION("7-Segment Display driver for Sun Microsystems CP1400/1500");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("d7s");
struct d7s {
void __iomem *regs;
MODULE_AUTHOR("Hewlett-Packard Company");
MODULE_DESCRIPTION("Driver for HP Smart Array Controller version " \
HPSA_DRIVER_VERSION);
-MODULE_SUPPORTED_DEVICE("HP Smart Array Controllers");
MODULE_VERSION(HPSA_DRIVER_VERSION);
MODULE_LICENSE("GPL");
MODULE_ALIAS("cciss");
#ifndef HPSA_CMD_H
#define HPSA_CMD_H
+#include <linux/compiler.h>
+
+#include <linux/build_bug.h> /* static_assert */
+#include <linux/stddef.h> /* offsetof */
+
/* general boundary defintions */
#define SENSEINFOBYTES 32 /* may vary between hbas */
#define SG_ENTRIES_IN_CMD 32 /* Max SG entries excluding chain blocks */
MAX_EXT_TARGETS + 1) /* + 1 is for the controller itself */
/* SCSI-3 Commands */
-#pragma pack(1)
-
#define HPSA_INQUIRY 0x12
struct InquiryData {
u8 data_byte[36];
-};
+} __packed;
#define HPSA_REPORT_LOG 0xc2 /* Report Logical LUNs */
#define HPSA_REPORT_PHYS 0xc3 /* Report Physical LUNs */
u8 xor_mult[2]; /**< XOR multipliers for this position,
* valid for data disks only */
u8 reserved[2];
-};
+} __packed;
struct raid_map_data {
__le32 structure_size; /* Size of entire structure in bytes */
__le16 dekindex; /* Data encryption key index. */
u8 reserved[16];
struct raid_map_disk_data data[RAID_MAP_MAX_ENTRIES];
-};
+} __packed;
struct ReportLUNdata {
u8 LUNListLength[4];
u8 extended_response_flag;
u8 reserved[3];
u8 LUN[HPSA_MAX_LUN][8];
-};
+} __packed;
struct ext_report_lun_entry {
u8 lunid[8];
u8 lun_count; /* multi-lun device, how many luns */
u8 redundant_paths;
u32 ioaccel_handle; /* ioaccel1 only uses lower 16 bits */
-};
+} __packed;
struct ReportExtendedLUNdata {
u8 LUNListLength[4];
u8 extended_response_flag;
u8 reserved[3];
struct ext_report_lun_entry LUN[HPSA_MAX_PHYS_LUN];
-};
+} __packed;
struct SenseSubsystem_info {
u8 reserved[36];
u8 portname[8];
u8 reserved1[1108];
-};
+} __packed;
/* BMIC commands */
#define BMIC_READ 0x26
u8 Targ:6;
u8 Mode:2; /* b10 */
} LogUnit;
-};
+} __packed;
struct PhysDevAddr {
u32 TargetId:24;
u32 Mode:2;
/* 2 level target device addr */
union SCSI3Addr Target[2];
-};
+} __packed;
struct LogDevAddr {
u32 VolId:30;
u32 Mode:2;
u8 reserved[4];
-};
+} __packed;
union LUNAddr {
u8 LunAddrBytes[8];
union SCSI3Addr SCSI3Lun[4];
struct PhysDevAddr PhysDev;
struct LogDevAddr LogDev;
-};
+} __packed;
struct CommandListHeader {
u8 ReplyQueue;
__le16 SGTotal;
__le64 tag;
union LUNAddr LUN;
-};
+} __packed;
struct RequestBlock {
u8 CDBLen;
#define GET_DIR(tad) (((tad) >> 6) & 0x03)
u16 Timeout;
u8 CDB[16];
-};
+} __packed;
struct ErrDescriptor {
__le64 Addr;
__le32 Len;
-};
+} __packed;
struct SGDescriptor {
__le64 Addr;
__le32 Len;
__le32 Ext;
-};
+} __packed;
union MoreErrInfo {
struct {
u8 offense_num; /* byte # of offense 0-base */
u32 offense_value;
} Invalid_Cmd;
-};
+} __packed;
+
struct ErrorInfo {
u8 ScsiStatus;
u8 SenseLen;
u32 ResidualCnt;
union MoreErrInfo MoreErrInfo;
u8 SenseInfo[SENSEINFOBYTES];
-};
+} __packed;
/* Command types */
#define CMD_IOCTL_PEND 0x01
#define CMD_SCSI 0x03
atomic_t refcount; /* Must be last to avoid memset in hpsa_cmd_init() */
} __aligned(COMMANDLIST_ALIGNMENT);
+/*
+ * Make sure our embedded atomic variable is aligned. Otherwise we break atomic
+ * operations on architectures that don't support unaligned atomics like IA64.
+ *
+ * The assert guards against reintroductin against unwanted __packed to
+ * the struct CommandList.
+ */
+static_assert(offsetof(struct CommandList, refcount) % __alignof__(atomic_t) == 0);
+
/* Max S/G elements in I/O accelerator command */
#define IOACCEL1_MAXSGENTRIES 24
#define IOACCEL2_MAXSGENTRIES 28
__le64 host_addr; /* 0x70 - 0x77 */
u8 CISS_LUN[8]; /* 0x78 - 0x7F */
struct SGDescriptor SG[IOACCEL1_MAXSGENTRIES];
-} __aligned(IOACCEL1_COMMANDLIST_ALIGNMENT);
+} __packed __aligned(IOACCEL1_COMMANDLIST_ALIGNMENT);
#define IOACCEL1_FUNCTION_SCSIIO 0x00
#define IOACCEL1_SGLOFFSET 32
u8 chain_indicator;
#define IOACCEL2_CHAIN 0x80
#define IOACCEL2_LAST_SG 0x40
-};
+} __packed;
/*
* SCSI Response Format structure for IO Accelerator Mode 2
u8 sense_data_len; /* sense/response data length */
u8 resid_cnt[4]; /* residual count */
u8 sense_data_buff[32]; /* sense/response data buffer */
-};
+} __packed;
/*
* Structure for I/O accelerator (mode 2 or m2) commands.
__le32 tweak_upper; /* Encryption tweak, upper 4 bytes */
struct ioaccel2_sg_element sg[IOACCEL2_MAXSGENTRIES];
struct io_accel2_scsi_response error_data;
-} __aligned(IOACCEL2_COMMANDLIST_ALIGNMENT);
+} __packed __aligned(IOACCEL2_COMMANDLIST_ALIGNMENT);
/*
* defines for Mode 2 command struct
__le64 abort_tag; /* cciss tag of SCSI cmd or TMF to abort */
__le64 error_ptr; /* Error Pointer */
__le32 error_len; /* Error Length */
-} __aligned(IOACCEL2_COMMANDLIST_ALIGNMENT);
+} __packed __aligned(IOACCEL2_COMMANDLIST_ALIGNMENT);
/* Configuration Table Structure */
struct HostWrite {
__le32 command_pool_addr_hi;
__le32 CoalIntDelay;
__le32 CoalIntCount;
-};
+} __packed;
#define SIMPLE_MODE 0x02
#define PERFORMANT_MODE 0x04
#define HPSA_EVENT_NOTIFY_ACCEL_IO_PATH_STATE_CHANGE (1 << 30)
#define HPSA_EVENT_NOTIFY_ACCEL_IO_PATH_CONFIG_CHANGE (1 << 31)
__le32 clear_event_notify;
-};
+} __packed;
#define NUM_BLOCKFETCH_ENTRIES 8
struct TransTable_struct {
__le32 RepQCtrAddrHigh32;
#define MAX_REPLY_QUEUES 64
struct vals32 RepQAddr[MAX_REPLY_QUEUES];
-};
+} __packed;
struct hpsa_pci_info {
unsigned char bus;
unsigned char dev_fn;
unsigned short domain;
u32 board_id;
-};
+} __packed;
struct bmic_identify_controller {
u8 configured_logical_drive_count; /* offset 0 */
u8 pad2[136];
u8 controller_mode; /* offset 292 */
u8 pad3[32];
-};
+} __packed;
struct bmic_identify_physical_device {
u8 max_link_rate[256];
u8 neg_phys_link_rate[256];
u8 box_conn_name[8];
-} __attribute((aligned(512)));
+} __packed __attribute((aligned(512)));
struct bmic_sense_subsystem_info {
u8 primary_slot_number;
u8 secondary_array_serial_number[32];
u8 secondary_cache_serial_number[32];
u8 pad[332];
-};
+} __packed;
struct bmic_sense_storage_box_params {
u8 reserved[36];
u8 reserver_3[84];
u8 phys_connector[2];
u8 reserved_4[296];
-};
+} __packed;
-#pragma pack()
#endif /* HPSA_CMD_H */
return 0;
}
+/**
+ * ibmvfc_event_is_free - Check if event is free or not
+ * @evt: ibmvfc event struct
+ *
+ * Returns:
+ * true / false
+ **/
+static bool ibmvfc_event_is_free(struct ibmvfc_event *evt)
+{
+ struct ibmvfc_event *loop_evt;
+
+ list_for_each_entry(loop_evt, &evt->queue->free, queue_list)
+ if (loop_evt == evt)
+ return true;
+
+ return false;
+}
+
/**
* ibmvfc_wait_for_ops - Wait for ops to complete
* @vhost: ibmvfc host struct
{
struct ibmvfc_event *evt;
DECLARE_COMPLETION_ONSTACK(comp);
- int wait;
+ int wait, i, q_index, q_size;
unsigned long flags;
signed long timeout = IBMVFC_ABORT_WAIT_TIMEOUT * HZ;
+ struct ibmvfc_queue *queues;
ENTER;
+ if (vhost->mq_enabled && vhost->using_channels) {
+ queues = vhost->scsi_scrqs.scrqs;
+ q_size = vhost->scsi_scrqs.active_queues;
+ } else {
+ queues = &vhost->crq;
+ q_size = 1;
+ }
+
do {
wait = 0;
- spin_lock_irqsave(&vhost->crq.l_lock, flags);
- list_for_each_entry(evt, &vhost->crq.sent, queue_list) {
- if (match(evt, device)) {
- evt->eh_comp = ∁
- wait++;
+ spin_lock_irqsave(vhost->host->host_lock, flags);
+ for (q_index = 0; q_index < q_size; q_index++) {
+ spin_lock(&queues[q_index].l_lock);
+ for (i = 0; i < queues[q_index].evt_pool.size; i++) {
+ evt = &queues[q_index].evt_pool.events[i];
+ if (!ibmvfc_event_is_free(evt)) {
+ if (match(evt, device)) {
+ evt->eh_comp = ∁
+ wait++;
+ }
+ }
}
+ spin_unlock(&queues[q_index].l_lock);
}
- spin_unlock_irqrestore(&vhost->crq.l_lock, flags);
+ spin_unlock_irqrestore(vhost->host->host_lock, flags);
if (wait) {
timeout = wait_for_completion_timeout(&comp, timeout);
if (!timeout) {
wait = 0;
- spin_lock_irqsave(&vhost->crq.l_lock, flags);
- list_for_each_entry(evt, &vhost->crq.sent, queue_list) {
- if (match(evt, device)) {
- evt->eh_comp = NULL;
- wait++;
+ spin_lock_irqsave(vhost->host->host_lock, flags);
+ for (q_index = 0; q_index < q_size; q_index++) {
+ spin_lock(&queues[q_index].l_lock);
+ for (i = 0; i < queues[q_index].evt_pool.size; i++) {
+ evt = &queues[q_index].evt_pool.events[i];
+ if (!ibmvfc_event_is_free(evt)) {
+ if (match(evt, device)) {
+ evt->eh_comp = NULL;
+ wait++;
+ }
+ }
}
+ spin_unlock(&queues[q_index].l_lock);
}
- spin_unlock_irqrestore(&vhost->crq.l_lock, flags);
+ spin_unlock_irqrestore(vhost->host->host_lock, flags);
if (wait)
dev_err(vhost->dev, "Timed out waiting for aborted commands\n");
LEAVE;
vhost->disc_buf_dma);
dma_free_coherent(vhost->dev, sizeof(*vhost->login_buf),
vhost->login_buf, vhost->login_buf_dma);
+ dma_free_coherent(vhost->dev, sizeof(*vhost->channel_setup_buf),
+ vhost->channel_setup_buf, vhost->channel_setup_dma);
dma_pool_destroy(vhost->sg_pool);
ibmvfc_free_queue(vhost, async_q);
LEAVE;
memset(dstbuf, 0, 33);
size = (nbytes < 32) ? nbytes : 32;
if (copy_from_user(dstbuf, buf, size))
- return 0;
+ return -EFAULT;
if (dent == phba->debug_InjErrLBA) {
if ((dstbuf[0] == 'o') && (dstbuf[1] == 'f') &&
}
if ((tmp == 0) && (kstrtoull(dstbuf, 0, &tmp)))
- return 0;
+ return -EINVAL;
if (dent == phba->debug_writeGuard)
phba->lpfc_injerr_wgrd_cnt = (uint32_t)tmp;
ioc->pend_os_device_add_sz++;
ioc->pend_os_device_add = kzalloc(ioc->pend_os_device_add_sz,
GFP_KERNEL);
- if (!ioc->pend_os_device_add)
+ if (!ioc->pend_os_device_add) {
+ r = -ENOMEM;
goto out_free_resources;
+ }
ioc->device_remove_in_progress_sz = ioc->pend_os_device_add_sz;
ioc->device_remove_in_progress =
kzalloc(ioc->device_remove_in_progress_sz, GFP_KERNEL);
- if (!ioc->device_remove_in_progress)
+ if (!ioc->device_remove_in_progress) {
+ r = -ENOMEM;
goto out_free_resources;
+ }
ioc->fwfault_debug = mpt3sas_fwfault_debug;
* And add this object to port_table_list.
*/
if (!ioc->multipath_on_hba) {
- port = kzalloc(sizeof(struct hba_port), GFP_KERNEL);
+ port = kzalloc(sizeof(struct hba_port), GFP_ATOMIC);
if (!port)
return NULL;
if (cs->mmio_base) {
cs->disable_intr(cs);
iounmap(cs->mmio_base);
+ cs->mmio_base = NULL;
}
if (cs->irq)
free_irq(cs->irq, cs);
if (cs->io_addr)
release_region(cs->io_addr, 0x80);
- iounmap(cs->mmio_base);
pci_set_drvdata(pdev, NULL);
pci_disable_device(pdev);
scsi_host_put(cs->host);
MODULE_AUTHOR("YOKOTA Hiroshi <yokota@netlab.is.tsukuba.ac.jp>");
MODULE_DESCRIPTION("WorkBit NinjaSCSI-3 / NinjaSCSI-32Bi(16bit) PCMCIA SCSI host adapter module");
-MODULE_SUPPORTED_DEVICE("sd,sr,sg,st");
MODULE_LICENSE("GPL");
#include "nsp_io.h"
PM8001_EVENT_LOG_SIZE;
pm8001_ha->main_cfg_tbl.pm8001_tbl.iop_event_log_option = 0x01;
pm8001_ha->main_cfg_tbl.pm8001_tbl.fatal_err_interrupt = 0x01;
- for (i = 0; i < PM8001_MAX_INB_NUM; i++) {
+ for (i = 0; i < pm8001_ha->max_q_num; i++) {
pm8001_ha->inbnd_q_tbl[i].element_pri_size_cnt =
PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x00<<30);
pm8001_ha->inbnd_q_tbl[i].upper_base_addr =
pm8001_ha->inbnd_q_tbl[i].producer_idx = 0;
pm8001_ha->inbnd_q_tbl[i].consumer_index = 0;
}
- for (i = 0; i < PM8001_MAX_OUTB_NUM; i++) {
+ for (i = 0; i < pm8001_ha->max_q_num; i++) {
pm8001_ha->outbnd_q_tbl[i].element_size_cnt =
PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x01<<30);
pm8001_ha->outbnd_q_tbl[i].upper_base_addr =
read_outbnd_queue_table(pm8001_ha);
/* update main config table ,inbound table and outbound table */
update_main_config_table(pm8001_ha);
- for (i = 0; i < PM8001_MAX_INB_NUM; i++)
+ for (i = 0; i < pm8001_ha->max_q_num; i++)
update_inbnd_queue_table(pm8001_ha, i);
- for (i = 0; i < PM8001_MAX_OUTB_NUM; i++)
+ for (i = 0; i < pm8001_ha->max_q_num; i++)
update_outbnd_queue_table(pm8001_ha, i);
/* 8081 controller donot require these operations */
if (deviceid != 0x8081 && deviceid != 0x0042) {
if (!qedi->global_queues[i]) {
QEDI_ERR(&qedi->dbg_ctx,
"Unable to allocation global queue %d.\n", i);
+ status = -ENOMEM;
goto mem_alloc_failure;
}
if (!qpair->fw_started || (cmd->reset_count != qpair->chip_reset) ||
(cmd->sess && cmd->sess->deleted)) {
cmd->state = QLA_TGT_STATE_PROCESSED;
- res = 0;
- goto free;
+ return 0;
}
ql_dbg_qp(ql_dbg_tgt, qpair, 0xe018,
res = qlt_pre_xmit_response(cmd, &prm, xmit_type, scsi_status,
&full_req_cnt);
- if (unlikely(res != 0))
- goto free;
+ if (unlikely(res != 0)) {
+ return res;
+ }
spin_lock_irqsave(qpair->qp_lock_ptr, flags);
vha->flags.online, qla2x00_reset_active(vha),
cmd->reset_count, qpair->chip_reset);
spin_unlock_irqrestore(qpair->qp_lock_ptr, flags);
- res = 0;
- goto free;
+ return 0;
}
/* Does F/W have an IOCBs for this request */
qlt_unmap_sg(vha, cmd);
spin_unlock_irqrestore(qpair->qp_lock_ptr, flags);
-free:
- vha->hw->tgt.tgt_ops->free_cmd(cmd);
return res;
}
EXPORT_SYMBOL(qlt_xmit_response);
(min(1270, ((ql) > 0) ? (QLA_TGT_DATASEGS_PER_CMD_24XX + \
QLA_TGT_DATASEGS_PER_CONT_24XX*((ql) - 1)) : 0))
#endif
-#endif
#define GET_TARGET_ID(ha, iocb) ((HAS_EXTENDED_IDS(ha)) \
? le16_to_cpu((iocb)->u.isp2x.target.extended) \
#ifndef CTIO_RET_TYPE
#define CTIO_RET_TYPE 0x17 /* CTIO return entry */
#define ATIO_TYPE7 0x06 /* Accept target I/O entry for 24xx */
+#endif
struct fcp_hdr {
uint8_t r_ctl;
{
struct qla_tgt_cmd *cmd = container_of(se_cmd,
struct qla_tgt_cmd, se_cmd);
- struct scsi_qla_host *vha = cmd->vha;
if (cmd->aborted) {
/* Cmd can loop during Q-full. tcm_qla2xxx_aborted_task
cmd->se_cmd.transport_state,
cmd->se_cmd.t_state,
cmd->se_cmd.se_cmd_flags);
- vha->hw->tgt.tgt_ops->free_cmd(cmd);
return 0;
}
{
struct qla_tgt_cmd *cmd = container_of(se_cmd,
struct qla_tgt_cmd, se_cmd);
- struct scsi_qla_host *vha = cmd->vha;
int xmit_type = QLA_TGT_XMIT_STATUS;
if (cmd->aborted) {
cmd, kref_read(&cmd->se_cmd.cmd_kref),
cmd->se_cmd.transport_state, cmd->se_cmd.t_state,
cmd->se_cmd.se_cmd_flags);
- vha->hw->tgt.tgt_ops->free_cmd(cmd);
return 0;
}
cmd->bufflen = se_cmd->data_length;
*/
mutex_lock(&conn_mutex);
conn->transport->stop_conn(conn, flag);
+ conn->state = ISCSI_CONN_DOWN;
mutex_unlock(&conn_mutex);
}
default:
err = transport->set_param(conn, ev->u.set_param.param,
data, ev->u.set_param.len);
+ if ((conn->state == ISCSI_CONN_BOUND) ||
+ (conn->state == ISCSI_CONN_UP)) {
+ err = transport->set_param(conn, ev->u.set_param.param,
+ data, ev->u.set_param.len);
+ } else {
+ return -ENOTCONN;
+ }
}
return err;
mutex_lock(&conn->ep_mutex);
conn->ep = NULL;
mutex_unlock(&conn->ep_mutex);
+ conn->state = ISCSI_CONN_DOWN;
}
transport->ep_disconnect(ep);
ev->r.retcode = transport->bind_conn(session, conn,
ev->u.b_conn.transport_eph,
ev->u.b_conn.is_leading);
+ if (!ev->r.retcode)
+ conn->state = ISCSI_CONN_BOUND;
mutex_unlock(&conn_mutex);
if (ev->r.retcode || !transport->ep_connect)
static const char *const connection_state_names[] = {
[ISCSI_CONN_UP] = "up",
[ISCSI_CONN_DOWN] = "down",
- [ISCSI_CONN_FAILED] = "failed"
+ [ISCSI_CONN_FAILED] = "failed",
+ [ISCSI_CONN_BOUND] = "bound"
};
static ssize_t show_conn_state(struct device *dev,
res = mutex_lock_interruptible(&rport->mutex);
if (res)
goto out;
- if (rport->state != SRP_RPORT_FAIL_FAST)
+ if (rport->state != SRP_RPORT_FAIL_FAST && rport->state != SRP_RPORT_LOST)
/*
* sdev state must be SDEV_TRANSPORT_OFFLINE, transition
* to SDEV_BLOCK is illegal. Calling scsi_target_unblock()
static void sd_zbc_update_wp_offset_workfn(struct work_struct *work)
{
struct scsi_disk *sdkp;
+ unsigned long flags;
unsigned int zno;
int ret;
sdkp = container_of(work, struct scsi_disk, zone_wp_offset_work);
- spin_lock_bh(&sdkp->zones_wp_offset_lock);
+ spin_lock_irqsave(&sdkp->zones_wp_offset_lock, flags);
for (zno = 0; zno < sdkp->nr_zones; zno++) {
if (sdkp->zones_wp_offset[zno] != SD_ZBC_UPDATING_WP_OFST)
continue;
- spin_unlock_bh(&sdkp->zones_wp_offset_lock);
+ spin_unlock_irqrestore(&sdkp->zones_wp_offset_lock, flags);
ret = sd_zbc_do_report_zones(sdkp, sdkp->zone_wp_update_buf,
SD_BUF_SIZE,
zno * sdkp->zone_blocks, true);
- spin_lock_bh(&sdkp->zones_wp_offset_lock);
+ spin_lock_irqsave(&sdkp->zones_wp_offset_lock, flags);
if (!ret)
sd_zbc_parse_report(sdkp, sdkp->zone_wp_update_buf + 64,
zno, sd_zbc_update_wp_offset_cb,
sdkp);
}
- spin_unlock_bh(&sdkp->zones_wp_offset_lock);
+ spin_unlock_irqrestore(&sdkp->zones_wp_offset_lock, flags);
scsi_device_put(sdkp->device);
}
struct request *rq = cmd->request;
struct scsi_disk *sdkp = scsi_disk(rq->rq_disk);
unsigned int wp_offset, zno = blk_rq_zone_no(rq);
+ unsigned long flags;
blk_status_t ret;
ret = sd_zbc_cmnd_checks(cmd);
if (!blk_req_zone_write_trylock(rq))
return BLK_STS_ZONE_RESOURCE;
- spin_lock_bh(&sdkp->zones_wp_offset_lock);
+ spin_lock_irqsave(&sdkp->zones_wp_offset_lock, flags);
wp_offset = sdkp->zones_wp_offset[zno];
switch (wp_offset) {
case SD_ZBC_INVALID_WP_OFST:
*lba += wp_offset;
}
- spin_unlock_bh(&sdkp->zones_wp_offset_lock);
+ spin_unlock_irqrestore(&sdkp->zones_wp_offset_lock, flags);
if (ret)
blk_req_zone_write_unlock(rq);
return ret;
struct scsi_disk *sdkp = scsi_disk(rq->rq_disk);
unsigned int zno = blk_rq_zone_no(rq);
enum req_opf op = req_op(rq);
+ unsigned long flags;
/*
* If we got an error for a command that needs updating the write
* invalid to force an update from disk the next time a zone append
* command is issued.
*/
- spin_lock_bh(&sdkp->zones_wp_offset_lock);
+ spin_lock_irqsave(&sdkp->zones_wp_offset_lock, flags);
if (result && op != REQ_OP_ZONE_RESET_ALL) {
if (op == REQ_OP_ZONE_APPEND) {
}
unlock_wp_offset:
- spin_unlock_bh(&sdkp->zones_wp_offset_lock);
+ spin_unlock_irqrestore(&sdkp->zones_wp_offset_lock, flags);
return good_bytes;
}
MODULE_AUTHOR("Microsemi");
MODULE_DESCRIPTION("Driver for Microsemi Smart Family Controller version "
DRIVER_VERSION);
-MODULE_SUPPORTED_DEVICE("Microsemi Smart Family Controllers");
MODULE_VERSION(DRIVER_VERSION);
MODULE_LICENSE("GPL");
spin_lock(&st_use_lock);
if (STp->in_use) {
spin_unlock(&st_use_lock);
- scsi_tape_put(STp);
DEBC_printk(STp, "Device already in use.\n");
+ scsi_tape_put(STp);
return (-EBUSY);
}
if (!hba->vreg_info.vccq2 || !hba->vreg_info.vcc)
return;
- if (lpm & !hba->vreg_info.vcc->enabled)
+ if (lpm && !hba->vreg_info.vcc->enabled)
regulator_set_mode(hba->vreg_info.vccq2->reg,
REGULATOR_MODE_IDLE);
else if (!lpm)
DECLARE_COMPLETION_ONSTACK(wait);
struct request *req;
unsigned long flags;
- int free_slot, task_tag, err;
+ int task_tag, err;
/*
- * Get free slot, sleep if slots are unavailable.
- * Even though we use wait_event() which sleeps indefinitely,
- * the maximum wait time is bounded by %TM_CMD_TIMEOUT.
+ * blk_get_request() is used here only to get a free tag.
*/
req = blk_get_request(q, REQ_OP_DRV_OUT, 0);
if (IS_ERR(req))
return PTR_ERR(req);
req->end_io_data = &wait;
- free_slot = req->tag;
- WARN_ON_ONCE(free_slot < 0 || free_slot >= hba->nutmrs);
ufshcd_hold(hba, false);
spin_lock_irqsave(host->host_lock, flags);
- task_tag = hba->nutrs + free_slot;
+ blk_mq_start_request(req);
+ task_tag = req->tag;
treq->req_header.dword_0 |= cpu_to_be32(task_tag);
- memcpy(hba->utmrdl_base_addr + free_slot, treq, sizeof(*treq));
- ufshcd_vops_setup_task_mgmt(hba, free_slot, tm_function);
+ memcpy(hba->utmrdl_base_addr + task_tag, treq, sizeof(*treq));
+ ufshcd_vops_setup_task_mgmt(hba, task_tag, tm_function);
/* send command to the controller */
- __set_bit(free_slot, &hba->outstanding_tasks);
+ __set_bit(task_tag, &hba->outstanding_tasks);
/* Make sure descriptors are ready before ringing the task doorbell */
wmb();
- ufshcd_writel(hba, 1 << free_slot, REG_UTP_TASK_REQ_DOOR_BELL);
+ ufshcd_writel(hba, 1 << task_tag, REG_UTP_TASK_REQ_DOOR_BELL);
/* Make sure that doorbell is committed immediately */
wmb();
ufshcd_add_tm_upiu_trace(hba, task_tag, UFS_TM_ERR);
dev_err(hba->dev, "%s: task management cmd 0x%.2x timed-out\n",
__func__, tm_function);
- if (ufshcd_clear_tm_cmd(hba, free_slot))
- dev_WARN(hba->dev, "%s: unable clear tm cmd (slot %d) after timeout\n",
- __func__, free_slot);
+ if (ufshcd_clear_tm_cmd(hba, task_tag))
+ dev_WARN(hba->dev, "%s: unable to clear tm cmd (slot %d) after timeout\n",
+ __func__, task_tag);
err = -ETIMEDOUT;
} else {
err = 0;
- memcpy(treq, hba->utmrdl_base_addr + free_slot, sizeof(*treq));
+ memcpy(treq, hba->utmrdl_base_addr + task_tag, sizeof(*treq));
ufshcd_add_tm_upiu_trace(hba, task_tag, UFS_TM_COMP);
}
spin_lock_irqsave(hba->host->host_lock, flags);
- __clear_bit(free_slot, &hba->outstanding_tasks);
+ __clear_bit(task_tag, &hba->outstanding_tasks);
spin_unlock_irqrestore(hba->host->host_lock, flags);
+ ufshcd_release(hba);
blk_put_request(req);
- ufshcd_release(hba);
return err;
}
MODULE_AUTHOR("Adrian McMenamin <adrian@mcmen.demon.co.uk>");
MODULE_DESCRIPTION("Maple bus driver for Dreamcast");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{SEGA, Dreamcast/Maple}}");
static void maple_dma_handler(struct work_struct *work);
static void maple_vblank_handler(struct work_struct *work);
__be32 tag;
struct qm_fd fd;
u8 __reserved3[32];
-} __packed;
+} __packed __aligned(8);
#define QM_EQCR_VERB_VBIT 0x80
#define QM_EQCR_VERB_CMD_MASK 0x61 /* but only one value; */
#define QM_EQCR_VERB_CMD_ENQUEUE 0x01
#include <linux/platform_device.h>
#include <linux/printk.h>
#include <linux/module.h>
-#include <linux/errno.h>
#include <linux/io.h>
#include <linux/reboot.h>
#include <linux/acpi.h>
#include <linux/clk.h>
-#include <linux/console.h>
#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include <linux/io.h>
struct device *dev;
void __iomem *base;
struct clk_bulk_data ahb_clks[NUM_AHB_CLKS];
- struct geni_icc_path to_core;
};
static const char * const icc_path_names[] = {"qup-core", "qup-config",
"qup-memory"};
-static struct geni_wrapper *earlycon_wrapper;
-
#define QUP_HW_VER_REG 0x4
/* Common SE registers */
}
EXPORT_SYMBOL(geni_icc_disable);
-void geni_remove_earlycon_icc_vote(void)
-{
- struct platform_device *pdev;
- struct geni_wrapper *wrapper;
- struct device_node *parent;
- struct device_node *child;
-
- if (!earlycon_wrapper)
- return;
-
- wrapper = earlycon_wrapper;
- parent = of_get_next_parent(wrapper->dev->of_node);
- for_each_child_of_node(parent, child) {
- if (!of_device_is_compatible(child, "qcom,geni-se-qup"))
- continue;
-
- pdev = of_find_device_by_node(child);
- if (!pdev)
- continue;
-
- wrapper = platform_get_drvdata(pdev);
- icc_put(wrapper->to_core.path);
- wrapper->to_core.path = NULL;
-
- }
- of_node_put(parent);
-
- earlycon_wrapper = NULL;
-}
-EXPORT_SYMBOL(geni_remove_earlycon_icc_vote);
-
static int geni_se_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct resource *res;
struct geni_wrapper *wrapper;
- struct console __maybe_unused *bcon;
- bool __maybe_unused has_earlycon = false;
int ret;
wrapper = devm_kzalloc(dev, sizeof(*wrapper), GFP_KERNEL);
}
}
-#ifdef CONFIG_SERIAL_EARLYCON
- for_each_console(bcon) {
- if (!strcmp(bcon->name, "qcom_geni")) {
- has_earlycon = true;
- break;
- }
- }
- if (!has_earlycon)
- goto exit;
-
- wrapper->to_core.path = devm_of_icc_get(dev, "qup-core");
- if (IS_ERR(wrapper->to_core.path))
- return PTR_ERR(wrapper->to_core.path);
- /*
- * Put minmal BW request on core clocks on behalf of early console.
- * The vote will be removed earlycon exit function.
- *
- * Note: We are putting vote on each QUP wrapper instead only to which
- * earlycon is connected because QUP core clock of different wrapper
- * share same voltage domain. If core1 is put to 0, then core2 will
- * also run at 0, if not voted. Default ICC vote will be removed ASA
- * we touch any of the core clock.
- * core1 = core2 = max(core1, core2)
- */
- ret = icc_set_bw(wrapper->to_core.path, GENI_DEFAULT_BW,
- GENI_DEFAULT_BW);
- if (ret) {
- dev_err(&pdev->dev, "%s: ICC BW voting failed for core: %d\n",
- __func__, ret);
- return ret;
- }
-
- if (of_get_compatible_child(pdev->dev.of_node, "qcom,geni-debug-uart"))
- earlycon_wrapper = wrapper;
- of_node_put(pdev->dev.of_node);
-exit:
-#endif
dev_set_drvdata(dev, wrapper);
dev_dbg(dev, "GENI SE Driver probed\n");
return devm_of_platform_populate(dev);
{
.name = "l3init", .base = 0x4ae07300,
.pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon,
- .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012,
+ .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01,
.clkdm_name = "pcie"
},
{
reset->prm->data->name, id);
exit:
- if (reset->clkdm)
+ if (reset->clkdm) {
+ /* At least dra7 iva needs a delay before clkdm idle */
+ if (has_rstst)
+ udelay(1);
pdata->clkdm_allow_idle(reset->clkdm);
+ }
return ret;
}
cqspi = spi_master_get_devdata(master);
cqspi->pdev = pdev;
+ platform_set_drvdata(pdev, cqspi);
/* Obtain configuration from OF. */
ret = cqspi_of_get_pdata(cqspi);
devpriv->amcc + AMCC_OP_REG_INTCSR);
ret = request_irq(pcidev->irq, cb_pcidas_interrupt, IRQF_SHARED,
- dev->board_name, dev);
+ "cb_pcidas", dev);
if (ret) {
dev_dbg(dev->class_dev, "unable to allocate irq %d\n",
pcidev->irq);
init_stc_registers(dev);
retval = request_irq(pcidev->irq, handle_interrupt, IRQF_SHARED,
- dev->board_name, dev);
+ "cb_pcidas64", dev);
if (retval) {
dev_dbg(dev->class_dev, "unable to allocate irq %u\n",
pcidev->irq);
MODULE_AUTHOR("Manuel Gebele <forensixs@gmx.de>");
MODULE_DESCRIPTION("Velleman USB Board Low-Level Driver");
-MODULE_SUPPORTED_DEVICE("K8055/K8061 aka VM110/VM140");
MODULE_LICENSE("GPL");
int i;
for (i = 0; i < chan->numgangports; i++) {
- host1x_syncpt_free(chan->mw_ack_sp[i]);
- host1x_syncpt_free(chan->frame_start_sp[i]);
+ host1x_syncpt_put(chan->mw_ack_sp[i]);
+ host1x_syncpt_put(chan->frame_start_sp[i]);
}
}
mw_sp = host1x_syncpt_request(&vi->client, flags);
if (!mw_sp) {
dev_err(vi->dev, "failed to request memory ack syncpoint\n");
- host1x_syncpt_free(fs_sp);
+ host1x_syncpt_put(fs_sp);
ret = -ENOMEM;
goto free_syncpts;
}
bool bWithAironetIE;
bool bCkipSupported;
bool bCcxRmEnable;
- u16 CcxRmState[2];
+ u8 CcxRmState[2];
bool bMBssidValid;
u8 MBssidMask;
u8 MBssid[ETH_ALEN];
info_element->data[2] == 0x96 &&
info_element->data[3] == 0x01) {
if (info_element->len == 6) {
- memcpy(network->CcxRmState, &info_element[4], 2);
+ memcpy(network->CcxRmState, &info_element->data[4], 2);
if (network->CcxRmState[0] != 0)
network->bCcxRmEnable = true;
else
u16 reserved;
struct ieee80211_cts data;
u16 reserved2;
-} __packed;
+} __packed __aligned(2);
struct vnt_cts_fb {
struct vnt_phy_field b;
__le16 cts_duration_ba_f1;
struct ieee80211_cts data;
u16 reserved2;
-} __packed;
+} __packed __aligned(2);
struct vnt_tx_fifo_head {
u8 tx_key[WLAN_KEY_LEN_CCMP];
target_get_sess_cmd(&cmd->se_cmd, true);
+ cmd->se_cmd.tag = (__force u32)cmd->init_task_tag;
cmd->sense_reason = target_cmd_init_cdb(&cmd->se_cmd, hdr->cdb);
if (cmd->sense_reason) {
if (cmd->sense_reason == TCM_OUT_OF_RESOURCES) {
if (cmd->sense_reason)
goto attach_cmd;
- /* only used for printks or comparing with ->ref_task_tag */
- cmd->se_cmd.tag = (__force u32)cmd->init_task_tag;
cmd->sense_reason = target_cmd_parse_cdb(&cmd->se_cmd);
if (cmd->sense_reason)
goto attach_cmd;
if (!bio) {
new_bio:
nr_vecs = bio_max_segs(nr_pages);
- nr_pages -= nr_vecs;
/*
* Calls bio_kmalloc() and sets bio->bi_end_io()
*/
return 0;
fail:
+ if (bio)
+ bio_put(bio);
+ while (req->bio) {
+ bio = req->bio;
+ req->bio = bio->bi_next;
+ bio_put(bio);
+ }
+ req->biotail = NULL;
return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
}
MODULE_AUTHOR("Linaro");
MODULE_DESCRIPTION("OP-TEE driver");
-MODULE_SUPPORTED_DEVICE("");
MODULE_VERSION("1.0");
MODULE_LICENSE("GPL v2");
MODULE_ALIAS("platform:optee");
{
struct cooling_dev_stats *stats = cdev->stats;
+ if (!stats)
+ return;
+
spin_lock(&stats->lock);
if (stats->state == new_state)
ret = tb_retimer_nvm_add(rt);
if (ret) {
dev_err(&rt->dev, "failed to add NVM devices: %d\n", ret);
- device_del(&rt->dev);
+ device_unregister(&rt->dev);
return ret;
}
*/
int tb_retimer_scan(struct tb_port *port)
{
- u32 status[TB_MAX_RETIMER_INDEX] = {};
+ u32 status[TB_MAX_RETIMER_INDEX + 1] = {};
int ret, i, last_idx = 0;
if (!port->cap_usb4)
tb_dump_port(port->sw->tb, &port->config);
- /* Control port does not need HopID allocation */
- if (port->port) {
- ida_init(&port->in_hopids);
- ida_init(&port->out_hopids);
- }
-
INIT_LIST_HEAD(&port->list);
return 0;
dma_port_free(sw->dma_port);
tb_switch_for_each_port(sw, port) {
- if (!port->disabled) {
- ida_destroy(&port->in_hopids);
- ida_destroy(&port->out_hopids);
- }
+ ida_destroy(&port->in_hopids);
+ ida_destroy(&port->out_hopids);
}
kfree(sw->uuid);
/* minimum setup for tb_find_cap and tb_drom_read to work */
sw->ports[i].sw = sw;
sw->ports[i].port = i;
+
+ /* Control port does not need HopID allocation */
+ if (i) {
+ ida_init(&sw->ports[i].in_hopids);
+ ida_init(&sw->ports[i].out_hopids);
+ }
}
ret = tb_switch_find_vse_cap(sw, TB_VSE_CAP_PLUG_EVENTS);
parent->boot = true;
parent = tb_switch_parent(parent);
}
+ } else if (tb_tunnel_is_dp(tunnel)) {
+ /* Keep the domain from powering down */
+ pm_runtime_get_sync(&tunnel->src_port->sw->dev);
+ pm_runtime_get_sync(&tunnel->dst_port->sw->dev);
}
list_add_tail(&tunnel->list, &tcm->tunnel_list);
MODULE_AUTHOR("Michael Anderson <mjanders@us.ibm.com>");
MODULE_DESCRIPTION("IBM iSeries Serial IOA driver");
-MODULE_SUPPORTED_DEVICE
- ("IBM iSeries 2745, 2771, 2772, 2742, 2793 and 2805 Communications adapters");
MODULE_LICENSE("GPL");
MODULE_FIRMWARE("icom_call_setup.bin");
MODULE_FIRMWARE("icom_res_dce.bin");
MODULE_AUTHOR("Digi International, https://www.digi.com");
MODULE_DESCRIPTION("Driver for the Digi International Neo and Classic PCI based product line");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("jsm");
#define JSM_DRIVER_NAME "jsm"
#define NR_PORTS 32
struct console *con) { }
#endif
-static int qcom_geni_serial_earlycon_exit(struct console *con)
-{
- geni_remove_earlycon_icc_vote();
- return 0;
-}
-
static struct qcom_geni_private_data earlycon_private_data;
static int __init qcom_geni_serial_earlycon_setup(struct earlycon_device *dev,
writel(stop_bit_len, uport->membase + SE_UART_TX_STOP_BIT_LEN);
dev->con->write = qcom_geni_serial_earlycon_write;
- dev->con->exit = qcom_geni_serial_earlycon_exit;
dev->con->setup = NULL;
qcom_geni_serial_enable_early_read(&se, dev->con);
return -ESHUTDOWN;
}
+ /* Requests has been dequeued during disabling endpoint. */
+ if (!(pep->ep_state & EP_ENABLED))
+ return 0;
+
spin_lock_irqsave(&pdev->lock, flags);
ret = cdnsp_ep_dequeue(pep, to_cdnsp_request(request));
spin_unlock_irqrestore(&pdev->lock, flags);
* inverted in the first TDs isoc TRB.
*/
field = TRB_TYPE(TRB_ISOC) | TRB_TLBPC(last_burst_pkt) |
- start_cycle ? 0 : 1 | TRB_SIA | TRB_TBC(burst_count);
+ TRB_SIA | TRB_TBC(burst_count);
+
+ if (!start_cycle)
+ field |= TRB_CYCLE;
/* Fill the rest of the TRB fields, and remaining normal TRBs. */
for (i = 0; i < trbs_per_td; i++) {
#define acm_send_break(acm, ms) \
acm_ctrl_msg(acm, USB_CDC_REQ_SEND_BREAK, ms, NULL, 0)
-static void acm_kill_urbs(struct acm *acm)
+static void acm_poison_urbs(struct acm *acm)
{
int i;
- usb_kill_urb(acm->ctrlurb);
+ usb_poison_urb(acm->ctrlurb);
for (i = 0; i < ACM_NW; i++)
- usb_kill_urb(acm->wb[i].urb);
+ usb_poison_urb(acm->wb[i].urb);
for (i = 0; i < acm->rx_buflimit; i++)
- usb_kill_urb(acm->read_urbs[i]);
+ usb_poison_urb(acm->read_urbs[i]);
+}
+
+static void acm_unpoison_urbs(struct acm *acm)
+{
+ int i;
+
+ for (i = 0; i < acm->rx_buflimit; i++)
+ usb_unpoison_urb(acm->read_urbs[i]);
+ for (i = 0; i < ACM_NW; i++)
+ usb_unpoison_urb(acm->wb[i].urb);
+ usb_unpoison_urb(acm->ctrlurb);
}
+
/*
* Write buffer management.
* All of these assume proper locks taken by the caller.
rc = usb_submit_urb(wb->urb, GFP_ATOMIC);
if (rc < 0) {
- dev_err(&acm->data->dev,
- "%s - usb_submit_urb(write bulk) failed: %d\n",
- __func__, rc);
+ if (rc != -EPERM)
+ dev_err(&acm->data->dev,
+ "%s - usb_submit_urb(write bulk) failed: %d\n",
+ __func__, rc);
acm_write_done(acm, wb);
}
return rc;
acm->iocount.dsr++;
if (difference & ACM_CTRL_DCD)
acm->iocount.dcd++;
- if (newctrl & ACM_CTRL_BRK)
+ if (newctrl & ACM_CTRL_BRK) {
acm->iocount.brk++;
+ tty_insert_flip_char(&acm->port, 0, TTY_BREAK);
+ }
if (newctrl & ACM_CTRL_RI)
acm->iocount.rng++;
if (newctrl & ACM_CTRL_FRAMING)
dev_vdbg(&acm->data->dev, "got urb %d, len %d, status %d\n",
rb->index, urb->actual_length, status);
- if (!acm->dev) {
- dev_dbg(&acm->data->dev, "%s - disconnected\n", __func__);
- return;
- }
-
switch (status) {
case 0:
usb_mark_last_busy(acm->dev);
res = acm_set_control(acm, val);
if (res && (acm->ctrl_caps & USB_CDC_CAP_LINE))
- dev_err(&acm->control->dev, "failed to set dtr/rts\n");
+ /* This is broken in too many devices to spam the logs */
+ dev_dbg(&acm->control->dev, "failed to set dtr/rts\n");
}
static int acm_port_activate(struct tty_port *port, struct tty_struct *tty)
* Need to grab write_lock to prevent race with resume, but no need to
* hold it due to the tty-port initialised flag.
*/
+ acm_poison_urbs(acm);
spin_lock_irq(&acm->write_lock);
spin_unlock_irq(&acm->write_lock);
usb_autopm_put_interface_async(acm->control);
}
- acm_kill_urbs(acm);
+ acm_unpoison_urbs(acm);
+
}
static void acm_tty_cleanup(struct tty_struct *tty)
if (!combined_interfaces && intf != control_interface)
return -ENODEV;
- if (!combined_interfaces && usb_interface_claimed(data_interface)) {
- /* valid in this context */
- dev_dbg(&intf->dev, "The data interface isn't available\n");
- return -EBUSY;
- }
-
-
if (data_interface->cur_altsetting->desc.bNumEndpoints < 2 ||
control_interface->cur_altsetting->desc.bNumEndpoints == 0)
return -EINVAL;
dev_dbg(&intf->dev, "interfaces are valid\n");
acm = kzalloc(sizeof(struct acm), GFP_KERNEL);
- if (acm == NULL)
- goto alloc_fail;
+ if (!acm)
+ return -ENOMEM;
tty_port_init(&acm->port);
acm->port.ops = &acm_port_ops;
minor = acm_alloc_minor(acm);
if (minor < 0)
- goto alloc_fail1;
+ goto err_put_port;
acm->minor = minor;
acm->dev = usb_dev;
buf = usb_alloc_coherent(usb_dev, ctrlsize, GFP_KERNEL, &acm->ctrl_dma);
if (!buf)
- goto alloc_fail1;
+ goto err_put_port;
acm->ctrl_buffer = buf;
if (acm_write_buffers_alloc(acm) < 0)
- goto alloc_fail2;
+ goto err_free_ctrl_buffer;
acm->ctrlurb = usb_alloc_urb(0, GFP_KERNEL);
if (!acm->ctrlurb)
- goto alloc_fail3;
+ goto err_free_write_buffers;
for (i = 0; i < num_rx_buf; i++) {
struct acm_rb *rb = &(acm->read_buffers[i]);
rb->base = usb_alloc_coherent(acm->dev, readsize, GFP_KERNEL,
&rb->dma);
if (!rb->base)
- goto alloc_fail4;
+ goto err_free_read_urbs;
rb->index = i;
rb->instance = acm;
urb = usb_alloc_urb(0, GFP_KERNEL);
if (!urb)
- goto alloc_fail4;
+ goto err_free_read_urbs;
urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP;
urb->transfer_dma = rb->dma;
struct acm_wb *snd = &(acm->wb[i]);
snd->urb = usb_alloc_urb(0, GFP_KERNEL);
- if (snd->urb == NULL)
- goto alloc_fail5;
+ if (!snd->urb)
+ goto err_free_write_urbs;
if (usb_endpoint_xfer_int(epwrite))
usb_fill_int_urb(snd->urb, usb_dev, acm->out,
i = device_create_file(&intf->dev, &dev_attr_bmCapabilities);
if (i < 0)
- goto alloc_fail5;
+ goto err_free_write_urbs;
if (h.usb_cdc_country_functional_desc) { /* export the country data */
struct usb_cdc_country_functional_desc * cfd =
acm->nb_index = 0;
acm->nb_size = 0;
- dev_info(&intf->dev, "ttyACM%d: USB ACM device\n", minor);
-
acm->line.dwDTERate = cpu_to_le32(9600);
acm->line.bDataBits = 8;
acm_set_line(acm, &acm->line);
- usb_driver_claim_interface(&acm_driver, data_interface, acm);
- usb_set_intfdata(data_interface, acm);
+ if (!acm->combined_interfaces) {
+ rv = usb_driver_claim_interface(&acm_driver, data_interface, acm);
+ if (rv)
+ goto err_remove_files;
+ }
tty_dev = tty_port_register_device(&acm->port, acm_tty_driver, minor,
&control_interface->dev);
if (IS_ERR(tty_dev)) {
rv = PTR_ERR(tty_dev);
- goto alloc_fail6;
+ goto err_release_data_interface;
}
if (quirks & CLEAR_HALT_CONDITIONS) {
usb_clear_halt(usb_dev, acm->out);
}
+ dev_info(&intf->dev, "ttyACM%d: USB ACM device\n", minor);
+
return 0;
-alloc_fail6:
+
+err_release_data_interface:
+ if (!acm->combined_interfaces) {
+ /* Clear driver data so that disconnect() returns early. */
+ usb_set_intfdata(data_interface, NULL);
+ usb_driver_release_interface(&acm_driver, data_interface);
+ }
+err_remove_files:
if (acm->country_codes) {
device_remove_file(&acm->control->dev,
&dev_attr_wCountryCodes);
device_remove_file(&acm->control->dev,
&dev_attr_iCountryCodeRelDate);
- kfree(acm->country_codes);
}
device_remove_file(&acm->control->dev, &dev_attr_bmCapabilities);
-alloc_fail5:
- usb_set_intfdata(intf, NULL);
+err_free_write_urbs:
for (i = 0; i < ACM_NW; i++)
usb_free_urb(acm->wb[i].urb);
-alloc_fail4:
+err_free_read_urbs:
for (i = 0; i < num_rx_buf; i++)
usb_free_urb(acm->read_urbs[i]);
acm_read_buffers_free(acm);
usb_free_urb(acm->ctrlurb);
-alloc_fail3:
+err_free_write_buffers:
acm_write_buffers_free(acm);
-alloc_fail2:
+err_free_ctrl_buffer:
usb_free_coherent(usb_dev, ctrlsize, acm->ctrl_buffer, acm->ctrl_dma);
-alloc_fail1:
+err_put_port:
tty_port_put(&acm->port);
-alloc_fail:
+
return rv;
}
if (!acm)
return;
- mutex_lock(&acm->mutex);
acm->disconnected = true;
+ /*
+ * there is a circular dependency. acm_softint() can resubmit
+ * the URBs in error handling so we need to block any
+ * submission right away
+ */
+ acm_poison_urbs(acm);
+ mutex_lock(&acm->mutex);
if (acm->country_codes) {
device_remove_file(&acm->control->dev,
&dev_attr_wCountryCodes);
tty_kref_put(tty);
}
- acm_kill_urbs(acm);
cancel_delayed_work_sync(&acm->dwork);
tty_unregister_device(acm_tty_driver, acm->minor);
if (cnt)
return 0;
- acm_kill_urbs(acm);
+ acm_poison_urbs(acm);
cancel_delayed_work_sync(&acm->dwork);
acm->urbs_in_error_delay = 0;
struct urb *urb;
int rv = 0;
+ acm_unpoison_urbs(acm);
spin_lock_irq(&acm->write_lock);
if (--acm->susp_count)
/* DJI CineSSD */
{ USB_DEVICE(0x2ca3, 0x0031), .driver_info = USB_QUIRK_NO_LPM },
+ /* Fibocom L850-GL LTE Modem */
+ { USB_DEVICE(0x2cb7, 0x0007), .driver_info =
+ USB_QUIRK_IGNORE_REMOTE_WAKEUP },
+
/* INTEL VALUE SSD */
{ USB_DEVICE(0x8086, 0xf1a5), .driver_info = USB_QUIRK_RESET_RESUME },
if (hsotg->op_state == OTG_STATE_B_PERIPHERAL)
goto unlock;
- if (hsotg->params.power_down > DWC2_POWER_DOWN_PARAM_PARTIAL)
+ if (hsotg->params.power_down != DWC2_POWER_DOWN_PARAM_PARTIAL ||
+ hsotg->flags.b.port_connect_status == 0)
goto skip_power_saving;
/*
dwc2_writel(hsotg, hprt0, HPRT0);
/* Wait for the HPRT0.PrtSusp register field to be set */
- if (dwc2_hsotg_wait_bit_set(hsotg, HPRT0, HPRT0_SUSP, 3000))
+ if (dwc2_hsotg_wait_bit_set(hsotg, HPRT0, HPRT0_SUSP, 5000))
dev_warn(hsotg->dev, "Suspend wasn't generated\n");
/*
static const struct property_entry dwc3_pci_mrfld_properties[] = {
PROPERTY_ENTRY_STRING("dr_mode", "otg"),
PROPERTY_ENTRY_STRING("linux,extcon-name", "mrfld_bcove_pwrsrc"),
+ PROPERTY_ENTRY_BOOL("snps,dis_u3_susphy_quirk"),
+ PROPERTY_ENTRY_BOOL("snps,dis_u2_susphy_quirk"),
PROPERTY_ENTRY_BOOL("linux,sysdev_is_parent"),
{}
};
struct device *dev = qcom->dev;
int ret;
+ if (has_acpi_companion(dev))
+ return 0;
+
qcom->icc_path_ddr = of_icc_get(dev, "usb-ddr");
if (IS_ERR(qcom->icc_path_ddr)) {
dev_err(dev, "failed to get usb-ddr path: %ld\n",
trace_dwc3_gadget_ep_disable(dep);
- dwc3_remove_requests(dwc, dep);
-
/* make sure HW endpoint isn't stalled */
if (dep->flags & DWC3_EP_STALL)
__dwc3_gadget_ep_set_halt(dep, 0, false);
reg &= ~DWC3_DALEPENA_EP(dep->number);
dwc3_writel(dwc->regs, DWC3_DALEPENA, reg);
- dep->stream_capable = false;
- dep->type = 0;
- dep->flags = 0;
-
/* Clear out the ep descriptors for non-ep0 */
if (dep->number > 1) {
dep->endpoint.comp_desc = NULL;
dep->endpoint.desc = NULL;
}
+ dwc3_remove_requests(dwc, dep);
+
+ dep->stream_capable = false;
+ dep->type = 0;
+ dep->flags = 0;
+
return 0;
}
{
struct dwc3 *dwc = dep->dwc;
- if (!dep->endpoint.desc || !dwc->pullups_connected) {
+ if (!dep->endpoint.desc || !dwc->pullups_connected || !dwc->connected) {
dev_err(dwc->dev, "%s: can't queue to disabled endpoint\n",
dep->name);
return -ESHUTDOWN;
u32 reg;
speed = dwc->gadget_max_speed;
- if (speed > dwc->maximum_speed)
+ if (speed == USB_SPEED_UNKNOWN || speed > dwc->maximum_speed)
speed = dwc->maximum_speed;
if (speed == USB_SPEED_SUPER_PLUS &&
if (!is_on) {
u32 count;
+ dwc->connected = false;
/*
* In the Synopsis DesignWare Cores USB3 Databook Rev. 3.30a
* Section 4.1.8 Table 4-7, it states that for a device-initiated
dwc->ev_buf->lpos = (dwc->ev_buf->lpos + count) %
dwc->ev_buf->length;
}
- dwc->connected = false;
} else {
__dwc3_gadget_start(dwc);
}
unsigned long flags;
spin_lock_irqsave(&dwc->lock, flags);
+ dwc->gadget_max_speed = USB_SPEED_SUPER_PLUS;
dwc->gadget_ssp_rate = rate;
spin_unlock_irqrestore(&dwc->lock, flags);
}
{
u32 reg;
- dwc->connected = true;
-
/*
* WORKAROUND: DWC3 revisions <1.88a have an issue which
* would cause a missing Disconnect Event if there's a
* transfers."
*/
dwc3_stop_active_transfers(dwc);
+ dwc->connected = true;
reg = dwc3_readl(dwc->regs, DWC3_DCTL);
reg &= ~DWC3_DCTL_TSTCTRL_MASK;
struct list_head list;
};
+#define USB_MAX_STRING_WITH_NULL_LEN (USB_MAX_STRING_LEN+1)
+
static int usb_string_copy(const char *s, char **s_copy)
{
int ret;
if (ret > USB_MAX_STRING_LEN)
return -EOVERFLOW;
- str = kstrdup(s, GFP_KERNEL);
- if (!str)
- return -ENOMEM;
+ if (copy) {
+ str = copy;
+ } else {
+ str = kmalloc(USB_MAX_STRING_WITH_NULL_LEN, GFP_KERNEL);
+ if (!str)
+ return -ENOMEM;
+ }
+ strcpy(str, s);
if (str[ret - 1] == '\n')
str[ret - 1] = '\0';
- kfree(copy);
*s_copy = str;
return 0;
}
pci_set_master(pdev);
pci_try_set_mwi(pdev);
+ dev->phys_addr = resource;
+ dev->irq = pdev->irq;
+ dev->pdev = pdev;
+ dev->dev = &pdev->dev;
+
/* init dma pools */
if (use_dma) {
retval = init_dma_pools(dev);
goto err_dma;
}
- dev->phys_addr = resource;
- dev->irq = pdev->irq;
- dev->pdev = pdev;
- dev->dev = &pdev->dev;
-
/* general probing */
if (udc_probe(dev)) {
retval = -ENODEV;
xhci->quirks |= XHCI_SPURIOUS_SUCCESS;
if (mtk->lpm_support)
xhci->quirks |= XHCI_LPM_SUPPORT;
+
+ /*
+ * MTK xHCI 0.96: PSA is 1 by default even if doesn't support stream,
+ * and it's 3 when support it.
+ */
+ if (xhci->hci_version < 0x100 && HCC_MAX_PSA(xhci->hcc_params) == 4)
+ xhci->quirks |= XHCI_BROKEN_STREAMS;
}
/* called during probe() after chip reset completes */
if (ret)
goto put_usb3_hcd;
- if (HCC_MAX_PSA(xhci->hcc_params) >= 4)
+ if (HCC_MAX_PSA(xhci->hcc_params) >= 4 &&
+ !(xhci->quirks & XHCI_BROKEN_STREAMS))
xhci->shared_hcd->can_do_streams = 1;
ret = usb_add_hcd(xhci->shared_hcd, irq, IRQF_SHARED);
MODULE_AUTHOR("Michael Hund <mhund@ld-didactic.de>");
MODULE_DESCRIPTION("LD USB Driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("LD USB Devices");
/* All interrupt in transfers are collected in a ring buffer to
* avoid racing conditions and get better performance of the driver.
MUSB_DEVCTL_HR;
switch (devctl & ~s) {
case MUSB_QUIRK_B_DISCONNECT_99:
- musb_dbg(musb, "Poll devctl in case of suspend after disconnect\n");
- schedule_delayed_work(&musb->irq_work,
- msecs_to_jiffies(1000));
- break;
+ if (musb->quirk_retries && !musb->flush_irq_work) {
+ musb_dbg(musb, "Poll devctl in case of suspend after disconnect\n");
+ schedule_delayed_work(&musb->irq_work,
+ msecs_to_jiffies(1000));
+ musb->quirk_retries--;
+ break;
+ }
+ fallthrough;
case MUSB_QUIRK_B_INVALID_VBUS_91:
if (musb->quirk_retries && !musb->flush_irq_work) {
musb_dbg(musb,
need_auto_sense = 1;
}
+ /* Some devices (Kindle) require another command after SYNC CACHE */
+ if ((us->fflags & US_FL_SENSE_AFTER_SYNC) &&
+ srb->cmnd[0] == SYNCHRONIZE_CACHE) {
+ usb_stor_dbg(us, "-- sense after SYNC CACHE\n");
+ need_auto_sense = 1;
+ }
+
/*
* If we have a failure, we're going to do a REQUEST_SENSE
* automatically. Note that we differentiate between a command
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_NO_READ_DISC_INFO ),
+/*
+ * Reported by Matthias Schwarzott <zzam@gentoo.org>
+ * The Amazon Kindle treats SYNCHRONIZE CACHE as an indication that
+ * the host may be finished with it, and automatically ejects its
+ * emulated media unless it receives another command within one second.
+ */
+UNUSUAL_DEV( 0x1949, 0x0004, 0x0000, 0x9999,
+ "Amazon",
+ "Kindle",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_SENSE_AFTER_SYNC ),
+
/*
* Reported by Oliver Neukum <oneukum@suse.com>
* This device morphes spontaneously into another device if the access
port->supply_voltage = mv;
port->current_limit = max_ma;
+ power_supply_changed(port->psy);
if (port->tcpc->set_current_limit)
ret = port->tcpc->set_current_limit(port->tcpc, max_ma, mv);
port->pps_data.supported = false;
port->usb_type = POWER_SUPPLY_USB_TYPE_PD;
+ power_supply_changed(port->psy);
/*
* Select the source PDO providing the most power which has a
port->pps_data.supported = true;
port->usb_type =
POWER_SUPPLY_USB_TYPE_PD_PPS;
+ power_supply_changed(port->psy);
}
continue;
default:
port->pps_data.out_volt));
port->pps_data.op_curr = min(port->pps_data.max_curr,
port->pps_data.op_curr);
+ power_supply_changed(port->psy);
}
return src_pdo;
return ret;
}
port->vbus_charge = charge;
+ power_supply_changed(port->psy);
return 0;
}
port->try_src_count = 0;
port->try_snk_count = 0;
port->usb_type = POWER_SUPPLY_USB_TYPE_C;
+ power_supply_changed(port->psy);
port->nr_sink_caps = 0;
port->sink_cap_done = false;
if (port->tcpc->enable_frs)
goto unlock;
/* Send when the state machine is idle */
- if (port->state != SNK_READY || port->vdm_state != VDM_STATE_DONE || port->send_discover)
+ if (port->state != SNK_READY || port->vdm_sm_running || port->send_discover)
goto resched;
port->upcoming_state = GET_SINK_CAP;
ret = -EINVAL;
break;
}
-
+ power_supply_changed(port->psy);
return ret;
}
err = devm_tcpm_psy_register(port);
if (err)
goto out_role_sw_put;
+ power_supply_changed(port->psy);
port->typec_port = typec_register_port(port->dev, &port->typec_caps);
if (IS_ERR(port->typec_port)) {
struct tps6598x_rx_identity_reg {
u8 status;
struct usb_pd_identity identity;
- u32 vdo[3];
} __packed;
/* Standard Task return codes */
dev_info(dev, "stub up\n");
+ mutex_lock(&sdev->ud.sysfs_lock);
spin_lock_irq(&sdev->ud.lock);
if (sdev->ud.status != SDEV_ST_AVAILABLE) {
tcp_rx = kthread_create(stub_rx_loop, &sdev->ud, "stub_rx");
if (IS_ERR(tcp_rx)) {
sockfd_put(socket);
- return -EINVAL;
+ goto unlock_mutex;
}
tcp_tx = kthread_create(stub_tx_loop, &sdev->ud, "stub_tx");
if (IS_ERR(tcp_tx)) {
kthread_stop(tcp_rx);
sockfd_put(socket);
- return -EINVAL;
+ goto unlock_mutex;
}
/* get task structs now */
wake_up_process(sdev->ud.tcp_rx);
wake_up_process(sdev->ud.tcp_tx);
+ mutex_unlock(&sdev->ud.sysfs_lock);
+
} else {
dev_info(dev, "stub down\n");
spin_unlock_irq(&sdev->ud.lock);
usbip_event_add(&sdev->ud, SDEV_EVENT_DOWN);
+ mutex_unlock(&sdev->ud.sysfs_lock);
}
return count;
sockfd_put(socket);
err:
spin_unlock_irq(&sdev->ud.lock);
+unlock_mutex:
+ mutex_unlock(&sdev->ud.sysfs_lock);
return -EINVAL;
}
static DEVICE_ATTR_WO(usbip_sockfd);
sdev->ud.side = USBIP_STUB;
sdev->ud.status = SDEV_ST_AVAILABLE;
spin_lock_init(&sdev->ud.lock);
+ mutex_init(&sdev->ud.sysfs_lock);
sdev->ud.tcp_socket = NULL;
sdev->ud.sockfd = -1;
/* lock for status */
spinlock_t lock;
+ /* mutex for synchronizing sysfs store paths */
+ struct mutex sysfs_lock;
+
int sockfd;
struct socket *tcp_socket;
while ((ud = get_event()) != NULL) {
usbip_dbg_eh("pending event %lx\n", ud->event);
+ mutex_lock(&ud->sysfs_lock);
/*
* NOTE: shutdown must come first.
* Shutdown the device.
ud->eh_ops.unusable(ud);
unset_event(ud, USBIP_EH_UNUSABLE);
}
+ mutex_unlock(&ud->sysfs_lock);
wake_up(&ud->eh_waitq);
}
pr_err("invalid port number %d\n", wIndex);
goto error;
}
+ if (wValue >= 32)
+ goto error;
if (hcd->speed == HCD_USB3) {
if ((vhci_hcd->port_status[rhport] &
USB_SS_PORT_STAT_POWER) != 0) {
vdev->ud.side = USBIP_VHCI;
vdev->ud.status = VDEV_ST_NULL;
spin_lock_init(&vdev->ud.lock);
+ mutex_init(&vdev->ud.sysfs_lock);
INIT_LIST_HEAD(&vdev->priv_rx);
INIT_LIST_HEAD(&vdev->priv_tx);
usbip_dbg_vhci_sysfs("enter\n");
+ mutex_lock(&vdev->ud.sysfs_lock);
+
/* lock */
spin_lock_irqsave(&vhci->lock, flags);
spin_lock(&vdev->ud.lock);
/* unlock */
spin_unlock(&vdev->ud.lock);
spin_unlock_irqrestore(&vhci->lock, flags);
+ mutex_unlock(&vdev->ud.sysfs_lock);
return -EINVAL;
}
usbip_event_add(&vdev->ud, VDEV_EVENT_DOWN);
+ mutex_unlock(&vdev->ud.sysfs_lock);
+
return 0;
}
else
vdev = &vhci->vhci_hcd_hs->vdev[rhport];
+ mutex_lock(&vdev->ud.sysfs_lock);
+
/* Extract socket from fd. */
socket = sockfd_lookup(sockfd, &err);
if (!socket) {
dev_err(dev, "failed to lookup sock");
- return -EINVAL;
+ err = -EINVAL;
+ goto unlock_mutex;
}
if (socket->type != SOCK_STREAM) {
dev_err(dev, "Expecting SOCK_STREAM - found %d",
socket->type);
sockfd_put(socket);
- return -EINVAL;
+ err = -EINVAL;
+ goto unlock_mutex;
}
/* create threads before locking */
tcp_rx = kthread_create(vhci_rx_loop, &vdev->ud, "vhci_rx");
if (IS_ERR(tcp_rx)) {
sockfd_put(socket);
- return -EINVAL;
+ err = -EINVAL;
+ goto unlock_mutex;
}
tcp_tx = kthread_create(vhci_tx_loop, &vdev->ud, "vhci_tx");
if (IS_ERR(tcp_tx)) {
kthread_stop(tcp_rx);
sockfd_put(socket);
- return -EINVAL;
+ err = -EINVAL;
+ goto unlock_mutex;
}
/* get task structs now */
* Will be retried from userspace
* if there's another free port.
*/
- return -EBUSY;
+ err = -EBUSY;
+ goto unlock_mutex;
}
dev_info(dev, "pdev(%u) rhport(%u) sockfd(%d)\n",
rh_port_connect(vdev, speed);
+ dev_info(dev, "Device attached\n");
+
+ mutex_unlock(&vdev->ud.sysfs_lock);
+
return count;
+
+unlock_mutex:
+ mutex_unlock(&vdev->ud.sysfs_lock);
+ return err;
}
static DEVICE_ATTR_WO(attach);
init_waitqueue_head(&udc->tx_waitq);
spin_lock_init(&ud->lock);
+ mutex_init(&ud->sysfs_lock);
ud->status = SDEV_ST_AVAILABLE;
ud->side = USBIP_VUDC;
dev_err(dev, "no device");
return -ENODEV;
}
+ mutex_lock(&udc->ud.sysfs_lock);
spin_lock_irqsave(&udc->lock, flags);
/* Don't export what we don't have */
if (!udc->driver || !udc->pullup) {
udc->ud.tcp_socket = socket;
udc->ud.tcp_rx = tcp_rx;
- udc->ud.tcp_rx = tcp_tx;
+ udc->ud.tcp_tx = tcp_tx;
udc->ud.status = SDEV_ST_USED;
spin_unlock_irq(&udc->ud.lock);
wake_up_process(udc->ud.tcp_rx);
wake_up_process(udc->ud.tcp_tx);
+
+ mutex_unlock(&udc->ud.sysfs_lock);
return count;
} else {
}
spin_unlock_irqrestore(&udc->lock, flags);
+ mutex_unlock(&udc->ud.sysfs_lock);
return count;
spin_unlock_irq(&udc->ud.lock);
unlock:
spin_unlock_irqrestore(&udc->lock, flags);
+ mutex_unlock(&udc->ud.sysfs_lock);
return ret;
}
}
adapter = vdpa_alloc_device(struct ifcvf_adapter, vdpa,
- dev, &ifc_vdpa_ops,
- IFCVF_MAX_QUEUE_PAIRS * 2, NULL);
+ dev, &ifc_vdpa_ops, NULL);
if (adapter == NULL) {
IFCVF_ERR(pdev, "Failed to allocate vDPA structure");
return -ENOMEM;
for (i = 0; i < IFCVF_MAX_QUEUE_PAIRS * 2; i++)
vf->vring[i].irq = -EINVAL;
- ret = vdpa_register_device(&adapter->vdpa);
+ ret = vdpa_register_device(&adapter->vdpa, IFCVF_MAX_QUEUE_PAIRS * 2);
if (ret) {
IFCVF_ERR(pdev, "Failed to register ifcvf to vdpa bus");
goto err;
#ifndef __MLX5_VDPA_H__
#define __MLX5_VDPA_H__
+#include <linux/etherdevice.h>
+#include <linux/if_vlan.h>
#include <linux/vdpa.h>
#include <linux/mlx5/driver.h>
+#define MLX5V_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN + ETH_FCS_LEN)
+
struct mlx5_vdpa_direct_mr {
u64 start;
u64 end;
mlx5_vdpa_destroy_mkey(mvdev, &mkey->mkey);
}
+static struct device *get_dma_device(struct mlx5_vdpa_dev *mvdev)
+{
+ return &mvdev->mdev->pdev->dev;
+}
+
static int map_direct_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_direct_mr *mr,
struct vhost_iotlb *iotlb)
{
u64 pa;
u64 paend;
struct scatterlist *sg;
- struct device *dma = mvdev->mdev->device;
+ struct device *dma = get_dma_device(mvdev);
for (map = vhost_iotlb_itree_first(iotlb, mr->start, mr->end - 1);
map; map = vhost_iotlb_itree_next(map, start, mr->end - 1)) {
static void unmap_direct_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_direct_mr *mr)
{
- struct device *dma = mvdev->mdev->device;
+ struct device *dma = get_dma_device(mvdev);
destroy_direct_mr(mvdev, mr);
dma_unmap_sg_attrs(dma, mr->sg_head.sgl, mr->nsg, DMA_BIDIRECTIONAL, 0);
if (err)
goto err_key;
- kick_addr = pci_resource_start(mdev->pdev, 0) + offset;
+ kick_addr = mdev->bar_addr + offset;
+
res->kick_addr = ioremap(kick_addr, PAGE_SIZE);
if (!res->kick_addr) {
err = -ENOMEM;
MLX5_SET(virtio_q, vq_ctx, event_qpn_or_msix, mvq->fwqp.mqp.qpn);
MLX5_SET(virtio_q, vq_ctx, queue_size, mvq->num_ent);
MLX5_SET(virtio_q, vq_ctx, virtio_version_1_0,
- !!(ndev->mvdev.actual_features & VIRTIO_F_VERSION_1));
+ !!(ndev->mvdev.actual_features & BIT_ULL(VIRTIO_F_VERSION_1)));
MLX5_SET64(virtio_q, vq_ctx, desc_addr, mvq->desc_addr);
MLX5_SET64(virtio_q, vq_ctx, used_addr, mvq->device_addr);
MLX5_SET64(virtio_q, vq_ctx, available_addr, mvq->driver_addr);
return;
}
mvq->avail_idx = attr.available_index;
+ mvq->used_idx = attr.used_index;
}
static void suspend_vqs(struct mlx5_vdpa_net *ndev)
return -EINVAL;
}
+ mvq->used_idx = state->avail_index;
mvq->avail_idx = state->avail_index;
return 0;
}
* that cares about emulating the index after vq is stopped.
*/
if (!mvq->initialized) {
- state->avail_index = mvq->avail_idx;
+ /* Firmware returns a wrong value for the available index.
+ * Since both values should be identical, we take the value of
+ * used_idx which is reported correctly.
+ */
+ state->avail_index = mvq->used_idx;
return 0;
}
mlx5_vdpa_warn(mvdev, "failed to query virtqueue\n");
return err;
}
- state->avail_index = attr.available_index;
+ state->avail_index = attr.used_index;
return 0;
}
}
}
-static void clear_virtqueues(struct mlx5_vdpa_net *ndev)
-{
- int i;
-
- for (i = ndev->mvdev.max_vqs - 1; i >= 0; i--) {
- ndev->vqs[i].avail_idx = 0;
- ndev->vqs[i].used_idx = 0;
- }
-}
-
/* TODO: cross-endian support */
static inline bool mlx5_vdpa_is_little_endian(struct mlx5_vdpa_dev *mvdev)
{
return virtio_legacy_is_little_endian() ||
- (mvdev->actual_features & (1ULL << VIRTIO_F_VERSION_1));
+ (mvdev->actual_features & BIT_ULL(VIRTIO_F_VERSION_1));
}
static __virtio16 cpu_to_mlx5vdpa16(struct mlx5_vdpa_dev *mvdev, u16 val)
if (!status) {
mlx5_vdpa_info(mvdev, "performing device reset\n");
teardown_driver(ndev);
- clear_virtqueues(ndev);
mlx5_vdpa_destroy_mr(&ndev->mvdev);
ndev->mvdev.status = 0;
ndev->mvdev.mlx_features = 0;
.free = mlx5_vdpa_free,
};
+static int query_mtu(struct mlx5_core_dev *mdev, u16 *mtu)
+{
+ u16 hw_mtu;
+ int err;
+
+ err = mlx5_query_nic_vport_mtu(mdev, &hw_mtu);
+ if (err)
+ return err;
+
+ *mtu = hw_mtu - MLX5V_ETH_HARD_MTU;
+ return 0;
+}
+
static int alloc_resources(struct mlx5_vdpa_net *ndev)
{
struct mlx5_vdpa_net_resources *res = &ndev->res;
max_vqs = min_t(u32, max_vqs, MLX5_MAX_SUPPORTED_VQS);
ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev, mdev->device, &mlx5_vdpa_ops,
- 2 * mlx5_vdpa_max_qps(max_vqs), NULL);
+ NULL);
if (IS_ERR(ndev))
return PTR_ERR(ndev);
init_mvqs(ndev);
mutex_init(&ndev->reslock);
config = &ndev->config;
- err = mlx5_query_nic_vport_mtu(mdev, &ndev->mtu);
+ err = query_mtu(mdev, &ndev->mtu);
if (err)
goto err_mtu;
if (err)
goto err_res;
- err = vdpa_register_device(&mvdev->vdev);
+ err = vdpa_register_device(&mvdev->vdev, 2 * mlx5_vdpa_max_qps(max_vqs));
if (err)
goto err_reg;
* initialized but before registered.
* @parent: the parent device
* @config: the bus operations that is supported by this device
- * @nvqs: number of virtqueues supported by this device
* @size: size of the parent structure that contains private data
* @name: name of the vdpa device; optional.
*
*/
struct vdpa_device *__vdpa_alloc_device(struct device *parent,
const struct vdpa_config_ops *config,
- int nvqs, size_t size, const char *name)
+ size_t size, const char *name)
{
struct vdpa_device *vdev;
int err = -EINVAL;
vdev->index = err;
vdev->config = config;
vdev->features_valid = false;
- vdev->nvqs = nvqs;
if (name)
err = dev_set_name(&vdev->dev, "%s", name);
return (strcmp(dev_name(&vdev->dev), data) == 0);
}
-static int __vdpa_register_device(struct vdpa_device *vdev)
+static int __vdpa_register_device(struct vdpa_device *vdev, int nvqs)
{
struct device *dev;
+ vdev->nvqs = nvqs;
+
lockdep_assert_held(&vdpa_dev_mutex);
dev = bus_find_device(&vdpa_bus, NULL, dev_name(&vdev->dev), vdpa_name_match);
if (dev) {
* Caller must invoke this routine in the management device dev_add()
* callback after setting up valid mgmtdev for this vdpa device.
* @vdev: the vdpa device to be registered to vDPA bus
+ * @nvqs: number of virtqueues supported by this device
*
* Returns an error when fail to add device to vDPA bus
*/
-int _vdpa_register_device(struct vdpa_device *vdev)
+int _vdpa_register_device(struct vdpa_device *vdev, int nvqs)
{
if (!vdev->mdev)
return -EINVAL;
- return __vdpa_register_device(vdev);
+ return __vdpa_register_device(vdev, nvqs);
}
EXPORT_SYMBOL_GPL(_vdpa_register_device);
* vdpa_register_device - register a vDPA device
* Callers must have a succeed call of vdpa_alloc_device() before.
* @vdev: the vdpa device to be registered to vDPA bus
+ * @nvqs: number of virtqueues supported by this device
*
* Returns an error when fail to add to vDPA bus
*/
-int vdpa_register_device(struct vdpa_device *vdev)
+int vdpa_register_device(struct vdpa_device *vdev, int nvqs)
{
int err;
mutex_lock(&vdpa_dev_mutex);
- err = __vdpa_register_device(vdev);
+ err = __vdpa_register_device(vdev, nvqs);
mutex_unlock(&vdpa_dev_mutex);
return err;
}
ops = &vdpasim_config_ops;
vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops,
- dev_attr->nvqs, dev_attr->name);
+ dev_attr->name);
if (!vdpasim)
goto err_alloc;
static void vdpasim_net_get_config(struct vdpasim *vdpasim, void *config)
{
- struct virtio_net_config *net_config =
- (struct virtio_net_config *)config;
+ struct virtio_net_config *net_config = config;
net_config->mtu = cpu_to_vdpasim16(vdpasim, 1500);
net_config->status = cpu_to_vdpasim16(vdpasim, VIRTIO_NET_S_LINK_UP);
if (IS_ERR(simdev))
return PTR_ERR(simdev);
- ret = _vdpa_register_device(&simdev->vdpa);
+ ret = _vdpa_register_device(&simdev->vdpa, VDPASIM_NET_VQ_NUM);
if (ret)
goto reg_err;
menuconfig VFIO
tristate "VFIO Non-Privileged userspace driver framework"
- depends on IOMMU_API
- select VFIO_IOMMU_TYPE1 if (X86 || S390 || ARM || ARM64)
+ select IOMMU_API
+ select VFIO_IOMMU_TYPE1 if MMU && (X86 || S390 || ARM || ARM64)
help
VFIO provides a framework for secure userspace device drivers.
See Documentation/driver-api/vfio.rst for more details.
config VFIO_PCI_NVLINK2
def_bool y
- depends on VFIO_PCI && PPC_POWERNV
+ depends on VFIO_PCI && PPC_POWERNV && SPAPR_TCE_IOMMU
help
VFIO PCI support for P9 Witherspoon machine with NVIDIA V100 GPUs
# SPDX-License-Identifier: GPL-2.0-only
config VFIO_PLATFORM
tristate "VFIO support for platform devices"
- depends on VFIO && EVENTFD && (ARM || ARM64)
+ depends on VFIO && EVENTFD && (ARM || ARM64 || COMPILE_TEST)
select VFIO_VIRQFD
help
Support for platform devices with VFIO. This is required to make
config VFIO_AMBA
tristate "VFIO support for AMBA devices"
- depends on VFIO_PLATFORM && ARM_AMBA
+ depends on VFIO_PLATFORM && (ARM_AMBA || COMPILE_TEST)
help
Support for ARM AMBA devices with VFIO. This is required to make
use of ARM AMBA devices present on the system using the VFIO
}
static struct rb_node *vfio_find_dma_first_node(struct vfio_iommu *iommu,
- dma_addr_t start, size_t size)
+ dma_addr_t start, u64 size)
{
struct rb_node *res = NULL;
struct rb_node *node = iommu->dma_list.rb_node;
ret = vfio_lock_acct(dma, lock_acct, false);
unpin_out:
+ if (batch->size == 1 && !batch->offset) {
+ /* May be a VM_PFNMAP pfn, which the batch can't remember. */
+ put_pfn(pfn, dma->prot);
+ batch->size = 0;
+ }
+
if (ret < 0) {
if (pinned && !rsvd) {
for (pfn = *pfn_base ; pinned ; pfn++, pinned--)
return -ENODEV;
ret = vaddr_get_pfns(mm, vaddr, 1, dma->prot, pfn_base, pages);
- if (ret == 1 && do_accounting && !is_invalid_reserved_pfn(*pfn_base)) {
+ if (ret != 1)
+ goto out;
+
+ ret = 0;
+
+ if (do_accounting && !is_invalid_reserved_pfn(*pfn_base)) {
ret = vfio_lock_acct(dma, 1, true);
if (ret) {
put_pfn(*pfn_base, dma->prot);
}
}
+out:
mmput(mm);
return ret;
}
int ret = -EINVAL, retries = 0;
unsigned long pgshift;
dma_addr_t iova = unmap->iova;
- unsigned long size = unmap->size;
+ u64 size = unmap->size;
bool unmap_all = unmap->flags & VFIO_DMA_UNMAP_FLAG_ALL;
bool invalidate_vaddr = unmap->flags & VFIO_DMA_UNMAP_FLAG_VADDR;
struct rb_node *n, *first_n;
if (unmap_all) {
if (iova || size)
goto unlock;
- size = SIZE_MAX;
- } else if (!size || size & (pgsize - 1)) {
+ size = U64_MAX;
+ } else if (!size || size & (pgsize - 1) ||
+ iova + size - 1 < iova || size > SIZE_MAX) {
goto unlock;
}
- if (iova + size - 1 < iova || size > SIZE_MAX)
- goto unlock;
-
/* When dirty tracking is enabled, allow only min supported pgsize */
if ((unmap->flags & VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP) &&
(!iommu->dirty_page_tracking || (bitmap->pgsize != pgsize))) {
static void vhost_vdpa_config_put(struct vhost_vdpa *v)
{
- if (v->config_ctx)
+ if (v->config_ctx) {
eventfd_ctx_put(v->config_ctx);
+ v->config_ctx = NULL;
+ }
}
static long vhost_vdpa_set_config_call(struct vhost_vdpa *v, u32 __user *argp)
if (!IS_ERR_OR_NULL(ctx))
eventfd_ctx_put(ctx);
- if (IS_ERR(v->config_ctx))
- return PTR_ERR(v->config_ctx);
+ if (IS_ERR(v->config_ctx)) {
+ long ret = PTR_ERR(v->config_ctx);
+
+ v->config_ctx = NULL;
+ return ret;
+ }
v->vdpa->config->set_config_cb(v->vdpa, &cb);
static void vhost_vdpa_clean_irq(struct vhost_vdpa *v)
{
- struct vhost_virtqueue *vq;
int i;
- for (i = 0; i < v->nvqs; i++) {
- vq = &v->vqs[i];
- if (vq->call_ctx.producer.irq)
- irq_bypass_unregister_producer(&vq->call_ctx.producer);
- }
+ for (i = 0; i < v->nvqs; i++)
+ vhost_vdpa_unsetup_vq_irq(v, i);
}
static int vhost_vdpa_release(struct inode *inode, struct file *filep)
vq->error_ctx = NULL;
vq->kick = NULL;
vq->log_ctx = NULL;
- vhost_reset_is_le(vq);
vhost_disable_cross_endian(vq);
+ vhost_reset_is_le(vq);
vq->busyloop_timeout = 0;
vq->umem = NULL;
vq->iotlb = NULL;
ops->cursor_flash = (mode == CM_ERASE) ? 0 : 1;
+ if (!ops->cursor)
+ return;
+
ops->cursor(vc, info, mode, get_color(vc, info, c, 1),
get_color(vc, info, c, 0));
}
PCI_DEVICE_ID_HYPERV_VIDEO, NULL);
if (!pdev) {
pr_err("Unable to find PCI Hyper-V video\n");
- kfree(info->apertures);
return -ENODEV;
}
} else {
pci_dev_put(pdev);
}
- kfree(info->apertures);
return 0;
err1:
if (!gen2vm)
pci_dev_put(pdev);
- kfree(info->apertures);
return -ENOMEM;
}
}
EXPORT_SYMBOL_GPL(virtio_config_changed);
-void virtio_config_disable(struct virtio_device *dev)
+static void virtio_config_disable(struct virtio_device *dev)
{
spin_lock_irq(&dev->config_lock);
dev->config_enabled = false;
spin_unlock_irq(&dev->config_lock);
}
-EXPORT_SYMBOL_GPL(virtio_config_disable);
-void virtio_config_enable(struct virtio_device *dev)
+static void virtio_config_enable(struct virtio_device *dev)
{
spin_lock_irq(&dev->config_lock);
dev->config_enabled = true;
dev->config_change_pending = false;
spin_unlock_irq(&dev->config_lock);
}
-EXPORT_SYMBOL_GPL(virtio_config_enable);
void virtio_add_status(struct virtio_device *dev, unsigned int status)
{
{
struct virtio_device *vdev =
container_of(_d, struct virtio_device, dev);
- struct virtio_mmio_device *vm_dev =
- container_of(vdev, struct virtio_mmio_device, vdev);
+ struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
struct platform_device *pdev = vm_dev->pdev;
devm_kfree(&pdev->dev, vm_dev);
/*
* Watchdog driver for Marvell Armada 37xx SoCs
*
- * Author: Marek Behun <marek.behun@nic.cz>
+ * Author: Marek Behún <kabel@kernel.org>
*/
#include <linux/clk.h>
module_platform_driver(armada_37xx_wdt_driver);
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
MODULE_DESCRIPTION("Armada 37xx CPU Watchdog");
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("Heiko Ronsdorf <hero@ihg.uni-duisburg.de>");
MODULE_DESCRIPTION("sma cpu5 watchdog driver");
-MODULE_SUPPORTED_DEVICE("sma cpu5 watchdog");
MODULE_LICENSE("GPL");
module_param_hw(port, int, ioport, 0);
MODULE_AUTHOR("Eric Brower <ebrower@usa.net>");
MODULE_DESCRIPTION("Hardware watchdog driver for Sun Microsystems CP1400/1500");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("watchdog");
static void cpwd_writew(u16 val, void __iomem *addr)
{
MODULE_AUTHOR("David S. Miller <davem@davemloft.net>");
MODULE_DESCRIPTION("Hardware watchdog driver for Sun RIO");
-MODULE_SUPPORTED_DEVICE("watchdog");
MODULE_LICENSE("GPL");
#define DRIVER_NAME "riowd"
SUBSYSTEM=="memory", ACTION=="add", RUN+="/bin/sh -c '[ -f /sys$devpath/state ] && echo online > /sys$devpath/state'"
-config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
+config XEN_MEMORY_HOTPLUG_LIMIT
int "Hotplugged memory limit (in GiB) for a PV guest"
default 512
depends on XEN_HAVE_PVMMU
- depends on XEN_BALLOON_MEMORY_HOTPLUG
+ depends on MEMORY_HOTPLUG
help
Maxmium amount of memory (in GiB) that a PV guest can be
expanded to when using memory hotplug.
unsigned short eoi_cpu; /* EOI must happen on this cpu-1 */
unsigned int irq_epoch; /* If eoi_cpu valid: irq_epoch of event */
u64 eoi_time; /* Time in jiffies when to EOI. */
- spinlock_t lock;
+ raw_spinlock_t lock;
union {
unsigned short virq;
info->evtchn = evtchn;
info->cpu = cpu;
info->mask_reason = EVT_MASK_REASON_EXPLICIT;
- spin_lock_init(&info->lock);
+ raw_spin_lock_init(&info->lock);
ret = set_evtchn_to_irq(evtchn, irq);
if (ret < 0)
{
unsigned long flags;
- spin_lock_irqsave(&info->lock, flags);
+ raw_spin_lock_irqsave(&info->lock, flags);
if (!info->mask_reason)
mask_evtchn(info->evtchn);
info->mask_reason |= reason;
- spin_unlock_irqrestore(&info->lock, flags);
+ raw_spin_unlock_irqrestore(&info->lock, flags);
}
static void do_unmask(struct irq_info *info, u8 reason)
{
unsigned long flags;
- spin_lock_irqsave(&info->lock, flags);
+ raw_spin_lock_irqsave(&info->lock, flags);
info->mask_reason &= ~reason;
if (!info->mask_reason)
unmask_evtchn(info->evtchn);
- spin_unlock_irqrestore(&info->lock, flags);
+ raw_spin_unlock_irqrestore(&info->lock, flags);
}
#ifdef CONFIG_X86
.permission = afs_permission,
.getattr = afs_getattr,
.setattr = afs_setattr,
- .listxattr = afs_listxattr,
};
const struct address_space_operations afs_dir_aops = {
.getattr = afs_getattr,
.setattr = afs_setattr,
.permission = afs_permission,
- .listxattr = afs_listxattr,
};
const struct address_space_operations afs_fs_aops = {
if (test_bit(AFS_SERVER_FL_IS_YFS, &op->server->flags) &&
op->ops->issue_yfs_rpc)
op->ops->issue_yfs_rpc(op);
- else
+ else if (op->ops->issue_afs_rpc)
op->ops->issue_afs_rpc(op);
+ else
+ op->ac.error = -ENOTSUPP;
- op->error = afs_wait_for_call_to_complete(op->call, &op->ac);
+ if (op->call)
+ op->error = afs_wait_for_call_to_complete(op->call, &op->ac);
}
switch (op->error) {
static const struct inode_operations afs_symlink_inode_operations = {
.get_link = page_get_link,
- .listxattr = afs_listxattr,
};
static noinline void dump_vnode(struct afs_vnode *vnode, struct afs_vnode *parent_vnode)
* xattr.c
*/
extern const struct xattr_handler *afs_xattr_handlers[];
-extern ssize_t afs_listxattr(struct dentry *, char *, size_t);
/*
* yfsclient.c
.lookup = afs_mntpt_lookup,
.readlink = page_readlink,
.getattr = afs_getattr,
- .listxattr = afs_listxattr,
};
const struct inode_operations afs_autocell_inode_operations = {
fscache_wait_on_page_write(vnode->cache, vmf->page);
#endif
- if (PageWriteback(vmf->page) &&
- wait_on_page_bit_killable(vmf->page, PG_writeback) < 0)
+ if (wait_on_page_writeback_killable(vmf->page))
return VM_FAULT_RETRY;
if (lock_page_killable(vmf->page) < 0)
#include <linux/xattr.h>
#include "internal.h"
-static const char afs_xattr_list[] =
- "afs.acl\0"
- "afs.cell\0"
- "afs.fid\0"
- "afs.volume\0"
- "afs.yfs.acl\0"
- "afs.yfs.acl_inherited\0"
- "afs.yfs.acl_num_cleaned\0"
- "afs.yfs.vol_acl";
-
-/*
- * Retrieve a list of the supported xattrs.
- */
-ssize_t afs_listxattr(struct dentry *dentry, char *buffer, size_t size)
-{
- if (size == 0)
- return sizeof(afs_xattr_list);
- if (size < sizeof(afs_xattr_list))
- return -ERANGE;
- memcpy(buffer, afs_xattr_list, sizeof(afs_xattr_list));
- return sizeof(afs_xattr_list);
-}
-
/*
* Deal with the result of a successful fetch ACL operation.
*/
else
ret = -ERANGE;
}
+ } else if (ret == -ENOTSUPP) {
+ ret = -ENODATA;
}
error_yacl:
{
struct afs_operation *op;
struct afs_vnode *vnode = AFS_FS_I(inode);
+ int ret;
if (flags == XATTR_CREATE ||
strcmp(name, "acl") != 0)
return afs_put_operation(op);
op->ops = &yfs_store_opaque_acl2_operation;
- return afs_do_sync_operation(op);
+ ret = afs_do_sync_operation(op);
+ if (ret == -ENOTSUPP)
+ ret = -ENODATA;
+ return ret;
}
static const struct xattr_handler afs_xattr_yfs_handler = {
bio.bi_opf = dio_bio_write_op(iocb);
task_io_account_write(ret);
}
+ if (iocb->ki_flags & IOCB_NOWAIT)
+ bio.bi_opf |= REQ_NOWAIT;
if (iocb->ki_flags & IOCB_HIPRI)
bio_set_polled(&bio, iocb);
bio->bi_opf = dio_bio_write_op(iocb);
task_io_account_write(bio->bi_iter.bi_size);
}
+ if (iocb->ki_flags & IOCB_NOWAIT)
+ bio->bi_opf |= REQ_NOWAIT;
dio->size += bio->bi_iter.bi_size;
pos += bio->bi_iter.bi_size;
lockdep_assert_held(&bdev->bd_mutex);
- clear_bit(GD_NEED_PART_SCAN, &bdev->bd_disk->state);
-
rescan:
ret = blk_drop_partitions(bdev);
if (ret)
return ret;
+ clear_bit(GD_NEED_PART_SCAN, &disk->state);
+
/*
* Historically we only set the capacity to zero for devices that
* support partitions (independ of actually having partitions created).
subdir-ccflags-y += -Wmissing-prototypes
subdir-ccflags-y += -Wold-style-definition
subdir-ccflags-y += -Wmissing-include-dirs
-subdir-ccflags-y += $(call cc-option, -Wunused-but-set-variable)
-subdir-ccflags-y += $(call cc-option, -Wunused-const-variable)
-subdir-ccflags-y += $(call cc-option, -Wpacked-not-aligned)
-subdir-ccflags-y += $(call cc-option, -Wstringop-truncation)
+condflags := \
+ $(call cc-option, -Wunused-but-set-variable) \
+ $(call cc-option, -Wunused-const-variable) \
+ $(call cc-option, -Wpacked-not-aligned) \
+ $(call cc-option, -Wstringop-truncation)
+subdir-ccflags-y += $(condflags)
# The following turn off the warnings enabled by -Wextra
subdir-ccflags-y += -Wno-missing-field-initializers
subdir-ccflags-y += -Wno-sign-compare
"failed to read tree block %llu from get_old_root",
logical);
} else {
+ btrfs_tree_read_lock(old);
eb = btrfs_clone_extent_buffer(old);
+ btrfs_tree_read_unlock(old);
free_extent_buffer(old);
}
} else if (old_root) {
struct btrfs_dev_replace_item *ptr;
u64 src_devid;
+ if (!dev_root)
+ return 0;
+
path = btrfs_alloc_path();
if (!path) {
ret = -ENOMEM;
} else {
set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
fs_info->dev_root = root;
- btrfs_init_devices_late(fs_info);
}
+ /* Initialize fs_info for all devices in any case */
+ btrfs_init_devices_late(fs_info);
/* If IGNOREDATACSUMS is set don't bother reading the csum root. */
if (!btrfs_test_opt(fs_info, IGNOREDATACSUMS)) {
}
}
+ /*
+ * btrfs_find_orphan_roots() is responsible for finding all the dead
+ * roots (with 0 refs), flag them with BTRFS_ROOT_DEAD_TREE and load
+ * them into the fs_info->fs_roots_radix tree. This must be done before
+ * calling btrfs_orphan_cleanup() on the tree root. If we don't do it
+ * first, then btrfs_orphan_cleanup() will delete a dead root's orphan
+ * item before the root's tree is deleted - this means that if we unmount
+ * or crash before the deletion completes, on the next mount we will not
+ * delete what remains of the tree because the orphan item does not
+ * exists anymore, which is what tells us we have a pending deletion.
+ */
+ ret = btrfs_find_orphan_roots(fs_info);
+ if (ret)
+ goto out;
+
ret = btrfs_cleanup_fs_roots(fs_info);
if (ret)
goto out;
}
}
- ret = btrfs_find_orphan_roots(fs_info);
out:
return ret;
}
if (last_ref && btrfs_header_generation(buf) == trans->transid) {
struct btrfs_block_group *cache;
+ bool must_pin = false;
if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
ret = check_ref_cleanup(trans, buf->start);
goto out;
}
- if (btrfs_is_zoned(fs_info)) {
+ /*
+ * If this is a leaf and there are tree mod log users, we may
+ * have recorded mod log operations that point to this leaf.
+ * So we must make sure no one reuses this leaf's extent before
+ * mod log operations are applied to a node, otherwise after
+ * rewinding a node using the mod log operations we get an
+ * inconsistent btree, as the leaf's extent may now be used as
+ * a node or leaf for another different btree.
+ * We are safe from races here because at this point no other
+ * node or root points to this extent buffer, so if after this
+ * check a new tree mod log user joins, it will not be able to
+ * find a node pointing to this leaf and record operations that
+ * point to this leaf.
+ */
+ if (btrfs_header_level(buf) == 0) {
+ read_lock(&fs_info->tree_mod_log_lock);
+ must_pin = !list_empty(&fs_info->tree_mod_seq_list);
+ read_unlock(&fs_info->tree_mod_log_lock);
+ }
+
+ if (must_pin || btrfs_is_zoned(fs_info)) {
btrfs_redirty_list_add(trans->transaction, buf);
pin_down_extent(trans, cache, buf->start, buf->len, 1);
btrfs_put_block_group(cache);
btrfs_subpage_end_reader(fs_info, page, start, len);
}
+/*
+ * Find extent buffer for a givne bytenr.
+ *
+ * This is for end_bio_extent_readpage(), thus we can't do any unsafe locking
+ * in endio context.
+ */
+static struct extent_buffer *find_extent_buffer_readpage(
+ struct btrfs_fs_info *fs_info, struct page *page, u64 bytenr)
+{
+ struct extent_buffer *eb;
+
+ /*
+ * For regular sectorsize, we can use page->private to grab extent
+ * buffer
+ */
+ if (fs_info->sectorsize == PAGE_SIZE) {
+ ASSERT(PagePrivate(page) && page->private);
+ return (struct extent_buffer *)page->private;
+ }
+
+ /* For subpage case, we need to lookup buffer radix tree */
+ rcu_read_lock();
+ eb = radix_tree_lookup(&fs_info->buffer_radix,
+ bytenr >> fs_info->sectorsize_bits);
+ rcu_read_unlock();
+ ASSERT(eb);
+ return eb;
+}
+
/*
* after a readpage IO is done, we need to:
* clear the uptodate bits on error
} else {
struct extent_buffer *eb;
- eb = (struct extent_buffer *)page->private;
+ eb = find_extent_buffer_readpage(fs_info, page, start);
set_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags);
eb->read_mirror = mirror;
atomic_dec(&eb->io_pages);
*/
if (page->index == end_index && i_size <= end) {
u32 zero_start = max(offset_in_page(i_size),
- offset_in_page(end));
+ offset_in_page(start));
zero_user_segment(page, zero_start,
offset_in_page(end) + 1);
* @bio_offset: offset to the beginning of the bio (in bytes)
* @page: page where is the data to be verified
* @pgoff: offset inside the page
+ * @start: logical offset in the file
*
* The length of such check is always one sector size.
*/
static int check_data_csum(struct inode *inode, struct btrfs_io_bio *io_bio,
- u32 bio_offset, struct page *page, u32 pgoff)
+ u32 bio_offset, struct page *page, u32 pgoff,
+ u64 start)
{
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
kunmap_atomic(kaddr);
return 0;
zeroit:
- btrfs_print_data_csum_error(BTRFS_I(inode), page_offset(page) + pgoff,
- csum, csum_expected, io_bio->mirror_num);
+ btrfs_print_data_csum_error(BTRFS_I(inode), start, csum, csum_expected,
+ io_bio->mirror_num);
if (io_bio->device)
btrfs_dev_stat_inc_and_print(io_bio->device,
BTRFS_DEV_STAT_CORRUPTION_ERRS);
pg_off += sectorsize, bio_offset += sectorsize) {
int ret;
- ret = check_data_csum(inode, io_bio, bio_offset, page, pg_off);
+ ret = check_data_csum(inode, io_bio, bio_offset, page, pg_off,
+ page_offset(page) + pg_off);
if (ret < 0)
return -EIO;
}
ASSERT(pgoff < PAGE_SIZE);
if (uptodate &&
(!csum || !check_data_csum(inode, io_bio,
- bio_offset, bvec.bv_page, pgoff))) {
+ bio_offset, bvec.bv_page,
+ pgoff, start))) {
clean_io_failure(fs_info, failure_tree, io_tree,
start, bvec.bv_page,
btrfs_ino(BTRFS_I(inode)),
bio->bi_end_io = btrfs_end_dio_bio;
btrfs_io_bio(bio)->logical = file_offset;
- WARN_ON_ONCE(write && btrfs_is_zoned(fs_info) &&
- fs_info->max_zone_append_size &&
- bio_op(bio) != REQ_OP_ZONE_APPEND);
-
if (bio_op(bio) == REQ_OP_ZONE_APPEND) {
status = extract_ordered_extent(BTRFS_I(inode), bio,
file_offset);
btrfs_free_space_bitmap_cachep = kmem_cache_create("btrfs_free_space_bitmap",
PAGE_SIZE, PAGE_SIZE,
- SLAB_RED_ZONE, NULL);
+ SLAB_MEM_SPREAD, NULL);
if (!btrfs_free_space_bitmap_cachep)
goto fail;
struct btrfs_path *path;
u64 start = ins->objectid;
u64 len = ins->offset;
+ int qgroup_released;
int ret;
memset(&stack_fi, 0, sizeof(stack_fi));
btrfs_set_stack_file_extent_compression(&stack_fi, BTRFS_COMPRESS_NONE);
/* Encryption and other encoding is reserved and all 0 */
- ret = btrfs_qgroup_release_data(inode, file_offset, len);
- if (ret < 0)
- return ERR_PTR(ret);
+ qgroup_released = btrfs_qgroup_release_data(inode, file_offset, len);
+ if (qgroup_released < 0)
+ return ERR_PTR(qgroup_released);
if (trans) {
ret = insert_reserved_file_extent(trans, inode,
file_offset, &stack_fi,
- true, ret);
+ true, qgroup_released);
if (ret)
- return ERR_PTR(ret);
+ goto free_qgroup;
return trans;
}
extent_info.file_offset = file_offset;
extent_info.extent_buf = (char *)&stack_fi;
extent_info.is_new_extent = true;
- extent_info.qgroup_reserved = ret;
+ extent_info.qgroup_reserved = qgroup_released;
extent_info.insertions = 0;
path = btrfs_alloc_path();
- if (!path)
- return ERR_PTR(-ENOMEM);
+ if (!path) {
+ ret = -ENOMEM;
+ goto free_qgroup;
+ }
ret = btrfs_replace_file_extents(&inode->vfs_inode, path, file_offset,
file_offset + len - 1, &extent_info,
&trans);
btrfs_free_path(path);
if (ret)
- return ERR_PTR(ret);
-
+ goto free_qgroup;
return trans;
+
+free_qgroup:
+ /*
+ * We have released qgroup data range at the beginning of the function,
+ * and normally qgroup_released bytes will be freed when committing
+ * transaction.
+ * But if we error out early, we have to free what we have released
+ * or we leak qgroup data reservation.
+ */
+ btrfs_qgroup_free_refroot(inode->root->fs_info,
+ inode->root->root_key.objectid, qgroup_released,
+ BTRFS_QGROUP_RSV_DATA);
+ return ERR_PTR(ret);
}
static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
{
struct btrfs_qgroup_list *list;
- btrfs_sysfs_del_one_qgroup(fs_info, qgroup);
list_del(&qgroup->dirty);
while (!list_empty(&qgroup->groups)) {
list = list_first_entry(&qgroup->groups,
list_del(&list->next_member);
kfree(list);
}
- kfree(qgroup);
}
/* must be called with qgroup_lock held */
qgroup = rb_entry(n, struct btrfs_qgroup, node);
rb_erase(n, &fs_info->qgroup_tree);
__del_qgroup_rb(fs_info, qgroup);
+ btrfs_sysfs_del_one_qgroup(fs_info, qgroup);
+ kfree(qgroup);
}
/*
* We call btrfs_free_qgroup_config() when unmounting
spin_lock(&fs_info->qgroup_lock);
del_qgroup_rb(fs_info, qgroupid);
spin_unlock(&fs_info->qgroup_lock);
+
+ /*
+ * Remove the qgroup from sysfs now without holding the qgroup_lock
+ * spinlock, since the sysfs_remove_group() function needs to take
+ * the mutex kernfs_mutex through kernfs_remove_by_name_ns().
+ */
+ btrfs_sysfs_del_one_qgroup(fs_info, qgroup);
+ kfree(qgroup);
out:
mutex_unlock(&fs_info->qgroup_ioctl_lock);
return ret;
/* find extent */
spin_lock(&fs_info->reada_lock);
re = radix_tree_lookup(&fs_info->reada_tree,
- eb->start >> PAGE_SHIFT);
+ eb->start >> fs_info->sectorsize_bits);
if (re)
re->refcnt++;
spin_unlock(&fs_info->reada_lock);
zone = NULL;
spin_lock(&fs_info->reada_lock);
ret = radix_tree_gang_lookup(&dev->reada_zones, (void **)&zone,
- logical >> PAGE_SHIFT, 1);
+ logical >> fs_info->sectorsize_bits, 1);
if (ret == 1 && logical >= zone->start && logical <= zone->end) {
kref_get(&zone->refcnt);
spin_unlock(&fs_info->reada_lock);
spin_lock(&fs_info->reada_lock);
ret = radix_tree_insert(&dev->reada_zones,
- (unsigned long)(zone->end >> PAGE_SHIFT),
- zone);
+ (unsigned long)(zone->end >> fs_info->sectorsize_bits),
+ zone);
if (ret == -EEXIST) {
kfree(zone);
ret = radix_tree_gang_lookup(&dev->reada_zones, (void **)&zone,
- logical >> PAGE_SHIFT, 1);
+ logical >> fs_info->sectorsize_bits, 1);
if (ret == 1 && logical >= zone->start && logical <= zone->end)
kref_get(&zone->refcnt);
else
u64 length;
int real_stripes;
int nzones = 0;
- unsigned long index = logical >> PAGE_SHIFT;
+ unsigned long index = logical >> fs_info->sectorsize_bits;
int dev_replace_is_ongoing;
int have_zone = 0;
struct reada_extent *re)
{
int i;
- unsigned long index = re->logical >> PAGE_SHIFT;
+ unsigned long index = re->logical >> fs_info->sectorsize_bits;
spin_lock(&fs_info->reada_lock);
if (--re->refcnt) {
static void reada_zone_release(struct kref *kref)
{
struct reada_zone *zone = container_of(kref, struct reada_zone, refcnt);
+ struct btrfs_fs_info *fs_info = zone->device->fs_info;
- lockdep_assert_held(&zone->device->fs_info->reada_lock);
+ lockdep_assert_held(&fs_info->reada_lock);
radix_tree_delete(&zone->device->reada_zones,
- zone->end >> PAGE_SHIFT);
+ zone->end >> fs_info->sectorsize_bits);
kfree(zone);
}
static void reada_peer_zones_set_lock(struct reada_zone *zone, int lock)
{
int i;
- unsigned long index = zone->end >> PAGE_SHIFT;
+ unsigned long index = zone->end >> zone->device->fs_info->sectorsize_bits;
for (i = 0; i < zone->ndevs; ++i) {
struct reada_zone *peer;
(void **)&zone, index, 1);
if (ret == 0)
break;
- index = (zone->end >> PAGE_SHIFT) + 1;
+ index = (zone->end >> dev->fs_info->sectorsize_bits) + 1;
if (zone->locked) {
if (zone->elems > top_locked_elems) {
top_locked_elems = zone->elems;
* plugging to speed things up
*/
ret = radix_tree_gang_lookup(&dev->reada_extents, (void **)&re,
- dev->reada_next >> PAGE_SHIFT, 1);
+ dev->reada_next >> fs_info->sectorsize_bits, 1);
if (ret == 0 || re->logical > dev->reada_curr_zone->end) {
ret = reada_pick_zone(dev);
if (!ret) {
}
re = NULL;
ret = radix_tree_gang_lookup(&dev->reada_extents, (void **)&re,
- dev->reada_next >> PAGE_SHIFT, 1);
+ dev->reada_next >> fs_info->sectorsize_bits, 1);
}
if (ret == 0) {
spin_unlock(&fs_info->reada_lock);
pr_cont(" curr off %llu",
device->reada_next - zone->start);
pr_cont("\n");
- index = (zone->end >> PAGE_SHIFT) + 1;
+ index = (zone->end >> fs_info->sectorsize_bits) + 1;
}
cnt = 0;
index = 0;
}
}
pr_cont("\n");
- index = (re->logical >> PAGE_SHIFT) + 1;
+ index = (re->logical >> fs_info->sectorsize_bits) + 1;
if (++cnt > 15)
break;
}
if (ret == 0)
break;
if (!re->scheduled) {
- index = (re->logical >> PAGE_SHIFT) + 1;
+ index = (re->logical >> fs_info->sectorsize_bits) + 1;
continue;
}
pr_debug("re: logical %llu size %u list empty %d scheduled %d",
}
}
pr_cont("\n");
- index = (re->logical >> PAGE_SHIFT) + 1;
+ index = (re->logical >> fs_info->sectorsize_bits) + 1;
}
spin_unlock(&fs_info->reada_lock);
}
mutex_lock(&log_root_tree->log_mutex);
- index2 = log_root_tree->log_transid % 2;
- list_add_tail(&root_log_ctx.list, &log_root_tree->log_ctxs[index2]);
- root_log_ctx.log_transid = log_root_tree->log_transid;
-
if (btrfs_is_zoned(fs_info)) {
if (!log_root_tree->node) {
ret = btrfs_alloc_log_tree_node(trans, log_root_tree);
}
}
+ index2 = log_root_tree->log_transid % 2;
+ list_add_tail(&root_log_ctx.list, &log_root_tree->log_ctxs[index2]);
+ root_log_ctx.log_transid = log_root_tree->log_transid;
+
/*
* Now we are safe to update the log_root_tree because we're under the
* log_mutex, and we're a current writer so we're holding the commit
int item_size;
int i, ret, slot;
+ if (!device->fs_info->dev_root)
+ return 0;
+
key.objectid = BTRFS_DEV_STATS_OBJECTID;
key.type = BTRFS_PERSISTENT_ITEM_KEY;
key.offset = device->devid;
/* Pseudo write pointer value for conventional zone */
#define WP_CONVENTIONAL ((u64)-2)
+/*
+ * Location of the first zone of superblock logging zone pairs.
+ *
+ * - primary superblock: 0B (zone 0)
+ * - first copy: 512G (zone starting at that offset)
+ * - second copy: 4T (zone starting at that offset)
+ */
+#define BTRFS_SB_LOG_PRIMARY_OFFSET (0ULL)
+#define BTRFS_SB_LOG_FIRST_OFFSET (512ULL * SZ_1G)
+#define BTRFS_SB_LOG_SECOND_OFFSET (4096ULL * SZ_1G)
+
+#define BTRFS_SB_LOG_FIRST_SHIFT const_ilog2(BTRFS_SB_LOG_FIRST_OFFSET)
+#define BTRFS_SB_LOG_SECOND_SHIFT const_ilog2(BTRFS_SB_LOG_SECOND_OFFSET)
+
/* Number of superblock log zones */
#define BTRFS_NR_SB_LOG_ZONES 2
+/*
+ * Maximum supported zone size. Currently, SMR disks have a zone size of
+ * 256MiB, and we are expecting ZNS drives to be in the 1-4GiB range. We do not
+ * expect the zone size to become larger than 8GiB in the near future.
+ */
+#define BTRFS_MAX_ZONE_SIZE SZ_8G
+
static int copy_zone_info_cb(struct blk_zone *zone, unsigned int idx, void *data)
{
struct blk_zone *zones = data;
}
/*
- * The following zones are reserved as the circular buffer on ZONED btrfs.
- * - The primary superblock: zones 0 and 1
- * - The first copy: zones 16 and 17
- * - The second copy: zones 1024 or zone at 256GB which is minimum, and
- * the following one
+ * Get the first zone number of the superblock mirror
*/
static inline u32 sb_zone_number(int shift, int mirror)
{
- ASSERT(mirror < BTRFS_SUPER_MIRROR_MAX);
+ u64 zone;
+ ASSERT(mirror < BTRFS_SUPER_MIRROR_MAX);
switch (mirror) {
- case 0: return 0;
- case 1: return 16;
- case 2: return min_t(u64, btrfs_sb_offset(mirror) >> shift, 1024);
+ case 0: zone = 0; break;
+ case 1: zone = 1ULL << (BTRFS_SB_LOG_FIRST_SHIFT - shift); break;
+ case 2: zone = 1ULL << (BTRFS_SB_LOG_SECOND_SHIFT - shift); break;
}
- return 0;
+ ASSERT(zone <= U32_MAX);
+
+ return (u32)zone;
}
/*
zone_sectors = bdev_zone_sectors(bdev);
}
- nr_sectors = bdev_nr_sectors(bdev);
/* Check if it's power of 2 (see is_power_of_2) */
ASSERT(zone_sectors != 0 && (zone_sectors & (zone_sectors - 1)) == 0);
zone_info->zone_size = zone_sectors << SECTOR_SHIFT;
+
+ /* We reject devices with a zone size larger than 8GB */
+ if (zone_info->zone_size > BTRFS_MAX_ZONE_SIZE) {
+ btrfs_err_in_rcu(fs_info,
+ "zoned: %s: zone size %llu larger than supported maximum %llu",
+ rcu_str_deref(device->name),
+ zone_info->zone_size, BTRFS_MAX_ZONE_SIZE);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ nr_sectors = bdev_nr_sectors(bdev);
zone_info->zone_size_shift = ilog2(zone_info->zone_size);
zone_info->max_zone_append_size =
(u64)queue_max_zone_append_sectors(queue) << SECTOR_SHIFT;
cache->mnt = path.mnt;
root = path.dentry;
+ ret = -EINVAL;
+ if (mnt_user_ns(path.mnt) != &init_user_ns) {
+ pr_warn("File cache on idmapped mounts not supported");
+ goto error_unsupported;
+ }
+
/* check parameters */
ret = -EOPNOTSUPP;
if (d_is_negative(root) ||
container_of(wait, struct cachefiles_one_read, monitor);
struct cachefiles_object *object;
struct fscache_retrieval *op = monitor->op;
- struct wait_bit_key *key = _key;
+ struct wait_page_key *key = _key;
struct page *page = wait->private;
ASSERT(key);
_enter("{%lu},%u,%d,{%p,%u}",
monitor->netfs_page->index, mode, sync,
- key->flags, key->bit_nr);
+ key->page, key->bit_nr);
- if (key->flags != &page->flags ||
- key->bit_nr != PG_locked)
+ if (key->page != page || key->bit_nr != PG_locked)
return 0;
_debug("--- monitor %p %lx ---", page, page->flags);
select CRYPTO_AES
select CRYPTO_LIB_DES
select KEYS
+ select DNS_RESOLVER
help
This is the client VFS module for the SMB3 family of NAS protocols,
(including support for the most recent, most secure dialect SMB3.1.1)
config CIFS_UPCALL
bool "Kerberos/SPNEGO advanced session setup"
depends on CIFS
- select DNS_RESOLVER
help
Enables an upcall mechanism for CIFS which accesses userspace helper
utilities to provide SPNEGO packaged (RFC 4178) Kerberos tickets
config CIFS_DFS_UPCALL
bool "DFS feature support"
depends on CIFS
- select DNS_RESOLVER
help
Distributed File System (DFS) support is used to access shares
transparently in an enterprise name space, even if the share
cifs_unicode.o nterr.o cifsencrypt.o \
readdir.o ioctl.o sess.o export.o smb1ops.o unc.o winucase.o \
smb2ops.o smb2maperror.o smb2transport.o \
- smb2misc.o smb2pdu.o smb2inode.o smb2file.o cifsacl.o fs_context.o
+ smb2misc.o smb2pdu.o smb2inode.o smb2file.o cifsacl.o fs_context.o \
+ dns_resolve.o
cifs-$(CONFIG_CIFS_XATTR) += xattr.o
cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
-cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o dfs_cache.o
+cifs-$(CONFIG_CIFS_DFS_UPCALL) += cifs_dfs_ref.o dfs_cache.o
cifs-$(CONFIG_CIFS_SWN_UPCALL) += netlink.o cifs_swn.o
/*
* Try to find a matching registration for the tcon's server name and share name.
- * Calls to this funciton must be protected by cifs_swnreg_idr_mutex.
+ * Calls to this function must be protected by cifs_swnreg_idr_mutex.
* TODO Try to avoid memory allocations
*/
static struct cifs_swn_reg *cifs_find_swn_reg(struct cifs_tcon *tcon)
/* Retain old ACEs which we can retain */
for (i = 0; i < src_num_aces; ++i) {
pntace = (struct cifs_ace *) (acl_base + size);
- pnntace = (struct cifs_ace *) (nacl_base + nsize);
if (!new_aces_set && (pntace->flags & INHERITED_ACE)) {
/* Place the new ACEs in between existing explicit and inherited */
}
/* If it's any one of the ACE we're replacing, skip! */
- if ((compare_sids(&pntace->sid, &sid_unix_NFS_mode) == 0) ||
+ if (((compare_sids(&pntace->sid, &sid_unix_NFS_mode) == 0) ||
(compare_sids(&pntace->sid, pownersid) == 0) ||
(compare_sids(&pntace->sid, pgrpsid) == 0) ||
(compare_sids(&pntace->sid, &sid_everyone) == 0) ||
- (compare_sids(&pntace->sid, &sid_authusers) == 0)) {
+ (compare_sids(&pntace->sid, &sid_authusers) == 0))) {
goto next_ace;
}
+ /* update the pointer to the next ACE to populate*/
+ pnntace = (struct cifs_ace *) (nacl_base + nsize);
+
nsize += cifs_copy_ace(pnntace, pntace, NULL);
num_aces++;
seq_puts(m, "none");
else {
convert_delimiter(devname, '/');
- seq_puts(m, devname);
+ /* escape all spaces in share names */
+ seq_escape(m, devname, " \t");
kfree(devname);
}
return 0;
bool binding:1; /* are we binding the session? */
__u16 session_flags;
__u8 smb3signingkey[SMB3_SIGN_KEY_SIZE];
- __u8 smb3encryptionkey[SMB3_SIGN_KEY_SIZE];
- __u8 smb3decryptionkey[SMB3_SIGN_KEY_SIZE];
+ __u8 smb3encryptionkey[SMB3_ENC_DEC_KEY_SIZE];
+ __u8 smb3decryptionkey[SMB3_ENC_DEC_KEY_SIZE];
__u8 preauth_sha_hash[SMB2_PREAUTH_HASH_SIZE];
__u8 binding_preauth_sha_hash[SMB2_PREAUTH_HASH_SIZE];
bool direct_io;
};
-struct cifs_readdata;
-
/* asynchronous read support */
struct cifs_readdata {
struct kref refcount;
*/
#define SMB3_SIGN_KEY_SIZE (16)
+/*
+ * Size of the smb3 encryption/decryption keys
+ */
+#define SMB3_ENC_DEC_KEY_SIZE (32)
+
#define CIFS_CLIENT_CHALLENGE_SIZE (8)
#define CIFS_SERVER_CHALLENGE_SIZE (8)
#define CIFS_HMAC_MD5_HASH_SIZE (16)
*
* This should be called with server->srv_mutex held.
*/
-#ifdef CONFIG_CIFS_DFS_UPCALL
static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
{
int rc;
return !rc ? -1 : 0;
}
+#ifdef CONFIG_CIFS_DFS_UPCALL
/* These functions must be called with server->srv_mutex held */
static void reconn_set_next_dfs_target(struct TCP_Server_Info *server,
struct cifs_sb_info *cifs_sb,
#endif
#ifdef CONFIG_CIFS_DFS_UPCALL
+ if (cifs_sb && cifs_sb->origin_fullpath)
/*
* Set up next DFS target server (if any) for reconnect. If DFS
* feature is disabled, then we will retry last server we
* connected to before.
*/
reconn_set_next_dfs_target(server, cifs_sb, &tgt_list, &tgt_it);
+ else {
+#endif
+ /*
+ * Resolve the hostname again to make sure that IP address is up-to-date.
+ */
+ rc = reconn_set_ipaddr_from_hostname(server);
+ if (rc) {
+ cifs_dbg(FYI, "%s: failed to resolve hostname: %d\n",
+ __func__, rc);
+ }
+
+#ifdef CONFIG_CIFS_DFS_UPCALL
+ }
#endif
+
#ifdef CONFIG_CIFS_SWN_UPCALL
}
#endif
goto posix_open_ret;
}
} else {
+ cifs_revalidate_mapping(*pinode);
cifs_fattr_to_inode(*pinode, &fattr);
}
pr_warn_once("Witness protocol support is experimental\n");
break;
case Opt_rootfs:
-#ifdef CONFIG_CIFS_ROOT
- ctx->rootfs = true;
+#ifndef CONFIG_CIFS_ROOT
+ cifs_dbg(VFS, "rootfs support requires CONFIG_CIFS_ROOT config option\n");
+ goto cifs_parse_mount_err;
#endif
+ ctx->rootfs = true;
break;
case Opt_posixpaths:
if (result.negated)
* We need to be sure that all dirty pages are written and the server
* has actual ctime, mtime and file length.
*/
- if ((request_mask & (STATX_CTIME | STATX_MTIME | STATX_SIZE)) &&
+ if ((request_mask & (STATX_CTIME | STATX_MTIME | STATX_SIZE | STATX_BLOCKS)) &&
!CIFS_CACHE_READ(CIFS_I(inode)) &&
inode->i_mapping && inode->i_mapping->nrpages != 0) {
rc = filemap_fdatawait(inode->i_mapping);
if (rc == 0) {
cifsInode->server_eof = attrs->ia_size;
cifs_setsize(inode, attrs->ia_size);
+ /*
+ * i_blocks is not related to (i_size / i_blksize), but instead
+ * 512 byte (2**9) size is required for calculating num blocks.
+ * Until we can query the server for actual allocation size,
+ * this is best estimate we have for blocks allocated for a file
+ * Number of blocks must be rounded up so size 1 is not 0 blocks
+ */
+ inode->i_blocks = (512 - 1 + attrs->ia_size) >> 9;
/*
* The man page of truncate says if the size changed,
#define SMB2_HMACSHA256_SIZE (32)
#define SMB2_CMACAES_SIZE (16)
#define SMB3_SIGNKEY_SIZE (16)
+#define SMB3_GCM128_CRYPTKEY_SIZE (16)
#define SMB3_GCM256_CRYPTKEY_SIZE (32)
/* Maximum buffer size value we can send with 1 credit */
}
}
spin_unlock(&cifs_tcp_ses_lock);
- cifs_dbg(FYI, "Can not process oplock break for non-existent connection\n");
- return false;
+ cifs_dbg(FYI, "No file id matched, oplock break ignored\n");
+ return true;
}
void
{
int rc;
unsigned int ret_data_len;
+ struct inode *inode;
struct duplicate_extents_to_file dup_ext_buf;
struct cifs_tcon *tcon = tlink_tcon(trgtfile->tlink);
cifs_dbg(FYI, "Duplicate extents: src off %lld dst off %lld len %lld\n",
src_off, dest_off, len);
- rc = smb2_set_file_size(xid, tcon, trgtfile, dest_off + len, false);
- if (rc)
- goto duplicate_extents_out;
+ inode = d_inode(trgtfile->dentry);
+ if (inode->i_size < dest_off + len) {
+ rc = smb2_set_file_size(xid, tcon, trgtfile, dest_off + len, false);
+ if (rc)
+ goto duplicate_extents_out;
+ /*
+ * Although also could set plausible allocation size (i_blocks)
+ * here in addition to setting the file size, in reflink
+ * it is likely that the target file is sparse. Its allocation
+ * size will be queried on next revalidate, but it is important
+ * to make sure that file's cached size is updated immediately
+ */
+ cifs_setsize(inode, dest_off + len);
+ }
rc = SMB2_ioctl(xid, tcon, trgtfile->fid.persistent_fid,
trgtfile->fid.volatile_fid,
FSCTL_DUPLICATE_EXTENTS_TO_FILE,
if (ses->Suid == ses_id) {
ses_enc_key = enc ? ses->smb3encryptionkey :
ses->smb3decryptionkey;
- memcpy(key, ses_enc_key, SMB3_SIGN_KEY_SIZE);
+ memcpy(key, ses_enc_key, SMB3_ENC_DEC_KEY_SIZE);
spin_unlock(&cifs_tcp_ses_lock);
return 0;
}
int rc = 0;
struct scatterlist *sg;
u8 sign[SMB2_SIGNATURE_SIZE] = {};
- u8 key[SMB3_SIGN_KEY_SIZE];
+ u8 key[SMB3_ENC_DEC_KEY_SIZE];
struct aead_request *req;
char *iv;
unsigned int iv_len;
tfm = enc ? server->secmech.ccmaesencrypt :
server->secmech.ccmaesdecrypt;
- if (server->cipher_type == SMB2_ENCRYPTION_AES256_GCM)
+ if ((server->cipher_type == SMB2_ENCRYPTION_AES256_CCM) ||
+ (server->cipher_type == SMB2_ENCRYPTION_AES256_GCM))
rc = crypto_aead_setkey(tfm, key, SMB3_GCM256_CRYPTKEY_SIZE);
else
- rc = crypto_aead_setkey(tfm, key, SMB3_SIGN_KEY_SIZE);
+ rc = crypto_aead_setkey(tfm, key, SMB3_GCM128_CRYPTKEY_SIZE);
if (rc) {
cifs_server_dbg(VFS, "%s: Failed to set aead key %d\n", __func__, rc);
{
unsigned char zero = 0x0;
__u8 i[4] = {0, 0, 0, 1};
- __u8 L[4] = {0, 0, 0, 128};
+ __u8 L128[4] = {0, 0, 0, 128};
+ __u8 L256[4] = {0, 0, 1, 0};
int rc = 0;
unsigned char prfhash[SMB2_HMACSHA256_SIZE];
unsigned char *hashptr = prfhash;
goto smb3signkey_ret;
}
- rc = crypto_shash_update(&server->secmech.sdeschmacsha256->shash,
- L, 4);
+ if ((server->cipher_type == SMB2_ENCRYPTION_AES256_CCM) ||
+ (server->cipher_type == SMB2_ENCRYPTION_AES256_GCM)) {
+ rc = crypto_shash_update(&server->secmech.sdeschmacsha256->shash,
+ L256, 4);
+ } else {
+ rc = crypto_shash_update(&server->secmech.sdeschmacsha256->shash,
+ L128, 4);
+ }
if (rc) {
cifs_server_dbg(VFS, "%s: Could not update with L\n", __func__);
goto smb3signkey_ret;
const struct derivation_triplet *ptriplet)
{
int rc;
+#ifdef CONFIG_CIFS_DEBUG_DUMP_KEYS
+ struct TCP_Server_Info *server = ses->server;
+#endif
/*
* All channels use the same encryption/decryption keys but
rc = generate_key(ses, ptriplet->encryption.label,
ptriplet->encryption.context,
ses->smb3encryptionkey,
- SMB3_SIGN_KEY_SIZE);
+ SMB3_ENC_DEC_KEY_SIZE);
rc = generate_key(ses, ptriplet->decryption.label,
ptriplet->decryption.context,
ses->smb3decryptionkey,
- SMB3_SIGN_KEY_SIZE);
+ SMB3_ENC_DEC_KEY_SIZE);
if (rc)
return rc;
}
*/
cifs_dbg(VFS, "Session Id %*ph\n", (int)sizeof(ses->Suid),
&ses->Suid);
+ cifs_dbg(VFS, "Cipher type %d\n", server->cipher_type);
cifs_dbg(VFS, "Session Key %*ph\n",
SMB2_NTLMV2_SESSKEY_SIZE, ses->auth_key.response);
cifs_dbg(VFS, "Signing Key %*ph\n",
SMB3_SIGN_KEY_SIZE, ses->smb3signingkey);
- cifs_dbg(VFS, "ServerIn Key %*ph\n",
- SMB3_SIGN_KEY_SIZE, ses->smb3encryptionkey);
- cifs_dbg(VFS, "ServerOut Key %*ph\n",
- SMB3_SIGN_KEY_SIZE, ses->smb3decryptionkey);
+ if ((server->cipher_type == SMB2_ENCRYPTION_AES256_CCM) ||
+ (server->cipher_type == SMB2_ENCRYPTION_AES256_GCM)) {
+ cifs_dbg(VFS, "ServerIn Key %*ph\n",
+ SMB3_GCM256_CRYPTKEY_SIZE, ses->smb3encryptionkey);
+ cifs_dbg(VFS, "ServerOut Key %*ph\n",
+ SMB3_GCM256_CRYPTKEY_SIZE, ses->smb3decryptionkey);
+ } else {
+ cifs_dbg(VFS, "ServerIn Key %*ph\n",
+ SMB3_GCM128_CRYPTKEY_SIZE, ses->smb3encryptionkey);
+ cifs_dbg(VFS, "ServerOut Key %*ph\n",
+ SMB3_GCM128_CRYPTKEY_SIZE, ses->smb3decryptionkey);
+ }
#endif
return rc;
}
/*
* Compounding is never used during session establish.
*/
- if ((ses->status == CifsNew) || (optype & CIFS_NEG_OP) || (optype & CIFS_SESS_OP))
+ if ((ses->status == CifsNew) || (optype & CIFS_NEG_OP) || (optype & CIFS_SESS_OP)) {
+ mutex_lock(&server->srv_mutex);
smb311_update_preauth_hash(ses, rqst[0].rq_iov,
rqst[0].rq_nvec);
+ mutex_unlock(&server->srv_mutex);
+ }
for (i = 0; i < num_rqst; i++) {
rc = wait_for_response(server, midQ[i]);
.iov_base = resp_iov[0].iov_base,
.iov_len = resp_iov[0].iov_len
};
+ mutex_lock(&server->srv_mutex);
smb311_update_preauth_hash(ses, &iov, 1);
+ mutex_unlock(&server->srv_mutex);
}
out:
struct buffer_head *map_bh)
{
int ret = 0;
+ int boundary = sdio->boundary; /* dio_send_cur_page may clear it */
if (dio->op == REQ_OP_WRITE) {
/*
sdio->cur_page_fs_offset = sdio->block_in_file << sdio->blkbits;
out:
/*
- * If sdio->boundary then we want to schedule the IO now to
+ * If boundary then we want to schedule the IO now to
* avoid metadata seeks.
*/
- if (sdio->boundary) {
+ if (boundary) {
ret = dio_send_cur_page(dio, sdio, map_bh);
if (sdio->bio)
dio_bio_submit(dio, sdio);
/**
* ext4_should_retry_alloc() - check if a block allocation should be retried
- * @sb: super block
- * @retries: number of attemps has been made
+ * @sb: superblock
+ * @retries: number of retry attempts made so far
*
- * ext4_should_retry_alloc() is called when ENOSPC is returned, and if
- * it is profitable to retry the operation, this function will wait
- * for the current or committing transaction to complete, and then
- * return TRUE. We will only retry once.
+ * ext4_should_retry_alloc() is called when ENOSPC is returned while
+ * attempting to allocate blocks. If there's an indication that a pending
+ * journal transaction might free some space and allow another attempt to
+ * succeed, this function will wait for the current or committing transaction
+ * to complete and then return TRUE.
*/
int ext4_should_retry_alloc(struct super_block *sb, int *retries)
{
- if (!ext4_has_free_clusters(EXT4_SB(sb), 1, 0) ||
- (*retries)++ > 1 ||
- !EXT4_SB(sb)->s_journal)
+ struct ext4_sb_info *sbi = EXT4_SB(sb);
+
+ if (!sbi->s_journal)
return 0;
- smp_mb();
- if (EXT4_SB(sb)->s_mb_free_pending == 0)
+ if (++(*retries) > 3) {
+ percpu_counter_inc(&sbi->s_sra_exceeded_retry_limit);
return 0;
+ }
+ /*
+ * if there's no indication that blocks are about to be freed it's
+ * possible we just missed a transaction commit that did so
+ */
+ smp_mb();
+ if (sbi->s_mb_free_pending == 0)
+ return ext4_has_free_clusters(sbi, 1, 0);
+
+ /*
+ * it's possible we've just missed a transaction commit here,
+ * so ignore the returned status
+ */
jbd_debug(1, "%s: retrying operation after ENOSPC\n", sb->s_id);
- jbd2_journal_force_commit_nested(EXT4_SB(sb)->s_journal);
+ (void) jbd2_journal_force_commit_nested(sbi->s_journal);
return 1;
}
struct percpu_counter s_freeinodes_counter;
struct percpu_counter s_dirs_counter;
struct percpu_counter s_dirtyclusters_counter;
+ struct percpu_counter s_sra_exceeded_retry_limit;
struct blockgroup_lock *s_blockgroup_lock;
struct proc_dir_entry *s_proc;
struct kobject s_kobj;
struct dentry *dentry);
void ext4_fc_track_unlink(handle_t *handle, struct dentry *dentry);
void ext4_fc_track_link(handle_t *handle, struct dentry *dentry);
+void __ext4_fc_track_create(handle_t *handle, struct inode *inode,
+ struct dentry *dentry);
void ext4_fc_track_create(handle_t *handle, struct dentry *dentry);
void ext4_fc_track_inode(handle_t *handle, struct inode *inode);
void ext4_fc_mark_ineligible(struct super_block *sb, int reason);
{
struct inode *inode = file_inode(file);
handle_t *handle;
- int ret, ret2 = 0, ret3 = 0;
+ int ret = 0, ret2 = 0, ret3 = 0;
int retries = 0;
int depth = 0;
struct ext4_map_blocks map;
__ext4_fc_track_link(handle, d_inode(dentry), dentry);
}
-void ext4_fc_track_create(handle_t *handle, struct dentry *dentry)
+void __ext4_fc_track_create(handle_t *handle, struct inode *inode,
+ struct dentry *dentry)
{
struct __track_dentry_update_args args;
- struct inode *inode = d_inode(dentry);
int ret;
args.dentry = dentry;
trace_ext4_fc_track_create(inode, dentry, ret);
}
+void ext4_fc_track_create(handle_t *handle, struct dentry *dentry)
+{
+ __ext4_fc_track_create(handle, d_inode(dentry), dentry);
+}
+
/* __track_fn for inode tracking */
static int __track_inode(struct inode *inode, void *arg, bool update)
{
if (!ret)
ret = err;
- if (!ext4_has_inline_data(inode))
- ext4_walk_page_buffers(NULL, page_bufs, 0, len,
- NULL, bput_one);
ext4_set_inode_state(inode, EXT4_STATE_JDATA);
out:
unlock_page(page);
out_no_pagelock:
+ if (!inline_data && page_bufs)
+ ext4_walk_page_buffers(NULL, page_bufs, 0, len,
+ NULL, bput_one);
brelse(inode_bh);
return ret;
}
struct ext4_inode_info *ei = EXT4_I(inode);
struct buffer_head *bh = iloc->bh;
struct super_block *sb = inode->i_sb;
- int err = 0, rc, block;
+ int err = 0, block;
int need_datasync = 0, set_large_file = 0;
uid_t i_uid;
gid_t i_gid;
bh->b_data);
BUFFER_TRACE(bh, "call ext4_handle_dirty_metadata");
- rc = ext4_handle_dirty_metadata(handle, NULL, bh);
- if (!err)
- err = rc;
+ err = ext4_handle_dirty_metadata(handle, NULL, bh);
+ if (err)
+ goto out_brelse;
ext4_clear_inode_state(inode, EXT4_STATE_NEW);
if (set_large_file) {
BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get write access");
inode->i_gid = attr->ia_gid;
error = ext4_mark_inode_dirty(handle, inode);
ext4_journal_stop(handle);
- if (unlikely(error))
+ if (unlikely(error)) {
+ ext4_fc_stop_update(inode);
return error;
+ }
}
if (attr->ia_valid & ATTR_SIZE) {
}
if (ext4_has_feature_flex_bg(sb)) {
- /* a single flex group is supposed to be read by a single IO */
- sbi->s_mb_prefetch = min(1 << sbi->s_es->s_log_groups_per_flex,
+ /* a single flex group is supposed to be read by a single IO.
+ * 2 ^ s_log_groups_per_flex != UINT_MAX as s_mb_prefetch is
+ * unsigned integer, so the maximum shift is 32.
+ */
+ if (sbi->s_es->s_log_groups_per_flex >= 32) {
+ ext4_msg(sb, KERN_ERR, "too many log groups per flexible block group");
+ goto err_freesgi;
+ }
+ sbi->s_mb_prefetch = min_t(uint, 1 << sbi->s_es->s_log_groups_per_flex,
BLK_MAX_SEGMENT_SIZE >> (sb->s_blocksize_bits - 9));
sbi->s_mb_prefetch *= 8; /* 8 prefetch IOs in flight at most */
} else {
return retval;
}
+static void ext4_resetent(handle_t *handle, struct ext4_renament *ent,
+ unsigned ino, unsigned file_type)
+{
+ struct ext4_renament old = *ent;
+ int retval = 0;
+
+ /*
+ * old->de could have moved from under us during make indexed dir,
+ * so the old->de may no longer valid and need to find it again
+ * before reset old inode info.
+ */
+ old.bh = ext4_find_entry(old.dir, &old.dentry->d_name, &old.de, NULL);
+ if (IS_ERR(old.bh))
+ retval = PTR_ERR(old.bh);
+ if (!old.bh)
+ retval = -ENOENT;
+ if (retval) {
+ ext4_std_error(old.dir->i_sb, retval);
+ return;
+ }
+
+ ext4_setent(handle, &old, ino, file_type);
+ brelse(old.bh);
+}
+
static int ext4_find_delete_entry(handle_t *handle, struct inode *dir,
const struct qstr *d_name)
{
*/
retval = -ENOENT;
if (!old.bh || le32_to_cpu(old.de->inode) != old.inode->i_ino)
- goto end_rename;
+ goto release_bh;
new.bh = ext4_find_entry(new.dir, &new.dentry->d_name,
&new.de, &new.inlined);
if (IS_ERR(new.bh)) {
retval = PTR_ERR(new.bh);
new.bh = NULL;
- goto end_rename;
+ goto release_bh;
}
if (new.bh) {
if (!new.inode) {
handle = ext4_journal_start(old.dir, EXT4_HT_DIR, credits);
if (IS_ERR(handle)) {
retval = PTR_ERR(handle);
- handle = NULL;
- goto end_rename;
+ goto release_bh;
}
} else {
whiteout = ext4_whiteout_for_rename(mnt_userns, &old, credits, &handle);
if (IS_ERR(whiteout)) {
retval = PTR_ERR(whiteout);
- whiteout = NULL;
- goto end_rename;
+ goto release_bh;
}
}
retval = ext4_mark_inode_dirty(handle, whiteout);
if (unlikely(retval))
goto end_rename;
+
}
if (!new.bh) {
retval = ext4_add_entry(handle, new.dentry, old.inode);
ext4_fc_track_unlink(handle, new.dentry);
__ext4_fc_track_link(handle, old.inode, new.dentry);
__ext4_fc_track_unlink(handle, old.inode, old.dentry);
+ if (whiteout)
+ __ext4_fc_track_create(handle, whiteout, old.dentry);
}
if (new.inode) {
end_rename:
if (whiteout) {
if (retval) {
- ext4_setent(handle, &old,
- old.inode->i_ino, old_file_type);
+ ext4_resetent(handle, &old,
+ old.inode->i_ino, old_file_type);
drop_nlink(whiteout);
+ ext4_orphan_add(handle, whiteout);
}
unlock_new_inode(whiteout);
+ ext4_journal_stop(handle);
iput(whiteout);
-
+ } else {
+ ext4_journal_stop(handle);
}
+release_bh:
brelse(old.dir_bh);
brelse(old.bh);
brelse(new.bh);
- if (handle)
- ext4_journal_stop(handle);
return retval;
}
percpu_counter_destroy(&sbi->s_freeinodes_counter);
percpu_counter_destroy(&sbi->s_dirs_counter);
percpu_counter_destroy(&sbi->s_dirtyclusters_counter);
+ percpu_counter_destroy(&sbi->s_sra_exceeded_retry_limit);
percpu_free_rwsem(&sbi->s_writepages_rwsem);
#ifdef CONFIG_QUOTA
for (i = 0; i < EXT4_MAXQUOTAS; i++)
if (!err)
err = percpu_counter_init(&sbi->s_dirtyclusters_counter, 0,
GFP_KERNEL);
+ if (!err)
+ err = percpu_counter_init(&sbi->s_sra_exceeded_retry_limit, 0,
+ GFP_KERNEL);
if (!err)
err = percpu_init_rwsem(&sbi->s_writepages_rwsem);
percpu_counter_destroy(&sbi->s_freeinodes_counter);
percpu_counter_destroy(&sbi->s_dirs_counter);
percpu_counter_destroy(&sbi->s_dirtyclusters_counter);
+ percpu_counter_destroy(&sbi->s_sra_exceeded_retry_limit);
percpu_free_rwsem(&sbi->s_writepages_rwsem);
failed_mount5:
ext4_ext_release(sb);
failed_mount3a:
ext4_es_unregister_shrinker(sbi);
failed_mount3:
- del_timer_sync(&sbi->s_err_report);
flush_work(&sbi->s_error_work);
+ del_timer_sync(&sbi->s_err_report);
if (sbi->s_mmp_tsk)
kthread_stop(sbi->s_mmp_tsk);
failed_mount2:
attr_session_write_kbytes,
attr_lifetime_write_kbytes,
attr_reserved_clusters,
+ attr_sra_exceeded_retry_limit,
attr_inode_readahead,
attr_trigger_test_error,
attr_first_error_time,
EXT4_ATTR_FUNC(session_write_kbytes, 0444);
EXT4_ATTR_FUNC(lifetime_write_kbytes, 0444);
EXT4_ATTR_FUNC(reserved_clusters, 0644);
+EXT4_ATTR_FUNC(sra_exceeded_retry_limit, 0444);
EXT4_ATTR_OFFSET(inode_readahead_blks, 0644, inode_readahead,
ext4_sb_info, s_inode_readahead_blks);
ATTR_LIST(session_write_kbytes),
ATTR_LIST(lifetime_write_kbytes),
ATTR_LIST(reserved_clusters),
+ ATTR_LIST(sra_exceeded_retry_limit),
ATTR_LIST(inode_readahead_blks),
ATTR_LIST(inode_goal),
ATTR_LIST(mb_stats),
return snprintf(buf, PAGE_SIZE, "%llu\n",
(unsigned long long)
atomic64_read(&sbi->s_resv_clusters));
+ case attr_sra_exceeded_retry_limit:
+ return snprintf(buf, PAGE_SIZE, "%llu\n",
+ (unsigned long long)
+ percpu_counter_sum(&sbi->s_sra_exceeded_retry_limit));
case attr_inode_readahead:
case attr_pointer_ui:
if (!ptr)
struct inode *inode = file_inode(filp);
const int credits = 2; /* superblock and inode for ext4_orphan_del() */
handle_t *handle;
+ struct ext4_iloc iloc;
int err = 0;
- int err2;
- if (desc != NULL) {
- /* Succeeded; write the verity descriptor. */
- err = ext4_write_verity_descriptor(inode, desc, desc_size,
- merkle_tree_size);
-
- /* Write all pages before clearing VERITY_IN_PROGRESS. */
- if (!err)
- err = filemap_write_and_wait(inode->i_mapping);
- }
+ /*
+ * If an error already occurred (which fs/verity/ signals by passing
+ * desc == NULL), then only clean-up is needed.
+ */
+ if (desc == NULL)
+ goto cleanup;
- /* If we failed, truncate anything we wrote past i_size. */
- if (desc == NULL || err)
- ext4_truncate(inode);
+ /* Append the verity descriptor. */
+ err = ext4_write_verity_descriptor(inode, desc, desc_size,
+ merkle_tree_size);
+ if (err)
+ goto cleanup;
/*
- * We must always clean up by clearing EXT4_STATE_VERITY_IN_PROGRESS and
- * deleting the inode from the orphan list, even if something failed.
- * If everything succeeded, we'll also set the verity bit in the same
- * transaction.
+ * Write all pages (both data and verity metadata). Note that this must
+ * happen before clearing EXT4_STATE_VERITY_IN_PROGRESS; otherwise pages
+ * beyond i_size won't be written properly. For crash consistency, this
+ * also must happen before the verity inode flag gets persisted.
*/
+ err = filemap_write_and_wait(inode->i_mapping);
+ if (err)
+ goto cleanup;
- ext4_clear_inode_state(inode, EXT4_STATE_VERITY_IN_PROGRESS);
+ /*
+ * Finally, set the verity inode flag and remove the inode from the
+ * orphan list (in a single transaction).
+ */
handle = ext4_journal_start(inode, EXT4_HT_INODE, credits);
if (IS_ERR(handle)) {
- ext4_orphan_del(NULL, inode);
- return PTR_ERR(handle);
+ err = PTR_ERR(handle);
+ goto cleanup;
}
- err2 = ext4_orphan_del(handle, inode);
- if (err2)
- goto out_stop;
+ err = ext4_orphan_del(handle, inode);
+ if (err)
+ goto stop_and_cleanup;
- if (desc != NULL && !err) {
- struct ext4_iloc iloc;
+ err = ext4_reserve_inode_write(handle, inode, &iloc);
+ if (err)
+ goto stop_and_cleanup;
- err = ext4_reserve_inode_write(handle, inode, &iloc);
- if (err)
- goto out_stop;
- ext4_set_inode_flag(inode, EXT4_INODE_VERITY);
- ext4_set_inode_flags(inode, false);
- err = ext4_mark_iloc_dirty(handle, inode, &iloc);
- }
-out_stop:
+ ext4_set_inode_flag(inode, EXT4_INODE_VERITY);
+ ext4_set_inode_flags(inode, false);
+ err = ext4_mark_iloc_dirty(handle, inode, &iloc);
+ if (err)
+ goto stop_and_cleanup;
+
+ ext4_journal_stop(handle);
+
+ ext4_clear_inode_state(inode, EXT4_STATE_VERITY_IN_PROGRESS);
+ return 0;
+
+stop_and_cleanup:
ext4_journal_stop(handle);
- return err ?: err2;
+cleanup:
+ /*
+ * Verity failed to be enabled, so clean up by truncating any verity
+ * metadata that was written beyond i_size (both from cache and from
+ * disk), removing the inode from the orphan list (if it wasn't done
+ * already), and clearing EXT4_STATE_VERITY_IN_PROGRESS.
+ */
+ truncate_inode_pages(inode->i_mapping, inode->i_size);
+ ext4_truncate(inode);
+ ext4_orphan_del(NULL, inode);
+ ext4_clear_inode_state(inode, EXT4_STATE_VERITY_IN_PROGRESS);
+ return err;
}
static int ext4_get_verity_descriptor_location(struct inode *inode,
if (!ce)
return NULL;
+ WARN_ON_ONCE(ext4_handle_valid(journal_current_handle()) &&
+ !(current->flags & PF_MEMALLOC_NOFS));
+
ea_data = kvmalloc(value_len, GFP_KERNEL);
if (!ea_data) {
mb_cache_entry_put(ea_inode_cache, ce);
error = -ENOSPC;
goto cleanup;
}
+ WARN_ON_ONCE(!(current->flags & PF_MEMALLOC_NOFS));
}
error = ext4_reserve_inode_write(handle, inode, &is.iloc);
* external inode if possible.
*/
if (ext4_has_feature_ea_inode(inode->i_sb) &&
- !i.in_inode) {
+ i.value_len && !i.in_inode) {
i.in_inode = 1;
goto retry_inode;
}
}
EXPORT_SYMBOL(close_fd); /* for ksys_close() */
+/**
+ * last_fd - return last valid index into fd table
+ * @cur_fds: files struct
+ *
+ * Context: Either rcu read lock or files_lock must be held.
+ *
+ * Returns: Last valid index into fdtable.
+ */
+static inline unsigned last_fd(struct fdtable *fdt)
+{
+ return fdt->max_fds - 1;
+}
+
static inline void __range_cloexec(struct files_struct *cur_fds,
unsigned int fd, unsigned int max_fd)
{
struct fdtable *fdt;
- if (fd > max_fd)
- return;
-
+ /* make sure we're using the correct maximum value */
spin_lock(&cur_fds->file_lock);
fdt = files_fdtable(cur_fds);
- bitmap_set(fdt->close_on_exec, fd, max_fd - fd + 1);
+ max_fd = min(last_fd(fdt), max_fd);
+ if (fd <= max_fd)
+ bitmap_set(fdt->close_on_exec, fd, max_fd - fd + 1);
spin_unlock(&cur_fds->file_lock);
}
static long fuse_dev_ioctl(struct file *file, unsigned int cmd,
unsigned long arg)
{
- int err = -ENOTTY;
+ int res;
+ int oldfd;
+ struct fuse_dev *fud = NULL;
- if (cmd == FUSE_DEV_IOC_CLONE) {
- int oldfd;
+ if (_IOC_TYPE(cmd) != FUSE_DEV_IOC_MAGIC)
+ return -ENOTTY;
- err = -EFAULT;
- if (!get_user(oldfd, (__u32 __user *) arg)) {
+ switch (_IOC_NR(cmd)) {
+ case _IOC_NR(FUSE_DEV_IOC_CLONE):
+ res = -EFAULT;
+ if (!get_user(oldfd, (__u32 __user *)arg)) {
struct file *old = fget(oldfd);
- err = -EINVAL;
+ res = -EINVAL;
if (old) {
- struct fuse_dev *fud = NULL;
-
/*
* Check against file->f_op because CUSE
* uses the same ioctl handler.
if (fud) {
mutex_lock(&fuse_mutex);
- err = fuse_device_clone(fud->fc, file);
+ res = fuse_device_clone(fud->fc, file);
mutex_unlock(&fuse_mutex);
}
fput(old);
}
}
+ break;
+ default:
+ res = -ENOTTY;
+ break;
}
- return err;
+ return res;
}
const struct file_operations fuse_dev_operations = {
static inline void fuse_make_bad(struct inode *inode)
{
+ remove_inode_hash(inode);
set_bit(FUSE_I_BAD, &get_fuse_inode(inode)->state);
}
/* virtiofs allocates and installs its own fuse devices */
ctx->fudptr = NULL;
- if (ctx->dax)
+ if (ctx->dax) {
+ if (!fs->dax_dev) {
+ err = -EINVAL;
+ pr_err("virtio-fs: dax can't be enabled as filesystem"
+ " device does not support it.\n");
+ goto err_free_fuse_devs;
+ }
ctx->dax_dev = fs->dax_dev;
+ }
err = fuse_fill_super_common(sb, ctx);
if (err < 0)
goto err_free_fuse_devs;
int error;
error = init_threads(sdp);
- if (error)
+ if (error) {
+ gfs2_withdraw_delayed(sdp);
return error;
+ }
j_gl->gl_ops->go_inval(j_gl, DIO_METADATA);
if (gfs2_withdrawn(sdp)) {
static int gfs2_freeze(struct super_block *sb)
{
struct gfs2_sbd *sdp = sb->s_fs_info;
- int error = 0;
+ int error;
mutex_lock(&sdp->sd_freeze_mutex);
- if (atomic_read(&sdp->sd_freeze_state) != SFS_UNFROZEN)
+ if (atomic_read(&sdp->sd_freeze_state) != SFS_UNFROZEN) {
+ error = -EBUSY;
goto out;
+ }
for (;;) {
if (gfs2_withdrawn(sdp)) {
struct gfs2_sbd *sdp = sb->s_fs_info;
mutex_lock(&sdp->sd_freeze_mutex);
- if (atomic_read(&sdp->sd_freeze_state) != SFS_FROZEN ||
+ if (atomic_read(&sdp->sd_freeze_state) != SFS_FROZEN ||
!gfs2_holder_initialized(&sdp->sd_freeze_gh)) {
mutex_unlock(&sdp->sd_freeze_mutex);
- return 0;
+ return -EINVAL;
}
gfs2_freeze_unlock(&sdp->sd_freeze_gh);
char *name, *resolved, *end;
int n;
- name = __getname();
+ name = kmalloc(PATH_MAX, GFP_KERNEL);
if (!name) {
n = -ENOMEM;
goto out_free;
goto out_free;
}
- __putname(name);
- kfree(link);
+ kfree(name);
return resolved;
out_free:
- __putname(name);
+ kfree(name);
return ERR_PTR(n);
}
#include <linux/rculist_nulls.h>
#include <linux/cpu.h>
#include <linux/tracehook.h>
-#include <linux/freezer.h>
#include "../kernel/sched/sched.h"
#include "io-wq.h"
return NULL;
}
-static void io_flush_signals(void)
+static bool io_flush_signals(void)
{
- if (unlikely(test_tsk_thread_flag(current, TIF_NOTIFY_SIGNAL))) {
- if (current->task_works)
- task_work_run();
- clear_tsk_thread_flag(current, TIF_NOTIFY_SIGNAL);
+ if (unlikely(test_thread_flag(TIF_NOTIFY_SIGNAL))) {
+ __set_current_state(TASK_RUNNING);
+ tracehook_notify_signal();
+ return true;
}
+ return false;
}
static void io_assign_current_work(struct io_worker *worker,
{
struct io_wqe *wqe = worker->wqe;
struct io_wq *wq = wqe->wq;
+ bool do_kill = test_bit(IO_WQ_BIT_EXIT, &wq->state);
do {
struct io_wq_work *work;
unsigned int hash = io_get_work_hash(work);
next_hashed = wq_next_work(work);
+
+ if (unlikely(do_kill) && (work->flags & IO_WQ_WORK_UNBOUND))
+ work->flags |= IO_WQ_WORK_CANCEL;
wq->do_work(work);
io_assign_current_work(worker, NULL);
worker->flags |= (IO_WORKER_F_UP | IO_WORKER_F_RUNNING);
io_wqe_inc_running(worker);
- sprintf(buf, "iou-wrk-%d", wq->task_pid);
+ snprintf(buf, sizeof(buf), "iou-wrk-%d", wq->task_pid);
set_task_comm(current, buf);
while (!test_bit(IO_WQ_BIT_EXIT, &wq->state)) {
+ long ret;
+
set_current_state(TASK_INTERRUPTIBLE);
loop:
raw_spin_lock_irq(&wqe->lock);
}
__io_worker_idle(wqe, worker);
raw_spin_unlock_irq(&wqe->lock);
- io_flush_signals();
- if (schedule_timeout(WORKER_IDLE_TIMEOUT))
+ if (io_flush_signals())
continue;
- if (fatal_signal_pending(current))
+ ret = schedule_timeout(WORKER_IDLE_TIMEOUT);
+ if (signal_pending(current)) {
+ struct ksignal ksig;
+
+ if (!get_signal(&ksig))
+ continue;
break;
+ }
+ if (ret)
+ continue;
/* timed out, exit unless we're the fixed worker */
if (test_bit(IO_WQ_BIT_EXIT, &wq->state) ||
!(worker->flags & IO_WORKER_F_FIXED))
char buf[TASK_COMM_LEN];
int node;
- sprintf(buf, "iou-mgr-%d", wq->task_pid);
+ snprintf(buf, sizeof(buf), "iou-mgr-%d", wq->task_pid);
set_task_comm(current, buf);
do {
set_current_state(TASK_INTERRUPTIBLE);
io_wq_check_workers(wq);
schedule_timeout(HZ);
- if (fatal_signal_pending(current))
+ if (signal_pending(current)) {
+ struct ksignal ksig;
+
+ if (!get_signal(&ksig))
+ continue;
set_bit(IO_WQ_BIT_EXIT, &wq->state);
+ }
} while (!test_bit(IO_WQ_BIT_EXIT, &wq->state));
io_wq_check_workers(wq);
for_each_node(node) {
struct io_wqe *wqe = wq->wqes[node];
- WARN_ON_ONCE(!wq_list_empty(&wqe->work_list));
+ struct io_cb_cancel_data match = {
+ .fn = io_wq_work_match_all,
+ .cancel_all = true,
+ };
+ io_wqe_cancel_pending_work(wqe, &match);
kfree(wqe);
}
io_wq_put_hash(wq->hash);
#define INTERNAL_IO_WQ_H
#include <linux/refcount.h>
-#include <linux/io_uring.h>
struct io_wq;
IO_WQ_CANCEL_NOTFOUND, /* work not found */
};
+struct io_wq_work_node {
+ struct io_wq_work_node *next;
+};
+
+struct io_wq_work_list {
+ struct io_wq_work_node *first;
+ struct io_wq_work_node *last;
+};
+
static inline void wq_list_add_after(struct io_wq_work_node *node,
struct io_wq_work_node *pos,
struct io_wq_work_list *list)
#include <linux/task_work.h>
#include <linux/pagemap.h>
#include <linux/io_uring.h>
-#include <linux/freezer.h>
#define CREATE_TRACE_POINTS
#include <trace/events/io_uring.h>
struct io_sq_data {
refcount_t refs;
- struct rw_semaphore rw_lock;
+ atomic_t park_pending;
+ struct mutex lock;
/* ctx's that are using this sqd */
struct list_head ctx_list;
unsigned long state;
struct completion exited;
+ struct callback_head *park_task_work;
};
#define IO_IOPOLL_BATCH 8
struct socket *ring_sock;
#endif
- struct idr io_buffer_idr;
+ struct xarray io_buffers;
struct xarray personalities;
u32 pers_next;
struct list_head tctx_list;
};
+struct io_uring_task {
+ /* submission side */
+ struct xarray xa;
+ struct wait_queue_head wait;
+ const struct io_ring_ctx *last;
+ struct io_wq *io_wq;
+ struct percpu_counter inflight;
+ atomic_t in_idle;
+ bool sqpoll;
+
+ spinlock_t task_lock;
+ struct io_wq_work_list task_list;
+ unsigned long task_state;
+ struct callback_head task_work;
+};
+
/*
* First field must be the file pointer in all the
* iocb unions! See also 'struct kiocb' in <linux/fs.h>
REQ_F_NO_FILE_TABLE_BIT,
REQ_F_LTIMEOUT_ACTIVE_BIT,
REQ_F_COMPLETE_INLINE_BIT,
+ REQ_F_REISSUE_BIT,
/* not a real bit, just to check we're not overflowing the space */
__REQ_F_LAST_BIT,
REQ_F_LTIMEOUT_ACTIVE = BIT(REQ_F_LTIMEOUT_ACTIVE_BIT),
/* completion is deferred through io_comp_state */
REQ_F_COMPLETE_INLINE = BIT(REQ_F_COMPLETE_INLINE_BIT),
+ /* caller should reissue async */
+ REQ_F_REISSUE = BIT(REQ_F_REISSUE_BIT),
};
struct async_poll {
io_for_each_link(req, head) {
if (req->flags & REQ_F_INFLIGHT)
return true;
- if (req->task->files == files)
- return true;
}
return false;
}
init_waitqueue_head(&ctx->cq_wait);
INIT_LIST_HEAD(&ctx->cq_overflow_list);
init_completion(&ctx->ref_comp);
- idr_init(&ctx->io_buffer_idr);
+ xa_init_flags(&ctx->io_buffers, XA_FLAGS_ALLOC1);
xa_init_flags(&ctx->personalities, XA_FLAGS_ALLOC1);
mutex_init(&ctx->uring_lock);
init_waitqueue_head(&ctx->wait);
if (req->flags & REQ_F_ISREG) {
if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
io_wq_hash_work(&req->work, file_inode(req->file));
- } else {
+ } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
if (def->unbound_nonreg_file)
req->work.flags |= IO_WQ_WORK_UNBOUND;
}
BUG_ON(!tctx);
BUG_ON(!tctx->io_wq);
- trace_io_uring_queue_async_work(ctx, io_wq_is_hashed(&req->work), req,
- &req->work, req->flags);
/* init ->work of the whole link before punting */
io_prep_async_link(req);
+ trace_io_uring_queue_async_work(ctx, io_wq_is_hashed(&req->work), req,
+ &req->work, req->flags);
io_wq_enqueue(tctx->io_wq, &req->work);
if (link)
io_queue_linked_timeout(link);
}
-static void io_kill_timeout(struct io_kiocb *req)
+static void io_kill_timeout(struct io_kiocb *req, int status)
{
struct io_timeout_data *io = req->async_data;
int ret;
atomic_set(&req->ctx->cq_timeouts,
atomic_read(&req->ctx->cq_timeouts) + 1);
list_del_init(&req->timeout.list);
- io_cqring_fill_event(req, 0);
+ io_cqring_fill_event(req, status);
io_put_req_deferred(req, 1);
}
}
-/*
- * Returns true if we found and killed one or more timeouts
- */
-static bool io_kill_timeouts(struct io_ring_ctx *ctx, struct task_struct *tsk,
- struct files_struct *files)
-{
- struct io_kiocb *req, *tmp;
- int canceled = 0;
-
- spin_lock_irq(&ctx->completion_lock);
- list_for_each_entry_safe(req, tmp, &ctx->timeout_list, timeout.list) {
- if (io_match_task(req, tsk, files)) {
- io_kill_timeout(req);
- canceled++;
- }
- }
- spin_unlock_irq(&ctx->completion_lock);
- return canceled != 0;
-}
-
static void __io_queue_deferred(struct io_ring_ctx *ctx)
{
do {
break;
list_del_init(&req->timeout.list);
- io_kill_timeout(req);
+ io_kill_timeout(req, 0);
} while (!list_empty(&ctx->timeout_list));
ctx->cq_last_tm_flush = seq;
io_put_task(req->task, 1);
list_add(&req->compl.list, &cs->locked_free_list);
cs->locked_free_nr++;
- } else
- req = NULL;
+ } else {
+ if (!percpu_ref_tryget(&ctx->refs))
+ req = NULL;
+ }
io_commit_cqring(ctx);
spin_unlock_irqrestore(&ctx->completion_lock, flags);
- io_cqring_ev_posted(ctx);
- if (req)
+ if (req) {
+ io_cqring_ev_posted(ctx);
percpu_ref_put(&ctx->refs);
+ }
}
static void io_req_complete_state(struct io_kiocb *req, long res,
return ret;
}
-static void io_req_task_work_add_fallback(struct io_kiocb *req,
- task_work_func_t cb)
+static bool io_run_task_work_head(struct callback_head **work_head)
+{
+ struct callback_head *work, *next;
+ bool executed = false;
+
+ do {
+ work = xchg(work_head, NULL);
+ if (!work)
+ break;
+
+ do {
+ next = work->next;
+ work->func(work);
+ work = next;
+ cond_resched();
+ } while (work);
+ executed = true;
+ } while (1);
+
+ return executed;
+}
+
+static void io_task_work_add_head(struct callback_head **work_head,
+ struct callback_head *task_work)
{
- struct io_ring_ctx *ctx = req->ctx;
struct callback_head *head;
- init_task_work(&req->task_work, cb);
do {
- head = READ_ONCE(ctx->exit_task_work);
- req->task_work.next = head;
- } while (cmpxchg(&ctx->exit_task_work, head, &req->task_work) != head);
+ head = READ_ONCE(*work_head);
+ task_work->next = head;
+ } while (cmpxchg(work_head, head, task_work) != head);
+}
+
+static void io_req_task_work_add_fallback(struct io_kiocb *req,
+ task_work_func_t cb)
+{
+ init_task_work(&req->task_work, cb);
+ io_task_work_add_head(&req->ctx->exit_task_work, &req->task_work);
}
static void __io_req_task_cancel(struct io_kiocb *req, int error)
return false;
return true;
}
+#else
+static bool io_rw_should_reissue(struct io_kiocb *req)
+{
+ return false;
+}
#endif
static bool io_rw_reissue(struct io_kiocb *req)
{
int cflags = 0;
- if ((res == -EAGAIN || res == -EOPNOTSUPP) && io_rw_reissue(req))
+ if (req->rw.kiocb.ki_flags & IOCB_WRITE)
+ kiocb_end_write(req);
+ if ((res == -EAGAIN || res == -EOPNOTSUPP) && io_rw_should_reissue(req)) {
+ req->flags |= REQ_F_REISSUE;
return;
+ }
if (res != req->result)
req_set_fail_links(req);
-
- if (req->rw.kiocb.ki_flags & IOCB_WRITE)
- kiocb_end_write(req);
if (req->flags & REQ_F_BUFFER_SELECTED)
cflags = io_put_rw_kbuf(req);
__io_req_complete(req, issue_flags, res, cflags);
{
struct io_kiocb *req = container_of(kiocb, struct io_kiocb, rw.kiocb);
struct io_async_rw *io = req->async_data;
+ bool check_reissue = kiocb->ki_complete == io_complete_rw;
/* add previously done IO, if any */
if (io && io->bytes_done > 0) {
__io_complete_rw(req, ret, 0, issue_flags);
else
io_rw_done(kiocb, ret);
+
+ if (check_reissue && req->flags & REQ_F_REISSUE) {
+ req->flags &= ~REQ_F_REISSUE;
+ if (!io_rw_reissue(req)) {
+ int cflags = 0;
+
+ req_set_fail_links(req);
+ if (req->flags & REQ_F_BUFFER_SELECTED)
+ cflags = io_put_rw_kbuf(req);
+ __io_req_complete(req, issue_flags, ret, cflags);
+ }
+ }
}
static int io_import_fixed(struct io_kiocb *req, int rw, struct iov_iter *iter)
lockdep_assert_held(&req->ctx->uring_lock);
- head = idr_find(&req->ctx->io_buffer_idr, bgid);
+ head = xa_load(&req->ctx->io_buffers, bgid);
if (head) {
if (!list_empty(&head->list)) {
kbuf = list_last_entry(&head->list, struct io_buffer,
list_del(&kbuf->list);
} else {
kbuf = head;
- idr_remove(&req->ctx->io_buffer_idr, bgid);
+ xa_erase(&req->ctx->io_buffers, bgid);
}
if (*len > kbuf->len)
*len = kbuf->len;
ret = io_iter_do_read(req, iter);
- if (ret == -EIOCBQUEUED) {
- if (req->async_data)
- iov_iter_revert(iter, io_size - iov_iter_count(iter));
- goto out_free;
- } else if (ret == -EAGAIN) {
+ if (ret == -EAGAIN || (req->flags & REQ_F_REISSUE)) {
+ req->flags &= ~REQ_F_REISSUE;
/* IOPOLL retry should happen for io-wq threads */
if (!force_nonblock && !(req->ctx->flags & IORING_SETUP_IOPOLL))
goto done;
/* some cases will consume bytes even on error returns */
iov_iter_revert(iter, io_size - iov_iter_count(iter));
ret = 0;
+ } else if (ret == -EIOCBQUEUED) {
+ goto out_free;
} else if (ret <= 0 || ret == io_size || !force_nonblock ||
(req->flags & REQ_F_NOWAIT) || !(req->flags & REQ_F_ISREG)) {
/* read all, failed, already did sync or don't want to retry */
else
ret2 = -EINVAL;
+ if (req->flags & REQ_F_REISSUE) {
+ req->flags &= ~REQ_F_REISSUE;
+ ret2 = -EAGAIN;
+ }
+
/*
* Raw bdev writes will return -EOPNOTSUPP for IOCB_NOWAIT. Just
* retry them without IOCB_NOWAIT.
/* no retry on NONBLOCK nor RWF_NOWAIT */
if (ret2 == -EAGAIN && (req->flags & REQ_F_NOWAIT))
goto done;
- if (ret2 == -EIOCBQUEUED && req->async_data)
- iov_iter_revert(iter, io_size - iov_iter_count(iter));
if (!force_nonblock || ret2 != -EAGAIN) {
/* IOPOLL retry should happen for io-wq threads */
if ((req->ctx->flags & IORING_SETUP_IOPOLL) && ret2 == -EAGAIN)
}
i++;
kfree(buf);
- idr_remove(&ctx->io_buffer_idr, bgid);
+ xa_erase(&ctx->io_buffers, bgid);
return i;
}
lockdep_assert_held(&ctx->uring_lock);
ret = -ENOENT;
- head = idr_find(&ctx->io_buffer_idr, p->bgid);
+ head = xa_load(&ctx->io_buffers, p->bgid);
if (head)
ret = __io_remove_buffers(ctx, head, p->bgid, p->nbufs);
if (ret < 0)
static int io_provide_buffers_prep(struct io_kiocb *req,
const struct io_uring_sqe *sqe)
{
+ unsigned long size;
struct io_provide_buf *p = &req->pbuf;
u64 tmp;
p->addr = READ_ONCE(sqe->addr);
p->len = READ_ONCE(sqe->len);
- if (!access_ok(u64_to_user_ptr(p->addr), (p->len * p->nbufs)))
+ size = (unsigned long)p->len * p->nbufs;
+ if (!access_ok(u64_to_user_ptr(p->addr), size))
return -EFAULT;
p->bgid = READ_ONCE(sqe->buf_group);
lockdep_assert_held(&ctx->uring_lock);
- list = head = idr_find(&ctx->io_buffer_idr, p->bgid);
+ list = head = xa_load(&ctx->io_buffers, p->bgid);
ret = io_add_buffers(p, &head);
- if (ret < 0)
- goto out;
-
- if (!list) {
- ret = idr_alloc(&ctx->io_buffer_idr, head, p->bgid, p->bgid + 1,
- GFP_KERNEL);
- if (ret < 0) {
+ if (ret >= 0 && !list) {
+ ret = xa_insert(&ctx->io_buffers, p->bgid, head, GFP_KERNEL);
+ if (ret < 0)
__io_remove_buffers(ctx, head, p->bgid, -1U);
- goto out;
- }
}
-out:
if (ret < 0)
req_set_fail_links(req);
struct io_async_msghdr iomsg, *kmsg;
struct socket *sock;
unsigned flags;
+ int min_ret = 0;
int ret;
sock = sock_from_file(req->file);
kmsg = &iomsg;
}
- flags = req->sr_msg.msg_flags;
+ flags = req->sr_msg.msg_flags | MSG_NOSIGNAL;
if (flags & MSG_DONTWAIT)
req->flags |= REQ_F_NOWAIT;
else if (issue_flags & IO_URING_F_NONBLOCK)
flags |= MSG_DONTWAIT;
+ if (flags & MSG_WAITALL)
+ min_ret = iov_iter_count(&kmsg->msg.msg_iter);
+
ret = __sys_sendmsg_sock(sock, &kmsg->msg, flags);
if ((issue_flags & IO_URING_F_NONBLOCK) && ret == -EAGAIN)
return io_setup_async_msg(req, kmsg);
if (kmsg->free_iov)
kfree(kmsg->free_iov);
req->flags &= ~REQ_F_NEED_CLEANUP;
- if (ret < 0)
+ if (ret < min_ret)
req_set_fail_links(req);
__io_req_complete(req, issue_flags, ret, 0);
return 0;
struct iovec iov;
struct socket *sock;
unsigned flags;
+ int min_ret = 0;
int ret;
sock = sock_from_file(req->file);
msg.msg_controllen = 0;
msg.msg_namelen = 0;
- flags = req->sr_msg.msg_flags;
+ flags = req->sr_msg.msg_flags | MSG_NOSIGNAL;
if (flags & MSG_DONTWAIT)
req->flags |= REQ_F_NOWAIT;
else if (issue_flags & IO_URING_F_NONBLOCK)
flags |= MSG_DONTWAIT;
+ if (flags & MSG_WAITALL)
+ min_ret = iov_iter_count(&msg.msg_iter);
+
msg.msg_flags = flags;
ret = sock_sendmsg(sock, &msg);
if ((issue_flags & IO_URING_F_NONBLOCK) && ret == -EAGAIN)
if (ret == -ERESTARTSYS)
ret = -EINTR;
- if (ret < 0)
+ if (ret < min_ret)
req_set_fail_links(req);
__io_req_complete(req, issue_flags, ret, 0);
return 0;
struct socket *sock;
struct io_buffer *kbuf;
unsigned flags;
+ int min_ret = 0;
int ret, cflags = 0;
bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK;
1, req->sr_msg.len);
}
- flags = req->sr_msg.msg_flags;
+ flags = req->sr_msg.msg_flags | MSG_NOSIGNAL;
if (flags & MSG_DONTWAIT)
req->flags |= REQ_F_NOWAIT;
else if (force_nonblock)
flags |= MSG_DONTWAIT;
+ if (flags & MSG_WAITALL)
+ min_ret = iov_iter_count(&kmsg->msg.msg_iter);
+
ret = __sys_recvmsg_sock(sock, &kmsg->msg, req->sr_msg.umsg,
kmsg->uaddr, flags);
if (force_nonblock && ret == -EAGAIN)
if (kmsg->free_iov)
kfree(kmsg->free_iov);
req->flags &= ~REQ_F_NEED_CLEANUP;
- if (ret < 0)
+ if (ret < min_ret || ((flags & MSG_WAITALL) && (kmsg->msg.msg_flags & (MSG_TRUNC | MSG_CTRUNC))))
req_set_fail_links(req);
__io_req_complete(req, issue_flags, ret, cflags);
return 0;
struct socket *sock;
struct iovec iov;
unsigned flags;
+ int min_ret = 0;
int ret, cflags = 0;
bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK;
msg.msg_iocb = NULL;
msg.msg_flags = 0;
- flags = req->sr_msg.msg_flags;
+ flags = req->sr_msg.msg_flags | MSG_NOSIGNAL;
if (flags & MSG_DONTWAIT)
req->flags |= REQ_F_NOWAIT;
else if (force_nonblock)
flags |= MSG_DONTWAIT;
+ if (flags & MSG_WAITALL)
+ min_ret = iov_iter_count(&msg.msg_iter);
+
ret = sock_recvmsg(sock, &msg, flags);
if (force_nonblock && ret == -EAGAIN)
return -EAGAIN;
out_free:
if (req->flags & REQ_F_BUFFER_SELECTED)
cflags = io_put_recv_kbuf(req);
- if (ret < 0)
+ if (ret < min_ret || ((flags & MSG_WAITALL) && (msg.msg_flags & (MSG_TRUNC | MSG_CTRUNC))))
req_set_fail_links(req);
__io_req_complete(req, issue_flags, ret, cflags);
return 0;
ret = -ENOMEM;
goto out;
}
- io = req->async_data;
memcpy(req->async_data, &__io, sizeof(__io));
return -EAGAIN;
}
data->mode = io_translate_timeout_mode(flags);
hrtimer_init(&data->timer, CLOCK_MONOTONIC, data->mode);
- io_req_track_inflight(req);
+ if (is_timeout_link)
+ io_req_track_inflight(req);
return 0;
}
spin_unlock_irqrestore(&ctx->completion_lock, flags);
if (prev) {
- req_set_fail_links(prev);
io_async_find_and_cancel(ctx, req, prev->user_data, -ETIME);
io_put_req_deferred(prev, 1);
} else {
ret = io_init_req(ctx, req, sqe);
if (unlikely(ret)) {
fail_req:
- io_put_req(req);
- io_req_complete(req, ret);
if (link->head) {
/* fail even hard links since we don't submit */
link->head->flags |= REQ_F_FAIL_LINK;
io_req_complete(link->head, -ECANCELED);
link->head = NULL;
}
+ io_put_req(req);
+ io_req_complete(req, ret);
return ret;
}
ret = io_req_prep(req, sqe);
char buf[TASK_COMM_LEN];
DEFINE_WAIT(wait);
- sprintf(buf, "iou-sqp-%d", sqd->task_pid);
+ snprintf(buf, sizeof(buf), "iou-sqp-%d", sqd->task_pid);
set_task_comm(current, buf);
current->pf_io_worker = NULL;
set_cpus_allowed_ptr(current, cpu_online_mask);
current->flags |= PF_NO_SETAFFINITY;
- down_read(&sqd->rw_lock);
-
+ mutex_lock(&sqd->lock);
while (!test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state)) {
int ret;
bool cap_entries, sqt_spin, needs_sched;
- if (test_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state)) {
- up_read(&sqd->rw_lock);
+ if (test_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state) ||
+ signal_pending(current)) {
+ bool did_sig = false;
+
+ mutex_unlock(&sqd->lock);
+ if (signal_pending(current)) {
+ struct ksignal ksig;
+
+ did_sig = get_signal(&ksig);
+ }
cond_resched();
- down_read(&sqd->rw_lock);
+ mutex_lock(&sqd->lock);
+ if (did_sig)
+ break;
io_run_task_work();
+ io_run_task_work_head(&sqd->park_task_work);
timeout = jiffies + sqd->sq_thread_idle;
continue;
}
- if (fatal_signal_pending(current))
- break;
sqt_spin = false;
cap_entries = !list_is_singular(&sqd->ctx_list);
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) {
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list)
io_ring_set_wakeup_flag(ctx);
- up_read(&sqd->rw_lock);
+ mutex_unlock(&sqd->lock);
schedule();
- down_read(&sqd->rw_lock);
+ mutex_lock(&sqd->lock);
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list)
io_ring_clear_wakeup_flag(ctx);
}
finish_wait(&sqd->wait, &wait);
+ io_run_task_work_head(&sqd->park_task_work);
timeout = jiffies + sqd->sq_thread_idle;
}
- up_read(&sqd->rw_lock);
- down_write(&sqd->rw_lock);
- /*
- * someone may have parked and added a cancellation task_work, run
- * it first because we don't want it in io_uring_cancel_sqpoll()
- */
- io_run_task_work();
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list)
io_uring_cancel_sqpoll(ctx);
sqd->thread = NULL;
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list)
io_ring_set_wakeup_flag(ctx);
- up_write(&sqd->rw_lock);
+ mutex_unlock(&sqd->lock);
io_run_task_work();
+ io_run_task_work_head(&sqd->park_task_work);
complete(&sqd->exited);
do_exit(0);
}
return 1;
if (!signal_pending(current))
return 0;
- if (test_tsk_thread_flag(current, TIF_NOTIFY_SIGNAL))
+ if (test_thread_flag(TIF_NOTIFY_SIGNAL))
return -ERESTARTSYS;
return -EINTR;
}
}
static void io_sq_thread_unpark(struct io_sq_data *sqd)
- __releases(&sqd->rw_lock)
+ __releases(&sqd->lock)
{
WARN_ON_ONCE(sqd->thread == current);
+ /*
+ * Do the dance but not conditional clear_bit() because it'd race with
+ * other threads incrementing park_pending and setting the bit.
+ */
clear_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state);
- up_write(&sqd->rw_lock);
+ if (atomic_dec_return(&sqd->park_pending))
+ set_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state);
+ mutex_unlock(&sqd->lock);
}
static void io_sq_thread_park(struct io_sq_data *sqd)
- __acquires(&sqd->rw_lock)
+ __acquires(&sqd->lock)
{
WARN_ON_ONCE(sqd->thread == current);
+ atomic_inc(&sqd->park_pending);
set_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state);
- down_write(&sqd->rw_lock);
- /* set again for consistency, in case concurrent parks are happening */
- set_bit(IO_SQ_THREAD_SHOULD_PARK, &sqd->state);
+ mutex_lock(&sqd->lock);
if (sqd->thread)
wake_up_process(sqd->thread);
}
{
WARN_ON_ONCE(sqd->thread == current);
- down_write(&sqd->rw_lock);
+ mutex_lock(&sqd->lock);
set_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state);
if (sqd->thread)
wake_up_process(sqd->thread);
- up_write(&sqd->rw_lock);
+ mutex_unlock(&sqd->lock);
wait_for_completion(&sqd->exited);
}
static void io_put_sq_data(struct io_sq_data *sqd)
{
if (refcount_dec_and_test(&sqd->refs)) {
+ WARN_ON_ONCE(atomic_read(&sqd->park_pending));
+
io_sq_thread_stop(sqd);
kfree(sqd);
}
if (!sqd)
return ERR_PTR(-ENOMEM);
+ atomic_set(&sqd->park_pending, 0);
refcount_set(&sqd->refs, 1);
INIT_LIST_HEAD(&sqd->ctx_list);
- init_rwsem(&sqd->rw_lock);
+ mutex_init(&sqd->lock);
init_waitqueue_head(&sqd->wait);
init_completion(&sqd->exited);
return sqd;
ret = 0;
io_sq_thread_park(sqd);
+ list_add(&ctx->sqd_list, &sqd->ctx_list);
+ io_sqd_update_thread_idle(sqd);
/* don't attach to a dying SQPOLL thread, would be racy */
- if (attached && !sqd->thread) {
+ if (attached && !sqd->thread)
ret = -ENXIO;
- } else {
- list_add(&ctx->sqd_list, &sqd->ctx_list);
- io_sqd_update_thread_idle(sqd);
- }
io_sq_thread_unpark(sqd);
- if (ret < 0) {
- io_put_sq_data(sqd);
- ctx->sq_data = NULL;
- return ret;
- } else if (attached) {
+ if (ret < 0)
+ goto err;
+ if (attached)
return 0;
- }
if (p->flags & IORING_SETUP_SQ_AFF) {
int cpu = p->sq_thread_cpu;
return -ENXIO;
}
-static int __io_destroy_buffers(int id, void *p, void *data)
-{
- struct io_ring_ctx *ctx = data;
- struct io_buffer *buf = p;
-
- __io_remove_buffers(ctx, buf, id, -1U);
- return 0;
-}
-
static void io_destroy_buffers(struct io_ring_ctx *ctx)
{
- idr_for_each(&ctx->io_buffer_idr, __io_destroy_buffers, ctx);
- idr_destroy(&ctx->io_buffer_idr);
+ struct io_buffer *buf;
+ unsigned long index;
+
+ xa_for_each(&ctx->io_buffers, index, buf)
+ __io_remove_buffers(ctx, buf, index, -1U);
}
static void io_req_cache_free(struct list_head *list, struct task_struct *tsk)
{
/*
* Some may use context even when all refs and requests have been put,
- * and they are free to do so while still holding uring_lock, see
- * __io_req_task_submit(). Wait for them to finish.
+ * and they are free to do so while still holding uring_lock or
+ * completion_lock, see __io_req_task_submit(). Wait for them to finish.
*/
mutex_lock(&ctx->uring_lock);
mutex_unlock(&ctx->uring_lock);
+ spin_lock_irq(&ctx->completion_lock);
+ spin_unlock_irq(&ctx->completion_lock);
io_sq_thread_finish(ctx);
io_sqe_buffers_unregister(ctx);
return -EINVAL;
}
-static bool io_run_ctx_fallback(struct io_ring_ctx *ctx)
+static inline bool io_run_ctx_fallback(struct io_ring_ctx *ctx)
{
- struct callback_head *work, *next;
- bool executed = false;
-
- do {
- work = xchg(&ctx->exit_task_work, NULL);
- if (!work)
- break;
-
- do {
- next = work->next;
- work->func(work);
- work = next;
- cond_resched();
- } while (work);
- executed = true;
- } while (1);
-
- return executed;
+ return io_run_task_work_head(&ctx->exit_task_work);
}
struct io_tctx_exit {
struct io_tctx_node *node;
int ret;
+ /* prevent SQPOLL from submitting new requests */
+ if (ctx->sq_data) {
+ io_sq_thread_park(ctx->sq_data);
+ list_del_init(&ctx->sqd_list);
+ io_sqd_update_thread_idle(ctx->sq_data);
+ io_sq_thread_unpark(ctx->sq_data);
+ }
+
/*
* If we're doing polled IO and end up having requests being
* submitted async (out-of-line), then completions can come in while
io_ring_ctx_free(ctx);
}
+/* Returns true if we found and killed one or more timeouts */
+static bool io_kill_timeouts(struct io_ring_ctx *ctx, struct task_struct *tsk,
+ struct files_struct *files)
+{
+ struct io_kiocb *req, *tmp;
+ int canceled = 0;
+
+ spin_lock_irq(&ctx->completion_lock);
+ list_for_each_entry_safe(req, tmp, &ctx->timeout_list, timeout.list) {
+ if (io_match_task(req, tsk, files)) {
+ io_kill_timeout(req, -ECANCELED);
+ canceled++;
+ }
+ }
+ if (canceled != 0)
+ io_commit_cqring(ctx);
+ spin_unlock_irq(&ctx->completion_lock);
+ if (canceled != 0)
+ io_cqring_ev_posted(ctx);
+ return canceled != 0;
+}
+
static void io_ring_ctx_wait_and_kill(struct io_ring_ctx *ctx)
{
unsigned long index;
if (task) {
init_completion(&work.completion);
init_task_work(&work.task_work, io_sqpoll_cancel_cb);
- WARN_ON_ONCE(task_work_add(task, &work.task_work, TWA_SIGNAL));
+ io_task_work_add_head(&sqd->park_task_work, &work.task_work);
wake_up_process(task);
}
io_sq_thread_unpark(sqd);
/* make sure overflow events are dropped */
atomic_inc(&tctx->in_idle);
+ __io_uring_files_cancel(NULL);
+
do {
/* read completions before cancelations */
inflight = tctx_inflight(tctx);
return ret;
}
+ /*
+ * If this swapfile doesn't contain even a single page-aligned
+ * contiguous range of blocks, reject this useless swapfile to
+ * prevent confusion later on.
+ */
+ if (isi.nr_pages == 0) {
+ pr_warn("swapon: Cannot find a single usable page in file.\n");
+ return -EINVAL;
+ }
+
*pagespan = 1 + isi.highest_ppage - isi.lowest_ppage;
sis->max = isi.nr_pages;
sis->pages = isi.nr_pages - 1;
if (flags & FL_LAYOUT)
return 0;
- if (flags & FL_DELEG)
- /* We leave these checks to the caller. */
- return 0;
if (arg == F_RDLCK)
return inode_is_open_for_write(inode) ? -EAGAIN : 0;
p->stack = p->internal;
p->dfd = dfd;
p->name = name;
+ p->path.mnt = NULL;
+ p->path.dentry = NULL;
p->total_link_count = old ? old->total_link_count : 0;
p->saved = old;
current->nameidata = p;
rcu_read_unlock();
}
nd->depth = 0;
+ nd->path.mnt = NULL;
+ nd->path.dentry = NULL;
}
/* path_put is needed afterwards regardless of success or failure */
}
nd->root.mnt = NULL;
- nd->path.mnt = NULL;
- nd->path.dentry = NULL;
/* Absolute pathname -- fetch the root (LOOKUP_IN_ROOT uses nd->dfd). */
if (*s == '/' && !(flags & LOOKUP_IN_ROOT)) {
while (!(err = link_path_walk(s, nd)) &&
(s = lookup_last(nd)) != NULL)
;
+ if (!err && unlikely(nd->flags & LOOKUP_MOUNTPOINT)) {
+ err = handle_lookup_down(nd);
+ nd->flags &= ~LOOKUP_JUMPED; // no d_weak_revalidate(), please...
+ }
if (!err)
err = complete_walk(nd);
if (!err && nd->flags & LOOKUP_DIRECTORY)
if (!d_can_lookup(nd->path.dentry))
err = -ENOTDIR;
- if (!err && unlikely(nd->flags & LOOKUP_MOUNTPOINT)) {
- err = handle_lookup_down(nd);
- nd->flags &= ~LOOKUP_JUMPED; // no d_weak_revalidate(), please...
- }
if (!err) {
*path = nd->path;
nd->path.mnt = NULL;
select NFSD_V3
select FS_POSIX_ACL
select SUNRPC_GSS
+ select CRYPTO
select CRYPTO_MD5
select CRYPTO_SHA256
select GRACE_PERIOD
continue;
if (!nfsd_match_cred(nf->nf_cred, current_cred()))
continue;
+ if (!test_bit(NFSD_FILE_HASHED, &nf->nf_flags))
+ continue;
if (nfsd_file_get(nf) != NULL)
return nf;
}
switch (task->tk_status) {
case -EIO:
case -ETIMEDOUT:
+ case -EACCES:
nfsd4_mark_cb_down(clp, task->tk_status);
}
break;
struct nfsd_file *dst)
{
nfs42_ssc_close(src->nf_file);
- /* 'src' is freed by nfsd4_do_async_copy */
+ fput(src->nf_file);
nfsd_file_put(dst);
mntput(ss_mnt);
}
return fl;
}
-static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
- struct nfs4_file *fp)
-{
- struct nfs4_clnt_odstate *co;
- struct file *f = fp->fi_deleg_file->nf_file;
- struct inode *ino = locks_inode(f);
- int writes = atomic_read(&ino->i_writecount);
-
- if (fp->fi_fds[O_WRONLY])
- writes--;
- if (fp->fi_fds[O_RDWR])
- writes--;
- if (writes > 0)
- return -EAGAIN;
- spin_lock(&fp->fi_lock);
- list_for_each_entry(co, &fp->fi_clnt_odstate, co_perfile) {
- if (co->co_client != clp) {
- spin_unlock(&fp->fi_lock);
- return -EAGAIN;
- }
- }
- spin_unlock(&fp->fi_lock);
- return 0;
-}
-
static struct nfs4_delegation *
nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh,
struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
nf = find_readable_file(fp);
if (!nf) {
- /*
- * We probably could attempt another open and get a read
- * delegation, but for now, don't bother until the
- * client actually sends us one.
- */
- return ERR_PTR(-EAGAIN);
+ /* We should always have a readable file here */
+ WARN_ON_ONCE(1);
+ return ERR_PTR(-EBADF);
}
spin_lock(&state_lock);
spin_lock(&fp->fi_lock);
if (!fl)
goto out_clnt_odstate;
- status = nfsd4_check_conflicting_opens(clp, fp);
- if (status) {
- locks_free_lock(fl);
- goto out_clnt_odstate;
- }
status = vfs_setlease(fp->fi_deleg_file->nf_file, fl->fl_type, &fl, NULL);
if (fl)
locks_free_lock(fl);
if (status)
goto out_clnt_odstate;
- status = nfsd4_check_conflicting_opens(clp, fp);
- if (status)
- goto out_clnt_odstate;
spin_lock(&state_lock);
spin_lock(&fp->fi_lock);
goto out_no_deleg;
if (!cb_up || !(oo->oo_flags & NFS4_OO_CONFIRMED))
goto out_no_deleg;
+ /*
+ * Also, if the file was opened for write or
+ * create, there's a good chance the client's
+ * about to write to it, resulting in an
+ * immediate recall (since we don't support
+ * write delegations):
+ */
+ if (open->op_share_access & NFS4_SHARE_ACCESS_WRITE)
+ goto out_no_deleg;
+ if (open->op_create == NFS4_OPEN_CREATE)
+ goto out_no_deleg;
break;
default:
goto out_no_deleg;
idr_for_each_entry(&nn->s2s_cp_stateids, cps_t, i) {
cps = container_of(cps_t, struct nfs4_cpntf_state, cp_stateid);
if (cps->cp_stateid.sc_type == NFS4_COPYNOTIFY_STID &&
- cps->cpntf_time > cutoff)
+ cps->cpntf_time < cutoff)
_free_cpntf_state_locked(nn, cps);
}
spin_unlock(&nn->s2s_cp_lock);
struct ocfs2_alloc_context *meta_ac = NULL;
handle_t *handle = NULL;
loff_t end = offset + bytes;
- int ret = 0, credits = 0, locked = 0;
+ int ret = 0, credits = 0;
ocfs2_init_dealloc_ctxt(&dealloc);
!dwc->dw_orphaned)
goto out;
- /* ocfs2_file_write_iter will get i_mutex, so we need not lock if we
- * are in that context. */
- if (dwc->dw_writer_pid != task_pid_nr(current)) {
- inode_lock(inode);
- locked = 1;
- }
-
ret = ocfs2_inode_lock(inode, &di_bh, 1);
if (ret < 0) {
mlog_errno(ret);
if (meta_ac)
ocfs2_free_alloc_context(meta_ac);
ocfs2_run_deallocs(osb, &dealloc);
- if (locked)
- inode_unlock(inode);
ocfs2_dio_free_write_ctx(inode, dwc);
return ret;
goto bail_unlock;
}
}
+ down_write(&OCFS2_I(inode)->ip_alloc_sem);
handle = ocfs2_start_trans(osb, OCFS2_INODE_UPDATE_CREDITS +
2 * ocfs2_quota_trans_credits(sb));
if (IS_ERR(handle)) {
status = PTR_ERR(handle);
mlog_errno(status);
- goto bail_unlock;
+ goto bail_unlock_alloc;
}
status = __dquot_transfer(inode, transfer_to);
if (status < 0)
goto bail_commit;
} else {
+ down_write(&OCFS2_I(inode)->ip_alloc_sem);
handle = ocfs2_start_trans(osb, OCFS2_INODE_UPDATE_CREDITS);
if (IS_ERR(handle)) {
status = PTR_ERR(handle);
mlog_errno(status);
- goto bail_unlock;
+ goto bail_unlock_alloc;
}
}
bail_commit:
ocfs2_commit_trans(osb, handle);
+bail_unlock_alloc:
+ up_write(&OCFS2_I(inode)->ip_alloc_sem);
bail_unlock:
if (status && inode_locked) {
ocfs2_inode_unlock_tracker(inode, 1, &oh, had_lock);
static inline int reiserfs_xattrs_initialized(struct super_block *sb)
{
- return REISERFS_SB(sb)->priv_root != NULL;
+ return REISERFS_SB(sb)->priv_root && REISERFS_SB(sb)->xattr_root;
}
#define xattr_size(size) ((size) + sizeof(struct reiserfs_xattr_header))
ret = do_sys_poll(ufds, nfds, to);
- if (ret == -ERESTARTNOHAND) {
- restart_block->fn = do_restart_poll;
- ret = -ERESTART_RESTARTBLOCK;
- }
+ if (ret == -ERESTARTNOHAND)
+ ret = set_restart_fn(restart_block, do_restart_poll);
+
return ret;
}
struct restart_block *restart_block;
restart_block = ¤t->restart_block;
- restart_block->fn = do_restart_poll;
restart_block->poll.ufds = ufds;
restart_block->poll.nfds = nfds;
} else
restart_block->poll.has_timeout = 0;
- ret = -ERESTART_RESTARTBLOCK;
+ ret = set_restart_fn(restart_block, do_restart_poll);
}
return ret;
}
start = le64_to_cpu(table[n]);
end = le64_to_cpu(table[n + 1]);
- if (start >= end || (end - start) > SQUASHFS_METADATA_SIZE) {
+ if (start >= end
+ || (end - start) >
+ (SQUASHFS_METADATA_SIZE + SQUASHFS_BLOCK_OFFSET)) {
kfree(table);
return ERR_PTR(-EINVAL);
}
}
start = le64_to_cpu(table[indexes - 1]);
- if (start >= lookup_table_start || (lookup_table_start - start) > SQUASHFS_METADATA_SIZE) {
+ if (start >= lookup_table_start ||
+ (lookup_table_start - start) >
+ (SQUASHFS_METADATA_SIZE + SQUASHFS_BLOCK_OFFSET)) {
kfree(table);
return ERR_PTR(-EINVAL);
}
start = le64_to_cpu(table[n]);
end = le64_to_cpu(table[n + 1]);
- if (start >= end || (end - start) > SQUASHFS_METADATA_SIZE) {
+ if (start >= end || (end - start) >
+ (SQUASHFS_METADATA_SIZE + SQUASHFS_BLOCK_OFFSET)) {
kfree(table);
return ERR_PTR(-EINVAL);
}
}
start = le64_to_cpu(table[indexes - 1]);
- if (start >= id_table_start || (id_table_start - start) > SQUASHFS_METADATA_SIZE) {
+ if (start >= id_table_start || (id_table_start - start) >
+ (SQUASHFS_METADATA_SIZE + SQUASHFS_BLOCK_OFFSET)) {
kfree(table);
return ERR_PTR(-EINVAL);
}
/* size of metadata (inode and directory) blocks */
#define SQUASHFS_METADATA_SIZE 8192
+#define SQUASHFS_BLOCK_OFFSET 2
/* default size of block device I/O */
#ifdef CONFIG_SQUASHFS_4K_DEVBLK_SIZE
start = le64_to_cpu(table[n]);
end = le64_to_cpu(table[n + 1]);
- if (start >= end || (end - start) > SQUASHFS_METADATA_SIZE) {
+ if (start >= end || (end - start) >
+ (SQUASHFS_METADATA_SIZE + SQUASHFS_BLOCK_OFFSET)) {
kfree(table);
return ERR_PTR(-EINVAL);
}
}
start = le64_to_cpu(table[indexes - 1]);
- if (start >= table_start || (table_start - start) > SQUASHFS_METADATA_SIZE) {
+ if (start >= table_start || (table_start - start) >
+ (SQUASHFS_METADATA_SIZE + SQUASHFS_BLOCK_OFFSET)) {
kfree(table);
return ERR_PTR(-EINVAL);
}
/*
* Make sure that we have allocated dquot(s) on disk.
*/
- error = xfs_qm_vop_dqalloc(dp, current_fsuid(), current_fsgid(), prid,
- XFS_QMOPT_QUOTALL | XFS_QMOPT_INHERIT,
- &udqp, &gdqp, &pdqp);
+ error = xfs_qm_vop_dqalloc(dp, fsuid_into_mnt(mnt_userns),
+ fsgid_into_mnt(mnt_userns), prid,
+ XFS_QMOPT_QUOTALL | XFS_QMOPT_INHERIT,
+ &udqp, &gdqp, &pdqp);
if (error)
return error;
/*
* Make sure that we have allocated dquot(s) on disk.
*/
- error = xfs_qm_vop_dqalloc(dp, current_fsuid(), current_fsgid(), prid,
- XFS_QMOPT_QUOTALL | XFS_QMOPT_INHERIT,
- &udqp, &gdqp, &pdqp);
+ error = xfs_qm_vop_dqalloc(dp, fsuid_into_mnt(mnt_userns),
+ fsgid_into_mnt(mnt_userns), prid,
+ XFS_QMOPT_QUOTALL | XFS_QMOPT_INHERIT,
+ &udqp, &gdqp, &pdqp);
if (error)
return error;
};
int error;
+ if (breq->mnt_userns != &init_user_ns) {
+ xfs_warn_ratelimited(breq->mp,
+ "bulkstat not supported inside of idmapped mounts.");
+ return -EINVAL;
+ }
+
ASSERT(breq->icount == 1);
bc.buf = kmem_zalloc(sizeof(struct xfs_bulkstat),
return xfs_initialize_perag_data(mp, mp->m_sb.sb_agcount);
}
+/*
+ * Flush and reclaim dirty inodes in preparation for unmount. Inodes and
+ * internal inode structures can be sitting in the CIL and AIL at this point,
+ * so we need to unpin them, write them back and/or reclaim them before unmount
+ * can proceed.
+ *
+ * An inode cluster that has been freed can have its buffer still pinned in
+ * memory because the transaction is still sitting in a iclog. The stale inodes
+ * on that buffer will be pinned to the buffer until the transaction hits the
+ * disk and the callbacks run. Pushing the AIL will skip the stale inodes and
+ * may never see the pinned buffer, so nothing will push out the iclog and
+ * unpin the buffer.
+ *
+ * Hence we need to force the log to unpin everything first. However, log
+ * forces don't wait for the discards they issue to complete, so we have to
+ * explicitly wait for them to complete here as well.
+ *
+ * Then we can tell the world we are unmounting so that error handling knows
+ * that the filesystem is going away and we should error out anything that we
+ * have been retrying in the background. This will prevent never-ending
+ * retries in AIL pushing from hanging the unmount.
+ *
+ * Finally, we can push the AIL to clean all the remaining dirty objects, then
+ * reclaim the remaining inodes that are still in memory at this point in time.
+ */
+static void
+xfs_unmount_flush_inodes(
+ struct xfs_mount *mp)
+{
+ xfs_log_force(mp, XFS_LOG_SYNC);
+ xfs_extent_busy_wait_all(mp);
+ flush_workqueue(xfs_discard_wq);
+
+ mp->m_flags |= XFS_MOUNT_UNMOUNTING;
+
+ xfs_ail_push_all_sync(mp->m_ail);
+ cancel_delayed_work_sync(&mp->m_reclaim_work);
+ xfs_reclaim_inodes(mp);
+ xfs_health_unmount(mp);
+}
+
/*
* This function does the following on an initial mount of a file system:
* - reads the superblock from disk and init the mount struct
/* Clean out dquots that might be in memory after quotacheck. */
xfs_qm_unmount(mp);
/*
- * Cancel all delayed reclaim work and reclaim the inodes directly.
+ * Flush all inode reclamation work and flush the log.
* We have to do this /after/ rtunmount and qm_unmount because those
* two will have scheduled delayed reclaim for the rt/quota inodes.
*
* qm_unmount_quotas and therefore rely on qm_unmount to release the
* quota inodes.
*/
- cancel_delayed_work_sync(&mp->m_reclaim_work);
- xfs_reclaim_inodes(mp);
- xfs_health_unmount(mp);
+ xfs_unmount_flush_inodes(mp);
out_log_dealloc:
- mp->m_flags |= XFS_MOUNT_UNMOUNTING;
xfs_log_mount_cancel(mp);
out_fail_wait:
if (mp->m_logdev_targp && mp->m_logdev_targp != mp->m_ddev_targp)
xfs_rtunmount_inodes(mp);
xfs_irele(mp->m_rootip);
- /*
- * We can potentially deadlock here if we have an inode cluster
- * that has been freed has its buffer still pinned in memory because
- * the transaction is still sitting in a iclog. The stale inodes
- * on that buffer will be pinned to the buffer until the
- * transaction hits the disk and the callbacks run. Pushing the AIL will
- * skip the stale inodes and may never see the pinned buffer, so
- * nothing will push out the iclog and unpin the buffer. Hence we
- * need to force the log here to ensure all items are flushed into the
- * AIL before we go any further.
- */
- xfs_log_force(mp, XFS_LOG_SYNC);
-
- /*
- * Wait for all busy extents to be freed, including completion of
- * any discard operation.
- */
- xfs_extent_busy_wait_all(mp);
- flush_workqueue(xfs_discard_wq);
-
- /*
- * We now need to tell the world we are unmounting. This will allow
- * us to detect that the filesystem is going away and we should error
- * out anything that we have been retrying in the background. This will
- * prevent neverending retries in AIL pushing from hanging the unmount.
- */
- mp->m_flags |= XFS_MOUNT_UNMOUNTING;
-
- /*
- * Flush all pending changes from the AIL.
- */
- xfs_ail_push_all_sync(mp->m_ail);
-
- /*
- * Reclaim all inodes. At this point there should be no dirty inodes and
- * none should be pinned or locked. Stop background inode reclaim here
- * if it is still running.
- */
- cancel_delayed_work_sync(&mp->m_reclaim_work);
- xfs_reclaim_inodes(mp);
- xfs_health_unmount(mp);
+ xfs_unmount_flush_inodes(mp);
xfs_qm_unmount(mp);
/*
* Make sure that we have allocated dquot(s) on disk.
*/
- error = xfs_qm_vop_dqalloc(dp, current_fsuid(), current_fsgid(), prid,
+ error = xfs_qm_vop_dqalloc(dp, fsuid_into_mnt(mnt_userns),
+ fsgid_into_mnt(mnt_userns), prid,
XFS_QMOPT_QUOTALL | XFS_QMOPT_INHERIT,
&udqp, &gdqp, &pdqp);
if (error)
return iomap_writepages(mapping, wbc, &wpc, &zonefs_writeback_ops);
}
+static int zonefs_swap_activate(struct swap_info_struct *sis,
+ struct file *swap_file, sector_t *span)
+{
+ struct inode *inode = file_inode(swap_file);
+ struct zonefs_inode_info *zi = ZONEFS_I(inode);
+
+ if (zi->i_ztype != ZONEFS_ZTYPE_CNV) {
+ zonefs_err(inode->i_sb,
+ "swap file: not a conventional zone file\n");
+ return -EINVAL;
+ }
+
+ return iomap_swapfile_activate(sis, swap_file, span, &zonefs_iomap_ops);
+}
+
static const struct address_space_operations zonefs_file_aops = {
.readpage = zonefs_readpage,
.readahead = zonefs_readahead,
.is_partially_uptodate = iomap_is_partially_uptodate,
.error_remove_page = generic_error_remove_page,
.direct_IO = noop_direct_IO,
+ .swap_activate = zonefs_swap_activate,
};
static void zonefs_update_stats(struct inode *inode, loff_t new_isize)
return ret;
}
+/*
+ * Do not exceed the LFS limits nor the file zone size. If pos is under the
+ * limit it becomes a short access. If it exceeds the limit, return -EFBIG.
+ */
+static loff_t zonefs_write_check_limits(struct file *file, loff_t pos,
+ loff_t count)
+{
+ struct inode *inode = file_inode(file);
+ struct zonefs_inode_info *zi = ZONEFS_I(inode);
+ loff_t limit = rlimit(RLIMIT_FSIZE);
+ loff_t max_size = zi->i_max_size;
+
+ if (limit != RLIM_INFINITY) {
+ if (pos >= limit) {
+ send_sig(SIGXFSZ, current, 0);
+ return -EFBIG;
+ }
+ count = min(count, limit - pos);
+ }
+
+ if (!(file->f_flags & O_LARGEFILE))
+ max_size = min_t(loff_t, MAX_NON_LFS, max_size);
+
+ if (unlikely(pos >= max_size))
+ return -EFBIG;
+
+ return min(count, max_size - pos);
+}
+
+static ssize_t zonefs_write_checks(struct kiocb *iocb, struct iov_iter *from)
+{
+ struct file *file = iocb->ki_filp;
+ struct inode *inode = file_inode(file);
+ struct zonefs_inode_info *zi = ZONEFS_I(inode);
+ loff_t count;
+
+ if (IS_SWAPFILE(inode))
+ return -ETXTBSY;
+
+ if (!iov_iter_count(from))
+ return 0;
+
+ if ((iocb->ki_flags & IOCB_NOWAIT) && !(iocb->ki_flags & IOCB_DIRECT))
+ return -EINVAL;
+
+ if (iocb->ki_flags & IOCB_APPEND) {
+ if (zi->i_ztype != ZONEFS_ZTYPE_SEQ)
+ return -EINVAL;
+ mutex_lock(&zi->i_truncate_mutex);
+ iocb->ki_pos = zi->i_wpoffset;
+ mutex_unlock(&zi->i_truncate_mutex);
+ }
+
+ count = zonefs_write_check_limits(file, iocb->ki_pos,
+ iov_iter_count(from));
+ if (count < 0)
+ return count;
+
+ iov_iter_truncate(from, count);
+ return iov_iter_count(from);
+}
+
/*
* Handle direct writes. For sequential zone files, this is the only possible
* write path. For these files, check that the user is issuing writes
struct super_block *sb = inode->i_sb;
bool sync = is_sync_kiocb(iocb);
bool append = false;
- size_t count;
- ssize_t ret;
+ ssize_t ret, count;
/*
* For async direct IOs to sequential zone files, refuse IOCB_NOWAIT
inode_lock(inode);
}
- ret = generic_write_checks(iocb, from);
- if (ret <= 0)
+ count = zonefs_write_checks(iocb, from);
+ if (count <= 0) {
+ ret = count;
goto inode_unlock;
-
- iov_iter_truncate(from, zi->i_max_size - iocb->ki_pos);
- count = iov_iter_count(from);
+ }
if ((iocb->ki_pos | count) & (sb->s_blocksize - 1)) {
ret = -EINVAL;
inode_lock(inode);
}
- ret = generic_write_checks(iocb, from);
+ ret = zonefs_write_checks(iocb, from);
if (ret <= 0)
goto inode_unlock;
- iov_iter_truncate(from, zi->i_max_size - iocb->ki_pos);
-
ret = iomap_file_buffered_write(iocb, from, &zonefs_iomap_ops);
if (ret > 0)
iocb->ki_pos += ret;
mutex_lock(&zi->i_truncate_mutex);
- zi->i_wr_refcnt++;
- if (zi->i_wr_refcnt == 1) {
-
+ if (!zi->i_wr_refcnt) {
if (atomic_inc_return(&sbi->s_open_zones) > sbi->s_max_open_zones) {
atomic_dec(&sbi->s_open_zones);
ret = -EBUSY;
if (i_size_read(inode) < zi->i_max_size) {
ret = zonefs_zone_mgmt(inode, REQ_OP_ZONE_OPEN);
if (ret) {
- zi->i_wr_refcnt--;
atomic_dec(&sbi->s_open_zones);
goto unlock;
}
}
}
+ zi->i_wr_refcnt++;
+
unlock:
mutex_unlock(&zi->i_truncate_mutex);
struct acpi_device_pnp {
acpi_bus_id bus_id; /* Object name */
+ int instance_no; /* Instance number of this object */
struct acpi_pnp_type type; /* ID type */
acpi_bus_address bus_address; /* _ADR */
char *unique_id; /* _UID */
* @crc_count: counter of captured frame CRCs
* @transfer: transfers a message representing a single AUX transaction
*
- * The .dev field should be set to a pointer to the device that implements
- * the AUX channel.
+ * The @dev field should be set to a pointer to the device that implements the
+ * AUX channel.
*
- * The .name field may be used to specify the name of the I2C adapter. If set to
- * NULL, dev_name() of .dev will be used.
+ * The @name field may be used to specify the name of the I2C adapter. If set to
+ * %NULL, dev_name() of @dev will be used.
*
- * Drivers provide a hardware-specific implementation of how transactions
- * are executed via the .transfer() function. A pointer to a drm_dp_aux_msg
+ * Drivers provide a hardware-specific implementation of how transactions are
+ * executed via the @transfer() function. A pointer to a &drm_dp_aux_msg
* structure describing the transaction is passed into this function. Upon
- * success, the implementation should return the number of payload bytes
- * that were transferred, or a negative error-code on failure. Helpers
- * propagate errors from the .transfer() function, with the exception of
- * the -EBUSY error, which causes a transaction to be retried. On a short,
- * helpers will return -EPROTO to make it simpler to check for failure.
+ * success, the implementation should return the number of payload bytes that
+ * were transferred, or a negative error-code on failure. Helpers propagate
+ * errors from the @transfer() function, with the exception of the %-EBUSY
+ * error, which causes a transaction to be retried. On a short, helpers will
+ * return %-EPROTO to make it simpler to check for failure.
*
* An AUX channel can also be used to transport I2C messages to a sink. A
- * typical application of that is to access an EDID that's present in the
- * sink device. The .transfer() function can also be used to execute such
- * transactions. The drm_dp_aux_register() function registers an I2C
- * adapter that can be passed to drm_probe_ddc(). Upon removal, drivers
- * should call drm_dp_aux_unregister() to remove the I2C adapter.
- * The I2C adapter uses long transfers by default; if a partial response is
- * received, the adapter will drop down to the size given by the partial
- * response for this transaction only.
+ * typical application of that is to access an EDID that's present in the sink
+ * device. The @transfer() function can also be used to execute such
+ * transactions. The drm_dp_aux_register() function registers an I2C adapter
+ * that can be passed to drm_probe_ddc(). Upon removal, drivers should call
+ * drm_dp_aux_unregister() to remove the I2C adapter. The I2C adapter uses long
+ * transfers by default; if a partial response is received, the adapter will
+ * drop down to the size given by the partial response for this transaction
+ * only.
*
- * Note that the aux helper code assumes that the .transfer() function
- * only modifies the reply field of the drm_dp_aux_msg structure. The
- * retry logic and i2c helpers assume this is the case.
+ * Note that the aux helper code assumes that the @transfer() function only
+ * modifies the reply field of the &drm_dp_aux_msg structure. The retry logic
+ * and i2c helpers assume this is the case.
*/
struct drm_dp_aux {
const char *name;
#define DRM_DEBUG_DP(fmt, ...) \
__drm_dbg(DRM_UT_DP, fmt, ## __VA_ARGS__)
-
-#define DRM_DEBUG_KMS_RATELIMITED(fmt, ...) \
-({ \
- static DEFINE_RATELIMIT_STATE(_rs, \
- DEFAULT_RATELIMIT_INTERVAL, \
- DEFAULT_RATELIMIT_BURST); \
- if (__ratelimit(&_rs)) \
- drm_dev_dbg(NULL, DRM_UT_KMS, fmt, ##__VA_ARGS__); \
+#define __DRM_DEFINE_DBG_RATELIMITED(category, drm, fmt, ...) \
+({ \
+ static DEFINE_RATELIMIT_STATE(rs_, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST);\
+ const struct drm_device *drm_ = (drm); \
+ \
+ if (drm_debug_enabled(DRM_UT_ ## category) && __ratelimit(&rs_)) \
+ drm_dev_printk(drm_ ? drm_->dev : NULL, KERN_DEBUG, fmt, ## __VA_ARGS__); \
})
+#define drm_dbg_kms_ratelimited(drm, fmt, ...) \
+ __DRM_DEFINE_DBG_RATELIMITED(KMS, drm, fmt, ## __VA_ARGS__)
+
+#define DRM_DEBUG_KMS_RATELIMITED(fmt, ...) drm_dbg_kms_ratelimited(NULL, fmt, ## __VA_ARGS__)
+
/*
* struct drm_device based WARNs
*
* @hang_limit: once the hangs by a job crosses this limit then it is marked
* guilty and it will be considered for scheduling further.
* @score: score to help loadbalancer pick a idle sched
+ * @_score: score used when the driver doesn't provide one
* @ready: marks if the underlying HW is ready to work
* @free_guilty: A hit to time out handler to free the guilty job.
*
void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad);
void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery);
void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched);
+void drm_sched_resubmit_jobs_ext(struct drm_gpu_scheduler *sched, int max);
void drm_sched_increase_karma(struct drm_sched_job *bad);
+void drm_sched_reset_karma(struct drm_sched_job *bad);
+void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type);
bool drm_sched_dependency_optimized(struct dma_fence* fence,
struct drm_sched_entity *entity);
void drm_sched_fault(struct drm_gpu_scheduler *sched);
static inline void ttm_bo_unpin(struct ttm_buffer_object *bo)
{
dma_resv_assert_held(bo->base.resv);
- WARN_ON_ONCE(!bo->pin_count);
WARN_ON_ONCE(!kref_read(&bo->kref));
- --bo->pin_count;
+ if (bo->pin_count)
+ --bo->pin_count;
+ else
+ WARN_ON_ONCE(true);
}
int ttm_mem_evict_first(struct ttm_device *bdev,
/*
* Constant for device tree bindings for Turris Mox module configuration bus
*
- * Copyright (C) 2019 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2019 Marek Behún <kabel@kernel.org>
*/
#ifndef _DT_BINDINGS_BUS_MOXTET_H
void __acpi_unmap_table(void __iomem *map, unsigned long size);
int early_acpi_boot_init(void);
int acpi_boot_init (void);
+void acpi_boot_table_prepare (void);
void acpi_boot_table_init (void);
int acpi_mps_check (void);
int acpi_numa_init (void);
+int acpi_locate_initial_tables (void);
+void acpi_reserve_initial_tables (void);
+void acpi_table_init_complete (void);
int acpi_table_init (void);
int acpi_table_parse(char *id, acpi_tbl_table_handler handler);
int __init acpi_table_parse_entries(char *id, unsigned long table_size,
return 0;
}
+static inline void acpi_boot_table_prepare(void)
+{
+}
+
static inline void acpi_boot_table_init(void)
{
- return;
}
static inline int acpi_mps_check(void)
#define amba_get_drvdata(d) dev_get_drvdata(&d->dev)
#define amba_set_drvdata(d,p) dev_set_drvdata(&d->dev, p)
+#ifdef CONFIG_ARM_AMBA
int amba_driver_register(struct amba_driver *);
void amba_driver_unregister(struct amba_driver *);
+#else
+static inline int amba_driver_register(struct amba_driver *drv)
+{
+ return -EINVAL;
+}
+static inline void amba_driver_unregister(struct amba_driver *drv)
+{
+}
+#endif
+
struct amba_device *amba_device_alloc(const char *, resource_size_t, size_t);
void amba_device_put(struct amba_device *);
int amba_device_add(struct amba_device *, struct resource *);
/*
* rWTM BIU Mailbox driver for Armada 37xx
*
- * Author: Marek Behun <marek.behun@nic.cz>
+ * Author: Marek Behún <kabel@kernel.org>
*/
#ifndef _LINUX_ARMADA_37XX_RWTM_MAILBOX_H_
u16 vsi_id;
u16 key_len;
u8 key[1]; /* RSS hash key, packed bytes */
- u8 pad[1];
};
VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
u16 vsi_id;
u16 lut_entries;
u8 lut[1]; /* RSS lookup table */
- u8 pad[1];
};
VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
#define RQF_ELVPRIV ((__force req_flags_t)(1 << 12))
/* account into disk and partition IO statistics */
#define RQF_IO_STAT ((__force req_flags_t)(1 << 13))
-/* request came from our alloc pool */
-#define RQF_ALLOCED ((__force req_flags_t)(1 << 14))
/* runtime pm request */
#define RQF_PM ((__force req_flags_t)(1 << 15))
/* on IO scheduler merge hash */
#include <linux/capability.h>
#include <linux/sched/mm.h>
#include <linux/slab.h>
+#include <linux/percpu-refcount.h>
struct bpf_verifier_env;
struct bpf_verifier_log;
struct bpf_local_storage_map;
struct kobject;
struct mem_cgroup;
+struct module;
extern struct idr btf_idr;
extern spinlock_t btf_idr_lock;
* fentry = a set of program to run before calling original function
* fexit = a set of program to run after original function
*/
-int arch_prepare_bpf_trampoline(void *image, void *image_end,
+struct bpf_tramp_image;
+int arch_prepare_bpf_trampoline(struct bpf_tramp_image *tr, void *image, void *image_end,
const struct btf_func_model *m, u32 flags,
struct bpf_tramp_progs *tprogs,
void *orig_call);
void notrace __bpf_prog_exit(struct bpf_prog *prog, u64 start);
u64 notrace __bpf_prog_enter_sleepable(struct bpf_prog *prog);
void notrace __bpf_prog_exit_sleepable(struct bpf_prog *prog, u64 start);
+void notrace __bpf_tramp_enter(struct bpf_tramp_image *tr);
+void notrace __bpf_tramp_exit(struct bpf_tramp_image *tr);
struct bpf_ksym {
unsigned long start;
BPF_TRAMP_REPLACE, /* more than MAX */
};
+struct bpf_tramp_image {
+ void *image;
+ struct bpf_ksym ksym;
+ struct percpu_ref pcref;
+ void *ip_after_call;
+ void *ip_epilogue;
+ union {
+ struct rcu_head rcu;
+ struct work_struct work;
+ };
+};
+
struct bpf_trampoline {
/* hlist for trampoline_table */
struct hlist_node hlist;
/* Number of attached programs. A counter per kind. */
int progs_cnt[BPF_TRAMP_MAX];
/* Executable image of trampoline */
- void *image;
+ struct bpf_tramp_image *cur_image;
u64 selector;
- struct bpf_ksym ksym;
+ struct module *mod;
};
struct bpf_attach_target_info {
void bpf_image_ksym_del(struct bpf_ksym *ksym);
void bpf_ksym_add(struct bpf_ksym *ksym);
void bpf_ksym_del(struct bpf_ksym *ksym);
+int bpf_jit_charge_modmem(u32 pages);
+void bpf_jit_uncharge_modmem(u32 pages);
#else
static inline int bpf_trampoline_link_prog(struct bpf_prog *prog,
struct bpf_trampoline *tr)
bool func_proto_unreliable;
bool sleepable;
bool tail_call_reachable;
- enum bpf_tramp_prog_type trampoline_prog_type;
struct hlist_node tramp_hlist;
/* BTF_KIND_FUNC_PROTO for valid attach_btf_id */
const struct btf_type *attach_func_proto;
_ret; \
})
-#define __BPF_PROG_RUN_ARRAY(array, ctx, func, check_non_null) \
+#define __BPF_PROG_RUN_ARRAY(array, ctx, func, check_non_null, set_cg_storage) \
({ \
struct bpf_prog_array_item *_item; \
struct bpf_prog *_prog; \
goto _out; \
_item = &_array->items[0]; \
while ((_prog = READ_ONCE(_item->prog))) { \
- bpf_cgroup_storage_set(_item->cgroup_storage); \
+ if (set_cg_storage) \
+ bpf_cgroup_storage_set(_item->cgroup_storage); \
_ret &= func(_prog, ctx); \
_item++; \
} \
})
#define BPF_PROG_RUN_ARRAY(array, ctx, func) \
- __BPF_PROG_RUN_ARRAY(array, ctx, func, false)
+ __BPF_PROG_RUN_ARRAY(array, ctx, func, false, true)
#define BPF_PROG_RUN_ARRAY_CHECK(array, ctx, func) \
- __BPF_PROG_RUN_ARRAY(array, ctx, func, true)
+ __BPF_PROG_RUN_ARRAY(array, ctx, func, true, false)
#ifdef CONFIG_BPF_SYSCALL
DECLARE_PER_CPU(int, bpf_prog_active);
(parent_data), (flags), (reg), (shift), \
(width), (clk_divider_flags), (table), \
(lock))
+/**
+ * devm_clk_hw_register_divider - register a divider clock with the clock framework
+ * @dev: device registering this clock
+ * @name: name of this clock
+ * @parent_name: name of clock's parent
+ * @flags: framework-specific flags
+ * @reg: register address to adjust divider
+ * @shift: number of bits to shift the bitfield
+ * @width: width of the bitfield
+ * @clk_divider_flags: divider-specific flags for this clock
+ * @lock: shared register lock for this clock
+ */
+#define devm_clk_hw_register_divider(dev, name, parent_name, flags, reg, shift, \
+ width, clk_divider_flags, lock) \
+ __devm_clk_hw_register_divider((dev), NULL, (name), (parent_name), NULL, \
+ NULL, (flags), (reg), (shift), (width), \
+ (clk_divider_flags), NULL, (lock))
/**
* devm_clk_hw_register_divider_table - register a table based divider clock
* with the clock framework (devres variant)
const struct clk_parent_data *parent_data,
unsigned long flags, void __iomem *reg, u8 shift, u32 mask,
u8 clk_mux_flags, u32 *table, spinlock_t *lock);
+struct clk_hw *__devm_clk_hw_register_mux(struct device *dev, struct device_node *np,
+ const char *name, u8 num_parents,
+ const char * const *parent_names,
+ const struct clk_hw **parent_hws,
+ const struct clk_parent_data *parent_data,
+ unsigned long flags, void __iomem *reg, u8 shift, u32 mask,
+ u8 clk_mux_flags, u32 *table, spinlock_t *lock);
struct clk *clk_register_mux_table(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents,
unsigned long flags, void __iomem *reg, u8 shift, u32 mask,
__clk_hw_register_mux((dev), NULL, (name), (num_parents), NULL, NULL, \
(parent_data), (flags), (reg), (shift), \
BIT((width)) - 1, (clk_mux_flags), NULL, (lock))
+#define devm_clk_hw_register_mux(dev, name, parent_names, num_parents, flags, reg, \
+ shift, width, clk_mux_flags, lock) \
+ __devm_clk_hw_register_mux((dev), NULL, (name), (num_parents), \
+ (parent_names), NULL, NULL, (flags), (reg), \
+ (shift), BIT((width)) - 1, (clk_mux_flags), \
+ NULL, (lock))
int clk_mux_val_to_index(struct clk_hw *hw, u32 *table, unsigned int flags,
unsigned int val);
#define dm_target_passes_integrity(type) ((type)->features & DM_TARGET_PASSES_INTEGRITY)
/*
- * Indicates that a target supports host-managed zoned block devices.
+ * Indicates support for zoned block devices:
+ * - DM_TARGET_ZONED_HM: the target also supports host-managed zoned
+ * block devices but does not support combining different zoned models.
+ * - DM_TARGET_MIXED_ZONED_MODEL: the target supports combining multiple
+ * devices with different zoned models.
*/
#ifdef CONFIG_BLK_DEV_ZONED
#define DM_TARGET_ZONED_HM 0x00000040
#define DM_TARGET_PASSES_CRYPTO 0x00000100
#define dm_target_passes_crypto(type) ((type)->features & DM_TARGET_PASSES_CRYPTO)
+#ifdef CONFIG_BLK_DEV_ZONED
+#define DM_TARGET_MIXED_ZONED_MODEL 0x00000200
+#define dm_target_supports_mixed_zoned_model(type) \
+ ((type)->features & DM_TARGET_MIXED_ZONED_MODEL)
+#else
+#define DM_TARGET_MIXED_ZONED_MODEL 0x00000000
+#define dm_target_supports_mixed_zoned_model(type) (false)
+#endif
+
struct dm_target {
struct dm_table *table;
struct target_type *type;
}
struct dma_fence *dma_fence_get_stub(void);
+struct dma_fence *dma_fence_allocate_private_stub(void);
u64 dma_fence_context_alloc(unsigned num);
#define DMA_FENCE_TRACE(f, fmt, args...) \
*/
typedef guid_t efi_guid_t __aligned(__alignof__(u32));
-#define EFI_GUID(a,b,c,d0,d1,d2,d3,d4,d5,d6,d7) \
- GUID_INIT(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7)
+#define EFI_GUID(a, b, c, d...) (efi_guid_t){ { \
+ (a) & 0xff, ((a) >> 8) & 0xff, ((a) >> 16) & 0xff, ((a) >> 24) & 0xff, \
+ (b) & 0xff, ((b) >> 8) & 0xff, \
+ (c) & 0xff, ((c) >> 8) & 0xff, d } }
/*
* Generic EFI table header
int ethtool_op_get_ts_info(struct net_device *dev, struct ethtool_ts_info *eti);
-/**
- * struct ethtool_link_ext_state_info - link extended state and substate.
- */
+/* Link extended state and substate. */
struct ethtool_link_ext_state_info {
enum ethtool_link_ext_state link_ext_state;
union {
__ETHTOOL_DECLARE_LINK_MODE_MASK(lp_advertising);
} link_modes;
u32 lanes;
- enum ethtool_link_mode_bit_indices link_mode;
};
/**
* do not attach ext_substate attribute to netlink message). If link_ext_state
* and link_ext_substate are unknown, return -ENODATA. If not implemented,
* link_ext_state and link_ext_substate will not be sent to userspace.
+ * @get_eeprom_len: Read range of EEPROM addresses for validation of
+ * @get_eeprom and @set_eeprom requests.
+ * Returns 0 if device does not support EEPROM access.
* @get_eeprom: Read data from the device EEPROM.
* Should fill in the magic field. Don't need to check len for zero
* or wraparound. Fill in the data argument with the eeprom values
* @get_module_eeprom: Get the eeprom information from the plug-in module
* @get_eee: Get Energy-Efficient (EEE) supported and status.
* @set_eee: Set EEE status (enable/disable) as well as LPI timers.
+ * @get_tunable: Read the value of a driver / device tunable.
+ * @set_tunable: Set the value of a driver / device tunable.
* @get_per_queue_coalesce: Get interrupt coalescing parameters per queue.
* It must check that the given queue number is valid. If neither a RX nor
* a TX queue has this number, return -EINVAL. If only a RX queue or a TX
* @get_sset_count: Get number of strings that @get_strings will write.
* @get_strings: Return a set of strings that describe the requested objects
* @get_stats: Return extended statistics about the PHY device.
- * @start_cable_test - Start a cable test
- * @start_cable_test_tdr - Start a Time Domain Reflectometry cable test
+ * @start_cable_test: Start a cable test
+ * @start_cable_test_tdr: Start a Time Domain Reflectometry cable test
*
* All operations are optional (i.e. the function pointer may be set to %NULL)
* and callers must take this into account. Callers must hold the RTNL lock.
*/
void ethtool_set_ethtool_phy_ops(const struct ethtool_phy_ops *ops);
+/*
+ * ethtool_params_from_link_mode - Derive link parameters from a given link mode
+ * @link_ksettings: Link parameters to be derived from the link mode
+ * @link_mode: Link mode
+ */
+void
+ethtool_params_from_link_mode(struct ethtool_link_ksettings *link_ksettings,
+ enum ethtool_link_mode_bit_indices link_mode);
#endif /* _LINUX_ETHTOOL_H */
struct extcon_dev *edev, unsigned int id,
struct notifier_block *nb) { }
+static inline int extcon_register_notifier_all(struct extcon_dev *edev,
+ struct notifier_block *nb)
+{
+ return 0;
+}
+
+static inline int extcon_unregister_notifier_all(struct extcon_dev *edev,
+ struct notifier_block *nb)
+{
+ return 0;
+}
+
+static inline int devm_extcon_register_notifier_all(struct device *dev,
+ struct extcon_dev *edev,
+ struct notifier_block *nb)
+{
+ return 0;
+}
+
+static inline void devm_extcon_unregister_notifier_all(struct device *dev,
+ struct extcon_dev *edev,
+ struct notifier_block *nb) { }
+
static inline struct extcon_dev *extcon_get_extcon_dev(const char *extcon_name)
{
return ERR_PTR(-ENODEV);
* COMMAND_RECONFIG_FLAG_PARTIAL:
* Set to FPGA configuration type (full or partial).
*/
-#define COMMAND_RECONFIG_FLAG_PARTIAL 1
+#define COMMAND_RECONFIG_FLAG_PARTIAL 0
/*
* Timeout settings for service clients:
/**
* struct host1x_client_ops - host1x client operations
+ * @early_init: host1x client early initialization code
* @init: host1x client initialization code
* @exit: host1x client tear down code
+ * @late_exit: host1x client late tear down code
* @suspend: host1x client suspend code
* @resume: host1x client resume code
*/
struct host1x_client_ops {
+ int (*early_init)(struct host1x_client *client);
int (*init)(struct host1x_client *client);
int (*exit)(struct host1x_client *client);
+ int (*late_exit)(struct host1x_client *client);
int (*suspend)(struct host1x_client *client);
int (*resume)(struct host1x_client *client);
};
struct host1x_syncpt;
struct host1x;
-struct host1x_syncpt *host1x_syncpt_get(struct host1x *host, u32 id);
+struct host1x_syncpt *host1x_syncpt_get_by_id(struct host1x *host, u32 id);
+struct host1x_syncpt *host1x_syncpt_get_by_id_noref(struct host1x *host, u32 id);
+struct host1x_syncpt *host1x_syncpt_get(struct host1x_syncpt *sp);
u32 host1x_syncpt_id(struct host1x_syncpt *sp);
u32 host1x_syncpt_read_min(struct host1x_syncpt *sp);
u32 host1x_syncpt_read_max(struct host1x_syncpt *sp);
u32 *value);
struct host1x_syncpt *host1x_syncpt_request(struct host1x_client *client,
unsigned long flags);
-void host1x_syncpt_free(struct host1x_syncpt *sp);
+void host1x_syncpt_put(struct host1x_syncpt *sp);
+struct host1x_syncpt *host1x_syncpt_alloc(struct host1x *host,
+ unsigned long flags,
+ const char *name);
struct host1x_syncpt_base *host1x_syncpt_get_base(struct host1x_syncpt *sp);
u32 host1x_syncpt_base_id(struct host1x_syncpt_base *base);
+void host1x_syncpt_release_vblank_reservation(struct host1x_client *client,
+ u32 syncpt_id);
+
/*
* host1x channel
*/
dma_addr_t *reloc_addr_phys;
/* Sync point id, number of increments and end related to the submit */
- u32 syncpt_id;
+ struct host1x_syncpt *syncpt;
u32 syncpt_incrs;
u32 syncpt_end;
int host1x_device_init(struct host1x_device *device);
int host1x_device_exit(struct host1x_device *device);
-int host1x_client_register(struct host1x_client *client);
+int __host1x_client_register(struct host1x_client *client,
+ struct lock_class_key *key);
+#define host1x_client_register(class) \
+ ({ \
+ static struct lock_class_key __key; \
+ __host1x_client_register(class, &__key); \
+ })
+
int host1x_client_unregister(struct host1x_client *client);
int host1x_client_suspend(struct host1x_client *client);
return !cgroup_subsys_enabled(hugetlb_cgrp_subsys);
}
+static inline void hugetlb_cgroup_put_rsvd_cgroup(struct hugetlb_cgroup *h_cg)
+{
+ css_put(&h_cg->css);
+}
+
extern int hugetlb_cgroup_charge_cgroup(int idx, unsigned long nr_pages,
struct hugetlb_cgroup **ptr);
extern int hugetlb_cgroup_charge_cgroup_rsvd(int idx, unsigned long nr_pages,
extern void hugetlb_cgroup_uncharge_file_region(struct resv_map *resv,
struct file_region *rg,
- unsigned long nr_pages);
+ unsigned long nr_pages,
+ bool region_del);
extern void hugetlb_cgroup_file_init(void) __init;
extern void hugetlb_cgroup_migrate(struct page *oldhpage,
#else
static inline void hugetlb_cgroup_uncharge_file_region(struct resv_map *resv,
struct file_region *rg,
- unsigned long nr_pages)
+ unsigned long nr_pages,
+ bool region_del)
{
}
return true;
}
+static inline void hugetlb_cgroup_put_rsvd_cgroup(struct hugetlb_cgroup *h_cg)
+{
+}
+
static inline int hugetlb_cgroup_charge_cgroup(int idx, unsigned long nr_pages,
struct hugetlb_cgroup **ptr)
{
if (likely(success)) {
struct vlan_pcpu_stats *pcpu_stats;
- pcpu_stats = this_cpu_ptr(vlan->pcpu_stats);
+ pcpu_stats = get_cpu_ptr(vlan->pcpu_stats);
u64_stats_update_begin(&pcpu_stats->syncp);
pcpu_stats->rx_packets++;
pcpu_stats->rx_bytes += len;
if (multicast)
pcpu_stats->rx_multicast++;
u64_stats_update_end(&pcpu_stats->syncp);
+ put_cpu_ptr(vlan->pcpu_stats);
} else {
this_cpu_inc(vlan->pcpu_stats->rx_errors);
}
#include <linux/sched.h>
#include <linux/xarray.h>
-struct io_wq_work_node {
- struct io_wq_work_node *next;
-};
-
-struct io_wq_work_list {
- struct io_wq_work_node *first;
- struct io_wq_work_node *last;
-};
-
-struct io_uring_task {
- /* submission side */
- struct xarray xa;
- struct wait_queue_head wait;
- void *last;
- void *io_wq;
- struct percpu_counter inflight;
- atomic_t in_idle;
- bool sqpoll;
-
- spinlock_t task_lock;
- struct io_wq_work_list task_list;
- unsigned long task_state;
- struct callback_head task_work;
-};
-
#if defined(CONFIG_IO_URING)
struct sock *io_uring_get_socket(struct file *file);
void __io_uring_task_cancel(void);
/*
* Set the allocation direction to bottom-up or top-down.
*/
-static inline __init void memblock_set_bottom_up(bool enable)
+static inline __init_memblock void memblock_set_bottom_up(bool enable)
{
memblock.bottom_up = enable;
}
* if this is true, that said, memblock will allocate memory
* in bottom-up direction.
*/
-static inline __init bool memblock_bottom_up(void)
+static inline __init_memblock bool memblock_bottom_up(void)
{
return memblock.bottom_up;
}
u8 reserved_at_60[0x18];
u8 log_max_ft_num[0x8];
- u8 reserved_at_80[0x18];
+ u8 reserved_at_80[0x10];
+ u8 log_max_flow_counter[0x8];
u8 log_max_destination[0x8];
- u8 log_max_flow_counter[0x8];
- u8 reserved_at_a8[0x10];
+ u8 reserved_at_a0[0x18];
u8 log_max_flow[0x8];
u8 reserved_at_c0[0x40];
u8 fec_override_admin_100g_2x[0x10];
u8 fec_override_admin_50g_1x[0x10];
+
+ u8 reserved_at_140[0x140];
};
struct mlx5_ifc_ppcnt_reg_bits {
struct mlx5_ifc_bufferx_reg_bits buffer[10];
- u8 reserved_at_2e0[0x40];
+ u8 reserved_at_2e0[0x80];
};
struct mlx5_ifc_qtct_reg_bits {
}
}
+static inline int mlx5_get_qp_default_ts(struct mlx5_core_dev *dev)
+{
+ return !MLX5_CAP_ROCE(dev, qp_ts_format) ?
+ MLX5_QPC_TIMESTAMP_FORMAT_FREE_RUNNING :
+ MLX5_QPC_TIMESTAMP_FORMAT_DEFAULT;
+}
+
#endif /* MLX5_QP_H */
#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
+/*
+ * KASAN per-page tags are stored xor'ed with 0xff. This allows to avoid
+ * setting tags for all pages to native kernel tag value 0xff, as the default
+ * value 0x00 maps to 0xff.
+ */
+
static inline u8 page_kasan_tag(const struct page *page)
{
- if (kasan_enabled())
- return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
- return 0xff;
+ u8 tag = 0xff;
+
+ if (kasan_enabled()) {
+ tag = (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
+ tag ^= 0xff;
+ }
+
+ return tag;
}
static inline void page_kasan_tag_set(struct page *page, u8 tag)
{
if (kasan_enabled()) {
+ tag ^= 0xff;
page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT);
page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT;
}
* the last refcount is dropped.
*
* If blockable argument is set to false then the callback cannot
- * sleep and has to return with -EAGAIN. 0 should be returned
- * otherwise. Please note that if invalidate_range_start approves
- * a non-blocking behavior then the same applies to
- * invalidate_range_end.
- *
+ * sleep and has to return with -EAGAIN if sleeping would be required.
+ * 0 should be returned otherwise. Please note that notifiers that can
+ * fail invalidate_range_start are not allowed to implement
+ * invalidate_range_end, as there is no mechanism for informing the
+ * notifier that its start failed.
*/
int (*invalidate_range_start)(struct mmu_notifier *subscription,
const struct mmu_notifier_range *range);
#include <linux/percpu.h>
#include <asm/module.h>
-/* Not Yet Implemented */
-#define MODULE_SUPPORTED_DEVICE(name)
-
#define MODULE_NAME_LEN MAX_PARAM_PREFIX_LEN
struct modversion_info {
/*
* Turris Mox module configuration bus driver
*
- * Copyright (C) 2019 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2019 Marek Behún <kabel@kernel.org>
*/
#ifndef __LINUX_MOXTET_H
# define mutex_lock_interruptible_nested(lock, subclass) mutex_lock_interruptible(lock)
# define mutex_lock_killable_nested(lock, subclass) mutex_lock_killable(lock)
# define mutex_lock_nest_lock(lock, nest_lock) mutex_lock(lock)
-# define mutex_lock_io_nested(lock, subclass) mutex_lock(lock)
+# define mutex_lock_io_nested(lock, subclass) mutex_lock_io(lock)
#endif
/*
NAPI_STATE_IN_BUSY_POLL, /* sk_busy_loop() owns this NAPI */
NAPI_STATE_PREFER_BUSY_POLL, /* prefer busy-polling over softirq processing*/
NAPI_STATE_THREADED, /* The poll is performed inside its own thread*/
+ NAPI_STATE_SCHED_THREADED, /* Napi is currently scheduled in threaded mode */
};
enum {
NAPIF_STATE_IN_BUSY_POLL = BIT(NAPI_STATE_IN_BUSY_POLL),
NAPIF_STATE_PREFER_BUSY_POLL = BIT(NAPI_STATE_PREFER_BUSY_POLL),
NAPIF_STATE_THREADED = BIT(NAPI_STATE_THREADED),
+ NAPIF_STATE_SCHED_THREADED = BIT(NAPI_STATE_SCHED_THREADED),
};
enum gro_result {
unsigned int valid_hooks;
/* Man behind the curtain... */
- struct xt_table_info __rcu *private;
+ struct xt_table_info *private;
/* Set this to THIS_MODULE if you are a module, otherwise NULL */
struct module *me;
* since addend is most likely 1
*/
__this_cpu_add(xt_recseq.sequence, addend);
- smp_wmb();
+ smp_mb();
return addend;
}
struct nf_hook_ops *xt_hook_ops_alloc(const struct xt_table *, nf_hookfn *);
-struct xt_table_info
-*xt_table_get_private_protected(const struct xt_table *table);
-
#ifdef CONFIG_COMPAT
#include <net/compat.h>
return pgoff;
}
-/* This has the same layout as wait_bit_key - see fs/cachefiles/rdwr.c */
struct wait_page_key {
struct page *page;
int bit_nr;
int put_and_wait_on_page_locked(struct page *page, int state);
void wait_on_page_writeback(struct page *page);
+int wait_on_page_writeback_killable(struct page *page);
extern void end_page_writeback(struct page *page);
void wait_for_stable_page(struct page *page);
int geni_icc_enable(struct geni_se *se);
int geni_icc_disable(struct geni_se *se);
-
-void geni_remove_earlycon_icc_vote(void);
#endif
#endif
* System call restart block.
*/
struct restart_block {
+ unsigned long arch_data;
long (*fn)(struct restart_block *);
union {
/* For futex_wait and futex_wait_requeue_pi */
struct tc_skb_ext {
__u32 chain;
__u16 mru;
+ bool post_ct;
};
#endif
static inline void sk_psock_restore_proto(struct sock *sk,
struct sk_psock *psock)
{
- sk->sk_prot->unhash = psock->saved_unhash;
if (inet_csk_has_ulp(sk)) {
+ /* TLS does not have an unhash proto in SW cases, but we need
+ * to ensure we stop using the sock_map unhash routine because
+ * the associated psock is being removed. So use the original
+ * unhash handler.
+ */
+ WRITE_ONCE(sk->sk_prot->unhash, psock->saved_unhash);
tcp_update_ulp(sk, psock->sk_proto, psock->saved_write_space);
} else {
sk->sk_write_space = psock->saved_write_space;
wait_queue_head_t sc_send_wait; /* SQ exhaustion waitlist */
unsigned long sc_flags;
- u32 sc_pending_recvs;
struct list_head sc_read_complete_q;
struct work_struct sc_work;
#include <linux/types.h>
#include <linux/bug.h>
#include <linux/restart_block.h>
+#include <linux/errno.h>
#ifdef CONFIG_THREAD_INFO_IN_TASK
/*
#ifdef __KERNEL__
+#ifndef arch_set_restart_data
+#define arch_set_restart_data(restart) do { } while (0)
+#endif
+
+static inline long set_restart_fn(struct restart_block *restart,
+ long (*fn)(struct restart_block *))
+{
+ restart->fn = fn;
+ arch_set_restart_data(restart);
+ return -ERESTART_RESTARTBLOCK;
+}
+
#ifndef THREAD_ALIGN
#define THREAD_ALIGN THREAD_SIZE
#endif
/* lies about caching, so always sync */ \
US_FLAG(NO_SAME, 0x40000000) \
/* Cannot handle WRITE_SAME */ \
+ US_FLAG(SENSE_AFTER_SYNC, 0x80000000) \
+ /* Do REQUEST_SENSE after SYNCHRONIZE_CACHE */ \
#define US_FLAG(name, value) US_FL_##name = value ,
enum { US_DO_ALL_FLAGS };
int umd_load_blob(struct umd_info *info, const void *data, size_t len);
int umd_unload_blob(struct umd_info *info);
int fork_usermode_driver(struct umd_info *info);
+void umd_cleanup_helper(struct umd_info *info);
#endif /* __LINUX_USERMODE_DRIVER_H__ */
struct vdpa_device *__vdpa_alloc_device(struct device *parent,
const struct vdpa_config_ops *config,
- int nvqs, size_t size, const char *name);
+ size_t size, const char *name);
-#define vdpa_alloc_device(dev_struct, member, parent, config, nvqs, name) \
+#define vdpa_alloc_device(dev_struct, member, parent, config, name) \
container_of(__vdpa_alloc_device( \
- parent, config, nvqs, \
+ parent, config, \
sizeof(dev_struct) + \
BUILD_BUG_ON_ZERO(offsetof( \
dev_struct, member)), name), \
dev_struct, member)
-int vdpa_register_device(struct vdpa_device *vdev);
+int vdpa_register_device(struct vdpa_device *vdev, int nvqs);
void vdpa_unregister_device(struct vdpa_device *vdev);
-int _vdpa_register_device(struct vdpa_device *vdev);
+int _vdpa_register_device(struct vdpa_device *vdev, int nvqs);
void _vdpa_unregister_device(struct vdpa_device *vdev);
/**
void virtio_break_device(struct virtio_device *dev);
void virtio_config_changed(struct virtio_device *dev);
-void virtio_config_disable(struct virtio_device *dev);
-void virtio_config_enable(struct virtio_device *dev);
int virtio_finalize_features(struct virtio_device *dev);
#ifdef CONFIG_PM_SLEEP
int virtio_device_freeze(struct virtio_device *dev);
return -EINVAL;
}
+ skb_reset_mac_header(skb);
+
if (hdr->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
- u16 start = __virtio16_to_cpu(little_endian, hdr->csum_start);
- u16 off = __virtio16_to_cpu(little_endian, hdr->csum_offset);
+ u32 start = __virtio16_to_cpu(little_endian, hdr->csum_start);
+ u32 off = __virtio16_to_cpu(little_endian, hdr->csum_offset);
+ u32 needed = start + max_t(u32, thlen, off + sizeof(__sum16));
+
+ if (!pskb_may_pull(skb, needed))
+ return -EINVAL;
if (!skb_partial_csum_set(skb, start, off))
return -EINVAL;
p_off = skb_transport_offset(skb) + thlen;
- if (p_off > skb_headlen(skb))
+ if (!pskb_may_pull(skb, p_off))
return -EINVAL;
} else {
/* gso packets without NEEDS_CSUM do not set transport_offset.
}
p_off = keys.control.thoff + thlen;
- if (p_off > skb_headlen(skb) ||
+ if (!pskb_may_pull(skb, p_off) ||
keys.basic.ip_proto != ip_proto)
return -EINVAL;
skb_set_transport_header(skb, keys.control.thoff);
} else if (gso_type) {
p_off = thlen;
- if (p_off > skb_headlen(skb))
+ if (!pskb_may_pull(skb, p_off))
return -EINVAL;
}
}
*/
static inline void ww_acquire_fini(struct ww_acquire_ctx *ctx)
{
-#ifdef CONFIG_DEBUG_MUTEXES
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
mutex_release(&ctx->dep_map, _THIS_IP_);
-
+#endif
+#ifdef CONFIG_DEBUG_MUTEXES
DEBUG_LOCKS_WARN_ON(ctx->acquired);
if (!IS_ENABLED(CONFIG_PROVE_LOCKING))
/*
*
* This structure is used either directly or via the XA_LIMIT() macro
* to communicate the range of IDs that are valid for allocation.
- * Two common ranges are predefined for you:
+ * Three common ranges are predefined for you:
* * xa_limit_32b - [0 - UINT_MAX]
* * xa_limit_31b - [0 - INT_MAX]
+ * * xa_limit_16b - [0 - USHRT_MAX]
*/
struct xa_limit {
u32 max;
#define xa_limit_32b XA_LIMIT(0, UINT_MAX)
#define xa_limit_31b XA_LIMIT(0, INT_MAX)
+#define xa_limit_16b XA_LIMIT(0, USHRT_MAX)
typedef unsigned __bitwise xa_mark_t;
#define XA_MARK_0 ((__force xa_mark_t)0U)
void tcf_idr_cleanup(struct tc_action_net *tn, u32 index);
int tcf_idr_check_alloc(struct tc_action_net *tn, u32 *index,
struct tc_action **a, int bind);
-int __tcf_idr_release(struct tc_action *a, bool bind, bool strict);
-
-static inline int tcf_idr_release(struct tc_action *a, bool bind)
-{
- return __tcf_idr_release(a, bind, false);
-}
+int tcf_idr_release(struct tc_action *a, bool bind);
int tcf_register_action(struct tc_action_ops *a, struct pernet_operations *ops);
int tcf_unregister_action(struct tc_action_ops *a,
int nr_actions, struct tcf_result *res);
int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
struct nlattr *est, char *name, int ovr, int bind,
- struct tc_action *actions[], size_t *attr_size,
+ struct tc_action *actions[], int init_res[], size_t *attr_size,
bool rtnl_held, struct netlink_ext_ack *extack);
struct tc_action_ops *tc_action_load_ops(char *name, struct nlattr *nla,
bool rtnl_held,
struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp,
struct nlattr *nla, struct nlattr *est,
char *name, int ovr, int bind,
- struct tc_action_ops *ops, bool rtnl_held,
+ struct tc_action_ops *a_o, int *init_res,
+ bool rtnl_held,
struct netlink_ext_ack *extack);
int tcf_action_dump(struct sk_buff *skb, struct tc_action *actions[], int bind,
int ref, bool terse);
dst->ops->update_pmtu(dst, NULL, skb, mtu, false);
}
+struct dst_entry *dst_blackhole_check(struct dst_entry *dst, u32 cookie);
+void dst_blackhole_update_pmtu(struct dst_entry *dst, struct sock *sk,
+ struct sk_buff *skb, u32 mtu, bool confirm_neigh);
+void dst_blackhole_redirect(struct dst_entry *dst, struct sock *sk,
+ struct sk_buff *skb);
+u32 *dst_blackhole_cow_metrics(struct dst_entry *dst, unsigned long old);
+struct neighbour *dst_blackhole_neigh_lookup(const struct dst_entry *dst,
+ struct sk_buff *skb,
+ const void *daddr);
+unsigned int dst_blackhole_mtu(const struct dst_entry *dst);
+
#endif /* _NET_DST_H */
return inet_csk_reqsk_queue_len(sk) >= sk->sk_max_ack_backlog;
}
-void inet_csk_reqsk_queue_drop(struct sock *sk, struct request_sock *req);
+bool inet_csk_reqsk_queue_drop(struct sock *sk, struct request_sock *req);
void inet_csk_reqsk_queue_drop_and_put(struct sock *sk, struct request_sock *req);
static inline void inet_csk_prepare_for_destroy_sock(struct sock *sk)
struct nft_flowtable *flowtable;
bool update;
struct list_head hook_list;
+ u32 flags;
};
#define nft_trans_flowtable(trans) \
(((struct nft_trans_flowtable *)trans->data)->update)
#define nft_trans_flowtable_hooks(trans) \
(((struct nft_trans_flowtable *)trans->data)->hook_list)
+#define nft_trans_flowtable_flags(trans) \
+ (((struct nft_trans_flowtable *)trans->data)->flags)
int __init nft_chain_filter_init(void);
void nft_chain_filter_fini(void);
#if IS_ENABLED(CONFIG_IPV6)
struct dst_ops xfrm6_dst_ops;
#endif
- spinlock_t xfrm_state_lock;
+ spinlock_t xfrm_state_lock;
+ seqcount_spinlock_t xfrm_state_hash_generation;
+
spinlock_t xfrm_policy_lock;
struct mutex xfrm_cfg_mutex;
};
int fib6_check_nexthop(struct nexthop *nh, struct fib6_config *cfg,
struct netlink_ext_ack *extack);
+/* Caller should either hold rcu_read_lock(), or RTNL. */
static inline struct fib6_nh *nexthop_fib6_nh(struct nexthop *nh)
{
struct nh_info *nhi;
return NULL;
}
+/* Variant of nexthop_fib6_nh().
+ * Caller should either hold rcu_read_lock_bh(), or RTNL.
+ */
+static inline struct fib6_nh *nexthop_fib6_nh_bh(struct nexthop *nh)
+{
+ struct nh_info *nhi;
+
+ if (nh->is_group) {
+ struct nh_group *nh_grp;
+
+ nh_grp = rcu_dereference_bh_rtnl(nh->nh_grp);
+ nh = nexthop_mpath_select(nh_grp, 0);
+ if (!nh)
+ return NULL;
+ }
+
+ nhi = rcu_dereference_bh_rtnl(nh->nh_info);
+ if (nhi->family == AF_INET6)
+ return &nhi->fib6_nh;
+
+ return NULL;
+}
+
static inline struct net_device *fib6_info_nh_dev(struct fib6_info *f6i)
{
struct fib6_nh *fib6_nh;
v->qcount = -1;
}
-static inline bool red_check_params(u32 qth_min, u32 qth_max, u8 Wlog, u8 Scell_log)
+static inline bool red_check_params(u32 qth_min, u32 qth_max, u8 Wlog,
+ u8 Scell_log, u8 *stab)
{
- if (fls(qth_min) + Wlog > 32)
+ if (fls(qth_min) + Wlog >= 32)
return false;
- if (fls(qth_max) + Wlog > 32)
+ if (fls(qth_max) + Wlog >= 32)
return false;
if (Scell_log >= 32)
return false;
if (qth_max < qth_min)
return false;
+ if (stab) {
+ int i;
+
+ for (i = 0; i < RED_STAB_SIZE; i++)
+ if (stab[i] >= 32)
+ return false;
+ }
return true;
}
int shift;
/*
- * The problem: ideally, average length queue recalcultion should
+ * The problem: ideally, average length queue recalculation should
* be done over constant clock intervals. This is too expensive, so
* that the calculation is driven by outgoing packets.
* When the queue is idle we have to model this clock by hand.
*
* @list: Used internally
* @kind: Identifier
+ * @netns_refund: Physical device, move to init_net on netns exit
* @maxtype: Highest device specific netlink attribute number
* @policy: Netlink policy for device specific attribute validation
* @validate: Optional validation function for netlink/changelink parameters
size_t priv_size;
void (*setup)(struct net_device *dev);
+ bool netns_refund;
unsigned int maxtype;
const struct nla_policy *policy;
int (*validate)(struct nlattr *tb[],
int (*validate_link_af)(const struct net_device *dev,
const struct nlattr *attr);
int (*set_link_af)(struct net_device *dev,
- const struct nlattr *attr);
-
+ const struct nlattr *attr,
+ struct netlink_ext_ack *extack);
int (*fill_stats_af)(struct sk_buff *skb,
const struct net_device *dev);
size_t (*get_stats_af_size)(const struct net_device *dev);
WRITE_ONCE(sk->sk_ack_backlog, sk->sk_ack_backlog + 1);
}
+/* Note: If you think the test should be:
+ * return READ_ONCE(sk->sk_ack_backlog) >= READ_ONCE(sk->sk_max_ack_backlog);
+ * Then please take a look at commit 64a146513f8f ("[NET]: Revert incorrect accept queue backlog changes.")
+ */
static inline bool sk_acceptq_is_full(const struct sock *sk)
{
return READ_ONCE(sk->sk_ack_backlog) > READ_ONCE(sk->sk_max_ack_backlog);
sk_mem_charge(sk, skb->truesize);
}
+static inline void skb_set_owner_sk_safe(struct sk_buff *skb, struct sock *sk)
+{
+ if (sk && refcount_inc_not_zero(&sk->sk_refcnt)) {
+ skb_orphan(skb);
+ skb->destructor = sock_efree;
+ skb->sk = sk;
+ }
+}
+
void sk_reset_timer(struct sock *sk, struct timer_list *timer,
unsigned long expires);
return __xfrm_policy_check(sk, ndir, skb, family);
return (!net->xfrm.policy_count[dir] && !secpath_exists(skb)) ||
- (skb_dst(skb)->flags & DST_NOPOLICY) ||
+ (skb_dst(skb) && (skb_dst(skb)->flags & DST_NOPOLICY)) ||
__xfrm_policy_check(sk, ndir, skb, family);
}
int xfrm_trans_queue(struct sk_buff *skb,
int (*finish)(struct net *, struct sock *,
struct sk_buff *));
-int xfrm_output_resume(struct sk_buff *skb, int err);
+int xfrm_output_resume(struct sock *sk, struct sk_buff *skb, int err);
int xfrm_output(struct sock *sk, struct sk_buff *skb);
#if IS_ENABLED(CONFIG_NET_PKTGEN)
ISCSI_CONN_UP = 0,
ISCSI_CONN_DOWN,
ISCSI_CONN_FAILED,
+ ISCSI_CONN_BOUND,
};
struct iscsi_cls_conn {
TP_STRUCT__entry(
__field( void *, work )
__field( void *, function)
- __field( const char *, workqueue)
+ __string( workqueue, pwq->wq->name)
__field( unsigned int, req_cpu )
__field( unsigned int, cpu )
),
TP_fast_assign(
__entry->work = work;
__entry->function = work->func;
- __entry->workqueue = pwq->wq->name;
+ __assign_str(workqueue, pwq->wq->name);
__entry->req_cpu = req_cpu;
__entry->cpu = pwq->pool->cpu;
),
TP_printk("work struct=%p function=%ps workqueue=%s req_cpu=%u cpu=%u",
- __entry->work, __entry->function, __entry->workqueue,
+ __entry->work, __entry->function, __get_str(workqueue),
__entry->req_cpu, __entry->cpu)
);
#define MSM_PARAM_NR_RINGS 0x07
#define MSM_PARAM_PP_PGTABLE 0x08 /* => 1 for per-process pagetables, else 0 */
#define MSM_PARAM_FAULTS 0x09
+#define MSM_PARAM_SUSPENDS 0x0a
struct drm_msm_param {
__u32 pipe; /* in, MSM_PIPE_x */
#ifndef _UAPI__LINUX_BLKPG_H
#define _UAPI__LINUX_BLKPG_H
-/*
- * Partition table and disk geometry handling
- *
- * A single ioctl with lots of subfunctions:
- *
- * Device number stuff:
- * get_whole_disk() (given the device number of a partition,
- * find the device number of the encompassing disk)
- * get_all_partitions() (given the device number of a disk, return the
- * device numbers of all its known partitions)
- *
- * Partition stuff:
- * add_partition()
- * delete_partition()
- * test_partition_in_use() (also for test_disk_in_use)
- *
- * Geometry stuff:
- * get_geometry()
- * set_geometry()
- * get_bios_drivedata()
- *
- * For today, only the partition stuff - aeb, 990515
- */
#include <linux/compiler.h>
#include <linux/ioctl.h>
long long start; /* starting offset in bytes */
long long length; /* length in bytes */
int pno; /* partition number */
- char devname[BLKPG_DEVNAMELTH]; /* partition name, like sda5 or c0d1p2,
- to be used in kernel messages */
- char volname[BLKPG_VOLNAMELTH]; /* volume label */
+ char devname[BLKPG_DEVNAMELTH]; /* unused / ignored */
+ char volname[BLKPG_VOLNAMELTH]; /* unused / ignore */
};
#endif /* _UAPI__LINUX_BLKPG_H */
*
* long bpf_check_mtu(void *ctx, u32 ifindex, u32 *mtu_len, s32 len_diff, u64 flags)
* Description
- * Check ctx packet size against exceeding MTU of net device (based
+ * Check packet size against exceeding MTU of net device (based
* on *ifindex*). This helper will likely be used in combination
* with helpers that adjust/change the packet size.
*
* against the current net device. This is practical if this isn't
* used prior to redirect.
*
+ * On input *mtu_len* must be a valid pointer, else verifier will
+ * reject BPF program. If the value *mtu_len* is initialized to
+ * zero then the ctx packet size is use. When value *mtu_len* is
+ * provided as input this specify the L3 length that the MTU check
+ * is done against. Remember XDP and TC length operate at L2, but
+ * this value is L3 as this correlate to MTU and IP-header tot_len
+ * values which are L3 (similar behavior as bpf_fib_lookup).
+ *
* The Linux kernel route table can configure MTUs on a more
* specific per route level, which is not provided by this helper.
* For route level MTU checks use the **bpf_fib_lookup**\ ()
*
* On return *mtu_len* pointer contains the MTU value of the net
* device. Remember the net device configured MTU is the L3 size,
- * which is returned here and XDP and TX length operate at L2.
+ * which is returned here and XDP and TC length operate at L2.
* Helper take this into account for you, but remember when using
- * MTU value in your BPF-code. On input *mtu_len* must be a valid
- * pointer and be initialized (to zero), else verifier will reject
- * BPF program.
+ * MTU value in your BPF-code.
*
* Return
* * 0 on success, and populate MTU value in *mtu_len* pointer.
*/
__u8 len;
__u8 can_dlc; /* deprecated */
- };
+ } __attribute__((packed)); /* disable padding added in some ABIs */
__u8 __pad; /* padding */
__u8 __res0; /* reserved / padding */
__u8 len8_dlc; /* optional DLC for 8 byte payload length (9 .. 15) */
* have the same layout for 32-bit and 64-bit userland.
*/
+/* Note on reserved space.
+ * Reserved fields must not be accessed directly by user space because
+ * they may be replaced by a different field in the future. They must
+ * be initialized to zero before making the request, e.g. via memset
+ * of the entire structure or implicitly by not being set in a structure
+ * initializer.
+ */
+
/**
* struct ethtool_cmd - DEPRECATED, link control and status
* This structure is DEPRECATED, please use struct ethtool_link_settings.
* and other link features that the link partner advertised
* through autonegotiation; 0 if unknown or not applicable.
* Read-only.
+ * @reserved: Reserved for future use; see the note on reserved space.
*
* The link speed in Mbps is split between @speed and @speed_hi. Use
* the ethtool_cmd_speed() and ethtool_cmd_speed_set() functions to
* @bus_info: Device bus address. This should match the dev_name()
* string for the underlying bus device, if there is one. May be
* an empty string.
+ * @reserved2: Reserved for future use; see the note on reserved space.
* @n_priv_flags: Number of flags valid for %ETHTOOL_GPFLAGS and
* %ETHTOOL_SPFLAGS commands; also the number of strings in the
* %ETH_SS_PRIV_FLAGS set
* @tx_lpi_timer: Time in microseconds the interface delays prior to asserting
* its tx lpi (after reaching 'idle' state). Effective only when eee
* was negotiated and tx_lpi_enabled was set.
+ * @reserved: Reserved for future use; see the note on reserved space.
*/
struct ethtool_eee {
__u32 cmd;
* @cmd: %ETHTOOL_GMODULEINFO
* @type: Standard the module information conforms to %ETH_MODULE_SFF_xxxx
* @eeprom_len: Length of the eeprom
+ * @reserved: Reserved for future use; see the note on reserved space.
*
* This structure is used to return the information to
* properly size memory for a subsequent call to %ETHTOOL_GMODULEEEPROM.
__u32 tx_pause;
};
-/**
- * enum ethtool_link_ext_state - link extended state
- */
+/* Link extended state */
enum ethtool_link_ext_state {
ETHTOOL_LINK_EXT_STATE_AUTONEG,
ETHTOOL_LINK_EXT_STATE_LINK_TRAINING_FAILURE,
ETHTOOL_LINK_EXT_STATE_OVERHEAT,
};
-/**
- * enum ethtool_link_ext_substate_autoneg - more information in addition to
- * ETHTOOL_LINK_EXT_STATE_AUTONEG.
- */
+/* More information in addition to ETHTOOL_LINK_EXT_STATE_AUTONEG. */
enum ethtool_link_ext_substate_autoneg {
ETHTOOL_LINK_EXT_SUBSTATE_AN_NO_PARTNER_DETECTED = 1,
ETHTOOL_LINK_EXT_SUBSTATE_AN_ACK_NOT_RECEIVED,
ETHTOOL_LINK_EXT_SUBSTATE_AN_NO_HCD,
};
-/**
- * enum ethtool_link_ext_substate_link_training - more information in addition to
- * ETHTOOL_LINK_EXT_STATE_LINK_TRAINING_FAILURE.
+/* More information in addition to ETHTOOL_LINK_EXT_STATE_LINK_TRAINING_FAILURE.
*/
enum ethtool_link_ext_substate_link_training {
ETHTOOL_LINK_EXT_SUBSTATE_LT_KR_FRAME_LOCK_NOT_ACQUIRED = 1,
ETHTOOL_LINK_EXT_SUBSTATE_LT_REMOTE_FAULT,
};
-/**
- * enum ethtool_link_ext_substate_logical_mismatch - more information in addition
- * to ETHTOOL_LINK_EXT_STATE_LINK_LOGICAL_MISMATCH.
+/* More information in addition to ETHTOOL_LINK_EXT_STATE_LINK_LOGICAL_MISMATCH.
*/
enum ethtool_link_ext_substate_link_logical_mismatch {
ETHTOOL_LINK_EXT_SUBSTATE_LLM_PCS_DID_NOT_ACQUIRE_BLOCK_LOCK = 1,
ETHTOOL_LINK_EXT_SUBSTATE_LLM_RS_FEC_IS_NOT_LOCKED,
};
-/**
- * enum ethtool_link_ext_substate_bad_signal_integrity - more information in
- * addition to ETHTOOL_LINK_EXT_STATE_BAD_SIGNAL_INTEGRITY.
+/* More information in addition to ETHTOOL_LINK_EXT_STATE_BAD_SIGNAL_INTEGRITY.
*/
enum ethtool_link_ext_substate_bad_signal_integrity {
ETHTOOL_LINK_EXT_SUBSTATE_BSI_LARGE_NUMBER_OF_PHYSICAL_ERRORS = 1,
ETHTOOL_LINK_EXT_SUBSTATE_BSI_UNSUPPORTED_RATE,
};
-/**
- * enum ethtool_link_ext_substate_cable_issue - more information in
- * addition to ETHTOOL_LINK_EXT_STATE_CABLE_ISSUE.
- */
+/* More information in addition to ETHTOOL_LINK_EXT_STATE_CABLE_ISSUE. */
enum ethtool_link_ext_substate_cable_issue {
ETHTOOL_LINK_EXT_SUBSTATE_CI_UNSUPPORTED_CABLE = 1,
ETHTOOL_LINK_EXT_SUBSTATE_CI_CABLE_TEST_FAILURE,
* now deprecated
* @ETH_SS_FEATURES: Device feature names
* @ETH_SS_RSS_HASH_FUNCS: RSS hush function names
+ * @ETH_SS_TUNABLES: tunable names
* @ETH_SS_PHY_STATS: Statistic names, for use with %ETHTOOL_GPHYSTATS
* @ETH_SS_PHY_TUNABLES: PHY tunable names
* @ETH_SS_LINK_MODES: link mode names
* @ETH_SS_TS_TX_TYPES: timestamping Tx types
* @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
* @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types
+ *
+ * @ETH_SS_COUNT: number of defined string sets
*/
enum ethtool_stringset {
ETH_SS_TEST = 0,
/**
* struct ethtool_sset_info - string set information
* @cmd: Command number = %ETHTOOL_GSSET_INFO
+ * @reserved: Reserved for future use; see the note on reserved space.
* @sset_mask: On entry, a bitmask of string sets to query, with bits
* numbered according to &enum ethtool_stringset. On return, a
* bitmask of those string sets queried that are supported.
* @flags: A bitmask of flags from &enum ethtool_test_flags. Some
* flags may be set by the user on entry; others may be set by
* the driver on return.
+ * @reserved: Reserved for future use; see the note on reserved space.
* @len: On return, the number of test results
* @data: Array of test results
*
* @vlan_etype: VLAN EtherType
* @vlan_tci: VLAN tag control information
* @data: user defined data
+ * @padding: Reserved for future use; see the note on reserved space.
*
* Note, @vlan_etype, @vlan_tci, and @data are only valid if %FLOW_EXT
* is set in &struct ethtool_rx_flow_spec @flow_type.
* hardware hash key.
* @hfunc: Defines the current RSS hash function used by HW (or to be set to).
* Valid values are one of the %ETH_RSS_HASH_*.
- * @rsvd: Reserved for future extensions.
+ * @rsvd8: Reserved for future use; see the note on reserved space.
+ * @rsvd32: Reserved for future use; see the note on reserved space.
* @rss_config: RX ring/queue index for each hash value i.e., indirection table
* of @indir_size __u32 elements, followed by hash key of @key_size
* bytes.
* @so_timestamping: bit mask of the sum of the supported SO_TIMESTAMPING flags
* @phc_index: device index of the associated PHC, or -1 if there is none
* @tx_types: bit mask of the supported hwtstamp_tx_types enumeration values
+ * @tx_reserved: Reserved for future use; see the note on reserved space.
* @rx_filters: bit mask of the supported hwtstamp_rx_filters enumeration values
+ * @rx_reserved: Reserved for future use; see the note on reserved space.
*
* The bits in the 'tx_types' and 'rx_filters' fields correspond to
* the 'hwtstamp_tx_types' and 'hwtstamp_rx_filters' enumeration values,
* autonegotiation; 0 if unknown or not applicable. Read-only.
* @transceiver: Used to distinguish different possible PHY types,
* reported consistently by PHYLIB. Read-only.
+ * @master_slave_cfg: Master/slave port mode.
+ * @master_slave_state: Master/slave port state.
+ * @reserved: Reserved for future use; see the note on reserved space.
+ * @reserved1: Reserved for future use; see the note on reserved space.
+ * @link_mode_masks: Variable length bitmaps.
*
* If autonegotiation is disabled, the speed and @duplex represent the
* fixed link mode and are writable if the driver supports multiple
};
/* Device ioctls: */
-#define FUSE_DEV_IOC_CLONE _IOR(229, 0, uint32_t)
+#define FUSE_DEV_IOC_MAGIC 229
+#define FUSE_DEV_IOC_CLONE _IOR(FUSE_DEV_IOC_MAGIC, 0, uint32_t)
struct fuse_lseek_in {
uint64_t fh;
#define __UAPI_PSAMPLE_H
enum {
- /* sampled packet metadata */
PSAMPLE_ATTR_IIFINDEX,
PSAMPLE_ATTR_OIFINDEX,
PSAMPLE_ATTR_ORIGSIZE,
PSAMPLE_ATTR_GROUP_SEQ,
PSAMPLE_ATTR_SAMPLE_RATE,
PSAMPLE_ATTR_DATA,
- PSAMPLE_ATTR_TUNNEL,
-
- /* commands attributes */
PSAMPLE_ATTR_GROUP_REFCOUNT,
+ PSAMPLE_ATTR_TUNNEL,
__PSAMPLE_ATTR_MAX
};
* @op: operation code
* @hard: hard state (0/1)
* @soft: soft state (0/1)
+ *
+ * Structure used for userspace communication on /dev/rfkill,
+ * used for events from the kernel and control to the kernel.
+ */
+struct rfkill_event {
+ __u32 idx;
+ __u8 type;
+ __u8 op;
+ __u8 soft;
+ __u8 hard;
+} __attribute__((packed));
+
+/**
+ * struct rfkill_event_ext - events for userspace on /dev/rfkill
+ * @idx: index of dev rfkill
+ * @type: type of the rfkill struct
+ * @op: operation code
+ * @hard: hard state (0/1)
+ * @soft: soft state (0/1)
* @hard_block_reasons: valid if hard is set. One or several reasons from
* &enum rfkill_hard_block_reasons.
*
* Structure used for userspace communication on /dev/rfkill,
* used for events from the kernel and control to the kernel.
+ *
+ * See the extensibility docs below.
*/
-struct rfkill_event {
+struct rfkill_event_ext {
__u32 idx;
__u8 type;
__u8 op;
__u8 soft;
__u8 hard;
+
+ /*
+ * older kernels will accept/send only up to this point,
+ * and if extended further up to any chunk marked below
+ */
+
__u8 hard_block_reasons;
} __attribute__((packed));
-/*
- * We are planning to be backward and forward compatible with changes
- * to the event struct, by adding new, optional, members at the end.
- * When reading an event (whether the kernel from userspace or vice
- * versa) we need to accept anything that's at least as large as the
- * version 1 event size, but might be able to accept other sizes in
- * the future.
+/**
+ * DOC: Extensibility
+ *
+ * Originally, we had planned to allow backward and forward compatible
+ * changes by just adding fields at the end of the structure that are
+ * then not reported on older kernels on read(), and not written to by
+ * older kernels on write(), with the kernel reporting the size it did
+ * accept as the result.
+ *
+ * This would have allowed userspace to detect on read() and write()
+ * which kernel structure version it was dealing with, and if was just
+ * recompiled it would have gotten the new fields, but obviously not
+ * accessed them, but things should've continued to work.
+ *
+ * Unfortunately, while actually exercising this mechanism to add the
+ * hard block reasons field, we found that userspace (notably systemd)
+ * did all kinds of fun things not in line with this scheme:
+ *
+ * 1. treat the (expected) short writes as an error;
+ * 2. ask to read sizeof(struct rfkill_event) but then compare the
+ * actual return value to RFKILL_EVENT_SIZE_V1 and treat any
+ * mismatch as an error.
+ *
+ * As a consequence, just recompiling with a new struct version caused
+ * things to no longer work correctly on old and new kernels.
+ *
+ * Hence, we've rolled back &struct rfkill_event to the original version
+ * and added &struct rfkill_event_ext. This effectively reverts to the
+ * old behaviour for all userspace, unless it explicitly opts in to the
+ * rules outlined here by using the new &struct rfkill_event_ext.
+ *
+ * Userspace using &struct rfkill_event_ext must adhere to the following
+ * rules
*
- * One exception is the kernel -- we already have two event sizes in
- * that we've made the 'hard' member optional since our only option
- * is to ignore it anyway.
+ * 1. accept short writes, optionally using them to detect that it's
+ * running on an older kernel;
+ * 2. accept short reads, knowing that this means it's running on an
+ * older kernel;
+ * 3. treat reads that are as long as requested as acceptable, not
+ * checking against RFKILL_EVENT_SIZE_V1 or such.
*/
-#define RFKILL_EVENT_SIZE_V1 8
+#define RFKILL_EVENT_SIZE_V1 sizeof(struct rfkill_event)
/* ioctl for turning off rfkill-input (if present) */
#define RFKILL_IOC_MAGIC 'R'
fd = *(int *)key;
f = fget_raw(fd);
if (!f)
- return NULL;
+ return ERR_PTR(-EBADF);
sdata = inode_storage_lookup(f->f_inode, map, true);
fput(f);
tprogs[BPF_TRAMP_FENTRY].progs[0] = prog;
tprogs[BPF_TRAMP_FENTRY].nr_progs = 1;
- err = arch_prepare_bpf_trampoline(image,
+ err = arch_prepare_bpf_trampoline(NULL, image,
st_map->image + PAGE_SIZE,
&st_ops->func_models[i], 0,
tprogs, NULL);
}
pure_initcall(bpf_jit_charge_init);
-static int bpf_jit_charge_modmem(u32 pages)
+int bpf_jit_charge_modmem(u32 pages)
{
if (atomic_long_add_return(pages, &bpf_jit_current) >
(bpf_jit_limit >> PAGE_SHIFT)) {
return 0;
}
-static void bpf_jit_uncharge_modmem(u32 pages)
+void bpf_jit_uncharge_modmem(u32 pages)
{
atomic_long_sub(pages, &bpf_jit_current);
}
[BPF_ADD >> 4] = "add",
[BPF_AND >> 4] = "and",
[BPF_OR >> 4] = "or",
- [BPF_XOR >> 4] = "or",
+ [BPF_XOR >> 4] = "xor",
};
static const char *const bpf_ldst_string[] = {
return PTR_ERR(raw);
if (type == BPF_TYPE_PROG)
- ret = bpf_prog_new_fd(raw);
+ ret = (f_flags != O_RDWR) ? -EINVAL : bpf_prog_new_fd(raw);
else if (type == BPF_TYPE_MAP)
ret = bpf_map_new_fd(raw, f_flags);
else if (type == BPF_TYPE_LINK)
- ret = bpf_link_new_fd(raw);
+ ret = (f_flags != O_RDWR) ? -EINVAL : bpf_link_new_fd(raw);
else
return -ENOENT;
&magic, sizeof(magic), &pos);
if (n != sizeof(magic))
return -EPIPE;
+
tgid = umd_ops.info.tgid;
- wait_event(tgid->wait_pidfd, thread_group_exited(tgid));
- umd_ops.info.tgid = NULL;
+ if (tgid) {
+ wait_event(tgid->wait_pidfd, thread_group_exited(tgid));
+ umd_cleanup_helper(&umd_ops.info);
+ }
return 0;
}
static void __exit fini_umd(void)
{
+ struct pid *tgid;
+
bpf_preload_ops = NULL;
+
/* kill UMD in case it's still there due to earlier error */
- kill_pid(umd_ops.info.tgid, SIGKILL, 1);
- umd_ops.info.tgid = NULL;
+ tgid = umd_ops.info.tgid;
+ if (tgid) {
+ kill_pid(tgid, SIGKILL, 1);
+
+ wait_event(tgid->wait_pidfd, thread_group_exited(tgid));
+ umd_cleanup_helper(&umd_ops.info);
+ }
umd_unload_blob(&umd_ops.info);
}
late_initcall(load_umd);
BPF_CALL_4(bpf_get_task_stack, struct task_struct *, task, void *, buf,
u32, size, u64, flags)
{
- struct pt_regs *regs = task_pt_regs(task);
+ struct pt_regs *regs;
+ long res;
- return __bpf_get_stack(regs, task, NULL, buf, size, flags);
+ if (!try_get_task_stack(task))
+ return -EFAULT;
+
+ regs = task_pt_regs(task);
+ res = __bpf_get_stack(regs, task, NULL, buf, size, flags);
+ put_task_stack(task);
+
+ return res;
}
BTF_ID_LIST_SINGLE(bpf_get_task_stack_btf_ids, struct, task_struct)
err = PTR_ERR(btf);
goto free_map;
}
+ if (btf_is_kernel(btf)) {
+ btf_put(btf);
+ err = -EACCES;
+ goto free_map;
+ }
map->btf = btf;
if (attr->btf_value_type_id) {
#include <linux/btf.h>
#include <linux/rcupdate_trace.h>
#include <linux/rcupdate_wait.h>
+#include <linux/module.h>
/* dummy _ops. The verifier will operate on target program's ops. */
const struct bpf_verifier_ops bpf_extension_verifier_ops = {
PAGE_SIZE, true, ksym->name);
}
-static void bpf_trampoline_ksym_add(struct bpf_trampoline *tr)
-{
- struct bpf_ksym *ksym = &tr->ksym;
-
- snprintf(ksym->name, KSYM_NAME_LEN, "bpf_trampoline_%llu", tr->key);
- bpf_image_ksym_add(tr->image, ksym);
-}
-
static struct bpf_trampoline *bpf_trampoline_lookup(u64 key)
{
struct bpf_trampoline *tr;
struct hlist_head *head;
- void *image;
int i;
mutex_lock(&trampoline_mutex);
if (!tr)
goto out;
- /* is_root was checked earlier. No need for bpf_jit_charge_modmem() */
- image = bpf_jit_alloc_exec_page();
- if (!image) {
- kfree(tr);
- tr = NULL;
- goto out;
- }
-
tr->key = key;
INIT_HLIST_NODE(&tr->hlist);
hlist_add_head(&tr->hlist, head);
mutex_init(&tr->mutex);
for (i = 0; i < BPF_TRAMP_MAX; i++)
INIT_HLIST_HEAD(&tr->progs_hlist[i]);
- tr->image = image;
- INIT_LIST_HEAD_RCU(&tr->ksym.lnode);
- bpf_trampoline_ksym_add(tr);
out:
mutex_unlock(&trampoline_mutex);
return tr;
}
+static int bpf_trampoline_module_get(struct bpf_trampoline *tr)
+{
+ struct module *mod;
+ int err = 0;
+
+ preempt_disable();
+ mod = __module_text_address((unsigned long) tr->func.addr);
+ if (mod && !try_module_get(mod))
+ err = -ENOENT;
+ preempt_enable();
+ tr->mod = mod;
+ return err;
+}
+
+static void bpf_trampoline_module_put(struct bpf_trampoline *tr)
+{
+ module_put(tr->mod);
+ tr->mod = NULL;
+}
+
static int is_ftrace_location(void *ip)
{
long addr;
ret = unregister_ftrace_direct((long)ip, (long)old_addr);
else
ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, old_addr, NULL);
+
+ if (!ret)
+ bpf_trampoline_module_put(tr);
return ret;
}
return ret;
tr->func.ftrace_managed = ret;
+ if (bpf_trampoline_module_get(tr))
+ return -ENOENT;
+
if (tr->func.ftrace_managed)
ret = register_ftrace_direct((long)ip, (long)new_addr);
else
ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, NULL, new_addr);
+
+ if (ret)
+ bpf_trampoline_module_put(tr);
return ret;
}
return tprogs;
}
+static void __bpf_tramp_image_put_deferred(struct work_struct *work)
+{
+ struct bpf_tramp_image *im;
+
+ im = container_of(work, struct bpf_tramp_image, work);
+ bpf_image_ksym_del(&im->ksym);
+ bpf_jit_free_exec(im->image);
+ bpf_jit_uncharge_modmem(1);
+ percpu_ref_exit(&im->pcref);
+ kfree_rcu(im, rcu);
+}
+
+/* callback, fexit step 3 or fentry step 2 */
+static void __bpf_tramp_image_put_rcu(struct rcu_head *rcu)
+{
+ struct bpf_tramp_image *im;
+
+ im = container_of(rcu, struct bpf_tramp_image, rcu);
+ INIT_WORK(&im->work, __bpf_tramp_image_put_deferred);
+ schedule_work(&im->work);
+}
+
+/* callback, fexit step 2. Called after percpu_ref_kill confirms. */
+static void __bpf_tramp_image_release(struct percpu_ref *pcref)
+{
+ struct bpf_tramp_image *im;
+
+ im = container_of(pcref, struct bpf_tramp_image, pcref);
+ call_rcu_tasks(&im->rcu, __bpf_tramp_image_put_rcu);
+}
+
+/* callback, fexit or fentry step 1 */
+static void __bpf_tramp_image_put_rcu_tasks(struct rcu_head *rcu)
+{
+ struct bpf_tramp_image *im;
+
+ im = container_of(rcu, struct bpf_tramp_image, rcu);
+ if (im->ip_after_call)
+ /* the case of fmod_ret/fexit trampoline and CONFIG_PREEMPTION=y */
+ percpu_ref_kill(&im->pcref);
+ else
+ /* the case of fentry trampoline */
+ call_rcu_tasks(&im->rcu, __bpf_tramp_image_put_rcu);
+}
+
+static void bpf_tramp_image_put(struct bpf_tramp_image *im)
+{
+ /* The trampoline image that calls original function is using:
+ * rcu_read_lock_trace to protect sleepable bpf progs
+ * rcu_read_lock to protect normal bpf progs
+ * percpu_ref to protect trampoline itself
+ * rcu tasks to protect trampoline asm not covered by percpu_ref
+ * (which are few asm insns before __bpf_tramp_enter and
+ * after __bpf_tramp_exit)
+ *
+ * The trampoline is unreachable before bpf_tramp_image_put().
+ *
+ * First, patch the trampoline to avoid calling into fexit progs.
+ * The progs will be freed even if the original function is still
+ * executing or sleeping.
+ * In case of CONFIG_PREEMPT=y use call_rcu_tasks() to wait on
+ * first few asm instructions to execute and call into
+ * __bpf_tramp_enter->percpu_ref_get.
+ * Then use percpu_ref_kill to wait for the trampoline and the original
+ * function to finish.
+ * Then use call_rcu_tasks() to make sure few asm insns in
+ * the trampoline epilogue are done as well.
+ *
+ * In !PREEMPT case the task that got interrupted in the first asm
+ * insns won't go through an RCU quiescent state which the
+ * percpu_ref_kill will be waiting for. Hence the first
+ * call_rcu_tasks() is not necessary.
+ */
+ if (im->ip_after_call) {
+ int err = bpf_arch_text_poke(im->ip_after_call, BPF_MOD_JUMP,
+ NULL, im->ip_epilogue);
+ WARN_ON(err);
+ if (IS_ENABLED(CONFIG_PREEMPTION))
+ call_rcu_tasks(&im->rcu, __bpf_tramp_image_put_rcu_tasks);
+ else
+ percpu_ref_kill(&im->pcref);
+ return;
+ }
+
+ /* The trampoline without fexit and fmod_ret progs doesn't call original
+ * function and doesn't use percpu_ref.
+ * Use call_rcu_tasks_trace() to wait for sleepable progs to finish.
+ * Then use call_rcu_tasks() to wait for the rest of trampoline asm
+ * and normal progs.
+ */
+ call_rcu_tasks_trace(&im->rcu, __bpf_tramp_image_put_rcu_tasks);
+}
+
+static struct bpf_tramp_image *bpf_tramp_image_alloc(u64 key, u32 idx)
+{
+ struct bpf_tramp_image *im;
+ struct bpf_ksym *ksym;
+ void *image;
+ int err = -ENOMEM;
+
+ im = kzalloc(sizeof(*im), GFP_KERNEL);
+ if (!im)
+ goto out;
+
+ err = bpf_jit_charge_modmem(1);
+ if (err)
+ goto out_free_im;
+
+ err = -ENOMEM;
+ im->image = image = bpf_jit_alloc_exec_page();
+ if (!image)
+ goto out_uncharge;
+
+ err = percpu_ref_init(&im->pcref, __bpf_tramp_image_release, 0, GFP_KERNEL);
+ if (err)
+ goto out_free_image;
+
+ ksym = &im->ksym;
+ INIT_LIST_HEAD_RCU(&ksym->lnode);
+ snprintf(ksym->name, KSYM_NAME_LEN, "bpf_trampoline_%llu_%u", key, idx);
+ bpf_image_ksym_add(image, ksym);
+ return im;
+
+out_free_image:
+ bpf_jit_free_exec(im->image);
+out_uncharge:
+ bpf_jit_uncharge_modmem(1);
+out_free_im:
+ kfree(im);
+out:
+ return ERR_PTR(err);
+}
+
static int bpf_trampoline_update(struct bpf_trampoline *tr)
{
- void *old_image = tr->image + ((tr->selector + 1) & 1) * PAGE_SIZE/2;
- void *new_image = tr->image + (tr->selector & 1) * PAGE_SIZE/2;
+ struct bpf_tramp_image *im;
struct bpf_tramp_progs *tprogs;
u32 flags = BPF_TRAMP_F_RESTORE_REGS;
int err, total;
return PTR_ERR(tprogs);
if (total == 0) {
- err = unregister_fentry(tr, old_image);
+ err = unregister_fentry(tr, tr->cur_image->image);
+ bpf_tramp_image_put(tr->cur_image);
+ tr->cur_image = NULL;
tr->selector = 0;
goto out;
}
+ im = bpf_tramp_image_alloc(tr->key, tr->selector);
+ if (IS_ERR(im)) {
+ err = PTR_ERR(im);
+ goto out;
+ }
+
if (tprogs[BPF_TRAMP_FEXIT].nr_progs ||
tprogs[BPF_TRAMP_MODIFY_RETURN].nr_progs)
flags = BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_SKIP_FRAME;
- /* Though the second half of trampoline page is unused a task could be
- * preempted in the middle of the first half of trampoline and two
- * updates to trampoline would change the code from underneath the
- * preempted task. Hence wait for tasks to voluntarily schedule or go
- * to userspace.
- * The same trampoline can hold both sleepable and non-sleepable progs.
- * synchronize_rcu_tasks_trace() is needed to make sure all sleepable
- * programs finish executing.
- * Wait for these two grace periods together.
- */
- synchronize_rcu_mult(call_rcu_tasks, call_rcu_tasks_trace);
-
- err = arch_prepare_bpf_trampoline(new_image, new_image + PAGE_SIZE / 2,
+ err = arch_prepare_bpf_trampoline(im, im->image, im->image + PAGE_SIZE,
&tr->func.model, flags, tprogs,
tr->func.addr);
if (err < 0)
goto out;
- if (tr->selector)
+ WARN_ON(tr->cur_image && tr->selector == 0);
+ WARN_ON(!tr->cur_image && tr->selector);
+ if (tr->cur_image)
/* progs already running at this address */
- err = modify_fentry(tr, old_image, new_image);
+ err = modify_fentry(tr, tr->cur_image->image, im->image);
else
/* first time registering */
- err = register_fentry(tr, new_image);
+ err = register_fentry(tr, im->image);
if (err)
goto out;
+ if (tr->cur_image)
+ bpf_tramp_image_put(tr->cur_image);
+ tr->cur_image = im;
tr->selector++;
out:
kfree(tprogs);
goto out;
if (WARN_ON_ONCE(!hlist_empty(&tr->progs_hlist[BPF_TRAMP_FEXIT])))
goto out;
- bpf_image_ksym_del(&tr->ksym);
- /* This code will be executed when all bpf progs (both sleepable and
- * non-sleepable) went through
- * bpf_prog_put()->call_rcu[_tasks_trace]()->bpf_prog_free_deferred().
- * Hence no need for another synchronize_rcu_tasks_trace() here,
- * but synchronize_rcu_tasks() is still needed, since trampoline
- * may not have had any sleepable programs and we need to wait
- * for tasks to get out of trampoline code before freeing it.
+ /* This code will be executed even when the last bpf_tramp_image
+ * is alive. All progs are detached from the trampoline and the
+ * trampoline image is patched with jmp into epilogue to skip
+ * fexit progs. The fentry-only trampoline will be freed via
+ * multiple rcu callbacks.
*/
- synchronize_rcu_tasks();
- bpf_jit_free_exec(tr->image);
hlist_del(&tr->hlist);
kfree(tr);
out:
rcu_read_unlock_trace();
}
+void notrace __bpf_tramp_enter(struct bpf_tramp_image *tr)
+{
+ percpu_ref_get(&tr->pcref);
+}
+
+void notrace __bpf_tramp_exit(struct bpf_tramp_image *tr)
+{
+ percpu_ref_put(&tr->pcref);
+}
+
int __weak
-arch_prepare_bpf_trampoline(void *image, void *image_end,
+arch_prepare_bpf_trampoline(struct bpf_tramp_image *tr, void *image, void *image_end,
const struct btf_func_model *m, u32 flags,
struct bpf_tramp_progs *tprogs,
void *orig_call)
{
bool mask_to_left = (opcode == BPF_ADD && off_is_neg) ||
(opcode == BPF_SUB && !off_is_neg);
- u32 off;
+ u32 off, max;
switch (ptr_reg->type) {
case PTR_TO_STACK:
+ /* Offset 0 is out-of-bounds, but acceptable start for the
+ * left direction, see BPF_REG_FP.
+ */
+ max = MAX_BPF_STACK + mask_to_left;
/* Indirect variable offset stack access is prohibited in
* unprivileged mode so it's not handled here.
*/
if (mask_to_left)
*ptr_limit = MAX_BPF_STACK + off;
else
- *ptr_limit = -off;
- return 0;
+ *ptr_limit = -off - 1;
+ return *ptr_limit >= max ? -ERANGE : 0;
case PTR_TO_MAP_VALUE:
+ max = ptr_reg->map_ptr->value_size;
if (mask_to_left) {
*ptr_limit = ptr_reg->umax_value + ptr_reg->off;
} else {
off = ptr_reg->smin_value + ptr_reg->off;
- *ptr_limit = ptr_reg->map_ptr->value_size - off;
+ *ptr_limit = ptr_reg->map_ptr->value_size - off - 1;
}
- return 0;
+ return *ptr_limit >= max ? -ERANGE : 0;
default:
return -EINVAL;
}
u32 alu_state, alu_limit;
struct bpf_reg_state tmp;
bool ret;
+ int err;
if (can_skip_alu_sanitation(env, insn))
return 0;
alu_state |= ptr_is_dst_reg ?
BPF_ALU_SANITIZE_SRC : BPF_ALU_SANITIZE_DST;
- if (retrieve_ptr_limit(ptr_reg, &alu_limit, opcode, off_is_neg))
- return 0;
- if (update_alu_sanitation_state(aux, alu_state, alu_limit))
- return -EACCES;
+ err = retrieve_ptr_limit(ptr_reg, &alu_limit, opcode, off_is_neg);
+ if (err < 0)
+ return err;
+
+ err = update_alu_sanitation_state(aux, alu_state, alu_limit);
+ if (err < 0)
+ return err;
do_sim:
/* Simulate and find potential out-of-bounds access under
* speculative execution from truncation as a result of
case BPF_ADD:
ret = sanitize_ptr_alu(env, insn, ptr_reg, dst_reg, smin_val < 0);
if (ret < 0) {
- verbose(env, "R%d tried to add from different maps or paths\n", dst);
+ verbose(env, "R%d tried to add from different maps, paths, or prohibited types\n", dst);
return ret;
}
/* We can take a fixed offset as long as it doesn't overflow
case BPF_SUB:
ret = sanitize_ptr_alu(env, insn, ptr_reg, dst_reg, smin_val < 0);
if (ret < 0) {
- verbose(env, "R%d tried to sub from different maps or paths\n", dst);
+ verbose(env, "R%d tried to sub from different maps, paths, or prohibited types\n", dst);
return ret;
}
if (dst_reg == off_reg) {
btf = btf_get_by_fd(attr->prog_btf_fd);
if (IS_ERR(btf))
return PTR_ERR(btf);
+ if (btf_is_kernel(btf)) {
+ btf_put(btf);
+ return -EACCES;
+ }
env->prog->aux->btf = btf;
err = check_btf_func(env, attr, uattr);
off_reg = issrc ? insn->src_reg : insn->dst_reg;
if (isneg)
*patch++ = BPF_ALU64_IMM(BPF_MUL, off_reg, -1);
- *patch++ = BPF_MOV32_IMM(BPF_REG_AX, aux->alu_limit - 1);
+ *patch++ = BPF_MOV32_IMM(BPF_REG_AX, aux->alu_limit);
*patch++ = BPF_ALU64_REG(BPF_SUB, BPF_REG_AX, off_reg);
*patch++ = BPF_ALU64_REG(BPF_OR, BPF_REG_AX, off_reg);
*patch++ = BPF_ALU64_IMM(BPF_NEG, BPF_REG_AX, 0);
u32 btf_id, member_idx;
const char *mname;
+ if (!prog->gpl_compatible) {
+ verbose(env, "struct ops programs must have a GPL compatible license\n");
+ return -EINVAL;
+ }
+
btf_id = prog->aux->attach_btf_id;
st_ops = bpf_struct_ops_find(btf_id);
if (!st_ops) {
p = dup_task_struct(current, node);
if (!p)
goto fork_out;
- if (args->io_thread)
+ if (args->io_thread) {
+ /*
+ * Mark us an IO worker, and block any signal that isn't
+ * fatal or STOP
+ */
p->flags |= PF_IO_WORKER;
+ siginitsetinv(&p->blocked, sigmask(SIGKILL)|sigmask(SIGSTOP));
+ }
/*
* This _must_ happen before we call free_task(), i.e. before we jump
.stack_size = (unsigned long)arg,
.io_thread = 1,
};
- struct task_struct *tsk;
- tsk = copy_process(NULL, 0, node, &args);
- if (!IS_ERR(tsk)) {
- sigfillset(&tsk->blocked);
- sigdelsetmask(&tsk->blocked, sigmask(SIGKILL));
- tsk->flags |= PF_NOFREEZE;
- }
- return tsk;
+ return copy_process(NULL, 0, node, &args);
}
/*
goto out;
restart = ¤t->restart_block;
- restart->fn = futex_wait_restart;
restart->futex.uaddr = uaddr;
restart->futex.val = val;
restart->futex.time = *abs_time;
restart->futex.bitset = bitset;
restart->futex.flags = flags | FLAGS_HAS_TIMEOUT;
- ret = -ERESTART_RESTARTBLOCK;
+ ret = set_restart_fn(restart, futex_wait_restart);
out:
if (to) {
u32 ident;
u32 checksum;
+#if CONFIG_CLANG_VERSION < 110000
u8 use_extra_checksum;
+#endif
u32 cfg_checksum;
u32 num_counters;
u64 *counters;
+#if CONFIG_CLANG_VERSION < 110000
const char *function_name;
+#endif
};
static struct gcov_info *current_info;
}
EXPORT_SYMBOL(llvm_gcov_init);
+#if CONFIG_CLANG_VERSION < 110000
void llvm_gcda_start_file(const char *orig_filename, const char version[4],
u32 checksum)
{
current_info->checksum = checksum;
}
EXPORT_SYMBOL(llvm_gcda_start_file);
+#else
+void llvm_gcda_start_file(const char *orig_filename, u32 version, u32 checksum)
+{
+ current_info->filename = orig_filename;
+ current_info->version = version;
+ current_info->checksum = checksum;
+}
+EXPORT_SYMBOL(llvm_gcda_start_file);
+#endif
+#if CONFIG_CLANG_VERSION < 110000
void llvm_gcda_emit_function(u32 ident, const char *function_name,
u32 func_checksum, u8 use_extra_checksum, u32 cfg_checksum)
{
list_add_tail(&info->head, ¤t_info->functions);
}
+#else
+void llvm_gcda_emit_function(u32 ident, u32 func_checksum, u32 cfg_checksum)
+{
+ struct gcov_fn_info *info = kzalloc(sizeof(*info), GFP_KERNEL);
+
+ if (!info)
+ return;
+
+ INIT_LIST_HEAD(&info->head);
+ info->ident = ident;
+ info->checksum = func_checksum;
+ info->cfg_checksum = cfg_checksum;
+ list_add_tail(&info->head, ¤t_info->functions);
+}
+#endif
EXPORT_SYMBOL(llvm_gcda_emit_function);
void llvm_gcda_emit_arcs(u32 num_counters, u64 *counters)
!list_is_last(&fn_ptr2->head, &info2->functions)) {
if (fn_ptr1->checksum != fn_ptr2->checksum)
return false;
+#if CONFIG_CLANG_VERSION < 110000
if (fn_ptr1->use_extra_checksum != fn_ptr2->use_extra_checksum)
return false;
if (fn_ptr1->use_extra_checksum &&
fn_ptr1->cfg_checksum != fn_ptr2->cfg_checksum)
return false;
+#else
+ if (fn_ptr1->cfg_checksum != fn_ptr2->cfg_checksum)
+ return false;
+#endif
fn_ptr1 = list_next_entry(fn_ptr1, head);
fn_ptr2 = list_next_entry(fn_ptr2, head);
}
}
}
+#if CONFIG_CLANG_VERSION < 110000
static struct gcov_fn_info *gcov_fn_info_dup(struct gcov_fn_info *fn)
{
size_t cv_size; /* counter values size */
kfree(fn_dup);
return NULL;
}
+#else
+static struct gcov_fn_info *gcov_fn_info_dup(struct gcov_fn_info *fn)
+{
+ size_t cv_size; /* counter values size */
+ struct gcov_fn_info *fn_dup = kmemdup(fn, sizeof(*fn),
+ GFP_KERNEL);
+ if (!fn_dup)
+ return NULL;
+ INIT_LIST_HEAD(&fn_dup->head);
+
+ cv_size = fn->num_counters * sizeof(fn->counters[0]);
+ fn_dup->counters = vmalloc(cv_size);
+ if (!fn_dup->counters) {
+ kfree(fn_dup);
+ return NULL;
+ }
+
+ memcpy(fn_dup->counters, fn->counters, cv_size);
+
+ return fn_dup;
+}
+#endif
/**
* gcov_info_dup - duplicate profiling data set
* gcov_info_free - release memory for profiling data set duplicate
* @info: profiling data set duplicate to free
*/
+#if CONFIG_CLANG_VERSION < 110000
void gcov_info_free(struct gcov_info *info)
{
struct gcov_fn_info *fn, *tmp;
kfree(info->filename);
kfree(info);
}
+#else
+void gcov_info_free(struct gcov_info *info)
+{
+ struct gcov_fn_info *fn, *tmp;
+
+ list_for_each_entry_safe(fn, tmp, &info->functions, head) {
+ vfree(fn->counters);
+ list_del(&fn->head);
+ kfree(fn);
+ }
+ kfree(info->filename);
+ kfree(info);
+}
+#endif
#define ITER_STRIDE PAGE_SIZE
list_for_each_entry(fi_ptr, &info->functions, head) {
u32 i;
- u32 len = 2;
-
- if (fi_ptr->use_extra_checksum)
- len++;
pos += store_gcov_u32(buffer, pos, GCOV_TAG_FUNCTION);
- pos += store_gcov_u32(buffer, pos, len);
+#if CONFIG_CLANG_VERSION < 110000
+ pos += store_gcov_u32(buffer, pos,
+ fi_ptr->use_extra_checksum ? 3 : 2);
+#else
+ pos += store_gcov_u32(buffer, pos, 3);
+#endif
pos += store_gcov_u32(buffer, pos, fi_ptr->ident);
pos += store_gcov_u32(buffer, pos, fi_ptr->checksum);
+#if CONFIG_CLANG_VERSION < 110000
if (fi_ptr->use_extra_checksum)
pos += store_gcov_u32(buffer, pos, fi_ptr->cfg_checksum);
+#else
+ pos += store_gcov_u32(buffer, pos, fi_ptr->cfg_checksum);
+#endif
pos += store_gcov_u32(buffer, pos, GCOV_TAG_COUNTER_BASE);
pos += store_gcov_u32(buffer, pos, fi_ptr->num_counters * 2);
* irq_domain_create_sim - Create a new interrupt simulator irq_domain and
* allocate a range of dummy interrupts.
*
- * @fnode: struct fwnode_handle to be associated with this domain.
+ * @fwnode: struct fwnode_handle to be associated with this domain.
* @num_irqs: Number of interrupts to allocate.
*
* On success: return a new irq_domain object.
* a managed device.
*
* @dev: Device to initialize the simulator object for.
- * @fnode: struct fwnode_handle to be associated with this domain.
+ * @fwnode: struct fwnode_handle to be associated with this domain.
* @num_irqs: Number of interrupts to allocate
*
* On success: return a new irq_domain object.
irqreturn_t ret;
local_bh_disable();
+ if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+ local_irq_disable();
ret = action->thread_fn(action->irq, action->dev_id);
if (ret == IRQ_HANDLED)
atomic_inc(&desc->threads_handled);
irq_finalize_oneshot(desc, action);
+ if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+ local_irq_enable();
local_bh_enable();
return ret;
}
return false;
if (!kernel_text_address(jump_entry_code(entry))) {
+ /*
+ * This skips patching built-in __exit, which
+ * is part of init_section_contains() but is
+ * not part of kernel_text_address().
+ *
+ * Skipping built-in __exit is fine since it
+ * will never be executed.
+ */
WARN_ONCE(!jump_entry_is_init(entry),
"can't patch jump_label at %pS",
(void *)jump_entry_code(entry));
printk(KERN_CONT " (");
__print_lock_name(class);
- printk(KERN_CONT "){%s}-{%hd:%hd}", usage,
+ printk(KERN_CONT "){%s}-{%d:%d}", usage,
class->wait_type_outer ?: class->wait_type_inner,
class->wait_type_inner);
}
/* Debug-check: all keys must be persistent! */
debug_locks_off();
pr_err("INFO: trying to register non-static key.\n");
- pr_err("the code is fine but needs lockdep annotation.\n");
+ pr_err("The code is fine but needs lockdep annotation, or maybe\n");
+ pr_err("you didn't initialize this object before use?\n");
pr_err("turning off the locking correctness validator.\n");
dump_stack();
return false;
*/
static __always_inline bool
mutex_optimistic_spin(struct mutex *lock, struct ww_acquire_ctx *ww_ctx,
- const bool use_ww_ctx, struct mutex_waiter *waiter)
+ struct mutex_waiter *waiter)
{
if (!waiter) {
/*
#else
static __always_inline bool
mutex_optimistic_spin(struct mutex *lock, struct ww_acquire_ctx *ww_ctx,
- const bool use_ww_ctx, struct mutex_waiter *waiter)
+ struct mutex_waiter *waiter)
{
return false;
}
struct ww_mutex *ww;
int ret;
+ if (!use_ww_ctx)
+ ww_ctx = NULL;
+
might_sleep();
#ifdef CONFIG_DEBUG_MUTEXES
#endif
ww = container_of(lock, struct ww_mutex, base);
- if (use_ww_ctx && ww_ctx) {
+ if (ww_ctx) {
if (unlikely(ww_ctx == READ_ONCE(ww->ctx)))
return -EALREADY;
mutex_acquire_nest(&lock->dep_map, subclass, 0, nest_lock, ip);
if (__mutex_trylock(lock) ||
- mutex_optimistic_spin(lock, ww_ctx, use_ww_ctx, NULL)) {
+ mutex_optimistic_spin(lock, ww_ctx, NULL)) {
/* got the lock, yay! */
lock_acquired(&lock->dep_map, ip);
- if (use_ww_ctx && ww_ctx)
+ if (ww_ctx)
ww_mutex_set_context_fastpath(ww, ww_ctx);
preempt_enable();
return 0;
* After waiting to acquire the wait_lock, try again.
*/
if (__mutex_trylock(lock)) {
- if (use_ww_ctx && ww_ctx)
+ if (ww_ctx)
__ww_mutex_check_waiters(lock, ww_ctx);
goto skip_wait;
goto err;
}
- if (use_ww_ctx && ww_ctx) {
+ if (ww_ctx) {
ret = __ww_mutex_check_kill(lock, &waiter, ww_ctx);
if (ret)
goto err;
* ww_mutex needs to always recheck its position since its waiter
* list is not FIFO ordered.
*/
- if ((use_ww_ctx && ww_ctx) || !first) {
+ if (ww_ctx || !first) {
first = __mutex_waiter_is_first(lock, &waiter);
if (first)
__mutex_set_flag(lock, MUTEX_FLAG_HANDOFF);
* or we must see its unlock and acquire.
*/
if (__mutex_trylock(lock) ||
- (first && mutex_optimistic_spin(lock, ww_ctx, use_ww_ctx, &waiter)))
+ (first && mutex_optimistic_spin(lock, ww_ctx, &waiter)))
break;
spin_lock(&lock->wait_lock);
acquired:
__set_current_state(TASK_RUNNING);
- if (use_ww_ctx && ww_ctx) {
+ if (ww_ctx) {
/*
* Wound-Wait; we stole the lock (!first_waiter), check the
* waiters as anyone might want to wound us.
/* got the lock - cleanup and rejoice! */
lock_acquired(&lock->dep_map, ip);
- if (use_ww_ctx && ww_ctx)
+ if (ww_ctx)
ww_mutex_lock_acquired(ww, ww_ctx);
spin_unlock(&lock->wait_lock);
return 0;
}
-core_initcall(em_debug_init);
+fs_initcall(em_debug_init);
#else /* CONFIG_DEBUG_FS */
static void em_debug_create_pd(struct device *dev) {}
static void em_debug_remove_pd(struct device *dev) {}
audit_ptrace(task);
retval = -EPERM;
- if (unlikely(task->flags & (PF_KTHREAD | PF_IO_WORKER)))
+ if (unlikely(task->flags & PF_KTHREAD))
goto out;
if (same_thread_group(task, current))
goto out;
void kernel_restart(char *cmd)
{
kernel_restart_prepare(cmd);
- if (pm_power_off_prepare)
- pm_power_off_prepare();
migrate_to_reboot_cpu();
syscore_shutdown();
if (!cmd)
return true;
/* Only allow kernel generated signals to this kthread */
- if (unlikely((t->flags & (PF_KTHREAD | PF_IO_WORKER)) &&
+ if (unlikely((t->flags & PF_KTHREAD) &&
(handler == SIG_KTHREAD_KERNEL) && !force))
return true;
/*
* Skip useless siginfo allocation for SIGKILL and kernel threads.
*/
- if ((sig == SIGKILL) || (t->flags & (PF_KTHREAD | PF_IO_WORKER)))
+ if ((sig == SIGKILL) || (t->flags & PF_KTHREAD))
goto out_set;
/*
do_coredump(&ksig->info);
}
+ /*
+ * PF_IO_WORKER threads will catch and exit on fatal signals
+ * themselves. They have cleanup that must be performed, so
+ * we cannot call do_exit() on their behalf.
+ */
+ if (current->flags & PF_IO_WORKER)
+ goto out;
+
/*
* Death signals, no core dump.
*/
/* NOTREACHED */
}
spin_unlock_irq(&sighand->siglock);
-
+out:
ksig->sig = signr;
if (!(ksig->ka.sa.sa_flags & SA_EXPOSE_TAGBITS))
return (void *)((long)site->addr + (long)&site->addr);
}
+static inline unsigned long __static_call_key(const struct static_call_site *site)
+{
+ return (long)site->key + (long)&site->key;
+}
static inline struct static_call_key *static_call_key(const struct static_call_site *site)
{
- return (struct static_call_key *)
- (((long)site->key + (long)&site->key) & ~STATIC_CALL_SITE_FLAGS);
+ return (void *)(__static_call_key(site) & ~STATIC_CALL_SITE_FLAGS);
}
/* These assume the key is word-aligned. */
static inline bool static_call_is_init(struct static_call_site *site)
{
- return ((long)site->key + (long)&site->key) & STATIC_CALL_SITE_INIT;
+ return __static_call_key(site) & STATIC_CALL_SITE_INIT;
}
static inline bool static_call_is_tail(struct static_call_site *site)
{
- return ((long)site->key + (long)&site->key) & STATIC_CALL_SITE_TAIL;
+ return __static_call_key(site) & STATIC_CALL_SITE_TAIL;
}
static inline void static_call_set_init(struct static_call_site *site)
{
- site->key = ((long)static_call_key(site) | STATIC_CALL_SITE_INIT) -
+ site->key = (__static_call_key(site) | STATIC_CALL_SITE_INIT) -
(long)&site->key;
}
};
for (site_mod = &first; site_mod; site_mod = site_mod->next) {
+ bool init = system_state < SYSTEM_RUNNING;
struct module *mod = site_mod->mod;
if (!site_mod->sites) {
if (mod) {
stop = mod->static_call_sites +
mod->num_static_call_sites;
+ init = mod->state == MODULE_STATE_COMING;
}
#endif
site < stop && static_call_key(site) == key; site++) {
void *site_addr = static_call_addr(site);
- if (static_call_is_init(site)) {
- /*
- * Don't write to call sites which were in
- * initmem and have since been freed.
- */
- if (!mod && system_state >= SYSTEM_RUNNING)
- continue;
- if (mod && !within_module_init((unsigned long)site_addr, mod))
- continue;
- }
+ if (!init && static_call_is_init(site))
+ continue;
if (!kernel_text_address((unsigned long)site_addr)) {
- WARN_ONCE(1, "can't patch static call site at %pS",
+ /*
+ * This skips patching built-in __exit, which
+ * is part of init_section_contains() but is
+ * not part of kernel_text_address().
+ *
+ * Skipping built-in __exit is fine since it
+ * will never be executed.
+ */
+ WARN_ONCE(!static_call_is_init(site),
+ "can't patch static call site at %pS",
site_addr);
continue;
}
arch_static_call_transform(site_addr, NULL, func,
- static_call_is_tail(site));
+ static_call_is_tail(site));
}
}
struct static_call_site *site;
for (site = start; site != stop; site++) {
- unsigned long s_key = (long)site->key + (long)&site->key;
+ unsigned long s_key = __static_call_key(site);
unsigned long addr = s_key & ~STATIC_CALL_SITE_FLAGS;
unsigned long key;
if (flags == TIMER_ABSTIME)
return -ERESTARTNOHAND;
- restart->fn = alarm_timer_nsleep_restart;
restart->nanosleep.clockid = type;
restart->nanosleep.expires = exp;
+ set_restart_fn(restart, alarm_timer_nsleep_restart);
return ret;
}
}
restart = ¤t->restart_block;
- restart->fn = hrtimer_nanosleep_restart;
restart->nanosleep.clockid = t.timer.base->clockid;
restart->nanosleep.expires = hrtimer_get_expires_tv64(&t.timer);
+ set_restart_fn(restart, hrtimer_nanosleep_restart);
out:
destroy_hrtimer_on_stack(&t.timer);
return ret;
if (flags & TIMER_ABSTIME)
return -ERESTARTNOHAND;
- restart_block->fn = posix_cpu_nsleep_restart;
restart_block->nanosleep.clockid = which_clock;
+ set_restart_fn(restart_block, posix_cpu_nsleep_restart);
}
return error;
}
pg = start_pg;
while (pg) {
order = get_count_order(pg->size / ENTRIES_PER_PAGE);
- free_pages((unsigned long)pg->records, order);
+ if (order >= 0)
+ free_pages((unsigned long)pg->records, order);
start_pg = pg->next;
kfree(pg);
pg = start_pg;
return NULL;
}
+static struct ftrace_direct_func *ftrace_alloc_direct_func(unsigned long addr)
+{
+ struct ftrace_direct_func *direct;
+
+ direct = kmalloc(sizeof(*direct), GFP_KERNEL);
+ if (!direct)
+ return NULL;
+ direct->addr = addr;
+ direct->count = 0;
+ list_add_rcu(&direct->next, &ftrace_direct_funcs);
+ ftrace_direct_func_count++;
+ return direct;
+}
+
/**
* register_ftrace_direct - Call a custom trampoline directly
* @ip: The address of the nop at the beginning of a function
direct = ftrace_find_direct_func(addr);
if (!direct) {
- direct = kmalloc(sizeof(*direct), GFP_KERNEL);
+ direct = ftrace_alloc_direct_func(addr);
if (!direct) {
kfree(entry);
goto out_unlock;
}
- direct->addr = addr;
- direct->count = 0;
- list_add_rcu(&direct->next, &ftrace_direct_funcs);
- ftrace_direct_func_count++;
}
entry->ip = ip;
int modify_ftrace_direct(unsigned long ip,
unsigned long old_addr, unsigned long new_addr)
{
+ struct ftrace_direct_func *direct, *new_direct = NULL;
struct ftrace_func_entry *entry;
struct dyn_ftrace *rec;
int ret = -ENODEV;
if (entry->direct != old_addr)
goto out_unlock;
+ direct = ftrace_find_direct_func(old_addr);
+ if (WARN_ON(!direct))
+ goto out_unlock;
+ if (direct->count > 1) {
+ ret = -ENOMEM;
+ new_direct = ftrace_alloc_direct_func(new_addr);
+ if (!new_direct)
+ goto out_unlock;
+ direct->count--;
+ new_direct->count++;
+ } else {
+ direct->addr = new_addr;
+ }
+
/*
* If there's no other ftrace callback on the rec->ip location,
* then it can be changed directly by the architecture.
ret = 0;
}
+ if (unlikely(ret && new_direct)) {
+ direct->count++;
+ list_del_rcu(&new_direct->next);
+ synchronize_rcu_tasks();
+ kfree(new_direct);
+ ftrace_direct_func_count--;
+ }
+
out_unlock:
mutex_unlock(&ftrace_lock);
mutex_unlock(&direct_mutex);
clear_mod_from_hashes(pg);
order = get_count_order(pg->size / ENTRIES_PER_PAGE);
- free_pages((unsigned long)pg->records, order);
+ if (order >= 0)
+ free_pages((unsigned long)pg->records, order);
tmp_page = pg->next;
kfree(pg);
ftrace_number_of_pages -= 1 << order;
if (!pg->index) {
*last_pg = pg->next;
order = get_count_order(pg->size / ENTRIES_PER_PAGE);
- free_pages((unsigned long)pg->records, order);
+ if (order >= 0)
+ free_pages((unsigned long)pg->records, order);
ftrace_number_of_pages -= 1 << order;
ftrace_number_of_groups--;
kfree(pg);
size = nr_entries * sizeof(unsigned long);
event = __trace_buffer_lock_reserve(buffer, TRACE_STACK,
- sizeof(*entry) + size, trace_ctx);
+ (sizeof(*entry) - sizeof(entry->caller)) + size,
+ trace_ctx);
if (!event)
goto out;
entry = ring_buffer_event_data(event);
struct umd_info *umd_info = info->data;
/* cleanup if umh_setup() was successful but exec failed */
- if (info->retval) {
- fput(umd_info->pipe_to_umh);
- fput(umd_info->pipe_from_umh);
- put_pid(umd_info->tgid);
- umd_info->tgid = NULL;
- }
+ if (info->retval)
+ umd_cleanup_helper(umd_info);
+}
+
+/**
+ * umd_cleanup_helper - release the resources which were allocated in umd_setup
+ * @info: information about usermode driver
+ */
+void umd_cleanup_helper(struct umd_info *info)
+{
+ fput(info->pipe_to_umh);
+ fput(info->pipe_from_umh);
+ put_pid(info->tgid);
+ info->tgid = NULL;
}
+EXPORT_SYMBOL_GPL(umd_cleanup_helper);
/**
* fork_usermode_driver - fork a usermode driver
* update as well, the only side effect might be a cycle delay for
* the softlockup check.
*/
- for_each_cpu(cpu, &watchdog_allowed_mask)
+ for_each_cpu(cpu, &watchdog_allowed_mask) {
per_cpu(watchdog_touch_ts, cpu) = SOFTLOCKUP_RESET;
- wq_watchdog_touch(-1);
+ wq_watchdog_touch(cpu);
+ }
}
void touch_softlockup_watchdog_sync(void)
*/
lockdep_assert_irqs_disabled();
- debug_work_activate(work);
/* if draining, only works from the same workqueue are allowed */
if (unlikely(wq->flags & __WQ_DRAINING) &&
worklist = &pwq->delayed_works;
}
+ debug_work_activate(work);
insert_work(pwq, work, worklist, work_flags);
out:
continue;
/* get the latest of pool and touched timestamps */
+ if (pool->cpu >= 0)
+ touched = READ_ONCE(per_cpu(wq_watchdog_touched_cpu, pool->cpu));
+ else
+ touched = READ_ONCE(wq_watchdog_touched);
pool_ts = READ_ONCE(pool->watchdog_ts);
- touched = READ_ONCE(wq_watchdog_touched);
if (time_after(pool_ts, touched))
ts = pool_ts;
else
ts = touched;
- if (pool->cpu >= 0) {
- unsigned long cpu_touched =
- READ_ONCE(per_cpu(wq_watchdog_touched_cpu,
- pool->cpu));
- if (time_after(cpu_touched, ts))
- ts = cpu_touched;
- }
-
/* did we stall? */
if (time_after(jiffies, ts + thresh)) {
lockup_detected = true;
{
if (cpu >= 0)
per_cpu(wq_watchdog_touched_cpu, cpu) = jiffies;
- else
- wq_watchdog_touched = jiffies;
+
+ wq_watchdog_touched = jiffies;
}
static void wq_watchdog_set_thresh(unsigned long thresh)
bool
depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT
select STACKTRACE
- select FRAME_POINTER if !MIPS && !PPC && !ARM && !S390 && !MICROBLAZE && !ARC && !X86
+ depends on FRAME_POINTER || MIPS || PPC || S390 || MICROBLAZE || ARM || ARC || X86
select KALLSYMS
select KALLSYMS_ALL
depends on DEBUG_KERNEL
depends on STACKTRACE_SUPPORT
depends on PROC_FS
- select FRAME_POINTER if !MIPS && !PPC && !S390 && !MICROBLAZE && !ARM && !ARC && !X86
+ depends on FRAME_POINTER || MIPS || PPC || S390 || MICROBLAZE || ARM || ARC || X86
select KALLSYMS
select KALLSYMS_ALL
select STACKTRACE
depends on FAULT_INJECTION_DEBUG_FS && STACKTRACE_SUPPORT
depends on !X86_64
select STACKTRACE
- select FRAME_POINTER if !MIPS && !PPC && !S390 && !MICROBLAZE && !ARM && !ARC && !X86
+ depends on FRAME_POINTER || MIPS || PPC || S390 || MICROBLAZE || ARM || ARC || X86
help
Provide stacktrace filter for fault-injection capabilities
return res + div64_u64(a * b, c);
}
+EXPORT_SYMBOL(mul_u64_u64_div_u64);
#endif
char *kmem;
char __user *usermem;
size_t size = 10;
- int unused;
+ int __maybe_unused unused;
kmem = kmalloc(size, GFP_KERNEL);
if (!kmem)
#ifdef CONFIG_XARRAY_MULTI
static void check_split_1(struct xarray *xa, unsigned long index,
- unsigned int order)
+ unsigned int order, unsigned int new_order)
{
- XA_STATE(xas, xa, index);
- void *entry;
- unsigned int i = 0;
+ XA_STATE_ORDER(xas, xa, index, new_order);
+ unsigned int i;
xa_store_order(xa, index, order, xa, GFP_KERNEL);
xas_split_alloc(&xas, xa, order, GFP_KERNEL);
xas_lock(&xas);
xas_split(&xas, xa, order);
+ for (i = 0; i < (1 << order); i += (1 << new_order))
+ __xa_store(xa, index + i, xa_mk_index(index + i), 0);
xas_unlock(&xas);
- xa_for_each(xa, index, entry) {
- XA_BUG_ON(xa, entry != xa);
- i++;
+ for (i = 0; i < (1 << order); i++) {
+ unsigned int val = index + (i & ~((1 << new_order) - 1));
+ XA_BUG_ON(xa, xa_load(xa, index + i) != xa_mk_index(val));
}
- XA_BUG_ON(xa, i != 1 << order);
xa_set_mark(xa, index, XA_MARK_0);
XA_BUG_ON(xa, !xa_get_mark(xa, index, XA_MARK_0));
static noinline void check_split(struct xarray *xa)
{
- unsigned int order;
+ unsigned int order, new_order;
XA_BUG_ON(xa, !xa_empty(xa));
for (order = 1; order < 2 * XA_CHUNK_SHIFT; order++) {
- check_split_1(xa, 0, order);
- check_split_1(xa, 1UL << order, order);
- check_split_1(xa, 3UL << order, order);
+ for (new_order = 0; new_order < order; new_order++) {
+ check_split_1(xa, 0, order, new_order);
+ check_split_1(xa, 1UL << order, order, new_order);
+ check_split_1(xa, 3UL << order, order, new_order);
+ }
}
}
#else
* xas_split_alloc() - Allocate memory for splitting an entry.
* @xas: XArray operation state.
* @entry: New entry which will be stored in the array.
- * @order: New entry order.
+ * @order: Current entry order.
* @gfp: Memory allocation flags.
*
* This function should be called before calling xas_split().
do {
unsigned int i;
- void *sibling;
+ void *sibling = NULL;
struct xa_node *node;
node = kmem_cache_alloc(radix_tree_node_cachep, gfp);
for (i = 0; i < XA_CHUNK_SIZE; i++) {
if ((i & mask) == 0) {
RCU_INIT_POINTER(node->slots[i], entry);
- sibling = xa_mk_sibling(0);
+ sibling = xa_mk_sibling(i);
} else {
RCU_INIT_POINTER(node->slots[i], sibling);
}
* xas_split() - Split a multi-index entry into smaller entries.
* @xas: XArray operation state.
* @entry: New entry to store in the array.
- * @order: New entry order.
+ * @order: Current entry order.
*
- * The value in the entry is copied to all the replacement entries.
+ * The size of the new entries is set in @xas. The value in @entry is
+ * copied to all the replacement entries.
*
* Context: Any context. The caller should hold the xa_lock.
*/
FOLL_FORCE | FOLL_DUMP | FOLL_GET);
if (locked)
mmap_read_unlock(mm);
+
+ if (ret == 1 && is_page_poisoned(page))
+ return NULL;
+
return (ret == 1) ? page : NULL;
}
#endif /* CONFIG_ELF_CORE */
int idx;
/* With debug all even slots are unmapped and act as guard */
- if (IS_ENABLED(CONFIG_DEBUG_HIGHMEM) && !(i & 0x01)) {
+ if (IS_ENABLED(CONFIG_DEBUG_KMAP_LOCAL) && !(i & 0x01)) {
WARN_ON_ONCE(!pte_none(pteval));
continue;
}
int idx;
/* With debug all even slots are unmapped and act as guard */
- if (IS_ENABLED(CONFIG_DEBUG_HIGHMEM) && !(i & 0x01)) {
+ if (IS_ENABLED(CONFIG_DEBUG_KMAP_LOCAL) && !(i & 0x01)) {
WARN_ON_ONCE(!pte_none(pteval));
continue;
}
nrg->reservation_counter =
&h_cg->rsvd_hugepage[hstate_index(h)];
nrg->css = &h_cg->css;
+ /*
+ * The caller will hold exactly one h_cg->css reference for the
+ * whole contiguous reservation region. But this area might be
+ * scattered when there are already some file_regions reside in
+ * it. As a result, many file_regions may share only one css
+ * reference. In order to ensure that one file_region must hold
+ * exactly one h_cg->css reference, we should do css_get for
+ * each file_region and leave the reference held by caller
+ * untouched.
+ */
+ css_get(&h_cg->css);
if (!resv->pages_per_hpage)
resv->pages_per_hpage = pages_per_huge_page(h);
/* pages_per_hpage should be the same for all entries in
#endif
}
+static void put_uncharge_info(struct file_region *rg)
+{
+#ifdef CONFIG_CGROUP_HUGETLB
+ if (rg->css)
+ css_put(rg->css);
+#endif
+}
+
static bool has_same_uncharge_info(struct file_region *rg,
struct file_region *org)
{
prg->to = rg->to;
list_del(&rg->link);
+ put_uncharge_info(rg);
kfree(rg);
rg = prg;
nrg->from = rg->from;
list_del(&rg->link);
+ put_uncharge_info(rg);
kfree(rg);
}
}
del += t - f;
hugetlb_cgroup_uncharge_file_region(
- resv, rg, t - f);
+ resv, rg, t - f, false);
/* New entry for end of split region */
nrg->from = t;
if (f <= rg->from && t >= rg->to) { /* Remove entire region */
del += rg->to - rg->from;
hugetlb_cgroup_uncharge_file_region(resv, rg,
- rg->to - rg->from);
+ rg->to - rg->from, true);
list_del(&rg->link);
kfree(rg);
continue;
if (f <= rg->from) { /* Trim beginning of region */
hugetlb_cgroup_uncharge_file_region(resv, rg,
- t - rg->from);
+ t - rg->from, false);
del += t - rg->from;
rg->from = t;
} else { /* Trim end of region */
hugetlb_cgroup_uncharge_file_region(resv, rg,
- rg->to - f);
+ rg->to - f, false);
del += rg->to - f;
rg->to = f;
*/
long rsv_adjust;
+ /*
+ * hugetlb_cgroup_uncharge_cgroup_rsvd() will put the
+ * reference to h_cg->css. See comment below for detail.
+ */
hugetlb_cgroup_uncharge_cgroup_rsvd(
hstate_index(h),
(chg - add) * pages_per_huge_page(h), h_cg);
rsv_adjust = hugepage_subpool_put_pages(spool,
chg - add);
hugetlb_acct_memory(h, -rsv_adjust);
+ } else if (h_cg) {
+ /*
+ * The file_regions will hold their own reference to
+ * h_cg->css. So we should release the reference held
+ * via hugetlb_cgroup_charge_cgroup_rsvd() when we are
+ * done.
+ */
+ hugetlb_cgroup_put_rsvd_cgroup(h_cg);
}
}
return true;
void hugetlb_cgroup_uncharge_file_region(struct resv_map *resv,
struct file_region *rg,
- unsigned long nr_pages)
+ unsigned long nr_pages,
+ bool region_del)
{
if (hugetlb_cgroup_disabled() || !resv || !rg || !nr_pages)
return;
!resv->reservation_counter) {
page_counter_uncharge(rg->reservation_counter,
nr_pages * resv->pages_per_hpage);
- css_put(rg->css);
+ /*
+ * Only do css_put(rg->css) when we delete the entire region
+ * because one file_region must hold exactly one css reference.
+ */
+ if (region_del)
+ css_put(rg->css);
}
}
set_page_count(page, 1);
}
+/*
+ * When kernel touch the user page, the user page may be have been marked
+ * poison but still mapped in user space, if without this page, the kernel
+ * can guarantee the data integrity and operation success, the kernel is
+ * better to check the posion status and avoid touching it, be good not to
+ * panic, coredump for process fatal signal is a sample case matching this
+ * scenario. Or if kernel can't guarantee the data integrity, it's better
+ * not to call this function, let kernel touch the poison page and get to
+ * panic.
+ */
+static inline bool is_page_poisoned(struct page *page)
+{
+ if (PageHWPoison(page))
+ return true;
+ else if (PageHuge(page) && PageHWPoison(compound_head(page)))
+ return true;
+
+ return false;
+}
+
extern unsigned long highest_memmap_pfn;
/*
#include <linux/debugfs.h>
#include <linux/kcsan-checks.h>
#include <linux/kfence.h>
+#include <linux/kmemleak.h>
#include <linux/list.h>
#include <linux/lockdep.h>
#include <linux/memblock.h>
addr += 2 * PAGE_SIZE;
}
+ /*
+ * The pool is live and will never be deallocated from this point on.
+ * Remove the pool object from the kmemleak object tree, as it would
+ * otherwise overlap with allocations returned by kfence_alloc(), which
+ * are registered with kmemleak through the slab post-alloc hook.
+ */
+ kmemleak_free(__kfence_pool);
+
return true;
err:
#include <linux/atomic.h>
#include <linux/kasan.h>
+#include <linux/kfence.h>
#include <linux/kmemleak.h>
#include <linux/memory_hotplug.h>
atomic_set(&object->use_count, 1);
object->flags = OBJECT_ALLOCATED;
object->pointer = ptr;
- object->size = size;
+ object->size = kfence_ksize((void *)ptr) ?: size;
object->excess_ref = 0;
object->min_count = min_count;
object->count = 0; /* white color initially */
zero_pfn = page_to_pfn(ZERO_PAGE(0));
return 0;
}
-core_initcall(init_zero_pfn);
+early_initcall(init_zero_pfn);
void mm_trace_rss_stat(struct mm_struct *mm, int member, long count)
{
"");
WARN_ON(mmu_notifier_range_blockable(range) ||
_ret != -EAGAIN);
+ /*
+ * We call all the notifiers on any EAGAIN,
+ * there is no way for a notifier to know if
+ * its start method failed, thus a start that
+ * does EAGAIN can't also do end.
+ */
+ WARN_ON(ops->invalidate_range_end);
ret = _ret;
}
}
}
+
+ if (ret) {
+ /*
+ * Must be non-blocking to get here. If there are multiple
+ * notifiers and one or more failed start, any that succeeded
+ * start are expecting their end to be called. Do so now.
+ */
+ hlist_for_each_entry_rcu(subscription, &subscriptions->list,
+ hlist, srcu_read_lock_held(&srcu)) {
+ if (!subscription->ops->invalidate_range_end)
+ continue;
+
+ subscription->ops->invalidate_range_end(subscription,
+ range);
+ }
+ }
srcu_read_unlock(&srcu, id);
return ret;
}
EXPORT_SYMBOL_GPL(wait_on_page_writeback);
+/*
+ * Wait for a page to complete writeback. Returns -EINTR if we get a
+ * fatal signal while waiting.
+ */
+int wait_on_page_writeback_killable(struct page *page)
+{
+ while (PageWriteback(page)) {
+ trace_wait_on_page_writeback(page, page_mapping(page));
+ if (wait_on_page_bit_killable(page, PG_writeback))
+ return -EINTR;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(wait_on_page_writeback_killable);
+
/**
* wait_for_stable_page() - wait for writeback to finish, if necessary.
* @page: The page to wait on.
void *addr;
addr = kmap_atomic(page);
+ kasan_disable_current();
/*
* Page poisoning when enabled poisons each and every page
* that is freed to buddy. Thus no extra check is done to
* see if a page was poisoned.
*/
- check_poison_mem(addr, PAGE_SIZE);
+ check_poison_mem(kasan_reset_tag(addr), PAGE_SIZE);
+ kasan_enable_current();
kunmap_atomic(addr);
}
extern struct list_head *pcpu_chunk_lists;
extern int pcpu_nr_slots;
-extern int pcpu_nr_empty_pop_pages;
+extern int pcpu_nr_empty_pop_pages[];
extern struct pcpu_chunk *pcpu_first_chunk;
extern struct pcpu_chunk *pcpu_reserved_chunk;
int slot, max_nr_alloc;
int *buffer;
enum pcpu_chunk_type type;
+ int nr_empty_pop_pages;
alloc_buffer:
spin_lock_irq(&pcpu_lock);
goto alloc_buffer;
}
-#define PL(X) \
+ nr_empty_pop_pages = 0;
+ for (type = 0; type < PCPU_NR_CHUNK_TYPES; type++)
+ nr_empty_pop_pages += pcpu_nr_empty_pop_pages[type];
+
+#define PL(X) \
seq_printf(m, " %-20s: %12lld\n", #X, (long long int)pcpu_stats_ai.X)
seq_printf(m,
PU(nr_max_chunks);
PU(min_alloc_size);
PU(max_alloc_size);
- P("empty_pop_pages", pcpu_nr_empty_pop_pages);
+ P("empty_pop_pages", nr_empty_pop_pages);
seq_putc(m, '\n');
#undef PU
static LIST_HEAD(pcpu_map_extend_chunks);
/*
- * The number of empty populated pages, protected by pcpu_lock. The
- * reserved chunk doesn't contribute to the count.
+ * The number of empty populated pages by chunk type, protected by pcpu_lock.
+ * The reserved chunk doesn't contribute to the count.
*/
-int pcpu_nr_empty_pop_pages;
+int pcpu_nr_empty_pop_pages[PCPU_NR_CHUNK_TYPES];
/*
* The number of populated pages in use by the allocator, protected by
{
chunk->nr_empty_pop_pages += nr;
if (chunk != pcpu_reserved_chunk)
- pcpu_nr_empty_pop_pages += nr;
+ pcpu_nr_empty_pop_pages[pcpu_chunk_type(chunk)] += nr;
}
/*
mutex_unlock(&pcpu_alloc_mutex);
}
- if (pcpu_nr_empty_pop_pages < PCPU_EMPTY_POP_PAGES_LOW)
+ if (pcpu_nr_empty_pop_pages[type] < PCPU_EMPTY_POP_PAGES_LOW)
pcpu_schedule_balance_work();
/* clear the areas and return address relative to base address */
pcpu_atomic_alloc_failed = false;
} else {
nr_to_pop = clamp(PCPU_EMPTY_POP_PAGES_HIGH -
- pcpu_nr_empty_pop_pages,
+ pcpu_nr_empty_pop_pages[type],
0, PCPU_EMPTY_POP_PAGES_HIGH);
}
/* link the first chunk in */
pcpu_first_chunk = chunk;
- pcpu_nr_empty_pop_pages = pcpu_first_chunk->nr_empty_pop_pages;
+ pcpu_nr_empty_pop_pages[PCPU_CHUNK_ROOT] = pcpu_first_chunk->nr_empty_pop_pages;
pcpu_chunk_relocate(pcpu_first_chunk, -1);
/* include all regions of the first chunk */
page = list_entry(pos, struct page, lru);
zhdr = page_address(page);
- if (test_bit(PAGE_HEADLESS, &page->private))
+ if (test_bit(PAGE_HEADLESS, &page->private)) {
+ /*
+ * For non-headless pages, we wait to do this
+ * until we have the page lock to avoid racing
+ * with __z3fold_alloc(). Headless pages don't
+ * have a lock (and __z3fold_alloc() will never
+ * see them), but we still need to test and set
+ * PAGE_CLAIMED to avoid racing with
+ * z3fold_free(), so just do it now before
+ * leaving the loop.
+ */
+ if (test_and_set_bit(PAGE_CLAIMED, &page->private))
+ continue;
+
break;
+ }
if (kref_get_unless_zero(&zhdr->refcount) == 0) {
zhdr = NULL;
MODULE_AUTHOR(BATADV_DRIVER_AUTHOR);
MODULE_DESCRIPTION(BATADV_DRIVER_DESC);
-MODULE_SUPPORTED_DEVICE(BATADV_DRIVER_DEVICE);
MODULE_VERSION(BATADV_SOURCE_VERSION);
MODULE_ALIAS_RTNL_LINK("batadv");
MODULE_ALIAS_GENL_FAMILY(BATADV_NL_NAME);
hlist_for_each_entry(vlan, &orig_node->vlan_list, list) {
tt_vlan->vid = htons(vlan->vid);
tt_vlan->crc = htonl(vlan->tt.crc);
+ tt_vlan->reserved = 0;
tt_vlan++;
}
tt_vlan->vid = htons(vlan->vid);
tt_vlan->crc = htonl(vlan->tt.crc);
+ tt_vlan->reserved = 0;
tt_vlan++;
}
{
if (!fdb->dst)
return;
+ if (test_bit(BR_FDB_LOCAL, &fdb->flags))
+ return;
switch (type) {
case RTM_DELNEIGH:
MODULE_AUTHOR("Oliver Hartkopp <oliver.hartkopp@volkswagen.de>");
MODULE_ALIAS("can-proto-2");
+#define BCM_MIN_NAMELEN CAN_REQUIRED_SIZE(struct sockaddr_can, can_ifindex)
+
/*
* easy access to the first 64 bit of can(fd)_frame payload. cp->data is
* 64 bit aligned so the offset has to be multiples of 8 which is ensured
/* no bound device as default => check msg_name */
DECLARE_SOCKADDR(struct sockaddr_can *, addr, msg->msg_name);
- if (msg->msg_namelen < CAN_REQUIRED_SIZE(*addr, can_ifindex))
+ if (msg->msg_namelen < BCM_MIN_NAMELEN)
return -EINVAL;
if (addr->can_family != AF_CAN)
struct net *net = sock_net(sk);
int ret = 0;
- if (len < CAN_REQUIRED_SIZE(*addr, can_ifindex))
+ if (len < BCM_MIN_NAMELEN)
return -EINVAL;
lock_sock(sk);
sock_recv_ts_and_drops(msg, sk, skb);
if (msg->msg_name) {
- __sockaddr_check_size(sizeof(struct sockaddr_can));
- msg->msg_namelen = sizeof(struct sockaddr_can);
+ __sockaddr_check_size(BCM_MIN_NAMELEN);
+ msg->msg_namelen = BCM_MIN_NAMELEN;
memcpy(msg->msg_name, skb->cb, msg->msg_namelen);
}
MODULE_AUTHOR("Oliver Hartkopp <socketcan@hartkopp.net>");
MODULE_ALIAS("can-proto-6");
+#define ISOTP_MIN_NAMELEN CAN_REQUIRED_SIZE(struct sockaddr_can, can_addr.tp)
+
#define SINGLE_MASK(id) (((id) & CAN_EFF_FLAG) ? \
(CAN_EFF_MASK | CAN_EFF_FLAG | CAN_RTR_FLAG) : \
(CAN_SFF_MASK | CAN_EFF_FLAG | CAN_RTR_FLAG))
nskb->dev = dev;
can_skb_set_owner(nskb, sk);
ncf = (struct canfd_frame *)nskb->data;
- skb_put(nskb, so->ll.mtu);
+ skb_put_zero(nskb, so->ll.mtu);
/* create & send flow control reply */
ncf->can_id = so->txid;
if (ae)
ncf->data[0] = so->opt.ext_address;
- if (so->ll.mtu == CANFD_MTU)
- ncf->flags = so->ll.tx_flags;
+ ncf->flags = so->ll.tx_flags;
can_send_ret = can_send(nskb, 1);
if (can_send_ret)
can_skb_prv(skb)->skbcnt = 0;
cf = (struct canfd_frame *)skb->data;
- skb_put(skb, so->ll.mtu);
+ skb_put_zero(skb, so->ll.mtu);
/* create consecutive frame */
isotp_fill_dataframe(cf, so, ae, 0);
so->tx.sn %= 16;
so->tx.bs++;
- if (so->ll.mtu == CANFD_MTU)
- cf->flags = so->ll.tx_flags;
+ cf->flags = so->ll.tx_flags;
skb->dev = dev;
can_skb_set_owner(skb, sk);
so->tx.idx = 0;
cf = (struct canfd_frame *)skb->data;
- skb_put(skb, so->ll.mtu);
+ skb_put_zero(skb, so->ll.mtu);
/* check for single frame transmission depending on TX_DL */
if (size <= so->tx.ll_dl - SF_PCI_SZ4 - ae - off) {
}
/* send the first or only CAN frame */
- if (so->ll.mtu == CANFD_MTU)
- cf->flags = so->ll.tx_flags;
+ cf->flags = so->ll.tx_flags;
skb->dev = dev;
skb->sk = sk;
sock_recv_timestamp(msg, sk, skb);
if (msg->msg_name) {
- msg->msg_namelen = sizeof(struct sockaddr_can);
+ __sockaddr_check_size(ISOTP_MIN_NAMELEN);
+ msg->msg_namelen = ISOTP_MIN_NAMELEN;
memcpy(msg->msg_name, skb->cb, msg->msg_namelen);
}
int notify_enetdown = 0;
int do_rx_reg = 1;
- if (len < CAN_REQUIRED_SIZE(struct sockaddr_can, can_addr.tp))
+ if (len < ISOTP_MIN_NAMELEN)
return -EINVAL;
/* do not register frame reception for functional addressing */
if (peer)
return -EOPNOTSUPP;
- memset(addr, 0, sizeof(*addr));
+ memset(addr, 0, ISOTP_MIN_NAMELEN);
addr->can_family = AF_CAN;
addr->can_ifindex = so->ifindex;
addr->can_addr.tp.rx_id = so->rxid;
addr->can_addr.tp.tx_id = so->txid;
- return sizeof(*addr);
+ return ISOTP_MIN_NAMELEN;
}
static int isotp_setsockopt(struct socket *sock, int level, int optname,
if (ll.mtu != CAN_MTU && ll.mtu != CANFD_MTU)
return -EINVAL;
- if (ll.mtu == CAN_MTU && ll.tx_dl > CAN_MAX_DLEN)
+ if (ll.mtu == CAN_MTU &&
+ (ll.tx_dl > CAN_MAX_DLEN || ll.tx_flags != 0))
return -EINVAL;
memcpy(&so->ll, &ll, sizeof(ll));
MODULE_AUTHOR("Urs Thuermann <urs.thuermann@volkswagen.de>");
MODULE_ALIAS("can-proto-1");
+#define RAW_MIN_NAMELEN CAN_REQUIRED_SIZE(struct sockaddr_can, can_ifindex)
+
#define MASK_ALL 0
/* A raw socket has a list of can_filters attached to it, each receiving
int err = 0;
int notify_enetdown = 0;
- if (len < CAN_REQUIRED_SIZE(*addr, can_ifindex))
+ if (len < RAW_MIN_NAMELEN)
return -EINVAL;
if (addr->can_family != AF_CAN)
return -EINVAL;
if (peer)
return -EOPNOTSUPP;
- memset(addr, 0, sizeof(*addr));
+ memset(addr, 0, RAW_MIN_NAMELEN);
addr->can_family = AF_CAN;
addr->can_ifindex = ro->ifindex;
- return sizeof(*addr);
+ return RAW_MIN_NAMELEN;
}
static int raw_setsockopt(struct socket *sock, int level, int optname,
if (msg->msg_name) {
DECLARE_SOCKADDR(struct sockaddr_can *, addr, msg->msg_name);
- if (msg->msg_namelen < CAN_REQUIRED_SIZE(*addr, can_ifindex))
+ if (msg->msg_namelen < RAW_MIN_NAMELEN)
return -EINVAL;
if (addr->can_family != AF_CAN)
sock_recv_ts_and_drops(msg, sk, skb);
if (msg->msg_name) {
- __sockaddr_check_size(sizeof(struct sockaddr_can));
- msg->msg_namelen = sizeof(struct sockaddr_can);
+ __sockaddr_check_size(RAW_MIN_NAMELEN);
+ msg->msg_namelen = RAW_MIN_NAMELEN;
memcpy(msg->msg_name, skb->cb, msg->msg_namelen);
}
return -ENOMEM;
for_each_netdev(net, d) {
+ struct netdev_name_node *name_node;
+ list_for_each_entry(name_node, &d->name_node->list, list) {
+ if (!sscanf(name_node->name, name, &i))
+ continue;
+ if (i < 0 || i >= max_netdevices)
+ continue;
+
+ /* avoid cases where sscanf is not exact inverse of printf */
+ snprintf(buf, IFNAMSIZ, name, i);
+ if (!strncmp(buf, name_node->name, IFNAMSIZ))
+ set_bit(i, inuse);
+ }
if (!sscanf(d->name, name, &i))
continue;
if (i < 0 || i >= max_netdevices)
*/
thread = READ_ONCE(napi->thread);
if (thread) {
+ /* Avoid doing set_bit() if the thread is in
+ * INTERRUPTIBLE state, cause napi_thread_wait()
+ * makes sure to proceed with napi polling
+ * if the thread is explicitly woken from here.
+ */
+ if (READ_ONCE(thread->state) != TASK_INTERRUPTIBLE)
+ set_bit(NAPI_STATE_SCHED_THREADED, &napi->state);
wake_up_process(thread);
return;
}
WARN_ON_ONCE(!(val & NAPIF_STATE_SCHED));
new = val & ~(NAPIF_STATE_MISSED | NAPIF_STATE_SCHED |
+ NAPIF_STATE_SCHED_THREADED |
NAPIF_STATE_PREFER_BUSY_POLL);
/* If STATE_MISSED was set, leave STATE_SCHED set,
static int napi_thread_wait(struct napi_struct *napi)
{
+ bool woken = false;
+
set_current_state(TASK_INTERRUPTIBLE);
- while (!kthread_should_stop() && !napi_disable_pending(napi)) {
- if (test_bit(NAPI_STATE_SCHED, &napi->state)) {
+ while (!kthread_should_stop()) {
+ /* Testing SCHED_THREADED bit here to make sure the current
+ * kthread owns this napi and could poll on this napi.
+ * Testing SCHED bit is not enough because SCHED bit might be
+ * set by some other busy poll thread or by napi_disable().
+ */
+ if (test_bit(NAPI_STATE_SCHED_THREADED, &napi->state) || woken) {
WARN_ON(!list_empty(&napi->poll_list));
__set_current_state(TASK_RUNNING);
return 0;
}
schedule();
+ /* woken being true indicates this thread owns this napi. */
+ woken = true;
set_current_state(TASK_INTERRUPTIBLE);
}
__set_current_state(TASK_RUNNING);
+
return -1;
}
continue;
/* Leave virtual devices for the generic cleanup */
- if (dev->rtnl_link_ops)
+ if (dev->rtnl_link_ops && !dev->rtnl_link_ops->netns_refund)
continue;
/* Push remaining network devices to init_net */
return 0;
err_module_put:
+ for_each_possible_cpu(cpu) {
+ struct per_cpu_dm_data *hw_data = &per_cpu(dm_hw_cpu_data, cpu);
+ struct sk_buff *skb;
+
+ del_timer_sync(&hw_data->send_timer);
+ cancel_work_sync(&hw_data->dm_alert_work);
+ while ((skb = __skb_dequeue(&hw_data->drop_queue))) {
+ struct devlink_trap_metadata *hw_metadata;
+
+ hw_metadata = NET_DM_SKB_CB(skb)->hw_metadata;
+ net_dm_hw_metadata_free(hw_metadata);
+ consume_skb(skb);
+ }
+ }
module_put(THIS_MODULE);
return rc;
}
err_unregister_trace:
unregister_trace_kfree_skb(ops->kfree_skb_probe, NULL);
err_module_put:
+ for_each_possible_cpu(cpu) {
+ struct per_cpu_dm_data *data = &per_cpu(dm_cpu_data, cpu);
+ struct sk_buff *skb;
+
+ del_timer_sync(&data->send_timer);
+ cancel_work_sync(&data->dm_alert_work);
+ while ((skb = __skb_dequeue(&data->drop_queue)))
+ consume_skb(skb);
+ }
module_put(THIS_MODULE);
return rc;
}
}
EXPORT_SYMBOL(__dst_destroy_metrics_generic);
-static struct dst_ops md_dst_ops = {
- .family = AF_UNSPEC,
-};
+struct dst_entry *dst_blackhole_check(struct dst_entry *dst, u32 cookie)
+{
+ return NULL;
+}
-static int dst_md_discard_out(struct net *net, struct sock *sk, struct sk_buff *skb)
+u32 *dst_blackhole_cow_metrics(struct dst_entry *dst, unsigned long old)
{
- WARN_ONCE(1, "Attempting to call output on metadata dst\n");
- kfree_skb(skb);
- return 0;
+ return NULL;
}
-static int dst_md_discard(struct sk_buff *skb)
+struct neighbour *dst_blackhole_neigh_lookup(const struct dst_entry *dst,
+ struct sk_buff *skb,
+ const void *daddr)
{
- WARN_ONCE(1, "Attempting to call input on metadata dst\n");
- kfree_skb(skb);
- return 0;
+ return NULL;
+}
+
+void dst_blackhole_update_pmtu(struct dst_entry *dst, struct sock *sk,
+ struct sk_buff *skb, u32 mtu,
+ bool confirm_neigh)
+{
+}
+EXPORT_SYMBOL_GPL(dst_blackhole_update_pmtu);
+
+void dst_blackhole_redirect(struct dst_entry *dst, struct sock *sk,
+ struct sk_buff *skb)
+{
+}
+EXPORT_SYMBOL_GPL(dst_blackhole_redirect);
+
+unsigned int dst_blackhole_mtu(const struct dst_entry *dst)
+{
+ unsigned int mtu = dst_metric_raw(dst, RTAX_MTU);
+
+ return mtu ? : dst->dev->mtu;
}
+EXPORT_SYMBOL_GPL(dst_blackhole_mtu);
+
+static struct dst_ops dst_blackhole_ops = {
+ .family = AF_UNSPEC,
+ .neigh_lookup = dst_blackhole_neigh_lookup,
+ .check = dst_blackhole_check,
+ .cow_metrics = dst_blackhole_cow_metrics,
+ .update_pmtu = dst_blackhole_update_pmtu,
+ .redirect = dst_blackhole_redirect,
+ .mtu = dst_blackhole_mtu,
+};
static void __metadata_dst_init(struct metadata_dst *md_dst,
enum metadata_type type, u8 optslen)
-
{
struct dst_entry *dst;
dst = &md_dst->dst;
- dst_init(dst, &md_dst_ops, NULL, 1, DST_OBSOLETE_NONE,
+ dst_init(dst, &dst_blackhole_ops, NULL, 1, DST_OBSOLETE_NONE,
DST_METADATA | DST_NOCOUNT);
-
- dst->input = dst_md_discard;
- dst->output = dst_md_discard_out;
-
memset(dst + 1, 0, sizeof(*md_dst) + optslen - sizeof(*dst));
md_dst->type = type;
}
if (unlikely(flags & ~(BPF_MTU_CHK_SEGS)))
return -EINVAL;
- if (unlikely(flags & BPF_MTU_CHK_SEGS && len_diff))
+ if (unlikely(flags & BPF_MTU_CHK_SEGS && (len_diff || *mtu_len)))
return -EINVAL;
dev = __dev_via_ifindex(dev, ifindex);
mtu = READ_ONCE(dev->mtu);
dev_len = mtu + dev->hard_header_len;
- skb_len = skb->len + len_diff; /* minus result pass check */
+
+ /* If set use *mtu_len as input, L3 as iph->tot_len (like fib_lookup) */
+ skb_len = *mtu_len ? *mtu_len + dev->hard_header_len : skb->len;
+
+ skb_len += len_diff; /* minus result pass check */
if (skb_len <= dev_len) {
ret = BPF_MTU_CHK_RET_SUCCESS;
goto out;
/* Add L2-header as dev MTU is L3 size */
dev_len = mtu + dev->hard_header_len;
+ /* Use *mtu_len as input, L3 as iph->tot_len (like fib_lookup) */
+ if (*mtu_len)
+ xdp_len = *mtu_len + dev->hard_header_len;
+
xdp_len += len_diff; /* minus result pass check */
if (xdp_len > dev_len)
ret = BPF_MTU_CHK_RET_FRAG_NEEDED;
* avoid confusion with packets without such field
*/
if (icmp_has_id(ih->type))
- key_icmp->id = ih->un.echo.id ? : 1;
+ key_icmp->id = ih->un.echo.id ? ntohs(ih->un.echo.id) : 1;
else
key_icmp->id = 0;
}
* we can reinject the packet there.
*/
n2 = NULL;
- if (dst) {
+ if (dst && dst->obsolete != DST_OBSOLETE_DEAD) {
n2 = dst_neigh_lookup_skb(dst, skb);
if (n2)
n1 = n2;
BUG_ON(!(af_ops = rtnl_af_lookup(nla_type(af))));
- err = af_ops->set_link_af(dev, af);
+ err = af_ops->set_link_af(dev, af, extack);
if (err < 0) {
rcu_read_unlock();
goto errout;
if (unlikely(!msg))
return -EAGAIN;
sk_msg_init(msg);
+ skb_set_owner_r(skb, sk);
return sk_psock_skb_ingress_enqueue(skb, psock, sk, msg);
}
{
switch (verdict) {
case __SK_REDIRECT:
- skb_set_owner_r(skb, sk);
sk_psock_skb_redirect(skb);
break;
case __SK_PASS:
rcu_read_lock();
prog = READ_ONCE(psock->progs.skb_verdict);
if (likely(prog)) {
- /* We skip full set_owner_r here because if we do a SK_PASS
- * or SK_DROP we can skip skb memory accounting and use the
- * TLS context.
- */
skb->sk = psock->sk;
tcp_skb_bpf_redirect_clear(skb);
ret = sk_psock_bpf_run(psock, prog, skb);
kfree_skb(skb);
goto out;
}
- skb_set_owner_r(skb, sk);
prog = READ_ONCE(psock->progs.skb_verdict);
if (likely(prog)) {
+ skb->sk = sk;
tcp_skb_bpf_redirect_clear(skb);
ret = sk_psock_bpf_run(psock, prog, skb);
ret = sk_psock_map_verd(ret, tcp_skb_bpf_redirect_fetch(skb));
+ skb->sk = NULL;
}
sk_psock_verdict_apply(psock, skb, ret);
out:
kfree_skb(skb);
goto out;
}
- skb_set_owner_r(skb, sk);
prog = READ_ONCE(psock->progs.skb_verdict);
if (likely(prog)) {
+ skb->sk = sk;
tcp_skb_bpf_redirect_clear(skb);
ret = sk_psock_bpf_run(psock, prog, skb);
ret = sk_psock_map_verd(ret, tcp_skb_bpf_redirect_fetch(skb));
+ skb->sk = NULL;
}
sk_psock_verdict_apply(psock, skb, ret);
out:
if (skb_is_tcp_pure_ack(skb))
return;
- if (can_skb_orphan_partial(skb)) {
- struct sock *sk = skb->sk;
-
- if (refcount_inc_not_zero(&sk->sk_refcnt)) {
- WARN_ON(refcount_sub_and_test(skb->truesize, &sk->sk_wmem_alloc));
- skb->destructor = sock_efree;
- }
- } else {
+ if (can_skb_orphan_partial(skb))
+ skb_set_owner_sk_safe(skb, skb->sk);
+ else
skb_orphan(skb);
- }
}
EXPORT_SYMBOL(skb_orphan_partial);
twsk_prot->twsk_slab = NULL;
}
+static int tw_prot_init(const struct proto *prot)
+{
+ struct timewait_sock_ops *twsk_prot = prot->twsk_prot;
+
+ if (!twsk_prot)
+ return 0;
+
+ twsk_prot->twsk_slab_name = kasprintf(GFP_KERNEL, "tw_sock_%s",
+ prot->name);
+ if (!twsk_prot->twsk_slab_name)
+ return -ENOMEM;
+
+ twsk_prot->twsk_slab =
+ kmem_cache_create(twsk_prot->twsk_slab_name,
+ twsk_prot->twsk_obj_size, 0,
+ SLAB_ACCOUNT | prot->slab_flags,
+ NULL);
+ if (!twsk_prot->twsk_slab) {
+ pr_crit("%s: Can't create timewait sock SLAB cache!\n",
+ prot->name);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
static void req_prot_cleanup(struct request_sock_ops *rsk_prot)
{
if (!rsk_prot)
if (req_prot_init(prot))
goto out_free_request_sock_slab;
- if (prot->twsk_prot != NULL) {
- prot->twsk_prot->twsk_slab_name = kasprintf(GFP_KERNEL, "tw_sock_%s", prot->name);
-
- if (prot->twsk_prot->twsk_slab_name == NULL)
- goto out_free_request_sock_slab;
-
- prot->twsk_prot->twsk_slab =
- kmem_cache_create(prot->twsk_prot->twsk_slab_name,
- prot->twsk_prot->twsk_obj_size,
- 0,
- SLAB_ACCOUNT |
- prot->slab_flags,
- NULL);
- if (prot->twsk_prot->twsk_slab == NULL)
- goto out_free_timewait_sock_slab;
- }
+ if (tw_prot_init(prot))
+ goto out_free_timewait_sock_slab;
}
mutex_lock(&proto_list_mutex);
/* mem->id is valid, checked in xdp_rxq_info_reg_mem_model() */
xa = rhashtable_lookup(mem_id_ht, &mem->id, mem_id_rht_params);
page = virt_to_head_page(data);
- napi_direct &= !xdp_return_frame_no_direct();
+ if (napi_direct && xdp_return_frame_no_direct())
+ napi_direct = false;
page_pool_put_full_page(xa->page_pool, page, napi_direct);
rcu_read_unlock();
break;
if (!ipv6_unicast_destination(skb))
return 0; /* discard, don't send a reset here */
+ if (ipv6_addr_v4mapped(&ipv6_hdr(skb)->saddr)) {
+ __IP6_INC_STATS(sock_net(sk), NULL, IPSTATS_MIB_INHDRERRORS);
+ return 0;
+ }
+
if (dccp_bad_service_code(sk, service)) {
dcb->dccpd_reset_code = DCCP_RESET_CODE_BAD_SERVICE_CODE;
goto drop;
list_for_each_entry(dp, &dst->ports, list) {
err = dsa_port_setup(dp);
- if (err)
+ if (err) {
+ dsa_port_devlink_teardown(dp);
+ dp->type = DSA_PORT_TYPE_UNUSED;
+ err = dsa_port_devlink_setup(dp);
+ if (err)
+ goto teardown;
continue;
+ }
}
return 0;
{
struct dsa_switch *ds = dp->ds;
struct dsa_switch_tree *dst = ds->dst;
+ const struct dsa_device_ops *tag_ops;
enum dsa_tag_protocol tag_protocol;
tag_protocol = dsa_get_tag_protocol(dp, master);
* nothing to do here.
*/
} else {
- dst->tag_ops = dsa_tag_driver_get(tag_protocol);
- if (IS_ERR(dst->tag_ops)) {
- if (PTR_ERR(dst->tag_ops) == -ENOPROTOOPT)
+ tag_ops = dsa_tag_driver_get(tag_protocol);
+ if (IS_ERR(tag_ops)) {
+ if (PTR_ERR(tag_ops) == -ENOPROTOOPT)
return -EPROBE_DEFER;
dev_warn(ds->dev, "No tagger for this switch\n");
dp->master = NULL;
- return PTR_ERR(dst->tag_ops);
+ return PTR_ERR(tag_ops);
}
+
+ dst->tag_ops = tag_ops;
}
dp->master = master;
bool unset_vlan_filtering = br_vlan_enabled(info->br);
struct dsa_switch_tree *dst = ds->dst;
struct netlink_ext_ack extack = {0};
- int err, i;
+ int err, port;
if (dst->index == info->tree_index && ds->index == info->sw_index &&
ds->ops->port_bridge_join)
* it. That is a good thing, because that lets us handle it and also
* handle the case where the switch's vlan_filtering setting is global
* (not per port). When that happens, the correct moment to trigger the
- * vlan_filtering callback is only when the last port left this bridge.
+ * vlan_filtering callback is only when the last port leaves the last
+ * VLAN-aware bridge.
*/
if (unset_vlan_filtering && ds->vlan_filtering_is_global) {
- for (i = 0; i < ds->num_ports; i++) {
- if (i == info->port)
- continue;
- if (dsa_to_port(ds, i)->bridge_dev == info->br) {
+ for (port = 0; port < ds->num_ports; port++) {
+ struct net_device *bridge_dev;
+
+ bridge_dev = dsa_to_port(ds, port)->bridge_dev;
+
+ if (bridge_dev && br_vlan_enabled(bridge_dev)) {
unset_vlan_filtering = false;
break;
}
__DEFINE_LINK_MODE_PARAMS(10000, KR, Full),
[ETHTOOL_LINK_MODE_10000baseR_FEC_BIT] = {
.speed = SPEED_10000,
+ .lanes = 1,
.duplex = DUPLEX_FULL,
},
__DEFINE_LINK_MODE_PARAMS(20000, MLD2, Full),
rtnl_unlock();
}
EXPORT_SYMBOL_GPL(ethtool_set_ethtool_phy_ops);
+
+void
+ethtool_params_from_link_mode(struct ethtool_link_ksettings *link_ksettings,
+ enum ethtool_link_mode_bit_indices link_mode)
+{
+ const struct link_mode_info *link_info;
+
+ if (WARN_ON_ONCE(link_mode >= __ETHTOOL_LINK_MODE_MASK_NBITS))
+ return;
+
+ link_info = &link_mode_params[link_mode];
+ link_ksettings->base.speed = link_info->speed;
+ link_ksettings->lanes = link_info->lanes;
+ link_ksettings->base.duplex = link_info->duplex;
+}
+EXPORT_SYMBOL_GPL(ethtool_params_from_link_mode);
ethnl_update_bool32(&eee.eee_enabled, tb[ETHTOOL_A_EEE_ENABLED], &mod);
ethnl_update_bool32(&eee.tx_lpi_enabled,
tb[ETHTOOL_A_EEE_TX_LPI_ENABLED], &mod);
- ethnl_update_bool32(&eee.tx_lpi_timer, tb[ETHTOOL_A_EEE_TX_LPI_TIMER],
- &mod);
+ ethnl_update_u32(&eee.tx_lpi_timer, tb[ETHTOOL_A_EEE_TX_LPI_TIMER],
+ &mod);
ret = 0;
if (!mod)
goto out_ops;
int __ethtool_get_link_ksettings(struct net_device *dev,
struct ethtool_link_ksettings *link_ksettings)
{
- const struct link_mode_info *link_info;
- int err;
-
ASSERT_RTNL();
if (!dev->ethtool_ops->get_link_ksettings)
return -EOPNOTSUPP;
memset(link_ksettings, 0, sizeof(*link_ksettings));
-
- link_ksettings->link_mode = -1;
- err = dev->ethtool_ops->get_link_ksettings(dev, link_ksettings);
- if (err)
- return err;
-
- if (link_ksettings->link_mode != -1) {
- link_info = &link_mode_params[link_ksettings->link_mode];
- link_ksettings->base.speed = link_info->speed;
- link_ksettings->lanes = link_info->lanes;
- link_ksettings->base.duplex = link_info->duplex;
- }
-
- return 0;
+ return dev->ethtool_ops->get_link_ksettings(dev, link_ksettings);
}
EXPORT_SYMBOL(__ethtool_get_link_ksettings);
master = hsr_port_get_hsr(hsr, HSR_PT_MASTER);
if (master) {
skb->dev = master->dev;
+ skb_reset_mac_header(skb);
hsr_forward_skb(skb, master);
} else {
atomic_long_inc(&dev->tx_dropped);
{
struct hsr_frame_info frame;
- if (skb_mac_header(skb) != skb->data) {
- WARN_ONCE(1, "%s:%d: Malformed frame (port_src %s)\n",
- __FILE__, __LINE__, port->dev->name);
- goto out_drop;
- }
-
if (fill_frame_info(&frame, skb, port) < 0)
goto out_drop;
desc->mode = nla_get_u8(info->attrs[IEEE802154_ATTR_LLSEC_KEY_MODE]);
if (desc->mode == IEEE802154_SCF_KEY_IMPLICIT) {
- if (!info->attrs[IEEE802154_ATTR_PAN_ID] &&
- !(info->attrs[IEEE802154_ATTR_SHORT_ADDR] ||
- info->attrs[IEEE802154_ATTR_HW_ADDR]))
+ if (!info->attrs[IEEE802154_ATTR_PAN_ID])
return -EINVAL;
desc->device_addr.pan_id = nla_get_shortaddr(info->attrs[IEEE802154_ATTR_PAN_ID]);
desc->device_addr.mode = IEEE802154_ADDR_SHORT;
desc->device_addr.short_addr = nla_get_shortaddr(info->attrs[IEEE802154_ATTR_SHORT_ADDR]);
} else {
+ if (!info->attrs[IEEE802154_ATTR_HW_ADDR])
+ return -EINVAL;
+
desc->device_addr.mode = IEEE802154_ADDR_LONG;
desc->device_addr.extended_addr = nla_get_hwaddr(info->attrs[IEEE802154_ATTR_HW_ADDR]);
}
goto nla_put_failure;
#ifdef CONFIG_IEEE802154_NL802154_EXPERIMENTAL
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ goto out;
+
if (nl802154_get_llsec_params(msg, rdev, wpan_dev) < 0)
goto nla_put_failure;
+
+out:
#endif /* CONFIG_IEEE802154_NL802154_EXPERIMENTAL */
genlmsg_end(msg, hdr);
u32 changed = 0;
int ret;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
if (info->attrs[NL802154_ATTR_SEC_ENABLED]) {
u8 enabled;
if (err)
return err;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR) {
+ err = skb->len;
+ goto out_err;
+ }
+
if (!wpan_dev->netdev) {
err = -EINVAL;
goto out_err;
struct ieee802154_llsec_key_id id = { };
u32 commands[NL802154_CMD_FRAME_NR_IDS / 32] = { };
- if (nla_parse_nested_deprecated(attrs, NL802154_KEY_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_KEY], nl802154_key_policy, info->extack))
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
+ if (!info->attrs[NL802154_ATTR_SEC_KEY] ||
+ nla_parse_nested_deprecated(attrs, NL802154_KEY_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_KEY], nl802154_key_policy, info->extack))
return -EINVAL;
if (!attrs[NL802154_KEY_ATTR_USAGE_FRAMES] ||
struct nlattr *attrs[NL802154_KEY_ATTR_MAX + 1];
struct ieee802154_llsec_key_id id;
- if (nla_parse_nested_deprecated(attrs, NL802154_KEY_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_KEY], nl802154_key_policy, info->extack))
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
+ if (!info->attrs[NL802154_ATTR_SEC_KEY] ||
+ nla_parse_nested_deprecated(attrs, NL802154_KEY_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_KEY], nl802154_key_policy, info->extack))
return -EINVAL;
if (ieee802154_llsec_parse_key_id(attrs[NL802154_KEY_ATTR_ID], &id) < 0)
if (err)
return err;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR) {
+ err = skb->len;
+ goto out_err;
+ }
+
if (!wpan_dev->netdev) {
err = -EINVAL;
goto out_err;
struct wpan_dev *wpan_dev = dev->ieee802154_ptr;
struct ieee802154_llsec_device dev_desc;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
if (ieee802154_llsec_parse_device(info->attrs[NL802154_ATTR_SEC_DEVICE],
&dev_desc) < 0)
return -EINVAL;
struct nlattr *attrs[NL802154_DEV_ATTR_MAX + 1];
__le64 extended_addr;
- if (nla_parse_nested_deprecated(attrs, NL802154_DEV_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_DEVICE], nl802154_dev_policy, info->extack))
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
+ if (!info->attrs[NL802154_ATTR_SEC_DEVICE] ||
+ nla_parse_nested_deprecated(attrs, NL802154_DEV_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_DEVICE], nl802154_dev_policy, info->extack))
return -EINVAL;
if (!attrs[NL802154_DEV_ATTR_EXTENDED_ADDR])
if (err)
return err;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR) {
+ err = skb->len;
+ goto out_err;
+ }
+
if (!wpan_dev->netdev) {
err = -EINVAL;
goto out_err;
struct ieee802154_llsec_device_key key;
__le64 extended_addr;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
if (!info->attrs[NL802154_ATTR_SEC_DEVKEY] ||
nla_parse_nested_deprecated(attrs, NL802154_DEVKEY_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_DEVKEY], nl802154_devkey_policy, info->extack) < 0)
return -EINVAL;
struct ieee802154_llsec_device_key key;
__le64 extended_addr;
- if (nla_parse_nested_deprecated(attrs, NL802154_DEVKEY_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_DEVKEY], nl802154_devkey_policy, info->extack))
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
+ if (!info->attrs[NL802154_ATTR_SEC_DEVKEY] ||
+ nla_parse_nested_deprecated(attrs, NL802154_DEVKEY_ATTR_MAX, info->attrs[NL802154_ATTR_SEC_DEVKEY], nl802154_devkey_policy, info->extack))
return -EINVAL;
if (!attrs[NL802154_DEVKEY_ATTR_EXTENDED_ADDR])
if (err)
return err;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR) {
+ err = skb->len;
+ goto out_err;
+ }
+
if (!wpan_dev->netdev) {
err = -EINVAL;
goto out_err;
struct wpan_dev *wpan_dev = dev->ieee802154_ptr;
struct ieee802154_llsec_seclevel sl;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
if (llsec_parse_seclevel(info->attrs[NL802154_ATTR_SEC_LEVEL],
&sl) < 0)
return -EINVAL;
struct wpan_dev *wpan_dev = dev->ieee802154_ptr;
struct ieee802154_llsec_seclevel sl;
+ if (wpan_dev->iftype == NL802154_IFTYPE_MONITOR)
+ return -EOPNOTSUPP;
+
if (!info->attrs[NL802154_ATTR_SEC_LEVEL] ||
llsec_parse_seclevel(info->attrs[NL802154_ATTR_SEC_LEVEL],
&sl) < 0)
#define NL802154_FLAG_NEED_NETDEV 0x02
#define NL802154_FLAG_NEED_RTNL 0x04
#define NL802154_FLAG_CHECK_NETDEV_UP 0x08
-#define NL802154_FLAG_NEED_NETDEV_UP (NL802154_FLAG_NEED_NETDEV |\
- NL802154_FLAG_CHECK_NETDEV_UP)
#define NL802154_FLAG_NEED_WPAN_DEV 0x10
-#define NL802154_FLAG_NEED_WPAN_DEV_UP (NL802154_FLAG_NEED_WPAN_DEV |\
- NL802154_FLAG_CHECK_NETDEV_UP)
static int nl802154_pre_doit(const struct genl_ops *ops, struct sk_buff *skb,
struct genl_info *info)
}
kfree(AH_SKB_CB(skb)->tmp);
- xfrm_output_resume(skb, err);
+ xfrm_output_resume(skb->sk, skb, err);
}
static int ah_output(struct xfrm_state *x, struct sk_buff *skb)
return 0;
}
-static int inet_set_link_af(struct net_device *dev, const struct nlattr *nla)
+static int inet_set_link_af(struct net_device *dev, const struct nlattr *nla,
+ struct netlink_ext_ack *extack)
{
struct in_device *in_dev = __in_dev_get_rcu(dev);
struct nlattr *a, *tb[IFLA_INET_MAX+1];
x->encap && x->encap->encap_type == TCP_ENCAP_ESPINTCP)
esp_output_tail_tcp(x, skb);
else
- xfrm_output_resume(skb, err);
+ xfrm_output_resume(skb->sk, skb, err);
}
}
if ((!(skb->dev->gso_partial_features & NETIF_F_HW_ESP) &&
!(features & NETIF_F_HW_ESP)) || x->xso.dev != skb->dev)
- esp_features = features & ~(NETIF_F_SG | NETIF_F_CSUM_MASK);
+ esp_features = features & ~(NETIF_F_SG | NETIF_F_CSUM_MASK |
+ NETIF_F_SCTP_CRC);
else if (!(features & NETIF_F_HW_ESP_TX_CSUM) &&
!(skb->dev->gso_partial_features & NETIF_F_HW_ESP_TX_CSUM))
- esp_features = features & ~NETIF_F_CSUM_MASK;
+ esp_features = features & ~(NETIF_F_CSUM_MASK |
+ NETIF_F_SCTP_CRC);
xo->flags |= XFRM_GSO_SEGMENT;
ip_hdr(skb)->tot_len = htons(skb->len);
ip_send_check(ip_hdr(skb));
- if (hw_offload)
+ if (hw_offload) {
+ if (!skb_ext_add(skb, SKB_EXT_SEC_PATH))
+ return -ENOMEM;
+
+ xo = xfrm_offload(skb);
+ if (!xo)
+ return -EINVAL;
+
+ xo->flags |= XFRM_XMIT;
return 0;
+ }
err = esp_output_tail(x, skb, &esp);
if (err)
return found;
}
-void inet_csk_reqsk_queue_drop(struct sock *sk, struct request_sock *req)
+bool inet_csk_reqsk_queue_drop(struct sock *sk, struct request_sock *req)
{
- if (reqsk_queue_unlink(req)) {
+ bool unlinked = reqsk_queue_unlink(req);
+
+ if (unlinked) {
reqsk_queue_removed(&inet_csk(sk)->icsk_accept_queue, req);
reqsk_put(req);
}
+ return unlinked;
}
EXPORT_SYMBOL(inet_csk_reqsk_queue_drop);
}
if (dst->flags & DST_XFRM_QUEUE)
- goto queued;
+ goto xmit;
if (!vti_state_check(dst->xfrm, parms->iph.daddr, parms->iph.saddr)) {
dev->stats.tx_carrier_errors++;
if (skb->len > mtu) {
skb_dst_update_pmtu_no_confirm(skb, mtu);
if (skb->protocol == htons(ETH_P_IP)) {
+ if (!(ip_hdr(skb)->frag_off & htons(IP_DF)))
+ goto xmit;
icmp_ndo_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED,
htonl(mtu));
} else {
goto tx_error;
}
-queued:
+xmit:
skb_scrub_packet(skb, !net_eq(tunnel->net, dev_net(dev)));
skb_dst_set(skb, dst);
skb->dev = skb_dst(skb)->dev;
*/
static void __init ic_close_devs(void)
{
- struct net_device *selected_dev = ic_dev->dev;
+ struct net_device *selected_dev = ic_dev ? ic_dev->dev : NULL;
struct ic_device *d, *next;
struct net_device *dev;
next = ic_first_dev;
while ((d = next)) {
bool bring_down = (d != ic_dev);
- struct net_device *lower_dev;
+ struct net_device *lower;
struct list_head *iter;
next = d->next;
dev = d->dev;
- netdev_for_each_lower_dev(selected_dev, lower_dev, iter) {
- if (dev == lower_dev) {
- bring_down = false;
- break;
+ if (selected_dev) {
+ netdev_for_each_lower_dev(selected_dev, lower, iter) {
+ if (dev == lower) {
+ bring_down = false;
+ break;
+ }
}
}
if (bring_down) {
local_bh_disable();
addend = xt_write_recseq_begin();
- private = rcu_access_pointer(table->private);
+ private = READ_ONCE(table->private); /* Address dependency. */
cpu = smp_processor_id();
table_base = private->entries;
jumpstack = (struct arpt_entry **)private->jumpstack[cpu];
{
unsigned int countersize;
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
/* We need atomic snapshot of counters: rest doesn't change
* (other than comefrom, which userspace doesn't care
unsigned int off, num;
const struct arpt_entry *e;
struct xt_counters *counters;
- struct xt_table_info *private = xt_table_get_private_protected(table);
+ struct xt_table_info *private = table->private;
int ret = 0;
void *loc_cpu_entry;
t = xt_request_find_table_lock(net, NFPROTO_ARP, name);
if (!IS_ERR(t)) {
struct arpt_getinfo info;
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
#ifdef CONFIG_COMPAT
struct xt_table_info tmp;
t = xt_find_table_lock(net, NFPROTO_ARP, get.name);
if (!IS_ERR(t)) {
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
if (get.size == private->size)
ret = copy_entries_to_user(private->size,
}
local_bh_disable();
- private = xt_table_get_private_protected(t);
+ private = t->private;
if (private->number != tmp.num_counters) {
ret = -EINVAL;
goto unlock_up_free;
void __user *userptr)
{
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
void __user *pos;
unsigned int size;
int ret = 0;
xt_compat_lock(NFPROTO_ARP);
t = xt_find_table_lock(net, NFPROTO_ARP, get.name);
if (!IS_ERR(t)) {
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
struct xt_table_info info;
ret = compat_table_info(private, &info);
WARN_ON(!(table->valid_hooks & (1 << hook)));
local_bh_disable();
addend = xt_write_recseq_begin();
- private = rcu_access_pointer(table->private);
+ private = READ_ONCE(table->private); /* Address dependency. */
cpu = smp_processor_id();
table_base = private->entries;
jumpstack = (struct ipt_entry **)private->jumpstack[cpu];
{
unsigned int countersize;
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
/* We need atomic snapshot of counters: rest doesn't change
(other than comefrom, which userspace doesn't care
unsigned int off, num;
const struct ipt_entry *e;
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
int ret = 0;
const void *loc_cpu_entry;
t = xt_request_find_table_lock(net, AF_INET, name);
if (!IS_ERR(t)) {
struct ipt_getinfo info;
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
#ifdef CONFIG_COMPAT
struct xt_table_info tmp;
t = xt_find_table_lock(net, AF_INET, get.name);
if (!IS_ERR(t)) {
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
if (get.size == private->size)
ret = copy_entries_to_user(private->size,
t, uptr->entrytable);
}
local_bh_disable();
- private = xt_table_get_private_protected(t);
+ private = t->private;
if (private->number != tmp.num_counters) {
ret = -EINVAL;
goto unlock_up_free;
void __user *userptr)
{
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
void __user *pos;
unsigned int size;
int ret = 0;
xt_compat_lock(AF_INET);
t = xt_find_table_lock(net, AF_INET, get.name);
if (!IS_ERR(t)) {
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
struct xt_table_info info;
ret = compat_table_info(private, &info);
if (!ret && get.size == info.size)
return rth;
}
-static struct dst_entry *ipv4_blackhole_dst_check(struct dst_entry *dst, u32 cookie)
-{
- return NULL;
-}
-
-static unsigned int ipv4_blackhole_mtu(const struct dst_entry *dst)
-{
- unsigned int mtu = dst_metric_raw(dst, RTAX_MTU);
-
- return mtu ? : dst->dev->mtu;
-}
-
-static void ipv4_rt_blackhole_update_pmtu(struct dst_entry *dst, struct sock *sk,
- struct sk_buff *skb, u32 mtu,
- bool confirm_neigh)
-{
-}
-
-static void ipv4_rt_blackhole_redirect(struct dst_entry *dst, struct sock *sk,
- struct sk_buff *skb)
-{
-}
-
-static u32 *ipv4_rt_blackhole_cow_metrics(struct dst_entry *dst,
- unsigned long old)
-{
- return NULL;
-}
-
static struct dst_ops ipv4_dst_blackhole_ops = {
- .family = AF_INET,
- .check = ipv4_blackhole_dst_check,
- .mtu = ipv4_blackhole_mtu,
- .default_advmss = ipv4_default_advmss,
- .update_pmtu = ipv4_rt_blackhole_update_pmtu,
- .redirect = ipv4_rt_blackhole_redirect,
- .cow_metrics = ipv4_rt_blackhole_cow_metrics,
- .neigh_lookup = ipv4_neigh_lookup,
+ .family = AF_INET,
+ .default_advmss = ipv4_default_advmss,
+ .neigh_lookup = ipv4_neigh_lookup,
+ .check = dst_blackhole_check,
+ .cow_metrics = dst_blackhole_cow_metrics,
+ .update_pmtu = dst_blackhole_update_pmtu,
+ .redirect = dst_blackhole_redirect,
+ .mtu = dst_blackhole_mtu,
};
struct dst_entry *ipv4_blackhole_route(struct net *net, struct dst_entry *dst_orig)
tcp_reset(sk, skb);
}
if (!fastopen) {
- inet_csk_reqsk_queue_drop(sk, req);
- __NET_INC_STATS(sock_net(sk), LINUX_MIB_EMBRYONICRSTS);
+ bool unlinked = inet_csk_reqsk_queue_drop(sk, req);
+
+ if (unlinked)
+ __NET_INC_STATS(sock_net(sk), LINUX_MIB_EMBRYONICRSTS);
+ *req_stolen = !unlinked;
}
return NULL;
}
val = up->gso_size;
break;
+ case UDP_GRO:
+ val = up->gro_enabled;
+ break;
+
/* The following two cannot be changed on UDP sockets, the return is
* always 0 (which corresponds to the full checksum coverage of UDP). */
case UDPLITE_SEND_CSCOV:
return 0;
}
-static int inet6_set_iftoken(struct inet6_dev *idev, struct in6_addr *token)
+static int inet6_set_iftoken(struct inet6_dev *idev, struct in6_addr *token,
+ struct netlink_ext_ack *extack)
{
struct inet6_ifaddr *ifp;
struct net_device *dev = idev->dev;
if (!token)
return -EINVAL;
- if (dev->flags & (IFF_LOOPBACK | IFF_NOARP))
+
+ if (dev->flags & IFF_LOOPBACK) {
+ NL_SET_ERR_MSG_MOD(extack, "Device is loopback");
return -EINVAL;
- if (!ipv6_accept_ra(idev))
+ }
+
+ if (dev->flags & IFF_NOARP) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Device does not do neighbour discovery");
+ return -EINVAL;
+ }
+
+ if (!ipv6_accept_ra(idev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Router advertisement is disabled on device");
return -EINVAL;
- if (idev->cnf.rtr_solicits == 0)
+ }
+
+ if (idev->cnf.rtr_solicits == 0) {
+ NL_SET_ERR_MSG(extack,
+ "Router solicitation is disabled on device");
return -EINVAL;
+ }
write_lock_bh(&idev->lock);
return 0;
}
-static int inet6_set_link_af(struct net_device *dev, const struct nlattr *nla)
+static int inet6_set_link_af(struct net_device *dev, const struct nlattr *nla,
+ struct netlink_ext_ack *extack)
{
struct inet6_dev *idev = __in6_dev_get(dev);
struct nlattr *tb[IFLA_INET6_MAX + 1];
BUG();
if (tb[IFLA_INET6_TOKEN]) {
- err = inet6_set_iftoken(idev, nla_data(tb[IFLA_INET6_TOKEN]));
+ err = inet6_set_iftoken(idev, nla_data(tb[IFLA_INET6_TOKEN]),
+ extack);
if (err)
return err;
}
}
kfree(AH_SKB_CB(skb)->tmp);
- xfrm_output_resume(skb, err);
+ xfrm_output_resume(skb->sk, skb, err);
}
static int ah6_output(struct xfrm_state *x, struct sk_buff *skb)
x->encap && x->encap->encap_type == TCP_ENCAP_ESPINTCP)
esp_output_tail_tcp(x, skb);
else
- xfrm_output_resume(skb, err);
+ xfrm_output_resume(skb->sk, skb, err);
}
}
skb->encap_hdr_csum = 1;
if (!(features & NETIF_F_HW_ESP) || x->xso.dev != skb->dev)
- esp_features = features & ~(NETIF_F_SG | NETIF_F_CSUM_MASK);
+ esp_features = features & ~(NETIF_F_SG | NETIF_F_CSUM_MASK |
+ NETIF_F_SCTP_CRC);
else if (!(features & NETIF_F_HW_ESP_TX_CSUM))
- esp_features = features & ~NETIF_F_CSUM_MASK;
+ esp_features = features & ~(NETIF_F_CSUM_MASK |
+ NETIF_F_SCTP_CRC);
xo->flags |= XFRM_GSO_SEGMENT;
ipv6_hdr(skb)->payload_len = htons(len);
- if (hw_offload)
+ if (hw_offload) {
+ if (!skb_ext_add(skb, SKB_EXT_SEC_PATH))
+ return -ENOMEM;
+
+ xo = xfrm_offload(skb);
+ if (!xo)
+ return -EINVAL;
+
+ xo->flags |= XFRM_XMIT;
return 0;
+ }
err = esp6_output_tail(x, skb, &esp);
if (err)
const struct net_device *dev;
if (rt->nh)
- fib6_nh = nexthop_fib6_nh(rt->nh);
+ fib6_nh = nexthop_fib6_nh_bh(rt->nh);
seq_printf(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen);
if (ipv6_addr_is_multicast(&hdr->saddr))
goto err;
- /* While RFC4291 is not explicit about v4mapped addresses
- * in IPv6 headers, it seems clear linux dual-stack
- * model can not deal properly with these.
- * Security models could be fooled by ::ffff:127.0.0.1 for example.
- *
- * https://tools.ietf.org/html/draft-itojun-v6ops-v4mapped-harmful-02
- */
- if (ipv6_addr_v4mapped(&hdr->saddr))
- goto err;
-
skb->transport_header = skb->network_header + sizeof(*hdr);
IP6CB(skb)->nhoff = offsetof(struct ipv6hdr, nexthdr);
}
if (dst->flags & DST_XFRM_QUEUE)
- goto queued;
+ goto xmit;
x = dst->xfrm;
if (!vti6_state_check(x, &t->parms.raddr, &t->parms.laddr))
icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu);
} else {
+ if (!(ip_hdr(skb)->frag_off & htons(IP_DF)))
+ goto xmit;
icmp_ndo_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED,
htonl(mtu));
}
goto tx_err_dst_release;
}
-queued:
+xmit:
skb_scrub_packet(skb, !net_eq(t->net, dev_net(dev)));
skb_dst_set(skb, dst);
skb->dev = skb_dst(skb)->dev;
local_bh_disable();
addend = xt_write_recseq_begin();
- private = rcu_access_pointer(table->private);
+ private = READ_ONCE(table->private); /* Address dependency. */
cpu = smp_processor_id();
table_base = private->entries;
jumpstack = (struct ip6t_entry **)private->jumpstack[cpu];
{
unsigned int countersize;
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
/* We need atomic snapshot of counters: rest doesn't change
(other than comefrom, which userspace doesn't care
unsigned int off, num;
const struct ip6t_entry *e;
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
int ret = 0;
const void *loc_cpu_entry;
t = xt_request_find_table_lock(net, AF_INET6, name);
if (!IS_ERR(t)) {
struct ip6t_getinfo info;
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
#ifdef CONFIG_COMPAT
struct xt_table_info tmp;
t = xt_find_table_lock(net, AF_INET6, get.name);
if (!IS_ERR(t)) {
- struct xt_table_info *private = xt_table_get_private_protected(t);
+ struct xt_table_info *private = t->private;
if (get.size == private->size)
ret = copy_entries_to_user(private->size,
t, uptr->entrytable);
}
local_bh_disable();
- private = xt_table_get_private_protected(t);
+ private = t->private;
if (private->number != tmp.num_counters) {
ret = -EINVAL;
goto unlock_up_free;
void __user *userptr)
{
struct xt_counters *counters;
- const struct xt_table_info *private = xt_table_get_private_protected(table);
+ const struct xt_table_info *private = table->private;
void __user *pos;
unsigned int size;
int ret = 0;
xt_compat_lock(AF_INET6);
t = xt_find_table_lock(net, AF_INET6, get.name);
if (!IS_ERR(t)) {
- const struct xt_table_info *private = xt_table_get_private_protected(t);
+ const struct xt_table_info *private = t->private;
struct xt_table_info info;
ret = compat_table_info(private, &info);
if (!ret && get.size == info.size)
*/
v4addr = LOOPBACK4_IPV6;
if (!(addr_type & IPV6_ADDR_MULTICAST) &&
- !sock_net(sk)->ipv6.sysctl.ip_nonlocal_bind) {
+ !ipv6_can_nonlocal_bind(sock_net(sk), inet)) {
err = -EADDRNOTAVAIL;
if (!ipv6_chk_addr(sock_net(sk), &addr->sin6_addr,
dev, 0)) {
.confirm_neigh = ip6_confirm_neigh,
};
-static unsigned int ip6_blackhole_mtu(const struct dst_entry *dst)
-{
- unsigned int mtu = dst_metric_raw(dst, RTAX_MTU);
-
- return mtu ? : dst->dev->mtu;
-}
-
-static void ip6_rt_blackhole_update_pmtu(struct dst_entry *dst, struct sock *sk,
- struct sk_buff *skb, u32 mtu,
- bool confirm_neigh)
-{
-}
-
-static void ip6_rt_blackhole_redirect(struct dst_entry *dst, struct sock *sk,
- struct sk_buff *skb)
-{
-}
-
static struct dst_ops ip6_dst_blackhole_ops = {
- .family = AF_INET6,
- .destroy = ip6_dst_destroy,
- .check = ip6_dst_check,
- .mtu = ip6_blackhole_mtu,
- .default_advmss = ip6_default_advmss,
- .update_pmtu = ip6_rt_blackhole_update_pmtu,
- .redirect = ip6_rt_blackhole_redirect,
- .cow_metrics = dst_cow_metrics_generic,
- .neigh_lookup = ip6_dst_neigh_lookup,
+ .family = AF_INET6,
+ .default_advmss = ip6_default_advmss,
+ .neigh_lookup = ip6_dst_neigh_lookup,
+ .check = ip6_dst_check,
+ .destroy = ip6_dst_destroy,
+ .cow_metrics = dst_cow_metrics_generic,
+ .update_pmtu = dst_blackhole_update_pmtu,
+ .redirect = dst_blackhole_redirect,
+ .mtu = dst_blackhole_mtu,
};
static const u32 ip6_template_metrics[RTAX_MAX] = {
* nexthops have been replaced by first new, the rest should
* be added to it.
*/
- cfg->fc_nlinfo.nlh->nlmsg_flags &= ~(NLM_F_EXCL |
- NLM_F_REPLACE);
- cfg->fc_nlinfo.nlh->nlmsg_flags |= NLM_F_CREATE;
+ if (cfg->fc_nlinfo.nlh) {
+ cfg->fc_nlinfo.nlh->nlmsg_flags &= ~(NLM_F_EXCL |
+ NLM_F_REPLACE);
+ cfg->fc_nlinfo.nlh->nlmsg_flags |= NLM_F_CREATE;
+ }
nhn++;
}
if (!ipv6_unicast_destination(skb))
goto drop;
+ if (ipv6_addr_v4mapped(&ipv6_hdr(skb)->saddr)) {
+ __IP6_INC_STATS(sock_net(sk), NULL, IPSTATS_MIB_INHDRERRORS);
+ return 0;
+ }
+
return tcp_conn_request(&tcp6_request_sock_ops,
&tcp_request_sock_ipv6_ops, sk, skb);
struct aead_request *aead_req;
int reqsize = sizeof(*aead_req) + crypto_aead_reqsize(tfm);
u8 *__aad;
+ int ret;
aead_req = kzalloc(reqsize + aad_len, GFP_ATOMIC);
if (!aead_req)
aead_request_set_crypt(aead_req, sg, sg, data_len, b_0);
aead_request_set_ad(aead_req, sg[0].length);
- crypto_aead_encrypt(aead_req);
+ ret = crypto_aead_encrypt(aead_req);
kfree_sensitive(aead_req);
- return 0;
+ return ret;
}
int aead_decrypt(struct crypto_aead *tfm, u8 *b_0, u8 *aad, size_t aad_len,
struct aead_request *aead_req;
int reqsize = sizeof(*aead_req) + crypto_aead_reqsize(tfm);
const __le16 *fc;
+ int ret;
if (data_len < GMAC_MIC_LEN)
return -EINVAL;
aead_request_set_crypt(aead_req, sg, sg, 0, iv);
aead_request_set_ad(aead_req, GMAC_AAD_LEN + data_len);
- crypto_aead_encrypt(aead_req);
+ ret = crypto_aead_encrypt(aead_req);
kfree_sensitive(aead_req);
- return 0;
+ return ret;
}
struct crypto_aead *ieee80211_aes_gmac_key_setup(const u8 key[],
}
if (sta->sdata->vif.type == NL80211_IFTYPE_AP_VLAN &&
- sta->sdata->u.vlan.sta)
+ sta->sdata->u.vlan.sta) {
+ ieee80211_clear_fast_rx(sta);
RCU_INIT_POINTER(sta->sdata->u.vlan.sta, NULL);
+ }
if (test_sta_flag(sta, WLAN_STA_AUTHORIZED))
ieee80211_vif_dec_num_mcast(sta->sdata);
continue;
for (j = 0; j < IEEE80211_HT_MCS_MASK_LEN; j++) {
- if (~sdata->rc_rateidx_mcs_mask[i][j]) {
+ if (sdata->rc_rateidx_mcs_mask[i][j] != 0xff) {
sdata->rc_has_mcs_mask[i] = true;
break;
}
}
for (j = 0; j < NL80211_VHT_NSS_MAX; j++) {
- if (~sdata->rc_rateidx_vht_mcs_mask[i][j]) {
+ if (sdata->rc_rateidx_vht_mcs_mask[i][j] != 0xffff) {
sdata->rc_has_vht_mcs_mask[i] = true;
break;
}
/* remove beacon */
kfree(sdata->u.ibss.ie);
+ sdata->u.ibss.ie = NULL;
+ sdata->u.ibss.ie_len = 0;
/* on the next join, re-program HT parameters */
memset(&ifibss->ht_capa, 0, sizeof(ifibss->ht_capa));
continue;
if (!dflt_chandef.chan) {
+ /*
+ * Assign the first enabled channel to dflt_chandef
+ * from the list of channels
+ */
+ for (i = 0; i < sband->n_channels; i++)
+ if (!(sband->channels[i].flags &
+ IEEE80211_CHAN_DISABLED))
+ break;
+ /* if none found then use the first anyway */
+ if (i == sband->n_channels)
+ i = 0;
cfg80211_chandef_create(&dflt_chandef,
- &sband->channels[0],
+ &sband->channels[i],
NL80211_CHAN_NO_HT);
/* init channel we're on */
if (!local->use_chanctx && !local->_oper_chandef.chan) {
timeout = sta->rx_stats.last_rx;
timeout += IEEE80211_CONNECTION_IDLE_TIME;
- if (time_is_before_jiffies(timeout)) {
+ /* If timeout is after now, then update timer to fire at
+ * the later date, but do not actually probe at this time.
+ */
+ if (time_is_after_jiffies(timeout)) {
mod_timer(&ifmgd->conn_mon_timer, round_jiffies_up(timeout));
return;
}
he_oper_ie = cfg80211_find_ext_ie(WLAN_EID_EXT_HE_OPERATION,
ies->data, ies->len);
if (he_oper_ie &&
- he_oper_ie[1] == ieee80211_he_oper_size(&he_oper_ie[3]))
+ he_oper_ie[1] >= ieee80211_he_oper_size(&he_oper_ie[3]))
he_oper = (void *)(he_oper_ie + 3);
else
he_oper = NULL;
static u16
minstrel_ht_next_inc_rate(struct minstrel_ht_sta *mi, u32 fast_rate_dur)
{
- struct minstrel_mcs_group_data *mg;
u8 type = MINSTREL_SAMPLE_TYPE_INC;
int i, index = 0;
u8 group;
group = mi->sample[type].sample_group;
for (i = 0; i < ARRAY_SIZE(minstrel_mcs_groups); i++) {
group = (group + 1) % ARRAY_SIZE(minstrel_mcs_groups);
- mg = &mi->groups[group];
index = minstrel_ht_group_min_rate_offset(mi, group,
fast_rate_dur);
test_bit(IEEE80211_TXQ_STOP_NETIF_TX, &txqi->flags))
goto out;
- if (vif->txqs_stopped[ieee80211_ac_from_tid(txq->tid)]) {
+ if (vif->txqs_stopped[txq->ac]) {
set_bit(IEEE80211_TXQ_STOP_NETIF_TX, &txqi->flags);
goto out;
}
break;
case WLAN_EID_EXT_HE_OPERATION:
if (len >= sizeof(*elems->he_operation) &&
- len == ieee80211_he_oper_size(data) - 1) {
+ len >= ieee80211_he_oper_size(data) - 1) {
if (crc)
*crc = crc32_be(*crc, (void *)elem,
elem->datalen + 2);
crypto_free_sync_skcipher(key->tfm0);
err_tfm:
for (i = 0; i < ARRAY_SIZE(key->tfm); i++)
- if (key->tfm[i])
+ if (!IS_ERR_OR_NULL(key->tfm[i]))
crypto_free_aead(key->tfm[i]);
kfree_sensitive(key);
}
static u64 add_addr_generate_hmac(u64 key1, u64 key2, u8 addr_id,
- struct in_addr *addr)
+ struct in_addr *addr, u16 port)
{
u8 hmac[SHA256_DIGEST_SIZE];
u8 msg[7];
msg[0] = addr_id;
memcpy(&msg[1], &addr->s_addr, 4);
- msg[5] = 0;
- msg[6] = 0;
+ msg[5] = port >> 8;
+ msg[6] = port & 0xFF;
mptcp_crypto_hmac_sha(key1, key2, msg, 7, hmac);
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
static u64 add_addr6_generate_hmac(u64 key1, u64 key2, u8 addr_id,
- struct in6_addr *addr)
+ struct in6_addr *addr, u16 port)
{
u8 hmac[SHA256_DIGEST_SIZE];
u8 msg[19];
msg[0] = addr_id;
memcpy(&msg[1], &addr->s6_addr, 16);
- msg[17] = 0;
- msg[18] = 0;
+ msg[17] = port >> 8;
+ msg[18] = port & 0xFF;
mptcp_crypto_hmac_sha(key1, key2, msg, 19, hmac);
opts->ahmac = add_addr_generate_hmac(msk->local_key,
msk->remote_key,
opts->addr_id,
- &opts->addr);
+ &opts->addr,
+ opts->port);
}
}
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
opts->ahmac = add_addr6_generate_hmac(msk->local_key,
msk->remote_key,
opts->addr_id,
- &opts->addr6);
+ &opts->addr6,
+ opts->port);
}
}
#endif
if (mp_opt->family == MPTCP_ADDR_IPVERSION_4)
hmac = add_addr_generate_hmac(msk->remote_key,
msk->local_key,
- mp_opt->addr_id, &mp_opt->addr);
+ mp_opt->addr_id, &mp_opt->addr,
+ mp_opt->port);
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
else
hmac = add_addr6_generate_hmac(msk->remote_key,
msk->local_key,
- mp_opt->addr_id, &mp_opt->addr6);
+ mp_opt->addr_id, &mp_opt->addr6,
+ mp_opt->port);
#endif
pr_debug("msk=%p, ahmac=%llu, mp_opt->ahmac=%llu\n",
#include <linux/netdevice.h>
#include <linux/sched/signal.h>
#include <linux/atomic.h>
-#include <linux/igmp.h>
#include <net/sock.h>
#include <net/inet_common.h>
#include <net/inet_hashtables.h>
#include <net/tcp_states.h>
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
#include <net/transp_v6.h>
-#include <net/addrconf.h>
#endif
#include <net/mptcp.h>
#include <net/xfrm.h>
return ret;
}
+static bool mptcp_unsupported(int level, int optname)
+{
+ if (level == SOL_IP) {
+ switch (optname) {
+ case IP_ADD_MEMBERSHIP:
+ case IP_ADD_SOURCE_MEMBERSHIP:
+ case IP_DROP_MEMBERSHIP:
+ case IP_DROP_SOURCE_MEMBERSHIP:
+ case IP_BLOCK_SOURCE:
+ case IP_UNBLOCK_SOURCE:
+ case MCAST_JOIN_GROUP:
+ case MCAST_LEAVE_GROUP:
+ case MCAST_JOIN_SOURCE_GROUP:
+ case MCAST_LEAVE_SOURCE_GROUP:
+ case MCAST_BLOCK_SOURCE:
+ case MCAST_UNBLOCK_SOURCE:
+ case MCAST_MSFILTER:
+ return true;
+ }
+ return false;
+ }
+ if (level == SOL_IPV6) {
+ switch (optname) {
+ case IPV6_ADDRFORM:
+ case IPV6_ADD_MEMBERSHIP:
+ case IPV6_DROP_MEMBERSHIP:
+ case IPV6_JOIN_ANYCAST:
+ case IPV6_LEAVE_ANYCAST:
+ case MCAST_JOIN_GROUP:
+ case MCAST_LEAVE_GROUP:
+ case MCAST_JOIN_SOURCE_GROUP:
+ case MCAST_LEAVE_SOURCE_GROUP:
+ case MCAST_BLOCK_SOURCE:
+ case MCAST_UNBLOCK_SOURCE:
+ case MCAST_MSFILTER:
+ return true;
+ }
+ return false;
+ }
+ return false;
+}
+
static int mptcp_setsockopt(struct sock *sk, int level, int optname,
sockptr_t optval, unsigned int optlen)
{
pr_debug("msk=%p", msk);
+ if (mptcp_unsupported(level, optname))
+ return -ENOPROTOOPT;
+
if (level == SOL_SOCKET)
return mptcp_setsockopt_sol_socket(msk, optname, optval, optlen);
for (;;) {
flags = 0;
if (test_and_clear_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags))
- flags |= MPTCP_PUSH_PENDING;
+ flags |= BIT(MPTCP_PUSH_PENDING);
if (!flags)
break;
*/
spin_unlock_bh(&sk->sk_lock.slock);
- if (flags & MPTCP_PUSH_PENDING)
+ if (flags & BIT(MPTCP_PUSH_PENDING))
__mptcp_push_pending(sk, 0);
cond_resched();
return mask;
}
-static int mptcp_release(struct socket *sock)
-{
- struct mptcp_subflow_context *subflow;
- struct sock *sk = sock->sk;
- struct mptcp_sock *msk;
-
- if (!sk)
- return 0;
-
- lock_sock(sk);
-
- msk = mptcp_sk(sk);
-
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
-
- ip_mc_drop_socket(ssk);
- }
-
- release_sock(sk);
-
- return inet_release(sock);
-}
-
static const struct proto_ops mptcp_stream_ops = {
.family = PF_INET,
.owner = THIS_MODULE,
- .release = mptcp_release,
+ .release = inet_release,
.bind = mptcp_bind,
.connect = mptcp_stream_connect,
.socketpair = sock_no_socketpair,
}
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
-static int mptcp6_release(struct socket *sock)
-{
- struct mptcp_subflow_context *subflow;
- struct mptcp_sock *msk;
- struct sock *sk = sock->sk;
-
- if (!sk)
- return 0;
-
- lock_sock(sk);
-
- msk = mptcp_sk(sk);
-
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
-
- ip_mc_drop_socket(ssk);
- ipv6_sock_mc_close(ssk);
- ipv6_sock_ac_close(ssk);
- }
-
- release_sock(sk);
- return inet6_release(sock);
-}
-
static const struct proto_ops mptcp_v6_stream_ops = {
.family = PF_INET6,
.owner = THIS_MODULE,
- .release = mptcp6_release,
+ .release = inet6_release,
.bind = mptcp_bind,
.connect = mptcp_stream_connect,
.socketpair = sock_no_socketpair,
if (!ipv6_unicast_destination(skb))
goto drop;
+ if (ipv6_addr_v4mapped(&ipv6_hdr(skb)->saddr)) {
+ __IP6_INC_STATS(sock_net(sk), NULL, IPSTATS_MIB_INHDRERRORS);
+ return 0;
+ }
+
return tcp_conn_request(&mptcp_subflow_request_sock_ops,
&subflow_request_sock_ipv6_ops, sk, skb);
monitor_state = nc->monitor.state;
spin_unlock_irqrestore(&nc->lock, flags);
- if (!enabled || chained) {
- ncsi_stop_channel_monitor(nc);
- return;
- }
+ if (!enabled)
+ return; /* expected race disabling timer */
+ if (WARN_ON_ONCE(chained))
+ goto bad_state;
+
if (state != NCSI_CHANNEL_INACTIVE &&
state != NCSI_CHANNEL_ACTIVE) {
- ncsi_stop_channel_monitor(nc);
+bad_state:
+ netdev_warn(ndp->ndev.dev,
+ "Bad NCSI monitor state channel %d 0x%x %s queue\n",
+ nc->id, state, chained ? "on" : "off");
+ spin_lock_irqsave(&nc->lock, flags);
+ nc->monitor.enabled = false;
+ spin_unlock_irqrestore(&nc->lock, flags);
return;
}
ncsi_report_link(ndp, true);
ndp->flags |= NCSI_DEV_RESHUFFLE;
- ncsi_stop_channel_monitor(nc);
-
ncm = &nc->modes[NCSI_MODE_LINK];
spin_lock_irqsave(&nc->lock, flags);
+ nc->monitor.enabled = false;
nc->state = NCSI_CHANNEL_INVISIBLE;
ncm->data[2] &= ~0x1;
spin_unlock_irqrestore(&nc->lock, flags);
memset(&m, 0xFF, sizeof(m));
memcpy(&m.src.u3, &mask->src.u3, sizeof(m.src.u3));
m.src.u.all = mask->src.u.all;
+ m.src.l3num = tuple->src.l3num;
m.dst.protonum = tuple->dst.protonum;
nest_parms = nla_nest_start(skb, CTA_EXPECT_MASK);
enum ip_conntrack_info ctinfo,
const struct nf_hook_state *state)
{
- if (state->pf != NFPROTO_IPV4)
- return -NF_ACCEPT;
-
if (!nf_ct_is_confirmed(ct)) {
unsigned int *timeouts = nf_ct_timeout_lookup(ct);
{
int err;
- INIT_DEFERRABLE_WORK(&flowtable->gc_work, nf_flow_offload_work_gc);
+ INIT_DELAYED_WORK(&flowtable->gc_work, nf_flow_offload_work_gc);
flow_block_init(&flowtable->flow_block);
init_rwsem(&flowtable->flow_block_lock);
list_for_each_entry(hook, hook_list, list) {
list_for_each_entry(ft, &table->flowtables, list) {
+ if (!nft_is_active_next(net, ft))
+ continue;
+
list_for_each_entry(hook2, &ft->hook_list, list) {
if (hook->ops.dev == hook2->ops.dev &&
hook->ops.pf == hook2->ops.pf) {
struct nft_hook *hook, *next;
struct nft_trans *trans;
bool unregister = false;
+ u32 flags;
int err;
err = nft_flowtable_parse_hook(ctx, nla[NFTA_FLOWTABLE_HOOK],
}
}
+ if (nla[NFTA_FLOWTABLE_FLAGS]) {
+ flags = ntohl(nla_get_be32(nla[NFTA_FLOWTABLE_FLAGS]));
+ if (flags & ~NFT_FLOWTABLE_MASK)
+ return -EOPNOTSUPP;
+ if ((flowtable->data.flags & NFT_FLOWTABLE_HW_OFFLOAD) ^
+ (flags & NFT_FLOWTABLE_HW_OFFLOAD))
+ return -EOPNOTSUPP;
+ } else {
+ flags = flowtable->data.flags;
+ }
+
err = nft_register_flowtable_net_hooks(ctx->net, ctx->table,
&flowtable_hook.list, flowtable);
if (err < 0)
goto err_flowtable_update_hook;
}
+ nft_trans_flowtable_flags(trans) = flags;
nft_trans_flowtable(trans) = flowtable;
nft_trans_flowtable_update(trans) = true;
INIT_LIST_HEAD(&nft_trans_flowtable_hooks(trans));
if (nla[NFTA_FLOWTABLE_FLAGS]) {
flowtable->data.flags =
ntohl(nla_get_be32(nla[NFTA_FLOWTABLE_FLAGS]));
- if (flowtable->data.flags & ~NFT_FLOWTABLE_MASK)
+ if (flowtable->data.flags & ~NFT_FLOWTABLE_MASK) {
+ err = -EOPNOTSUPP;
goto err3;
+ }
}
write_pnet(&flowtable->data.net, net);
break;
case NFT_MSG_NEWFLOWTABLE:
if (nft_trans_flowtable_update(trans)) {
+ nft_trans_flowtable(trans)->data.flags =
+ nft_trans_flowtable_flags(trans);
nf_tables_flowtable_notify(&trans->ctx,
nft_trans_flowtable(trans),
&nft_trans_flowtable_hooks(trans),
}
EXPORT_SYMBOL(xt_counters_alloc);
-struct xt_table_info
-*xt_table_get_private_protected(const struct xt_table *table)
-{
- return rcu_dereference_protected(table->private,
- mutex_is_locked(&xt[table->af].mutex));
-}
-EXPORT_SYMBOL(xt_table_get_private_protected);
-
struct xt_table_info *
xt_replace_table(struct xt_table *table,
unsigned int num_counters,
int *error)
{
struct xt_table_info *private;
+ unsigned int cpu;
int ret;
ret = xt_jumpstack_alloc(newinfo);
}
/* Do the substitution. */
- private = xt_table_get_private_protected(table);
+ local_bh_disable();
+ private = table->private;
/* Check inside lock: is the old number correct? */
if (num_counters != private->number) {
pr_debug("num_counters != table->private->number (%u/%u)\n",
num_counters, private->number);
+ local_bh_enable();
*error = -EAGAIN;
return NULL;
}
newinfo->initial_entries = private->initial_entries;
+ /*
+ * Ensure contents of newinfo are visible before assigning to
+ * private.
+ */
+ smp_wmb();
+ table->private = newinfo;
+
+ /* make sure all cpus see new ->private value */
+ smp_mb();
- rcu_assign_pointer(table->private, newinfo);
- synchronize_rcu();
+ /*
+ * Even though table entries have now been swapped, other CPU's
+ * may still be using the old entries...
+ */
+ local_bh_enable();
+
+ /* ... so wait for even xt_recseq on all cpus */
+ for_each_possible_cpu(cpu) {
+ seqcount_t *s = &per_cpu(xt_recseq, cpu);
+ u32 seq = raw_read_seqcount(s);
+
+ if (seq & 1) {
+ do {
+ cond_resched();
+ cpu_relax();
+ } while (seq == raw_read_seqcount(s));
+ }
+ }
audit_log_nfcfg(table->name, table->af, private->number,
!private->number ? AUDIT_XT_OP_REGISTER :
}
/* Simplifies replace_table code. */
- rcu_assign_pointer(table->private, bootstrap);
+ table->private = bootstrap;
if (!xt_replace_table(table, 0, newinfo, &ret))
goto unlock;
- private = xt_table_get_private_protected(table);
+ private = table->private;
pr_debug("table->private->number = %u\n", private->number);
/* save number of initial entries */
struct xt_table_info *private;
mutex_lock(&xt[table->af].mutex);
- private = xt_table_get_private_protected(table);
- RCU_INIT_POINTER(table->private, NULL);
+ private = table->private;
list_del(&table->list);
mutex_unlock(&xt[table->af].mutex);
audit_log_nfcfg(table->name, table->af, private->number,
llcp_sock->service_name_len,
GFP_KERNEL);
if (!llcp_sock->service_name) {
+ nfc_llcp_local_put(llcp_sock->local);
ret = -ENOMEM;
goto put_dev;
}
llcp_sock->ssap = nfc_llcp_get_sdp_ssap(local, llcp_sock);
if (llcp_sock->ssap == LLCP_SAP_MAX) {
+ nfc_llcp_local_put(llcp_sock->local);
kfree(llcp_sock->service_name);
llcp_sock->service_name = NULL;
ret = -EADDRINUSE;
ret = -EISCONN;
goto error;
}
+ if (sk->sk_state == LLCP_CONNECTING) {
+ ret = -EINPROGRESS;
+ goto error;
+ }
dev = nfc_get_device(addr->dev_idx);
if (dev == NULL) {
llcp_sock->local = nfc_llcp_local_get(local);
llcp_sock->ssap = nfc_llcp_get_local_ssap(local);
if (llcp_sock->ssap == LLCP_SAP_MAX) {
+ nfc_llcp_local_put(llcp_sock->local);
ret = -ENOMEM;
goto put_dev;
}
sock_unlink:
nfc_llcp_sock_unlink(&local->connecting_sockets, sk);
+ kfree(llcp_sock->service_name);
+ llcp_sock->service_name = NULL;
sock_llcp_release:
nfc_llcp_put_ssap(local, llcp_sock->ssap);
+ nfc_llcp_local_put(llcp_sock->local);
put_dev:
nfc_put_device(dev);
/* This is called to initialize CT key fields possibly coming in from the local
* stack.
*/
-void ovs_ct_fill_key(const struct sk_buff *skb, struct sw_flow_key *key)
+void ovs_ct_fill_key(const struct sk_buff *skb,
+ struct sw_flow_key *key,
+ bool post_ct)
{
- ovs_ct_update_key(skb, NULL, key, false, false);
+ ovs_ct_update_key(skb, NULL, key, post_ct, false);
}
int ovs_ct_put_key(const struct sw_flow_key *swkey,
if (skb_nfct(skb)) {
nf_conntrack_put(skb_nfct(skb));
nf_ct_set(skb, NULL, IP_CT_UNTRACKED);
- ovs_ct_fill_key(skb, key);
+ ovs_ct_fill_key(skb, key, false);
}
return 0;
static int ovs_ct_limit_get_default_limit(struct ovs_ct_limit_info *info,
struct sk_buff *reply)
{
- struct ovs_zone_limit zone_limit;
-
- zone_limit.zone_id = OVS_ZONE_LIMIT_DEFAULT_ZONE;
- zone_limit.limit = info->default_limit;
+ struct ovs_zone_limit zone_limit = {
+ .zone_id = OVS_ZONE_LIMIT_DEFAULT_ZONE,
+ .limit = info->default_limit,
+ };
return nla_put_nohdr(reply, sizeof(zone_limit), &zone_limit);
}
const struct ovs_conntrack_info *);
int ovs_ct_clear(struct sk_buff *skb, struct sw_flow_key *key);
-void ovs_ct_fill_key(const struct sk_buff *skb, struct sw_flow_key *key);
+void ovs_ct_fill_key(const struct sk_buff *skb, struct sw_flow_key *key,
+ bool post_ct);
int ovs_ct_put_key(const struct sw_flow_key *swkey,
const struct sw_flow_key *output, struct sk_buff *skb);
void ovs_ct_free_action(const struct nlattr *a);
}
static inline void ovs_ct_fill_key(const struct sk_buff *skb,
- struct sw_flow_key *key)
+ struct sw_flow_key *key,
+ bool post_ct)
{
key->ct_state = 0;
key->ct_zone = 0;
#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
struct tc_skb_ext *tc_ext;
#endif
+ bool post_ct = false;
int res, err;
/* Extract metadata from packet. */
tc_ext = skb_ext_find(skb, TC_SKB_EXT);
key->recirc_id = tc_ext ? tc_ext->chain : 0;
OVS_CB(skb)->mru = tc_ext ? tc_ext->mru : 0;
+ post_ct = tc_ext ? tc_ext->post_ct : false;
} else {
key->recirc_id = 0;
}
err = key_extract(skb, key);
if (!err)
- ovs_ct_fill_key(skb, key); /* Must be after key_extract(). */
+ ovs_ct_fill_key(skb, key, post_ct); /* Must be after key_extract(). */
return err;
}
flow = kzalloc(sizeof(*flow), GFP_KERNEL);
if (flow) {
init_waitqueue_head(&flow->resume_tx);
- radix_tree_insert(&node->qrtr_tx_flow, key, flow);
+ if (radix_tree_insert(&node->qrtr_tx_flow, key, flow)) {
+ kfree(flow);
+ flow = NULL;
+ }
}
}
mutex_unlock(&node->qrtr_tx_lock);
rc = copied;
if (addr) {
+ /* There is an anonymous 2-byte hole after sq_family,
+ * make sure to clear it.
+ */
+ memset(addr, 0, sizeof(*addr));
+
addr->sq_family = AF_QIPCRTR;
addr->sq_node = cb->src_node;
addr->sq_port = cb->src_port;
rds_message_purge(rm);
kfree(rm);
+ rm = NULL;
}
}
EXPORT_SYMBOL_GPL(rds_message_put);
rm->data.op_nents = DIV_ROUND_UP(total_len, PAGE_SIZE);
rm->data.op_sg = rds_message_alloc_sgs(rm, num_sgs);
if (IS_ERR(rm->data.op_sg)) {
+ void *err = ERR_CAST(rm->data.op_sg);
rds_message_put(rm);
- return ERR_CAST(rm->data.op_sg);
+ return err;
}
for (i = 0; i < rm->data.op_nents; ++i) {
unlock_and_drop:
spin_unlock_irqrestore(&rm->m_rs_lock, flags);
rds_message_put(rm);
- if (was_on_sock)
+ if (was_on_sock && rm)
rds_message_put(rm);
}
struct rfkill_int_event {
struct list_head list;
- struct rfkill_event ev;
+ struct rfkill_event_ext ev;
};
struct rfkill_data {
}
#endif /* CONFIG_RFKILL_LEDS */
-static void rfkill_fill_event(struct rfkill_event *ev, struct rfkill *rfkill,
+static void rfkill_fill_event(struct rfkill_event_ext *ev,
+ struct rfkill *rfkill,
enum rfkill_operation op)
{
unsigned long flags;
size_t count, loff_t *pos)
{
struct rfkill *rfkill;
- struct rfkill_event ev;
+ struct rfkill_event_ext ev;
int ret;
/* we don't need the 'hard' variable but accept it */
return 0;
}
-int __tcf_idr_release(struct tc_action *p, bool bind, bool strict)
+static int __tcf_idr_release(struct tc_action *p, bool bind, bool strict)
{
int ret = 0;
return ret;
}
-EXPORT_SYMBOL(__tcf_idr_release);
+
+int tcf_idr_release(struct tc_action *a, bool bind)
+{
+ const struct tc_action_ops *ops = a->ops;
+ int ret;
+
+ ret = __tcf_idr_release(a, bind, false);
+ if (ret == ACT_P_DELETED)
+ module_put(ops->owner);
+ return ret;
+}
+EXPORT_SYMBOL(tcf_idr_release);
static size_t tcf_action_shared_attrs_size(const struct tc_action *act)
{
}
p->idrinfo = idrinfo;
+ __module_get(ops->owner);
p->ops = ops;
*a = p;
return 0;
struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp,
struct nlattr *nla, struct nlattr *est,
char *name, int ovr, int bind,
- struct tc_action_ops *a_o, bool rtnl_held,
+ struct tc_action_ops *a_o, int *init_res,
+ bool rtnl_held,
struct netlink_ext_ack *extack)
{
struct nla_bitfield32 flags = { 0, 0 };
}
if (err < 0)
goto err_out;
+ *init_res = err;
if (!name && tb[TCA_ACT_COOKIE])
tcf_set_action_cookie(&a->act_cookie, cookie);
if (!name)
a->hw_stats = hw_stats;
- /* module count goes up only when brand new policy is created
- * if it exists and is only bound to in a_o->init() then
- * ACT_P_CREATED is not returned (a zero is).
- */
- if (err != ACT_P_CREATED)
- module_put(a_o->owner);
-
return a;
err_out:
int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
struct nlattr *est, char *name, int ovr, int bind,
- struct tc_action *actions[], size_t *attr_size,
+ struct tc_action *actions[], int init_res[], size_t *attr_size,
bool rtnl_held, struct netlink_ext_ack *extack)
{
struct tc_action_ops *ops[TCA_ACT_MAX_PRIO] = {};
for (i = 1; i <= TCA_ACT_MAX_PRIO && tb[i]; i++) {
act = tcf_action_init_1(net, tp, tb[i], est, name, ovr, bind,
- ops[i - 1], rtnl_held, extack);
+ ops[i - 1], &init_res[i - 1], rtnl_held,
+ extack);
if (IS_ERR(act)) {
err = PTR_ERR(act);
goto err;
tcf_idr_insert_many(actions);
*attr_size = tcf_action_full_attrs_size(sz);
- return i - 1;
+ err = i - 1;
+ goto err_mod;
err:
tcf_action_destroy(actions, bind);
struct netlink_ext_ack *extack)
{
size_t attr_size = 0;
- int loop, ret;
+ int loop, ret, i;
struct tc_action *actions[TCA_ACT_MAX_PRIO] = {};
+ int init_res[TCA_ACT_MAX_PRIO] = {};
for (loop = 0; loop < 10; loop++) {
ret = tcf_action_init(net, NULL, nla, NULL, NULL, ovr, 0,
- actions, &attr_size, true, extack);
+ actions, init_res, &attr_size, true, extack);
if (ret != -EAGAIN)
break;
}
if (ret < 0)
return ret;
ret = tcf_add_notify(net, n, actions, portid, attr_size, extack);
- if (ovr)
- tcf_action_put_many(actions);
+
+ /* only put existing actions */
+ for (i = 0; i < TCA_ACT_MAX_PRIO; i++)
+ if (init_res[i] == ACT_P_CREATED)
+ actions[i] = NULL;
+ tcf_action_put_many(actions);
return ret;
}
tcf_lastuse_update(&c->tcf_tm);
if (clear) {
+ qdisc_skb_cb(skb)->post_ct = false;
ct = nf_ct_get(skb, &ctinfo);
if (ct) {
nf_conntrack_put(&ct->ct_general);
nf_ct_set(skb, NULL, IP_CT_UNTRACKED);
}
- goto out;
+ goto out_clear;
}
family = tcf_ct_skb_nf_family(skb);
skb_push_rcsum(skb, nh_ofs);
out:
- tcf_action_update_bstats(&c->common, skb);
qdisc_skb_cb(skb)->post_ct = true;
+out_clear:
+ tcf_action_update_bstats(&c->common, skb);
if (defrag)
qdisc_skb_cb(skb)->pkt_len = skb->len;
return retval;
struct net_device *dev = block_cb->indr.dev;
struct Qdisc *sch = block_cb->indr.sch;
struct netlink_ext_ack extack = {};
- struct flow_block_offload bo;
+ struct flow_block_offload bo = {};
tcf_block_offload_init(&bo, dev, sch, FLOW_BLOCK_UNBIND,
block_cb->indr.binder_type,
return TC_ACT_SHOT;
ext->chain = last_executed_chain;
ext->mru = qdisc_skb_cb(skb)->mru;
+ ext->post_ct = qdisc_skb_cb(skb)->post_ct;
}
return ret;
{
#ifdef CONFIG_NET_CLS_ACT
{
+ int init_res[TCA_ACT_MAX_PRIO] = {};
struct tc_action *act;
size_t attr_size = 0;
return PTR_ERR(a_o);
act = tcf_action_init_1(net, tp, tb[exts->police],
rate_tlv, "police", ovr,
- TCA_ACT_BIND, a_o, rtnl_held,
- extack);
- if (IS_ERR(act)) {
- module_put(a_o->owner);
+ TCA_ACT_BIND, a_o, init_res,
+ rtnl_held, extack);
+ module_put(a_o->owner);
+ if (IS_ERR(act))
return PTR_ERR(act);
- }
act->type = exts->type = TCA_OLD_COMPAT;
exts->actions[0] = act;
err = tcf_action_init(net, tp, tb[exts->action],
rate_tlv, NULL, ovr, TCA_ACT_BIND,
- exts->actions, &attr_size,
- rtnl_held, extack);
+ exts->actions, init_res,
+ &attr_size, rtnl_held, extack);
if (err < 0)
return err;
exts->nr_actions = err;
&mask->ct_state, TCA_FLOWER_KEY_CT_STATE_MASK,
sizeof(key->ct_state));
- err = fl_validate_ct_state(mask->ct_state,
+ err = fl_validate_ct_state(key->ct_state & mask->ct_state,
tb[TCA_FLOWER_KEY_CT_STATE_MASK],
extack);
if (err)
struct sk_buff **old = NULL;
unsigned int mask;
u32 max_P;
+ u8 *stab;
if (opt == NULL)
return -EINVAL;
max_P = tb[TCA_CHOKE_MAX_P] ? nla_get_u32(tb[TCA_CHOKE_MAX_P]) : 0;
ctl = nla_data(tb[TCA_CHOKE_PARMS]);
-
- if (!red_check_params(ctl->qth_min, ctl->qth_max, ctl->Wlog, ctl->Scell_log))
+ stab = nla_data(tb[TCA_CHOKE_STAB]);
+ if (!red_check_params(ctl->qth_min, ctl->qth_max, ctl->Wlog, ctl->Scell_log, stab))
return -EINVAL;
if (ctl->limit > CHOKE_MAX_QUEUE)
red_set_parms(&q->parms, ctl->qth_min, ctl->qth_max, ctl->Wlog,
ctl->Plog, ctl->Scell_log,
- nla_data(tb[TCA_CHOKE_STAB]),
+ stab,
max_P);
red_set_vars(&q->vars);
struct gred_sched *table = qdisc_priv(sch);
struct gred_sched_data *q = table->tab[dp];
- if (!red_check_params(ctl->qth_min, ctl->qth_max, ctl->Wlog, ctl->Scell_log)) {
+ if (!red_check_params(ctl->qth_min, ctl->qth_max, ctl->Wlog, ctl->Scell_log, stab)) {
NL_SET_ERR_MSG_MOD(extack, "invalid RED parameters");
return -EINVAL;
}
struct nlattr *tb[TCA_HTB_MAX + 1];
struct tc_htb_glob *gopt;
unsigned int ntx;
+ bool offload;
int err;
qdisc_watchdog_init(&q->watchdog, sch);
if (gopt->version != HTB_VER >> 16)
return -EINVAL;
- q->offload = nla_get_flag(tb[TCA_HTB_OFFLOAD]);
+ offload = nla_get_flag(tb[TCA_HTB_OFFLOAD]);
- if (q->offload) {
+ if (offload) {
if (sch->parent != TC_H_ROOT)
return -EOPNOTSUPP;
q->rate2quantum = 1;
q->defcls = gopt->defcls;
- if (!q->offload)
+ if (!offload)
return 0;
for (ntx = 0; ntx < q->num_direct_qdiscs; ntx++) {
if (err)
goto err_free_qdiscs;
+ /* Defer this assignment, so that htb_destroy skips offload-related
+ * parts (especially calling ndo_setup_tc) on errors.
+ */
+ q->offload = true;
+
return 0;
err_free_qdiscs:
- /* TC_HTB_CREATE call failed, avoid any further calls to the driver. */
- q->offload = false;
-
for (ntx = 0; ntx < q->num_direct_qdiscs && q->direct_qdiscs[ntx];
ntx++)
qdisc_put(q->direct_qdiscs[ntx]);
{
struct net_device *dev = qdisc_dev(sch);
struct tc_htb_qopt_offload offload_opt;
+ struct htb_sched *q = qdisc_priv(sch);
int err;
+ if (!q->offload)
+ return sch->dev_queue;
+
offload_opt = (struct tc_htb_qopt_offload) {
.command = TC_HTB_LEAF_QUERY_QUEUE,
.classid = TC_H_MIN(tcm->tcm_parent),
cl->parent->common.classid,
NULL);
if (q->offload) {
- if (new_q)
+ if (new_q) {
htb_set_lockdep_class_child(new_q);
- htb_parent_to_leaf_offload(sch, dev_queue, new_q);
+ htb_parent_to_leaf_offload(sch, dev_queue, new_q);
+ }
}
}
unsigned char flags;
int err;
u32 max_P;
+ u8 *stab;
if (tb[TCA_RED_PARMS] == NULL ||
tb[TCA_RED_STAB] == NULL)
max_P = tb[TCA_RED_MAX_P] ? nla_get_u32(tb[TCA_RED_MAX_P]) : 0;
ctl = nla_data(tb[TCA_RED_PARMS]);
- if (!red_check_params(ctl->qth_min, ctl->qth_max, ctl->Wlog, ctl->Scell_log))
+ stab = nla_data(tb[TCA_RED_STAB]);
+ if (!red_check_params(ctl->qth_min, ctl->qth_max, ctl->Wlog,
+ ctl->Scell_log, stab))
return -EINVAL;
err = red_get_flags(ctl->flags, TC_RED_HISTORIC_FLAGS,
red_set_parms(&q->parms,
ctl->qth_min, ctl->qth_max, ctl->Wlog,
ctl->Plog, ctl->Scell_log,
- nla_data(tb[TCA_RED_STAB]),
+ stab,
max_P);
red_set_vars(&q->vars);
}
if (ctl_v1 && !red_check_params(ctl_v1->qth_min, ctl_v1->qth_max,
- ctl_v1->Wlog, ctl_v1->Scell_log))
+ ctl_v1->Wlog, ctl_v1->Scell_log, NULL))
return -EINVAL;
if (ctl_v1 && ctl_v1->qth_min) {
p = kmalloc(sizeof(*p), GFP_KERNEL);
struct teql_sched_data *dat = qdisc_priv(sch);
struct teql_master *master = dat->m;
+ if (!master)
+ return;
+
prev = master->slaves;
if (prev) {
do {
if (!(type & IPV6_ADDR_UNICAST))
return 0;
- return sp->inet.freebind || net->ipv6.sysctl.ip_nonlocal_bind ||
- ipv6_chk_addr(net, in6, NULL, 0);
+ return ipv6_can_nonlocal_bind(net, &sp->inet) ||
+ ipv6_chk_addr(net, in6, NULL, 0);
}
/* This function checks if the address is a valid address to be used for
net = sock_net(&opt->inet.sk);
rcu_read_lock();
dev = dev_get_by_index_rcu(net, addr->v6.sin6_scope_id);
- if (!dev || !(opt->inet.freebind ||
- net->ipv6.sysctl.ip_nonlocal_bind ||
+ if (!dev || !(ipv6_can_nonlocal_bind(net, &opt->inet) ||
ipv6_chk_addr(net, &addr->v6.sin6_addr,
dev, 0))) {
rcu_read_unlock();
goto out;
}
- rcu_read_lock();
- if (__sk_dst_get(sk) != tp->dst) {
- dst_hold(tp->dst);
- sk_setup_caps(sk, tp->dst);
- }
- rcu_read_unlock();
-
/* pack up chunks */
pkt_count = sctp_packet_pack(packet, head, gso, gfp);
if (!pkt_count) {
static void sctp_outq_flush_transports(struct sctp_flush_ctx *ctx)
{
+ struct sock *sk = ctx->asoc->base.sk;
struct list_head *ltransport;
struct sctp_packet *packet;
struct sctp_transport *t;
t = list_entry(ltransport, struct sctp_transport, send_ready);
packet = &t->packet;
if (!sctp_packet_empty(packet)) {
+ rcu_read_lock();
+ if (t->dst && __sk_dst_get(sk) != t->dst) {
+ dst_hold(t->dst);
+ sk_setup_caps(sk, t->dst);
+ }
+ rcu_read_unlock();
error = sctp_packet_transmit(packet, ctx->gfp);
if (error < 0)
ctx->q->asoc->base.sk->sk_err = -error;
svcauth_gss_release(struct svc_rqst *rqstp)
{
struct gss_svc_data *gsd = (struct gss_svc_data *)rqstp->rq_auth_data;
- struct rpc_gss_wire_cred *gc = &gsd->clcred;
+ struct rpc_gss_wire_cred *gc;
struct xdr_buf *resbuf = &rqstp->rq_res;
int stat = -EINVAL;
struct sunrpc_net *sn = net_generic(SVC_NET(rqstp), sunrpc_net_id);
+ if (!gsd)
+ goto out;
+ gc = &gsd->clcred;
if (gc->gc_proc != RPC_GSS_PROC_DATA)
goto out;
/* Release can be called twice, but we only wrap once. */
if (rqstp->rq_cred.cr_group_info)
put_group_info(rqstp->rq_cred.cr_group_info);
rqstp->rq_cred.cr_group_info = NULL;
- if (gsd->rsci)
+ if (gsd && gsd->rsci) {
cache_put(&gsd->rsci->h, sn->rsc_cache);
- gsd->rsci = NULL;
-
+ gsd->rsci = NULL;
+ }
return stat;
}
sendit:
if (svc_authorise(rqstp))
- goto close;
+ goto close_xprt;
return 1; /* Caller can now send it */
release_dropit:
return 0;
close:
+ svc_authorise(rqstp);
+close_xprt:
if (rqstp->rq_xprt && test_bit(XPT_TEMP, &rqstp->rq_xprt->xpt_flags))
svc_close_xprt(rqstp->rq_xprt);
dprintk("svc: svc_process close\n");
err_short_len:
svc_printk(rqstp, "short len %zd, dropping request\n",
argv->iov_len);
- goto close;
+ goto close_xprt;
err_bad_rpc:
serv->sv_stats->rpcbadfmt++;
struct svc_xprt *xprt;
int ret = 0;
- spin_lock(&serv->sv_lock);
+ spin_lock_bh(&serv->sv_lock);
list_for_each_entry(xprt, xprt_list, xpt_list) {
if (xprt->xpt_net != net)
continue;
set_bit(XPT_CLOSE, &xprt->xpt_flags);
svc_xprt_enqueue(xprt);
}
- spin_unlock(&serv->sv_lock);
+ spin_unlock_bh(&serv->sv_lock);
return ret;
}
xprt->timeout = &xprt_rdma_bc_timeout;
xprt_set_bound(xprt);
xprt_set_connected(xprt);
- xprt->bind_timeout = RPCRDMA_BIND_TO;
- xprt->reestablish_timeout = RPCRDMA_INIT_REEST_TO;
- xprt->idle_timeout = RPCRDMA_IDLE_DISC_TO;
+ xprt->bind_timeout = 0;
+ xprt->reestablish_timeout = 0;
+ xprt->idle_timeout = 0;
xprt->prot = XPRT_TRANSPORT_BC_RDMA;
xprt->ops = &xprt_rdma_bc_procs;
svc_rdma_recv_ctxt_put(rdma, ctxt);
}
-static bool svc_rdma_refresh_recvs(struct svcxprt_rdma *rdma,
- unsigned int wanted, bool temp)
+static int __svc_rdma_post_recv(struct svcxprt_rdma *rdma,
+ struct svc_rdma_recv_ctxt *ctxt)
{
- const struct ib_recv_wr *bad_wr = NULL;
- struct svc_rdma_recv_ctxt *ctxt;
- struct ib_recv_wr *recv_chain;
int ret;
- recv_chain = NULL;
- while (wanted--) {
- ctxt = svc_rdma_recv_ctxt_get(rdma);
- if (!ctxt)
- break;
-
- trace_svcrdma_post_recv(ctxt);
- ctxt->rc_temp = temp;
- ctxt->rc_recv_wr.next = recv_chain;
- recv_chain = &ctxt->rc_recv_wr;
- rdma->sc_pending_recvs++;
- }
- if (!recv_chain)
- return false;
-
- ret = ib_post_recv(rdma->sc_qp, recv_chain, &bad_wr);
+ trace_svcrdma_post_recv(ctxt);
+ ret = ib_post_recv(rdma->sc_qp, &ctxt->rc_recv_wr, NULL);
if (ret)
goto err_post;
- return true;
+ return 0;
err_post:
- while (bad_wr) {
- ctxt = container_of(bad_wr, struct svc_rdma_recv_ctxt,
- rc_recv_wr);
- bad_wr = bad_wr->next;
- svc_rdma_recv_ctxt_put(rdma, ctxt);
- }
-
trace_svcrdma_rq_post_err(rdma, ret);
- /* Since we're destroying the xprt, no need to reset
- * sc_pending_recvs. */
- return false;
+ svc_rdma_recv_ctxt_put(rdma, ctxt);
+ return ret;
+}
+
+static int svc_rdma_post_recv(struct svcxprt_rdma *rdma)
+{
+ struct svc_rdma_recv_ctxt *ctxt;
+
+ if (test_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags))
+ return 0;
+ ctxt = svc_rdma_recv_ctxt_get(rdma);
+ if (!ctxt)
+ return -ENOMEM;
+ return __svc_rdma_post_recv(rdma, ctxt);
}
/**
*/
bool svc_rdma_post_recvs(struct svcxprt_rdma *rdma)
{
- return svc_rdma_refresh_recvs(rdma, rdma->sc_max_requests, true);
+ struct svc_rdma_recv_ctxt *ctxt;
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < rdma->sc_max_requests; i++) {
+ ctxt = svc_rdma_recv_ctxt_get(rdma);
+ if (!ctxt)
+ return false;
+ ctxt->rc_temp = true;
+ ret = __svc_rdma_post_recv(rdma, ctxt);
+ if (ret)
+ return false;
+ }
+ return true;
}
/**
* @cq: Completion Queue context
* @wc: Work Completion object
*
+ * NB: The svc_xprt/svcxprt_rdma is pinned whenever it's possible that
+ * the Receive completion handler could be running.
*/
static void svc_rdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc)
{
struct ib_cqe *cqe = wc->wr_cqe;
struct svc_rdma_recv_ctxt *ctxt;
- rdma->sc_pending_recvs--;
-
/* WARNING: Only wc->wr_cqe and wc->status are reliable */
ctxt = container_of(cqe, struct svc_rdma_recv_ctxt, rc_cqe);
if (wc->status != IB_WC_SUCCESS)
goto flushed;
+ if (svc_rdma_post_recv(rdma))
+ goto post_err;
+
/* All wc fields are now known to be valid */
ctxt->rc_byte_len = wc->byte_len;
spin_unlock(&rdma->sc_rq_dto_lock);
if (!test_bit(RDMAXPRT_CONN_PENDING, &rdma->sc_flags))
svc_xprt_enqueue(&rdma->sc_xprt);
-
- if (!test_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags) &&
- rdma->sc_pending_recvs < rdma->sc_max_requests)
- if (!svc_rdma_refresh_recvs(rdma, RPCRDMA_MAX_RECV_BATCH,
- false))
- goto post_err;
-
return;
flushed:
- svc_rdma_recv_ctxt_put(rdma, ctxt);
post_err:
+ svc_rdma_recv_ctxt_put(rdma, ctxt);
set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags);
svc_xprt_enqueue(&rdma->sc_xprt);
}
* care of initializing all other fields.
*/
struct tipc_bearer {
- void __rcu *media_ptr; /* initalized by media */
- u32 mtu; /* initalized by media */
- struct tipc_media_addr addr; /* initalized by media */
+ void __rcu *media_ptr; /* initialized by media */
+ u32 mtu; /* initialized by media */
+ struct tipc_media_addr addr; /* initialized by media */
char name[TIPC_MAX_BEARER_NAME];
struct tipc_media *media;
struct tipc_media_addr bcast_addr;
goto rcv;
if (tipc_aead_clone(&tmp, aead) < 0)
goto rcv;
+ WARN_ON(!refcount_inc_not_zero(&tmp->refcnt));
if (tipc_crypto_key_attach(rx, tmp, ehdr->tx_key, false) < 0) {
tipc_aead_free(&tmp->rcu);
goto rcv;
}
tipc_aead_put(aead);
- aead = tipc_aead_get(tmp);
+ aead = tmp;
}
if (unlikely(err)) {
* - A spin lock to protect the registry of kernel/driver users (reg.c)
* - A global spin_lock (tipc_port_lock), which only task is to ensure
* consistency where more than one port is involved in an operation,
- * i.e., whe a port is part of a linked list of ports.
+ * i.e., when a port is part of a linked list of ports.
* There are two such lists; 'port_list', which is used for management,
* and 'wait_list', which is used to queue ports during congestion.
*
}
/* tipc_node_xmit_skb(): send single buffer to destination
- * Buffers sent via this functon are generally TIPC_SYSTEM_IMPORTANCE
+ * Buffers sent via this function are generally TIPC_SYSTEM_IMPORTANCE
* messages, which will not be rejected
* The only exception is datagram messages rerouted after secondary
* lookup, which are rare and safe to dispose of anyway.
#ifdef CONFIG_TIPC_CRYPTO
static int tipc_nl_retrieve_key(struct nlattr **attrs,
- struct tipc_aead_key **key)
+ struct tipc_aead_key **pkey)
{
struct nlattr *attr = attrs[TIPC_NLA_NODE_KEY];
+ struct tipc_aead_key *key;
if (!attr)
return -ENODATA;
- *key = (struct tipc_aead_key *)nla_data(attr);
- if (nla_len(attr) < tipc_aead_key_size(*key))
+ if (nla_len(attr) < sizeof(*key))
+ return -EINVAL;
+ key = (struct tipc_aead_key *)nla_data(attr);
+ if (key->keylen > TIPC_AEAD_KEYLEN_MAX ||
+ nla_len(attr) < tipc_aead_key_size(key))
return -EINVAL;
+ *pkey = key;
return 0;
}
spin_lock_bh(&inputq->lock);
if (skb_peek(arrvq) == skb) {
skb_queue_splice_tail_init(&tmpq, inputq);
- kfree_skb(__skb_dequeue(arrvq));
+ __skb_dequeue(arrvq);
}
spin_unlock_bh(&inputq->lock);
__skb_queue_purge(&tmpq);
vsk->buffer_size = psk->buffer_size;
vsk->buffer_min_size = psk->buffer_min_size;
vsk->buffer_max_size = psk->buffer_max_size;
+ security_sk_clone(parent, sk);
} else {
vsk->trusted = ns_capable_noaudit(&init_user_ns, CAP_NET_ADMIN);
vsk->owner = get_current_cred();
* Copyright 2006-2010 Johannes Berg <johannes@sipsolutions.net>
* Copyright 2013-2014 Intel Mobile Communications GmbH
* Copyright 2015-2017 Intel Deutschland GmbH
- * Copyright (C) 2018-2020 Intel Corporation
+ * Copyright (C) 2018-2021 Intel Corporation
*/
#include <linux/if.h>
struct wireless_dev *result = NULL;
bool have_ifidx = attrs[NL80211_ATTR_IFINDEX];
bool have_wdev_id = attrs[NL80211_ATTR_WDEV];
- u64 wdev_id;
+ u64 wdev_id = 0;
int wiphy_idx = -1;
int ifidx = -1;
unsigned int len = nla_len(attr);
const struct element *elem;
const struct ieee80211_mgmt *mgmt = (void *)data;
- bool s1g_bcn = ieee80211_is_s1g_beacon(mgmt->frame_control);
unsigned int fixedlen, hdrlen;
+ bool s1g_bcn;
+ if (len < offsetofend(typeof(*mgmt), frame_control))
+ goto err;
+
+ s1g_bcn = ieee80211_is_s1g_beacon(mgmt->frame_control);
if (s1g_bcn) {
fixedlen = offsetof(struct ieee80211_ext,
u.s1g_beacon.variable);
rdev, info->attrs[NL80211_ATTR_UNSOL_BCAST_PROBE_RESP],
¶ms);
if (err)
- return err;
+ goto out;
}
nl80211_calculate_ap_params(¶ms);
#define NL80211_FLAG_NEED_WDEV_UP (NL80211_FLAG_NEED_WDEV |\
NL80211_FLAG_CHECK_NETDEV_UP)
#define NL80211_FLAG_CLEAR_SKB 0x20
+#define NL80211_FLAG_NO_WIPHY_MTX 0x40
static int nl80211_pre_doit(const struct genl_ops *ops, struct sk_buff *skb,
struct genl_info *info)
info->user_ptr[0] = rdev;
}
- if (rdev) {
+ if (rdev && !(ops->internal_flags & NL80211_FLAG_NO_WIPHY_MTX)) {
wiphy_lock(&rdev->wiphy);
/* we keep the mutex locked until post_doit */
__release(&rdev->wiphy.mtx);
}
}
- if (info->user_ptr[0]) {
+ if (info->user_ptr[0] &&
+ !(ops->internal_flags & NL80211_FLAG_NO_WIPHY_MTX)) {
struct cfg80211_registered_device *rdev = info->user_ptr[0];
/* we kept the mutex locked since pre_doit */
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
.doit = nl80211_wiphy_netns,
.flags = GENL_UNS_ADMIN_PERM,
- .internal_flags = NL80211_FLAG_NEED_WIPHY,
+ .internal_flags = NL80211_FLAG_NEED_WIPHY |
+ NL80211_FLAG_NEED_RTNL |
+ NL80211_FLAG_NO_WIPHY_MTX,
},
{
.cmd = NL80211_CMD_GET_SURVEY,
return NULL;
if (ext) {
- struct ieee80211_s1g_bcn_compat_ie *compat;
- u8 *ie;
+ const struct ieee80211_s1g_bcn_compat_ie *compat;
+ const struct element *elem;
- ie = (void *)cfg80211_find_ie(WLAN_EID_S1G_BCN_COMPAT,
- variable, ielen);
- if (!ie)
+ elem = cfg80211_find_elem(WLAN_EID_S1G_BCN_COMPAT,
+ variable, ielen);
+ if (!elem)
+ return NULL;
+ if (elem->datalen < sizeof(*compat))
return NULL;
- compat = (void *)(ie + 2);
+ compat = (void *)elem->data;
bssid = ext->u.s1g_beacon.sa;
capability = le16_to_cpu(compat->compat_info);
beacon_int = le16_to_cpu(compat->beacon_int);
cfg80211_sme_free(wdev);
}
- if (WARN_ON(wdev->conn))
+ if (wdev->conn)
return -EINPROGRESS;
wdev->conn = kzalloc(sizeof(*wdev->conn), GFP_KERNEL);
case XFRM_MSG_GETSADINFO:
case XFRM_MSG_GETSPDINFO:
default:
- WARN_ONCE(1, "unsupported nlmsg_type %d", nlh_src->nlmsg_type);
+ pr_warn_once("unsupported nlmsg_type %d\n", nlh_src->nlmsg_type);
return ERR_PTR(-EOPNOTSUPP);
}
return xfrm_nla_cpy(dst, src, nla_len(src));
default:
BUILD_BUG_ON(XFRMA_MAX != XFRMA_IF_ID);
- WARN_ONCE(1, "unsupported nla_type %d", src->nla_type);
+ pr_warn_once("unsupported nla_type %d\n", src->nla_type);
return -EOPNOTSUPP;
}
}
struct sk_buff *new = NULL;
int err;
- if (WARN_ON_ONCE(type >= ARRAY_SIZE(xfrm_msg_min)))
+ if (type >= ARRAY_SIZE(xfrm_msg_min)) {
+ pr_warn_once("unsupported nlmsg_type %d\n", nlh_src->nlmsg_type);
return -EOPNOTSUPP;
+ }
if (skb_shinfo(skb)->frag_list == NULL) {
new = alloc_skb(skb->len + skb_tailroom(skb), GFP_ATOMIC);
struct nlmsghdr *nlmsg = dst;
struct nlattr *nla;
+ /* xfrm_user_rcv_msg_compat() relies on fact that 32-bit messages
+ * have the same len or shorted than 64-bit ones.
+ * 32-bit translation that is bigger than 64-bit original is unexpected.
+ */
if (WARN_ON_ONCE(copy_len > payload))
copy_len = payload;
return skb;
}
- xo->flags |= XFRM_XMIT;
-
if (skb_is_gso(skb) && unlikely(x->xso.dev != dev)) {
struct sk_buff *segs;
icmpv6_ndo_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu);
} else {
+ if (!(ip_hdr(skb)->frag_off & htons(IP_DF)))
+ goto xmit;
icmp_ndo_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED,
htonl(mtu));
}
return -EMSGSIZE;
}
+xmit:
xfrmi_scrub_packet(skb, !net_eq(xi->net, dev_net(dev)));
skb_dst_set(skb, dst);
skb->dev = tdev;
return err;
}
-int xfrm_output_resume(struct sk_buff *skb, int err)
+int xfrm_output_resume(struct sock *sk, struct sk_buff *skb, int err)
{
struct net *net = xs_net(skb_dst(skb)->xfrm);
while (likely((err = xfrm_output_one(skb, err)) == 0)) {
nf_reset_ct(skb);
- err = skb_dst(skb)->ops->local_out(net, skb->sk, skb);
+ err = skb_dst(skb)->ops->local_out(net, sk, skb);
if (unlikely(err != 1))
goto out;
if (!skb_dst(skb)->xfrm)
- return dst_output(net, skb->sk, skb);
+ return dst_output(net, sk, skb);
err = nf_hook(skb_dst(skb)->ops->family,
- NF_INET_POST_ROUTING, net, skb->sk, skb,
+ NF_INET_POST_ROUTING, net, sk, skb,
NULL, skb_dst(skb)->dev, xfrm_output2);
if (unlikely(err != 1))
goto out;
static int xfrm_output2(struct net *net, struct sock *sk, struct sk_buff *skb)
{
- return xfrm_output_resume(skb, 1);
+ return xfrm_output_resume(sk, skb, 1);
}
static int xfrm_output_gso(struct net *net, struct sock *sk, struct sk_buff *skb)
{
int err;
+ if (x->outer_mode.encap == XFRM_MODE_BEET &&
+ ip_is_fragment(ip_hdr(skb))) {
+ net_warn_ratelimited("BEET mode doesn't support inner IPv4 fragments\n");
+ return -EAFNOSUPPORT;
+ }
+
err = xfrm4_tunnel_check_size(skb);
if (err)
return err;
static int xfrm6_extract_output(struct xfrm_state *x, struct sk_buff *skb)
{
#if IS_ENABLED(CONFIG_IPV6)
+ unsigned int ptr = 0;
int err;
+ if (x->outer_mode.encap == XFRM_MODE_BEET &&
+ ipv6_find_hdr(skb, &ptr, NEXTHDR_FRAGMENT, NULL, NULL) >= 0) {
+ net_warn_ratelimited("BEET mode doesn't support inner IPv6 fragments\n");
+ return -EAFNOSUPPORT;
+ }
+
err = xfrm6_tunnel_check_size(skb);
if (err)
return err;
*/
static unsigned int xfrm_state_hashmax __read_mostly = 1 * 1024 * 1024;
-static __read_mostly seqcount_t xfrm_state_hash_generation = SEQCNT_ZERO(xfrm_state_hash_generation);
static struct kmem_cache *xfrm_state_cache __ro_after_init;
static DECLARE_WORK(xfrm_state_gc_work, xfrm_state_gc_task);
}
spin_lock_bh(&net->xfrm.xfrm_state_lock);
- write_seqcount_begin(&xfrm_state_hash_generation);
+ write_seqcount_begin(&net->xfrm.xfrm_state_hash_generation);
nhashmask = (nsize / sizeof(struct hlist_head)) - 1U;
odst = xfrm_state_deref_prot(net->xfrm.state_bydst, net);
rcu_assign_pointer(net->xfrm.state_byspi, nspi);
net->xfrm.state_hmask = nhashmask;
- write_seqcount_end(&xfrm_state_hash_generation);
+ write_seqcount_end(&net->xfrm.xfrm_state_hash_generation);
spin_unlock_bh(&net->xfrm.xfrm_state_lock);
osize = (ohashmask + 1) * sizeof(struct hlist_head);
to_put = NULL;
- sequence = read_seqcount_begin(&xfrm_state_hash_generation);
+ sequence = read_seqcount_begin(&net->xfrm.xfrm_state_hash_generation);
rcu_read_lock();
h = xfrm_dst_hash(net, daddr, saddr, tmpl->reqid, encap_family);
if (to_put)
xfrm_state_put(to_put);
- if (read_seqcount_retry(&xfrm_state_hash_generation, sequence)) {
+ if (read_seqcount_retry(&net->xfrm.xfrm_state_hash_generation, sequence)) {
*err = -EAGAIN;
if (x) {
xfrm_state_put(x);
net->xfrm.state_num = 0;
INIT_WORK(&net->xfrm.state_hash_work, xfrm_hash_resize);
spin_lock_init(&net->xfrm.xfrm_state_lock);
+ seqcount_spinlock_init(&net->xfrm.xfrm_state_hash_generation,
+ &net->xfrm.xfrm_state_lock);
return 0;
out_byspi:
__patchable_function_entries : { *(__patchable_function_entries) }
+#ifdef CONFIG_LTO_CLANG
/*
* With CONFIG_LTO_CLANG, LLD always enables -fdata-sections and
* -ffunction-sections, which increases the size of the final module.
}
.text : { *(.text .text.[0-9a-zA-Z_]*) }
+#endif
}
/* bring in arch-specific sections */
struct rb_node *node, *parent = NULL;
struct integrity_iint_cache *iint, *test_iint;
+ /*
+ * The integrity's "iint_cache" is initialized at security_init(),
+ * unless it is not included in the ordered list of LSMs enabled
+ * on the boot command line.
+ */
+ if (!iint_cache)
+ panic("%s: lsm=integrity required.\n", __func__);
+
iint = integrity_iint_find(inode);
if (iint)
return iint;
return READ_ONCE(state->policycap[POLICYDB_CAPABILITY_GENFS_SECLABEL_SYMLINKS]);
}
+struct selinux_policy_convert_data;
+
+struct selinux_load_state {
+ struct selinux_policy *policy;
+ struct selinux_policy_convert_data *convert_data;
+};
+
int security_mls_enabled(struct selinux_state *state);
int security_load_policy(struct selinux_state *state,
- void *data, size_t len,
- struct selinux_policy **newpolicyp);
+ void *data, size_t len,
+ struct selinux_load_state *load_state);
void selinux_policy_commit(struct selinux_state *state,
- struct selinux_policy *newpolicy);
+ struct selinux_load_state *load_state);
void selinux_policy_cancel(struct selinux_state *state,
- struct selinux_policy *policy);
+ struct selinux_load_state *load_state);
int security_read_policy(struct selinux_state *state,
void **data, size_t *len);
int security_read_state_kernel(struct selinux_state *state,
ret = sel_make_bools(newpolicy, tmp_bool_dir, &tmp_bool_num,
&tmp_bool_names, &tmp_bool_values);
- if (ret) {
- pr_err("SELinux: failed to load policy booleans\n");
+ if (ret)
goto out;
- }
ret = sel_make_classes(newpolicy, tmp_class_dir,
&fsi->last_class_ino);
- if (ret) {
- pr_err("SELinux: failed to load policy classes\n");
+ if (ret)
goto out;
- }
/* booleans */
old_dentry = fsi->bool_dir;
{
struct selinux_fs_info *fsi = file_inode(file)->i_sb->s_fs_info;
- struct selinux_policy *newpolicy;
+ struct selinux_load_state load_state;
ssize_t length;
void *data = NULL;
if (copy_from_user(data, buf, count) != 0)
goto out;
- length = security_load_policy(fsi->state, data, count, &newpolicy);
+ length = security_load_policy(fsi->state, data, count, &load_state);
if (length) {
pr_warn_ratelimited("SELinux: failed to load policy\n");
goto out;
}
- length = sel_make_policy_nodes(fsi, newpolicy);
+ length = sel_make_policy_nodes(fsi, load_state.policy);
if (length) {
- selinux_policy_cancel(fsi->state, newpolicy);
- goto out1;
+ pr_warn_ratelimited("SELinux: failed to initialize selinuxfs\n");
+ selinux_policy_cancel(fsi->state, &load_state);
+ goto out;
}
- selinux_policy_commit(fsi->state, newpolicy);
+ selinux_policy_commit(fsi->state, &load_state);
length = count;
-out1:
audit_log(audit_context(), GFP_KERNEL, AUDIT_MAC_POLICY_LOAD,
"auid=%u ses=%u lsm=selinux res=1",
from_kuid(&init_user_ns, audit_get_loginuid(current)),
struct avtab_node *prev, *cur, *newnode;
u16 specified = key->specified & ~(AVTAB_ENABLED|AVTAB_ENABLED_OLD);
- if (!h)
+ if (!h || !h->nslot)
return -EINVAL;
hvalue = avtab_hash(key, h->mask);
struct avtab_node *prev, *cur;
u16 specified = key->specified & ~(AVTAB_ENABLED|AVTAB_ENABLED_OLD);
- if (!h)
+ if (!h || !h->nslot)
return NULL;
hvalue = avtab_hash(key, h->mask);
for (prev = NULL, cur = h->htable[hvalue];
struct avtab_node *cur;
u16 specified = key->specified & ~(AVTAB_ENABLED|AVTAB_ENABLED_OLD);
- if (!h)
+ if (!h || !h->nslot)
return NULL;
hvalue = avtab_hash(key, h->mask);
struct avtab_node *cur;
u16 specified = key->specified & ~(AVTAB_ENABLED|AVTAB_ENABLED_OLD);
- if (!h)
+ if (!h || !h->nslot)
return NULL;
hvalue = avtab_hash(key, h->mask);
}
kvfree(h->htable);
h->htable = NULL;
+ h->nel = 0;
h->nslot = 0;
h->mask = 0;
}
{
h->htable = NULL;
h->nel = 0;
+ h->nslot = 0;
+ h->mask = 0;
}
-int avtab_alloc(struct avtab *h, u32 nrules)
+static int avtab_alloc_common(struct avtab *h, u32 nslot)
{
- u32 mask = 0;
- u32 shift = 0;
- u32 work = nrules;
- u32 nslot = 0;
-
- if (nrules == 0)
- goto avtab_alloc_out;
-
- while (work) {
- work = work >> 1;
- shift++;
- }
- if (shift > 2)
- shift = shift - 2;
- nslot = 1 << shift;
- if (nslot > MAX_AVTAB_HASH_BUCKETS)
- nslot = MAX_AVTAB_HASH_BUCKETS;
- mask = nslot - 1;
+ if (!nslot)
+ return 0;
h->htable = kvcalloc(nslot, sizeof(void *), GFP_KERNEL);
if (!h->htable)
return -ENOMEM;
- avtab_alloc_out:
- h->nel = 0;
h->nslot = nslot;
- h->mask = mask;
- pr_debug("SELinux: %d avtab hash slots, %d rules.\n",
- h->nslot, nrules);
+ h->mask = nslot - 1;
return 0;
}
-int avtab_duplicate(struct avtab *new, struct avtab *orig)
+int avtab_alloc(struct avtab *h, u32 nrules)
{
- int i;
- struct avtab_node *node, *tmp, *tail;
-
- memset(new, 0, sizeof(*new));
+ int rc;
+ u32 nslot = 0;
- new->htable = kvcalloc(orig->nslot, sizeof(void *), GFP_KERNEL);
- if (!new->htable)
- return -ENOMEM;
- new->nslot = orig->nslot;
- new->mask = orig->mask;
-
- for (i = 0; i < orig->nslot; i++) {
- tail = NULL;
- for (node = orig->htable[i]; node; node = node->next) {
- tmp = kmem_cache_zalloc(avtab_node_cachep, GFP_KERNEL);
- if (!tmp)
- goto error;
- tmp->key = node->key;
- if (tmp->key.specified & AVTAB_XPERMS) {
- tmp->datum.u.xperms =
- kmem_cache_zalloc(avtab_xperms_cachep,
- GFP_KERNEL);
- if (!tmp->datum.u.xperms) {
- kmem_cache_free(avtab_node_cachep, tmp);
- goto error;
- }
- tmp->datum.u.xperms = node->datum.u.xperms;
- } else
- tmp->datum.u.data = node->datum.u.data;
-
- if (tail)
- tail->next = tmp;
- else
- new->htable[i] = tmp;
-
- tail = tmp;
- new->nel++;
+ if (nrules != 0) {
+ u32 shift = 1;
+ u32 work = nrules >> 3;
+ while (work) {
+ work >>= 1;
+ shift++;
}
+ nslot = 1 << shift;
+ if (nslot > MAX_AVTAB_HASH_BUCKETS)
+ nslot = MAX_AVTAB_HASH_BUCKETS;
+
+ rc = avtab_alloc_common(h, nslot);
+ if (rc)
+ return rc;
}
+ pr_debug("SELinux: %d avtab hash slots, %d rules.\n", nslot, nrules);
return 0;
-error:
- avtab_destroy(new);
- return -ENOMEM;
+}
+
+int avtab_alloc_dup(struct avtab *new, const struct avtab *orig)
+{
+ return avtab_alloc_common(new, orig->nslot);
}
void avtab_hash_eval(struct avtab *h, char *tag)
void avtab_init(struct avtab *h);
int avtab_alloc(struct avtab *, u32);
-int avtab_duplicate(struct avtab *new, struct avtab *orig);
+int avtab_alloc_dup(struct avtab *new, const struct avtab *orig);
struct avtab_datum *avtab_search(struct avtab *h, struct avtab_key *k);
void avtab_destroy(struct avtab *h);
void avtab_hash_eval(struct avtab *h, char *tag);
struct cond_av_list *orig,
struct avtab *avtab)
{
- struct avtab_node *avnode;
u32 i;
memset(new, 0, sizeof(*new));
return -ENOMEM;
for (i = 0; i < orig->len; i++) {
- avnode = avtab_search_node(avtab, &orig->nodes[i]->key);
- if (WARN_ON(!avnode))
- return -EINVAL;
- new->nodes[i] = avnode;
+ new->nodes[i] = avtab_insert_nonunique(avtab,
+ &orig->nodes[i]->key,
+ &orig->nodes[i]->datum);
+ if (!new->nodes[i])
+ return -ENOMEM;
new->len++;
}
{
int rc, i, j;
- rc = avtab_duplicate(&newp->te_cond_avtab, &origp->te_cond_avtab);
+ rc = avtab_alloc_dup(&newp->te_cond_avtab, &origp->te_cond_avtab);
if (rc)
return rc;
#include "policycap_names.h"
#include "ima.h"
+struct convert_context_args {
+ struct selinux_state *state;
+ struct policydb *oldp;
+ struct policydb *newp;
+};
+
+struct selinux_policy_convert_data {
+ struct convert_context_args args;
+ struct sidtab_convert_params sidtab_params;
+};
+
/* Forward declaration. */
static int context_struct_to_string(struct policydb *policydb,
struct context *context,
if (!str)
goto out;
}
+retry:
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
} else if (rc)
goto out_unlock;
rc = sidtab_context_to_sid(sidtab, &context, sid);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ if (context.str) {
+ str = context.str;
+ context.str = NULL;
+ }
+ context_destroy(&context);
+ goto retry;
+ }
context_destroy(&context);
out_unlock:
rcu_read_unlock();
struct selinux_policy *policy;
struct policydb *policydb;
struct sidtab *sidtab;
- struct class_datum *cladatum = NULL;
+ struct class_datum *cladatum;
struct context *scontext, *tcontext, newcontext;
struct sidtab_entry *sentry, *tentry;
struct avtab_key avkey;
goto out;
}
+retry:
+ cladatum = NULL;
context_init(&newcontext);
rcu_read_lock();
}
/* Obtain the sid for the context. */
rc = sidtab_context_to_sid(sidtab, &newcontext, out_sid);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ context_destroy(&newcontext);
+ goto retry;
+ }
out_unlock:
rcu_read_unlock();
context_destroy(&newcontext);
return 0;
}
-struct convert_context_args {
- struct selinux_state *state;
- struct policydb *oldp;
- struct policydb *newp;
-};
-
/*
* Convert the values in the security context
* structure `oldc' from the values specified
}
void selinux_policy_cancel(struct selinux_state *state,
- struct selinux_policy *policy)
+ struct selinux_load_state *load_state)
{
struct selinux_policy *oldpolicy;
lockdep_is_held(&state->policy_mutex));
sidtab_cancel_convert(oldpolicy->sidtab);
- selinux_policy_free(policy);
+ selinux_policy_free(load_state->policy);
+ kfree(load_state->convert_data);
}
static void selinux_notify_policy_change(struct selinux_state *state,
}
void selinux_policy_commit(struct selinux_state *state,
- struct selinux_policy *newpolicy)
+ struct selinux_load_state *load_state)
{
- struct selinux_policy *oldpolicy;
+ struct selinux_policy *oldpolicy, *newpolicy = load_state->policy;
+ unsigned long flags;
u32 seqno;
oldpolicy = rcu_dereference_protected(state->policy,
seqno = newpolicy->latest_granting;
/* Install the new policy. */
- rcu_assign_pointer(state->policy, newpolicy);
+ if (oldpolicy) {
+ sidtab_freeze_begin(oldpolicy->sidtab, &flags);
+ rcu_assign_pointer(state->policy, newpolicy);
+ sidtab_freeze_end(oldpolicy->sidtab, &flags);
+ } else {
+ rcu_assign_pointer(state->policy, newpolicy);
+ }
/* Load the policycaps from the new policy */
security_load_policycaps(state, newpolicy);
/* Free the old policy */
synchronize_rcu();
selinux_policy_free(oldpolicy);
+ kfree(load_state->convert_data);
/* Notify others of the policy change */
selinux_notify_policy_change(state, seqno);
* loading the new policy.
*/
int security_load_policy(struct selinux_state *state, void *data, size_t len,
- struct selinux_policy **newpolicyp)
+ struct selinux_load_state *load_state)
{
struct selinux_policy *newpolicy, *oldpolicy;
- struct sidtab_convert_params convert_params;
- struct convert_context_args args;
+ struct selinux_policy_convert_data *convert_data;
int rc = 0;
struct policy_file file = { data, len }, *fp = &file;
goto err_mapping;
}
-
if (!selinux_initialized(state)) {
/* First policy load, so no need to preserve state from old policy */
- *newpolicyp = newpolicy;
+ load_state->policy = newpolicy;
+ load_state->convert_data = NULL;
return 0;
}
goto err_free_isids;
}
+ convert_data = kmalloc(sizeof(*convert_data), GFP_KERNEL);
+ if (!convert_data) {
+ rc = -ENOMEM;
+ goto err_free_isids;
+ }
+
/*
* Convert the internal representations of contexts
* in the new SID table.
*/
- args.state = state;
- args.oldp = &oldpolicy->policydb;
- args.newp = &newpolicy->policydb;
+ convert_data->args.state = state;
+ convert_data->args.oldp = &oldpolicy->policydb;
+ convert_data->args.newp = &newpolicy->policydb;
- convert_params.func = convert_context;
- convert_params.args = &args;
- convert_params.target = newpolicy->sidtab;
+ convert_data->sidtab_params.func = convert_context;
+ convert_data->sidtab_params.args = &convert_data->args;
+ convert_data->sidtab_params.target = newpolicy->sidtab;
- rc = sidtab_convert(oldpolicy->sidtab, &convert_params);
+ rc = sidtab_convert(oldpolicy->sidtab, &convert_data->sidtab_params);
if (rc) {
pr_err("SELinux: unable to convert the internal"
" representation of contexts in the new SID"
" table\n");
- goto err_free_isids;
+ goto err_free_convert_data;
}
- *newpolicyp = newpolicy;
+ load_state->policy = newpolicy;
+ load_state->convert_data = convert_data;
return 0;
+err_free_convert_data:
+ kfree(convert_data);
err_free_isids:
sidtab_destroy(newpolicy->sidtab);
err_mapping:
struct policydb *policydb;
struct sidtab *sidtab;
struct ocontext *c;
- int rc = 0;
+ int rc;
if (!selinux_initialized(state)) {
*out_sid = SECINITSID_PORT;
return 0;
}
+retry:
+ rc = 0;
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
if (!c->sid[0]) {
rc = sidtab_context_to_sid(sidtab, &c->context[0],
&c->sid[0]);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out;
}
struct policydb *policydb;
struct sidtab *sidtab;
struct ocontext *c;
- int rc = 0;
+ int rc;
if (!selinux_initialized(state)) {
*out_sid = SECINITSID_UNLABELED;
return 0;
}
+retry:
+ rc = 0;
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
rc = sidtab_context_to_sid(sidtab,
&c->context[0],
&c->sid[0]);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out;
}
struct policydb *policydb;
struct sidtab *sidtab;
struct ocontext *c;
- int rc = 0;
+ int rc;
if (!selinux_initialized(state)) {
*out_sid = SECINITSID_UNLABELED;
return 0;
}
+retry:
+ rc = 0;
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
if (!c->sid[0]) {
rc = sidtab_context_to_sid(sidtab, &c->context[0],
&c->sid[0]);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out;
}
struct selinux_policy *policy;
struct policydb *policydb;
struct sidtab *sidtab;
- int rc = 0;
+ int rc;
struct ocontext *c;
if (!selinux_initialized(state)) {
return 0;
}
+retry:
+ rc = 0;
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
if (!c->sid[0] || !c->sid[1]) {
rc = sidtab_context_to_sid(sidtab, &c->context[0],
&c->sid[0]);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out;
rc = sidtab_context_to_sid(sidtab, &c->context[1],
&c->sid[1]);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out;
}
return 0;
}
+retry:
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
rc = sidtab_context_to_sid(sidtab,
&c->context[0],
&c->sid[0]);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out;
}
struct sidtab *sidtab;
struct context *fromcon, usercon;
u32 *mysids = NULL, *mysids2, sid;
- u32 mynel = 0, maxnel = SIDS_NEL;
+ u32 i, j, mynel, maxnel = SIDS_NEL;
struct user_datum *user;
struct role_datum *role;
struct ebitmap_node *rnode, *tnode;
- int rc = 0, i, j;
+ int rc;
*sids = NULL;
*nel = 0;
if (!selinux_initialized(state))
- goto out;
+ return 0;
+
+ mysids = kcalloc(maxnel, sizeof(*mysids), GFP_KERNEL);
+ if (!mysids)
+ return -ENOMEM;
+retry:
+ mynel = 0;
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
usercon.user = user->value;
- rc = -ENOMEM;
- mysids = kcalloc(maxnel, sizeof(*mysids), GFP_ATOMIC);
- if (!mysids)
- goto out_unlock;
-
ebitmap_for_each_positive_bit(&user->roles, rnode, i) {
role = policydb->role_val_to_struct[i];
usercon.role = i + 1;
continue;
rc = sidtab_context_to_sid(sidtab, &usercon, &sid);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out_unlock;
if (mynel < maxnel) {
rcu_read_unlock();
if (rc || !mynel) {
kfree(mysids);
- goto out;
+ return rc;
}
rc = -ENOMEM;
mysids2 = kcalloc(mynel, sizeof(*mysids2), GFP_KERNEL);
if (!mysids2) {
kfree(mysids);
- goto out;
+ return rc;
}
for (i = 0, j = 0; i < mynel; i++) {
struct av_decision dummy_avd;
mysids2[j++] = mysids[i];
cond_resched();
}
- rc = 0;
kfree(mysids);
*sids = mysids2;
*nel = j;
-out:
- return rc;
+ return 0;
}
/**
* Obtain a SID to use for a file in a filesystem that
* cannot support xattr or use a fixed labeling behavior like
* transition SIDs or task SIDs.
+ *
+ * WARNING: This function may return -ESTALE, indicating that the caller
+ * must retry the operation after re-acquiring the policy pointer!
*/
static inline int __security_genfs_sid(struct selinux_policy *policy,
const char *fstype,
return 0;
}
- rcu_read_lock();
- policy = rcu_dereference(state->policy);
- retval = __security_genfs_sid(policy,
- fstype, path, orig_sclass, sid);
- rcu_read_unlock();
+ do {
+ rcu_read_lock();
+ policy = rcu_dereference(state->policy);
+ retval = __security_genfs_sid(policy, fstype, path,
+ orig_sclass, sid);
+ rcu_read_unlock();
+ } while (retval == -ESTALE);
return retval;
}
struct selinux_policy *policy;
struct policydb *policydb;
struct sidtab *sidtab;
- int rc = 0;
+ int rc;
struct ocontext *c;
struct superblock_security_struct *sbsec = sb->s_security;
const char *fstype = sb->s_type->name;
return 0;
}
+retry:
+ rc = 0;
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
if (!c->sid[0]) {
rc = sidtab_context_to_sid(sidtab, &c->context[0],
&c->sid[0]);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
goto out;
}
} else {
rc = __security_genfs_sid(policy, fstype, "/",
SECCLASS_DIR, &sbsec->sid);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc) {
sbsec->behavior = SECURITY_FS_USE_NONE;
rc = 0;
u32 len;
int rc;
- rc = 0;
if (!selinux_initialized(state)) {
*new_sid = sid;
- goto out;
+ return 0;
}
+retry:
+ rc = 0;
context_init(&newcon);
rcu_read_lock();
}
}
rc = sidtab_context_to_sid(sidtab, &newcon, new_sid);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ context_destroy(&newcon);
+ goto retry;
+ }
out_unlock:
rcu_read_unlock();
context_destroy(&newcon);
-out:
return rc;
}
return 0;
}
+retry:
+ rc = 0;
rcu_read_lock();
policy = rcu_dereference(state->policy);
policydb = &policy->policydb;
goto out;
}
rc = -EIDRM;
- if (!mls_context_isvalid(policydb, &ctx_new))
- goto out_free;
+ if (!mls_context_isvalid(policydb, &ctx_new)) {
+ ebitmap_destroy(&ctx_new.range.level[0].cat);
+ goto out;
+ }
rc = sidtab_context_to_sid(sidtab, &ctx_new, sid);
+ ebitmap_destroy(&ctx_new.range.level[0].cat);
+ if (rc == -ESTALE) {
+ rcu_read_unlock();
+ goto retry;
+ }
if (rc)
- goto out_free;
+ goto out;
security_netlbl_cache_add(secattr, *sid);
-
- ebitmap_destroy(&ctx_new.range.level[0].cat);
} else
*sid = SECSID_NULL;
- rcu_read_unlock();
- return 0;
-out_free:
- ebitmap_destroy(&ctx_new.range.level[0].cat);
out:
rcu_read_unlock();
return rc;
for (i = 0; i < SECINITSID_NUM; i++)
s->isids[i].set = 0;
+ s->frozen = false;
s->count = 0;
s->convert = NULL;
hash_init(s->context_to_sid);
if (*sid)
goto out_unlock;
+ if (unlikely(s->frozen)) {
+ /*
+ * This sidtab is now frozen - tell the caller to abort and
+ * get the new one.
+ */
+ rc = -ESTALE;
+ goto out_unlock;
+ }
+
count = s->count;
convert = s->convert;
spin_unlock_irqrestore(&s->lock, flags);
}
+void sidtab_freeze_begin(struct sidtab *s, unsigned long *flags) __acquires(&s->lock)
+{
+ spin_lock_irqsave(&s->lock, *flags);
+ s->frozen = true;
+ s->convert = NULL;
+}
+void sidtab_freeze_end(struct sidtab *s, unsigned long *flags) __releases(&s->lock)
+{
+ spin_unlock_irqrestore(&s->lock, *flags);
+}
+
static void sidtab_destroy_entry(struct sidtab_entry *entry)
{
context_destroy(&entry->context);
u32 count;
/* access only under spinlock */
struct sidtab_convert_params *convert;
+ bool frozen;
spinlock_t lock;
#if CONFIG_SECURITY_SELINUX_SID2STR_CACHE_SIZE > 0
void sidtab_cancel_convert(struct sidtab *s);
+void sidtab_freeze_begin(struct sidtab *s, unsigned long *flags) __acquires(&s->lock);
+void sidtab_freeze_end(struct sidtab *s, unsigned long *flags) __releases(&s->lock);
+
int sidtab_context_to_sid(struct sidtab *s, struct context *context, u32 *sid);
void sidtab_destroy(struct sidtab *s);
static bool tomoyo_kernel_service(void)
{
/* Nothing to do if I am a kernel service. */
- return (current->flags & (PF_KTHREAD | PF_IO_WORKER)) == PF_KTHREAD;
+ return current->flags & PF_KTHREAD;
}
/**
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("A loopback soundcard");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ALSA,Loopback soundcard}}");
#define MAX_PCM_SUBSTREAMS 8
return -ENOMEM;
kctl->id.device = dev;
kctl->id.subdevice = substr;
+
+ /* Add the control before copying the id so that
+ * the numid field of the id is set in the copy.
+ */
+ err = snd_ctl_add(card, kctl);
+ if (err < 0)
+ return err;
+
switch (idx) {
case ACTIVE_IDX:
setup->active_id = kctl->id;
default:
break;
}
- err = snd_ctl_add(card, kctl);
- if (err < 0)
- return err;
}
}
}
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Dummy soundcard (/dev/null)");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ALSA,Dummy soundcard}}");
#define MAX_PCM_DEVICES 4
#define MAX_PCM_SUBSTREAMS 128
MODULE_AUTHOR("Michael T. Mayers");
MODULE_DESCRIPTION("MOTU MidiTimePiece AV multiport MIDI");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{MOTU,MidiTimePiece AV multiport MIDI}}");
// io resources
#define MTPAV_IOBASE 0x378
MODULE_AUTHOR("Matthias Koenig <mk@phasorlab.de>");
MODULE_DESCRIPTION("ESI Miditerminal 4140");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ESI,Miditerminal 4140}}");
/*********************************************************************
* Chip specific
MODULE_AUTHOR("Stas Sergeev <stsp@users.sourceforge.net>");
MODULE_DESCRIPTION("PC-Speaker driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{PC-Speaker, pcsp}}");
MODULE_ALIAS("platform:pcspkr");
static int index = SNDRV_DEFAULT_IDX1; /* Index 0-MAX */
MODULE_AUTHOR("Levent Guendogdu, Tobias Gehrig, Matthias Koenig");
MODULE_DESCRIPTION("Midiman Portman2x4");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Midiman,Portman2x4}}");
/*********************************************************************
* Chip specific
MODULE_DESCRIPTION("MIDI serial u16550");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ALSA, MIDI serial u16550}}");
#define SNDRV_SERIAL_SOUNDCANVAS 0 /* Roland Soundcanvas; F5 NN selects part */
#define SNDRV_SERIAL_MS124T 1 /* Midiator MS-124T */
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("Dummy soundcard for virtual rawmidi devices");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ALSA,Virtual rawmidi device}}");
#define MAX_MIDI_DEVICES 4
struct reg_params tx_params, rx_params;
if (dice->substreams_counter == 0) {
- if (get_register_params(dice, &tx_params, &rx_params) >= 0) {
- amdtp_domain_stop(&dice->domain);
+ if (get_register_params(dice, &tx_params, &rx_params) >= 0)
finish_session(dice, &tx_params, &rx_params);
- }
+ amdtp_domain_stop(&dice->domain);
release_resources(dice);
}
}
MODULE_AUTHOR("Massimo Piccioni <dafastidio@libero.it>");
MODULE_DESCRIPTION("AD1816A, AD1815");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Highscreen,Sound-Boostar 16 3D},"
- "{Analog Devices,AD1815},"
- "{Analog Devices,AD1816A},"
- "{TerraTec,Base 64},"
- "{TerraTec,AudioSystem EWS64S},"
- "{Aztech/Newcom SC-16 3D},"
- "{Shark Predator ISA}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 1-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_DESCRIPTION(CRD_NAME);
MODULE_AUTHOR("Tugrul Galatali <galatalt@stuy.edu>, Jaroslav Kysela <perex@perex.cz>");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Analog Devices,AD1848},"
- "{Analog Devices,AD1847},"
- "{Crystal Semiconductors,CS4248}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
#define PFX "als100: "
MODULE_DESCRIPTION("Avance Logic ALS007/ALS1X0");
-MODULE_SUPPORTED_DEVICE("{{Diamond Technologies DT-019X},"
- "{Avance Logic ALS-007}}"
- "{{Avance Logic,ALS100 - PRO16PNP},"
- "{Avance Logic,ALS110},"
- "{Avance Logic,ALS120},"
- "{Avance Logic,ALS200},"
- "{3D Melody,MF1000},"
- "{Digimate,3D Sound},"
- "{Avance Logic,ALS120},"
- "{RTL,RTL3000}}");
-
MODULE_AUTHOR("Massimo Piccioni <dafastidio@libero.it>");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Massimo Piccioni <dafastidio@libero.it>");
MODULE_DESCRIPTION("Aztech Systems AZT2320");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Aztech Systems,PRO16V},"
- "{Aztech Systems,AZT2320},"
- "{Aztech Systems,AZT3300},"
- "{Aztech Systems,AZT2320},"
- "{Aztech Systems,AZT3000}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("George Talusan <gstalusan@uwaterloo.ca>");
MODULE_DESCRIPTION("C-Media CMI8330/CMI8329");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{C-Media,CMI8330,isapnp:{CMI0001,@@@0001,@X@0001}}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
MODULE_DESCRIPTION(CRD_NAME);
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Crystal Semiconductors,CS4231}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Cirrus Logic CS4232-9");
-MODULE_SUPPORTED_DEVICE("{{Turtle Beach,TBS-2000},"
- "{Turtle Beach,Tropez Plus},"
- "{SIC CrystalWave 32},"
- "{Hewlett Packard,Omnibook 5500},"
- "{TerraTec,Maestro 32/96},"
- "{Philips,PCA70PS}},"
- "{{Crystal Semiconductors,CS4235},"
- "{Crystal Semiconductors,CS4236},"
- "{Crystal Semiconductors,CS4237},"
- "{Crystal Semiconductors,CS4238},"
- "{Crystal Semiconductors,CS4239},"
- "{Acer,AW37},"
- "{Acer,AW35/Pro},"
- "{Crystal,3D},"
- "{Crystal Computer,TidalWave128},"
- "{Dell,Optiplex GX1},"
- "{Dell,Workstation 400 sound},"
- "{EliteGroup,P5TX-LA sound},"
- "{Gallant,SC-70P},"
- "{Gateway,E1000 Onboard CS4236B},"
- "{Genius,Sound Maker 3DJ},"
- "{Hewlett Packard,HP6330 sound},"
- "{IBM,PC 300PL sound},"
- "{IBM,Aptiva 2137 E24},"
- "{IBM,IntelliStation M Pro},"
- "{Intel,Marlin Spike Mobo CS4235},"
- "{Intel PR440FX Onboard},"
- "{Guillemot,MaxiSound 16 PnP},"
- "{NewClear,3D},"
- "{TerraTec,AudioSystem EWS64L/XL},"
- "{Typhoon Soundsystem,CS4236B},"
- "{Turtle Beach,Malibu},"
- "{Unknown,Digital PC 5000 Onboard}}");
-
MODULE_ALIAS("snd_cs4232");
#define IDENT "CS4232+"
MODULE_DESCRIPTION(CRD_NAME);
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ESS,ES688 PnP AudioDrive,pnp:ESS0100},"
- "{ESS,ES1688 PnP AudioDrive,pnp:ESS0102},"
- "{ESS,ES688 AudioDrive,pnp:ESS6881},"
- "{ESS,ES1688 AudioDrive,pnp:ESS1681}}");
-
MODULE_ALIAS("snd_es968");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
/* Card level */
-MODULE_AUTHOR("Christian Fischbach <fishbach@pool.informatik.rwth-aachen.de>, Abramo Bagnara <abramo@alsa-project.org>");
+MODULE_AUTHOR("Christian Fischbach <fishbach@pool.informatik.rwth-aachen.de>, Abramo Bagnara <abramo@alsa-project.org>");
MODULE_DESCRIPTION("ESS ES18xx AudioDrive");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ESS,ES1868 PnP AudioDrive},"
- "{ESS,ES1869 PnP AudioDrive},"
- "{ESS,ES1878 PnP AudioDrive},"
- "{ESS,ES1879 PnP AudioDrive},"
- "{ESS,ES1887 PnP AudioDrive},"
- "{ESS,ES1888 PnP AudioDrive},"
- "{ESS,ES1887 AudioDrive},"
- "{ESS,ES1888 AudioDrive}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_DESCRIPTION(CRD_NAME);
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Gravis,UltraSound Classic}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_DESCRIPTION(CRD_NAME);
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Gravis,UltraSound Extreme}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Gravis UltraSound MAX");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Gravis,UltraSound MAX}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_LICENSE("GPL");
#ifndef SNDRV_STB
MODULE_DESCRIPTION("AMD InterWave");
-MODULE_SUPPORTED_DEVICE("{{Gravis,UltraSound Plug & Play},"
- "{STB,SoundRage32},"
- "{MED,MED3210},"
- "{Dynasonix,Dynasonix Pro},"
- "{Panasonic,PCA761AW}}");
#else
MODULE_DESCRIPTION("AMD InterWave STB with TEA6330T");
-MODULE_SUPPORTED_DEVICE("{{AMD,InterWave STB with TEA6330T}}");
#endif
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Yamaha OPL3SA2+");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Yamaha,YMF719E-S},"
- "{Genius,Sound Maker 3DX},"
- "{Yamaha,OPL3SA3},"
- "{Intel,AL440LX sound},"
- "{NeoMagic,MagicWave 3DX}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Martin Langer <martin-langer@gmx.de>");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Miro miroSOUND PCM1 pro, PCM12, PCM20 Radio");
-MODULE_SUPPORTED_DEVICE("{{Miro,miroSOUND PCM1 pro}, "
- "{Miro,miroSOUND PCM12}, "
- "{Miro,miroSOUND PCM20 Radio}}");
static int index = SNDRV_DEFAULT_IDX1; /* Index 0-MAX */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_LICENSE("GPL");
#ifdef OPTi93X
MODULE_DESCRIPTION("OPTi93X");
-MODULE_SUPPORTED_DEVICE("{{OPTi,82C931/3}}");
#else /* OPTi93X */
#ifdef CS4231
MODULE_DESCRIPTION("OPTi92X - CS4231");
-MODULE_SUPPORTED_DEVICE("{{OPTi,82C924 (CS4231)},"
- "{OPTi,82C925 (CS4231)}}");
#else /* CS4231 */
MODULE_DESCRIPTION("OPTi92X - AD1848");
-MODULE_SUPPORTED_DEVICE("{{OPTi,82C924 (AD1848)},"
- "{OPTi,82C925 (AD1848)},"
- "{OAK,Mozart}}");
#endif /* CS4231 */
#endif /* OPTi93X */
#define PFX "jazz16: "
MODULE_DESCRIPTION("Media Vision Jazz16");
-MODULE_SUPPORTED_DEVICE("{{Media Vision ??? },"
- "{RTL,RTL3000}}");
-
MODULE_AUTHOR("Krzysztof Helt <krzysztof.h1@wp.pl>");
MODULE_LICENSE("GPL");
MODULE_LICENSE("GPL");
#ifndef SNDRV_SBAWE
MODULE_DESCRIPTION("Sound Blaster 16");
-MODULE_SUPPORTED_DEVICE("{{Creative Labs,SB 16},"
- "{Creative Labs,SB Vibra16S},"
- "{Creative Labs,SB Vibra16C},"
- "{Creative Labs,SB Vibra16CL},"
- "{Creative Labs,SB Vibra16X}}");
#else
MODULE_DESCRIPTION("Sound Blaster AWE");
-MODULE_SUPPORTED_DEVICE("{{Creative Labs,SB AWE 32},"
- "{Creative Labs,SB AWE 64},"
- "{Creative Labs,SB AWE 64 Gold}}");
#endif
#if 0
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Sound Blaster 1.0/2.0/Pro");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Creative Labs,SB 1.0/SB 2.0/SB Pro}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Krzysztof Helt");
MODULE_DESCRIPTION("Gallant SC-6000");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Gallant, SC-6000},"
- "{AudioExcel, Audio Excel DSP 16},"
- "{Zoltrix, AV302}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Paul Barton-Davis <pbd@op.net>");
MODULE_DESCRIPTION("Turtle Beach Wavefront");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Turtle Beach,Maui/Tropez/Tropez+}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Vivien Chappelier <vivien.chappelier@linux-mips.org>");
MODULE_DESCRIPTION("SGI O2 Audio");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Silicon Graphics, O2 Audio}}");
static int index = SNDRV_DEFAULT_IDX1; /* Index 0-MAX */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_AUTHOR("Kyle McMartin <kyle@parisc-linux.org>, Thibaut Varene <t-bone@parisc-linux.org>");
MODULE_DESCRIPTION("Analog Devices AD1889 ALSA sound driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Analog Devices,AD1889}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
module_param_array(index, int, NULL, 0444);
MODULE_AUTHOR("Matt Wu <Matt_Wu@acersoftech.com.cn>");
MODULE_DESCRIPTION("ALI M5451");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ALI,M5451,pci},{ALI,M5451}}");
static int index = SNDRV_DEFAULT_IDX1; /* Index */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_AUTHOR("Ash Willis <ashwillis@programmer.net>");
MODULE_DESCRIPTION("Avance Logic ALS300");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Avance Logic,ALS300},{Avance Logic,ALS300+}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
MODULE_AUTHOR("Bart Hartgers <bart@etpmod.phys.tue.nl>, Andreas Mohr");
MODULE_DESCRIPTION("Avance Logic ALS4000");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Avance Logic,ALS4000}}");
#if IS_REACHABLE(CONFIG_GAMEPORT)
#define SUPPORT_JOYSTICK 1
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("ATI IXP AC97 controller");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ATI,IXP150/200/250/300/400/600}}");
static int index = SNDRV_DEFAULT_IDX1; /* Index 0-MAX */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("ATI IXP MC97 controller");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ATI,IXP150/200/250}}");
static int index = -2; /* Exclude the first card */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_DESCRIPTION("Aureal vortex");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Aureal Semiconductor Inc., Aureal Vortex Sound Processor}}");
-
MODULE_DEVICE_TABLE(pci, snd_vortex_ids);
static void vortex_fix_latency(struct pci_dev *vortex)
MODULE_AUTHOR("Andreas Mohr <andi AT lisas.de>");
MODULE_DESCRIPTION("Aztech AZF3328 (PCI168)");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Aztech,AZF3328}}");
#if IS_REACHABLE(CONFIG_GAMEPORT)
#define SUPPORT_GAMEPORT 1
MODULE_AUTHOR("Clemens Ladisch <clemens@ladisch.de>");
MODULE_DESCRIPTION("Brooktree Bt87x audio driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Brooktree,Bt878},"
- "{Brooktree,Bt879}}");
static int index[SNDRV_CARDS] = {[0 ... (SNDRV_CARDS - 1)] = -2}; /* Exclude the first card */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("James Courtier-Dutton <James@superbug.demon.co.uk>");
MODULE_DESCRIPTION("CA0106");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Creative,SB CA0106 chip}}");
// module parameters (see "Module Parameters")
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("C-Media CMI8x38 PCI");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{C-Media,CMI8738},"
- "{C-Media,CMI8738B},"
- "{C-Media,CMI8338A},"
- "{C-Media,CMI8338B}}");
#if IS_REACHABLE(CONFIG_GAMEPORT)
#define SUPPORT_JOYSTICK 1
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Cirrus Logic CS4281");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Cirrus Logic,CS4281}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Cirrus Logic Sound Fusion CS46XX");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Cirrus Logic,Sound Fusion (CS4280)},"
- "{Cirrus Logic,Sound Fusion (CS4610)},"
- "{Cirrus Logic,Sound Fusion (CS4612)},"
- "{Cirrus Logic,Sound Fusion (CS4615)},"
- "{Cirrus Logic,Sound Fusion (CS4622)},"
- "{Cirrus Logic,Sound Fusion (CS4624)},"
- "{Cirrus Logic,Sound Fusion (CS4630)}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaya Kumar");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("CS5535 Audio");
-MODULE_SUPPORTED_DEVICE("CS5535 Audio");
MODULE_AUTHOR("Creative Technology Ltd");
MODULE_DESCRIPTION("X-Fi driver version 1.03");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{Creative Labs, Sound Blaster X-Fi}");
static unsigned int reference_rate = 48000;
static unsigned int multiple = 2;
MODULE_AUTHOR("Giuliano Pochini <pochini@shiny.it>");
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("Echoaudio " ECHOCARD_NAME " soundcards driver");
-MODULE_SUPPORTED_DEVICE("{{Echoaudio," ECHOCARD_NAME "}}");
MODULE_DEVICE_TABLE(pci, snd_echo_ids);
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("EMU10K1");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Creative Labs,SB Live!/PCI512/E-mu APS},"
- "{Creative Labs,SB Audigy}}");
#if IS_ENABLED(CONFIG_SND_SEQUENCER)
#define ENABLE_SYNTH
MODULE_AUTHOR("Francisco Moraes <fmoraes@nc.rr.com>");
MODULE_DESCRIPTION("EMU10K1X");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Dell Creative Labs,SB Live!}");
// module parameters (see "Module Parameters")
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
MODULE_LICENSE("GPL");
#ifdef CHIP1370
MODULE_DESCRIPTION("Ensoniq AudioPCI ES1370");
-MODULE_SUPPORTED_DEVICE("{{Ensoniq,AudioPCI-97 ES1370},"
- "{Creative Labs,SB PCI64/128 (ES1370)}}");
#endif
#ifdef CHIP1371
MODULE_DESCRIPTION("Ensoniq/Creative AudioPCI ES1371+");
-MODULE_SUPPORTED_DEVICE("{{Ensoniq,AudioPCI ES1371/73},"
- "{Ensoniq,AudioPCI ES1373},"
- "{Creative Labs,Ectiva EV1938},"
- "{Creative Labs,SB PCI64/128 (ES1371/73)},"
- "{Creative Labs,Vibra PCI128},"
- "{Ectiva,EV1938}}");
#endif
#if IS_REACHABLE(CONFIG_GAMEPORT)
MODULE_AUTHOR("Jaromir Koutek <miri@punknet.cz>");
MODULE_DESCRIPTION("ESS Solo-1");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ESS,ES1938},"
- "{ESS,ES1946},"
- "{ESS,ES1969},"
- "{TerraTec,128i PCI}}");
#if IS_REACHABLE(CONFIG_GAMEPORT)
#define SUPPORT_JOYSTICK 1
MODULE_DESCRIPTION("ESS Maestro");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ESS,Maestro 2e},"
- "{ESS,Maestro 2},"
- "{ESS,Maestro 1},"
- "{TerraTec,DMX}}");
#if IS_REACHABLE(CONFIG_GAMEPORT)
#define SUPPORT_JOYSTICK 1
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("ForteMedia FM801");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ForteMedia,FM801},"
- "{Genius,SoundMaker Live 5.1}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
spec->micmute_led.led_mode = MICMUTE_LED_FOLLOW_MUTE;
spec->micmute_led.capture = 0;
- spec->micmute_led.led_value = 0;
+ spec->micmute_led.led_value = -1;
spec->micmute_led.old_hook = spec->cap_sync_hook;
spec->cap_sync_hook = update_micmute_led;
if (!snd_hda_gen_add_kctl(spec, NULL, &micmute_led_mode_ctl))
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Intel, ICH6},"
- "{Intel, ICH6M},"
- "{Intel, ICH7},"
- "{Intel, ESB2},"
- "{Intel, ICH8},"
- "{Intel, ICH9},"
- "{Intel, ICH10},"
- "{Intel, PCH},"
- "{Intel, CPT},"
- "{Intel, PPT},"
- "{Intel, LPT},"
- "{Intel, LPT_LP},"
- "{Intel, WPT_LP},"
- "{Intel, SPT},"
- "{Intel, SPT_LP},"
- "{Intel, HPT},"
- "{Intel, PBG},"
- "{Intel, SCH},"
- "{ATI, SB450},"
- "{ATI, SB600},"
- "{ATI, RS600},"
- "{ATI, RS690},"
- "{ATI, RS780},"
- "{ATI, R600},"
- "{ATI, RV630},"
- "{ATI, RV610},"
- "{ATI, RV670},"
- "{ATI, RV635},"
- "{ATI, RV620},"
- "{ATI, RV770},"
- "{VIA, VT8251},"
- "{VIA, VT8237A},"
- "{SiS, SIS966},"
- "{ULI, M5461}}");
MODULE_DESCRIPTION("Intel HDA driver");
#if defined(CONFIG_PM) && defined(CONFIG_VGA_SWITCHEROO)
struct snd_card *card = dev_get_drvdata(dev);
struct azx *chip;
+ if (!azx_is_pm_ready(card))
+ return 0;
+
chip = card->private_data;
chip->pm_prepared = 1;
+ snd_power_change_state(card, SNDRV_CTL_POWER_D3hot);
flush_work(&azx_bus(chip)->unsol_work);
struct snd_card *card = dev_get_drvdata(dev);
struct azx *chip;
+ if (!azx_is_pm_ready(card))
+ return;
+
chip = card->private_data;
+ snd_power_change_state(card, SNDRV_CTL_POWER_D0);
chip->pm_prepared = 0;
}
SND_PCI_QUIRK(0x103c, 0x829a, "HP 800 G3 DM", CXT_FIXUP_HP_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x103c, 0x8402, "HP ProBook 645 G4", CXT_FIXUP_MUTE_LED_GPIO),
SND_PCI_QUIRK(0x103c, 0x8427, "HP ZBook Studio G5", CXT_FIXUP_HP_ZBOOK_MUTE_LED),
+ SND_PCI_QUIRK(0x103c, 0x844f, "HP ZBook Studio G5", CXT_FIXUP_HP_ZBOOK_MUTE_LED),
SND_PCI_QUIRK(0x103c, 0x8455, "HP Z2 G4", CXT_FIXUP_HP_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x103c, 0x8456, "HP Z2 G4 SFF", CXT_FIXUP_HP_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x103c, 0x8457, "HP Z2 G4 mini", CXT_FIXUP_HP_MIC_NO_PRESENCE),
snd_hda_sequence_write(codec, verbs);
}
+/* Fix the speaker amp after resume, etc */
+static void alc269vb_fixup_aspire_e1_coef(struct hda_codec *codec,
+ const struct hda_fixup *fix,
+ int action)
+{
+ if (action == HDA_FIXUP_ACT_INIT)
+ alc_update_coef_idx(codec, 0x0d, 0x6000, 0x6000);
+}
+
static void alc269_fixup_pcm_44k(struct hda_codec *codec,
const struct hda_fixup *fix, int action)
{
}
}
+static void alc236_fixup_hp_gpio_led(struct hda_codec *codec,
+ const struct hda_fixup *fix, int action)
+{
+ alc_fixup_hp_gpio_led(codec, action, 0x02, 0x01);
+}
+
static void alc269_fixup_hp_gpio_led(struct hda_codec *codec,
const struct hda_fixup *fix, int action)
{
case 0x10ec0274:
case 0x10ec0294:
alc_process_coef_fw(codec, coef0274);
- msleep(80);
+ msleep(850);
val = alc_read_coef_idx(codec, 0x46);
is_ctia = (val & 0x00f0) == 0x00f0;
break;
struct hda_jack_callback *jack)
{
snd_hda_gen_hp_automute(codec, jack);
+ alc_update_headset_mode(codec);
}
static void alc_probe_headset_mode(struct hda_codec *codec)
ALC283_FIXUP_HEADSET_MIC,
ALC255_FIXUP_MIC_MUTE_LED,
ALC282_FIXUP_ASPIRE_V5_PINS,
+ ALC269VB_FIXUP_ASPIRE_E1_COEF,
ALC280_FIXUP_HP_GPIO4,
ALC286_FIXUP_HP_GPIO_LED,
ALC280_FIXUP_HP_GPIO2_MIC_HOTKEY,
ALC294_FIXUP_ASUS_GX502_VERBS,
ALC285_FIXUP_HP_GPIO_LED,
ALC285_FIXUP_HP_MUTE_LED,
+ ALC236_FIXUP_HP_GPIO_LED,
ALC236_FIXUP_HP_MUTE_LED,
ALC298_FIXUP_SAMSUNG_HEADPHONE_VERY_QUIET,
ALC295_FIXUP_ASUS_MIC_NO_PRESENCE,
{ },
},
},
+ [ALC269VB_FIXUP_ASPIRE_E1_COEF] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc269vb_fixup_aspire_e1_coef,
+ },
[ALC280_FIXUP_HP_GPIO4] = {
.type = HDA_FIXUP_FUNC,
.v.func = alc280_fixup_hp_gpio4,
.type = HDA_FIXUP_FUNC,
.v.func = alc285_fixup_hp_mute_led,
},
+ [ALC236_FIXUP_HP_GPIO_LED] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc236_fixup_hp_gpio_led,
+ },
[ALC236_FIXUP_HP_MUTE_LED] = {
.type = HDA_FIXUP_FUNC,
.v.func = alc236_fixup_hp_mute_led,
SND_PCI_QUIRK(0x1025, 0x0762, "Acer Aspire E1-472", ALC271_FIXUP_HP_GATE_MIC_JACK_E1_572),
SND_PCI_QUIRK(0x1025, 0x0775, "Acer Aspire E1-572", ALC271_FIXUP_HP_GATE_MIC_JACK_E1_572),
SND_PCI_QUIRK(0x1025, 0x079b, "Acer Aspire V5-573G", ALC282_FIXUP_ASPIRE_V5_PINS),
+ SND_PCI_QUIRK(0x1025, 0x0840, "Acer Aspire E1", ALC269VB_FIXUP_ASPIRE_E1_COEF),
SND_PCI_QUIRK(0x1025, 0x101c, "Acer Veriton N2510G", ALC269_FIXUP_LIFEBOOK),
SND_PCI_QUIRK(0x1025, 0x102b, "Acer Aspire C24-860", ALC286_FIXUP_ACER_AIO_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1025, 0x1065, "Acer Aspire C20-820", ALC269VC_FIXUP_ACER_HEADSET_MIC),
SND_PCI_QUIRK(0x103c, 0x8783, "HP ZBook Fury 15 G7 Mobile Workstation",
ALC285_FIXUP_HP_GPIO_AMP_INIT),
SND_PCI_QUIRK(0x103c, 0x87c8, "HP", ALC287_FIXUP_HP_GPIO_LED),
+ SND_PCI_QUIRK(0x103c, 0x87e5, "HP ProBook 440 G8 Notebook PC", ALC236_FIXUP_HP_GPIO_LED),
+ SND_PCI_QUIRK(0x103c, 0x87f2, "HP ProBook 640 G8 Notebook PC", ALC236_FIXUP_HP_GPIO_LED),
SND_PCI_QUIRK(0x103c, 0x87f4, "HP", ALC287_FIXUP_HP_GPIO_LED),
SND_PCI_QUIRK(0x103c, 0x87f5, "HP", ALC287_FIXUP_HP_GPIO_LED),
SND_PCI_QUIRK(0x103c, 0x87f7, "HP Spectre x360 14", ALC245_FIXUP_HP_X360_AMP),
+ SND_PCI_QUIRK(0x103c, 0x8846, "HP EliteBook 850 G8 Notebook PC", ALC285_FIXUP_HP_GPIO_LED),
+ SND_PCI_QUIRK(0x103c, 0x884c, "HP EliteBook 840 G8 Notebook PC", ALC285_FIXUP_HP_GPIO_LED),
SND_PCI_QUIRK(0x1043, 0x103e, "ASUS X540SA", ALC256_FIXUP_ASUS_MIC),
SND_PCI_QUIRK(0x1043, 0x103f, "ASUS TX300", ALC282_FIXUP_ASUS_TX300),
SND_PCI_QUIRK(0x1043, 0x106d, "Asus K53BE", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x1b35, 0x1237, "CZC L101", ALC269_FIXUP_CZC_L101),
SND_PCI_QUIRK(0x1b7d, 0xa831, "Ordissimo EVE2 ", ALC269VB_FIXUP_ORDISSIMO_EVE2), /* Also known as Malata PC-B1303 */
SND_PCI_QUIRK(0x1d72, 0x1602, "RedmiBook", ALC255_FIXUP_XIAOMI_HEADSET_MIC),
+ SND_PCI_QUIRK(0x1d72, 0x1701, "XiaomiNotebook Pro", ALC298_FIXUP_DELL1_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1d72, 0x1901, "RedmiBook 14", ALC256_FIXUP_ASUS_HEADSET_MIC),
+ SND_PCI_QUIRK(0x1d72, 0x1947, "RedmiBook Air", ALC255_FIXUP_XIAOMI_HEADSET_MIC),
SND_PCI_QUIRK(0x10ec, 0x118c, "Medion EE4254 MD62100", ALC256_FIXUP_MEDION_HEADSET_NO_PRESENCE),
SND_PCI_QUIRK(0x1c06, 0x2013, "Lemote A1802", ALC269_FIXUP_LEMOTE_A1802),
SND_PCI_QUIRK(0x1c06, 0x2015, "Lemote A190X", ALC269_FIXUP_LEMOTE_A190X),
{.id = ALC283_FIXUP_HEADSET_MIC, .name = "alc283-headset"},
{.id = ALC255_FIXUP_MIC_MUTE_LED, .name = "alc255-dell-mute"},
{.id = ALC282_FIXUP_ASPIRE_V5_PINS, .name = "aspire-v5"},
+ {.id = ALC269VB_FIXUP_ASPIRE_E1_COEF, .name = "aspire-e1-coef"},
{.id = ALC280_FIXUP_HP_GPIO4, .name = "hp-gpio4"},
{.id = ALC286_FIXUP_HP_GPIO_LED, .name = "hp-gpio-led"},
{.id = ALC280_FIXUP_HP_GPIO2_MIC_HOTKEY, .name = "hp-gpio2-hotkey"},
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("ICEnsemble ICE1712 (Envy24)");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{"
- HOONTECH_DEVICE_DESC
- DELTA_DEVICE_DESC
- EWS_DEVICE_DESC
- "{ICEnsemble,Generic ICE1712},"
- "{ICEnsemble,Generic Envy24}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("VIA ICEnsemble ICE1724/1720 (Envy24HT/PT)");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{"
- REVO_DEVICE_DESC
- AMP_AUDIO2000_DEVICE_DESC
- AUREON_DEVICE_DESC
- VT1720_MOBO_DEVICE_DESC
- PONTIS_DEVICE_DESC
- PRODIGY192_DEVICE_DESC
- PRODIGY_HIFI_DEVICE_DESC
- JULI_DEVICE_DESC
- MAYA44_DEVICE_DESC
- PHASE_DEVICE_DESC
- WTM_DEVICE_DESC
- SE_DEVICE_DESC
- QTET_DEVICE_DESC
- "{VIA,VT1720},"
- "{VIA,VT1724},"
- "{ICEnsemble,Generic ICE1724},"
- "{ICEnsemble,Generic Envy24HT}"
- "{ICEnsemble,Generic Envy24PT}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Intel 82801AA,82901AB,i810,i820,i830,i840,i845,MX440; SiS 7012; Ali 5455");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Intel,82801AA-ICH},"
- "{Intel,82901AB-ICH0},"
- "{Intel,82801BA-ICH2},"
- "{Intel,82801CA-ICH3},"
- "{Intel,82801DB-ICH4},"
- "{Intel,ICH5},"
- "{Intel,ICH6},"
- "{Intel,ICH7},"
- "{Intel,6300ESB},"
- "{Intel,ESB2},"
- "{Intel,MX440},"
- "{SiS,SI7012},"
- "{NVidia,nForce Audio},"
- "{NVidia,nForce2 Audio},"
- "{NVidia,nForce3 Audio},"
- "{NVidia,MCP04},"
- "{NVidia,MCP501},"
- "{NVidia,CK804},"
- "{NVidia,CK8},"
- "{NVidia,CK8S},"
- "{AMD,AMD768},"
- "{AMD,AMD8111},"
- "{ALI,M5455}}");
static int index = SNDRV_DEFAULT_IDX1; /* Index 0-MAX */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_DESCRIPTION("Intel 82801AA,82901AB,i810,i820,i830,i840,i845,MX440; "
"SiS 7013; NVidia MCP/2/2S/3 modems");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Intel,82801AA-ICH},"
- "{Intel,82901AB-ICH0},"
- "{Intel,82801BA-ICH2},"
- "{Intel,82801CA-ICH3},"
- "{Intel,82801DB-ICH4},"
- "{Intel,ICH5},"
- "{Intel,ICH6},"
- "{Intel,ICH7},"
- "{Intel,MX440},"
- "{SiS,7013},"
- "{NVidia,NForce Modem},"
- "{NVidia,NForce2 Modem},"
- "{NVidia,NForce2s Modem},"
- "{NVidia,NForce3 Modem},"
- "{AMD,AMD768}}");
static int index = -2; /* Exclude the first card */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_DESCRIPTION("korg1212");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{KORG,korg1212}}");
MODULE_FIRMWARE("korg/k1212.dsp");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
*/
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Digigram, Lola}}");
MODULE_DESCRIPTION("Digigram Lola driver");
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_AUTHOR("Tim Blechmann");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("digigram lx6464es");
-MODULE_SUPPORTED_DEVICE("{digigram lx6464es{}}");
-
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
MODULE_AUTHOR("Zach Brown <zab@zabbo.net>, Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("ESS Maestro3 PCI");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{ESS,Maestro3 PCI},"
- "{ESS,ES1988},"
- "{ESS,Allegro PCI},"
- "{ESS,Allegro-1 PCI},"
- "{ESS,Canyon3D-2/LE PCI}}");
MODULE_FIRMWARE("ess/maestro3_assp_kernel.fw");
MODULE_FIRMWARE("ess/maestro3_assp_minisrc.fw");
MODULE_AUTHOR("Digigram <alsa@digigram.com>");
MODULE_DESCRIPTION("Digigram " CARD_NAME);
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Digigram," CARD_NAME "}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("NeoMagic NM256AV/ZX");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{NeoMagic,NM256AV},"
- "{NeoMagic,NM256ZX}}");
/*
* some compile conditions.
MODULE_AUTHOR("Clemens Ladisch <clemens@ladisch.de>");
MODULE_DESCRIPTION("C-Media CMI8788 driver");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{C-Media,CMI8786}"
- ",{C-Media,CMI8787}"
- ",{C-Media,CMI8788}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
MODULE_AUTHOR("Clemens Ladisch <clemens@ladisch.de>");
MODULE_DESCRIPTION("Studio Evolution SE6X driver");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{Studio Evolution,SE6X}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
MODULE_AUTHOR("Clemens Ladisch <clemens@ladisch.de>");
MODULE_DESCRIPTION("Asus Virtuoso driver");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{Asus,AV66},{Asus,AV100},{Asus,AV200}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
"Marc Titinger <titinger@digigram.com>");
MODULE_DESCRIPTION("Digigram " DRIVER_NAME " " PCXHR_DRIVER_VERSION_STRING);
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Digigram," DRIVER_NAME "}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Peter Gruber <nokos@gmx.net>");
MODULE_DESCRIPTION("riptide");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Conexant,Riptide}}");
MODULE_FIRMWARE("riptide.hex");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
MODULE_AUTHOR("Martin Langer <martin-langer@gmx.de>, Pilo Chambert <pilo.c@wanadoo.fr>");
MODULE_DESCRIPTION("RME Digi32, Digi32/8, Digi32 PRO");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{RME,Digi32}," "{RME,Digi32/8}," "{RME,Digi32 PRO}}");
/* Defines for RME Digi32 series */
#define RME32_SPDIF_NCHANNELS 2
MODULE_DESCRIPTION("RME Digi96, Digi96/8, Digi96/8 PRO, Digi96/8 PST, "
"Digi96/8 PAD");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{RME,Digi96},"
- "{RME,Digi96/8},"
- "{RME,Digi96/8 PRO},"
- "{RME,Digi96/8 PST},"
- "{RME,Digi96/8 PAD}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Paul Davis <paul@linuxaudiosystems.com>, Marcus Andersson, Thomas Charbonnel <thomas@undata.org>");
MODULE_DESCRIPTION("RME Hammerfall DSP");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{RME Hammerfall-DSP},"
- "{RME HDSP-9652},"
- "{RME HDSP-9632}}");
MODULE_FIRMWARE("rpm_firmware.bin");
MODULE_FIRMWARE("multiface_firmware.bin");
MODULE_FIRMWARE("multiface_firmware_rev11.bin");
);
MODULE_DESCRIPTION("RME HDSPM");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{RME HDSPM-MADI}}");
/* --- Write registers. ---
These are defined as byte-offsets from the iobase value. */
MODULE_AUTHOR("Paul Davis <pbd@op.net>, Winfried Ritsch");
MODULE_DESCRIPTION("RME Digi9652/Digi9636");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{RME,Hammerfall},"
- "{RME,Hammerfall-Light}}");
/* The Hammerfall has two sets of 24 ADAT + 2 S/PDIF channels, one for
capture, one for playback. Both the ADAT and S/PDIF channels appear
MODULE_AUTHOR("David Dillow <dave@thedillows.org>");
MODULE_DESCRIPTION("SiS7019");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{SiS,SiS7019 Audio Accelerator}}");
static int index = SNDRV_DEFAULT_IDX1; /* Index 0-MAX */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("S3 SonicVibes PCI");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{S3,SonicVibes PCI}}");
#if IS_REACHABLE(CONFIG_GAMEPORT)
#define SUPPORT_JOYSTICK 1
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>, <audio@tridentmicro.com>");
MODULE_DESCRIPTION("Trident 4D-WaveDX/NX & SiS SI7018");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Trident,4DWave DX},"
- "{Trident,4DWave NX},"
- "{SiS,SI7018 PCI Audio},"
- "{Best Union,Miss Melody 4DWave PCI},"
- "{HIS,4DWave PCI},"
- "{Warpspeed,ONSpeed 4DWave PCI},"
- "{Aztech Systems,PCI 64-Q3D},"
- "{Addonics,SV 750},"
- "{CHIC,True Sound 4Dwave},"
- "{Shark,Predator4D-PCI},"
- "{Jaton,SonicWave 4D},"
- "{Hoontech,SoundTrack Digital 4DWave NX}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("VIA VT82xx audio");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{VIA,VT82C686A/B/C,pci},{VIA,VT8233A/C,8235}}");
#if IS_REACHABLE(CONFIG_GAMEPORT)
#define SUPPORT_JOYSTICK 1
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("VIA VT82xx modem");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{VIA,VT82C686A/B/C modem,pci}}");
static int index = -2; /* Exclude the first card */
static char *id = SNDRV_DEFAULT_STR1; /* ID for this card */
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("Digigram VX222 V2/Mic");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Digigram," CARD_NAME "}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Yamaha DS-1 PCI");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Yamaha,YMF724},"
- "{Yamaha,YMF724F},"
- "{Yamaha,YMF740},"
- "{Yamaha,YMF740C},"
- "{Yamaha,YMF744},"
- "{Yamaha,YMF754}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Sound Core " CARD_NAME);
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Sound Core," CARD_NAME "}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
#include <sound/initval.h>
#include <sound/tlv.h>
-/*
- */
-
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("Digigram VXPocket");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Digigram,VXPocket},{Digigram,VXPocket440}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
#define CHIP_NAME "PMac"
MODULE_DESCRIPTION("PowerMac");
-MODULE_SUPPORTED_DEVICE("{{Apple,PowerMac}}");
MODULE_LICENSE("GPL");
static int index = SNDRV_DEFAULT_IDX1; /* Index 0-MAX */
MODULE_AUTHOR("Adrian McMenamin <adrian@mcmen.demon.co.uk>");
MODULE_DESCRIPTION("Dreamcast AICA sound (pcm) driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Yamaha/SEGA, AICA}}");
MODULE_FIRMWARE("aica_firmware.bin");
/* module parameters */
MODULE_AUTHOR("Rafael Ignacio Zurita <rizurita@yahoo.com>");
MODULE_DESCRIPTION("SuperH DAC audio driver");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{SuperH DAC audio support}}");
/* Module Parameters */
static int index = SNDRV_DEFAULT_IDX1;
&cygnus_ssp_dai[active_port_count]);
/* negative is err, 0 is active and good, 1 is disabled */
- if (err < 0)
+ if (err < 0) {
+ of_node_put(child_node);
return err;
+ }
else if (!err) {
dev_dbg(dev, "Activating DAI: %s\n",
cygnus_ssp_dai[active_port_count].name);
imply SND_SOC_SI476X
imply SND_SOC_SIMPLE_AMPLIFIER
imply SND_SOC_SIMPLE_MUX
- imply SND_SOC_SIRF_AUDIO_CODEC
imply SND_SOC_SPDIF
imply SND_SOC_SSM2305
imply SND_SOC_SSM2518
tristate "Simple Audio Mux"
select GPIOLIB
-config SND_SOC_SIRF_AUDIO_CODEC
- tristate "SiRF SoC internal audio codec"
- select REGMAP_MMIO
-
config SND_SOC_SPDIF
tristate "S/PDIF CODEC"
{ .compatible = "asahi-kasei,ak4497", .data = &ak4497_drvdata},
{ },
};
+MODULE_DEVICE_TABLE(of, ak4458_of_match);
static struct i2c_driver ak4458_i2c_driver = {
.driver = {
{ .compatible = "asahi-kasei,ak5558"},
{ }
};
+MODULE_DEVICE_TABLE(of, ak5558_i2c_dt_ids);
static struct i2c_driver ak5558_i2c_driver = {
.driver = {
};
static DECLARE_TLV_DB_SCALE(adc_tlv, -9600, 100, false);
-static DECLARE_TLV_DB_SCALE(mixer_tlv, -6200, 100, false);
+static DECLARE_TLV_DB_SCALE(mixer_tlv, -6300, 100, true);
static const char * const cs42l42_hpf_freq_text[] = {
"1.86Hz", "120Hz", "235Hz", "466Hz"
CS42L42_DAC_HPF_EN_SHIFT, true, false),
SOC_DOUBLE_R_TLV("Mixer Volume", CS42L42_MIXER_CHA_VOL,
CS42L42_MIXER_CHB_VOL, CS42L42_MIXER_CH_VOL_SHIFT,
- 0x3e, 1, mixer_tlv)
+ 0x3f, 1, mixer_tlv)
};
static int cs42l42_hpdrv_evt(struct snd_soc_dapm_widget *w,
{"HP", NULL, "HPDRV"}
};
-static int cs42l42_set_bias_level(struct snd_soc_component *component,
- enum snd_soc_bias_level level)
-{
- struct cs42l42_private *cs42l42 = snd_soc_component_get_drvdata(component);
- int ret;
-
- switch (level) {
- case SND_SOC_BIAS_ON:
- break;
- case SND_SOC_BIAS_PREPARE:
- break;
- case SND_SOC_BIAS_STANDBY:
- if (snd_soc_component_get_bias_level(component) == SND_SOC_BIAS_OFF) {
- regcache_cache_only(cs42l42->regmap, false);
- regcache_sync(cs42l42->regmap);
- ret = regulator_bulk_enable(
- ARRAY_SIZE(cs42l42->supplies),
- cs42l42->supplies);
- if (ret != 0) {
- dev_err(component->dev,
- "Failed to enable regulators: %d\n",
- ret);
- return ret;
- }
- }
- break;
- case SND_SOC_BIAS_OFF:
-
- regcache_cache_only(cs42l42->regmap, true);
- regulator_bulk_disable(ARRAY_SIZE(cs42l42->supplies),
- cs42l42->supplies);
- break;
- }
-
- return 0;
-}
-
static int cs42l42_component_probe(struct snd_soc_component *component)
{
struct cs42l42_private *cs42l42 =
static const struct snd_soc_component_driver soc_component_dev_cs42l42 = {
.probe = cs42l42_component_probe,
- .set_bias_level = cs42l42_set_bias_level,
.dapm_widgets = cs42l42_dapm_widgets,
.num_dapm_widgets = ARRAY_SIZE(cs42l42_dapm_widgets),
.dapm_routes = cs42l42_audio_map,
CS42L42_CLK_OASRC_SEL_MASK,
CS42L42_CLK_OASRC_SEL_12 <<
CS42L42_CLK_OASRC_SEL_SHIFT);
- /* channel 1 on low LRCLK, 32 bit */
- snd_soc_component_update_bits(component,
- CS42L42_ASP_RX_DAI0_CH1_AP_RES,
- CS42L42_ASP_RX_CH_AP_MASK |
- CS42L42_ASP_RX_CH_RES_MASK,
- (CS42L42_ASP_RX_CH_AP_LOW <<
- CS42L42_ASP_RX_CH_AP_SHIFT) |
- (CS42L42_ASP_RX_CH_RES_32 <<
- CS42L42_ASP_RX_CH_RES_SHIFT));
- /* Channel 2 on high LRCLK, 32 bit */
- snd_soc_component_update_bits(component,
- CS42L42_ASP_RX_DAI0_CH2_AP_RES,
- CS42L42_ASP_RX_CH_AP_MASK |
- CS42L42_ASP_RX_CH_RES_MASK,
- (CS42L42_ASP_RX_CH_AP_HI <<
- CS42L42_ASP_RX_CH_AP_SHIFT) |
- (CS42L42_ASP_RX_CH_RES_32 <<
- CS42L42_ASP_RX_CH_RES_SHIFT));
if (pll_ratio_table[i].mclk_src_sel == 0) {
/* Pass the clock straight through */
snd_soc_component_update_bits(component,
/* Bitclock/frame inversion */
switch (fmt & SND_SOC_DAIFMT_INV_MASK) {
case SND_SOC_DAIFMT_NB_NF:
+ asp_cfg_val |= CS42L42_ASP_SCPOL_NOR << CS42L42_ASP_SCPOL_SHIFT;
break;
case SND_SOC_DAIFMT_NB_IF:
- asp_cfg_val |= CS42L42_ASP_POL_INV <<
- CS42L42_ASP_LCPOL_IN_SHIFT;
+ asp_cfg_val |= CS42L42_ASP_SCPOL_NOR << CS42L42_ASP_SCPOL_SHIFT;
+ asp_cfg_val |= CS42L42_ASP_LCPOL_INV << CS42L42_ASP_LCPOL_SHIFT;
break;
case SND_SOC_DAIFMT_IB_NF:
- asp_cfg_val |= CS42L42_ASP_POL_INV <<
- CS42L42_ASP_SCPOL_IN_DAC_SHIFT;
break;
case SND_SOC_DAIFMT_IB_IF:
- asp_cfg_val |= CS42L42_ASP_POL_INV <<
- CS42L42_ASP_LCPOL_IN_SHIFT;
- asp_cfg_val |= CS42L42_ASP_POL_INV <<
- CS42L42_ASP_SCPOL_IN_DAC_SHIFT;
+ asp_cfg_val |= CS42L42_ASP_LCPOL_INV << CS42L42_ASP_LCPOL_SHIFT;
break;
}
- snd_soc_component_update_bits(component, CS42L42_ASP_CLK_CFG,
- CS42L42_ASP_MODE_MASK |
- CS42L42_ASP_SCPOL_IN_DAC_MASK |
- CS42L42_ASP_LCPOL_IN_MASK, asp_cfg_val);
+ snd_soc_component_update_bits(component, CS42L42_ASP_CLK_CFG, CS42L42_ASP_MODE_MASK |
+ CS42L42_ASP_SCPOL_MASK |
+ CS42L42_ASP_LCPOL_MASK,
+ asp_cfg_val);
return 0;
}
{
struct snd_soc_component *component = dai->component;
struct cs42l42_private *cs42l42 = snd_soc_component_get_drvdata(component);
- int retval;
+ unsigned int width = (params_width(params) / 8) - 1;
+ unsigned int val = 0;
cs42l42->srate = params_rate(params);
- cs42l42->swidth = params_width(params);
- retval = cs42l42_pll_config(component);
+ switch(substream->stream) {
+ case SNDRV_PCM_STREAM_PLAYBACK:
+ val |= width << CS42L42_ASP_RX_CH_RES_SHIFT;
+ /* channel 1 on low LRCLK */
+ snd_soc_component_update_bits(component, CS42L42_ASP_RX_DAI0_CH1_AP_RES,
+ CS42L42_ASP_RX_CH_AP_MASK |
+ CS42L42_ASP_RX_CH_RES_MASK, val);
+ /* Channel 2 on high LRCLK */
+ val |= CS42L42_ASP_RX_CH_AP_HI << CS42L42_ASP_RX_CH_AP_SHIFT;
+ snd_soc_component_update_bits(component, CS42L42_ASP_RX_DAI0_CH2_AP_RES,
+ CS42L42_ASP_RX_CH_AP_MASK |
+ CS42L42_ASP_RX_CH_RES_MASK, val);
+ break;
+ default:
+ break;
+ }
- return retval;
+ return cs42l42_pll_config(component);
}
static int cs42l42_set_sysclk(struct snd_soc_dai *dai,
return 0;
}
-#define CS42L42_FORMATS (SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S18_3LE | \
- SNDRV_PCM_FMTBIT_S20_3LE | SNDRV_PCM_FMTBIT_S24_LE | \
- SNDRV_PCM_FMTBIT_S32_LE)
+#define CS42L42_FORMATS (SNDRV_PCM_FMTBIT_S16_LE |\
+ SNDRV_PCM_FMTBIT_S24_LE |\
+ SNDRV_PCM_FMTBIT_S32_LE )
static const struct snd_soc_dai_ops cs42l42_ops = {
dev_dbg(&i2c_client->dev, "Found reset GPIO\n");
gpiod_set_value_cansleep(cs42l42->reset_gpio, 1);
}
- mdelay(3);
+ usleep_range(CS42L42_BOOT_TIME_US, CS42L42_BOOT_TIME_US * 2);
/* Request IRQ */
ret = devm_request_threaded_irq(&i2c_client->dev,
}
gpiod_set_value_cansleep(cs42l42->reset_gpio, 1);
+ usleep_range(CS42L42_BOOT_TIME_US, CS42L42_BOOT_TIME_US * 2);
regcache_cache_only(cs42l42->regmap, false);
regcache_sync(cs42l42->regmap);
#define CS42L42_ASP_SLAVE_MODE 0x00
#define CS42L42_ASP_MODE_SHIFT 4
#define CS42L42_ASP_MODE_MASK (1 << CS42L42_ASP_MODE_SHIFT)
-#define CS42L42_ASP_SCPOL_IN_DAC_SHIFT 2
-#define CS42L42_ASP_SCPOL_IN_DAC_MASK (1 << CS42L42_ASP_SCPOL_IN_DAC_SHIFT)
-#define CS42L42_ASP_LCPOL_IN_SHIFT 0
-#define CS42L42_ASP_LCPOL_IN_MASK (1 << CS42L42_ASP_LCPOL_IN_SHIFT)
-#define CS42L42_ASP_POL_INV 1
+#define CS42L42_ASP_SCPOL_SHIFT 2
+#define CS42L42_ASP_SCPOL_MASK (3 << CS42L42_ASP_SCPOL_SHIFT)
+#define CS42L42_ASP_SCPOL_NOR 3
+#define CS42L42_ASP_LCPOL_SHIFT 0
+#define CS42L42_ASP_LCPOL_MASK (3 << CS42L42_ASP_LCPOL_SHIFT)
+#define CS42L42_ASP_LCPOL_INV 3
#define CS42L42_ASP_FRM_CFG (CS42L42_PAGE_12 + 0x08)
#define CS42L42_ASP_STP_SHIFT 4
#define CS42L42_FRAC2_VAL(val) (((val) & 0xff0000) >> 16)
#define CS42L42_NUM_SUPPLIES 5
+#define CS42L42_BOOT_TIME_US 3000
static const char *const cs42l42_supply_names[CS42L42_NUM_SUPPLIES] = {
"VA",
struct completion pdn_done;
u32 sclk;
u32 srate;
- u32 swidth;
u8 plug_state;
u8 hs_type;
u8 ts_inv;
1, 1, TLV_DB_SCALE_ITEM(0, 0, 0),
2, 2, TLV_DB_SCALE_ITEM(250, 0, 0),
3, 3, TLV_DB_SCALE_ITEM(450, 0, 0),
- 4, 4, TLV_DB_SCALE_ITEM(700, 0, 0),
- 5, 5, TLV_DB_SCALE_ITEM(1000, 0, 0),
- 6, 6, TLV_DB_SCALE_ITEM(1300, 0, 0),
- 7, 7, TLV_DB_SCALE_ITEM(1600, 0, 0),
- 8, 8, TLV_DB_SCALE_ITEM(1800, 0, 0),
- 9, 9, TLV_DB_SCALE_ITEM(2100, 0, 0),
- 10, 10, TLV_DB_SCALE_ITEM(2400, 0, 0),
+ 4, 7, TLV_DB_SCALE_ITEM(700, 300, 0),
+ 8, 10, TLV_DB_SCALE_ITEM(1800, 300, 0),
);
static const SNDRV_CTL_TLVD_DECLARE_DB_RANGE(hpout_vol_tlv,
{
struct snd_soc_component *component = snd_soc_dapm_to_component(w->dapm);
u16 val, ec_hq_reg;
- int ec_tx;
+ int ec_tx = -1;
val = snd_soc_component_read(component,
CDC_RX_INP_MUX_RX_MIX_CFG4);
/* set MCLK and NPL rates */
clk_set_rate(rx->clks[2].clk, MCLK_FREQ);
- clk_set_rate(rx->clks[3].clk, MCLK_FREQ);
+ clk_set_rate(rx->clks[3].clk, 2 * MCLK_FREQ);
ret = clk_bulk_prepare_enable(RX_NUM_CLKS_MAX, rx->clks);
if (ret)
/* set MCLK and NPL rates */
clk_set_rate(tx->clks[2].clk, MCLK_FREQ);
- clk_set_rate(tx->clks[3].clk, MCLK_FREQ);
+ clk_set_rate(tx->clks[3].clk, 2 * MCLK_FREQ);
ret = clk_bulk_prepare_enable(TX_NUM_CLKS_MAX, tx->clks);
if (ret)
struct device *dev;
unsigned long active_ch_mask[VA_MACRO_MAX_DAIS];
unsigned long active_ch_cnt[VA_MACRO_MAX_DAIS];
- unsigned long active_decimator[VA_MACRO_MAX_DAIS];
u16 dmic_clk_div;
int dec_mode[VA_MACRO_NUM_DECIMATORS];
if (enable) {
set_bit(dec_id, &va->active_ch_mask[dai_id]);
va->active_ch_cnt[dai_id]++;
- va->active_decimator[dai_id] = dec_id;
} else {
clear_bit(dec_id, &va->active_ch_mask[dai_id]);
va->active_ch_cnt[dai_id]--;
- va->active_decimator[dai_id] = -1;
}
snd_soc_dapm_mixer_update_power(widget->dapm, kcontrol, enable, update);
struct va_macro *va = snd_soc_component_get_drvdata(component);
u16 tx_vol_ctl_reg, decimator;
- decimator = va->active_decimator[dai->id];
-
- tx_vol_ctl_reg = CDC_VA_TX0_TX_PATH_CTL +
- VA_MACRO_TX_PATH_OFFSET * decimator;
- if (mute)
- snd_soc_component_update_bits(component, tx_vol_ctl_reg,
- CDC_VA_TX_PATH_PGA_MUTE_EN_MASK,
- CDC_VA_TX_PATH_PGA_MUTE_EN);
- else
- snd_soc_component_update_bits(component, tx_vol_ctl_reg,
- CDC_VA_TX_PATH_PGA_MUTE_EN_MASK,
- CDC_VA_TX_PATH_PGA_MUTE_DISABLE);
+ for_each_set_bit(decimator, &va->active_ch_mask[dai->id],
+ VA_MACRO_DEC_MAX) {
+ tx_vol_ctl_reg = CDC_VA_TX0_TX_PATH_CTL +
+ VA_MACRO_TX_PATH_OFFSET * decimator;
+ if (mute)
+ snd_soc_component_update_bits(component, tx_vol_ctl_reg,
+ CDC_VA_TX_PATH_PGA_MUTE_EN_MASK,
+ CDC_VA_TX_PATH_PGA_MUTE_EN);
+ else
+ snd_soc_component_update_bits(component, tx_vol_ctl_reg,
+ CDC_VA_TX_PATH_PGA_MUTE_EN_MASK,
+ CDC_VA_TX_PATH_PGA_MUTE_DISABLE);
+ }
return 0;
}
struct snd_kcontrol *kcontrol, int event)
{
struct snd_soc_component *component = snd_soc_dapm_to_component(w->dapm);
- u16 gain_reg;
+ u16 path_reg, gain_reg;
int val;
- switch (w->reg) {
- case CDC_WSA_RX0_RX_PATH_MIX_CTL:
+ switch (w->shift) {
+ case WSA_MACRO_RX_MIX0:
+ path_reg = CDC_WSA_RX0_RX_PATH_MIX_CTL;
gain_reg = CDC_WSA_RX0_RX_VOL_MIX_CTL;
break;
- case CDC_WSA_RX1_RX_PATH_MIX_CTL:
+ case WSA_MACRO_RX_MIX1:
+ path_reg = CDC_WSA_RX1_RX_PATH_MIX_CTL;
gain_reg = CDC_WSA_RX1_RX_VOL_MIX_CTL;
break;
default:
snd_soc_component_write(component, gain_reg, val);
break;
case SND_SOC_DAPM_POST_PMD:
- snd_soc_component_update_bits(component, w->reg,
+ snd_soc_component_update_bits(component, path_reg,
CDC_WSA_RX_PATH_MIX_CLK_EN_MASK,
CDC_WSA_RX_PATH_MIX_CLK_DISABLE);
break;
SND_SOC_DAPM_MUX("WSA_RX0 INP0", SND_SOC_NOPM, 0, 0, &rx0_prim_inp0_mux),
SND_SOC_DAPM_MUX("WSA_RX0 INP1", SND_SOC_NOPM, 0, 0, &rx0_prim_inp1_mux),
SND_SOC_DAPM_MUX("WSA_RX0 INP2", SND_SOC_NOPM, 0, 0, &rx0_prim_inp2_mux),
- SND_SOC_DAPM_MUX_E("WSA_RX0 MIX INP", CDC_WSA_RX0_RX_PATH_MIX_CTL,
- 0, 0, &rx0_mix_mux, wsa_macro_enable_mix_path,
+ SND_SOC_DAPM_MUX_E("WSA_RX0 MIX INP", SND_SOC_NOPM, WSA_MACRO_RX_MIX0,
+ 0, &rx0_mix_mux, wsa_macro_enable_mix_path,
SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_POST_PMD),
SND_SOC_DAPM_MUX("WSA_RX1 INP0", SND_SOC_NOPM, 0, 0, &rx1_prim_inp0_mux),
SND_SOC_DAPM_MUX("WSA_RX1 INP1", SND_SOC_NOPM, 0, 0, &rx1_prim_inp1_mux),
SND_SOC_DAPM_MUX("WSA_RX1 INP2", SND_SOC_NOPM, 0, 0, &rx1_prim_inp2_mux),
- SND_SOC_DAPM_MUX_E("WSA_RX1 MIX INP", CDC_WSA_RX1_RX_PATH_MIX_CTL,
- 0, 0, &rx1_mix_mux, wsa_macro_enable_mix_path,
+ SND_SOC_DAPM_MUX_E("WSA_RX1 MIX INP", SND_SOC_NOPM, WSA_MACRO_RX_MIX1,
+ 0, &rx1_mix_mux, wsa_macro_enable_mix_path,
SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_POST_PMD),
SND_SOC_DAPM_MIXER_E("WSA_RX INT0 MIX", SND_SOC_NOPM, 0, 0, NULL, 0,
case MAX98373_R2054_MEAS_ADC_PVDD_CH_READBACK:
case MAX98373_R2055_MEAS_ADC_THERM_CH_READBACK:
case MAX98373_R20B6_BDE_CUR_STATE_READBACK:
+ case MAX98373_R20FF_GLOBAL_SHDN:
case MAX98373_R21FF_REV_ID:
return true;
default:
case MAX98373_R2054_MEAS_ADC_PVDD_CH_READBACK:
case MAX98373_R2055_MEAS_ADC_THERM_CH_READBACK:
case MAX98373_R20B6_BDE_CUR_STATE_READBACK:
+ case MAX98373_R20FF_GLOBAL_SHDN:
case MAX98373_R21FF_REV_ID:
/* SoundWire Control Port Registers */
case MAX98373_R0040_SCP_INIT_STAT_1 ... MAX98373_R0070_SCP_FRAME_CTLR:
regmap_update_bits(max98373->regmap,
MAX98373_R20FF_GLOBAL_SHDN,
MAX98373_GLOBAL_EN_MASK, 1);
+ usleep_range(30000, 31000);
break;
case SND_SOC_DAPM_POST_PMD:
regmap_update_bits(max98373->regmap,
MAX98373_R20FF_GLOBAL_SHDN,
MAX98373_GLOBAL_EN_MASK, 0);
+ usleep_range(30000, 31000);
max98373->tdm_mode = false;
break;
default:
case RT1015_VENDOR_ID:
case RT1015_DEVICE_ID:
case RT1015_PRO_ALT:
+ case RT1015_MAN_I2C:
case RT1015_DAC3:
case RT1015_VBAT_TEST_OUT1:
case RT1015_VBAT_TEST_OUT2:
msleep(300);
regmap_write(regmap, RT1015_PWR_STATE_CTRL, 0x0008);
regmap_write(regmap, RT1015_SYS_RST1, 0x05F5);
+ regmap_write(regmap, RT1015_CLK_DET, 0x8000);
regcache_cache_bypass(regmap, false);
regcache_mark_dirty(regmap);
}
static const DECLARE_TLV_DB_SCALE(out_vol_tlv, -4650, 150, 0);
-static const DECLARE_TLV_DB_SCALE(dac_vol_tlv, -65625, 375, 0);
+static const DECLARE_TLV_DB_MINMAX(dac_vol_tlv, -6562, 0);
static const DECLARE_TLV_DB_SCALE(in_vol_tlv, -3450, 150, 0);
-static const DECLARE_TLV_DB_SCALE(adc_vol_tlv, -17625, 375, 0);
+static const DECLARE_TLV_DB_MINMAX(adc_vol_tlv, -1762, 3000);
static const DECLARE_TLV_DB_SCALE(adc_bst_tlv, 0, 1200, 0);
/* {0, +20, +24, +30, +35, +40, +44, +50, +52} dB */
}
static const DECLARE_TLV_DB_SCALE(out_vol_tlv, -4650, 150, 0);
-static const DECLARE_TLV_DB_SCALE(dac_vol_tlv, -65625, 375, 0);
+static const DECLARE_TLV_DB_MINMAX(dac_vol_tlv, -6562, 0);
static const DECLARE_TLV_DB_SCALE(in_vol_tlv, -3450, 150, 0);
-static const DECLARE_TLV_DB_SCALE(adc_vol_tlv, -17625, 375, 0);
+static const DECLARE_TLV_DB_MINMAX(adc_vol_tlv, -1762, 3000);
static const DECLARE_TLV_DB_SCALE(adc_bst_tlv, 0, 1200, 0);
/* {0, +20, +24, +30, +35, +40, +44, +50, +52} dB */
{
struct rt5659_priv *rt5659 = snd_soc_component_get_drvdata(component);
unsigned int reg_val = 0;
+ int ret;
if (freq == rt5659->sysclk && clk_id == rt5659->sysclk_src)
return 0;
switch (clk_id) {
case RT5659_SCLK_S_MCLK:
+ ret = clk_set_rate(rt5659->mclk, freq);
+ if (ret)
+ return ret;
+
reg_val |= RT5659_SCLK_SRC_MCLK;
break;
case RT5659_SCLK_S_PLL1:
static SOC_ENUM_SINGLE_DECL(rt5670_if2_adc_enum, RT5670_DIG_INF1_DATA,
RT5670_IF2_ADC_SEL_SFT, rt5670_data_select);
+/*
+ * For reliable output-mute LED control we need a "DAC1 Playback Switch" control.
+ * We emulate this by only clearing the RT5670_M_DAC1_L/_R AD_DA_MIXER register
+ * bits when both our emulated DAC1 Playback Switch control and the DAC1 MIXL/R
+ * DAPM-mixer DAC1 input are enabled.
+ */
+static void rt5670_update_ad_da_mixer_dac1_m_bits(struct rt5670_priv *rt5670)
+{
+ int val = RT5670_M_DAC1_L | RT5670_M_DAC1_R;
+
+ if (rt5670->dac1_mixl_dac1_switch && rt5670->dac1_playback_switch_l)
+ val &= ~RT5670_M_DAC1_L;
+
+ if (rt5670->dac1_mixr_dac1_switch && rt5670->dac1_playback_switch_r)
+ val &= ~RT5670_M_DAC1_R;
+
+ regmap_update_bits(rt5670->regmap, RT5670_AD_DA_MIXER,
+ RT5670_M_DAC1_L | RT5670_M_DAC1_R, val);
+}
+
+static int rt5670_dac1_playback_switch_get(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_value *ucontrol)
+{
+ struct snd_soc_component *component = snd_soc_kcontrol_component(kcontrol);
+ struct rt5670_priv *rt5670 = snd_soc_component_get_drvdata(component);
+
+ ucontrol->value.integer.value[0] = rt5670->dac1_playback_switch_l;
+ ucontrol->value.integer.value[1] = rt5670->dac1_playback_switch_r;
+
+ return 0;
+}
+
+static int rt5670_dac1_playback_switch_put(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_value *ucontrol)
+{
+ struct snd_soc_component *component = snd_soc_kcontrol_component(kcontrol);
+ struct rt5670_priv *rt5670 = snd_soc_component_get_drvdata(component);
+
+ if (rt5670->dac1_playback_switch_l == ucontrol->value.integer.value[0] &&
+ rt5670->dac1_playback_switch_r == ucontrol->value.integer.value[1])
+ return 0;
+
+ rt5670->dac1_playback_switch_l = ucontrol->value.integer.value[0];
+ rt5670->dac1_playback_switch_r = ucontrol->value.integer.value[1];
+
+ rt5670_update_ad_da_mixer_dac1_m_bits(rt5670);
+
+ return 1;
+}
+
static const struct snd_kcontrol_new rt5670_snd_controls[] = {
/* Headphone Output Volume */
- SOC_DOUBLE("HP Playback Switch", RT5670_HP_VOL,
- RT5670_L_MUTE_SFT, RT5670_R_MUTE_SFT, 1, 1),
SOC_DOUBLE_TLV("HP Playback Volume", RT5670_HP_VOL,
RT5670_L_VOL_SFT, RT5670_R_VOL_SFT,
39, 1, out_vol_tlv),
/* OUTPUT Control */
- SOC_DOUBLE("OUT Channel Switch", RT5670_LOUT1,
- RT5670_VOL_L_SFT, RT5670_VOL_R_SFT, 1, 1),
SOC_DOUBLE_TLV("OUT Playback Volume", RT5670_LOUT1,
RT5670_L_VOL_SFT, RT5670_R_VOL_SFT, 39, 1, out_vol_tlv),
/* DAC Digital Volume */
SOC_DOUBLE("DAC2 Playback Switch", RT5670_DAC_CTRL,
RT5670_M_DAC_L2_VOL_SFT, RT5670_M_DAC_R2_VOL_SFT, 1, 1),
+ SOC_DOUBLE_EXT("DAC1 Playback Switch", SND_SOC_NOPM, 0, 1, 1, 0,
+ rt5670_dac1_playback_switch_get, rt5670_dac1_playback_switch_put),
SOC_DOUBLE_TLV("DAC1 Playback Volume", RT5670_DAC1_DIG_VOL,
RT5670_L_VOL_SFT, RT5670_R_VOL_SFT,
175, 0, dac_vol_tlv),
RT5670_M_MONO_ADC_R2_SFT, 1, 1),
};
+/* See comment above rt5670_update_ad_da_mixer_dac1_m_bits() */
+static int rt5670_put_dac1_mix_dac1_switch(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_value *ucontrol)
+{
+ struct soc_mixer_control *mc = (struct soc_mixer_control *)kcontrol->private_value;
+ struct snd_soc_component *component = snd_soc_dapm_kcontrol_component(kcontrol);
+ struct rt5670_priv *rt5670 = snd_soc_component_get_drvdata(component);
+ int ret;
+
+ if (mc->shift == 0)
+ rt5670->dac1_mixl_dac1_switch = ucontrol->value.integer.value[0];
+ else
+ rt5670->dac1_mixr_dac1_switch = ucontrol->value.integer.value[0];
+
+ /* Apply the update (if any) */
+ ret = snd_soc_dapm_put_volsw(kcontrol, ucontrol);
+ if (ret == 0)
+ return 0;
+
+ rt5670_update_ad_da_mixer_dac1_m_bits(rt5670);
+
+ return 1;
+}
+
+#define SOC_DAPM_SINGLE_RT5670_DAC1_SW(name, shift) \
+ SOC_SINGLE_EXT(name, SND_SOC_NOPM, shift, 1, 0, \
+ snd_soc_dapm_get_volsw, rt5670_put_dac1_mix_dac1_switch)
+
static const struct snd_kcontrol_new rt5670_dac_l_mix[] = {
SOC_DAPM_SINGLE("Stereo ADC Switch", RT5670_AD_DA_MIXER,
RT5670_M_ADCMIX_L_SFT, 1, 1),
- SOC_DAPM_SINGLE("DAC1 Switch", RT5670_AD_DA_MIXER,
- RT5670_M_DAC1_L_SFT, 1, 1),
+ SOC_DAPM_SINGLE_RT5670_DAC1_SW("DAC1 Switch", 0),
};
static const struct snd_kcontrol_new rt5670_dac_r_mix[] = {
SOC_DAPM_SINGLE("Stereo ADC Switch", RT5670_AD_DA_MIXER,
RT5670_M_ADCMIX_R_SFT, 1, 1),
- SOC_DAPM_SINGLE("DAC1 Switch", RT5670_AD_DA_MIXER,
- RT5670_M_DAC1_R_SFT, 1, 1),
+ SOC_DAPM_SINGLE_RT5670_DAC1_SW("DAC1 Switch", 1),
};
static const struct snd_kcontrol_new rt5670_sto_dac_l_mix[] = {
RT5670_PWR_ADC_S1F_BIT, 0, NULL, 0),
SND_SOC_DAPM_SUPPLY("ADC Stereo2 Filter", RT5670_PWR_DIG2,
RT5670_PWR_ADC_S2F_BIT, 0, NULL, 0),
- SND_SOC_DAPM_MIXER("Sto1 ADC MIXL", RT5670_STO1_ADC_DIG_VOL,
- RT5670_L_MUTE_SFT, 1, rt5670_sto1_adc_l_mix,
- ARRAY_SIZE(rt5670_sto1_adc_l_mix)),
- SND_SOC_DAPM_MIXER("Sto1 ADC MIXR", RT5670_STO1_ADC_DIG_VOL,
- RT5670_R_MUTE_SFT, 1, rt5670_sto1_adc_r_mix,
- ARRAY_SIZE(rt5670_sto1_adc_r_mix)),
+ SND_SOC_DAPM_MIXER("Sto1 ADC MIXL", SND_SOC_NOPM, 0, 0,
+ rt5670_sto1_adc_l_mix, ARRAY_SIZE(rt5670_sto1_adc_l_mix)),
+ SND_SOC_DAPM_MIXER("Sto1 ADC MIXR", SND_SOC_NOPM, 0, 0,
+ rt5670_sto1_adc_r_mix, ARRAY_SIZE(rt5670_sto1_adc_r_mix)),
SND_SOC_DAPM_MIXER("Sto2 ADC MIXL", SND_SOC_NOPM, 0, 0,
rt5670_sto2_adc_l_mix,
ARRAY_SIZE(rt5670_sto2_adc_l_mix)),
dev_info(&i2c->dev, "quirk JD mode 3\n");
}
+ /*
+ * Enable the emulated "DAC1 Playback Switch" by default to avoid
+ * muting the output with older UCM profiles.
+ */
+ rt5670->dac1_playback_switch_l = true;
+ rt5670->dac1_playback_switch_r = true;
+ /* The Power-On-Reset values for the DAC1 mixer have the DAC1 input enabled. */
+ rt5670->dac1_mixl_dac1_switch = true;
+ rt5670->dac1_mixr_dac1_switch = true;
+
rt5670->regmap = devm_regmap_init_i2c(i2c, &rt5670_regmap);
if (IS_ERR(rt5670->regmap)) {
ret = PTR_ERR(rt5670->regmap);
/* global definition */
#define RT5670_L_MUTE (0x1 << 15)
#define RT5670_L_MUTE_SFT 15
-#define RT5670_VOL_L_MUTE (0x1 << 14)
-#define RT5670_VOL_L_SFT 14
#define RT5670_R_MUTE (0x1 << 7)
#define RT5670_R_MUTE_SFT 7
-#define RT5670_VOL_R_MUTE (0x1 << 6)
-#define RT5670_VOL_R_SFT 6
#define RT5670_L_VOL_MASK (0x3f << 8)
#define RT5670_L_VOL_SFT 8
#define RT5670_R_VOL_MASK (0x3f)
int dsp_rate;
int jack_type;
int jack_type_saved;
+
+ bool dac1_mixl_dac1_switch;
+ bool dac1_mixr_dac1_switch;
+ bool dac1_playback_switch_l;
+ bool dac1_playback_switch_r;
};
void rt5670_jack_suspend(struct snd_soc_component *component);
return 0;
}
+static void rt711_remove(struct snd_soc_component *component)
+{
+ struct rt711_priv *rt711 = snd_soc_component_get_drvdata(component);
+
+ regcache_cache_only(rt711->regmap, true);
+}
+
static const struct snd_soc_component_driver soc_codec_dev_rt711 = {
.probe = rt711_probe,
.set_bias_level = rt711_set_bias_level,
.dapm_routes = rt711_audio_map,
.num_dapm_routes = ARRAY_SIZE(rt711_audio_map),
.set_jack = rt711_set_jack_detect,
+ .remove = rt711_remove,
};
static int rt711_set_sdw_stream(struct snd_soc_dai *dai, void *sdw_stream,
{ SGTL5000_DAP_EQ_BASS_BAND4, 0x002f },
{ SGTL5000_DAP_MAIN_CHAN, 0x8000 },
{ SGTL5000_DAP_MIX_CHAN, 0x0000 },
- { SGTL5000_DAP_AVC_CTRL, 0x0510 },
+ { SGTL5000_DAP_AVC_CTRL, 0x5100 },
{ SGTL5000_DAP_AVC_THRESHOLD, 0x1473 },
{ SGTL5000_DAP_AVC_ATTACK, 0x0028 },
{ SGTL5000_DAP_AVC_DECAY, 0x0050 },
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * SiRF inner codec controllers define
- *
- * Copyright (c) 2011 Cambridge Silicon Radio Limited, a CSR plc group company.
- */
-
-#ifndef _SIRF_AUDIO_CODEC_H
-#define _SIRF_AUDIO_CODEC_H
-
-
-#define AUDIO_IC_CODEC_PWR (0x00E0)
-#define AUDIO_IC_CODEC_CTRL0 (0x00E4)
-#define AUDIO_IC_CODEC_CTRL1 (0x00E8)
-#define AUDIO_IC_CODEC_CTRL2 (0x00EC)
-#define AUDIO_IC_CODEC_CTRL3 (0x00F0)
-
-#define MICBIASEN (1 << 3)
-
-#define IC_RDACEN (1 << 0)
-#define IC_LDACEN (1 << 1)
-#define IC_HSREN (1 << 2)
-#define IC_HSLEN (1 << 3)
-#define IC_SPEN (1 << 4)
-#define IC_CPEN (1 << 5)
-
-#define IC_HPRSELR (1 << 6)
-#define IC_HPLSELR (1 << 7)
-#define IC_HPRSELL (1 << 8)
-#define IC_HPLSELL (1 << 9)
-#define IC_SPSELR (1 << 10)
-#define IC_SPSELL (1 << 11)
-
-#define IC_MONOR (1 << 12)
-#define IC_MONOL (1 << 13)
-
-#define IC_RXOSRSEL (1 << 28)
-#define IC_CPFREQ (1 << 29)
-#define IC_HSINVEN (1 << 30)
-
-#define IC_MICINREN (1 << 0)
-#define IC_MICINLEN (1 << 1)
-#define IC_MICIN1SEL (1 << 2)
-#define IC_MICIN2SEL (1 << 3)
-#define IC_MICDIFSEL (1 << 4)
-#define IC_LINEIN1SEL (1 << 5)
-#define IC_LINEIN2SEL (1 << 6)
-#define IC_RADCEN (1 << 7)
-#define IC_LADCEN (1 << 8)
-#define IC_ALM (1 << 9)
-
-#define IC_DIGMICEN (1 << 22)
-#define IC_DIGMICFREQ (1 << 23)
-#define IC_ADC14B_12 (1 << 24)
-#define IC_FIRDAC_HSL_EN (1 << 25)
-#define IC_FIRDAC_HSR_EN (1 << 26)
-#define IC_FIRDAC_LOUT_EN (1 << 27)
-#define IC_POR (1 << 28)
-#define IC_CODEC_CLK_EN (1 << 29)
-#define IC_HP_3DB_BOOST (1 << 30)
-
-#define IC_ADC_LEFT_GAIN_SHIFT 16
-#define IC_ADC_RIGHT_GAIN_SHIFT 10
-#define IC_ADC_GAIN_MASK 0x3F
-#define IC_MIC_MAX_GAIN 0x39
-
-#define IC_RXPGAR_MASK 0x3F
-#define IC_RXPGAR_SHIFT 14
-#define IC_RXPGAL_MASK 0x3F
-#define IC_RXPGAL_SHIFT 21
-#define IC_RXPGAR 0x7B
-#define IC_RXPGAL 0x7B
-
-#define AUDIO_PORT_TX_FIFO_LEVEL_CHECK_MASK 0x3F
-#define AUDIO_PORT_TX_FIFO_SC_OFFSET 0
-#define AUDIO_PORT_TX_FIFO_LC_OFFSET 10
-#define AUDIO_PORT_TX_FIFO_HC_OFFSET 20
-
-#define TX_FIFO_SC(x) (((x) & AUDIO_PORT_TX_FIFO_LEVEL_CHECK_MASK) \
- << AUDIO_PORT_TX_FIFO_SC_OFFSET)
-#define TX_FIFO_LC(x) (((x) & AUDIO_PORT_TX_FIFO_LEVEL_CHECK_MASK) \
- << AUDIO_PORT_TX_FIFO_LC_OFFSET)
-#define TX_FIFO_HC(x) (((x) & AUDIO_PORT_TX_FIFO_LEVEL_CHECK_MASK) \
- << AUDIO_PORT_TX_FIFO_HC_OFFSET)
-
-#define AUDIO_PORT_RX_FIFO_LEVEL_CHECK_MASK 0x0F
-#define AUDIO_PORT_RX_FIFO_SC_OFFSET 0
-#define AUDIO_PORT_RX_FIFO_LC_OFFSET 10
-#define AUDIO_PORT_RX_FIFO_HC_OFFSET 20
-
-#define RX_FIFO_SC(x) (((x) & AUDIO_PORT_RX_FIFO_LEVEL_CHECK_MASK) \
- << AUDIO_PORT_RX_FIFO_SC_OFFSET)
-#define RX_FIFO_LC(x) (((x) & AUDIO_PORT_RX_FIFO_LEVEL_CHECK_MASK) \
- << AUDIO_PORT_RX_FIFO_LC_OFFSET)
-#define RX_FIFO_HC(x) (((x) & AUDIO_PORT_RX_FIFO_LEVEL_CHECK_MASK) \
- << AUDIO_PORT_RX_FIFO_HC_OFFSET)
-#define AUDIO_PORT_IC_CODEC_TX_CTRL (0x00F4)
-#define AUDIO_PORT_IC_CODEC_RX_CTRL (0x00F8)
-
-#define AUDIO_PORT_IC_TXFIFO_OP (0x00FC)
-#define AUDIO_PORT_IC_TXFIFO_LEV_CHK (0x0100)
-#define AUDIO_PORT_IC_TXFIFO_STS (0x0104)
-#define AUDIO_PORT_IC_TXFIFO_INT (0x0108)
-#define AUDIO_PORT_IC_TXFIFO_INT_MSK (0x010C)
-
-#define AUDIO_PORT_IC_RXFIFO_OP (0x0110)
-#define AUDIO_PORT_IC_RXFIFO_LEV_CHK (0x0114)
-#define AUDIO_PORT_IC_RXFIFO_STS (0x0118)
-#define AUDIO_PORT_IC_RXFIFO_INT (0x011C)
-#define AUDIO_PORT_IC_RXFIFO_INT_MSK (0x0120)
-
-#define AUDIO_FIFO_START (1 << 0)
-#define AUDIO_FIFO_RESET (1 << 1)
-
-#define AUDIO_FIFO_FULL (1 << 0)
-#define AUDIO_FIFO_EMPTY (1 << 1)
-#define AUDIO_FIFO_OFLOW (1 << 2)
-#define AUDIO_FIFO_UFLOW (1 << 3)
-
-#define IC_TX_ENABLE (0x03)
-#define IC_RX_ENABLE_MONO (0x01)
-#define IC_RX_ENABLE_STEREO (0x03)
-
-#endif /*__SIRF_AUDIO_CODEC_H*/
wcd = snd_soc_component_get_drvdata(dai->component);
+ if (tx_num > WCD934X_TX_MAX || rx_num > WCD934X_RX_MAX) {
+ dev_err(wcd->dev, "Invalid tx %d or rx %d channel count\n",
+ tx_num, rx_num);
+ return -EINVAL;
+ }
+
if (!tx_slot || !rx_slot) {
dev_err(wcd->dev, "Invalid tx_slot=%p, rx_slot=%p\n",
tx_slot, rx_slot);
best_freq_out = -EINVAL;
*sysclk_idx = *dac_idx = *bclk_idx = -1;
- for (i = 0; i < ARRAY_SIZE(sysclk_divs); ++i) {
+ /*
+ * From Datasheet, the PLL performs best when f2 is between
+ * 90MHz and 100MHz, the desired sysclk output is 11.2896MHz
+ * or 12.288MHz, then sysclkdiv = 2 is the best choice.
+ * So search sysclk_divs from 2 to 1 other than from 1 to 2.
+ */
+ for (i = ARRAY_SIZE(sysclk_divs) - 1; i >= 0; --i) {
if (sysclk_divs[i] == -1)
continue;
for (j = 0; j < ARRAY_SIZE(dac_divs); ++j) {
ESAI_SAICR_SYNC, esai_priv->synchronous ?
ESAI_SAICR_SYNC : 0);
- /* Set a default slot number -- 2 */
+ /* Set slots count */
regmap_update_bits(esai_priv->regmap, REG_ESAI_TCCR,
- ESAI_xCCR_xDC_MASK, ESAI_xCCR_xDC(2));
+ ESAI_xCCR_xDC_MASK,
+ ESAI_xCCR_xDC(esai_priv->slots));
regmap_update_bits(esai_priv->regmap, REG_ESAI_RCCR,
- ESAI_xCCR_xDC_MASK, ESAI_xCCR_xDC(2));
+ ESAI_xCCR_xDC_MASK,
+ ESAI_xCCR_xDC(esai_priv->slots));
}
return 0;
static int _fsl_ssi_set_dai_fmt(struct fsl_ssi *ssi, unsigned int fmt)
{
u32 strcr = 0, scr = 0, stcr, srcr, mask;
+ unsigned int slots;
ssi->dai_fmt = fmt;
return -EINVAL;
}
+ slots = ssi->slots ? : 2;
regmap_update_bits(ssi->regs, REG_SSI_STCCR,
- SSI_SxCCR_DC_MASK, SSI_SxCCR_DC(2));
+ SSI_SxCCR_DC_MASK, SSI_SxCCR_DC(slots));
regmap_update_bits(ssi->regs, REG_SSI_SRCCR,
- SSI_SxCCR_DC_MASK, SSI_SxCCR_DC(2));
+ SSI_SxCCR_DC_MASK, SSI_SxCCR_DC(slots));
/* Data on rising edge of bclk, frame low, 1clk before data */
strcr |= SSI_STCR_TFSI | SSI_STCR_TSCKP | SSI_STCR_TEFS;
* or device's module clock.
*/
clk = devm_get_clk_from_child(dev, node, NULL);
- if (IS_ERR(clk))
- clk = devm_get_clk_from_child(dev, dlc->of_node, NULL);
-
if (!IS_ERR(clk)) {
- simple_dai->clk = clk;
simple_dai->sysclk = clk_get_rate(clk);
- } else if (!of_property_read_u32(node, "system-clock-frequency",
- &val)) {
+
+ simple_dai->clk = clk;
+ } else if (!of_property_read_u32(node, "system-clock-frequency", &val)) {
simple_dai->sysclk = val;
+ } else {
+ clk = devm_get_clk_from_child(dev, dlc->of_node, NULL);
+ if (!IS_ERR(clk))
+ simple_dai->sysclk = clk_get_rate(clk);
}
if (of_property_read_bool(node, "system-clock-direction-out"))
.stream_name = "Headset Playback",
.channels_min = SST_STEREO,
.channels_max = SST_STEREO,
- .rates = SNDRV_PCM_RATE_44100|SNDRV_PCM_RATE_48000,
- .formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
},
.capture = {
.stream_name = "Headset Capture",
.channels_min = 1,
.channels_max = 2,
- .rates = SNDRV_PCM_RATE_44100|SNDRV_PCM_RATE_48000,
- .formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
},
},
{
.stream_name = "Deepbuffer Playback",
.channels_min = SST_STEREO,
.channels_max = SST_STEREO,
- .rates = SNDRV_PCM_RATE_44100|SNDRV_PCM_RATE_48000,
- .formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
},
},
{
},
.driver_data = (void *)(BYT_RT5640_DMIC1_MAP |
BYT_RT5640_JD_SRC_JD1_IN4P |
- BYT_RT5640_OVCD_TH_1500UA |
+ BYT_RT5640_OVCD_TH_2000UA |
BYT_RT5640_OVCD_SF_0P75 |
BYT_RT5640_MCLK_EN),
},
/* set tdm */
if (tdm_priv->bck_invert)
- tdm_con |= 1 << BCK_INVERSE_SFT;
+ regmap_update_bits(afe->regmap, AUDIO_TOP_CON3,
+ BCK_INVERSE_MASK_SFT,
+ 0x1 << BCK_INVERSE_SFT);
if (tdm_priv->lck_invert)
tdm_con |= 1 << LRCK_INVERSE_SFT;
/*****************************************************************************
* R E G I S T E R D E F I N I T I O N
*****************************************************************************/
+/* AUDIO_TOP_CON3 */
+#define BCK_INVERSE_SFT 3
+#define BCK_INVERSE_MASK 0x1
+#define BCK_INVERSE_MASK_SFT (0x1 << 3)
+
/* AFE_DAC_CON0 */
#define VUL12_ON_SFT 31
#define VUL12_ON_MASK 0x1
#define TDM_EN_SFT 0
#define TDM_EN_MASK 0x1
#define TDM_EN_MASK_SFT (0x1 << 0)
-#define BCK_INVERSE_SFT 1
-#define BCK_INVERSE_MASK 0x1
-#define BCK_INVERSE_MASK_SFT (0x1 << 1)
#define LRCK_INVERSE_SFT 2
#define LRCK_INVERSE_MASK 0x1
#define LRCK_INVERSE_MASK_SFT (0x1 << 2)
for_each_child_of_node(dev->of_node, node) {
ret = of_property_read_u32(node, "reg", &id);
- if (ret || id < 0 || id >= data->variant->num_dai) {
+ if (ret || id < 0) {
dev_err(dev, "valid dai id not found: %d\n", ret);
continue;
}
#define SPK_TDM_RX_MASK 0x03
#define NUM_TDM_SLOTS 8
#define SLIM_MAX_TX_PORTS 16
-#define SLIM_MAX_RX_PORTS 16
+#define SLIM_MAX_RX_PORTS 13
#define WCD934X_DEFAULT_MCLK_RATE 9600000
struct sdm845_snd_data {
struct snd_soc_jack jack;
bool jack_setup;
- bool stream_prepared[SLIM_MAX_RX_PORTS];
+ bool stream_prepared[AFE_PORT_MAX];
struct snd_soc_card *card;
uint32_t pri_mi2s_clk_count;
uint32_t sec_mi2s_clk_count;
uint32_t quat_tdm_clk_count;
- struct sdw_stream_runtime *sruntime[SLIM_MAX_RX_PORTS];
+ struct sdw_stream_runtime *sruntime[AFE_PORT_MAX];
};
static unsigned int tdm_slot_offset[8] = {0, 4, 8, 12, 16, 20, 24, 28};
#include <linux/of.h>
#include <linux/of_graph.h>
#include <linux/dmi.h>
+#include <linux/acpi.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
if (card->long_name)
return 0; /* long name already set by driver or from DMI */
+ if (!is_acpi_device_node(card->dev->fwnode))
+ return 0;
+
/* make up dmi long name as: vendor-product-version-board */
vendor = dmi_get_system_info(DMI_BOARD_VENDOR);
if (!vendor || !is_dmi_valid(vendor)) {
{
struct snd_sof_dev *sdev = dev_get_drvdata(dev);
- return snd_sof_shutdown(sdev);
+ if (IS_ENABLED(CONFIG_SND_SOC_SOF_PROBE_WORK_QUEUE))
+ cancel_work_sync(&sdev->probe_work);
+
+ if (sdev->fw_state == SOF_FW_BOOT_COMPLETE)
+ return snd_sof_shutdown(sdev);
+
+ return 0;
}
EXPORT_SYMBOL(snd_sof_device_shutdown);
/* apollolake ops */
const struct snd_sof_dsp_ops sof_apl_ops = {
- /* probe and remove */
+ /* probe/remove/shutdown */
.probe = hda_dsp_probe,
.remove = hda_dsp_remove,
+ .shutdown = hda_dsp_shutdown,
/* Register IO */
.write = sof_io_write,
/* cannonlake ops */
const struct snd_sof_dsp_ops sof_cnl_ops = {
- /* probe and remove */
+ /* probe/remove/shutdown */
.probe = hda_dsp_probe,
.remove = hda_dsp_remove,
+ .shutdown = hda_dsp_shutdown,
/* Register IO */
.write = sof_io_write,
};
EXPORT_SYMBOL_NS(cnl_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON);
-const struct sof_intel_dsp_desc ehl_chip_info = {
- /* Elkhartlake */
- .cores_num = 4,
- .init_core_mask = 1,
- .host_managed_cores_mask = BIT(0),
- .ipc_req = CNL_DSP_REG_HIPCIDR,
- .ipc_req_mask = CNL_DSP_REG_HIPCIDR_BUSY,
- .ipc_ack = CNL_DSP_REG_HIPCIDA,
- .ipc_ack_mask = CNL_DSP_REG_HIPCIDA_DONE,
- .ipc_ctl = CNL_DSP_REG_HIPCCTL,
- .rom_init_timeout = 300,
- .ssp_count = ICL_SSP_COUNT,
- .ssp_base_offset = CNL_SSP_BASE_OFFSET,
-};
-EXPORT_SYMBOL_NS(ehl_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON);
-
const struct sof_intel_dsp_desc jsl_chip_info = {
/* Jasperlake */
.cores_num = 2,
ret = snd_sof_dsp_read_poll_timeout(sdev, HDA_DSP_BAR,
HDA_DSP_REG_ADSPCS, adspcs,
- !(adspcs & HDA_DSP_ADSPCS_SPA_MASK(core_mask)),
+ !(adspcs & HDA_DSP_ADSPCS_CPA_MASK(core_mask)),
HDA_DSP_REG_POLL_INTERVAL_US,
HDA_DSP_PD_TIMEOUT * USEC_PER_MSEC);
if (ret < 0)
val = snd_sof_dsp_read(sdev, HDA_DSP_BAR, HDA_DSP_REG_ADSPCS);
- is_enable = (val & HDA_DSP_ADSPCS_CPA_MASK(core_mask)) &&
- (val & HDA_DSP_ADSPCS_SPA_MASK(core_mask)) &&
- !(val & HDA_DSP_ADSPCS_CRST_MASK(core_mask)) &&
- !(val & HDA_DSP_ADSPCS_CSTALL_MASK(core_mask));
+#define MASK_IS_EQUAL(v, m, field) ({ \
+ u32 _m = field(m); \
+ ((v) & _m) == _m; \
+})
+
+ is_enable = MASK_IS_EQUAL(val, core_mask, HDA_DSP_ADSPCS_CPA_MASK) &&
+ MASK_IS_EQUAL(val, core_mask, HDA_DSP_ADSPCS_SPA_MASK) &&
+ !(val & HDA_DSP_ADSPCS_CRST_MASK(core_mask)) &&
+ !(val & HDA_DSP_ADSPCS_CSTALL_MASK(core_mask));
+
+#undef MASK_IS_EQUAL
dev_dbg(sdev->dev, "DSP core(s) enabled? %d : core_mask %x\n",
is_enable, core_mask);
return snd_sof_dsp_set_power_state(sdev, &target_dsp_state);
}
+int hda_dsp_shutdown(struct snd_sof_dev *sdev)
+{
+ sdev->system_suspend_target = SOF_SUSPEND_S3;
+ return snd_sof_suspend(sdev->dev);
+}
+
int hda_dsp_set_hw_params_upon_resume(struct snd_sof_dev *sdev)
{
#if IS_ENABLED(CONFIG_SND_SOC_SOF_HDA)
/* dsp_unmap: not currently used */
iounmap(sdev->bar[HDA_DSP_BAR]);
hdac_bus_unmap:
+ platform_device_unregister(hdev->dmic_dev);
iounmap(bus->remap_addr);
hda_codec_i915_exit(sdev);
err:
int hda_dsp_runtime_suspend(struct snd_sof_dev *sdev);
int hda_dsp_runtime_resume(struct snd_sof_dev *sdev);
int hda_dsp_runtime_idle(struct snd_sof_dev *sdev);
+int hda_dsp_shutdown(struct snd_sof_dev *sdev);
int hda_dsp_set_hw_params_upon_resume(struct snd_sof_dev *sdev);
void hda_dsp_dump(struct snd_sof_dev *sdev, u32 flags);
void hda_ipc_dump(struct snd_sof_dev *sdev);
/* Icelake ops */
const struct snd_sof_dsp_ops sof_icl_ops = {
- /* probe and remove */
+ /* probe/remove/shutdown */
.probe = hda_dsp_probe,
.remove = hda_dsp_remove,
+ .shutdown = hda_dsp_shutdown,
/* Register IO */
.write = sof_io_write,
.default_tplg_path = "intel/sof-tplg",
.default_fw_filename = "sof-ehl.ri",
.nocodec_tplg_filename = "sof-ehl-nocodec.tplg",
- .ops = &sof_cnl_ops,
+ .ops = &sof_tgl_ops,
};
static const struct sof_dev_desc adls_desc = {
/* probe/remove/shutdown */
.probe = hda_dsp_probe,
.remove = hda_dsp_remove,
- .shutdown = hda_dsp_remove,
+ .shutdown = hda_dsp_shutdown,
/* Register IO */
.write = sof_io_write,
};
EXPORT_SYMBOL_NS(tglh_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON);
+const struct sof_intel_dsp_desc ehl_chip_info = {
+ /* Elkhartlake */
+ .cores_num = 4,
+ .init_core_mask = 1,
+ .host_managed_cores_mask = BIT(0),
+ .ipc_req = CNL_DSP_REG_HIPCIDR,
+ .ipc_req_mask = CNL_DSP_REG_HIPCIDR_BUSY,
+ .ipc_ack = CNL_DSP_REG_HIPCIDA,
+ .ipc_ack_mask = CNL_DSP_REG_HIPCIDA_DONE,
+ .ipc_ctl = CNL_DSP_REG_HIPCCTL,
+ .rom_init_timeout = 300,
+ .ssp_count = ICL_SSP_COUNT,
+ .ssp_base_offset = CNL_SSP_BASE_OFFSET,
+};
+EXPORT_SYMBOL_NS(ehl_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON);
+
const struct sof_intel_dsp_desc adls_chip_info = {
/* Alderlake-S */
.cores_num = 2,
return ERR_PTR(-ENOMEM);
card->dev = dev;
+ card->owner = THIS_MODULE;
card->name = "sun4i-codec";
card->dapm_widgets = sun4i_codec_card_dapm_widgets;
card->num_dapm_widgets = ARRAY_SIZE(sun4i_codec_card_dapm_widgets);
return ERR_PTR(-ENOMEM);
card->dev = dev;
+ card->owner = THIS_MODULE;
card->name = "A31 Audio Codec";
card->dapm_widgets = sun6i_codec_card_dapm_widgets;
card->num_dapm_widgets = ARRAY_SIZE(sun6i_codec_card_dapm_widgets);
return ERR_PTR(-ENOMEM);
card->dev = dev;
+ card->owner = THIS_MODULE;
card->name = "A23 Audio Codec";
card->dapm_widgets = sun6i_codec_card_dapm_widgets;
card->num_dapm_widgets = ARRAY_SIZE(sun6i_codec_card_dapm_widgets);
return ERR_PTR(-ENOMEM);
card->dev = dev;
+ card->owner = THIS_MODULE;
card->name = "H3 Audio Codec";
card->dapm_widgets = sun6i_codec_card_dapm_widgets;
card->num_dapm_widgets = ARRAY_SIZE(sun6i_codec_card_dapm_widgets);
return ERR_PTR(-ENOMEM);
card->dev = dev;
+ card->owner = THIS_MODULE;
card->name = "V3s Audio Codec";
card->dapm_widgets = sun6i_codec_card_dapm_widgets;
card->num_dapm_widgets = ARRAY_SIZE(sun6i_codec_card_dapm_widgets);
MODULE_AUTHOR("Thomas K. Dyas and David S. Miller");
MODULE_DESCRIPTION("Sun AMD7930");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Sun,AMD7930}}");
/* Device register layout. */
MODULE_AUTHOR("Jaroslav Kysela, Derrick J. Brashear and David S. Miller");
MODULE_DESCRIPTION("Sun CS4231");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Sun,CS4231}}");
#ifdef SBUS_SUPPORT
struct sbus_dma_info {
MODULE_AUTHOR("Rudolf Koenig, Brent Baccala and Martin Habets");
MODULE_DESCRIPTION("Sun DBRI");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Sun,DBRI}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Torsten Schenk <torsten.schenk@zoho.com>");
MODULE_DESCRIPTION("TerraTec DMX 6Fire USB audio driver");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{TerraTec,DMX 6Fire USB}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-max */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* Id for card */
MODULE_AUTHOR("Daniel Mack <daniel@caiaq.de>");
MODULE_DESCRIPTION("caiaq USB audio");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Native Instruments,RigKontrol2},"
- "{Native Instruments,RigKontrol3},"
- "{Native Instruments,Kore Controller},"
- "{Native Instruments,Kore Controller 2},"
- "{Native Instruments,Audio Kontrol 1},"
- "{Native Instruments,Audio 2 DJ},"
- "{Native Instruments,Audio 4 DJ},"
- "{Native Instruments,Audio 8 DJ},"
- "{Native Instruments,Traktor Audio 2},"
- "{Native Instruments,Session I/O},"
- "{Native Instruments,GuitarRig mobile},"
- "{Native Instruments,Traktor Kontrol X1},"
- "{Native Instruments,Traktor Kontrol S4},"
- "{Native Instruments,Maschine Controller}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-max */
static char* id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* Id for this card */
MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
MODULE_DESCRIPTION("USB Audio");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{Generic,USB Audio}}");
-
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-MAX */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* ID for this card */
MODULE_AUTHOR("Antonio Ospite <ao2@amarulasolutions.com>");
MODULE_DESCRIPTION("M2Tech hiFace USB-SPDIF audio driver");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{M2Tech,Young},"
- "{M2Tech,hiFace},"
- "{M2Tech,North Star},"
- "{M2Tech,W4S Young},"
- "{M2Tech,Corrson},"
- "{M2Tech,AUDIA},"
- "{M2Tech,SL Audio},"
- "{M2Tech,Empirical},"
- "{M2Tech,Rockna},"
- "{M2Tech,Pathos},"
- "{M2Tech,Metronome},"
- "{M2Tech,CAD},"
- "{M2Tech,Audio Esclusive},"
- "{M2Tech,Rotel},"
- "{M2Tech,Eeaudio},"
- "{The Chord Company,CHORD},"
- "{AVA Group A/S,Vitus}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-max */
static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* Id for card */
MODULE_DESCRIPTION("Edirol UA-101/1000 driver");
MODULE_AUTHOR("Clemens Ladisch <clemens@ladisch.de>");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{{Edirol,UA-101},{Edirol,UA-1000}}");
/*
* Should not be lower than the minimum scheduling delay of the host
u8 group = (private_value & SND_DJM_GROUP_MASK) >> SND_DJM_GROUP_SHIFT;
u16 value = elem->value.enumerated.item[0];
- kctl->private_value = ((device << SND_DJM_DEVICE_SHIFT) |
+ kctl->private_value = (((unsigned long)device << SND_DJM_DEVICE_SHIFT) |
(group << SND_DJM_GROUP_SHIFT) |
value);
value = device->controls[i].default_value;
knew.name = device->controls[i].name;
knew.private_value = (
- (device_idx << SND_DJM_DEVICE_SHIFT) |
+ ((unsigned long)device_idx << SND_DJM_DEVICE_SHIFT) |
(i << SND_DJM_GROUP_SHIFT) |
value);
err = snd_djm_controls_update(mixer, device_idx, i, value);
case USB_ID(0x21b4, 0x0081): /* AudioQuest DragonFly */
case USB_ID(0x2912, 0x30c8): /* Audioengine D1 */
case USB_ID(0x413c, 0xa506): /* Dell AE515 sound bar */
+ case USB_ID(0x046d, 0x084c): /* Logitech ConferenceCam Connect */
return true;
}
MODULE_AUTHOR("Karsten Wiese <annabellesgarden@yahoo.de>");
MODULE_DESCRIPTION("TASCAM "NAME_ALLCAPS" Version 0.8.7.2");
MODULE_LICENSE("GPL");
-MODULE_SUPPORTED_DEVICE("{{TASCAM(0x1604),"NAME_ALLCAPS"(0x8001)(0x8005)(0x8007)}}");
static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX; /* Index 0-max */
static char* id[SNDRV_CARDS] = SNDRV_DEFAULT_STR; /* Id for this card */
MODULE_AUTHOR("Jerome Anand <jerome.anand@intel.com>");
MODULE_DESCRIPTION("Intel HDMI Audio driver");
MODULE_LICENSE("GPL v2");
-MODULE_SUPPORTED_DEVICE("{Intel,Intel_HAD}");
MODULE_DESCRIPTION("Xen virtual sound device frontend");
MODULE_LICENSE("GPL");
MODULE_ALIAS("xen:" XENSND_DRIVER_NAME);
-MODULE_SUPPORTED_DEVICE("{{ALSA,Virtual soundcard}}");
#define KVM_XEN_HVM_CONFIG_HYPERCALL_MSR (1 << 0)
#define KVM_XEN_HVM_CONFIG_INTERCEPT_HCALL (1 << 1)
#define KVM_XEN_HVM_CONFIG_SHARED_INFO (1 << 2)
+#define KVM_XEN_HVM_CONFIG_RUNSTATE (1 << 3)
struct kvm_xen_hvm_config {
__u32 flags;
union {
__u64 gpa;
__u64 pad[8];
+ struct {
+ __u64 state;
+ __u64 state_entry_time;
+ __u64 time_running;
+ __u64 time_runnable;
+ __u64 time_blocked;
+ __u64 time_offline;
+ } runstate;
} u;
};
/* Available with KVM_CAP_XEN_HVM / KVM_XEN_HVM_CONFIG_SHARED_INFO */
#define KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO 0x0
#define KVM_XEN_VCPU_ATTR_TYPE_VCPU_TIME_INFO 0x1
+#define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADDR 0x2
+#define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_CURRENT 0x3
+#define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_DATA 0x4
+#define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADJUST 0x5
/* Secure Encrypted Virtualization command */
enum sev_cmd_id {
ExecStart=/usr/bin/kvm_stat -dtcz -s 10 -L /var/log/kvm_stat.csv
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
+RestartSec=60s
SyslogIdentifier=kvm_stat
SyslogLevel=debug
if [ ! -d '$(DESTDIR_SQ)$2' ]; then \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$2'; \
fi; \
- $(INSTALL) $1 $(if $3,-m $3,) '$(DESTDIR_SQ)$2'
+ $(INSTALL) $(if $3,-m $3,) $1 '$(DESTDIR_SQ)$2'
endef
install_lib: all_cmd
return err;
case BTF_KIND_ARRAY:
- return btf_dump_order_type(d, btf_array(t)->type, through_ptr);
+ return btf_dump_order_type(d, btf_array(t)->type, false);
case BTF_KIND_STRUCT:
case BTF_KIND_UNION: {
if (!elf_rawdata(elf_getscn(obj->efile.elf, obj->efile.shstrndx), NULL)) {
pr_warn("elf: failed to get section names strings from %s: %s\n",
obj->path, elf_errmsg(-1));
- return -LIBBPF_ERRNO__FORMAT;
+ err = -LIBBPF_ERRNO__FORMAT;
+ goto errout;
}
/* Old LLVM set e_machine to EM_NONE */
memset(&sa, 0, sizeof(sa));
sa.nl_family = AF_NETLINK;
- sock = socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE);
+ sock = socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, NETLINK_ROUTE);
if (sock < 0)
return -errno;
if ((len & BPF_RINGBUF_DISCARD_BIT) == 0) {
sample = (void *)len_ptr + BPF_RINGBUF_HDR_SZ;
err = r->sample_cb(r->ctx, sample, len);
- if (err) {
+ if (err < 0) {
/* update consumer pos and bail out */
smp_store_release(r->consumer_pos,
cons_pos);
int fd;
int refcount;
struct list_head ctx_list;
+ bool rx_ring_setup_done;
+ bool tx_ring_setup_done;
};
struct xsk_ctx {
return NULL;
}
-static void xsk_put_ctx(struct xsk_ctx *ctx)
+static void xsk_put_ctx(struct xsk_ctx *ctx, bool unmap)
{
struct xsk_umem *umem = ctx->umem;
struct xdp_mmap_offsets off;
int err;
- if (--ctx->refcount == 0) {
- err = xsk_get_mmap_offsets(umem->fd, &off);
- if (!err) {
- munmap(ctx->fill->ring - off.fr.desc,
- off.fr.desc + umem->config.fill_size *
- sizeof(__u64));
- munmap(ctx->comp->ring - off.cr.desc,
- off.cr.desc + umem->config.comp_size *
- sizeof(__u64));
- }
+ if (--ctx->refcount)
+ return;
- list_del(&ctx->list);
- free(ctx);
- }
+ if (!unmap)
+ goto out_free;
+
+ err = xsk_get_mmap_offsets(umem->fd, &off);
+ if (err)
+ goto out_free;
+
+ munmap(ctx->fill->ring - off.fr.desc, off.fr.desc + umem->config.fill_size *
+ sizeof(__u64));
+ munmap(ctx->comp->ring - off.cr.desc, off.cr.desc + umem->config.comp_size *
+ sizeof(__u64));
+
+out_free:
+ list_del(&ctx->list);
+ free(ctx);
}
static struct xsk_ctx *xsk_create_ctx(struct xsk_socket *xsk,
memcpy(ctx->ifname, ifname, IFNAMSIZ - 1);
ctx->ifname[IFNAMSIZ - 1] = '\0';
- umem->fill_save = NULL;
- umem->comp_save = NULL;
ctx->fill = fill;
ctx->comp = comp;
list_add(&ctx->list, &umem->ctx_list);
struct xsk_socket *xsk;
struct xsk_ctx *ctx;
int err, ifindex;
+ bool unmap = umem->fill_save != fill;
+ bool rx_setup_done = false, tx_setup_done = false;
if (!umem || !xsk_ptr || !(rx || tx))
return -EFAULT;
}
} else {
xsk->fd = umem->fd;
+ rx_setup_done = umem->rx_ring_setup_done;
+ tx_setup_done = umem->tx_ring_setup_done;
}
ctx = xsk_get_ctx(umem, ifindex, queue_id);
}
xsk->ctx = ctx;
- if (rx) {
+ if (rx && !rx_setup_done) {
err = setsockopt(xsk->fd, SOL_XDP, XDP_RX_RING,
&xsk->config.rx_size,
sizeof(xsk->config.rx_size));
err = -errno;
goto out_put_ctx;
}
+ if (xsk->fd == umem->fd)
+ umem->rx_ring_setup_done = true;
}
- if (tx) {
+ if (tx && !tx_setup_done) {
err = setsockopt(xsk->fd, SOL_XDP, XDP_TX_RING,
&xsk->config.tx_size,
sizeof(xsk->config.tx_size));
err = -errno;
goto out_put_ctx;
}
+ if (xsk->fd == umem->fd)
+ umem->rx_ring_setup_done = true;
}
err = xsk_get_mmap_offsets(xsk->fd, &off);
}
*xsk_ptr = xsk;
+ umem->fill_save = NULL;
+ umem->comp_save = NULL;
return 0;
out_mmap_tx:
munmap(rx_map, off.rx.desc +
xsk->config.rx_size * sizeof(struct xdp_desc));
out_put_ctx:
- xsk_put_ctx(ctx);
+ xsk_put_ctx(ctx, unmap);
out_socket:
if (--umem->refcount)
close(xsk->fd);
struct xsk_ring_cons *rx, struct xsk_ring_prod *tx,
const struct xsk_socket_config *usr_config)
{
+ if (!umem)
+ return -EFAULT;
+
return xsk_socket__create_shared(xsk_ptr, ifname, queue_id, umem,
rx, tx, umem->fill_save,
umem->comp_save, usr_config);
}
}
- xsk_put_ctx(ctx);
+ xsk_put_ctx(ctx, true);
umem->refcount--;
/* Do not close an fd that also has an associated umem connected
int status;
pid_t pid;
+ /*
+ * Take signal fd data as pure signal notification and check all
+ * the sessions state. The reason is that multiple signals can get
+ * coalesced in kernel and we can receive only single signal even
+ * if multiple SIGCHLD were generated.
+ */
err = read(daemon->signal_fd, &si, sizeof(struct signalfd_siginfo));
- if (err != sizeof(struct signalfd_siginfo))
+ if (err != sizeof(struct signalfd_siginfo)) {
+ pr_err("failed to read signal fd\n");
return -1;
+ }
list_for_each_entry(session, &daemon->sessions, list) {
+ if (session->pid == -1)
+ continue;
- if (session->pid != (int) si.ssi_pid)
+ pid = waitpid(session->pid, &status, WNOHANG);
+ if (pid <= 0)
continue;
- pid = waitpid(session->pid, &status, 0);
- if (pid == session->pid) {
- if (WIFEXITED(status)) {
- pr_info("session '%s' exited, status=%d\n",
- session->name, WEXITSTATUS(status));
- } else if (WIFSIGNALED(status)) {
- pr_info("session '%s' killed (signal %d)\n",
- session->name, WTERMSIG(status));
- } else if (WIFSTOPPED(status)) {
- pr_info("session '%s' stopped (signal %d)\n",
- session->name, WSTOPSIG(status));
- } else {
- pr_info("session '%s' Unexpected status (0x%x)\n",
- session->name, status);
- }
+ if (WIFEXITED(status)) {
+ pr_info("session '%s' exited, status=%d\n",
+ session->name, WEXITSTATUS(status));
+ } else if (WIFSIGNALED(status)) {
+ pr_info("session '%s' killed (signal %d)\n",
+ session->name, WTERMSIG(status));
+ } else if (WIFSTOPPED(status)) {
+ pr_info("session '%s' stopped (signal %d)\n",
+ session->name, WSTOPSIG(status));
+ } else {
+ pr_info("session '%s' Unexpected status (0x%x)\n",
+ session->name, status);
}
session->state = KILL;
session->pid = -1;
- return pid;
}
return 0;
.fd = daemon->signal_fd,
.events = POLLIN,
};
- pid_t wpid = 0, pid = session->pid;
time_t start;
start = time(NULL);
int err = poll(&pollfd, 1, 1000);
if (err > 0) {
- wpid = handle_signalfd(daemon);
+ handle_signalfd(daemon);
} else if (err < 0) {
perror("failed: poll\n");
return -1;
if (start + secs < time(NULL))
return -1;
- } while (wpid != pid);
+ } while (session->pid != -1);
return 0;
}
daemon_session__signal(session, SIGKILL);
break;
default:
- break;
+ pr_err("failed to wait for session %s\n",
+ session->name);
+ return;
}
how++;
daemon__signal(daemon, SIGKILL);
break;
default:
- break;
+ pr_err("failed to wait for sessions\n");
+ return;
}
how++;
close(sock_fd);
if (conf_fd != -1)
close(conf_fd);
- if (conf_fd != -1)
+ if (signal_fd != -1)
close(signal_fd);
pr_info("daemon exited\n");
}
data.path = inject.input_name;
- inject.session = perf_session__new(&data, true, &inject.tool);
+ inject.session = perf_session__new(&data, inject.output.is_pipe, &inject.tool);
if (IS_ERR(inject.session))
return PTR_ERR(inject.session);
.msg_load_fail = "check your vmlinux setting?",
.target_func = &epoll_pwait_loop,
.expect_result = (NR_ITERS + 1) / 2,
- .pin = true,
+ .pin = true,
},
#ifdef HAVE_BPF_PROLOGUE
{
.expect_result = (NR_ITERS + 1) / 4,
},
#endif
- {
- .prog_id = LLVM_TESTCASE_BPF_RELOCATION,
- .desc = "BPF relocation checker",
- .name = "[bpf_relocation_test]",
- .msg_compile_fail = "fix 'perf test LLVM' first",
- .msg_load_fail = "libbpf error when dealing with relocation",
- },
};
static int do_test(struct bpf_object *obj, int (*func)(void),
-#!/bin/sh
+#!/bin/bash
# daemon operations
# SPDX-License-Identifier: GPL-2.0
if ((hdr & SPE_HEADER0_MASK2) == SPE_HEADER0_EXTENDED) {
/* 16-bit extended format header */
- ext_hdr = 1;
+ if (len == 1)
+ return ARM_SPE_BAD_PACKET;
+ ext_hdr = 1;
hdr = buf[1];
if (hdr == SPE_HEADER1_ALIGNMENT)
return arm_spe_get_alignment(buf, len, packet);
queue->set = true;
queue->tid = buffer->tid;
queue->cpu = buffer->cpu;
- } else if (buffer->cpu != queue->cpu || buffer->tid != queue->tid) {
- pr_err("auxtrace queue conflict: cpu %d, tid %d vs cpu %d, tid %d\n",
- queue->cpu, queue->tid, buffer->cpu, buffer->tid);
- return -EINVAL;
}
buffer->buffer_nr = queues->next_buffer_nr++;
double ratio = 0.0;
if (block_fmt->total_cycles)
- ratio = (double)bi->cycles / (double)block_fmt->total_cycles;
+ ratio = (double)bi->cycles_aggr / (double)block_fmt->total_cycles;
return color_pct(hpp, block_fmt->width, 100.0 * ratio);
}
double l, r;
if (block_fmt->total_cycles) {
- l = ((double)bi_l->cycles /
+ l = ((double)bi_l->cycles_aggr /
(double)block_fmt->total_cycles) * 100000.0;
- r = ((double)bi_r->cycles /
+ r = ((double)bi_r->cycles_aggr /
(double)block_fmt->total_cycles) * 100000.0;
return (int64_t)l - (int64_t)r;
}
}
if (info_linear->info_len < offsetof(struct bpf_prog_info, prog_tags)) {
+ free(info_linear);
pr_debug("%s: the kernel is too old, aborting\n", __func__);
return -2;
}
info = &info_linear->info;
+ if (!info->jited_ksyms) {
+ free(info_linear);
+ return -1;
+ }
/* number of ksyms, func_lengths, and tags should match */
sub_prog_cnt = info->nr_jited_ksyms;
if (sub_prog_cnt != info->nr_prog_tags ||
- sub_prog_cnt != info->nr_jited_func_lens)
+ sub_prog_cnt != info->nr_jited_func_lens) {
+ free(info_linear);
return -1;
+ }
/* check BTF func info support */
if (info->btf_id && info->nr_func_info && info->func_info_rec_size) {
/* btf func info number should be same as sub_prog_cnt */
if (sub_prog_cnt != info->nr_func_info) {
pr_debug("%s: mismatch in BPF sub program count and BTF function info count, aborting\n", __func__);
- err = -1;
- goto out;
+ free(info_linear);
+ return -1;
}
if (btf__get_from_id(info->btf_id, &btf)) {
pr_debug("%s: failed to get BTF of id %u, aborting\n", __func__, info->btf_id);
struct perf_cpu_map *cpus = pmu ? perf_cpu_map__get(pmu->cpus) :
cpu_list ? perf_cpu_map__new(cpu_list) : NULL;
+ if (pmu && attr->type == PERF_TYPE_RAW)
+ perf_pmu__warn_invalid_config(pmu, attr->config, name);
+
if (init_attr)
event_attr_init(attr);
return nr_caps;
}
+
+void perf_pmu__warn_invalid_config(struct perf_pmu *pmu, __u64 config,
+ char *name)
+{
+ struct perf_pmu_format *format;
+ __u64 masks = 0, bits;
+ char buf[100];
+ unsigned int i;
+
+ list_for_each_entry(format, &pmu->format, list) {
+ if (format->value != PERF_PMU_FORMAT_VALUE_CONFIG)
+ continue;
+
+ for_each_set_bit(i, format->bits, PERF_PMU_FORMAT_BITS)
+ masks |= 1ULL << i;
+ }
+
+ /*
+ * Kernel doesn't export any valid format bits.
+ */
+ if (masks == 0)
+ return;
+
+ bits = config & ~masks;
+ if (bits == 0)
+ return;
+
+ bitmap_scnprintf((unsigned long *)&bits, sizeof(bits) * 8, buf, sizeof(buf));
+
+ pr_warning("WARNING: event '%s' not valid (bits %s of config "
+ "'%llx' not supported by kernel)!\n",
+ name ?: "N/A", buf, config);
+}
int perf_pmu__caps_parse(struct perf_pmu *pmu);
+void perf_pmu__warn_invalid_config(struct perf_pmu *pmu, __u64 config,
+ char *name);
+
#endif /* __PMU_H */
while (!io.eof) {
static const char anonstr[] = "//anon";
- size_t size;
+ size_t size, aligned_size;
/* ensure null termination since stack will be reused. */
event->mmap2.filename[0] = '\0';
}
size = strlen(event->mmap2.filename) + 1;
- size = PERF_ALIGN(size, sizeof(u64));
+ aligned_size = PERF_ALIGN(size, sizeof(u64));
event->mmap2.len -= event->mmap.start;
event->mmap2.header.size = (sizeof(event->mmap2) -
- (sizeof(event->mmap2.filename) - size));
- memset(event->mmap2.filename + size, 0, machine->id_hdr_size);
+ (sizeof(event->mmap2.filename) - aligned_size));
+ memset(event->mmap2.filename + size, 0, machine->id_hdr_size +
+ (aligned_size - size));
event->mmap2.header.size += machine->id_hdr_size;
event->mmap2.pid = tgid;
event->mmap2.tid = pid;
for (i = 0; i < n; i++) {
char *end;
pid_t _pid;
- bool kernel_thread;
+ bool kernel_thread = false;
_pid = strtol(dirent[i]->d_name, &end, 10);
if (*end)
if (dso != NULL) {
__dsos__add(&machine->dsos, dso);
dso__set_long_name(dso, long_name, false);
+ /* Put dso here because __dsos_add already got it */
+ dso__put(dso);
}
return dso;
# CONFIG_RESET_BRCMSTB_RESCAL is not set
# CONFIG_RESET_INTEL_GW is not set
# CONFIG_ADI_AXI_ADC is not set
+# CONFIG_DEBUG_PAGEALLOC is not set
+# CONFIG_PAGE_POISONING is not set
CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$'
CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+|".*")$'
-KconfigEntryBase = collections.namedtuple('KconfigEntry', ['name', 'value'])
+KconfigEntryBase = collections.namedtuple('KconfigEntryBase', ['name', 'value'])
class KconfigEntry(KconfigEntryBase):
return NULL;
}
+/*
+ * There are always either 1 or 2 objects in the IDR. If we find nothing,
+ * or we find something at an ID we didn't expect, that's a bug.
+ */
void idr_find_test_1(int anchor_id, int throbber_id)
{
pthread_t throbber;
time_t start = time(NULL);
- pthread_create(&throbber, NULL, idr_throbber, &throbber_id);
-
BUG_ON(idr_alloc(&find_idr, xa_mk_value(anchor_id), anchor_id,
anchor_id + 1, GFP_KERNEL) != anchor_id);
+ pthread_create(&throbber, NULL, idr_throbber, &throbber_id);
+
+ rcu_read_lock();
do {
int id = 0;
void *entry = idr_get_next(&find_idr, &id);
- BUG_ON(entry != xa_mk_value(id));
+ rcu_read_unlock();
+ if ((id != anchor_id && id != throbber_id) ||
+ entry != xa_mk_value(id)) {
+ printf("%s(%d, %d): %p at %d\n", __func__, anchor_id,
+ throbber_id, entry, id);
+ abort();
+ }
+ rcu_read_lock();
} while (time(NULL) < start + 11);
+ rcu_read_unlock();
pthread_join(throbber, NULL);
int __weak main(void)
{
+ rcu_register_thread();
radix_tree_init();
idr_checks();
ida_tests();
rcu_barrier();
if (nr_allocated)
printf("nr_allocated = %d\n", nr_allocated);
+ rcu_unregister_thread();
return 0;
}
int __weak main(void)
{
+ rcu_register_thread();
radix_tree_init();
multiorder_checks();
+ rcu_unregister_thread();
return 0;
}
int __weak main(void)
{
+ rcu_register_thread();
radix_tree_init();
xarray_tests();
radix_tree_cpu_dead(1);
rcu_barrier();
if (nr_allocated)
printf("nr_allocated = %d\n", nr_allocated);
+ rcu_unregister_thread();
return 0;
}
// Set up test pattern in the FFR
// x0: pid
// x2: generation
+//
+// We need to generate a canonical FFR value, which consists of a number of
+// low "1" bits, followed by a number of zeros. This gives us 17 unique values
+// per 16 bits of FFR, so we create a 4 bit signature out of the PID and
+// generation, and use that as the initial number of ones in the pattern.
+// We fill the upper lanes of FFR with zeros.
// Beware: corrupts P0.
function setup_ffr
mov x4, x30
- bl pattern
+ and w0, w0, #0x3
+ bfi w0, w2, #2, #2
+ mov w1, #1
+ lsl w1, w1, w0
+ sub w1, w1, #1
+
ldr x0, =ffrref
- ldr x1, =scratch
- rdvl x2, #1
- lsr x2, x2, #3
- bl memcpy
+ strh w1, [x0], 2
+ rdvl x1, #1
+ lsr x1, x1, #3
+ sub x1, x1, #2
+ bl memclr
mov x0, #0
ldr x1, =ffrref
#include <test_progs.h>
#include "bpf_dctcp.skel.h"
#include "bpf_cubic.skel.h"
+#include "bpf_tcp_nogpl.skel.h"
#define min(a, b) ((a) < (b) ? (a) : (b))
bpf_dctcp__destroy(dctcp_skel);
}
+static char *err_str;
+static bool found;
+
+static int libbpf_debug_print(enum libbpf_print_level level,
+ const char *format, va_list args)
+{
+ char *log_buf;
+
+ if (level != LIBBPF_WARN ||
+ strcmp(format, "libbpf: \n%s\n")) {
+ vprintf(format, args);
+ return 0;
+ }
+
+ log_buf = va_arg(args, char *);
+ if (!log_buf)
+ goto out;
+ if (err_str && strstr(log_buf, err_str) != NULL)
+ found = true;
+out:
+ printf(format, log_buf);
+ return 0;
+}
+
+static void test_invalid_license(void)
+{
+ libbpf_print_fn_t old_print_fn;
+ struct bpf_tcp_nogpl *skel;
+
+ err_str = "struct ops programs must have a GPL compatible license";
+ found = false;
+ old_print_fn = libbpf_set_print(libbpf_debug_print);
+
+ skel = bpf_tcp_nogpl__open_and_load();
+ ASSERT_NULL(skel, "bpf_tcp_nogpl");
+ ASSERT_EQ(found, true, "expected_err_msg");
+
+ bpf_tcp_nogpl__destroy(skel);
+ libbpf_set_print(old_print_fn);
+}
+
void test_bpf_tcp_ca(void)
{
if (test__start_subtest("dctcp"))
test_dctcp();
if (test__start_subtest("cubic"))
test_cubic();
+ if (test__start_subtest("invalid_license"))
+ test_invalid_license();
}
test_check_mtu_run_xdp(skel, skel->progs.xdp_use_helper, mtu);
test_check_mtu_run_xdp(skel, skel->progs.xdp_exceed_mtu, mtu);
test_check_mtu_run_xdp(skel, skel->progs.xdp_minus_delta, mtu);
+ test_check_mtu_run_xdp(skel, skel->progs.xdp_input_len, mtu);
+ test_check_mtu_run_xdp(skel, skel->progs.xdp_input_len_exceed, mtu);
cleanup:
test_check_mtu__destroy(skel);
test_check_mtu_run_tc(skel, skel->progs.tc_exceed_mtu, mtu);
test_check_mtu_run_tc(skel, skel->progs.tc_exceed_mtu_da, mtu);
test_check_mtu_run_tc(skel, skel->progs.tc_minus_delta, mtu);
+ test_check_mtu_run_tc(skel, skel->progs.tc_input_len, mtu);
+ test_check_mtu_run_tc(skel, skel->progs.tc_input_len_exceed, mtu);
cleanup:
test_check_mtu__destroy(skel);
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#define _GNU_SOURCE
+#include <sched.h>
+#include <test_progs.h>
+#include <time.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+#include "fexit_sleep.skel.h"
+
+static int do_sleep(void *skel)
+{
+ struct fexit_sleep *fexit_skel = skel;
+ struct timespec ts1 = { .tv_nsec = 1 };
+ struct timespec ts2 = { .tv_sec = 10 };
+
+ fexit_skel->bss->pid = getpid();
+ (void)syscall(__NR_nanosleep, &ts1, NULL);
+ (void)syscall(__NR_nanosleep, &ts2, NULL);
+ return 0;
+}
+
+#define STACK_SIZE (1024 * 1024)
+static char child_stack[STACK_SIZE];
+
+void test_fexit_sleep(void)
+{
+ struct fexit_sleep *fexit_skel = NULL;
+ int wstatus, duration = 0;
+ pid_t cpid;
+ int err, fexit_cnt;
+
+ fexit_skel = fexit_sleep__open_and_load();
+ if (CHECK(!fexit_skel, "fexit_skel_load", "fexit skeleton failed\n"))
+ goto cleanup;
+
+ err = fexit_sleep__attach(fexit_skel);
+ if (CHECK(err, "fexit_attach", "fexit attach failed: %d\n", err))
+ goto cleanup;
+
+ cpid = clone(do_sleep, child_stack + STACK_SIZE, CLONE_FILES | SIGCHLD, fexit_skel);
+ if (CHECK(cpid == -1, "clone", strerror(errno)))
+ goto cleanup;
+
+ /* wait until first sys_nanosleep ends and second sys_nanosleep starts */
+ while (READ_ONCE(fexit_skel->bss->fentry_cnt) != 2);
+ fexit_cnt = READ_ONCE(fexit_skel->bss->fexit_cnt);
+ if (CHECK(fexit_cnt != 1, "fexit_cnt", "%d", fexit_cnt))
+ goto cleanup;
+
+ /* close progs and detach them. That will trigger two nop5->jmp5 rewrites
+ * in the trampolines to skip nanosleep_fexit prog.
+ * The nanosleep_fentry prog will get detached first.
+ * The nanosleep_fexit prog will get detached second.
+ * Detaching will trigger freeing of both progs JITed images.
+ * There will be two dying bpf_tramp_image-s, but only the initial
+ * bpf_tramp_image (with both _fentry and _fexit progs will be stuck
+ * waiting for percpu_ref_kill to confirm). The other one
+ * will be freed quickly.
+ */
+ close(bpf_program__fd(fexit_skel->progs.nanosleep_fentry));
+ close(bpf_program__fd(fexit_skel->progs.nanosleep_fexit));
+ fexit_sleep__detach(fexit_skel);
+
+ /* kill the thread to unwind sys_nanosleep stack through the trampoline */
+ kill(cpid, 9);
+
+ if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", strerror(errno)))
+ goto cleanup;
+ if (CHECK(WEXITSTATUS(wstatus) != 0, "exitstatus", "failed"))
+ goto cleanup;
+
+ /* The bypassed nanosleep_fexit prog shouldn't have executed.
+ * Unlike progs the maps were not freed and directly accessible.
+ */
+ fexit_cnt = READ_ONCE(fexit_skel->bss->fexit_cnt);
+ if (CHECK(fexit_cnt != 1, "fexit_cnt", "%d", fexit_cnt))
+ goto cleanup;
+
+cleanup:
+ fexit_sleep__destroy(fexit_skel);
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <linux/types.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_tcp_helpers.h"
+
+char _license[] SEC("license") = "X";
+
+void BPF_STRUCT_OPS(nogpltcp_init, struct sock *sk)
+{
+}
+
+SEC(".struct_ops")
+struct tcp_congestion_ops bpf_nogpltcp = {
+ .init = (void *)nogpltcp_init,
+ .name = "bpf_nogpltcp",
+};
};
};
+struct struct_in_array {};
+
+struct struct_in_array_typed {};
+
+typedef struct struct_in_array_typed struct_in_array_t[2];
+
struct struct_with_embedded_stuff {
int a;
struct {
} r[5];
struct struct_in_struct s[10];
int t[11];
+ struct struct_in_array (*u)[2];
+ struct_in_array_t *v;
};
struct root_struct {
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2021 Facebook */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+char LICENSE[] SEC("license") = "GPL";
+
+int pid = 0;
+int fentry_cnt = 0;
+int fexit_cnt = 0;
+
+SEC("fentry/__x64_sys_nanosleep")
+int BPF_PROG(nanosleep_fentry, const struct pt_regs *regs)
+{
+ if ((int)bpf_get_current_pid_tgid() != pid)
+ return 0;
+
+ fentry_cnt++;
+ return 0;
+}
+
+SEC("fexit/__x64_sys_nanosleep")
+int BPF_PROG(nanosleep_fexit, const struct pt_regs *regs, int ret)
+{
+ if ((int)bpf_get_current_pid_tgid() != pid)
+ return 0;
+
+ fexit_cnt++;
+ return 0;
+}
return retval;
}
+SEC("xdp")
+int xdp_input_len(struct xdp_md *ctx)
+{
+ int retval = XDP_PASS; /* Expected retval on successful test */
+ void *data_end = (void *)(long)ctx->data_end;
+ void *data = (void *)(long)ctx->data;
+ __u32 ifindex = GLOBAL_USER_IFINDEX;
+ __u32 data_len = data_end - data;
+
+ /* API allow user give length to check as input via mtu_len param,
+ * resulting MTU value is still output in mtu_len param after call.
+ *
+ * Input len is L3, like MTU and iph->tot_len.
+ * Remember XDP data_len is L2.
+ */
+ __u32 mtu_len = data_len - ETH_HLEN;
+
+ if (bpf_check_mtu(ctx, ifindex, &mtu_len, 0, 0))
+ retval = XDP_ABORTED;
+
+ global_bpf_mtu_xdp = mtu_len;
+ return retval;
+}
+
+SEC("xdp")
+int xdp_input_len_exceed(struct xdp_md *ctx)
+{
+ int retval = XDP_ABORTED; /* Fail */
+ __u32 ifindex = GLOBAL_USER_IFINDEX;
+ int err;
+
+ /* API allow user give length to check as input via mtu_len param,
+ * resulting MTU value is still output in mtu_len param after call.
+ *
+ * Input length value is L3 size like MTU.
+ */
+ __u32 mtu_len = GLOBAL_USER_MTU;
+
+ mtu_len += 1; /* Exceed with 1 */
+
+ err = bpf_check_mtu(ctx, ifindex, &mtu_len, 0, 0);
+ if (err == BPF_MTU_CHK_RET_FRAG_NEEDED)
+ retval = XDP_PASS ; /* Success in exceeding MTU check */
+
+ global_bpf_mtu_xdp = mtu_len;
+ return retval;
+}
+
SEC("classifier")
int tc_use_helper(struct __sk_buff *ctx)
{
global_bpf_mtu_xdp = mtu_len;
return retval;
}
+
+SEC("classifier")
+int tc_input_len(struct __sk_buff *ctx)
+{
+ int retval = BPF_OK; /* Expected retval on successful test */
+ __u32 ifindex = GLOBAL_USER_IFINDEX;
+
+ /* API allow user give length to check as input via mtu_len param,
+ * resulting MTU value is still output in mtu_len param after call.
+ *
+ * Input length value is L3 size.
+ */
+ __u32 mtu_len = GLOBAL_USER_MTU;
+
+ if (bpf_check_mtu(ctx, ifindex, &mtu_len, 0, 0))
+ retval = BPF_DROP;
+
+ global_bpf_mtu_xdp = mtu_len;
+ return retval;
+}
+
+SEC("classifier")
+int tc_input_len_exceed(struct __sk_buff *ctx)
+{
+ int retval = BPF_DROP; /* Fail */
+ __u32 ifindex = GLOBAL_USER_IFINDEX;
+ int err;
+
+ /* API allow user give length to check as input via mtu_len param,
+ * resulting MTU value is still output in mtu_len param after call.
+ *
+ * Input length value is L3 size like MTU.
+ */
+ __u32 mtu_len = GLOBAL_USER_MTU;
+
+ mtu_len += 1; /* Exceed with 1 */
+
+ err = bpf_check_mtu(ctx, ifindex, &mtu_len, 0, 0);
+ if (err == BPF_MTU_CHK_RET_FRAG_NEEDED)
+ retval = BPF_OK; /* Success in exceeding MTU check */
+
+ global_bpf_mtu_xdp = mtu_len;
+ return retval;
+}
}
ret = bpf_skb_get_tunnel_opt(skb, &gopt, sizeof(gopt));
- if (ret < 0) {
- ERROR(ret);
- return TC_ACT_SHOT;
- }
+ if (ret < 0)
+ gopt.opt_class = 0;
bpf_trace_printk(fmt, sizeof(fmt),
key.tunnel_id, key.remote_ipv4, gopt.opt_class);
BPF_ALU64_REG(BPF_SUB, BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
},
- .result = REJECT,
+ .errstr_unpriv = "R0 tried to sub from different maps, paths, or prohibited types",
.errstr = "R0 tried to subtract pointer from scalar",
+ .result = REJECT,
},
{
"check deducing bounds from const, 2",
BPF_ALU64_REG(BPF_SUB, BPF_REG_1, BPF_REG_0),
BPF_EXIT_INSN(),
},
+ .errstr_unpriv = "R1 tried to sub from different maps, paths, or prohibited types",
+ .result_unpriv = REJECT,
.result = ACCEPT,
.retval = 1,
},
BPF_ALU64_REG(BPF_SUB, BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
},
- .result = REJECT,
+ .errstr_unpriv = "R0 tried to sub from different maps, paths, or prohibited types",
.errstr = "R0 tried to subtract pointer from scalar",
+ .result = REJECT,
},
{
"check deducing bounds from const, 4",
BPF_ALU64_REG(BPF_SUB, BPF_REG_1, BPF_REG_0),
BPF_EXIT_INSN(),
},
+ .errstr_unpriv = "R1 tried to sub from different maps, paths, or prohibited types",
+ .result_unpriv = REJECT,
.result = ACCEPT,
},
{
BPF_ALU64_REG(BPF_SUB, BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
},
- .result = REJECT,
+ .errstr_unpriv = "R0 tried to sub from different maps, paths, or prohibited types",
.errstr = "R0 tried to subtract pointer from scalar",
+ .result = REJECT,
},
{
"check deducing bounds from const, 6",
BPF_ALU64_REG(BPF_SUB, BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
},
- .result = REJECT,
+ .errstr_unpriv = "R0 tried to sub from different maps, paths, or prohibited types",
.errstr = "R0 tried to subtract pointer from scalar",
+ .result = REJECT,
},
{
"check deducing bounds from const, 7",
offsetof(struct __sk_buff, mark)),
BPF_EXIT_INSN(),
},
- .result = REJECT,
+ .errstr_unpriv = "R1 tried to sub from different maps, paths, or prohibited types",
.errstr = "dereference of modified ctx ptr",
+ .result = REJECT,
.flags = F_NEEDS_EFFICIENT_UNALIGNED_ACCESS,
},
{
offsetof(struct __sk_buff, mark)),
BPF_EXIT_INSN(),
},
- .result = REJECT,
+ .errstr_unpriv = "R1 tried to add from different maps, paths, or prohibited types",
.errstr = "dereference of modified ctx ptr",
+ .result = REJECT,
.flags = F_NEEDS_EFFICIENT_UNALIGNED_ACCESS,
},
{
BPF_ALU64_REG(BPF_SUB, BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
},
- .result = REJECT,
+ .errstr_unpriv = "R0 tried to sub from different maps, paths, or prohibited types",
.errstr = "R0 tried to subtract pointer from scalar",
+ .result = REJECT,
},
{
"check deducing bounds from const, 10",
BPF_ALU64_REG(BPF_SUB, BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
},
- .result = REJECT,
.errstr = "math between ctx pointer and register with unbounded min value is not allowed",
+ .result = REJECT,
},
BPF_EXIT_INSN(),
},
.fixup_map_hash_16b = { 4 },
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "R1 tried to add from different maps, paths, or prohibited types",
.result = ACCEPT,
},
{
BPF_EXIT_INSN(),
},
.fixup_map_hash_16b = { 4 },
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "R1 tried to add from different maps, paths, or prohibited types",
.result = ACCEPT,
},
.result = ACCEPT,
},
{
- "unpriv: adding of fp",
+ "unpriv: adding of fp, reg",
.insns = {
BPF_MOV64_IMM(BPF_REG_0, 0),
BPF_MOV64_IMM(BPF_REG_1, 0),
BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, -8),
BPF_EXIT_INSN(),
},
+ .errstr_unpriv = "R1 tried to add from different maps, paths, or prohibited types",
+ .result_unpriv = REJECT,
+ .result = ACCEPT,
+},
+{
+ "unpriv: adding of fp, imm",
+ .insns = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 0),
+ BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, -8),
+ BPF_EXIT_INSN(),
+ },
.errstr_unpriv = "R1 stack pointer arithmetic goes out of range",
.result_unpriv = REJECT,
.result = ACCEPT,
.fixup_map_array_48b = { 1 },
.result = ACCEPT,
.result_unpriv = REJECT,
- .errstr_unpriv = "R2 tried to add from different maps or paths",
+ .errstr_unpriv = "R2 tried to add from different maps, paths, or prohibited types",
.retval = 0,
},
{
.result = ACCEPT,
.retval = 0xabcdef12,
},
+{
+ "map access: value_ptr += N, value_ptr -= N known scalar",
+ .insns = {
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 6),
+ BPF_MOV32_IMM(BPF_REG_1, 0x12345678),
+ BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, 0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, 2),
+ BPF_MOV64_IMM(BPF_REG_1, 2),
+ BPF_ALU64_REG(BPF_SUB, BPF_REG_0, BPF_REG_1),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_array_48b = { 3 },
+ .result = ACCEPT,
+ .retval = 0x12345678,
+},
{
"map access: unknown scalar += value_ptr, 1",
.insns = {
/x86_64/debug_regs
/x86_64/evmcs_test
/x86_64/get_cpuid_test
+/x86_64/get_msr_index_features
/x86_64/kvm_pv_test
+/x86_64/hyperv_clock
/x86_64/hyperv_cpuid
/x86_64/mmio_warning_test
/x86_64/platform_info_test
+/x86_64/set_boot_cpu_id
/x86_64/set_sregs_test
/x86_64/smm_test
/x86_64/state_test
LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
+TEST_GEN_PROGS_x86_64 += x86_64/get_msr_index_features
TEST_GEN_PROGS_x86_64 += x86_64/evmcs_test
TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test
+TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
TEST_GEN_PROGS_x86_64 += x86_64/platform_info_test
+TEST_GEN_PROGS_x86_64 += x86_64/set_boot_cpu_id
TEST_GEN_PROGS_x86_64 += x86_64/set_sregs_test
TEST_GEN_PROGS_x86_64 += x86_64/smm_test
TEST_GEN_PROGS_x86_64 += x86_64/state_test
kvm_vm_elf_load(vm, program_invocation_name, 0, 0);
vm_create_irqchip(vm);
- fprintf(stderr, "%s: [%d] start vcpus\n", __func__, run);
+ pr_debug("%s: [%d] start vcpus\n", __func__, run);
for (i = 0; i < VCPU_NUM; ++i) {
vm_vcpu_add_default(vm, i, guest_code);
payloads[i].vm = vm;
check_set_affinity(throw_away, &cpu_set);
}
}
- fprintf(stderr, "%s: [%d] all threads launched\n", __func__, run);
+ pr_debug("%s: [%d] all threads launched\n", __func__, run);
sem_post(sem);
for (i = 0; i < VCPU_NUM; ++i)
check_join(threads[i], &b);
if (pid == 0)
run_test(i); /* This function always exits */
- fprintf(stderr, "%s: [%d] waiting semaphore\n", __func__, i);
+ pr_debug("%s: [%d] waiting semaphore\n", __func__, i);
sem_wait(sem);
r = (rand() % DELAY_US_MAX) + 1;
- fprintf(stderr, "%s: [%d] waiting %dus\n", __func__, i, r);
+ pr_debug("%s: [%d] waiting %dus\n", __func__, i, r);
usleep(r);
r = waitpid(pid, &s, WNOHANG);
TEST_ASSERT(r != pid,
"%s: [%d] child exited unexpectedly status: [%d]",
__func__, i, s);
- fprintf(stderr, "%s: [%d] killing child\n", __func__, i);
+ pr_debug("%s: [%d] killing child\n", __func__, i);
kill(pid, SIGKILL);
}
#include "sparsebit.h"
+#define KVM_DEV_PATH "/dev/kvm"
#define KVM_MAX_VCPUS 512
/*
int _vcpu_ioctl(struct kvm_vm *vm, uint32_t vcpuid, unsigned long ioctl,
void *arg);
void vm_ioctl(struct kvm_vm *vm, unsigned long ioctl, void *arg);
+int _vm_ioctl(struct kvm_vm *vm, unsigned long cmd, void *arg);
void kvm_ioctl(struct kvm_vm *vm, unsigned long ioctl, void *arg);
int _kvm_ioctl(struct kvm_vm *vm, unsigned long ioctl, void *arg);
void vm_mem_region_set_flags(struct kvm_vm *vm, uint32_t slot, uint32_t flags);
{
int ret;
- ret = ioctl(vm->fd, cmd, arg);
+ ret = _vm_ioctl(vm, cmd, arg);
TEST_ASSERT(ret == 0, "vm ioctl %lu failed, rc: %i errno: %i (%s)",
cmd, ret, errno, strerror(errno));
}
+int _vm_ioctl(struct kvm_vm *vm, unsigned long cmd, void *arg)
+{
+ return ioctl(vm->fd, cmd, arg);
+}
+
/*
* KVM system ioctl
*
#include "sparsebit.h"
-#define KVM_DEV_PATH "/dev/kvm"
-
struct userspace_mem_region {
struct kvm_userspace_memory_region region;
struct sparsebit *unused_phy_pages;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test that KVM_GET_MSR_INDEX_LIST and
+ * KVM_GET_MSR_FEATURE_INDEX_LIST work as intended
+ *
+ * Copyright (C) 2020, Red Hat, Inc.
+ */
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+static int kvm_num_index_msrs(int kvm_fd, int nmsrs)
+{
+ struct kvm_msr_list *list;
+ int r;
+
+ list = malloc(sizeof(*list) + nmsrs * sizeof(list->indices[0]));
+ list->nmsrs = nmsrs;
+ r = ioctl(kvm_fd, KVM_GET_MSR_INDEX_LIST, list);
+ TEST_ASSERT(r == -1 && errno == E2BIG,
+ "Unexpected result from KVM_GET_MSR_INDEX_LIST probe, r: %i",
+ r);
+
+ r = list->nmsrs;
+ free(list);
+ return r;
+}
+
+static void test_get_msr_index(void)
+{
+ int old_res, res, kvm_fd, r;
+ struct kvm_msr_list *list;
+
+ kvm_fd = open(KVM_DEV_PATH, O_RDONLY);
+ if (kvm_fd < 0)
+ exit(KSFT_SKIP);
+
+ old_res = kvm_num_index_msrs(kvm_fd, 0);
+ TEST_ASSERT(old_res != 0, "Expecting nmsrs to be > 0");
+
+ if (old_res != 1) {
+ res = kvm_num_index_msrs(kvm_fd, 1);
+ TEST_ASSERT(res > 1, "Expecting nmsrs to be > 1");
+ TEST_ASSERT(res == old_res, "Expecting nmsrs to be identical");
+ }
+
+ list = malloc(sizeof(*list) + old_res * sizeof(list->indices[0]));
+ list->nmsrs = old_res;
+ r = ioctl(kvm_fd, KVM_GET_MSR_INDEX_LIST, list);
+
+ TEST_ASSERT(r == 0,
+ "Unexpected result from KVM_GET_MSR_FEATURE_INDEX_LIST, r: %i",
+ r);
+ TEST_ASSERT(list->nmsrs == old_res, "Expecting nmsrs to be identical");
+ free(list);
+
+ close(kvm_fd);
+}
+
+static int kvm_num_feature_msrs(int kvm_fd, int nmsrs)
+{
+ struct kvm_msr_list *list;
+ int r;
+
+ list = malloc(sizeof(*list) + nmsrs * sizeof(list->indices[0]));
+ list->nmsrs = nmsrs;
+ r = ioctl(kvm_fd, KVM_GET_MSR_FEATURE_INDEX_LIST, list);
+ TEST_ASSERT(r == -1 && errno == E2BIG,
+ "Unexpected result from KVM_GET_MSR_FEATURE_INDEX_LIST probe, r: %i",
+ r);
+
+ r = list->nmsrs;
+ free(list);
+ return r;
+}
+
+struct kvm_msr_list *kvm_get_msr_feature_list(int kvm_fd, int nmsrs)
+{
+ struct kvm_msr_list *list;
+ int r;
+
+ list = malloc(sizeof(*list) + nmsrs * sizeof(list->indices[0]));
+ list->nmsrs = nmsrs;
+ r = ioctl(kvm_fd, KVM_GET_MSR_FEATURE_INDEX_LIST, list);
+
+ TEST_ASSERT(r == 0,
+ "Unexpected result from KVM_GET_MSR_FEATURE_INDEX_LIST, r: %i",
+ r);
+
+ return list;
+}
+
+static void test_get_msr_feature(void)
+{
+ int res, old_res, i, kvm_fd;
+ struct kvm_msr_list *feature_list;
+
+ kvm_fd = open(KVM_DEV_PATH, O_RDONLY);
+ if (kvm_fd < 0)
+ exit(KSFT_SKIP);
+
+ old_res = kvm_num_feature_msrs(kvm_fd, 0);
+ TEST_ASSERT(old_res != 0, "Expecting nmsrs to be > 0");
+
+ if (old_res != 1) {
+ res = kvm_num_feature_msrs(kvm_fd, 1);
+ TEST_ASSERT(res > 1, "Expecting nmsrs to be > 1");
+ TEST_ASSERT(res == old_res, "Expecting nmsrs to be identical");
+ }
+
+ feature_list = kvm_get_msr_feature_list(kvm_fd, old_res);
+ TEST_ASSERT(old_res == feature_list->nmsrs,
+ "Unmatching number of msr indexes");
+
+ for (i = 0; i < feature_list->nmsrs; i++)
+ kvm_get_feature_msr(feature_list->indices[i]);
+
+ free(feature_list);
+ close(kvm_fd);
+}
+
+int main(int argc, char *argv[])
+{
+ if (kvm_check_cap(KVM_CAP_GET_MSR_FEATURES))
+ test_get_msr_feature();
+
+ test_get_msr_index();
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021, Red Hat, Inc.
+ *
+ * Tests for Hyper-V clocksources
+ */
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+struct ms_hyperv_tsc_page {
+ volatile u32 tsc_sequence;
+ u32 reserved1;
+ volatile u64 tsc_scale;
+ volatile s64 tsc_offset;
+} __packed;
+
+#define HV_X64_MSR_GUEST_OS_ID 0x40000000
+#define HV_X64_MSR_TIME_REF_COUNT 0x40000020
+#define HV_X64_MSR_REFERENCE_TSC 0x40000021
+#define HV_X64_MSR_TSC_FREQUENCY 0x40000022
+#define HV_X64_MSR_REENLIGHTENMENT_CONTROL 0x40000106
+#define HV_X64_MSR_TSC_EMULATION_CONTROL 0x40000107
+
+/* Simplified mul_u64_u64_shr() */
+static inline u64 mul_u64_u64_shr64(u64 a, u64 b)
+{
+ union {
+ u64 ll;
+ struct {
+ u32 low, high;
+ } l;
+ } rm, rn, rh, a0, b0;
+ u64 c;
+
+ a0.ll = a;
+ b0.ll = b;
+
+ rm.ll = (u64)a0.l.low * b0.l.high;
+ rn.ll = (u64)a0.l.high * b0.l.low;
+ rh.ll = (u64)a0.l.high * b0.l.high;
+
+ rh.l.low = c = rm.l.high + rn.l.high + rh.l.low;
+ rh.l.high = (c >> 32) + rh.l.high;
+
+ return rh.ll;
+}
+
+static inline void nop_loop(void)
+{
+ int i;
+
+ for (i = 0; i < 1000000; i++)
+ asm volatile("nop");
+}
+
+static inline void check_tsc_msr_rdtsc(void)
+{
+ u64 tsc_freq, r1, r2, t1, t2;
+ s64 delta_ns;
+
+ tsc_freq = rdmsr(HV_X64_MSR_TSC_FREQUENCY);
+ GUEST_ASSERT(tsc_freq > 0);
+
+ /* First, check MSR-based clocksource */
+ r1 = rdtsc();
+ t1 = rdmsr(HV_X64_MSR_TIME_REF_COUNT);
+ nop_loop();
+ r2 = rdtsc();
+ t2 = rdmsr(HV_X64_MSR_TIME_REF_COUNT);
+
+ GUEST_ASSERT(r2 > r1 && t2 > t1);
+
+ /* HV_X64_MSR_TIME_REF_COUNT is in 100ns */
+ delta_ns = ((t2 - t1) * 100) - ((r2 - r1) * 1000000000 / tsc_freq);
+ if (delta_ns < 0)
+ delta_ns = -delta_ns;
+
+ /* 1% tolerance */
+ GUEST_ASSERT(delta_ns * 100 < (t2 - t1) * 100);
+}
+
+static inline u64 get_tscpage_ts(struct ms_hyperv_tsc_page *tsc_page)
+{
+ return mul_u64_u64_shr64(rdtsc(), tsc_page->tsc_scale) + tsc_page->tsc_offset;
+}
+
+static inline void check_tsc_msr_tsc_page(struct ms_hyperv_tsc_page *tsc_page)
+{
+ u64 r1, r2, t1, t2;
+
+ /* Compare TSC page clocksource with HV_X64_MSR_TIME_REF_COUNT */
+ t1 = get_tscpage_ts(tsc_page);
+ r1 = rdmsr(HV_X64_MSR_TIME_REF_COUNT);
+
+ /* 10 ms tolerance */
+ GUEST_ASSERT(r1 >= t1 && r1 - t1 < 100000);
+ nop_loop();
+
+ t2 = get_tscpage_ts(tsc_page);
+ r2 = rdmsr(HV_X64_MSR_TIME_REF_COUNT);
+ GUEST_ASSERT(r2 >= t1 && r2 - t2 < 100000);
+}
+
+static void guest_main(struct ms_hyperv_tsc_page *tsc_page, vm_paddr_t tsc_page_gpa)
+{
+ u64 tsc_scale, tsc_offset;
+
+ /* Set Guest OS id to enable Hyper-V emulation */
+ GUEST_SYNC(1);
+ wrmsr(HV_X64_MSR_GUEST_OS_ID, (u64)0x8100 << 48);
+ GUEST_SYNC(2);
+
+ check_tsc_msr_rdtsc();
+
+ GUEST_SYNC(3);
+
+ /* Set up TSC page is disabled state, check that it's clean */
+ wrmsr(HV_X64_MSR_REFERENCE_TSC, tsc_page_gpa);
+ GUEST_ASSERT(tsc_page->tsc_sequence == 0);
+ GUEST_ASSERT(tsc_page->tsc_scale == 0);
+ GUEST_ASSERT(tsc_page->tsc_offset == 0);
+
+ GUEST_SYNC(4);
+
+ /* Set up TSC page is enabled state */
+ wrmsr(HV_X64_MSR_REFERENCE_TSC, tsc_page_gpa | 0x1);
+ GUEST_ASSERT(tsc_page->tsc_sequence != 0);
+
+ GUEST_SYNC(5);
+
+ check_tsc_msr_tsc_page(tsc_page);
+
+ GUEST_SYNC(6);
+
+ tsc_offset = tsc_page->tsc_offset;
+ /* Call KVM_SET_CLOCK from userspace, check that TSC page was updated */
+
+ GUEST_SYNC(7);
+ /* Sanity check TSC page timestamp, it should be close to 0 */
+ GUEST_ASSERT(get_tscpage_ts(tsc_page) < 100000);
+
+ GUEST_ASSERT(tsc_page->tsc_offset != tsc_offset);
+
+ nop_loop();
+
+ /*
+ * Enable Re-enlightenment and check that TSC page stays constant across
+ * KVM_SET_CLOCK.
+ */
+ wrmsr(HV_X64_MSR_REENLIGHTENMENT_CONTROL, 0x1 << 16 | 0xff);
+ wrmsr(HV_X64_MSR_TSC_EMULATION_CONTROL, 0x1);
+ tsc_offset = tsc_page->tsc_offset;
+ tsc_scale = tsc_page->tsc_scale;
+ GUEST_SYNC(8);
+ GUEST_ASSERT(tsc_page->tsc_offset == tsc_offset);
+ GUEST_ASSERT(tsc_page->tsc_scale == tsc_scale);
+
+ GUEST_SYNC(9);
+
+ check_tsc_msr_tsc_page(tsc_page);
+
+ /*
+ * Disable re-enlightenment and TSC page, check that KVM doesn't update
+ * it anymore.
+ */
+ wrmsr(HV_X64_MSR_REENLIGHTENMENT_CONTROL, 0);
+ wrmsr(HV_X64_MSR_TSC_EMULATION_CONTROL, 0);
+ wrmsr(HV_X64_MSR_REFERENCE_TSC, 0);
+ memset(tsc_page, 0, sizeof(*tsc_page));
+
+ GUEST_SYNC(10);
+ GUEST_ASSERT(tsc_page->tsc_sequence == 0);
+ GUEST_ASSERT(tsc_page->tsc_offset == 0);
+ GUEST_ASSERT(tsc_page->tsc_scale == 0);
+
+ GUEST_DONE();
+}
+
+#define VCPU_ID 0
+
+static void host_check_tsc_msr_rdtsc(struct kvm_vm *vm)
+{
+ u64 tsc_freq, r1, r2, t1, t2;
+ s64 delta_ns;
+
+ tsc_freq = vcpu_get_msr(vm, VCPU_ID, HV_X64_MSR_TSC_FREQUENCY);
+ TEST_ASSERT(tsc_freq > 0, "TSC frequency must be nonzero");
+
+ /* First, check MSR-based clocksource */
+ r1 = rdtsc();
+ t1 = vcpu_get_msr(vm, VCPU_ID, HV_X64_MSR_TIME_REF_COUNT);
+ nop_loop();
+ r2 = rdtsc();
+ t2 = vcpu_get_msr(vm, VCPU_ID, HV_X64_MSR_TIME_REF_COUNT);
+
+ TEST_ASSERT(t2 > t1, "Time reference MSR is not monotonic (%ld <= %ld)", t1, t2);
+
+ /* HV_X64_MSR_TIME_REF_COUNT is in 100ns */
+ delta_ns = ((t2 - t1) * 100) - ((r2 - r1) * 1000000000 / tsc_freq);
+ if (delta_ns < 0)
+ delta_ns = -delta_ns;
+
+ /* 1% tolerance */
+ TEST_ASSERT(delta_ns * 100 < (t2 - t1) * 100,
+ "Elapsed time does not match (MSR=%ld, TSC=%ld)",
+ (t2 - t1) * 100, (r2 - r1) * 1000000000 / tsc_freq);
+}
+
+int main(void)
+{
+ struct kvm_vm *vm;
+ struct kvm_run *run;
+ struct ucall uc;
+ vm_vaddr_t tsc_page_gva;
+ int stage;
+
+ vm = vm_create_default(VCPU_ID, 0, guest_main);
+ run = vcpu_state(vm, VCPU_ID);
+
+ vcpu_set_hv_cpuid(vm, VCPU_ID);
+
+ tsc_page_gva = vm_vaddr_alloc(vm, getpagesize(), 0x10000, 0, 0);
+ memset(addr_gpa2hva(vm, tsc_page_gva), 0x0, getpagesize());
+ TEST_ASSERT((addr_gva2gpa(vm, tsc_page_gva) & (getpagesize() - 1)) == 0,
+ "TSC page has to be page aligned\n");
+ vcpu_args_set(vm, VCPU_ID, 2, tsc_page_gva, addr_gva2gpa(vm, tsc_page_gva));
+
+ host_check_tsc_msr_rdtsc(vm);
+
+ for (stage = 1;; stage++) {
+ _vcpu_run(vm, VCPU_ID);
+ TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
+ "Stage %d: unexpected exit reason: %u (%s),\n",
+ stage, run->exit_reason,
+ exit_reason_str(run->exit_reason));
+
+ switch (get_ucall(vm, VCPU_ID, &uc)) {
+ case UCALL_ABORT:
+ TEST_FAIL("%s at %s:%ld", (const char *)uc.args[0],
+ __FILE__, uc.args[1]);
+ /* NOT REACHED */
+ case UCALL_SYNC:
+ break;
+ case UCALL_DONE:
+ /* Keep in sync with guest_main() */
+ TEST_ASSERT(stage == 11, "Testing ended prematurely, stage %d\n",
+ stage);
+ goto out;
+ default:
+ TEST_FAIL("Unknown ucall %lu", uc.cmd);
+ }
+
+ TEST_ASSERT(!strcmp((const char *)uc.args[0], "hello") &&
+ uc.args[1] == stage,
+ "Stage %d: Unexpected register values vmexit, got %lx",
+ stage, (ulong)uc.args[1]);
+
+ /* Reset kvmclock triggering TSC page update */
+ if (stage == 7 || stage == 8 || stage == 10) {
+ struct kvm_clock_data clock = {0};
+
+ vm_ioctl(vm, KVM_SET_CLOCK, &clock);
+ }
+ }
+
+out:
+ kvm_vm_free(vm);
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test that KVM_SET_BOOT_CPU_ID works as intended
+ *
+ * Copyright (C) 2020, Red Hat, Inc.
+ */
+#define _GNU_SOURCE /* for program_invocation_name */
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+#define N_VCPU 2
+#define VCPU_ID0 0
+#define VCPU_ID1 1
+
+static uint32_t get_bsp_flag(void)
+{
+ return rdmsr(MSR_IA32_APICBASE) & MSR_IA32_APICBASE_BSP;
+}
+
+static void guest_bsp_vcpu(void *arg)
+{
+ GUEST_SYNC(1);
+
+ GUEST_ASSERT(get_bsp_flag() != 0);
+
+ GUEST_DONE();
+}
+
+static void guest_not_bsp_vcpu(void *arg)
+{
+ GUEST_SYNC(1);
+
+ GUEST_ASSERT(get_bsp_flag() == 0);
+
+ GUEST_DONE();
+}
+
+static void test_set_boot_busy(struct kvm_vm *vm)
+{
+ int res;
+
+ res = _vm_ioctl(vm, KVM_SET_BOOT_CPU_ID, (void *) VCPU_ID0);
+ TEST_ASSERT(res == -1 && errno == EBUSY,
+ "KVM_SET_BOOT_CPU_ID set while running vm");
+}
+
+static void run_vcpu(struct kvm_vm *vm, uint32_t vcpuid)
+{
+ struct ucall uc;
+ int stage;
+
+ for (stage = 0; stage < 2; stage++) {
+
+ vcpu_run(vm, vcpuid);
+
+ switch (get_ucall(vm, vcpuid, &uc)) {
+ case UCALL_SYNC:
+ TEST_ASSERT(!strcmp((const char *)uc.args[0], "hello") &&
+ uc.args[1] == stage + 1,
+ "Stage %d: Unexpected register values vmexit, got %lx",
+ stage + 1, (ulong)uc.args[1]);
+ test_set_boot_busy(vm);
+ break;
+ case UCALL_DONE:
+ TEST_ASSERT(stage == 1,
+ "Expected GUEST_DONE in stage 2, got stage %d",
+ stage);
+ break;
+ case UCALL_ABORT:
+ TEST_ASSERT(false, "%s at %s:%ld\n\tvalues: %#lx, %#lx",
+ (const char *)uc.args[0], __FILE__,
+ uc.args[1], uc.args[2], uc.args[3]);
+ default:
+ TEST_ASSERT(false, "Unexpected exit: %s",
+ exit_reason_str(vcpu_state(vm, vcpuid)->exit_reason));
+ }
+ }
+}
+
+static struct kvm_vm *create_vm(void)
+{
+ struct kvm_vm *vm;
+ uint64_t vcpu_pages = (DEFAULT_STACK_PGS) * 2;
+ uint64_t extra_pg_pages = vcpu_pages / PTES_PER_MIN_PAGE * N_VCPU;
+ uint64_t pages = DEFAULT_GUEST_PHY_PAGES + vcpu_pages + extra_pg_pages;
+
+ pages = vm_adjust_num_guest_pages(VM_MODE_DEFAULT, pages);
+ vm = vm_create(VM_MODE_DEFAULT, pages, O_RDWR);
+
+ kvm_vm_elf_load(vm, program_invocation_name, 0, 0);
+ vm_create_irqchip(vm);
+
+ return vm;
+}
+
+static void add_x86_vcpu(struct kvm_vm *vm, uint32_t vcpuid, bool bsp_code)
+{
+ if (bsp_code)
+ vm_vcpu_add_default(vm, vcpuid, guest_bsp_vcpu);
+ else
+ vm_vcpu_add_default(vm, vcpuid, guest_not_bsp_vcpu);
+
+ vcpu_set_cpuid(vm, vcpuid, kvm_get_supported_cpuid());
+}
+
+static void run_vm_bsp(uint32_t bsp_vcpu)
+{
+ struct kvm_vm *vm;
+ bool is_bsp_vcpu1 = bsp_vcpu == VCPU_ID1;
+
+ vm = create_vm();
+
+ if (is_bsp_vcpu1)
+ vm_ioctl(vm, KVM_SET_BOOT_CPU_ID, (void *) VCPU_ID1);
+
+ add_x86_vcpu(vm, VCPU_ID0, !is_bsp_vcpu1);
+ add_x86_vcpu(vm, VCPU_ID1, is_bsp_vcpu1);
+
+ run_vcpu(vm, VCPU_ID0);
+ run_vcpu(vm, VCPU_ID1);
+
+ kvm_vm_free(vm);
+}
+
+static void check_set_bsp_busy(void)
+{
+ struct kvm_vm *vm;
+ int res;
+
+ vm = create_vm();
+
+ add_x86_vcpu(vm, VCPU_ID0, true);
+ add_x86_vcpu(vm, VCPU_ID1, false);
+
+ res = _vm_ioctl(vm, KVM_SET_BOOT_CPU_ID, (void *) VCPU_ID1);
+ TEST_ASSERT(res == -1 && errno == EBUSY, "KVM_SET_BOOT_CPU_ID set after adding vcpu");
+
+ run_vcpu(vm, VCPU_ID0);
+ run_vcpu(vm, VCPU_ID1);
+
+ res = _vm_ioctl(vm, KVM_SET_BOOT_CPU_ID, (void *) VCPU_ID1);
+ TEST_ASSERT(res == -1 && errno == EBUSY, "KVM_SET_BOOT_CPU_ID set to a terminated vcpu");
+
+ kvm_vm_free(vm);
+}
+
+int main(int argc, char *argv[])
+{
+ if (!kvm_check_cap(KVM_CAP_SET_BOOT_CPU_ID)) {
+ print_skip("set_boot_cpu_id not available");
+ return 0;
+ }
+
+ run_vm_bsp(VCPU_ID0);
+ run_vm_bsp(VCPU_ID1);
+ run_vm_bsp(VCPU_ID0);
+
+ check_set_bsp_busy();
+}
{
# In accordance with INET_ECN_decapsulate()
__test_ecn_decap 00 00 0x00
+ __test_ecn_decap 00 01 0x00
+ __test_ecn_decap 00 02 0x00
+ # 00 03 is tested in test_ecn_decap_error()
+ __test_ecn_decap 01 00 0x01
__test_ecn_decap 01 01 0x01
- __test_ecn_decap 02 01 0x02
+ __test_ecn_decap 01 02 0x01
__test_ecn_decap 01 03 0x03
+ __test_ecn_decap 02 00 0x02
+ __test_ecn_decap 02 01 0x01
+ __test_ecn_decap 02 02 0x02
__test_ecn_decap 02 03 0x03
+ __test_ecn_decap 03 00 0x03
+ __test_ecn_decap 03 01 0x03
+ __test_ecn_decap 03 02 0x03
+ __test_ecn_decap 03 03 0x03
test_ecn_decap_error
}
timeout=30
mptcp_connect=""
capture=0
+do_all_tests=1
TEST_COUNT=0
-j DROP
}
-for arg in "$@"; do
- if [ "$arg" = "-c" ]; then
- capture=1
- fi
-done
-
ip -Version > /dev/null 2>&1
if [ $? -ne 0 ];then
echo "SKIP: Could not run test without ip tool"
echo " -4 v4mapped_tests"
echo " -b backup_tests"
echo " -p add_addr_ports_tests"
- echo " -c syncookies_tests"
+ echo " -k syncookies_tests"
+ echo " -c capture pcap files"
echo " -h help"
}
make_file "$sin" "server" 1
trap cleanup EXIT
-if [ -z $1 ]; then
+for arg in "$@"; do
+ # check for "capture" arg before launching tests
+ if [[ "${arg}" =~ ^"-"[0-9a-zA-Z]*"c"[0-9a-zA-Z]*$ ]]; then
+ capture=1
+ fi
+
+ # exception for the capture option, the rest means: a part of the tests
+ if [ "${arg}" != "-c" ]; then
+ do_all_tests=0
+ fi
+done
+
+if [ $do_all_tests -eq 1 ]; then
all_tests
exit $ret
fi
-while getopts 'fsltra64bpch' opt; do
+while getopts 'fsltra64bpkch' opt; do
case $opt in
f)
subflows_tests
p)
add_addr_ports_tests
;;
- c)
+ k)
syncookies_tests
;;
+ c)
+ ;;
h | *)
usage
;;
};
struct reuse_opts unreusable_opts[12] = {
- {0, 0, 0, 0},
- {0, 0, 0, 1},
- {0, 0, 1, 0},
- {0, 0, 1, 1},
- {0, 1, 0, 0},
- {0, 1, 0, 1},
- {0, 1, 1, 0},
- {0, 1, 1, 1},
- {1, 0, 0, 0},
- {1, 0, 0, 1},
- {1, 0, 1, 0},
- {1, 0, 1, 1},
+ {{0, 0}, {0, 0}},
+ {{0, 0}, {0, 1}},
+ {{0, 0}, {1, 0}},
+ {{0, 0}, {1, 1}},
+ {{0, 1}, {0, 0}},
+ {{0, 1}, {0, 1}},
+ {{0, 1}, {1, 0}},
+ {{0, 1}, {1, 1}},
+ {{1, 0}, {0, 0}},
+ {{1, 0}, {0, 1}},
+ {{1, 0}, {1, 0}},
+ {{1, 0}, {1, 1}},
};
struct reuse_opts reusable_opts[4] = {
- {1, 1, 0, 0},
- {1, 1, 0, 1},
- {1, 1, 1, 0},
- {1, 1, 1, 1},
+ {{1, 1}, {0, 0}},
+ {{1, 1}, {0, 1}},
+ {{1, 1}, {1, 0}},
+ {{1, 1}, {1, 1}},
};
int bind_port(struct __test_metadata *_metadata, int reuseaddr, int reuseport)
ifeq ($(CAN_BUILD_I386),1)
$(BINARIES_32): CFLAGS += -m32
$(BINARIES_32): LDLIBS += -lrt -ldl -lm
-$(BINARIES_32): %_32: %.c
+$(BINARIES_32): $(OUTPUT)/%_32: %.c
$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
$(foreach t,$(TARGETS),$(eval $(call gen-target-rule-32,$(t))))
endif
ifeq ($(CAN_BUILD_X86_64),1)
$(BINARIES_64): CFLAGS += -m64
$(BINARIES_64): LDLIBS += -lrt -ldl
-$(BINARIES_64): %_64: %.c
+$(BINARIES_64): $(OUTPUT)/%_64: %.c
$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
$(foreach t,$(TARGETS),$(eval $(call gen-target-rule-64,$(t))))
endif
projects / linux-2.6-microblaze.git / commitdiff